HK40100729A - Deployment of software releases on datacenters configured in cloud platforms - Google Patents

Description

Deploying software on data centers configured within a cloud platform.

Background Technology

Technical Field

The publicly available text generally relates to managing software distribution within a cloud computing platform, and more specifically, to deploying software distribution on data centers configured within a cloud computing platform.

Related technical descriptions

Organizations are increasingly turning to cloud platforms (or cloud computing platforms), such as AWS (Amazon Web Services), Google Cloud Platform, and Microsoft Azure, to meet their infrastructure needs. Cloud platforms deliver servers, storage, databases, networks, software, and more to organizations via the internet. Typically, organizations maintain data centers containing the hardware and software they use. However, maintaining data centers can incur significant costs in terms of maintenance, personnel, and other aspects. Therefore, organizations are migrating their data centers to cloud platforms to provide the scalability and elasticity of computing resources.

Organizations use continuous delivery platforms (CDPs) to maintain cloud infrastructure on cloud platforms, enabling them to manage and deploy applications. These CDPs allow organizations to streamline software deployment processes and manage applications, firewalls, clusters, servers, load balancers, and other computing infrastructure on cloud platforms. However, deploying software releases for services offered on cloud platforms using CDPs can be complex. For example, different versions of software may need to be deployed on different services running on different cloud computing resources. Furthermore, each cloud platform uses different tools to manage resources.

Large systems, such as multi-tenant systems, can provide organizational management services for a large number of tenants representing the multi-tenant system and can interact with multiple cloud platforms. A multi-tenant system may have to maintain thousands of such data centers across cloud platforms. Each data center may have different requirements for software deployment. Furthermore, each cloud platform may support different software, languages, and features. For example, different cloud platforms may support different mechanisms for enforcing network policies or access controls. Therefore, depending on the number of cloud platforms supported by the data center, the multi-tenant system must maintain different implementations of the mechanisms used to publish and deploy services across the data center. This results in high maintenance costs for multi-tenant systems that support software deployment across data centers on multiple cloud platforms.

Attached Figure Description

Figure 1 is a block diagram illustrating the system environment for configuring a multi-tenant system for a data center on a cloud platform according to the implementation scheme.

Figure 2A is a block diagram illustrating the architecture of deployment module 210 according to the implementation scheme.

Figure 2B illustrates the overall process for deploying software artifacts in a data center according to the implementation plan.

Figure 3 is a block diagram illustrating the architecture of a software release management module according to one implementation scheme.

Figure 4 illustrates an example of a data center declarative specification according to one implementation plan.

Figure 5 illustrates an exemplary data center created on a cloud platform based on a declarative specification according to one implementation scheme.

Figure 6 is a block diagram illustrating the generation of a data center on a cloud platform based on a declarative specification according to one implementation scheme.

Figure 7 illustrates the overall process of generating a pipeline for deploying software artifacts on a data center configured on a cloud platform, according to the implementation scheme.

Figure 8 illustrates an exemplary main line according to the implementation scheme.

Figure 9 illustrates the overall process executed in stages by the main pipeline environment on the cloud platform according to the implementation scheme.

Figure 10 illustrates an exemplary mainline according to an implementation scheme.

Figure 11 illustrates an exemplary mainline for a data center according to an implementation scheme.

Figure 12 illustrates how the execution of the pipeline can be modified based on the artifact version mapping according to the implementation scheme.

Figure 13 illustrates the overall process for deploying software artifacts on a data center configured on a cloud platform, according to the implementation scheme.

Figure 14 is a block diagram showing a functional view of a typical computer system used in the environment of Figure 1 according to one embodiment.

The accompanying drawings depict various embodiments for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein can be employed without departing from the principles of the embodiments described herein.

The diagram uses similar reference numbers to identify similar features. A letter following the reference number, such as "115a," indicates that the text specifically refers to the feature with that particular reference number. Reference numbers without a following letter, such as "115," refer to any or all features in the diagram that have that reference number.

Detailed Implementation

Cloud platforms provide computing resources, such as storage, computing resources, and applications, to computing systems on demand via public networks like the internet. Cloud platforms allow businesses to minimize the upfront costs of building computing infrastructure and enable applications to be launched and running faster with less maintenance overhead. They also allow businesses to adjust computing resources to adapt to rapidly fluctuating and unpredictable demands. Businesses can use cloud platforms to create data centers for their users. However, implementing a data center on each cloud platform requires expertise in the cloud platform technology.

The implementation uses a cloud platform infrastructure language (LPIL) to create data centers within a cloud platform. The system receives a cloud platform-independent declarative specification for the data center. This declarative specification describes the structure of the data center and may not provide instructions on how to create it. The cloud platform-independent declarative specification is configured to generate data centers on any of multiple cloud platforms and is specified using the LPIL. The system receives information identifying the target cloud platform used to create the data center and compiles the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. The system sends the cloud platform-specific data center representation and a set of instructions for execution on the target cloud platform. The target cloud platform uses the platform-specific data center representation to execute instructions for configuring the data center. The system provides users with access to the data center computing resources configured on the cloud platform.

In one implementation, the system performs operations related to software releases configured on a data center on a cloud platform, such as deploying software releases, providing resources, performing software release rollbacks, etc. The system accesses a data center configured on a target cloud platform. The data center is generated based on a cloud platform-independent declarative specification that includes a hierarchy of data center entities. Each data center entity includes (1) a service and (2) one or more other data center entities. The system generates a cloud platform-independent pipeline that includes: (1) a sequence of phases for deploying software artifacts, such as development, testing, and production phases, and (2) standards for advancing software artifacts from one phase to subsequent phases in the phase sequence. The system compiles the cloud platform-independent pipeline to generate a cloud platform-dependent detailed pipeline for the target cloud platform, which has instructions for performing service-related operations according to the data center layout defined in the declarative specification. The system executes the cloud platform-dependent detailed pipeline on the target cloud platform, such as deploying software releases on data center entities within the data center.

In one implementation, the system accesses a data center configured on a target cloud platform. The system receives a cloud platform-independent artifact version map that associates a data center entity with versions of software artifacts targeted for deployment on that data center entity. Each software artifact includes executable instructions associated with a service configured to execute on one or more cloud platforms. Based on the cloud platform-independent artifact version map, the system generates a cloud platform-specific pipeline for the target cloud platform. The cloud platform-specific pipeline includes instructions to perform operations such as building and deploying appropriate versions of deployment artifacts for services on the data center entity according to the cloud platform-independent version map. The system transmits the cloud platform-specific deployment pipeline to the target cloud platform for execution. The artifact version map and pipeline can be used to perform various service-related actions, including deploying services, destroying services, providing service resources, destroying service resources, and so on.

The cloud platform is also referred to as the substrate in this document. The declarative specifications of a data center are substrate-independent or substrate-agnostic. If traditional technologies are used to perform data center-related operations, such as deploying software releases and providing resources, users must provide cloud platform-specific instructions. Therefore, users require expertise in the cloud platform they are using. Furthermore, these instructions are cloud platform-specific and not portable across multiple platforms. For example, the instructions for deploying software on the AWS cloud platform differ from those on the GCP cloud platform. Developers need to understand the details of implementing each feature on a specific cloud platform. The disclosed system provides a cloud platform infrastructure language that allows users to perform operations in the data center using cloud platform-independent instructions that can be executed on any cloud platform selected from multiple cloud platforms. The compiler of the cloud platform infrastructure language generates detailed cloud platform-specific instructions for the target cloud platform.

The language of cloud platform infrastructure can be called a domain-specific language (DSL). This system can represent a multi-tenant system, but is not limited to multi-tenant systems, and can be any online system or any computing system with network access to the cloud platform.

System Environment

Figure 1 is a block diagram illustrating a system environment for configuring a multi-tenant system for a data center on a cloud platform according to an implementation scheme. System environment 100 includes a multi-tenant system 110, one or more cloud platforms 120, and one or more client devices 105. In other implementations, system environment 100 may include more or fewer components.

Multitenant system 110 stores information about one or more tenants 115. Each tenant can be associated with an enterprise that represents a customer of multitenant system 110. Each tenant can have multiple users interacting with the multitenant system via client device 105.

A cloud platform can also be referred to as a cloud computing platform or a public cloud environment. Tenants can use the cloud platform infrastructure language to provide declarative specifications for data centers created on the target cloud platform 120 and use these data centers to perform operations, such as providing resources, performing software releases, etc. Tenant 115 can create one or more data centers on cloud platform 120. A data center represents a set of computing resources, including servers, applications, storage, memory, etc., that can be used by users (e.g., users associated with a tenant). Each tenant can provide different functionalities to its users. Therefore, each tenant can perform different services on the data center configured for that tenant. Multi-tenant systems can implement different software release and deployment mechanisms for each tenant. Tenants can further obtain or develop software versions that include instructions for various services performed in the data center. Implementation schemes allow tenants to deploy specific versions of software releases for different services running on different computing resources in the data center.

The computing resources in a data center are secure, and users without authorized access may not be able to access them. For example, data center 125a created for user 115a may not be accessible to user 115b unless access is explicitly authorized. Similarly, data center 125b created for user 115b may not be accessible to user 115a unless access is explicitly authorized. Furthermore, services provided by the data center can only be accessed by computing systems outside the data center if access is authorized to the computing system in accordance with the data center's declarative specifications.

In a multi-tenant system 110, data from multiple tenants can be stored in the same physical database. However, the database is configured to logically separate one tenant's data from that of other tenants, preventing one tenant from accessing another's data unless such data is explicitly shared. Tenant data can be stored in tables shared with other clients, transparent to the tenant. Database tables can store rows from multiple tenants. Therefore, in a multi-tenant system, various hardware and software components of the system can be shared by one or more tenants. For example, multi-tenant system 110 can execute application servers that simultaneously process requests from multiple tenants. However, multi-tenant systems enforce tenant-level data isolation to ensure that one tenant's job does not access other tenants' data.

Implementations of the cloud platform include AWS (Amazon Web Services), Google Cloud Platform, or Microsoft Azure. Cloud platform 120 provides computing infrastructure services that can be used on demand by tenant 115 or any computing system outside of cloud platform 120. Implementations of the computing infrastructure services provided by the cloud platform include server, storage, database, networking, security, load balancing, software, analytics, intelligence, and other infrastructure service functions. Tenant 115 can use these infrastructure services to build, deploy, and manage applications in a scalable and secure manner.

Multitenant system 110 may include a tenant data store that stores data for various tenants within the multitenant storage. The tenant data store may store data for different tenants in separate physical structures, such as separate database tables or separate databases. Alternatively, the tenant data store may store data for multiple tenants in a shared structure. For example, user accounts for all tenants may share the same database table. However, the multitenant system stores additional information to logically separate the data for different tenants.

Each component shown in Figure 1 represents one or more computing devices. For example, a computing device can be a conventional computer system running a Microsoft ^™ , Windows ^™ compatible operating system (OS), Apple ^™ OS X, and/or Linux distribution. A computing device can also be a client device with computer functionality, such as a personal digital assistant (PDA), mobile phone, video game system, etc. Each computing device stores software modules, which store instructions.

Interactions between various components of system environment 100 are typically performed via a network (not shown in Figure 1). In one embodiment, the network uses standard communication technologies and/or protocols. In another embodiment, entities may use custom and/or proprietary data communication technologies to replace or supplement the aforementioned technologies.

Although the techniques disclosed herein are described in the context of multi-tenant systems, these techniques can be implemented using other systems that may not be multi-tenant systems. For example, an online system used by a single organization or enterprise can use the techniques disclosed herein to create one or more data centers on one or more cloud platforms 120.

System Architecture

Multitenant system 110 includes a deployment module for deploying software artifacts on a cloud platform. The deployment module can perform various operations associated with software releases, such as providing resources on the cloud platform, deploying software releases, performing rollbacks of software artifacts installed on data center entities, and so on. Figure 2 is a block diagram illustrating the system architecture of deployment module 210 according to an implementation scheme. Deployment module 210 includes a data center generation module 220 and a software release management module 230. Other implementations may have components and/or other components different from those described herein, and functionality may be distributed among the components in different ways.

The data center creation module 220 includes instructions for creating a data center on a cloud platform. The software release management module 230 includes instructions for deploying software releases that run on the data center created by the data center creation module 220.

Data center generation module 220 receives a cloud platform-independent declarative specification of the data center from a user (e.g., a tenant's user). The cloud platform-independent declarative specification of the data center specifies various entities within the data center. In one implementation, the cloud platform-independent declarative specification of the data center includes a hierarchical organization of data center entities, where each data center entity may include one or more services, one or more other data center entities, or a combination of both. Figure 4 illustrates the various types of data center entities in more detail. Data center generation module 220 receives a platform-independent declarative specification and a target cloud platform as input and generates a cloud platform-specific metadata representation for the target cloud platform. Data center generation module 220 deploys the generated cloud platform-specific metadata representation on the target cloud platform to create the data center on the target cloud platform according to the declarative specification.

The software release management module 230 receives (1) an artifact version map 225 and (2) a supervisor line 235 as input. The artifact version map 225 identifies a specific version of a software release or deployment artifact targeted for deployment on a specific data center entity. The artifact version map 225 maps the data center entity to a software release version targeted for deployment on the data center entity. The supervisor line 235 includes instructions for operations related to software releases on the data center, such as deploying services, destroying services, providing service resources, destroying service resources, etc.

Supervisor line 235 may include instructions for performing operations related to software releases in different environments (such as development, testing, canary, and production environments), and instructions for determining when to advance a software release from one environment to another. For example, if a software release is deployed in the development environment and executes more than a threshold number of test cases, the release is advanced to the testing environment for further testing, such as system-level and integration testing. If the software release in the testing environment passes a test coverage threshold, it is advanced to the canary environment, where it is offered to a small group of users on a trial basis. If the software release in the canary environment executes without errors within a threshold time, it is advanced to the production environment, where it is offered to all users.

The software release management module 230 compiles the input artifact version map 225 and the main pipeline 235 to generate a cloud-platform-specific detailed pipeline 255 for transmission to the target cloud platform. The cloud-platform-specific detailed pipeline 255 includes instructions for deploying the software release or an appropriate version of the deployment artifact on the data center entity, as specified in the artifact version map 225. The software release management module 230 can accept modifications to one of the inputs. For example, a user can modify the input artifact version map 225 while providing the same main pipeline 235. Thus, the same main pipeline is used, but a different software release is deployed on the data center entity. The software release management module 230 recompiles the input to generate a new cloud-platform-specific detailed pipeline 255 that deploys the version of the software release according to the new artifact version map 225.

Artifact version mapping can also be referred to as deployment manifest, version manifest, software release mapping, or software artifact version mapping. The main pipeline can also be referred to as the main deployment pipeline or main orchestration pipeline.

Figure 2B illustrates the overall process for deploying software artifacts in a data center according to the implementation scheme. Figure 2B shows the layout of data center 265, which includes various data center entities. As shown in Figure 2B, artifact version mapping 225 identifies different versions of software targeted for release on different data center entities 275 of data center 265. Pipelines represent the deployment artifact flow through various environments of the data center. Software release management module 230 combines the information in pipeline 235 with artifact version mapping 225 to determine cloud platform-specific detailed pipelines 255 that map the appropriate versions of software artifacts on the data center entities according to artifact version mapping 225.

Figure 3 is a block diagram illustrating the architecture of a software release management module 230 according to one embodiment. The software release management module 230 includes a parsing module 310, a pipeline generator module 320, an artifact version mapping store 330, a pipeline store 340, and a pipeline execution engine 360. Other embodiments may include more, fewer, or different modules than those indicated herein in Figure 3.

The parsing module 310 parses various types of user input, including declarative specifications for data centers, artifact version mappings 225, and main pipelines 235. The parsing module 310 generates data structures and metadata representations of the processed inputs and provides these representations to other modules within the software release management module 230 for further processing.

Metadata store 340 stores metadata representations of various transformations in the data center generated by software release management module 230. If an issue is encountered in the current version of the data center, a rollback to a previous version can be performed using the metadata representation of the transformation. The metadata representation of the transformation can be used for verification, auditing, governance, etc., at various stages of the transformation process.

Pipeline generator module 320 processes main pipelines in conjunction with artifact version maps received as input to generate detailed pipelines for a target cloud platform. The pipeline includes stages containing instructions for providing services or deploying applications, and deployment software release versions for various services on the cloud platform based on the artifact version maps. Artifact version map store 330 stores artifact version maps received from the user, and pipeline store 340 stores the main pipelines as well as the pipelines generated by pipeline generator module 320.

Pipeline execution engine 360 executes the detailed pipeline generated by pipeline generator module 320. In one implementation, pipeline execution engine 360 is a system such as SPINNAKER that executes pipelines for publishing/deploying software. Pipeline execution engine 360 parses the pipeline and executes each stage of the pipeline on the target cloud computing platform.

Data center generation based on cloud platform

Figure 4 illustrates an embodiment of a declarative specification for a data center according to one implementation. The declarative specification 410 includes multiple data center entities. A data center entity is an instance of a data center entity type, and each data center entity type can have multiple instances. Embodiments of data center entities include data centers, service groups, services, teams, environments, and patterns.

Declarative specification 410 includes definitions for various types of data center entities, including service groups, services, teams, environments, and patterns. A declarative specification includes one or more instances of a data center. The following is a description of the various types of data center entities and their embodiments. These embodiments are illustrative and show some attributes of the data center entity. Other implementations may include different attributes, and attributes with the same functionality may be given names different from those indicated herein. In one implementation, a hierarchical object conforming to a predefined pattern (e.g., JSON (Javascript object notation)) is used to specify the declarative specification.

According to one implementation, service group 520 represents a set of capabilities, features, and services that can be independently built and delivered by one or more computing systems. A service group may also be referred to as a logical service group, functional unit, or bounded context. Service group 520 can also be viewed as a set of services containing a cohesive set of technical use case functions provided by one or more computing systems. Service group 520 enforces security boundaries. Service group 520 defines the scope of modifications. Therefore, any modifications to an entity, such as capabilities, features, or services provided by one or more computing systems within service group 520, may propagate as needed or suitable for the entities within the service group, but not to entities residing outside the bounded definition of service group 520. A data center may include multiple service groups 520. Service group definitions specify attributes, including name, description, identifier, schema version, and service set instance. An example of a service group is a blockchain service group, which includes a set of services for providing blockchain functionality. Similarly, a security service group provides security features. A user interface service group provides functionality for specific user interface features. A shared document service group provides functionality for sharing documents across users. Similarly, there may be several other service groups.

Service groups support specification reusability, providing tenants or users interested in developing data centers with a library of service groups they can readily use. The boundaries of services surrounding a service group are based on security and network issues, among others. A service group is associated with protocols used to perform interactions with it. In one implementation, a service group provides an API (Application Programming Interface) and a collection of services that implement those APIs. Furthermore, service groups are independent of the substrate. Service groups provide an blast radius for services within the group, ensuring that the impact of any service failure within the group is limited to the services within the group, with minimal impact outside the group.

The following is an example of a service group specification. A service group specifies various attributes representing the service group's metadata and includes the set of services within the service group. Other types of metadata may exist for service groups, which are not indicated herein.

As illustrated in the examples above, a service group can specify a cluster. A cluster represents a group of compute nodes, such as a group of servers, a group of virtual machines, or a group of containers (such as Kubernetes containers). A physical server can run multiple containers, each with its own share of file system, CPU, memory, process space, etc.

This service group specifies a set of services. A service group can specify a cluster for a service, enabling a cluster of compute nodes to run in a data center deployed on a cloud platform, and mapping the service to the cluster based on that specified mapping if the specified mapping is included in a declarative specification. For example, in the service group embodiment shown above, the service instance serviceinstance0002 is specified to run on the cluster instance cluster1.

Service groups can specify security groups, each specifying a set of services that are allowed to interact with each other. Services outside a security group require additional authentication to communicate with services within the security group. Alternatively, services within a security group may use one protocol to interact with each other, while services outside the security group may use a different protocol and require enhanced authentication to interact with services within the security group. Therefore, security groups specify the policies that determine how services interact with each other. Security policies can specify one or more environments to which a security policy applies. For example, security policy policy1 may apply to a specific environment env1 (e.g., a production environment), and another security policy policy policy2 may apply to another environment env2 (e.g., a development environment). Security policies can be specified for service group types or specific service types.

In the implementation, the security policy specifies expressions for filtering service groups based on various attributes, making the security policy applicable to the filtered set of service groups. For example, the security policy may specify a list of IP (Internet Protocol) addresses that are whitelisted as a set of service groups identified by the filtered groups, and thus allow these computing systems to access the service groups or specific sets of services within the service groups.

In the implementation scheme, a security policy can specify a set of source services and a set of destination services for a service group. The source services for a specific service specify services outside the security group that are allowed to connect to that specific service. The destination services for a specific service specify services outside the security group that that specific service needs to connect to. During provisioning and deployment, the data center generation module generates instructions for the cloud platform that use cloud platform-specific features and network functions to enforce specific network policies, causing the network policies to implement the security policies specified in the declarative specification.

A data center entity, referred to as a cell, represents a set of services that interact with each other vertically and can be expanded by additional instances or copies of the cell (i.e., copies of the group of services). Creating multiple instances of a cell allows the system to expand the set of services that interact with each other. A data center instance can include one or more cells. Each cell can include one or more services. A data center can include instances of service groups or cells.

A service definition specifies metadata for a service type (such as a database service, load balancer service, etc.). This metadata describes various attributes of the service, including the service name, description, location of the service's documentation, any associated sub-services, the service owner, the team associated with the service, service build dependencies specifying other services the service depends on during construction, service startup dependencies specifying other services that should run when the specific service starts, authorized clients, the DNS (Domain Name Server) name associated with the service, service status, support level for the service, and so on. The service definition also specifies the listening port attribute, which indicates the ports the service can listen on for different communication protocols. For example, the service can listen on port p1 for UDP and port p2 for TCP. Other services within the data center can interact with the service via the ports specified by the service.

The service definition specifies outbound access attributes, which specify the destination endpoint (e.g., an external URL (Uniform Resource Locator)) that the service needs to access a specified external URL. During deployment, the datacenter generation module ensures that the cloud platform enforces access policies to provide instances of this service type with the ability to request access to the external URL.

Outbound access specifications can identify one or more environment types for which outbound access applies. For example, outbound access for a set of endpoints S1 can be applied to a specific environment env1 (e.g., production environment), and outbound access for a set of endpoints S2 can be applied to another environment env2 (e.g., development environment).

The following is an example of a service definition.

Team definition 450 includes team member names and other team attributes such as name, email, communication channel, etc. The following is an example of a team definition. A service can be associated with one or more teams responsible for modifying that service. Therefore, any modifications made to the service are approved by the team. After being deployed to a cloud platform, a service can be associated with a team responsible for maintaining the service. A team can be associated with a service group, and correspondingly with all services within that service group. For example, a team approves any changes to a service group (e.g., services that are part of the service group). A team can be associated with a data center, and correspondingly with all service groups within that data center. Team associations specified at the data center level provide a default team for all service groups within the data center, and further provide a default team for all services within the service group.

According to the implementation scheme, team associations specified at the functional level override team associations provided at the data center level. Similarly, team associations specified at the service level override default values that may already be provided for team associations specified at the service group level or data center level. Teams can decide how to take certain actions for data center entities associated with them. Team associations also determine the number of accounts on the cloud platform created for the final metadata representation of the data center generated by the compiler for the cloud platform, as well as for provisioning and deploying the data center on the cloud platform. The data center generation module 210 creates one or more user accounts in the cloud platform and provides team members with access to these user accounts. This allows team members to perform specific actions associated with the data center entities associated with their team, such as making or approving structural changes to or maintaining the data center entity during deployment, including debugging and testing issues that can be identified for the data center entity.

Traditional techniques associate the same team with the data center throughout the design process, resulting in organizational structure influencing the design of the data center or service group. The implementation plan separates team definition from defining the construction of the data center entity, thereby reducing the team's influence on the design and architecture of the data center entity.

Environment definition 460 specifies the type of system environment represented by the data center, such as a development environment, simulation environment, test environment, or production environment. Schema definition 470 specifies a schema for the syntax of defining a particular data center entity. Schema definition 470 is used to validate various data center entity definitions. The data center generation module determines the security policy of the data center in the cloud platform-specific metadata representation based on the environment. For example, a specific set of security policies may apply to environment env1, and a different set of security policies may apply to environment env2. For example, the security policy in the production environment provides much more restricted access compared to the development environment. Security policies can specify the length of time a security token is allowed to exist for a specific purpose. For example, long access tokens (e.g., access tokens lasting one week) may be allowed in the development environment, but access tokens used in the production environment have a much shorter lifespan (e.g., a few hours). Access tokens can allow users or services to access specific cloud platform resources.

Datacenter definition 420 specifies the attributes and components of a datacenter instance. A declarative specification can specify multiple datacenter instances. Datacenter definition 420 specifies attributes including name, description, environment type, a set of service groups, teams, the datacenter's domain name servers, etc. A datacenter definition can specify a schema definition and validate any metadata representation generated from the datacenter definition against the specified schema definition. A datacenter includes a set of core services and functions that enable other services to function within the datacenter. Datacenter instances are deployed on a specific cloud platform and can be associated with specific environment types (e.g., development, testing, simulation, production, etc.).

The following is the definition of a data center instance. A data center instance definition includes a list of service groups included in the data center instance and other attributes, including the data center environment, data center identifier, name, region representing the geographic area, one or more teams associated with the data center, and mode version.

Figure 5 illustrates some exemplary data centers created on a cloud platform according to a declarative specification based on one implementation scheme. Data centers 510 can be created based on declarative specifications processed by the data center generation module 210. As shown in Figure 5, multiple data centers can be configured within the cloud platform 120. Each data center 510 can correspond to tenant 115 of the multi-tenant system 110. Tenant 115 can create one or more data centers 510. Alternatively, data centers 510 can be created by any computing system. Each data center includes one or more service groups. For example, data center 510a includes service groups 520a and 520b, and data center 510b includes service group 520c. A data center can include multiple instances of a specific type of service group. Each service group includes a set of services. For example, service group 520a includes services 530a and 530b, service group 520b includes services 530a, 530b, and 530c, and service group 520c includes services 530e, 530f, and 530g. A service group can include multiple instances of services of the same service type.

Data center generation module 220 uses the following steps to create a data center on a cloud platform based on a declarative specification. Data center generation module 210 receives a cloud platform-independent declarative specification for the data center. The cloud platform-independent declarative specification can be used by tenants in a multi-tenant system or by any other computing system, such as an online system. The cloud platform-independent declarative specification is specified using the cloud platform infrastructure language. The cloud platform-independent declarative specification of the data center is configured to generate a data center on any of multiple cloud platforms.

The data center generation module 210 receives information identifying the target cloud platform used to create a data center based on a declarative specification independent of the cloud platform. The target cloud platform can be any of several cloud platforms, such as AWS, Azure, GCP, etc. The data center generation module 210 further receives information related to the connection with the target cloud platform, such as credentials for creating a connection to the target cloud platform. The cloud platform can also be referred to as a cloud computing platform.

The data center generation module 210 compiles a cloud platform-independent declarative specification to generate a cloud platform-specific data center representation for creating a data center on the target cloud computing platform. For example, the cloud platform-specific data center representation can refer to user accounts, network addresses, etc., specific to the target cloud computing platform.

The data center generation module 210 sends a platform-specific data center representation and instructions for deploying the data center on the target cloud computing platform. The target cloud computing platform executes the instructions to configure its computing resources to generate the data center based on the platform-specific data center representation. The data center generation module 210 provides users with access to the computing resources of the data center configured on the cloud computing platform. For example, if the data center is created for a tenant in a multi-tenant system, access to the data center is provided to the users associated with that tenant.

Figure 6 is a block diagram illustrating the generation of a data center on a cloud platform based on a declarative specification according to one implementation. The data center generation module 210 receives a cloud platform-independent declarative specification 610 as input. The cloud platform-independent declarative specification 610 can be a version of the declarative specification that has been progressively modified by the user. The data center generation module 210 processes a specific version of the cloud platform-independent declarative specification 610. Since no cloud platform-independent declarative specification 610 is specified for any particular target cloud platform, the data center generation module 210 can configure a data center on any target cloud platform based on the cloud platform-independent declarative specification 610.

The data center generation module 210 processes the cloud platform-independent declarative specification 610 to generate a cloud platform-independent detailed metadata representation 620 for the data center. The cloud platform-independent detailed metadata representation 620 defines the details of each instance of the data center entity specified in the cloud platform-independent declarative specification 610. The data center generation module 210 creates unique identifiers for data center entity instances (e.g., service instances).

In the implementation, the cloud platform-independent detailed metadata representation 620 includes an array of instances of data center entity types, such as an array of service group instances of a specific service group type. Each service group instance includes an array of service instances. The service instances may also include details of user teams permitted to perform certain actions associated with the service instance. The data center generation module 210 uses these team details during provisioning and deployment, for example, to create user accounts for the service instances and allow team members to access those user accounts.

The cloud-independent detailed metadata representation 620 includes attributes for each instance of the data center entity. Therefore, the description of each instance of the data center entity is expanded to include all detailed information. As a result, the cloud-independent detailed metadata representation 620 of the data center can be significantly larger than the cloud-independent declarative specification 610. For example, the cloud-independent declarative specification 610 can be a few thousand lines of specification, while the cloud-independent detailed data center representation 620 can be millions of lines of generated code. Consequently, the data center generation module 210 keeps the cloud-independent detailed metadata representation 620 immutable; that is, once the representation is finalized, no modifications are made to it. For example, if any updates, deletions, or additions to the data center entity need to be performed, they are performed on the cloud-independent declarative specification 610.

The data center generation module 210 receives a target cloud platform on which a data center is intended to be provisioned and deployed, and generates a cloud platform-specific detailed metadata representation 630 for the data center. For example, the data center generation module 210 interacts with the target cloud platform to generate certain entities (or resources), such as user accounts, Virtual Private Clouds (VPCs), and network resources, such as subnets on the VPC, various connections between entities in the cloud platform, etc. The data center generation module 210 receives resource identifiers of the resources created in the target cloud platform, such as user account names, VPC IDs, etc., and incorporates these into a cloud platform-independent detailed metadata representation 620 to obtain a cloud platform-specific metadata representation 630 for the data center. In one implementation, the data center generation module 210 creates a unique user account on the cloud platform for each team for a given combination of service groups and services. The user account is used by the team to perform interactions with specific services of that service group, such as for debugging, for receiving alerts, etc.

The target cloud platform can perform several steps to process the cloud platform-specific detailed metadata representation 630. For example, a cloud platform-independent declarative specification can specify permitted interactions between services. These permitted interactions are specified in the cloud platform-specific detailed metadata representation 630 and enforced as cloud platform network policies. The cloud platform can further create security groups to enforce network policies to enforce the data center according to the declarative specification.

Declarative specifications, independent of the cloud platform, define the dependencies between services. For example, startup dependencies for each service list all services that should run when a specific service is started. The data center generation module 220 generates a cloud platform-specific, detailed metadata representation of the data center, including information describing these dependencies. This ensures that instructions for deploying services guarantee that the cloud platform starts services in the order specified by the dependencies, and that for each service, services that need to be started before it are running when that service is started. Therefore, the dependencies between services represent a dependency graph, and the cloud platform starts services in the order determined by the dependency graph, such that if service A depends on service B, service B starts before service A.

The data center generation module 220 creates trust relationships between user accounts, allowing services to access other services via secure communication channels. These trust relationships are generated using substrate-specific instructions based on declarative specifications (e.g., based on outbound access attributes specified for services). The data center generation module 220 sends instructions to the cloud platform to create network policies based on cloud platform-specific mechanisms that control interactions and access between service groups and services, such as outbound access, security groups, and security policies as specified in the structure of the declarative specifications.

The data center generation module 210 deploys a cloud platform-specific metadata representation 630 on the specific target cloud platform for which it generates the representation. The data center generation module 210 can use the generated metadata representation to perform various validations, including policy validation, format validation, and so on.

The cloud platform-independent declarative specification 610 can be referred to as the declarative data center representation, the cloud platform-independent detailed metadata representation 620 can be referred to as the data center derived metadata representation, and the cloud platform-specific metadata representation 630 can be referred to as the data center hydrated metadata representation.

The overall process for deploying software artifacts on a data center

Figure 7 illustrates the overall process, according to the implementation scheme, for generating a pipeline for deploying software artifacts on data centers configured on a cloud platform. The data center generation module generates one or more data centers on the target cloud platform. Each data center is generated according to a declarative specification independent of the cloud platform and has a hierarchy of data center entities.

The software release management module 230 generates 720 cloud-independent pipelines. In one implementation, the cloud-independent pipeline includes phases corresponding to data center environments (e.g., development, testing, canary, and production environments). The pipelines constitute a sequence of progressive and/or conditional deployments across various environments (e.g., development, testing, simulation, or production). A pipeline can be triggered by passing an image of a software artifact and includes phases or instructions for deploying a build in a development-type environment. The built software artifact is conditionally advanced to one or more testing environments, then one or more canary environments, and finally deployed to the production environment. The pipeline can be customized by users (e.g., service owners) to represent specific orchestrations across environments. The pipeline can be customized to capture specific advancement criteria for moving from one phase to the next. For example, different tenants in a multi-tenant system can customize the pipelines differently. In one implementation, the pipeline defaults to using the latest version of the software for the service's software artifacts and builds and deploys that version across various environments. Users can use artifact version mapping to ensure that a specific version of a software artifact is deployed on a specific data center entity.

In the implementation, each service deployed in the data center has a cloud platform-independent supervisor line generated by the data center entity as defined by the data center's declarative specification. Examples include supervisor lines for data center instances, supervisor lines for service groups, supervisor lines for cells, supervisor lines for services, and so on. Supervisor lines can be triggered when delivering images of software artifacts. Supervisor lines can implement continuous deployments controlled by the service owner. Supervisor lines can also implement on-demand deployments owned by the data center instance owner or the publishing owner.

Certain parts of the pipeline can be customized by users (e.g., by tenants of a multi-tenant system deploying services on a data center). For example, tenants can customize the advance decision pipeline to determine which test cases to execute and what the thresholds are. The software release management module 230 receives 730 customizations of the logic used to advance software artifacts from one stage of the pipeline to another, independent of the cloud platform.

The software release management module 230 compiles 740 cloud platform-independent main pipelines to generate cloud platform-specific detailed deployment pipelines, which are specific to the tiers of data center entities for each data center as specified by the cloud platform-independent declarative specification of the data center.

The software release management module 230 further receives 750 code for publishing one or more features of a service deployed on a data center. The software release management module 230 executes 760 a cloud platform-specific detailed deployment pipeline to deploy software artifacts based on the received code.

Figure 8 illustrates an exemplary pipeline 800 according to an implementation scheme. The pipeline represents a sequence of phases that represent progressive conditional deployments across various data center environments. Figure 8 shows the phases of different environments in a data center, including development, testing, canary, and production environments. Each phase further represents the pipeline executed for that phase. Thus, pipeline 800 includes a development environment pipeline 810 that feeds to a testing environment pipeline 820, a testing environment pipeline that feeds to a canary environment pipeline 830, and a canary environment pipeline that feeds to a production environment pipeline 840.

Each stage of the pipeline is a hierarchical pipeline that includes lower-level pipelines. For example, depending on the number of data centers specified as having development environments in the data center's declarative specification, the development environment pipeline 810 includes main development pipelines that feed to data center pipelines D11, D12, ...

Depending on the number of data centers specified as having a test environment in the data center's declarative specification, the test environment pipeline 820 includes main test pipelines that feed to the data center pipelines D21, D22, ...

Depending on the number of data centers specified as having a canary environment in the data center's declarative specification, the canary environment pipeline 820 includes canary main pipelines feeding into data center pipelines D31, D32, ...

Depending on the number of data centers specified as having a test environment in the data center's declarative specification, the production environment pipeline 820 includes the main production pipelines that feed to the data center pipelines D21, D22, ...

Each environment pipeline 810, 820, and 830 includes advancement decision pipelines 815a, 815b, and 815c, respectively. Advancement decision pipeline 815 collects the output from the data center pipeline of the environment pipelines and determines whether the software artifact is ready to advance to the next stage. Advancement decision pipeline 815 can determine whether the service's software artifact should be advanced to the next stage based on test case results obtained from the data center. For example, if more than a threshold of test cases pass, advancement decision pipeline 815 advances the software artifact to the next stage. The last environment stage, such as the production environment pipeline, may not have an advancement decision pipeline because there is no subsequent stage to which the software artifact needs to be advanced. As shown in Figure 8, advancement decision pipeline 815a of the development environment pipeline determines whether to advance the software artifact from the development stage to the testing stage; advancement decision pipeline 815b of the testing environment pipeline determines whether to advance the software artifact from the testing stage to the canary stage; and advancement decision pipeline 815c of the canary environment pipeline determines whether to advance the software artifact from the canary stage to the production stage.

The main pipeline comprises multiple pipelines, such as a provisioning pipeline for provisioning resources to the target cloud platform and a deployment pipeline for deploying software artifacts on the data center entity. Each pipeline includes a sequence of stages, each stage representing one or more actions that the target cloud platform needs to perform to provision and deploy the data center. The data center generation module 210 generates a detailed pipeline for deploying versions of software artifacts on the data center entity.

In the implementation, the pipeline generator module 320 generates detailed pipelines using a pipeline template that includes variables. The pipeline template is converted into a pipeline by providing specific values for the variables in the pipeline. The process of generating pipelines from a template is called pipeline template hydration. A pipeline template contains template expressions that serve as placeholders for the actual values used in the deployment. For example, a template expression can be replaced by a target-specific parameter value or expression. Multiple pipeline instances can be generated by hydrating pipeline templates for different targets. Template variables represent parameters that can be replaced with specific values for a given target to generate target-specific pipeline instances. For example, during hydration, the template variable "account_id" can be replaced with the actual value of account_id, such as "12345".

In one implementation, the pipeline generator module 320 generates pipelines hierarchically based on the hierarchy of data center entities. For example, a data center includes different types of data center entities, including data centers, service groups, services, etc. A data center entity may include one or more sub-data center entities. For example, a data center includes one or more service groups as sub-data center entities. A service group includes one or more services as sub-data center entities. Therefore, the data center generation module 210 starts with data center entities at a hierarchical level and generates pipelines for data center entities below that level. For example, the pipeline generator module 320 starts at the data center level and generates pipelines for service groups within the data center. For each service group, the pipeline generator module 320 generates pipelines for the services within the service group.

According to one implementation, the process for executing a pipeline is as follows: Software deployment module 230 receives a request to deploy software artifacts on a set of data center entities in a target cloud platform. Software deployment module 230 executes a pipeline for one or more data centers. Software deployment module 230 executes an aggregate pipeline for each service group in each data center. The aggregate pipeline includes pipelines for services within the service group. For each service in each service group, the pipeline is executed through all stages of the pipeline execution. Execution of the provisioning pipeline results in the provisioning of resources to the service, and deployment pipeline results in the deployment of the service in the target cloud platform.

Figure 9 illustrates the overall process of phased execution of the environment in the main pipeline on the cloud platform according to the implementation scheme. Steps 910, 920, 930, 940, and 950 can be performed by each environment pipeline 810, 820, and 830. Production environment pipeline 3 can perform only steps 910 and 920. The steps shown in Figure 9 can be performed for one or more services specified using the manifest file.

The environment pipeline for environment E includes instructions for deploying the 910 software on a set of data center entities (e.g., a set of data center entities designated as having environment E). In an implementation, software artifacts are generated by compiling the source code of the services. The source code can be obtained from version control software. The set of data center entities may include data center instances, service groups, cells, services, or any combination thereof.

The environment pipeline for Environment E also includes instructions for running Test 920, which tests software artifacts deployed on a set of data center entities. The environment pipeline for Environment E also includes instructions for evaluating the results of Test 930 against advancement criteria, for example, using Advancement Decision Pipeline 815. If the advancement criteria are not met, steps 910, 920, 930, and 940 can be repeated using revised software artifacts, such as those generated from source code, which include fixes for certain defects identified during Test 920. The environment pipeline for Environment E also includes instructions for advancing to the next stage, 950, if the advancement criteria are met.

In the implementation, the pipeline includes a hierarchy. This hierarchy comprises multiple levels, and a particular level of pipeline includes the next lower level's pipeline as a sub-pipeline. For example, at the highest level of the hierarchy, the pipeline includes the release pipeline, which deploys a set of services associated with the product. The next level of the hierarchy includes the service pipeline, which represents all deployments of a specific service across various environments. The next level of the hierarchy may include the service group pipeline, followed by the service pipeline.

Figure 10 illustrates an exemplary pipeline according to an implementation scheme. A pipeline is a hierarchical pipeline, where each stage of the pipeline may include a pipeline with detailed instructions for performing that stage. The pipeline hierarchy may mirror a data center hierarchy. For example, the top level of the pipeline represents a sequence of stages for different environments. Each environment may include one or more pipelines for data center instances or for other types of data center entities. Data center instance pipeline 1010 may include service group pipeline 1020. Each service group pipeline 1020 may include one or more service pipelines 1030. Data center instance pipeline 1010 may include cell pipelines 1025, each cell pipeline 1025 including one or more service pipelines 1030. Service pipeline 1030 may include stages, each stage representing a pipeline that represents instructions for deploying a service for a specific environment. The lowest-level pipeline or leaf-level pipeline in the hierarchy is called a unit pipeline and may include detailed service-specific instructions for performing service-related operations. For example, service deployment may include pre-deployment steps, deployment steps, post-deployment steps, and post-deployment testing and verification steps. A pipeline that is not a leaf-level pipeline and has one or more sub-pipelines is an aggregate pipeline that performs the arrangement of sub-pipelines.

A pipeline can be driven by pull requests, which occur when the software's version control system receives requests to consider committing changes to an external repository for inclusion in the project's main repository. Therefore, when a pull request is received, a pipeline is automatically triggered, and software artifacts are deployed based on the latest software version that received the request. The pipeline performs continuous delivery of software artifacts based on pull requests. For example, a pipeline can be driven on an on-demand basis by calling requests using the application programming interface (API) of deployment module 210. On-demand deployment based on a pipeline can be requested for any set of services and any version of a given service specified using the API. A pipeline can be invoked to request a rollback from the current version to a previous version, or a rollforward from the currently deployed version to a newer version.

In the implementation, deployment module 210 creates service pipelines for each service. These pipelines are triggered when a pull request for the software repository is received. Deployment module 210 receives pipeline templates from users for specific services. These pipeline templates include detailed instructions for testing, validating, building, etc., for specific services. Data center generation module 220 receives cloud platform-independent declarative specifications for one or more data centers. Data center generation module 220 generates (or configures) data centers based on the received cloud platform-independent declarative specifications. Deployment module 210 receives the Advancement Decision 815 pipeline. The Advancement Decision 815 pipeline is incorporated into the overall pipeline.

The pipeline generator creates all pipelines for each data center from a template and combines them hierarchically via main pipelines, as shown in Figure 10. In the implementation, the pipeline generator generates service pipelines for each service; the pipeline generator generates cell main pipelines to invoke service pipelines; the pipeline generator generates service group main pipelines to invoke cell main pipelines; the pipeline generator generates data center instance main pipelines to invoke service group pipelines; and the pipeline generator generates service main pipelines to invoke data center instance main pipelines.

The following is a snippet showing the pipeline for each stage. Each stage can specify attributes, including stage name, pipeline type, stage type (e.g., main deployment pipeline or advance pipeline), previous stage, and so on.

As shown in the review pipeline, the first phase is artifact version mapping. The next phase is the main deployment pipeline for deployment to the development environment. The next phase is the advancement pipeline for determining whether the software artifact can be advanced to the next phase. The next phase is the main deployment pipeline for deployment to the test environment. The next phase is the advancement pipeline for determining whether the software artifact can be advanced to the next phase (i.e., the simulation environment).

Software artifact version mapping

In the implementation, deployment module 210 receives an artifact version map that associates various software artifacts and their versions with data center entities. The artifact version map provides a declarative specification of the specific versions of the software artifacts that need to be deployed for services in different data center entities. Each data center entity can be uniquely identified based on its position in the data center hierarchy as specified by the data center's declarative specification. For example, for a service, a software library can act as a software artifact. Software artifacts can have multiple versions, such as V1, V2, V3, etc. The artifact version map can specify that version V1 needs to be deployed in data center entities C1 and C2, and version V2 needs to be deployed in data center entities C3 and C4. Deployment module 210 generates supervisor lines and instructions that ensure the appropriate software artifact versions are deployed in the data center entities as specified in the artifact version map.

In the implementation, artifact version mappings are specified as JSON (JavaScript Object Notation) files, YAML files, or files using any other syntax for representing nested objects. Artifact version mappings may include a set of <service>:<version> key pairs associated with various data center entities distributed across a data center hierarchy. The artifact version mapping key pairs act as a whitelist for corresponding pipelines. If a service's key is not included in the artifact version mapping, then all pipelines for that service are excluded during pipeline execution. Different artifact version mappings can be applied to the same main pipeline, resulting in different services being included/excluded during pipeline execution.

The following is an example artifact version mapping. The artifact version mapping uses the attribute "environment_type" to specify the environment type. In the following embodiments, an environment type development is specified. An environment type may include one or more data center instances; a data center instance may include one or more service groups, and a service group may include one or more services. In the following embodiments, the software artifact name is specified as library1, the version is specified as version1, and it is associated with the service instance instance001. However, the software artifact name and version can be associated with a data center entity at any level of the tier. For example, for a specified software artifact name and version or service group, the software artifact name and version apply to all services within the service group, unless the software artifact name and version are overridden by different values of the software artifact name and version specified for a specific service instance within the service group.

Similarly, unless an overridden value is specified for a service group, the software artifact name and version can be specified for the data center instance and apply to all service groups or cells within the data center instance.

In the implementation, artifact version mapping uses the full path of the data center entity, such as "stagger_group1/datacenter1/service_group2/service1", to specify the data center entity. Alternatively, artifact version mapping uses a regular expression within the full path of the data center entity to specify a group of data center entities. For example, a full path including service_group[?] includes service_group1, service_group2, service_group3, and so on.

The following is an example of artifact version mapping, which specifies regular expressions to define service sets. The environment type is specified as Development and Test, and the data center entities (including data center instances and service groups) in the full path are specified as wildcards, and the service instances are specified as "service*". Therefore, for all data center instances, all service groups, and service names matching "service*" in the Development and Test environments, version V1 of application app1 will be deployed.

In some implementations, the workpiece version mapping can specify the parameters used by the pipeline. Therefore, the specified parameters will apply to the staggered groups for which the parameters are specified.

Figure 11 illustrates an exemplary pipeline for a data center according to an implementation scheme. As shown in Figure 11, the pipeline executes on a deployment group (also referred to as an interleaved group), which refers to a collection of data center entities defined by a declarative specification. An artifact version mapping can specify interleaved group attributes to define a set of data center entities that can be associated with a particular artifact version. Data center entities can be identified by specifying a path from the root node in the data center hierarchy, such as a data center instance. As shown in Figure 11, the pipeline defines different phases for different environments. Each phase executes actions on a set of data center entities referred to as deployment group 1110. Figure 11 illustrates a development deployment group 1110a including data center entities, which includes service groups S11, S12, and S13; a test deployment group 1110b including data center entities, which includes service groups S21, S22, and S23; and a production deployment group 1110b including data center entities, which includes service groups S31, S32, S33, and S34. Evaluate the test results of 1120a on the data center entity of development deployment group 1110a to determine whether the software artifacts tested in development deployment group 1110a are advanced to test deployment group 1110b. Similarly, evaluate the test results of 1120b on the data center entity of test deployment group 1110a to determine whether the software artifacts tested in test deployment group 1110a are advanced to production deployment group 1110c.

Figure 12 illustrates how the execution of the pipeline is modified based on artifact version mapping according to an implementation scheme. Artifact version mapping associates versions of software artifacts with data center entities. The system modifies the detailed pipelines executed on data center entities to ensure that actions associated with the appropriate version of the software artifact are executed on each data center entity. For example, Figure 12 shows versions V1 and V2 of the software artifacts used for service SVC1. Version V1 is associated with service groups S11, S23, and S31, and version V2 is associated with service groups S11, S21, and S32. Assume the pipeline is deploying a version of service SVC1 on the data center. Therefore, the system modifies the executed pipelines to deploy version V1 on service groups S11, S23, and S31 and version V2 on service groups S11, S21, and S32. In one implementation, the system introduces a version mapping filter prior to the phase of the pipeline shown in Figure 11. The execution of the pipeline selects the version of the software artifact for each data center entity based on the version mapping filter. For example, the version mapping filter includes instructions to skip service groups S13, S22, S33, and S34 for that specific action. The version mapping filter specifies that version V1 is deployed on service groups S11, S23, and S31, and version V2 is deployed on service groups S11, S21, and S32.

Figure 13 illustrates the overall process for deploying software artifacts on a data center configured on a cloud platform, according to the implementation scheme. The data center generation module generates one or more data centers on the target cloud platform. Each data center is generated according to a declarative specification independent of the cloud platform and has a hierarchy of data center entities.

The software release management module 230 receives an artifact version mapping that maps data center entities to versions of software artifacts as input. The software release management module 230 also receives a cloud platform-independent main pipeline as input.

The software release management module 230, in conjunction with the artifact version mapping compilation 1340, generates a cloud-platform-independent main pipeline to produce a cloud-platform-specific detailed pipeline. In one implementation, the generated cloud-platform-specific detailed pipeline includes artifact version mapping filters prior to certain stages to determine whether certain stages should be enabled or disabled based on the artifact version mapping.

The software release management module 230 further receives 1350 code for releasing one or more features of a service deployed on a data center. For example, the code may represent source code obtained from a version control management system that stores a repository of source code to which developers submit changes. The software release management module 230 executes 1360 a cloud platform-specific deployment pipeline to deploy software artifacts based on the received code.

Artifact version mappings and pipelines can be used to orchestrate various types of operations related to the continuous delivery of software artifacts in cloud-based data centers. Artifact version mappings and pipelines can be configured to perform aggregated retry operations on services, service groups, or any data center entity. Artifact version mappings include configurations for retry operations on data center entities, including retry policies, a threshold number of retries to perform in the event of a failure in one phase of the execution pipeline, whether user acknowledgment is required before a retrieval or whether retries are performed automatically, and so on. For example, a retry policy could be a fixed fallback policy that pauses execution for a fixed period before a retry. Artifact version mappings and pipelines can be used to configure other retry policies. In an implementation, the pipeline generator introduces a retry phase within the aggregate pipeline to trigger the retry policy if a previous pipeline phase fails. The retry policy and configuration parameters specified for a data center entity apply to all data center entities and services within that data center entity, unless the value is overridden by nested data center entities.

Computer Architecture

Figure 14 is a high-level block diagram illustrating a functional view of a typical computer system used as one of the entities shown in environment 100 of Figure 1, according to an embodiment. At least one processor 1402 is shown connected to chipset 1404. Also connected to chipset 1404 are memory 1406, storage device 1408, keyboard 1410, graphics adapter 1412, pointing device 1414, and network adapter 1416. Display 1418 is connected to graphics adapter 1412. In one embodiment, the functionality of chipset 1404 is provided by memory controller hub 1420 and I/O controller hub 1422. In another embodiment, memory 1406 is directly connected to processor 1402 instead of chipset 1404.

Storage device 1408 is a non-transitory computer-readable storage medium, such as a hard disk drive, optical disc read-only memory (CD-ROM), DVD, or solid-state storage device. Memory 1406 stores instructions and data used by processor 1402. Pointing device 1414 may be a mouse, trackball, or other type of pointing device and is used in conjunction with keyboard 1410 to input data into computer system 200. Graphics adapter 1412 displays images and other information on monitor 1418. Network adapter 1416 connects computer system 1400 to a network.

As is known in the art, computer 1400 may have different and/or other components than those shown in FIG. 14. Furthermore, computer 1400 may lack certain components illustrated. For example, computer system 1400 acting as multi-tenant system 110 may lack keyboard 1410 and point-of-sale device 1414. Additionally, storage device 1408 may be local and/or remote from computer 1400 (e.g., embodied within a storage area network (SAN)).

Computer 1400 is adapted to execute computer modules for providing the functions described herein. As used herein, the term "module" refers to computer program instructions and other logic for providing specified functions. Modules may be implemented in hardware, software, and/or firmware. A module may include one or more processes, and/or be provided only by a portion of those processes. Modules are typically stored on storage device 1408, loaded into memory 1406, and executed by processor 1402.

The type of computer system 1400 used by the entities in the system environment can vary depending on the implementation scheme and the processing power used by the entities. For example, client devices may be mobile phones with limited processing power, small displays 1418, and may lack fixed-point devices 1414. In contrast, multi-tenant systems or cloud platforms may include multiple blade servers working together to provide the functionality described herein.

Other precautions

Specific naming of components, capitalization of terms, attributes, data structures, or any other programming or structural aspects are not mandatory or important, and the mechanisms for implementing the described implementation schemes may have different names, formats, or protocols. Furthermore, as described, the system may be implemented via a combination of hardware and software, or entirely with hardware components. Moreover, the specific division of functionality among the various system components described herein is merely exemplary and not mandatory; a function performed by a single system component may alternatively be performed by multiple components, and a function performed by multiple components may alternatively be performed by a single component.

Some of the descriptions above are characterized by the algorithms and symbolic representations of the operations on information. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. Although these operations are described functionally or logically, they are understood to be implemented by computer programs. Furthermore, it is sometimes convenient, without loss of generality, to refer to these operations as modules or functional names.

Unless explicitly stated in the discussion above, it should be understood that throughout the description, the use of terms such as “processing” or “calculating” or “determining” or “displaying” refers to the actions and processes of a computer system or similar electronic computing device that manipulate and convert data represented as physical (electronic) quantities within the computer system’s memory or registers or other such information storage, transmission or display devices.

Some implementations described herein include processing steps and instructions described in algorithmic form. It should be noted that the processing steps and instructions of the implementation can be embodied in software, firmware, or hardware, and when embodied in software, can be downloaded to reside on and operated from different platforms used by a real-time network operating system.

The described embodiments also relate to means for performing the operations described herein. The means may be specifically constructed for the desired purpose, or it may include a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer-readable medium accessible to a computer. Such a computer program may be stored in a non-transitory computer-readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards or optical cards, application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each medium is coupled to a computer system bus. Furthermore, the computer mentioned in the specification may include a single processor, or may be an architecture employing multiple processors to increase computing power.

The algorithms and operations presented herein are not substantially related to any specific computer or other device. Various general-purpose systems can also be used with the programs taught herein, or it can be demonstrated that it is convenient to construct more specialized devices to perform the required method steps. The necessary structures for various such systems, as well as equivalent variations, will be apparent to those skilled in the art. Furthermore, this embodiment is described without reference to any specific programming language. It should be understood that the teachings of the embodiments described herein can be implemented using various programming languages.

The implementation scheme is well-suited for a wide variety of computer network systems across diverse topologies. In this field, the configuration and management of large networks involves storage devices and computers communicatively connected to different computers and storage devices via a network (such as the Internet).

Finally, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and may not be intended to depict or limit the subject matter of the invention. Therefore, the disclosure of embodiments is intended to be illustrative, not restrictive.

Claims (57)

1. A method implemented on a computer for deploying service software artifacts configured to run in a data center on a cloud platform, the method comprising: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service or (2) one or more of one or more other data center entities; Receive an artifact version mapping independent of the cloud platform, the artifact version mapping associating a data center entity of the data center with a version of a software artifact targeted for deployment on the data center entity, each software artifact including executable instructions associated with a service configured for execution on one or more cloud platforms; Based on the cloud platform-independent artifact version mapping, a cloud platform-specific deployment pipeline is generated for the target cloud platform. The cloud platform-specific deployment pipeline includes instructions for building and deploying appropriate versions of deployment artifacts for services on data center entities according to the cloud platform-independent artifact version mapping. The cloud-specific deployment pipeline is transferred to the target cloud platform for execution. 2. The computer-implemented method according to claim 1, further comprising: Generate a cloud-independent deployment pipeline, which includes instructions for building and deploying software artifacts on the data center; and Combine the cloud-independent artifact version mapping to compile the cloud-independent deployment pipeline. 3. The computer-implemented method of claim 2, wherein the cloud-independent deployment pipeline includes a version mapping filter prior to one or more stages, wherein execution of the pipeline causes the selection of a version of a software artifact for each data center entity based on the version mapping filter. 4. The computer-implemented method according to claim 1, further comprising: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 5. The computer-implemented method of claim 1, wherein compiling the cloud-independent declarative specification comprises: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 6. The computer-implemented method according to claim 5, further comprising: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and The data center is deployed on the target cloud platform based on the platform-specific detailed metadata representation. 7. The computer-implemented method of claim 1, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 8. The computer-implemented method of claim 1, wherein the generated instructions comprise one or more pipelines, each pipeline comprising a sequence of stages, each stage performing one or more actions for deploying software artifacts on a data center entity. 9. A non-transitory computer-readable storage medium for storing instructions, which, when executed by a computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps comprising: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service and (2) one or more of one or more other data center entities; Receive an artifact version mapping independent of the cloud platform, the artifact version mapping associating a data center entity of the data center with a version of a software artifact targeted for deployment on the data center entity, each software artifact including executable instructions associated with a service configured for execution on one or more cloud platforms; Based on the cloud platform-independent artifact version mapping, a cloud platform-specific deployment pipeline is generated for the target cloud platform. The cloud platform-specific deployment pipeline includes instructions for building and deploying appropriate versions of deployment artifacts for services on data center entities according to the cloud platform-independent artifact version mapping. The cloud-specific deployment pipeline is transferred to the target cloud platform for execution. 10. The non-transitory computer-readable storage medium of claim 9, wherein the instructions further cause the processor to perform the following steps: Generate a cloud-independent deployment pipeline, which includes instructions for building and deploying software artifacts on the data center; and Combine the cloud-independent artifact version mapping to compile the cloud-independent deployment pipeline. 11. The non-transitory computer-readable storage medium of claim 10, wherein the cloud platform-independent deployment pipeline includes a version mapping filter prior to one or more stages, wherein execution of the pipeline causes the selection of a version of the software artifact for each data center entity based on the version mapping filter. 12. The non-transitory computer-readable storage medium of claim 9, wherein the instructions further cause the processor to perform the following steps: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 13. The non-transitory computer-readable storage medium of claim 9, wherein compiling the cloud-independent declarative specification causes the processor to perform the following steps: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further cause the processor to perform the following steps: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and The data center is deployed on the target cloud platform based on the platform-specific detailed metadata representation. 15. The non-transitory computer-readable storage medium of claim 9, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 16. The non-transitory computer-readable storage medium of claim 9, wherein the generated instructions comprise one or more pipelines, each pipeline comprising a sequence of stages, each stage performing one or more actions for deploying software artifacts on a data center entity. 17. A computer system, comprising: Computer processor; and A non-transitory computer-readable storage medium for storing instructions, which, when executed by the computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps including: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service and (2) one or more of one or more other data center entities; Receive an artifact version mapping independent of the cloud platform, the artifact version mapping associating a data center entity of the data center with a version of a software artifact targeted for deployment on the data center entity, each software artifact including executable instructions associated with a service configured for execution on one or more cloud platforms; Based on the cloud platform-independent artifact version mapping, a cloud platform-specific deployment pipeline is generated for the target cloud platform. The cloud platform-specific deployment pipeline includes instructions for building and deploying appropriate versions of deployment artifacts for services on data center entities according to the cloud platform-independent artifact version mapping. The cloud-specific deployment pipeline is transferred to the target cloud platform for execution. 18. The computer system of claim 17, wherein the instructions further cause the processor to perform the following steps: Generate a cloud-independent deployment pipeline, which includes instructions for building and deploying software artifacts on the data center; and Combine the cloud-independent artifact version mapping to compile the cloud-independent deployment pipeline. 19. The computer system of claim 18, wherein the cloud-independent deployment pipeline includes a version mapping filter prior to one or more stages, wherein execution of the pipeline causes the selection of a version of a software artifact for each data center entity based on the version mapping filter. 20. The computer system of claim 17, wherein the instructions further cause the processor to perform the following steps: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 21. A method implemented on a computer for deploying service software artifacts configured to run in a data center on a cloud platform, the method comprising: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service and (2) one or more of one or more other data center entities; Generate a cloud platform-independent pipeline, the pipeline including a sequence of phases for deploying software artifacts, the sequence of phases including one or more of development phases, testing phases, and production phases, wherein at least some of the phases in the sequence specify criteria for advancing the software artifacts from the phases in the sequence to subsequent phases in the sequence; Compile the cloud-platform-independent primary deployment pipeline to generate a cloud-platform-dependent detailed deployment pipeline for the target cloud platform, the detailed deployment pipeline having instructions for deploying services based on the data center layout defined according to the cloud-platform-independent declarative specification; and Software artifacts are deployed for one or more services by executing detailed pipelines that depend on the cloud platform on the target cloud platform. 22. The computer-implemented method according to claim 21, further comprising: Receive information for customizing the cloud platform-independent primary deployment pipeline, wherein for the customized cloud platform-independent primary deployment pipeline, generate the platform-dependent detailed deployment pipeline for the target cloud platform. 23. The computer-implemented method of claim 22, wherein the information modification for customizing the cloud platform-independent main deployment pipeline is used to advance the software artifact from a first stage of the sequence to a second stage of the sequence. 24. The computer-implemented method of claim 21, wherein a specific phase includes instructions for executing a set of test cases, and wherein the criteria for advancing the software artifact includes a success rate of executing the test cases. 25. The computer-implemented method according to claim 21, further comprising: Provide users with access to the services that are running in the data center deployed on the target cloud platform. 26. The computer-implemented method of claim 21, wherein the cloud-independent primary deployment pipeline includes a hierarchy of pipelines having a hierarchical structure corresponding to the hierarchy of a data center entity specified by the cloud-independent declarative specification. 27. The computer-implemented method of claim 26, wherein the pipeline hierarchy includes a data center instance pipeline, the data center instance pipeline including one or more service group pipelines, wherein the service group pipeline includes one or more service pipelines. 28. The computer-implemented method according to claim 21, further comprising: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 29. The computer-implemented method of claim 21, wherein compiling the cloud-platform-independent declarative specification comprises: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 30. The computer-implemented method according to claim 29, further comprising: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and The data center is deployed on the target cloud platform based on the platform-specific detailed metadata representation. 31. The computer-implemented method of claim 21, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 32. A non-transitory computer-readable storage medium for storing instructions, which, when executed by a computer processor, cause the computer processor to perform steps for a service deployment software artifact configured to be executed in a data center on a cloud platform, the steps comprising: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service and (2) one or more of one or more other data center entities; Generate a cloud platform-independent pipeline, the pipeline including a sequence of phases for deploying software artifacts, the sequence of phases including one or more of development phases, testing phases, and production phases, wherein at least some of the phases in the sequence specify criteria for advancing the software artifacts from the phases of the sequence to subsequent phases of the sequence; Compile the cloud-independent primary deployment pipeline to generate a cloud-platform-dependent detailed deployment pipeline for the target cloud platform, the detailed deployment pipeline having instructions for deploying services according to the layout of the data center defined according to the cloud-independent declarative specification; and Software artifacts are deployed for one or more services by executing the detailed pipeline that depends on the cloud platform on the target cloud platform. 33. The non-transitory computer-readable storage medium of claim 32, wherein the instructions further cause the processor to perform the following steps: Receive information for customizing the cloud-independent primary deployment pipeline, wherein for the customized cloud-independent primary deployment pipeline, generate the platform-dependent detailed deployment pipeline for the target cloud platform. 34. The non-transitory computer-readable storage medium of claim 33, wherein the information modification for customizing the cloud platform-independent main deployment pipeline is a standard for advancing the software artifact from the first stage of the sequence to the second stage of the sequence. 35. The non-transitory computer-readable storage medium of claim 32, wherein the information modification for customizing the cloud platform-independent main deployment pipeline is used to advance the software artifact from the first stage of the sequence to the second stage of the sequence according to the standard. 36. The non-transitory computer-readable storage medium of claim 32, wherein the cloud-independent primary deployment pipeline includes a hierarchy of pipelines having a hierarchical structure corresponding to the hierarchy of the data center entity specified by the cloud-independent declarative specification. 37. The non-transitory computer-readable storage medium of claim 36, wherein the pipeline hierarchy includes a data center instance pipeline, the data center instance pipeline including one or more service group pipelines, wherein the service group pipeline includes one or more service pipelines. 38. The non-transitory computer-readable storage medium of claim 32, wherein the instructions further cause the processor to perform the following steps: Receive the cloud platform-independent declarative specification; and Compiling the cloud-platform-independent declarative specification to generate a cloud-platform-specific data center representation, wherein compiling the cloud-platform-independent declarative specification causes the processor to perform the following steps: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 39. The non-transitory computer-readable storage medium of claim 32, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 40. A computer system, comprising: Computer processor; and A non-transitory computer-readable storage medium for storing instructions, which, when executed by the computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps including: Access a data center configured on a target cloud platform, the data center being generated based on a declarative specification independent of the cloud platform, the declarative specification including a hierarchy of data center entities, wherein each data center entity includes (1) a service and (2) one or more of one or more other data center entities; Generate a cloud platform-independent pipeline, the pipeline including a sequence of phases for deploying software artifacts, the sequence of phases including one or more of development phases, testing phases, and production phases, wherein at least some of the phases in the sequence specify criteria for advancing the software artifacts from the phases of the sequence to subsequent phases of the sequence; Compile the cloud-independent primary deployment pipeline to generate a cloud-platform-dependent detailed deployment pipeline for the target cloud platform, the detailed deployment pipeline having instructions for deploying services according to the layout of the data center defined according to the cloud-independent declarative specification; and Software artifacts are deployed for one or more services by executing the detailed pipeline that depends on the cloud platform on the target cloud platform. 41. The computer-implemented method of claim 1 or 2, wherein the cloud-independent deployment pipeline includes a version mapping filter prior to one or more stages, wherein execution of the pipeline causes the selection of a version of a software artifact for each data center entity based on the version mapping filter. 42. The computer-implemented method according to any one of claims 1, 2, and 41, further comprising: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 43. The computer-implemented method according to any one of claims 1, 2, 41, and 42, wherein compiling the cloud-platform-independent declarative specification comprises: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 44. The computer-implemented method according to any one of claims 1, 2, and 41-43, further comprising: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and The data center is deployed on the target cloud platform based on the platform-specific detailed metadata representation. 45. The computer-implemented method according to any one of claims 1, 2 and 41-44, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 46. The computer-implemented method according to any one of claims 1, 2 and 41-45, wherein the generated instructions comprise one or more pipelines, each pipeline comprising a sequence of stages, each stage performing one or more actions for deploying software artifacts on a data center entity. 47. The computer-implemented method of claim 21 or 22, wherein the information modification for customizing the cloud platform-independent main deployment pipeline is used to advance the software artifact from the first stage of the sequence to the second stage of the sequence according to the criteria. 48. The computer-implemented method according to any one of claims 21, 22 and 47, wherein a particular phase includes instructions for executing a set of test cases, and wherein the criteria for advancing the software artifact includes a success rate of executing the test cases. 49. The computer-implemented method according to any one of claims 21, 22, 47, and 48, further comprising: Provide users with access to the services that are running in the data center deployed on the target cloud platform. 50. The computer-implemented method according to any one of claims 21, 22, and 47-49, wherein the cloud-independent primary deployment pipeline includes a hierarchy of pipelines having a hierarchical structure corresponding to the hierarchy of the data center entity specified by the cloud-independent declarative specification. 51. The computer-implemented method according to any one of claims 21, 22, and 47-50, wherein the pipeline hierarchy includes a data center instance pipeline, the data center instance pipeline including one or more service group pipelines, wherein the service group pipeline includes one or more service pipelines. 52. The computer-implemented method according to any one of claims 21, 22, and 47-51, further comprising: Receive the cloud platform-independent declarative specification; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 53. The computer-implemented method according to any one of claims 21, 22, and 47-52, wherein compiling the cloud platform-independent declarative specification comprises: A first version of a cloud platform-independent detailed metadata representation of the data center is generated from the original declarative specification; and A second version of the cloud platform-independent detailed metadata representation of the data center is generated from the modified declarative specification. 54. The computer-implemented method according to any one of claims 21, 22, and 47-53, further comprising: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and The data center is deployed on the target cloud platform based on the platform-specific detailed metadata representation. 55. The computer-implemented method according to any one of claims 21, 22 and 47-54, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 56. A non-transitory computer-readable storage medium for storing instructions, which, when executed by a computer processor, cause the computer processor to perform the steps of any one of the preceding method claims. 57. A computer system comprising: Computer processor; and A non-transitory computer-readable storage medium for storing instructions, which, when executed by the computer processor, cause the computer processor to perform the steps of any one of the preceding method claims.