Cloud Database Services – A Roadmap for Successful Cloud Data Migration Overview

Data is the key to all business operations and strategic decision making. That translates to data and databases as the most important asset within the IT ecosystem, whether data resides on-premise or in the cloud. As a result, data driven organizations deal with overwhelming challenges today due to rapid and explosive data growth.

These challenges include, but are not limited to, sustaining on-premise enterprise architecture, storage complexities, scalability requirements, application and database technology stacks, data unification, business continuity and high availability, security, and compliance.

These challenges drive the strategic decisions to improve business operations and revenue based on emerging technologies. Today, cloud options are discussed as a viable strategic solution to meet some or all of these challenges. Successful adoption to the cloud requires a roadmap and how-to guideline related to all IT components, especially relating to data and databases.

Cloud Justification

There are various strategic reasons as to why data centric organizations consider deployment of business functions to the cloud. Among the prevalent reasons include rapid data growth, e-commerce/e-business/web service advancements, emerging mobile apps and enterprise application software (EAS) requirements, 24x7 operations and high availability, flexible cost models and other evolving requirements.

From a macro level view, overall cost/benefit and business agility justification are often the compelling overall drivers. 

Cloud Models

Cloud Service Providers (CSPs) come in all flavors and sizes from the lower tier commodity providers to the high-end enterprise level providers - with complex geographically dispersed redundant data centers.

Today’s most common cloud service infrastructures include the acronyms IaaS, PaaS and SaaS. In short, IaaS (Infrastructure-as-a-Service) includes converged physical and virtual infrastructure which is supplied and hosted in secure data centers. PaaS (Platform-as-a-Service) includes IaaS provisions along with the layer for software components including OS, web servers, databases, software development tools and others. SaaS (Software-as-a-Service) includes PaaS provisions and bundles the pre-configured application software with business rules that are adapted to.

Matrixed in with IaaS/PaaS/SaaS are Public, Private and Hybrid models. In short, Public cloud is a virtually configured publically shared multitenant environment. Private cloud is typically a virtualized single owner and segregated environment. Hybrid refers to multiple delivery and integration models that could include on-premise and various cloud infrastructures.

Organizations will utilize the appropriate public, private and hybrid model or combination of depending on the business requirements. Typically, organizations moving to cloud services implement a hybrid approach.

Cloud Adoption and Data Services

Cloud adoption typically takes an incremental, iterative and multi-stage hybrid approach with most organizations implementing only selected applications, functionalities, data and processes at a time. Most organizations also implement a “proof of concept” project to test out the viability of cloud solutions for their specific business requirement.

As a stakeholder within an organization’s strategic cloud adoption team, the Data Architect/Analyst will participate in the business case analysis and utilize the DBA as needed for additional technical input.

The Data Architect/DBA will take on the primary responsibility of looking at the whole ecosystem 

surrounding the data and database depending on the selected application(s), functionality, and work-flow being transitioned.  Data centric recommendations will contribute greatly to the decision making process and whether to proceed with a cloud based solution.

Once a clear decision is made to move forward with cloud adoption, the Data Architect/DBA should then be included in developing the Request for Proposal (RFP) for the Cloud Service Providers (CSPs). During the RFP development process, the DBA should focus on reviewing and analyzing various CSP components based upon data related provisioning requirements. 

Cloud Pre-Deployment – The Role of the DBA

The DBA should be heavily involved in the cloud pre-deployment planning phase.  Typically, the DBA gets involved with such tasks as estimating storage needs, scoping virtual computing usage requirements, evaluating database architectural needs (e.g. high availability), determining backup and recovery, evaluating data archiving requirements and scoping other detailed data related, pre-deployment planning and challenges.

Building out cloud architecture properly can be a challenging endeavor. Improper architectural implementations can cause complications immediately with production issues and outages. So the DBA, Systems Administrator (SA) and other relevant IT personnel should look at all architectural considerations when moving data to the cloud.

Cloud integration services include (1) integrating existing on-premise application and data to the cloud or (2) creating new cloud-based applications and loading existing data from legacy apps or a (3) using a hybrid approach with many possible configurations.

The DBA is involved with various aspects of integration services including database creation, testing, quality assurance and other support functions within the provisioning of cloud architectural decisions previously made.

The DBA may also be involved with data unification issues if there is an initiative to consolidate. If necessary, the DBA is involved with pre-migration data cleansing tasks.

Regardless of the methodologies used to deploy to the cloud, the DBA is still an integral part of the integration pre-deployment effort related to data and database specific initiatives.

Cloud Deployment – The Role of the DBA

Once the pre-deployment tasks of planning, architecture and integration services have been completed and approval has been granted to move forward, then the actual cloud deployment involving migration and monitoring setup tasks are started.

Cloud migrations services for the DBA includes migrating existing on-premise production and legacy data for cloud ready use. Risks should have been minimized based on successful pre-deployment tasks.

During the migration phase of cloud deployment, the DBA works with the application development teams and appropriate IT staff (or specialized outsourced providers) regarding the specific migration components and work-flow that touches the data and databases.

The DBA also communicates and coordinates with the CSP deployment team to make sure all of the architectural components decided upon are implemented properly, especially the backup and redundancy configuration.

Production databases are configured and created in the cloud. The DBA initiates and monitors the data transfer tasks based on the appropriate extract and load methodology chosen. Database security standards are checked. Background and batch jobs are established. Appropriate database tuning features are set (although they will likely be tweaked once cloud production has been established for a while).  If using a hybrid model, interoperability and all appropriate configurations are checked and tested.

As with on-premise database environments, databases in the cloud should be pro-actively monitored so the DBA can stay on-top of critical database issues and problems.

It is very important that the DBA understand what database monitoring requirements are needed for cloud databases. Just as in on-premise, there are many factors that go into the database monitoring decisions in the cloud. These include licensing costs, compatibility issues, feature/function capabilities, portability, interoperability issues and others.

There also tends to be some confusion today about database monitoring in the cloud. The perception that the database is self-sustaining once in the cloud is simply not true. Some CSPs, especially the large public CSPs, provide native monitoring tools. However, in reality these tools have limitations for deep-dive internal database monitoring, metric gathering and the enabling proactive database alerts.

Cloud Post-Deployment – The Role of the DBA

Once databases have been deployed and established in the cloud, just as with on-premise environments, the DBA must manage and administer the databases. However, two fundamental and important questions must be answered in order to understand how the databases in the cloud will be managed and administered.

1.) Does the cloud service provider cover DBA support?

In most cases the CSP does not provide the services to do so. While CSPs work with you to scope infrastructure requirements and help you ramp up, they generally do not provide ‘Database as a Service’ (DaaS) for the pre-deployment and deployment related database functions and tasks.

2.) Are there differences for DBA services in the cloud? 

Yes, and the DBA must adapt to those support changes.  For instance, physical storage located in the cloud is managed by the CSP. Virtualization is managed in the cloud. Physical backup/recovery is managed by the CSP but there may be limitations (e.g. point-in-time recovery). There are also additional or adapted knowledge, skills, practices and other provisions required for the Cloud DBA.

Once databases reside in the cloud, holistic and optimal database support coverage is necessary as discussed in the white paper “Top 10 Database Essentials – A Catalog of DBA Coverage Necessities”[1].  The Top 10 Database Essentials discusses why breaking down DBA tasks into categorized domains will allow database services to be organized holistically minimizing the potential for any service gaps.

The Roadmap for Successful Cloud Data Migration provides an organization a blueprint for necessary DBA services to run operations efficiently, whether on-premise or in the cloud.