Sea Change Series: Orchestration Software in the Mega Data Center

Published: May, 2017 | Pages: 115 | Publisher: WinterGreen Research
Industry: ICT | Report Format: Electronic (PDF)

Business growth depends on technology spending. Intelligent, automated process, not manual labor systems are what speed business growth. We have had the situation in the data center where 93% of spending is just to keep current systems running, many of those plagued with manual input. Mega data centers change that pattern of IT manual process.

The Internet has grown by a factor of 100 over the past 10 years. To accommodate that growth, mega data centers have evolved to provide processing at scale. Facebook for one, has increased the corporate data center compute capacity by a factor of 1,000, virtually eliminating much manual process. Orchestration software is a key aspect of that process. To meet future demands on the Internet over the next 10 years, companies with that capacity need to increase capacity by the same amount again while the other companies struggle to catch up. Nobody really knows how to get to increasing compute capacity by another factor of 1,000.

Realigning data center cost structures is a core job of orchestration software. The enterprise data centers and many cloud infrastructure operations all have similar problems of being mired in administrative expense. Containers address that issue by creating vastly more efficient operations for data center infrastructure.

According to Susan Eustis, lead author of the team that prepared the study, “The only way to realign cost structure is to automate infrastructure management and orchestration. Mega data centers automate server and connectivity management using orchestration software to manage multiple application containers. Other systems automate switching and storage, along with hypervisor, operating system, and virtual machine provisioning. “

As IT relies more on virtualization and cloud mega data center computing, the physical infrastructure is flexible and agile enough to support the virtual infrastructure. Comprehensive infrastructure management and orchestration is essential.

The Enterprise Data Center has become a bottleneck, it needs to be completely replaced. Category 5 and Category 6 Ethernet cable is spread throughout the existing enterprise data centers and is too slow to handle all the digital data coming through the data center. Cat 5 and Cat 6 Ethernet utilized by the servers to achieve data transport using that cable does not keep up with the data coming through the data center the way optical cable and optical transceivers do.

The existing servers and cable are a problem because they are too slow for modern systems. The cable is too slow to handle all the data coming at us in the new digital age, and the associated technology that operates at Ethernet category 5 and category 6 cable speeds is too slow as well, this is why the entire set of existing enterprise data centers is a bottleneck.

Mobile data traffic is set to increase by a factor of eight between 2015 and 2020. Growth is anticipated at 53 percent per year, faster than systems revenue or industry revenue.

The theme of this study is that the pace of data expansion creates the need for more modern means of managing data. There are some companies that are doing a better job, better than others of adapting to IT infrastructure to the wild influx of data.

The four superstar companies that are able to leverage IT to achieve growth, Microsoft, Google, Facebook, and the leader AWS all use Clos architecture. What is significant is that systems have to hit a certain scale before Clos networks work Clos networks are what work now for flexibility and supporting innovation in an affordable manner. There is no dipping your toe in to try the system to see if it will work, it will not and then the IT says, “We tried that, we failed,” but what the executive needs to understand is that scale matters. A little mega data center does not exist. Only scale works.

Many companies are using digital technology to create market disruption. Amazon, Uber, Google, IBM, and Microsoft represent companies using effective strategic positioning that protects the security of the data. As entire industries shift to the digital world, once buoyant companies are threatened with disappearing. A digital transformation represents an approach that enables organizations to drive changes in their business models and ecosystems leveraging cloud computing, and not just hyperscale systems but leveraging mega data centers. Just as robots make work more automated, so also cloud based communications systems implement the IoT digital connectivity transformation.

WinterGreen Research is an independent research organization funded by the sale of market research studies all over the world and by the implementation of ROI models that are used to calculate the total cost of ownership of equipment, services, and software. The company has 35 distributors worldwide, including Global Information Info Shop, Market Research.com, Research and Markets, electronics.ca, and Thompson Financial. It conducts its business with integrity.

The increasingly global nature of science, technology and engineering is a reflection of the implementation of the globally integrated enterprise. Customers trust wintergreen research to work alongside them to ensure the success of the participation in a particular market segment.

WinterGreen Research supports various market segment programs; provides trusted technical services to the marketing departments. It carries out accurate market share and forecast analysis services for a range of commercial and government customers globally. These are all vital market research support solutions requiring trust and integrity.

Companies Profiled

Market Leaders
• Amazon
• Microsoft
• Google
• Facebook

Key Topics

• Orchestration Software In The Mega Data Center
• Realign IT Cost Structure
• Mega Datacenter Physical Infrastructure
• Automation of Mega Data Center
• Networking Fabric
• Exchange Of Data Between Servers
• Complex Automation Of Process
• Applications Customized For Each User
• Machine-To-Machine Management of Traffic Growth
• Fabric Network Topology
• Building-Wide Connectivity
• Highly Modular Data Cebter Design
• Scale Capacity
• Back-End Service Tiers
• Applications Scaling
• Mega Data Center Network
• Fabric Next-Generation Data Center Network Design
• Pod Unit of Network
• Mega Data Center Server Pods
• Non-Blocking Network Architecture
• Data Center Auto Discovery
• Large-Scale Network
• Rapid Deployment Architecture
• Expedites Provisioning And Changes
• Programmable Access To Network Stack
• Software Defined Networking (SDN)-Supports Scale and Automation
• Compute Engine Load Balancing
• Load Balanced Requests Architecture
• Scale-Out: Server And Storage Expansion
• Switches and Routers Deployed in Fabrics
• Mega Data Center Multipathing
• Routing Destinations
• Clos Topology Network
• Capacity Scalability
• Aggregation Switches
• Intelligent Cloud Platform
• Linux For Azure

 Table of Contents

Sea Change Series: Orchestration Software in the Mega Data Center
Sea Change Series: Orchestration Software in the Mega Data Center, Amazon, Google, Microsoft, Facebook 2
Aim to Realign IT Cost Structure 3
Internet Has Grown by a Factor of 100 Over The Past 10 Years 4
Table of Contents 5
Orchestration Software Automates Data Center Infrastructure 14
Software Containers 15
Orchestration Schedulers Manage Containers 16
Orchestration Software Supports Container Automation 17
Realigning Data Center Cost Structures 18
IT Relies On Replacing Virtual Machine: VM Virtualization 19
Microservices 20
Microservices Features 21
Microservices Modules 23
Difficulties with Virtual Machines 24
Hypervisor a Difficulty 25
Virtual Machines Use Bare Metal, Containers Use Orchestration Software 27
Bare Metal an Inefficient Use of Compute Resource 28
Bare Metal Less Efficient 28
Industry Uses Robots Because Manual Labor Is Slow And Error Prone 29
IT processes Replace Manual Labor 30
Mega Data Center Orchestration Software 31
Large Fabric Network Not The Kind Of Environment That Can Be Realistically Configured And Operated In A Manual Way 32
To Automate the Data Center Fabric 33
Value of Data Center Fabric 33
Google Shift from Bare Metal To Container Controllers 34
Google Container Controller Shift From Bare Metal In A Mixed Workload And In Nested Compute Units 35
Google Kubernetes Groups Software Containers 36
Fabric Services Inside A Container 37
Architecting Microsoft Cloud 38
Microsoft Managed Clustering and Container Management: Docker and Mesos 39
Microsoft Azure Service Fabric 41
Microsoft Is Seen As The Overall Winner In The Move To Application Containers 42
Microsoft Dublin Cloud 2.0 Mega Data Center 43
Microsoft Data Center, Dublin, 550,000 Square Feet 43
Microsoft Dublin Center Operates at a Power Usage Effectiveness (PUE) of 1.25. 44
Microsoft Data Center Container Area in Chicago. 44
Microsoft 45
Advantages of Using Containers 46
Orchestration Software Used to Create Containers 47
Disadvantages of Using Containers 48
Advantages of Virtual Machines 49
Container Orchestration 50
Use of Containers Eliminate Manual Process 51
IT Pros Increasingly Turn to Chef and Puppet 52
Hardware Containers Do Not Scale 53
Facebook Data Center Positioning 54
IBM Data Center Orchestration Software Automates Application Integration 55
Docker Orchestration & Docker Swarm 56
Docker Container Platform 57
Common Feature Sets For Orchestration Tools 58
Not all Orchestrators Are Created Equal 59
AWS Cloud Container Adoption Criteria 60
AWS Cloud Adoption Methodology 63
AWS Cloud Adoption Framework 64
AWS Market Leader In Cloud Computing 65
Facebook Fabric and Node are Core Structures Leveraging Software Orchestration 66
Apache Mesos Orchestration Software 67
Google Kubernetes Container 68
Google Container Builder Step Toward Building Pluggable Components in a Pipeline 69
Google Programmable Access To Network Stack 70
Google Andromeda Software Defined Networking (SDN)- 71
Google Compute Engine Load Balancing 72
Google Compute Engine Load Balanced Requests Architecture 73
Google Scaling Of The Compute Engine Load Balancing 74
Google Compute Engine (GCE) TCP Stream Performance Improvements 75
Google Cloud Platform TCP Andromeda Throughput Advantages 76
Google Open Sourced Its Container Management System Called Kubernetes 77
Facebook 79
Ability To Move Fast And Support Rapid Growth At The Core Of Facebook Infrastructure Design Philosophy 80
The Right Type of Cloud: Mega Data Centers 82
AWS Has Been Able To Adapt To Change 83
Manual Labor Is Slow And Error Prone 84
Mega Data Center Orchestration Software 86
Amazon, Google, Microsoft, Facebook 87
Cloud 2.0 Mega Data Center Fabric Implementation 87
Fabric and Node are Core Structures Leveraging Software Orchestration 89
Multi-Threading, Dynamic Systems 91
Oracle Multi-Threading Mega Data Center 92
Orchestration Tools Manage A Cluster As A Single Deployment 93
Microservice Monitoring with Google Kubernetes 94
Docker Container 94
Cluster Functions and Pod Benefits 95
Mesosphere DC/OS an Open-Source Project Built on Apache Mesos 96
Mesosphere Enterprise DC/OS Orchestration Software 96
Mesosphere DC/OS Production Containers Uses 97
Mesosphere DC/OS Orchestration Software 97
Mesosphere DC/OS Extending Capabilities Within Container Orchestration 98
Mesosphere DC/OS Certification Compliance 98
Mesosphere Market Leadership Position 99
Mesosphere DC/OS Runs Data Services on One Single Platform 99
Cloud Computing Not Enough: Entire Warehouse Building As A Single Mega Data Center System 100
Red Hat Ansible 101
Red Hat Ansible Architecture, Agents, And Security 102
Red Hat Ansible Advanced Features 102
Red Hat / Ansible 103
Red Hat Ansible Tower 3 Job Run Metrics 104
Cisco Integrated Infrastructure Management 105
Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Mesosphere DC/OS: Mesos Features 108
Heart of DC/OS: Apache Mesos 109
DC/OS Implements Containers 110
WinterGreen Research, 111
WinterGreen Research Methodology 112

List of Figures

Mega Data Center Depends on Orchestration Software to Control Operations, Implement Microservices
Figure 1. Slow Growth Mode of Companies with Enterprise Data Centers 3
Figure 2. Mega Data Center Fabric Implementation 4
Figure 3. Business Innovation and Technology 14
Figure 4. Docker Orchestration Software Creates Containers 15
Figure 5. Docker Compose 16
Figure 6. Mesosphere Marathon 16
Figure 7. Google Kubernetes 16
Figure 8. Orchestration Software Supports Container Automation 17
Figure 9. Orchestration Software Decreases Data Center Cost Structure 18
Figure 10. Files Bundled into a Container 19
Figure 11. Microservices: Suite Of Independently Deployable Service Modules with a Unique Process And Well-Defined, Lightweight Communication Portal: Mechanism To Serve A Business Goal 20
Figure 12. Microservices Distinct Features: Taxi Hailing Example 21
Figure 13. Microservices Market Segments 22
Figure 14. Microservices Modules 23
Figure 15. Hypervisor Virtualization Operating System Interface 24
Figure 16. Hypervisor Virtualization Operating System Interface 25
Figure 17. Virtual Machines Less Efficient Than Containers 26
Figure 18. Difference Between Virtual Machines and Containers 26
Figure 19. Bare Metal Management Replaced by Container Controllers 27
Figure 20. Containers vs. VMs 28
Figure 21. Industrial Robots Eliminate Manual Labor 29
Figure 22. Industry Uses Robots To Replace Manual Labor 29
Figure 23. Data Centers Need The Precision and Automation Similar to that Provided by Multi-Step Sequential Task Industrial Robots 30
Figure 24. Mega Data Center Orchestration Software 31
Figure 25. Single-Fabric Data Center Network Architecture 32
Figure 26. Bare Metal Presents a Lot of Extra Parameters and Metrics, Significantly More than With Containers 34
Figure 27. Nested Compute Units 35
Figure 28. Kubernetes Orchestration Software Groups Containers That Make Up An Application Into Logical Units 36
Figure 29. Kubernetes Orchestration Software Functions 36
Figure 30. Container Features as it integrates with the Service Fabric Runtime 37
Figure 31. Microsoft Setting Up A Secure Service Fabric Cluster in Azure using the Azure Portal. 40
Figure 32. Microsoft Data Center, Dublin, 550,000 Sf 43
Figure 33. Container Area In The Microsoft Data Center In Chicago 44
Figure 34. Microsoft Cloud Network Features 45
Figure 35. Like Physical Containers on a Ship, Software Containers Bring Many Servers Densely Packed 46
Figure 36. Advantages of Using Containers 47
Figure 37. Software Orchestration Container Challenges 48
Figure 38. Manual Process 51
Figure 39. Containers Need Orchestration Software 51
Figure 40. Virtual Machine Data Center Management Tasks: 52
Figure 41. FaceBook Open Compute Project 53
Figure 42. Facebook Data Center Modernization Functions 54
Figure 43. Facebook Altoona Iowa Cloud 2.0 Mega Data Center 54
Figure 44. Manual Process for Application Integration Deployment 55
Figure 45. Feature Sets For Orchestration Tools 58
Figure 46. Issues for Orchestration Software 59
Figure 47. AWS Cloud Container Adoption Criteria 60
Figure 48. AWS Cloud Container 61
Figure 49. AWS Cloud Adoption Framework 62
Figure 50. AWS Market Leader In Cloud Computing 65
Figure 51. Description of the Orchestration Software 67
Figure 52. Advantages of Using the Container Builder Cloud Architecture as a Service: 69
Figure 53. Google Andromeda Cloud High-Level Architecture 70
Figure 54. Google Andromeda Software Defined Networking (SDN)-Based Substrate Functions 71
Figure 55. Google Andromeda Performance Factors Of The Underlying Network72
Figure 56. Google Compute Engine Load Balanced Requests Architecture 73
Figure 57. Google Compute Engine Load Balancing 74
Figure 58. Google Cloud Platform TCP Andromeda Throughput Advantages 76
Figure 59. IoT: Open Source IoT High Level Platform, OpenStack and Kubernetes 78
Figure 60. Facebook DuPont Fabros Technology Ashburn, VA Data Center 81
Figure 61. Cloud 2.0 Mega Data Centers Support 1.5 Billion Facebook Users Worldwide. 82
Figure 62. AWS Market Leader In Cloud Computing 83
Figure 63. Data Centers Need The Precision and Automation Provided by Multi-Step Sequential Task Industrial Robots 85
Figure 64. Mega Data Center Orchestration Software Functions 86
Figure 65. Multiple Pathways Open To Processing Nodes In The Cloud 2.0 Mega Data Center Functions 90
Figure 66. Dynamic Load Balancing 91
Figure 67. Mesosphere Customer References 96
Figure 68. Mesosphere DC/OS Certification Compliance 98
Figure 69. Cloud Is Not Enough 100
Figure 70. Red hat Ansible Playbook Language Advanced Features 103
Figure 71. Red Hat Ansible Tower 3 Job Run Metrics 104
Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Figure 72. Cisco UCS Helps Manage Administrative Costs And Reign In Complexity 106
Figure 73. Cisco UCS Director Delivers Comprehensive Infrastructure Management and Orchestration 107
Figure 74. Mesosphere DC/OS: Mesos Features: 108
Figure 75. Native Mesos Containerizer Functions 110