Data Center as A Bottleneck: Market Strategies, Analysis, and Opportunities

Published: Apr, 2017 | Pages: 138 | Publisher: WinterGreen Research
Industry: Technology & Media | Report Format: Electronic (PDF)

Worldwide hyperscale data center markets implement cloud computing with shared resource and the aim, more or less achieved of providing foolproof security systems that protect the integrity of corporate data. Cloud data centers are poised to achieve explosive growth as they replace enterprise web server farms with cloud computing and with cloud 2.0 automated process computing. The implementation of secure large computing capability inside data center buildings provides economies of scale not matched by current state of the art enterprise data center standalone server technology.

Economies of scale provide savings of between 50% to 100x less cost. These are savings that cannot be ignored by any person responsible or running a business.

Building size cloud 2.0 computer implementations feature simplicity of design achievable only with scale. These data centers implement cloud 2.0 in a move that works better than much of the current cloud computing. The cloud 2.0 data centers have been reduced to two types of components, an ASIC server: single chip servers and a network based on a matching ASIC switch. Data centers are implemented with a software controller for that ASIC server and switch infrastructure.

The major driving factors for Cloud 2.0 mega data center market are cost benefit, growing colocation services, need for data consolidation, and cloud. Amazon (AWS), Microsoft, Google, and Facebook data centers are in a class by themselves, they have functioning fully automatic, self-healing, networked mega datacenters that operate at fiber optic speeds to create a fabric that can access any node in any particular data center because there are multiple pathways to every node. In this manner, they automate applications integration for any data in the mega data center.

This module addresses the issue of data center bottlenecks initially by drawing the reader;s attention to an analogy: navigating a sailboat through Woods Hole on Cape cos Massachusetts. The navigation is tricky - potentially dangerous.

The bottleneck is potentially dangerous - for a combination of reasons. The current routinely flows through at over 4 knots, and can hit 7 knots. Full current on the nose makes transit slow and awkward. Full current from astern where the current runs slightly cross-channel causes awkward transit at an alarmingly rapid pace.
Existing Enterprise Data Center as a Bottleneck: Think Woods Hole

Viewed From The Cockpit: The Converging And Diverging Channels Can Look Like A Random Scattering Of Reds And Greens

The existing data centers have a lot of entrenched culture and equipment. Mainframes represent 86% of transaction data processing and function generally in a manner separated from web traffic, though they doo handle some web traffic. One issue is, “What to do with the existing mainframes with its separate culture, functioning at 115% of capacity, and utterly impregnable security?”

According to Susan Eustis, principal author of the study, “The mega data centers have stepped in to do the job of automated process in the data center, increasing compute capacity efficiently by simplifying the processing task into two simple component parts that can scale on demand. There is an infrastructure layer that functions with simple processor, switch, and transceiver hardware orchestrated by software. There is an application layer that functions in a manner entirely separate from the infrastructure layer. The added benefit of automated application integration at the application layer brings massive savings to the IT budget, replacing manual process for application integration. The mainframe remains separate from this mega data center adventure, staying the course, likely to hold onto the transaction management part o data processing.”

The only way to realign enterprise data center cost structures is to automate infrastructure management and orchestration. Mega data centers automate server and connectivity management. Cisco UCS Director illustrates software that automates everything beyond. Cisco UCS automates switching and storage, along with hypervisor, operating system, and virtual machine provisioning.

As IT relies more on virtualization and cloud mega data center computing, the physical infrastructure is flexible and agile enough to support the virtual infrastructure. Comprehensive infrastructure management and orchestration is essential. The enterprise data centers and many cloud infrastructure operations all have similar problems of being mired in administrative expense. This presents a problem for those tasked with running companies.

The Internet has grown by a factor of 100 over the past 10 years. To accommodate that growth, hyperscale data centers have evolved to provide processing at scale, known as cloud computing. Facebook for one, has increased the corporate data center compute capacity by a factor of 1,000. To meet future demands on the Internet over the next 10 years, the company needs to increase capacity by the same amount again. Nobody really knows how to get there. Tis study takes a hard look at the alternatives open to business leaders.

Everyone should know by now that the enterprise data center is dead. It will no longer exist in three years, that is the time it takes servers to become outdated and need replacement. In that timeframe, enterprises will migrate workload from the core enterprise servers to the large data center that can provide processing at half the cost of current processing. Maybe this forecast is too aggressive, but probably not. The mainframe stays around as detailed in a different WinterGreen Research report.

The Hyperscale Data Centers: market size at $86.9.7 million in 2016 is anticipated to be $359.7 billion in 2023. The market has astoundingly rapid growth for a market that really is not yet well defined. The increasing scope of applications across different industries, manufacturing, medical, retail, game, and automotive, all industries really, is expected to drive demand over the forecast period to these unprecedented levels, reaching into the trillion-dollar market arenas soon.

The hyperscale data centers are position to manage the explosion in web data, including data from IoT technology that is in the nascent stage with a huge growth potential, and has attracted large investments contributing to the industry growth.

WinterGreen Research is an independent research organization funded by the sale of market research studies all over the world and by the implementation of ROI models that are used to calculate the total cost of ownership of equipment, services, and software. The company has 35 distributors worldwide, including Global Information Info Shop, Market Research.com, Research and Markets, electronics.ca, and Thompson Financial. It conducts its business with integrity.

The increasingly global nature of science, technology and engineering is a reflection of the implementation of the globally integrated enterprise. Customers trust wintergreen research to work alongside them to ensure the success of the participation in a particular market segment.

WinterGreen Research supports various market segment programs; provides trusted technical services to the marketing departments. It carries out accurate market share and forecast analysis services for a range of commercial and government customers globally. These are all vital market research support solutions requiring trust and integrity.

Companies Profiled

Market Leaders
• Facebook
• Amazon (AWS)
• Microsoft
• Google

Market Participants
• 365 Data Centers
• Amazon
• Apple
• Alibaba
• Baidu
• Chef
• China Building A Cloud Computing Complex
• China Mobile
• Colocation America Data Center Bandwidth and Measurements
• Colo-D
• CoreSIte
• CyrusOne
• Digital Realty
• Docker
• DuPont Fabros Technology
• Edge ConneX
• Equinix
• Facebook
• Forsythe
• Google
• Hewlett Packard Enterprise
• IBM
• Intel
• I/O
• InterXion
• Mesosphere
• Microsoft
• US National Security Agency
• NEC
• NTT / RagingWire
• OpenStack Cloud Controller
• Puppet
• QTS
• Qualcom
• Rackspace
• Red Hat / Ansible
• Switch
• Tango
• Tencent
• Twitter
• Yahoo

Key Topics

• Hyperscale Data Center
• Scale
• Automation
• Cloud Computing
• Cloud 2.0
• Automatic Rules
• Push-Button Actions
• Cloud Application Integration
• Container Control System
• Open Source Container
• Bare Metal To Container Controllers
• Kubernetes Defacto Standard
• Container Management System
• Global IP Traffic
• Mega Data Center
• Google Kubernetes Defacto Standard Container
• Digital Data Expanding Exponentially
• Colocation Shared Infrastructure
• Power and Data Center Fault Tolerance
• 100 Gbps Adoption
• Data Center Architectures
• High-Performance Cloud Computing
• Core Routing Platform
• Datacenter Metrics
• Mega Data Center Fabric Implementation
• Digital Data
• Open Source Container Control System
• Defacto Standard Container Management System
• Co-Location, and Social Media Cloud
• Biggest Data Centers
• Cloud 2.0
• Intelligent Cloud Segment

 Table of Contents

SEA CHANGE SERIES: CLOUD 2.0, MEGA DATA CENTERS
Executive Summary 3

BOTTLENECKS: NAVIGATING WOODS HOLE IS TRICKY - POTENTIALLY DANGEROUS 5
Viewed From The Cockpit: The Converging And Diverging Channels Can Look Like A
Random Scattering Of Reds And Greens 6
Using the Red and Green Boys to Navigate 7
Nine-Foot Bay Of Fundy Tide 10
Video and Data Streams Create Bottlenecks: 11

Demand for New Types of Cloud 11
The Right Type of Cloud: Mega Data Centers, Cloud 2.0 12

Mega Data Center Scale and Automation 22
Only Way To Realign Data Center Cost Structure Is To Automate Infrastructure
Management And Orchestration 23
Entire Warehouse Building As A Single System 24
Half a Trillion Dollars 25
Two Tier Architecture to Achieve Simplicity 26
Bandwidth and Data Storage Demands Create Need For Application Integration 27

Cultural Shift 28
Line of Business Loses Control Of Hardware Servers 29
Cultural Change Needed to Move to Cloud 31
Adjusting to Rapid Change 32
Amazon Web Services (AWS) Fully Automatic, Self-Healing, Networked Mega
Systems Inside A Building. 33
Data Center Design Innovation 34
Shift To An All-Digital Business Environment 35
System Operates As A Whole, At Fiber Optic Speeds, To Create A Fabric 35

Mega Data Center Market Description and Market Dynamics 36
Advantages of Mega Data Center Cloud 2.0: Multi-Threading 37
Cloud 2.0 Mega Data Center Multi-Threading Automates Systems Integration 38
Advantages of Mega Data Center Cloud 2.0: Scale 39
Infrastructure Scale 41
Intense Tide Of Data Causing Bottlenecks 42
Application Integration Bare Metal vs. Container Controllers 43
Workload Schedulers, Cluster Managers, And Container Controllers Work Together 44
Google Kubernetes Container 45
Google Shift from Bare Metal To Mega Data Center Container Controllers 46
Mesosphere / Open Source Mesos Tool 46

Mega Data Center TCO and Pricing: Server vs. Mainframe vs. Cloud vs. Cloud 2.0 47
Labor Accounts For 75% Of The Cost Of An Enterprise Web Server Center 48
Cloud 2.0 Systems And The Mainframe Computing Systems Compared 49
Cloud 2.0 Mega Data Center Lower Operations Cost 50
Cloud 2.0 mega Data Center Is Changing the Hardware And Data Center Markets 51

Scale Needed to Make Mega Data Center Containers Work Automatically 52
Multipathing 53
Cloud 2.0 Mega Data Centers Simple Repetitive Systems 53
Simplifying The Process Of Handling Load Balanced Requests 54
Google Servers Are Linked Logically, Each With Their Own Switch 55

Internet Apps Trillion Dollar Markets 56
Clos Simplicity 57
Clos-Based Topologies Increase Network Capacity 59
Mega Data Centers Embrace Open Source: Scale Is Everything 60
Open Cloud Server 61

Mainframe Provides Security 62
IBM Mainframe Handles Transactions, Business Analytics, and Mobile Apps 63
IBM Excels in Mastering Large Size Of Data To Be Managed 64
Transaction Based Mainframe 65

Microsoft Market Presence 66
Observers See Enterprise Data Center Moving to Cloud 67
Public Cloud Adoption 68
Microsoft Positioned To Become A Hyperscaler, Open Sourcing Hardware 69
Google Shift from Bare Metal To Container Controllers 70
Rapid Cloud Adoption: Google Says No Bare Metal 71
IBM Uses Bare Metal Servers: Mainframe Not Dead 72
VMware Photon Controller: Open Source Container Infrastructure Platform 73
Why Mega-Datacenters? 74
Data Center Switching 75
Software-Defined Networks Represent the Future 76

Broadcom 40 Gigabit Ethernet Optical Transceiver 78
40G, 100GBPS Transceivers Evolving Place in Mega Data Center: 79
NeoPhotonics 400 Gbps CFP8 PAM4 80

Applications: Equinix and Oracle 81
Oracle Cloud Platform 82
Reason Companies Move to Cloud 2.0 Mega Data Center 83

System On A Chip (SoAc) 84
Optical Transceiver Vendors Have Noticed That Mega Data Centers Are at the
Center of Modern Processing 86

Fiber High Bandwidth Datacenters 87
400 Gbps Headed For The Data Center 87
100 Gbps Adoption 89
Optical Transceiver Vendors Have Noticed That Mega Data Centers Are at the
Center of Modern Processing 89

Digital Workloads Increasing 90
Optical Transceiver High Growth as Shift to Cloud Occurs 91
Google Disruptive Technology: Base Orchestration Enhancements 92
Digital Realty Trust Lakeside Technology in Chicago: 1.1 Million Square Foot Data Center 93
Cisco Cloud Index: Cloud Replaces Data Centers 94
NTT Has Dominant Market Position 95
Enterprise Networking Rapid Transition 96
Public Cloud Adoption 97
Cisco CRS-3 Core Routing Platform 98

Evolution of Data Center Strategy 99
Systems Integration 101
AWS, Amazon Cloud Services Facebook, Google, and Microsoft: AWS leads in
Mega Data Center Infrastructure 102
Conclusion 103
Cloud 2.0 Mega Data Center Evolution 103

APPENDIX A 104
Growth of Quantity of Data 104
Data Expanding And Tools Used To Share, Store And Analyze Evolving At Phenomenal Rates 104
Video Traffic 105
Cisco Analysis of Business IP Traffic 105
Increasing Video Definition: By 2020, More Than 40 Percent of Connected Flat-
Panel TV Sets Will Be 4K 113
M2M Applications 115
Applications, For Telemedicine And Smart Car Navigation Systems,
Require Greater Bandwidth And Lower Latency 117
Explosion of Data Inside Cloud 2.0 Mega Data Center with Multi Threading 122
Cloud 2.0 Mega Data Center Multi-Threading Automates Systems Integration 122
Fixed Broadband Speeds (in Mbps), 2015-2020 123
Internet Traffic Trends 127
Siemens Predicts IoT Growth 130
Appendix B: Things People Already Know About Cloud Computing 132
WINTERGREEN RESEARCH, 133
WinterGreen Research Methodology 134

List of Figures

Enterprise Data Center as a Bottleneck: Think Woods Hole
Figure 1. Existing Enterprise Data Center as a Bottleneck: Think Woods Hole 5
Figure 2. AWS Data Center Image 6
Figure 3. Achieving a Scalable Architecture from Simple Units 7
Figure 4. Facebook Sample Pod: Unit of Network 8
Figure 5. Facebook Data Center Fabric Network Topology 9
Figure 6. Cloud 2.0 Mega Data Center 11
Figure 7. Cloud 2.0 Mega Data Centers Support 1.5 Billion Facebook Users Worldwide. 12
Figure 8. Facebook DuPont Fabros Technology Ashburn, VA Data Center 24
Figure 9. SOA Foundation Business, Infrastructure, and Data Information Architecture 27
Figure 10. AWS Market Leader In Cloud Computing 32
Figure 11. 538,000SF: i/o Data Centers and Microsoft Phoenix One, Phoenix, Ariz. 34
Figure 12. Phoenix, Arizona i/o Data Center Design Innovations 34
Figure 13. Key Challenges Enterprise IT Datacenters: 36
Figure 14. Multi-threading Manages Pathways From One Node To Another Node 37
Figure 15. Cloud Types of System Implementation 38
Figure 16. Google Mega Data Center Scale 39
Figure 17. Key Advantage of Cloud 2.0 Mega IT Datacenters: 40
Figure 18. NTT RagingWire Ashburn Va2 Data Center 41
Figure 19. AWS Region Diagram 42
Figure 20. Google Shift from Bare Metal To Container Controllers Advantages 45
Figure 21. Cloud 2.0 Mega Data Center Advantages 51
Figure 22. Images for Google Container Cloud 3.0 Mega Data Centers 52
Figure 23. Facebook Fifth Data Center Fort Worth Complex. 53
Figure 24. Google Compute Engine Load Balanced Requests Architecture 56
Figure 25. Google Extends App Indexing 57
Figure 26. Google Clos Multistage Switching Network 58
Figure 27. The size of the basic switch element has an impact on the total
number of switching nodes requi Google Clos Multistage Switching Network 59
Figure 28. Mainframe Security 62
Figure 29. IBM Mainframe System z/OS 63
Figure 30. z13 Server Benefits 64
Figure 31. Aspects of Cloud 65
Figure 32. Observers See Enterprise Data Center Moving to Cloud 67
Figure 33. Broadcom 40 Gigabit Ethernet Optical Transceiver 78
Figure 34. 40G, 100GBPS Transceiver Target Markets 79
Figure 35. NeoPhotonics 400G CFP8 PAM4 80
Figure 36. Neophotonics 400 Gbps CFP8 PAM4 Features 80
Figure 37. Equinix LD6 data center in Slough, England 81
Figure 38. Cloud 2.0 Mega Data Centers Are Demanding Significant
Amounts Of Power And Network Management 85
Figure 39. Flow of Digital Data Creating Bottlenecks In Enterprise Data Center 90
Figure 40. Google Base Orchestration Enhancement Functions 92
Figure 41. Digital Realty Trust Lakeside Technology Center Industrial-
Strength Power And Fiber Infrastructure 93
Figure 42. NTT RagingWire Data Centers Image 95
Figure 43. Google Andromeda Cloud High-Level Architecture 99
Figure 44. Amazon AWS Global Cloud Infrastructure 102
Figure 45. Cisco VNI Forecast Overview 106
Figure 46. The Cisco VNI Forecast—Historical Internet Context 107
Figure 47. Global Devices and Connections Growth 108
Figure 48. Average Number of Devices and Connections per Capita 110
Figure 49. Global IP Traffic by Devices 110
Figure 50. Global Internet Traffic by Device Type 111
Figure 51. Global 4K Video Traffic 113
Figure 52. Global IPv6-Capable Devices and Connections Forecast 2015–2020 114
Figure 53. Projected Global Fixed and Mobile IPv6 Traffic Forecast 2015–2020 115
Figure 54. Global M2M Connection Growth 116
Figure 55. Global M2M Connection Growth by Industries 117
Figure 56. Global M2M Traffic Growth: Exabytes per Month 118
Figure 57. Global Residential Services Adoption and Growth 119
Figure 58. Global IP Traffic by Application Category 120
Figure 59. Mobile Video Growing Fastest; Online Video and Digital TV Grow Similarly 121
Figure 60. Global Cord Cutting Generates Double the Traffic 121
Figure 61. Fixed Broadband Speeds (in Mbps), 2015–2020 123
Figure 62. Future of Wi-Fi as Wired Complement 124
Figure 63. Global IP Traffic, Wired and Wireless 125
Figure 64. Global Internet Traffic, Wired and Wireless 126
Figure 65. Cisco VNI Forecasts 194 EB per Month of IP Traffic by 2020 129
Figure 66. Cisco Forecast of Global Devices and Connections Growth 130
Figure 67. Siemens Perspective of Billions of Things, Trillions of Dollars 131
Figure 68. Benefits of Cloud Computing 132