As the rapid growth of power density per rack in data centers becomes a trend, enterprises must find more effective ways to face the power, cooling and space challenges that come with it. Fortunately, experts at Delta are integrating the modularization concept into products and solutions designed for data centers to bring more flexibility and prepare for growing workloads.
In recent years, data centers have faced many challenges due to rapid changes in IT technologies and trends. High density data centers have become an issue that enterprises must face.
As people have stricter demands for internet services, enterprises must think of ways to satisfy user expectations for 24x7 availability. The number of servers keeps increasing while more computing capabilities and resources are incorporated in smaller and smaller cases. Small Form Factor (SFF), Blade Servers to Hyperscale servers to Composable Infrastructure used to achieve more flexible frames, are all being developed in response to this demand.
Higher density servers mean higher power consumption is needed for each server. As the world focuses more and more on energy consumption, enterprises have started to treat energy efficiency issues and environmental responsibilities more seriously. Virtualization technology offers solutions for enterprises that allow each server to handle a greater workload so they are utilized to their fullest. At the current stage of the global cloud trend, many enterprises already see “Cloud First” as one of their top strategies. More IT budgets are investing in “cloud” related IT infrastructure and software and some enterprise organizations or IT suppliers even use “Cloud Only” as their main strategy. When servers are densely placed in cloud data centers, it will definitely bring greater maintenance challenges.
Since the development of mobilization and social media platforms, the data required for massive computing power to analyze and extract has increased rapidly. With the continued development of the Internet of Things (IoT), data collected through sensors will be guided to backend data centers to perform big data analysis. These changes are leading data centers to develop towards high density. With the rapid growth in the density of data center equipment, data centers built along traditional concepts are no longer enough.
Potential issues from high power density
According to a research report by Colocation America (2014), the power density of a single rack cabinet in data centers was approximately 6kW in 2008, which reached 12kW in 2016. It is estimated that by 2020, the power density of a single rack cabinet in data centers will achieve 16.5kW. For example, when Intel retrofitted two foundries into a green data center with high power density, the power density per rack reached as high as 43kW.
Source: Colocation America, 2014
With the rapid growth of power density per rack in data centers as a leading trend, enterprises must find more effective ways to face the challenges that come with it. For example, high density achieves better space utilization and the response time of system failure is reduced significantly. However, once there is a power failure, the large amounts of heat generated by the equipment cannot be extracted and will result in a server shutdown.
The ever-increasing power density has also far exceeded the processing capabilities of most old facilities. In previous years, each rack in a data center was designed for 6kW power density. However, when faced with high density racks of 15kW or above, facilities clearly do not meet requirements. When enterprises use technology that requires massive resources such as cloud computing or big data analysis, they also face expansion problems for the difference between available and needed capacities. In the past data center cooling design assumed that the IT work load was even and well distributed, but the actual operating environment was not so, especially in certain high density rack cabinets. Enterprises are realizing that their cooling capacities are seriously insufficient.
The backup power mechanism originally designed for data centers may also disappear due to this deficiency. The original UPS and cooling system designed using N+1 configuration will be forced to become fully operational due to insufficient capacities, and lose their backup functions. In addition, after deploying virtualization solutions, IT staffs can also move virtual machines dynamically. Data center loads will also change due to this and hot spots will become elusive. Power requirements will also change and result in unnecessary shutdowns.
The emerging modularization provides higher flexibility
High density data centers still have many potential problems as administrators of data centers face greater pressure. In addition to maintaining an increasingly dense computing environment and improving its availability, they must also reduce cost and increase efficiency. Fortunately, industry professionals are integrating the modularization concept into equipment and products designed for data centers to bring greater flexibility and prepare for future workloads.
At the current stage, the main modularization concepts are applied to space and facility designs. Space modularization refers to the use of modules from IT infrastructure, rack cabinets and facilities provided for IT equipment to operate. Each depends on and relates to the other. In practice, the data center spaces at enterprises are used to assess the capacity needs of existing services and future expansion considerations, and are further divided into smaller spaces and viewed as modules. The modularization of facilities refers to the use of modular designs for infrastructure, including power systems such as UPSs, power distribution cabinets, in-row cooling, server racks and cold/hot-aisle containment.
“The development of IT technology is changing every day, and infrastructure is developing towards ‘microservice’ architectures. Simply put, ‘microservice’ architecture refers to the use of modularization to form complicated large-scale applications. Modular solutions were developed for data center infrastructure years ago to provide flexibility for enterprises responding to expansion needs and to overcome power and cooling insufficiency and space challenges,” said Dr. Charles Tsai, the general manager of Delta’s mission critical infrastructure solutions business unit.
The modularization of the UPS system
According to a research conducted by a UPS vendor, approximately 50% of respondents believe that the main cause of power outage at data centers is UPS equipment failure. This shows how important UPS systems are for maintaining data centers. As data centers acquire higher densities, they must replace old UPS systems with efficiency as a major consideration. UPS systems designed ten years ago usually have an efficiency of 85% when operating at 40% load to serve dual power input servers. The energy efficiency of current UPS systems is even greater. Take Delta’s UPS solution for example, under a light load of 20%, the AC-AC efficiency for the
DPH 500kVA series UPS can be around 95% and the peak efficiency can be up to 96.5% for obvious energy cost savings.
Another consideration is effective space utilization in the data centers. Generally speaking, power rooms plan to install power facilities even though the space of the power room may be very limited. When data centers develop towards higher density in each rack cabinet, backup power must also increase accordingly. Enterprises can replace their legacy UPSs by new generation units with a higher power capacity. For example, Delta recently released the Modulon DPH 500 kVA modular UPS for large data centers that only takes up a space of a 19” rack cabinet and provides the world’s highest power density. The parallel expansion can also be configured up to 8 units, providing a maximum power capacity of 4MVA.
Since it has self-diagnosis and aging detection mechanisms, it can detect the health of batteries, fan, IGBT module, DC capacitors and AC capacitors for preventive maintenance to reduce the risk of malfunctions and power loss, and protect the customer’s equipment investment.
For enterprises, another advantage of modular UPSs is the “plug and play” design of power modules. Either vertical (within a single system cabinet) or horizontal (in parallel) expansion can be achieved per enterprise needs. Enterprises can flexibly purchase UPSs according to their initial capacity needs and count on future operational expansion to further lower CAPEX.
RowCool systems near hot spots reduce power losses
As server and IT equipment densities become higher, the requirements for facilities are also becoming more rigorous. In addition to higher power supply density, cooling has become an issue in data centers. The cooling design of data centers assumes that the IT work load is even and well distributed, but in real enterprise environments, uneven heat distribution may be generated due to dynamic moving of virtual machines or improper deployment of IT equipment.
Insufficient cooling will become a common challenge that high-density data centers will face. The advantage of RowCool is that it is close to hot spots, and is different from RoomCool systems where losses are generated in the air delivery path under the raised floor. RowCool systems can provide sufficient cooling capacity nearby to eliminate hot spots. They are equipped with high power-saving DC or EC fans with variable fan speed control for more energy savings. A 10% fan speed reduction can save a maximum 27% of energy consumption. In addition, the N+1 backup design is used for the overall architecture and group control functions that are provided to perform linked control for the RowCool units in the area to solve the hot spot problems caused by sudden load increases.
RowCool systems also have modular designs. For example,
RowCool 29/43kW not only has built-in dual power supplies and can significantly increase the reliability and protection for system power; but it also supports hot-swappable power supplies and fans, and can reduce the maintenance time needed. Its variable fan speed control design can adjust fan speed according to the actual work load. The modularization concept is also applied to RowCool units, which can be added to required spots as needed.
Case Study: Medium and High Density Zones co-exist in a data center
According to the data center density categories of AFCOM, the data center managers' association, a low density environment is when each cabinet is under 4kW, mid density is 5kW-8kW, high density is 9kW-15kW and ultra-high density is 16kW and above. Different data centers may face different rack power density challenges. In some data centers, there might even be a mix of rack cabinets with different densities, but use the same design concepts for power and cooling systems. A leading IC design company in Taiwan plans to build a new data center at their headquarters because they need to integrate their existing IT equipment and related network and operating environments. According to onsite inspection, there are up to 80 mid-density and ultra-high density rack cabinets in this enterprise’s data center, with the ultra-high density cabinets up to 25kW.
Delta recommends that when enterprises need both low-density and high-density/ ultra-high density rack designs for different application needs, it should divide the data center space into a high-density cabinets area and an ultra-high density cabinets area to optimize its design. Facility planning should not only include overall power and cooling needs, but also space usage for the deployment of IT systems. There are different options for the use of cooling solutions. For example, rack cabinets under 4kW may only need RoomCool for effective cooling, while it is better to equip RowCool units for higher density racks from 9kW to 15kW for optimal heat removal.
In this case, Delta used the modular design concept to plan the ultra-high density area for its client, and placed all of the ultra-high density rack cabinets in one zone. For its high density applications, the client chose Delta’s RowCool 95kW with the industry largest cooling capability within a 600mm wide cabinet. Delta also recommended the use of hot-aisle containment technology to prevent the mixing of cold and hot air and ensure optimal cooling efficiency.
Summary
Technology is developing every day. Emerging IoT, artificial intelligence, AR/VR has integrated cloud, mobile, social media and big data technologies. For enterprises, the challenges for data centers have become more and more difficult. Development towards high-density will definitely continue and infrastructure will become the most important backup when enterprises promote their innovative application services.
For data center administrators the development of high-density data centers will definitely bring more maintenance and management issues. The availability of data centers will also be a challenge, while costs must decrease and efficiency must increase. The use of modular designs and related facilities, including space modularization, cold or hot aisle containment technology, RowCool systems and more efficient UPSs, can ensure higher reliability architectures and more deployment flexibility for data centers.