Design Considerations Relating to Non-Thermal Aspects of Oil Immersion Cooling

2016 ◽  
Author(s):  
Jimil M. Shah ◽  
Syed Haider I. Rizvi ◽  
Indu Sravani Kota ◽  
Sahithi Reddy Nagilla ◽  
Dhaval Thakkar ◽  
...  
Keyword(s):  
Data Center ◽  
Data Centers ◽  
Energy Savings ◽  
Cost Savings ◽  
Mineral Oil ◽  
Critical Element ◽  
Air Cooling ◽  
Cooling Fluid ◽  
Trade Offs ◽  
Oil Immersion

Full submersion of servers in dielectric oils offers an opportunity for significant cooling energy savings and increased power densities for data centers. The enhanced thermal properties of oil can lead to considerable savings in both the upfront and operating costs over traditional air cooling methods. Despite recent findings showing the improved cooling efficiency and cost savings of oil as a cooling fluid, this technique is still not widely adopted. Many uncertainties and concerns persist regarding the non-thermal aspects of an oil immersion cooled data center. This paper presents useful information regarding a variety of factors related to the operation of an oil cooled data center. Pertinent material property considerations such as the chemistry, flammability, material compatibility, human health effects, and sustainability of mineral oil are discussed. A general introduction as to the chemical composition and production of mineral oil is provided. A discussion of the trade-offs in thermal performance and cost of the mineral oil is presented. The dielectric nature of oils is critical to their success as a cooling fluid for electronic applications. Factors such as temperature, voltage, and age that affect this property are reviewed. Flammability of oils is a valid concern when immersing costly IT equipment and the pertinent concerns of this aspect are reviewed. The evaporation loss of oil is also mentioned as refueling and safety are important parameters in the establishment of any facility. Leeching of materials, especially plastics, is a reoccurring concern expressed regarding mineral oil immersed IT equipment. Mineral oils are by-products of petroleum refining processes and as such may bring forth sustainability concerns associated with their use and disposal. The long term stability and performance of key physical and material parameters of oils used in applications such as high voltage power are typically monitored. The similarity and implications of the longevity of oils, when used for data center applications, will be examined. Other issues related to the design, operation, and serviceability of submerged IT equipment and racks will also be addressed. Switching to an oil immersion cooled data center typically brings about several designs and operational changes compared to a typical air-cooled approach. A critical element of oil cooling often cited by opponents of the technology is the issue of serviceability of IT equipment. This paper will discuss some of the additional features a data center may need in place to help alleviate these concerns, as well as, best practices based on experience and observations by the authors. This paper also includes Cup Burner Experiment as per ISO 14520/NFPA 2001 standard to determine the minimum design concentration of fire extinguishing agent for the class B hazard of heavy mineral oil and the class C hazard of electronic equipment as a part of the safety concerns for oil cooled data centers. The visual observations of the servers after immersion in oil for 8 months are also explained for a better view of the system related issues. The discussion presented here is based primarily on literature gathered on the subject and quantifiable data gathered by the authors.

scholarly journals Thermal Performance and Efficiency of a Mineral Oil Immersed Server Over Varied Environmental Operating Conditions

2017 ◽  
Vol 139 (4) ◽  
Author(s):  
Richard Eiland ◽  
John Edward Fernandes ◽  
Marianna Vallejo ◽  
Ashwin Siddarth ◽  
Dereje Agonafer ◽  
...  
Keyword(s):  
Flow Rate ◽  
Thermal Performance ◽  
Data Center ◽  
Mineral Oil ◽  
Operating Conditions ◽  
Published Data ◽  
Inlet Temperature ◽  
Air Cooling ◽  
Bulk Fluid ◽  
Oil Immersion

Complete immersion of servers in dielectric mineral oil has recently become a promising technique for minimizing cooling energy consumption in data centers. However, a lack of sufficient published data and long-term documentation of oil immersion cooling performance make most data center operators hesitant to apply these approaches to their mission critical facilities. In this study, a single server was fully submerged horizontally in mineral oil. Experiments were conducted to observe the effects of varying the volumetric flow rate and oil inlet temperature on thermal performance and power consumption of the server. Specifically, temperature measurements of the central processing units (CPUs), motherboard (MB) components, and bulk fluid were recorded at steady-state conditions. These results provide an initial bounding envelope of environmental conditions suitable for an oil immersion data center. Comparing with results from baseline tests performed with traditional air cooling, the technology shows a 34.4% reduction in the thermal resistance of the system. Overall, the cooling loop was able to achieve partial power usage effectiveness (pPUECooling) values as low as 1.03. This server level study provides a preview of possible facility energy savings by utilizing high temperature, low flow rate oil for cooling. A discussion on additional opportunities for optimization of information technology (IT) hardware and implementation of oil cooling is also included.

Qualitative Study of Cumulative Corrosion Damage of IT Equipment in a Data Center Utilizing Air-Side Economizer

2016 ◽  
Author(s):  
Jimil M. Shah ◽  
Oluwaseun Awe ◽  
Pavan Agarwal ◽  
Iziren Akhigbe ◽  
Dereje Agonafer ◽  
...  
Keyword(s):  
Relative Humidity ◽  
Data Center ◽  
Data Centers ◽  
Energy Savings ◽  
Evaporative Cooling ◽  
Corrosion Damage ◽  
Air Cooling ◽  
Single Server ◽  
Severity Level ◽  
Particulate Contamination

Deployment of air-side economizers in data centers is rapidly gaining acceptance to reduce the cost of energy by reducing the hours of operation of CRAC units. Use of air-side economizers has the associated risk of introducing gaseous and particulate contamination into data centers, thus, degrading the reliability of Information Technology (IT) equipment. Sulfur-bearing gaseous contamination is of concern because it attacks the copper and silver metallization of the electronic components causing electrical opens and/or shorts. Particulate contamination with low deliquescence relative humidity is of concern because it becomes wet and therefore electrically conductive under normal data center relative humidity conditions. IT equipment manufacturers guarantee the reliability of their equipment operating in environment within ISA 71.04-2013 severity level G1 and within the ASHRAE recommended temperature-relative humidity envelope. The challenge is to determine the reliability degrading effect of contamination severity levels higher than G1 and the temperature and humidity allowable ranges A1–A3 well outside the recommended range. This paper is a first attempt at addressing this challenge by studying the cumulative corrosion damage to IT equipment operated in an experimental data center located in Dallas, known to have contaminated air with ISA 71.04-2013 severity level G2. The data center is cooled using an air-side economizer. This study serves several purposes including: the correlation of equipment reliability to levels of airborne corrosive contaminants and the study of the degree of reliability degradation when the equipment is operated, outside the recommended envelope, in the allowable temperature-relative humidity range in geographies with high levels of gaseous and particulate contamination. The operating and external conditions of a modular data center, located in a Dallas industrial area, using air-side economizer is described. The reliability degradation of servers exposed to outside air via an airside economizer was determined qualitatively examining the corrosion of components in the servers and comparing the results to the corrosion of components in a non-operating server stored in a protective environment. The corrosion-related reliability of the servers over almost the life of the product was related to continuous temperature and relative humidity for the duration of the experiment. This work provides guidance for data center administration for similar environment. From an industry perspective, it should be noted that in the four years of operation in the hot and humid Dallas climate using only evaporative cooling or fresh air cooling, we have not seen a single server failure in our research pod. That performance should highlight an opportunity for significant energy savings for data center operators in a much broader geographic area than currently envisioned with evaporative cooling.

Reliability Considerations for Oil Immersion-Cooled Data Centers

2019 ◽  
Vol 141 (2) ◽  
Author(s):  
Jimil M. Shah ◽  
Richard Eiland ◽  
Pavan Rajmane ◽  
Ashwin Siddarth ◽  
Dereje Agonafer ◽  
...  
Keyword(s):  
Printed Circuit Board ◽  
Cooling System ◽  
Chemical Properties ◽  
Cost Savings ◽  
Dielectric Strength ◽  
Mineral Oil ◽  
Circuit Board ◽  
Air Cooling ◽  
Immersion Cooling ◽  
Oil Immersion

The improved efficiency of mineral oil may offer simplicity in facility design compared to traditional air cooling and provide a means for cost savings. Despite its improved cooling efficiency and cost savings, a mineral oil immersion cooling technique is still not widely implemented and original equipment manufacturers are reluctant to jeopardize sales of existing air-based cooling system equipment. Only compelling physics regarding thermal performance of direct immersion cooling is not enough for data center operators. Many uncertainties and concerns persist regarding the effects of mineral oil immersion cooling on the reliability of information technology (IT) equipment both at the component and chassis level. This paper is a first attempt at addressing this challenge by reviewing the changes in physical and chemical properties of IT equipment materials like polyvinyl chloride (PVC), printed circuit board (PCB), and capacitors and characterizes the interconnect reliability of materials. The changes in properties of a mineral oil like kinematic viscosity and dielectric strength are also cited as important factors and discussed briefly. The changes in mechanical properties like elasticity, hardness, swelling, and creep are being shown in the paper for thermoplastic materials. The chemical reaction between material and mineral oil as a function of time and temperature is also conferred. The literature gathered on the subject and quantifiable data gathered by the authors provide the primary basis for this research document.

Experimental Description of Information Technology Equipment Reliability Exposed to a Data Center Using Airside Economizer Operating in Recommended and Allowable ASHRAE Envelopes in an ANSI/ISA Classified G2 Environment

2020 ◽  
Vol 142 (2) ◽  
Author(s):  
Oluwaseun Awe ◽  
Jimil M. Shah ◽  
Dereje Agonafer ◽  
Prabjit Singh ◽  
Naveen Kannan ◽  
...  
Keyword(s):  
Information Technology ◽  
Data Center ◽  
Data Centers ◽  
Operating Cost ◽  
Weather Data ◽  
Air Cooling ◽  
Single Server ◽  
Severity Level ◽  
Equipment Reliability ◽  
Free Air

Abstract Airside economizers lower the operating cost of data centers by reducing or eliminating mechanical cooling. It, however, increases the risk of reliability degradation of information technology (IT) equipment due to contaminants. IT Equipment manufacturers have tested equipment performance and guarantee the reliability of their equipment in environments within ISA 71.04-2013 severity level G1 and the ASHRAE recommended temperature-relative humidity (RH) envelope. IT Equipment manufacturers require data center operators to meet all the specified conditions consistently before fulfilling warranty on equipment failure. To determine the reliability of electronic hardware in higher severity conditions, field data obtained from real data centers are required. In this study, a corrosion classification coupon experiment as per ISA 71.04-2013 was performed to determine the severity level of a research data center (RDC) located in an industrial area of hot and humid Dallas. The temperature-RH excursions were analyzed based on time series and weather data bin analysis using trend data for the duration of operation. After some period, a failure was recorded on two power distribution units (PDUs) located in the hot aisle. The damaged hardware and other hardware were evaluated, and cumulative corrosion damage study was carried out. The hypothetical estimation of the end of life of components is provided to determine free air-cooling hours for the site. There was no failure of even a single server operated with fresh air-cooling shows that using evaporative/free air cooling is not detrimental to IT equipment reliability. This study, however, must be repeated in other geographical locations to determine if the contamination effect is location dependent.

Understanding the True Total Cost of Ownership of Water Cooling for Data Centers

2015 ◽  
Author(s):  
Dustin W. Demetriou ◽  
Vinod Kamath ◽  
Howard Mahaney
Keyword(s):  
Data Center ◽  
Data Centers ◽  
Heat Dissipation ◽  
Cost Effective ◽  
Electric Utility ◽  
Air Cooling ◽  
Water Cooling ◽  
Total Cost Of Ownership ◽  
Total Cost ◽  
Cost Of Ownership

The generation-to-generation IT performance and density demands continue to drive innovation in data center cooling technologies. For many applications, the ability to efficiently deliver cooling via traditional chilled air cooling approaches has become inadequate. Water cooling has been used in data centers for more than 50 years to improve heat dissipation, boost performance and increase efficiency. While water cooling can undoubtedly have a higher initial capital cost, water cooling can be very cost effective when looking at the true lifecycle cost of a water cooled data center. This study aims at addressing how one should evaluate the true total cost of ownership for water cooled data centers by considering the combined capital and operational cost for both the IT systems and the data center facility. It compares several metrics, including return-on-investment for three cooling technologies: traditional air cooling, rack-level cooling using rear door heat exchangers and direct water cooling via cold plates. The results highlight several important variables, namely, IT power, data center location, site electric utility cost, and construction costs and how each of these influence the total cost of ownership of water cooling. The study further looks at implementing water cooling as part of a new data center construction project versus a retrofit or upgrade into an existing data center facility.

Parametric Study of Hybrid Cooling Solution for Thermal Management of Data Centers

2007 ◽  
Author(s):  
Veerendra Mulay ◽  
Saket Karajgikar ◽  
Dereje Agonafer ◽  
Roger Schmidt ◽  
Madhusudan Iyengar
Keyword(s):  
Thermal Management ◽  
Data Center ◽  
Parametric Study ◽  
Heat Load ◽  
Data Centers ◽  
High Heat ◽  
Inlet Temperature ◽  
Air Cooling ◽  
Air Conditioners ◽  
Hybrid Cooling

The power trend for Server systems continues to grow thereby making thermal management of Data centers a very challenging task. Although various configurations exist, the raised floor plenum with Computer Room Air Conditioners (CRACs) providing cold air is a popular operating strategy. The air cooling of data center however, may not address the situation where more energy is expended in cooling infrastructure than the thermal load of data center. Revised power trend projections by ASHRAE TC 9.9 predict heat load as high as 5000W per square feet of compute servers’ equipment footprint by year 2010. These trend charts also indicate that heat load per product footprint has doubled for storage servers during 2000–2004. For the same period, heat load per product footprint for compute servers has tripled. Amongst the systems that are currently available and being shipped, many racks exceed 20kW. Such high heat loads have raised concerns over limits of air cooling of data centers similar to air cooling of microprocessors. A hybrid cooling strategy that incorporates liquid cooling along with air cooling can be very efficient in these situations. A parametric study of such solution is presented in this paper. A representative data center with 40 racks is modeled using commercially available CFD code. The variation in rack inlet temperature due to tile openings, underfloor plenum depths is reported.

Energy-Efficient Air Cooling of Data Centers at 2000 W/ft2

2009 ◽  
Author(s):  
Magnus K. Herrlin ◽  
Michael K. Patterson
Keyword(s):  
Energy Efficiency ◽  
Data Center ◽  
Energy Efficient ◽  
Data Centers ◽  
High Heat ◽  
High Density ◽  
Cfd Modeling ◽  
Air Cooling ◽  
Capital Costs ◽  
Design Considerations

Increased Information and Communications Technology (ICT) capability and improved energy-efficiency of today’s server platforms have created opportunities for the data center operator. However, these platforms also test the ability of many data center cooling systems. New design considerations are necessary to effectively cool high-density data centers. Challenges exist in both capital costs and operational costs in the thermal management of ICT equipment. This paper details how air cooling can be used to address both challenges to provide a low Total Cost of Ownership (TCO) and a highly energy-efficient design at high heat densities. We consider trends in heat generation from servers and how the resulting densities can be effectively cooled. A number of key factors are reviewed and appropriate design considerations developed to air cool 2000 W/ft2 (21,500 W/m2). Although there are requirements for greater engineering, such data centers can be built with current technology, hardware, and best practices. The density limitations are shown primarily from an airflow management and cooling system controls perspective. Computational Fluid Dynamics (CFD) modeling is discussed as a key part of the analysis allowing high-density designs to be successfully implemented. Well-engineered airflow management systems and control systems designed to minimize airflow by preventing mixing of cold and hot airflows allow high heat densities. Energy efficiency is gained by treating the whole equipment room as part of the airflow management strategy, making use of the extended environmental ranges now recommended and implementing air-side air economizers.

Consideration for Running Data Center at High Temperatures and Using Free Air Cooling

2015 ◽  
Author(s):  
Yongzhan He ◽  
Guofeng Chen ◽  
Jiajun Zhang ◽  
Tianyu Zhou ◽  
Tao Liu ◽  
...  
Keyword(s):  
High Temperatures ◽  
Data Center ◽  
Data Centers ◽  
Rapid Development ◽  
Mobile Internet ◽  
Maintenance Cost ◽  
Air Cooling ◽  
Corrosion Resistant ◽  
Corrosion Risk ◽  
Infrastructure Maintenance

The advent of the big data era, the rapid development of mobile internet, and the rising demand of cloud computing services require increasingly more compute capability from their data center. This compute increase will most likely come from higher rack and room power densities or even construction of new Internet data centers. But an increase in a data center’s business-critical IT equipment (servers, hubs, routers, wiring patch panels, and other network appliances), not to mention the infrastructure needed to keep these devices alive and protected, encroaches on another IT goal: to reduce long-term energy usage. Large Internet Data Centers are looking at every possible way to reduce the cooling cost and improve efficiency. One of the emerging trends in the industry is to move to higher ambient data center operation and use air side economizers. However, these two trends can have significant implications for corrosion risk in data centers. The prevailing practice surrounding the data centers has often been “The colder, the better.” However, some leading server manufacturers and data center efficiency experts share the opinion that data centers can run far hotter than they do today without sacrificing uptime, and with a huge savings in both cooling related costs and CO2 emissions. Why do we need to increase the temperatures? To cool data center requires huge refrigeration system which is energy hog and also cost of cooling infrastructure, maintenance cost and operation cost are heavy cost burden. Ahuja et al [1] studied cooling path management in data center at typical operating temperature as well as higher ambient operating temperatures. High Temperatures and Corrosion Resistance technology will reduce the refrigeration output and how this innovation will open up new direction in data centers. Note that, HTA is not to say that the higher the better. Before embracing HTA two keys points need to be addressed and understood. Firstly, server stability along with optimal temperature from data center perspective. Secondly, corrosion resistant technology. With Fresh air cooling the server has to bear with the seasons and diurnal variation of temperatures and these can be over 35 degree C, therefore to some extent, we have to say, HTA design is the premise of corrosion resistant design. In this paper, we present methods to realize precise HTA operation along with corrosive resistant technology. This is achieved through an orchestrated collaboration between the IT and cooling infrastructures.

Mineral Oil As an Alternative Cooling Method

2021 ◽  
Author(s):  
Bahareh Estejab ◽  
John Tobin
Keyword(s):  
Data Centers ◽  
Convective Heat Transfer Coefficient ◽  
Mineral Oil ◽  
Working Fluid ◽  
Noise Levels ◽  
Conductive Liquid ◽  
Cooling Fluid ◽  
Cooling Method ◽  
C 50 ◽  
Lower Noise

Abstract In an effort to increase the performance ceiling and reduce the size of modern personal computers especially gaming computers and consequently data centers and supercomputers, mineral oil is proposed as the working fluid to cool the system. Mineral oil operates as an avenue for transmission rather than a liquid heatsink for heat storage, which differentiates this project from traditional mineral oil systems. Using non-conductive liquid as cooling fluid brings many advantages due to its higher convective heat transfer coefficient, which leads to more compact computers, higher speed of data transition, more efficient processors, lower noise levels, and less upkeep costs with respect to data centers. Modeling and simulations are done in NX to gauge temperature expectations; which is the most important limitation when designing and testing a computer before construction. The temperature range was found to be 34°C–50°C in the motherboard and 45°C–67°C in the graphic card. Based on the modeling results, a prototype of the proposed computer is built and tested. Ultimately, this project is trying to open up an avenue through which processor design can be reconsidered.

Sign in / Sign up

Export Citation Format

Share Document