High-performance computing (HPC) is one of the areas in IT that is expected to grow rapidly in the years to come. A report from analyst firm Grand View Research suggests that demand for the technology will grow at a compound annual growth rate of 6.5% per year through 2027.

The underlying reasons are quite simple; the ongoing evolution of various industry sectors towards digitalization, AI, IoT, and other heavy data-driven applications fuels the growing need for increased computational power and the ability to calculate complex algorithms. As the Grand View Research argues, “The ability of HPC systems to process large volumes of data at higher speeds is prompting government agencies, defence agencies, academic institutions, energy companies, and utilities to adopt HPC systems, which also bodes well for the growth of the HPC market.”

In other words, one of the most critical drivers behind HPC adoption and investment is the desire for access to a faster and more powerful data processing capability – high-performance data analytics (HPDA).

Delivering Leading HPDA Platforms

HPDA plays an important social, cultural, and economic role. For instance, through HPC, whole-genome sequencing, which initially took 13 years to complete, can now be completed in just one day. HPC has also been leveraged to accelerate drug screening and molecular analytics, allowing critical new drugs to move through the R&D phase more quickly. While the enhanced data computation capabilities of HPC have been facilitating more accurate weather forecasts, in the long run, HPDA will be the key to power astronomically complex numbers of computations that are required for safe autonomous driving.

To help visualize just how much data is involved, training an autonomous car requires the generation of around 60TB of data a day. A third-generation sequencer generates 6TB of data daily, while that data needs to be stored and accessed permanently. Now that organizations in mining and resources exploration are engaging in 3D exploration, instead of the traditional 2D imaging, the intermediate-term data requirements are soaring by as much as 20 times.  

To tackle such complex data challenges, enterprises need a robust HPDA solution, coupled with advanced data storage technology with higher speed, lower latency, and greater bandwidth.

Following hands-on performance and function testing for the Huawei OceanStor Pacific storage system, including multiple-protocol support, hybrid workload, high-density design, and total cost of ownership (TCO) analysis, Enterprise Strategy Group has recommended the storage system as an ideal next-gen storage solution for HPDA technology.

One of the key features behind the Huawei OceanStor Pacific solution – and one of the reasons that it is so effective in large volumes of data processing – is its parallel file system, which has been designed to optimize I/O metadata placement, keeping it close to the data on the nodes that own it. This allows the storage system to handle AI/ML, big data and analytics, large-scale virtualization, content repositories, seismic analysis, life sciences, or applications that are used by massive numbers of people.

According to Enterprise Strategy Group, the OceanStor Pacific storage system delivers consistently high performance for extremely large data sets and is clearly well suited to support demanding real-world, data-centric applications running in a performance-critical environment. The five-year TCO analysis by Enterprise Strategy Group also indicates that, by deploying the OceanStor Pacific storage system, organizations can lower storage TCO by up to 61% when compared with a traditional scale-out NAS system, improving availability and reducing operational efforts at the same time.

In typical scenarios, processing efficiency is also improved by more than 30% using the OceanStor Pacific storage system. Additionally, when compared with general-purpose servers, the high-density design adopted by the OceanStor Pacific storage system reduces the cabinet footprint by 66.6%. At the same time, the storage systems efficiency allows for a disk utilization rate of up to 91.6%.

Huawei HPC in action across energy and science and technology sectors

Lundin Energy Norway is not only one of Europe’s largest exploration and production companies, but also one of the leading innovators in the sector. As it notes on its website, “Today we are not just looking for oil and gas. We explore new ideas and new solutions. When we find what we are looking for, we get a little better. When we do not find it, we become a little wiser.”

Innovation in the energy sector requires the processing of massive amounts of data, and Huawei’s disturbed storage product has provided that for Lundin. Through Huawei, the company has access to a storage system that offers high bandwidth and input/output operations per second (IOPS), and that supports Lundin's seismic data processing, interpretation, and simulation analytics. Such changes have improved Lundin’s data processing efficiency by 28%. Moreover, intelligent tiering of hot and cold data saves 73% of physical storage space and slashes the overall TCO by 35%.

Across the science and technology sector, Huawei has helped Tsinghua University and the Chinese Academy of Sciences promote cutting-edge scientific research using HPDA solutions, as well as other leaders of science and technology, such as BGI, Orbita, and China Meteorological Administration.

The applications for HPDA are many and varied. As Steve Conway, the Senior Advisor of Hyperion Research who coined the term “HPDA,” said in a recent interview: “The important use cases, like automated driving, precision medicine, smart city, IoT, edge computing . . . if you’re doing research at the forefront of those fields, you’re using HPC.”

In other words, with just about every field of innovation, computing demands have reached the point that HDPA is the best solution going forward.

For more information on OceanStor Pacific Storage and HPDA, and to read the ESG report, click here.