The World Is Moving Beyond Big Data, According to Ocient Survey of 500 Data and Technology Leaders

Hyperscale data analytics will scale with exponential data growth, generate critical business insights, drive efficient decision-making and predict outcomes

Ocient, the leading hyperscale data analytics solutions company serving organizations that derive value from analyzing trillions of data records in interactive time, released a report, “Beyond Big Data: The Rise of Hyperscale.” The report is based on a survey of 500 data and technology leaders, across a variety of industries, who are managing active data workloads of 150 terabytes or more. The purpose of the survey and report is to uncover key trends around how organizations are managing the shift from big-data volumes toward ingesting, storing and analyzing hyperscale data sets, which include trillions of data records, and the expected technical requirements and business results from that shift.

The survey was conducted in May 2022 by Propeller Insights. Respondents include partners, owners, presidents, C-level executives, vice presidents and directors in many industries including technology, manufacturing, financial services, retail, and government. Their organizations’ annual revenue ranges from $50 million to more than $5 billion. Approximately 50% of respondents represent companies with annual revenue greater than $500 million.

Key findings of the survey include:

• Data analysis is tied to financial success. More than 85% of C-level respondents indicated there is a strong relationship between implementing faster data analytics and growing the company’s bottom line.
• Data workloads are getting bigger, faster and more complex. Ninety-seven percent of respondents indicated the volume of data managed by their organization will grow fast to very fast over the next one to five years.
• Legacy systems aren’t built to handle hyperscale data analysis. More than 59% of respondents plan to switch data warehousing solutions, and 46% of respondents indicated a legacy system is motivating them to switch.
• Security and compliance are among top concerns. Sixty-three percent of respondents said maintaining security and compliance is a challenge in scaling data volumes and analytics to hyperscale workloads.
• Staffing is a challenge. Forty-nine percent of respondents wrote there is a lack of talent to analyze their data.

Extraordinary data growth

Data is growing at an extraordinary rate. According to John Rydning, research vice president of the IDC Global DataSphere, a measure of how much new data is created, captured, replicated, and consumed each year, “The Global DataSphere is expected to more than double in size from 2022 to 2026. The Enterprise DataSphere will grow more than twice as fast as the Consumer DataSphere over the next five years, putting even more pressure on enterprise organizations to manage and protect the world’s data while creating opportunities to activate data for business and societal benefits.”

IDC Global DataSphere research also documented that “in 2020, 64.2 zettabytes of data was created or replicated” and forecasted that “global data creation and replication will experience a compound annual growth rate (CAGR) of 23% over the 2020-2025 forecast period.” At that rate, more than 180 zettabytes — that’s 180 billion terabytes — will be created in 2025.

The survey respondents reflect forecasts of such exponential data growth. When asked how fast the volume of data managed by their organization will grow over the next one to five years, 97% of respondents answered “fast” to “very fast,” with 72% of C-level executives expecting the volume to grow “very fast” over the next five years.

Barriers to supporting data growth and hyperscale analytics

To support such tremendous data growth, 98% of respondents agreed it’s somewhat or very important to increase the amount of data analyzed by their organizations in the next one to three years. However, respondents are experiencing barriers to harnessing the full capacity of their data and cited these top three limiting factors:

• The volume of data is growing too fast (62% total, 65% C-level)
• There is a lack of talent to analyze the data (49% total, 47% C-level)
• Current solutions are not flexible enough (49% total, 34.8% C-level)

When asked about their biggest data analysis pain points today, security and risk ranked first among C-level respondents (68%), with metadata and governance (41%) and slow data ingestion (31%) being two other top concerns. When scaling data management and analysis within their organization, 63% said maintaining security and compliance as data volume and needs grow was a challenge they are currently facing.

Survey respondents also indicated legacy systems are another source of pain and a barrier to supporting data growth and hyperscale analytics. When asked if they plan to switch data warehousing solutions, more than 59% of respondents answered “yes,” with 46% of respondents citing a legacy system motivating them to switch. When ranking their most important considerations in choosing a new data warehouse technology, “modernizing our IT infrastructure” was ranked number one.

Faster data analytics improve decisions, revenue and success

The survey respondents believe hyperscale data analytics is crucial to their success. Sixty-four percent of respondents indicate hyperscale data analytics provides important insights used to make better business decisions, and 62% said it is essential for planning and strategy.

The survey respondents also indicated there is a strong relationship between implementing faster data analytics and growing the company’s bottom line. When asked about this relationship, an overwhelming 78% of respondents agreed there is a definite relationship. For the C-level audience, more than 85% cited the relationship.

“Data analysis is no longer a ‘nice-to-have’ for organizations. Hyperscale data intelligence has become a mission-critical component for modern enterprises and government agencies looking to drive more impact and grow their bottom line. With the rapid pace of growth, it’s imperative for enterprises and government agencies to enhance their ability to ingest, store, and analyze fast-growing data sets in a way that is secure and cost effective,” said Chris Gladwin, co-founder and CEO, Ocient. “The ability to migrate from legacy systems and buy or build new data analysis capabilities for rapidly growing workloads will enable enterprises and government organizations to drive new levels of agility and growth that were previously only imaginable.”

Marketing Technology News: Capitalizing on the Chaos in the Advertising and Marketing Hiring Market