Spec Cloud IaaS 2018 Benchmark Uses I/O- and CPU-Intensive Workloads to Test Provisioning and Runtime Aspects of Cloud Implementations

The Standard Performance Evaluation Corp. (SPEC) has released SPEC Cloud IaaS 2018, new benchmark software that measures the performance of infrastructure-as-a-service (IaaS) cloud implementations.

The SPEC Cloud IaaS 2018 benchmark measures replicated application instances, overall performance, relative scalability, and mean instance provisioning time of IaaS cloud implementations, both public and private. It replaces the 2016 version, which will be retired in three months.

Cloud providers, users of cloud services, vendors of cloud-related hardware and software, and researchers can use the benchmark to evaluate how different implementations affect cloud performance.

Also Read: New Study Reveals That Half of Auto Dealers Overspend on Marketing, With No Measurable Return On Investment

SPEC members participating in the cloud subcommittee include Dell, Digital Ocean, IBM, Intel, Oracle, Red Hat and VMware. Long-time SPEC benchmark developer Yun-seng Chao is a supporting contributor.

Flexible, representative workloads

Two multi-instance workloads are included within the SPEC Cloud IaaS framework:

• The Yahoo! Cloud Serving Benchmark (YCSB) that uses the Cassandra NoSQL database to store and retrieve data in a manner representative of social media applications.
• Big data analytics based on a K-means clustering workload using Hadoop MapReduce.

The workloads are designed to test compute, storage and network performance of an IaaS cloud subject to strict quality of service (QoS) metrics. Each workload runs as a distributed application comprising six or seven cloud instances that make up an application instance. Adding new application instances increases the stress on the cloud infrastructure’s instance-provisioning capabilities and on its available resources, including CPU, memory, disk and network.

Also Read: Shoclef Presents Live Shopping in the Global Marketplace

The benchmark supports multi-tenancy and provides the flexibility to use a physical machine, a virtual machine, or a container as an instance type.

“SPEC Cloud IaaS 2018 incorporates improved methodologies that help ensure fairness, reliability and real-world rigor,” says Paula Smith, chair of the SPEC Cloud subcommittee. “It provides metrics that go beyond relative performance and capacity measurement to show how typical cloud application workloads behave as underlying resources approach full capacity.”

What’s new for 2018

SPEC Cloud IaaS 2018 builds on the 2016 release with a variety of updates, including:

• Easier installation of CBTOOL and workload images.
• Improved support for public clouds, OpenStack, and containers and container managers.
• Revised internal methodologies for scoring, reporting and metric calculations.
• New versions of Hadoop for K-means and Cassandra for YCSB.
• An increased I/O workload that tests the limits of current storage technologies.

Due to the methodology changes, results for SPEC Cloud IaaS 2018 are not comparable to those from the 2016 version.

Recommended Read: Searchbloom Named Top Digital Marketing Service Provider in the World