New Experiment Groups allows teams to learn, iterate, and ship faster without compromising results or the customer experience

To better support data leaders as they improve their organization’s experimentation practices, digital analytics leader Amplitude, Inc. (Nasdaq: AMPL) today announced an expanded suite of product features for its world-class experimentation offering, Amplitude Experiment.

Product and data teams leverage experimentation to continually test hypotheses for ways to acquire, engage and retain their users. While data leaders want to drive innovation within their organization, many find their experimentation programs blocked by user identity issues, inconsistent metrics, long testing periods, and duplicative data sets. Built upon Amplitude’s Digital Analytics Platform and natively integrated with Amplitude Analytics, Amplitude Experiment helps data leaders unite analytics, experimentation, and new program management capabilities to scale their experimentation programs faster and further.

“Leading organizations have identified how to scale experimentation effectively, consistently measuring the impact of every feature they ship as they aim to build better products,” explained Justin Bauer, chief product officer at Amplitude.

Still, the biggest barrier to scaling product experimentation is having data and analytics to analyze the results of experiments. Disparate solutions for analytics, A/B testing, and feature management have worsened this problem, leading to inconsistent data sources and the complexity of duplicative datasets.

“With the enhancements we’re making today—and our native integration with Amplitude Analytics—we’re making it easier for teams to scale their A/B testing programs without sacrificing quality or overburdening data teams,” added Bauer. “With every leader focused on ROI, Amplitude Experiment is an obvious choice. It drives business value quickly while eliminating duplicative vendor costs and reducing data spend.”

Marketing Technology News: MarTech Interview with Amol Waishampayan, Chief Product Officer at FullThrottle

According to a report from Nucleus Research, Amplitude customers reported a return on investment within the first six months following deployment leveraging Amplitude Analytics and Amplitude Experiment together¹. By consolidating experimentation and analytics, customers saw a 20 percent reduction in costs on average. They also saw a direct impact on the bottom line metrics, including a more than 3x average increase in user signups and a 13 percent average increase in revenue.

New experimentation capabilities include:

• Experiment Groups: Scale experimentation without compromising results or customer experience. Separate subsets of users from every test with Mutual Exclusion Groups to ensure users are not subjected to colliding experiments, which obscures results. With Holdout Groups, designate a set of users as a control group to understand the cumulative effects across multiple experiments.
• Formula Metrics: Data analysts can build formula-based metrics that ensure consistent insights and faster, coordinated action without needing engineering teams to update data pipelines.
• Controlled-experiment Using Pre-existing Data (CUPED): Automatically reduce experiment variance with pre-experiment data to reach statistical significance faster, with less traffic. With CUPED, teams can better understand how product changes impact different customer segments.
• Data Quality Checklist: Trust results by easily monitoring for experiment design, implementation, instrumentation, and statistical integrity issues proactively.

“We recently adopted Amplitude Experiment to simplify A/B testing and run concurrent experiments without exposing users to more than one testing scenario at a time,” said Alexander Magnusson, manager of data analytics at Brainly. “Experiment lets us integrate testing and analytics within a single ecosystem, which makes it a no-brainer.”

Marketing Technology News: The Recession Reset