mParticle Launches Open-Source Data Validation Tools

New open-source toolset includes new APIs and Smartype, a new product to help engineers ensure proper event collection at run time

mParticle, the Customer Data Platform (CDP) of choice for multi-channel consumer brands, today announced the release of a new open-source developer toolset to give engineering teams instant data quality protection and feedback in their integrated development environments (IDE).

This toolset also features Smartype, which translates any JSON based data model into strongly-typed code. Additionally, this toolset pairs mParticle’s Data Planning API, the command line interface (CLI), and static code analysis (linting), to help developers validate their data model throughout their software development lifecycle.

New Data Planning API, and Smartype

Earlier this year, mParticle released the next generation of its Data Master product, allowing engineering, product, and marketing teams to collaboratively design and enforce a consistent data model. But once a development team has created a data plan, developers still need an easy and foolproof way to implement the plan without having to leave their editor to cross-reference a web UI—a process that can be slow, error-prone, and doesn’t provide change management.

Marketing Technology News: i-Sight Launches v5.5, With Improvements to Efficiency, Analysis and Communication

“With Data Master 2.0, we allowed mParticle customers to plan and specify their data model, and now with Smartype and the Linting tools, we’re helping engineers implement that data model,” said Sam Dozor, Senior Director of Engineering at mParticle. “Both Smartype and Linting enforce a Data Model, based on JSON schema, at compile time—engineers will actually receive compilation failures if an event is collected that doesn’t match your data plan. Smartype itself is designed to be an agnostic data quality layer—any engineering team in the world can use it to define and represent their data model in code.”

The newly available Data Planning API allows engineering teams to use Data Master programmatically through an HTTP API. With the Data Planning API, developers can easily store data plans in their source code, and use their own SDLC and approval processes to define the data model to best suit their needs. The Data Planning API can create, read, update, and delete Data Plans, as well as integrate mParticle’s suite of Data Planning tools to perform compile-time and runtime data quality verification.

Together, the Data Planning API and Smartype allow developers to translate a Data Plan (or more generally, Data Model) into usable code to improve implementations and minimize data quality issues by improving instrumentation quality and consistency.

Marketing Technology News: SpotX Teams Up With Pluto TV To Drive Its Video Monetization For Latin America Market

Smartype addresses common data quality issues through these features:

• Data Plans as code: All constants called for in the Data Plan (event names, attribute names, enum values) available as a machine-readable JSON Schema
• Data Planning API: Programmatically perform all CRUD operations on Data Plans to decrease the time-to-data-quality and time to implement
• Compile-time feedback: Feedback developers can use to conform to a Data Plan – thereby limiting the likelihood of an implementation issue.
• Generate typesafe libraries: Programmatically generate a strongly typed client library across a variety of languages from a centrally defined Data Model.
• Multiplatform code sharing: Built on Kotlin so that you can share your common Data Plan implementations across platforms (Android, iOS, Web, and server-side environments).
• Managed within codebase: Any changes to Data Plans result in a regenerated typesafe library that is always in-sync with the latest Data Plan.

Smartype makes it easier for development teams to define and collaborate against a data model programmatically, implement it correctly at compile-time, report validations in time to fix them, and transform data points in the plan on an ongoing basis.

CLI and Linting

In addition to the Data Planning API and Smartype, the new developer toolset also includes mParticle CLI and Linting. For developers, the new CLI offers an easy way to interface with the mParticle platform through the command-line instead of the user interface.

Using the CLI allows developers to create, maintain, update, and even delete data plans with version control provided by a git repository. The CLI also enables developers to use linting tools to statically lint code against a data plan as they develop to ensure adherence to their data plan and that only high-quality data is logged to their workspace. The lint tool can also be used within CI to ensure validity throughout the entire development lifecycle.

Marketing Technology News: DOSL App Provides Live, Uninterrupted, Unbiased Video Content