Datadog Releases Data Streams Monitoring to Assess Streaming Data Pipeline Performance

Data Streams Monitoring helps organizations measure and meet strict SLAs and avoid critical downtime by observing queue health, consumer lag and end-to-end latency in event-driven applications like those using Kafka and RabbitMQ

Datadog, Inc., the monitoring and security platform for cloud applications, today announced the general availability of Data Streams Monitoring, which makes it simple for organizations to track and manage the performance of applications that rely on messaging systems like RabbitMQ. Data Streams Monitoring automatically visualizes all interdependencies and key health metrics across all streaming data pipelines to help organizations prevent and troubleshoot latency and downtime.

Traditional solutions often provide only a limited view of streaming data pipelines and struggle to consistently identify the root causes of issues, such as drops in throughput or failing services. They may also lack context from upstream and downstream dependencies. This means internal teams must devote more time to manual tasks and troubleshooting in order to gather relevant information, analyze it contextually and determine the necessary collaborative efforts with other teams.

Streaming pipelines lie at the heart of numerous mission-critical applications, including banking and trading technologies, automotive systems, payment platforms and video infrastructures. Teams responsible for managing these streaming pipelines require precise measurement of end-to-end request latencies as they are frequently governed by service-level agreements (SLAs) and regulatory restrictions.

Marketing Technology News: MarTech Interview with Baba Diallo, Director of Creator Relations at Calaxy

As part of Datadog’s APM platform, Datadog Data Streams Monitoring automatically visualizes interdependencies and key health metrics across all streaming data pipelines. With Data Streams Monitoring, engineers can measure latency between any two points across an entire streaming data pipeline, locate faulty queues or services and mitigate floods of backed-up messages so they can avoid critical downtime.

“Building streaming data pipelines to power our digital trading platform requires visibility into throughput and latency across the entire system, including all queues and services,” said Darren Furr, Solutions Architect at MarketAxess. “With Data Streams Monitoring, our team is able to proactively discover performance bottlenecks and optimize stream processing to ensure maximum throughput and low latency delivery of data to our customers.”

“Kafka has become a key component in powering our data processing infrastructure and gaming data streams, which are built using highly sophisticated architectures,” said SeungYong Oh, VP of Engineering at Devsisters. “Datadog Data Streams Monitoring allows us to easily map the topology of our Kafka pipelines across our globally spread clusters, monitor end-to-end latency and identify sources of incidents across our entire pipeline in one place. Having Data Streams Monitoring integrate seamlessly with APM helps us ensure that millions of gamers around the world always get the best possible experience.”

“There is a gap in visibility between SREs and application developers today, which limits their understanding of how queues are impacting mission-critical services and end-to-end latency of high-value user experiences like the resolution of banking transactions, the generation of shopping recommendations or payment processing,” said Omri Sass, Group Product Manager at Datadog. “While existing solutions can only observe individual queues and services, Data Streams Monitoring provides visibility into an organization’s entire ecosystem to make troubleshooting quicker and easier.”.

Marketing Technology News: PowerDMARC Email Authentication Platform Announces Integration with Gradient MSP