How Matic scaled their fleet of autonomous robots and cut their most common crash by 90%

quote mark

“Sentry has made our lives much easier. Before, we were looking at logs and passively reacting to tickets. Now, we just look at our Sentry dashboard and see exactly what's wrong, why, and for who.”

90%

Decrease in most common customer issue

75%

Decrease in software-related support tickets

7,000+

Physical devices monitored

About Matic

Matic Robots makes autonomous floor-cleaning robots for the home. Unlike the robovacs of the past, Matic’s bots use AI neural networks to steer clear of obstructions while learning the layout of customers’ homes. They’re built to traverse all types of surfaces, run quietly, and they’re also quite cute.

Most importantly: they navigate and clean without cloud dependence, keeping their customer data private and performance as high as possible.

The founders are former Google engineers, so it’s no surprise that their bar for product quality is high. As they’ve scaled from a handful of customers to thousands of households, the Matic team made sure to implement a reliable way to monitor health across their robot fleet.

The challenge: Manually reading logs doesn’t scale

Making a physical product robust in a real world setting comes with unique challenges. When robots are navigating various environments in people’s homes, accounting for edge cases is part of the job.

In the early days, Matic’s debugging workflow was manual. When a robot had an issue, the team would get a customer report via their support channels, pull logs from that specific device, and diagnose the problem manually. For a small fleet, that worked. But for operating across thousands of homes, not so much.

“When we started, we were able to look at each individual robot, get feedback from customers about what wasn’t working, go look at logs, and fix those. As the fleet scaled and we shipped more robots into customers’ homes, fleet monitoring became a much bigger need for us.”

— Austin Pineault, Head of Product Engineering

Why Sentry

Matic’s engineering team was already using Sentry for monitoring their mobile app. When it came to monitoring the robots themselves, they had two core requirements: a clear dashboard for surfacing the most impactful issues, and the ability to tag releases and compare error rates over time.

Sentry fit the bill without the overhead of legacy enterprise monitoring tools, and the Rust SDK made it easy to implement.

“Our first use cases [for Sentry] were on the app side, surfacing crashes and getting insights that we weren’t able to readily get elsewhere. From there we felt it was a good tool to expand to the robot itself.”

— Austin Pineault, Head of Product Engineering

How Matic uses Sentry

Matic uses Sentry across their full product stack: the robot itself (Rust), an iOS companion app, and an Android companion app.

The core use case is monitoring “panics” (unexpected crashes) on their in-home robots. For Matic, panics include things like logical bugs, high memory pressure, camera failures, and segmentation faults. Every panic is reported to Sentry along with the bot ID and software release version, giving the team a full picture of what problems are happening, which should be prioritized, and how to solve them. Sentry’s stack traces pinpoint exactly which function failed, making it significantly faster to go from issue to fix.

The team relies on Sentry for two main use cases:

  • Fleet-wide triage. The Sentry dashboard surfaces which panics are most frequent and affecting the most users. Engineers can immediately see what needs attention without digging through logs.
  • Release health tracking. Matic ships software on a weekly cadence. After each release, the team checks Sentry to compare versions and identify any regressions or issues that need attention.

The most common issue? Camera failure

One of the most impactful issues Matic resolved using Sentry was a camera buffer overflow. Through Sentry’s dashboard, it surfaced immediately as the most frequent panic across the fleet, occurring hundreds of times per day. The team was able to quickly understand how many bots were affected by the issue and use stack traces to understand what was causing it. It turned out that the timeouts were caused by an overflow of camera frames that the bot wasn’t able to catch.

Release regression safety net

Matic is moving fast, with weekly software releases going out to their fleet. A bad build can reach customers quickly, and Sentry has become the team’s release regression safety net.

In one case, a TensorRT version upgrade caused a vision pipeline stall, a panic that appeared in the first candidate build after the update. The engineering team quickly paused the rollout and fixed the problem before it impacted customers.

What’s next

Matic plans to scale the fleet significantly from its current size of over 7,000 active customers. As hardware manufacturing ramps and more robots ship, the need for robust fleet monitoring only grows.

The team is also exploring more granular trend analysis of specific errors in Sentry in addition to their standard dashboard view.

Fix It

Get started with the only application monitoring platform that empowers developers to fix application problems without compromising on velocity.