How Kraken Technologies uses Sentry Error Monitoring for their Python Monolith

Kraken is an advanced operating system designed specifically for utilities, providing an all-in-one solution that automates the energy supply chain. This system enhances service and efficiency, supporting the global transition to a decentralized, decarbonized energy system.

Kraken Technologies promotes a customer-centric culture grounded on delivering simple processes and straightforward interactions. It is built around a scalable, cloud-based architecture in Python that is updated nearly 200 times per day using a continuous deployment process

Managing thousands of deployments dailyManaging thousands of deployments daily

As a Python monolith deployed via AWS, Kraken Technologies’ fundamental problem is an enviable one: its size and scale. Simply put, their codebase is huge, and they need an effective monitoring tool that will handle the scale of their architecture and team, enabling them to detect actionable errors and get them in front of the right developer to fix them as quickly as possible.

Using a monorepo approach to handle their development environments, they’re managing some five million lines of Python with a test suite of around 200,000 automated tests. Adding to the technological complexity, the codebase changes constantly. Continuous development is the name of the game at Kraken Technologies, and developers push up to 200 releases per day.

Driving a ‘you build it, you run it’ cultureDriving a ‘you build it, you run it’ culture

When it comes to successfully navigating a development environment of this size, many companies would have chosen to employ some rather draconian systems to manage this beast. Kraken Technologies has taken a different, more egalitarian approach, putting the ownership into the hands of the developers who write the code. This drives an engineering culture that values simplicity, a concept they refer to as “minimum viable everything,” and high trust.

By simplicity, Kraken Technologies believes in choosing the simplest possible “thing that works,” to keep both codebases and processes uncomplicated. They also focus on fostering an environment of high trust— but with an audit trail. What this means in practice is that individual developers are responsible for the lifecycle of their work: from writing specs to post-deploy verification, and it’s up to them to ensure that their code does what it’s supposed to do.

To continue to ship confidently and continuously, they required a scalable solution that could accommodate their expanding team while providing the necessary visibility and tools to identify and fix issues before their customers write in.

We embrace the ‘you build it, you run it’ philosophy, empowering our developers to take ownership of the code they create. Sentry is key for our engineering teams to deploy code multiple times a day, knowing they can identify and resolve issues quickly.

Using Sentry to empower 500+ developers to confidently ship 200 times a dayUsing Sentry to empower 500+ developers to confidently ship 200 times a day

Because of the autonomy with which Kraken Technologies’s developers operate, it’s imperative that they are able to address errors quickly. And this is where Sentry comes in.

With some 500+ developers working around the world, 200 releases of our monorepo every day, and 20+ production environments, it’s vitally important that when an error occurs, our teams are able to detect it and triage it quickly-and Sentry does just that.

With Sentry, developers at Kraken are given the context around everything that led to an error in so they can triage and debug faster— including the number of times the error occurred, the users impacted, the associated release, and complete stack trace of the error. Additionally, with tracing in Sentry, the team can track issues across systems and services, pinpointing exactly where in the code the error occurred.

One of Kraken Technologies’ primary conventions when logging errors is to only send something to Sentry if developers can fix it. The team utilizes Sentry’s release notifications so they know when to monitor their applications for any new issues. Teams at Kraken Technologies also use automatic alerts to issue assignments in Sentry via GitHub CODEOWNERS, integrating with specific Slack channels for alerts. Internal teams then monitor these channels and are able to triage new errors as they happen.

Kraken Technologies has also figured out how to use Sentry when onboarding. They’ve discovered that using Sentry is a great way to introduce new developers to their way of doing things. When someone joins the team, Kraken Technologies will use Sentry to demonstrate how to triage errors, how to debug issues, and how they resolve them.

Not only is Sentry a great error-monitoring tool, but it’s also a helpful learning tool for new team members.

Alert to resolution and tech-debt burndownAlert to resolution and tech-debt burndown

When managing a development environment this massive, how your organization deals with technical debt is extremely important. In order to reduce tech debt, the time between being alerted of an error to the resolution of that error has to be reasonable. For Kraken, when Sentry encounters an issue, that issue is auto-assigned to the accountable team via Slack. Once triggered via an alert in the appropriate Slack channel, someone on that team triages the error. The responsible parties are then able to look at all the available context and trace the issue to the code where the error happened. Sentry even provides metadata about the environment, interface, the releases affected, and related accounts. All of this is in service of helping Kraken Technologies stay lean, agile, and quick to respond.

Noise is always a problem, but Sentry’s issue grouping and codeowners integration reduces the noise, ensuring only the relevant developers get notified of the issues they care about.