#439 – Moving to a Monolithic Backend

Over the past couple weeks we overhauled our backend.

Previously, we had a cluster of web servers that sent requests via Redis to a cluster of stateless executors. We architected this over five years ago back when we were running everything (including our database!) on a single Droplet in Digital Ocean. As we moved to Kubernetes and scaled up our infrastructure, we realized we could simplify our architecture, reduce costs, and still provide the same quality of service by combining the services into a monolith.

The move was prompted by an upcoming feature where users can generate AI-powered insights that analyze previous meetings. It can take awhile to generate these insights, so instead of waiting for the whole report to get generated, we wanted to stream them to the user just like what users see when they use ChatGPT. We accomplished this by using graphql-yoga and the upcoming @stream directive. This replaced the homegrown solution that we built all those years ago and has greatly simplified how our backend works.

Of course the migration wasn’t without problems. On the new architecture, our Datadog tracing agent is still using far too much memory, and we have a stochastic memory leak that we’re still tracking down, but overall it has been a large win in terms of cost, latency, and simplicity. Ironing out the last couple bugs is just a part of the process. Once complete, we’ll have much better error tracing and latency metrics, making bug bashes even easier.

At the same time, we also merged Chronos, our cron job process, into the same monolithic service. This has also improved efficiency as well because the calls to the server are now coming from inside the house, so to speak.

It is nice to see an uptick in new signups, given we’ve been watching web sessions decline. We are currently working a new marketing strategy that includes AIO as we follow the changes in search.