Autoscaling Sidekiq Workers on Convox
I’ve become a big fan of Convox over the past few years. It’s a great platform for deploying, scaling, and managing web without having to spend large amounts of time managing infrastructure. It also works with multiple clouds, so you can easily run your app on AWS, GCP, Azure, and more.
Among the features is autoscaling, which allows you to horizontally scale your application based on specific metrics. For a web application, this is usually based on the number of requests, or CPU/memory utilization. But if you’re looking to scale Sidekiq workers then you’re going to need a different metric.
Typically here you would want to look at one of these two metrics:
- Enqueued Jobs. How many jobs do you have waiting to be processed?
- Worker Utilization. How many workers do you have idling?
The idea is that once your queue depth or utilization increases, to increase throughput you would...