Sidekiq vs Delayed Job vs GoodJob

Table of Contents

Why do we need such a comparison at all? #

Comparing the three most popular background job processing systems makes sense to understand which option the developer should choose to implement a required feature or migrate from one gem to another. Each of these systems has its advantages and disadvantages. The best choice for a particular application will depend on the project’s specific needs. Generally, all three gems support ActiveJob and are interchangeable on the top level and when we work with basic functionality. But what happens when we need to use more specific logic and performance is essential? Which gem will be the most relevant for the particular case?

In the article below we’re going to compare Sidekiq, GoodJob and Delayed Job by the following criteria:

Its main technology and features
Presence of dashboards with information
Flexibility, support and community
Possible usage scenarios

Short overview of technologies #

Sidekiq #

Sidekiq is a Ruby-based background job processing system designed to be simple to use and highly efficient. It is built on top of Redis, and is optimized for speed and scalability. As a result, it comes with the obvious benefit of being much faster than regular database systems. In addition, each Sidekiq process can spawn multiple threads to process the jobs in parallel even faster.

Additionally, Sidekiq provides a lot of functionality out of the box, such as support for middleware to update job metadata or skip queuing a job, and, of course, Web UI with informative dashboards with jobs statistic and information about workers.

Although, this library provides two paid plans: Pro and Enterprise.

Comparison between Sidekiq paid and free plans.

Each of these paid plans has very important additional and most notable killer-features:

Built-in Cron Jobs.
Custom Rate Limiting.
And, of course, Batch Jobs.

Also, one of the most important thing related to Sidekiq is that it is under active development and its creators are working full-time on it. So the support of this library is much better, while the community continues growing.

If you want to know more about Sidekiq, please, take a look at our previous article about it in detail!

When we talk about Sidekiq we should consider an important topic connected to its pros and cons.

Starting with advantages, we can speak about:

Its speed - Sidekiq is really fast!
Multithreading.
It’s scalable and automatically runs Redis with zero config.

Disadvantages that are worth mentioning:

It requires several dependencies.
Sidekiq depends heavily on Redis.
Possibility of data loss: if an instance of Redis crushes, you can lose all data that stored in.

Delayed Job #

Delayed Job, also known as DJ, is a direct extraction from Shopify and uses a table to maintain all background jobs. So we can say that the core technology here is a regular database system such as PostgreSQL.

Many developers choose Delayed Job because it is simple and uses their existing database. You don’t need to spend or maintain other additional resources. However, it will still take up space in your database table and if an app has a high database load using Delayed Job may not be a good background queueing library. You may need more disk space if you simultaneously have too many jobs in the queue.

The gem itself doesn’t have a built-in Web UI, but delayed_job_web gem gives access to a basic dashboard with similar features as Sidekiq has, but it lacks some functionality, unfortunately.

Also, it is worth mentioning that Delayed Job appears to be in maintenance-only mode and there are a lot of open issues on Github for now. There were only a few minor releases for the last couple of years.

Overall, Delayed Job is easy to integrate with Rails, it has minimal dependencies and offers reliable processes. But at the same time, the main disadvantages of this gem are its performance (if we compare it with Sidekiq), lack of built-in dashboards and dependence on the same database where you store regular data.

GoodJob #

GoodJob is a Postgres-based, ActiveJob backend for the Rails application. It’s pretty similar to the Delayed Job, but unlike it, GoodJob is multi-threaded. It also has rich documentation with all the needed information about the gem and its features in README. GoodJob’s UI provides some features within GoodJob such as history and search that are not free with the basic Sidekiq plan.

Generally, GoodJob could be called an inheritor of the well-known Delayed Job gem, but better and newer.

Since GoodJob was released only in early 2020, it’s still under active development and its community only starts growing.

Conclusions #

Gem	Queues, priority, retries	Database	Concurrency	Reliability/Integrity	Latency
GoodJob	✅ Yes	✅ Postgres	✅ Multithreaded	✅ ACID, Advisory Locks	✅ Postgres LISTEN/NOTIFY
Delayed Job	✅ Yes	✅ Postgres	🔴 Single-threaded	✅ ACID, record-based	🔶 Polling
Sidekiq	✅ Yes	🔴 Redis	✅ Multithreaded	🔴 Crashes lose jobs	✅ Redis BRPOP
Sidekiq Pro	✅ Yes	🔴 Redis	✅ Multithreaded	✅ Redis RPOPLPUSH	✅ Redis RPOPLPUSH

To summarize everything above, all these three gems offer great solutions for processing background jobs. Ultimately, the gem you choose should be based only on your specific developers’ needs, such as simplicity, performance, stability and way more!

Redis is much faster at querying data than traditional databases like Postgres because it stores data in memory instead of on the disk. If performance and long-term maintainability are important, Sidekiq is the only possible background job processor here. On the other hand, if running costs are a concern, Delayed Job or GoodJob can help you. We’d recommend you select a GoodJob gem only for small projects without large amounts of background calculations or for developers’ projects only because of its newness and lack of real production usage examples for now.