A Framework Comparison Of Grape vs. Rails::API

As part of Backupify’s mission to provide the definitive enterprise cloud-to-cloud backup solution, we have spent a lot of time this year working on a new API to support a partner offering that will have a huge impact on the industry.

Until this project began, Rails has been the application framework of choice for most projects at Backupify, powering everything from the core Backupify application to some of our lesser known offerings like Migrator. For each of these applications, a full-on framework like Rails was the right solution since each of these projects is a full featured, front-facing web application. However, given the distinct, typically back-end centric nature of an API, we opted to take some time to evaluate other alternatives for building a robust API on top of Ruby rather than jumping straight to ‘rails new awesomeness’.

From the get-go, we had two frameworks in mind: Initridea’s Grape and the trimmed down Rails offshoot, Rails::API. In this post and a later follow up, I’ll take a closer look at both of these frameworks and evaluate their strengths and weaknesses. However, before we dive into the nitty gritty, first, a (mostly harmless) disclaimer.

DISCLAIMER: I came into this comparison leaning toward Rails::API (gasp!). This initial leaning likely stems from the fact that I have experience with Rails and hadn’t used Grape prior to this experiment. Beyond merely just kindling an initial bias, my experience with Rails puts Grape at a distinct disadvantage because I have more experience optimizing Rails and when it really comes down to it, I’ve learned to think “The Rails Way.” There’s also some chance that there’s something fundamental about Grape that I’ve missed. That said, I’ve done my best to make an unbiased comparison and hope my findings will help you to make a more informed decision when deciding between Grape and Rails::API.

It’s also a good time to mention that this post assumes a familiarity with Ruby and Rack and, as such, makes relatively little effort to explain more introductory topics. If that concerns you, I’d still encourage you to peruse the rest of the post, as much of the content is fairly introductory as it is.

Disclaimers aside, this post focuses on performance, leaving other considerations to the followup article.

Okay, let’s do this!

Performance

In software, performance is a crucial factor, perhaps even the most critical factor. To this end, I’ve put together sample applications for both Grape and Rails::API that will allow us to dig into each of the frameworks and get a feel for the performance of each along the way.

For Grape, I’ve put together a very simple Rack application to support the API. It’s pretty standard Rack code, so if there’s anything that doesn’t make sense to you, I’d suggest taking a deeper dive into Rack. Source for the example Grape app can be found here: https://github.com/backupify/grape_v_rails_api_grape_app

For Rails::API, the sample application provided is fairly similar to a new application generated by running `rails-api new`. I’ve made a few tweaks to get the application into a working state and ready to rock JRuby style, but it’s essentially equivalent to what `rails-api new` gives you. Source for the sample Rails::API app can be found here: https://github.com/backupify/grape_v_rails_api_rails_app

Since both of these frameworks are built on top of Rack, I’ve chosen puma for the underlying server for both applications to allow for a more consistent comparison and to allow us to take advantage of multiple OS level threads in JRuby to get an idea of any differences related to concurrency.

I’ll be using ab (apache-bench) against a local application instance with a sample size of 100K requests for each of the performance tests.

To start the Grape application I’m using the command:

bundle exec puma -p 3000

For Rails::API, I’m using the same command except with a different port and I’m setting an environment variable to set the Rails environment to Production to avoid development conveniences that would slow Rails down:

RAILS_ENV=production bundle exec puma -p 3001

With our initial applications in place, let’s take some baseline measurements. The ab command I’m using looks like the following, with variable ports and concurrency levels depending on the test and framework being tested:

ab -c1 -n100000 http://localhost:3001

And here are the initial results:    

It’s pretty obvious from these results that Grape destroys the default Rails::API configuration. Beyond that, I found it surprising that Ruby 1.9.3 outperformed Ruby 2.0 and JRuby for both frameworks.

The Catch

Now these results may seem decisive, however the default configuration for Rails::API is 99% the same as Rails, so chances are there’s a lot of cruft that’s included that’s not needed for a pure API. These extras are likely a lot of what is slowing Rails down. Beyond that, the default configuration includes a lot of functionality not available in Grape, so realistically, the performance test we’ve just run isn’t comparing apples to apples (or grapes to grapes, if you prefer). But wait. What’s that you say? What if we try to trim down Rails::API to better reflect the functionality available in Grape? Well, I’m glad you asked!

If you look through the commit history for the Rails::API application, you’ll notice one of the first things I changed was to replace `require ‘rails/all’` with a require for each of the sub-frameworks (ActiveRecord, ActionMailer, etc) in config/application.rb. Depending on the needs of your API, you can probably remove many of these sub-frameworks by removing the respective require line. For the purposes of this comparison, I’ve removed most of the sub-frameworks to see how far we can push the performance of Rails::API.

I won’t get into the specifics of the optimizations I’ve made to accommodate the next round of tests, but I encourage you to look at the commits to the source for more detail if you’re curious. The high-level goal of each change was to try to make the functionality available in Rails::API more similar to that available in Grape.

In brief, I’ve:

• removed all sub-frameworks except for ‘action_controller/railtie’
• removed the Postgresql gems
• removed the cache_store
• removed all middleware but the router (See Rails::API docs for explanation of various middlewares)
• removed the LogFormatter
• set the log_level to :error

And here are the results of the optimized Rails::API app compared to Grape:

Wow, what a difference! Rails::API still trails Grape in every single test, but they are at least much closer now. Arguably, for many APIs the difference is negligible, but if performance is your Holy Grail, look no further than Grape.

If these optimizations still leave you feeling a need for speed, it is possible to further trim down Rails::API. In fact, if you look through the docs for Rails::API, you’ll notice that the version of ActionController::API that Rails::API provides includes a number of controller modules that may not be necessary, depending on the needs of the API. This means that we could go further to optimize Rails::API if we wanted to dig in and create a custom version of ActionController::API. We’ll leave such optimizations as an exercise for the reader, but it should be noted that many of the included controller modules mirror functionality available in Grape. So, while many of the optimizations I previously made brought Rails::API closer to the level of functionality of Grape, removing some of the controller modules included in ActionController::API would make Rails::API faster, but also make for a less fair comparison.

That said, let’s give credit where credit is due: though it may be possible to further tweak Rails::API to reduce the performance gap, Grape is definitely the winner when it comes to performance.

Other Considerations

Though I said at the start that performance is a big factor, maybe the biggest, there are a lot of other factors to take into consideration. From community to documentation to functionality to personal taste, what draws one person to a framework deters another. Though we won’t be able to do them all justice, we’ll take a deeper look at some of the other considerations you should make before making your final decision in the second part of this comparison.
So stay tuned! Same Bat time, same Bat channel!

Other Resources

• http://joshsymonds.com/blog/2013/02/22/existing-rails-api-solutions-suck/
• http://matthewlehner.net/rails-api-testing-guidelines/
• For something a little different but still part of our “engineering in the cloud blog series, check out our recent post on Deploying open source Platform Docker