If it loses its quack, does it still talk like a duck? Using Swagger to detect breaking API changes.

October 2, 2015 Jeff Cousens

A while back we decided that all new functionality in the Civis platform would be implemented as a combination of API endpoints and front-end code. At the same time, we decided that all endpoints will be available to our customers. That is, if you can use a piece of functionality via the Civis UI, you can leverage the underlying API for programmatic access, too. This has been a huge force multiplier, as you can read about in The Civis API: Scale Up Your Data Science. It also means that our Engineering Team can’t break our API contract just because it suits the needs of the UI.

One solution is to rely upon API versions, which were built in to the Civis API from day one. But even with versions, we still need to know whether a change constitutes a breaking change. We consider it a breaking change to:

  • remove an endpoint
  • add a new required request parameter
  • remove or change the type of a request parameter or response attribute

Published guidelines are good, but they can be fallible. We prefer to leverage automated solutions that codify our best practices, and make explicit and enforce our policies. With that in mind, we wrote Swagger::Diff. Swagger::Diff compares two Swagger specifications and identifies any breaking changes. For example, to compare the Swagger Project’s expanded petstore example against the minimal petstore example, verifying the expanded example is backwards compatible with the minimal example:


$ gem install swagger-diff
$ wget https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-minimal.json
$ wget https://raw.githubusercontent.com/swagger-api/swagger-spec/master/examples/v2.0/json/petstore-expanded.json
$ swagger-diff petstore-minimal.json petstore-expanded.json

Swapping the order of the arguments, it compares the minimal example against the expanded example, flagging the endpoints, parameters, and attributes that are missing from the minimal example:


$ swagger-diff petstore-expanded.json petstore-minimal.json
- missing endpoints
  - delete /pets/{}
  - get /pets/{}
  - post /pets
- incompatible request params
  - get /pets
    - missing request param: tags (type: array)
    - missing request param: limit (type: integer)
- incompatible response attributes
  - get /pets
    - missing default response

Swagger::Diff also integrates directly with our test suite. People might forget to run a check, but continuous integration (CI) does not. Coupled with an API endpoint that returns our current Swagger specification, it allows us to simply and easily compare two versions of our API via a request spec:


it 'is backwards compatible' do
  production_swagger = open('https://host.domain/endpoints').read

get '/endpoints' test_swagger = response.body

expect(test_swagger).to be_compatible_with(production_swagger) end

Every time we push code, our CI tests the development branch against the production specification, and fails with a message detailing backwards-incompatible changes if any were introduced. This guarantees that no backwards-incompatible changes will accidentally slip into our API.

We’ve open-sourced Swagger::Diff under the BSD 3-Clause License, published the gem to RubyGems.org, and made the source available on GitHub. We encourage you to add it to your CLI toolset, project’s CI, or test suite, and we welcome issues or pull requests.

Do you enjoy building APIs? We’re hiring.

The post If it loses its quack, does it still talk like a duck? Using Swagger to detect breaking API changes. appeared first on Civis Analytics.

Previous Article
Dear Madison Avenue: You shouldn’t quit TV.
Dear Madison Avenue: You shouldn’t quit TV.

We’re introducing the software to help you save it. “TV as an advertising medium is broken.” You hear...

Next Article
Guiding our Summer Interns to Become More Effective Engineers
Guiding our Summer Interns to Become More Effective Engineers

A great internship or first job blends responsibility, real-world experience, and great mentorship. As we s...