Announcing Cage: Develop and deploy complex Docker applications

This post is part of our devops series.
At Faraday, we rely heavily on microservices to analyze data, generate maps and make predictions. All our microservices run inside Docker containers, which makes it easy to run our code either locally or in the cloud.
But if you've ever worked on a large project with lots of services, you're aware that local development can be difficult:
- You need some way to run a complex set of microservices locally on your laptop.
- You need to be able access the source code for multiple projects easily, edit it locally, and see the changes immediately.
- You need to remember how to run tests for services written in multiple languages by multiple teams.
Originally, we used docker-compose to work on our services locally. It offered some great features, but it didn't quite do enough:
- We very quickly wound up needing multiple
docker-compose.yml
files. For example, you're encouraged to use them to define "task" containers. They're also helpful if you need to selectively restart individual portions of your application or if you use multiple load balancers. docker-compose.yml
files often contain a fair bit of duplication. There are ways to reduce this usingextends:
andenv_file:
, but it still requires manual maintenance.docker-compose
provides limited support for working with multiple source repositories.
But what if there were a tool that made complex microservice projects as simple and easy as a Rails web application? We decided to build some tools and see how simple we could make it.
Introducing Cage
Cage is an open source tool that wraps around docker-compose
, and it tries to make local development as easy as possible.
We can get started by using cage to generate a new project:
cage new myproj
cd myproj
Next, we can start up our database server and create a new database. This part should be familiar to docker-compose
users:
cage up db
cage run rake db:create
cage run rake db:migrate
Once the database is set up, we can start the rest of the application:
cage up
This should make a web application available at http://localhost:3000/.
Diving deeper
If we open up pods/frontend.yml
, we'll see a standard docker-compose.yml
file:
version: "2"
services:
web:
image: "faraday/rails_hello"
build: "https://github.com/faradayio/rails_hello.git"
ports:
- "3000:3000"
labels:
io.fdy.cage.srcdir: "/usr/src/app"
io.fdy.cage.shell: "bash"
io.fdy.cage.test: "bundle exec rake"
We see that frontend.yml
defines a single web
service using the faraday/rails_hello
image, with source code available from https://github.com/faradayio/rails_hello.git
. (There are also some labels
that we'll explain later.)
Let's get this source code and make a change! First, we need to "mount" the source code into our service, and restart the app:
cage source mount rails_hello
cage up
This will clone a copy of the rails_hello
source code in src/rails_hello
, and mount it into our web
service in the directory specified by io.fdy.cage.srcdir
above. So we can just go ahead and create an HTML file at src/rails_hello/public/index.html
:
<html>
<head><title>Sample page</title></head>
<body><h1>Sample page</h1></body>
</html>
If we go back to http://localhost:3000/ and reload, we should see our new page!
Testing and shell access
One challenge on large microservice projects is remembering how to test other people's code! We specified io.fdy.cage.test
above, which specifies how to run tests for our web service. We can invoke this as:
cage test web
If we have other services written in other languages, we could also test them using cage test $SERVICE_NAME
.
Similarly, if we want to get command-line access to our web
service, we can run:
cage shell web
How we built cage
cage
is a single binary with no dependencies. It's written in Rust and the Linux version links against musl-libc, so you should be able to install it on any modern Linux distribution using cp
.
cage
relies heavily on the compose_yml
library, which provides a typesafe API for working with the complex data structures in a docker-compose.yml
file.
Internally, cage is structured a bit like a multi-pass compiler. In this case, the intermediate language would be docker-compose.yml
files, and various transformation plugins each transform the files in some way.
Rust has been a great language for this project:
- Rust allows us to build fast, standalone binaries.
- Refactoring Rust code is a joy, because the compiler can catch so much.
- The
cargo
build tool and the crates ecosystem is great. - Rust's type system allows us keep careful track of exactly what's in a
docker-compose.yml
file, which fields are optional, and which fields require shell variable interpolation. (It's far more complex than it looks.) Without a strong type system, it would be very easy to overlook an important case when writing a transformation plugin.
Getting started
Cage is still extremely new, and few people outside of Faraday have ever used it! So we encourage you to contact us and to ask us questions.
- To learn more about
cage
and install it, see the website. - To look at the source code or report issues, please see the GitHub repository.
We're interested in hearing about what works, what doesn't, and what's too confusing. We're also interested in ideas for new features to simplify your development workflow.