How and where should you be storing runtime configuration, especially secrets?

Store config in the environment

No credentials should be checked in with your source code.  This allows the identical application to be deployed in a number of different environments, from local dev to production, with no code or even binary changes. Paths to any external resources should be defined in the environment, allowing services to move without requiring anything more than a restart of the application.

What’s the Point?

It turns out that Sun was correct, all those years ago: The Network Is The Computer. Most modern systems rely on a variety of external systems to operate correctly, and generally expose their own interface over a network for external users to access remotely.

Now the thing about external systems is that they tend to change their endpoints.  Having to recompile an application because you’ve upgraded your database server isn’t any fun, and needing to change your source code to do so is an order of magnitude worse. By storing this information in the environment, at the very worst you’ll need to restart your services when they inevitably change.

More than One

Even in a small shop, your application is likely to be executed in a variety of situations.  Developers and QA staff may run it locally, your CI server may execute it in order to run automated integration tests, it may be deployed into a staging environment for testing purposes, and of course there’ll be at least one production environment that actually runs “the system”.  More to the point, you probably don’t know - and shouldn’t know - all of these cases during product development.


A completely separate environment that may or may not share external services with any previously existing runtime may be created by setting the appropriate environment variables prior to running the application. At no point should code changes be required.

Furthermore, these variables are read at runtime.  Not during the build.  A built application should be able to be launched into a completely new network environment without being rebuilt.  As a bonus, this keeps sensitive data out of your code repositories and also out of your executables.

What’s Modifiable

In short, anything that deals with your application reaching outside of its own process space should be defined through the environment.  This would include, for example, the network port that an HTTP server is exposed over, or the location of a remote DBMS or Redis server. Changing any of the ways that the process interacts with the outside world at an infrastructure level should never require a new binary to be created.

Additional behaviors may be defined in the environment, but that’s not a 12 Factor requirement.

Out of the code!

There are basically three places in which external connection information can be stored: your source code, your environment, and a connected third party such as your database. Of all of these, the code is the most problematic - changing values here require a recompile which should, in theory at least, trigger a full round of testing before release (which makes updating database credentials from a test DB to a production DB, for example, problematic at best).

You could work around this by storing multiple credentials in the code, but then you’re still exposing them to every developer who has access to your git repository.   Its possible that you could use a separate repo and create tiny libraries to be linked in per-build, but now you’re going to a large amount of work in order to avoid doing a small amount of work and are still missing out on many of the benefits of a fixed build.

Out of the database!

Let’s run through a different scenario.  You’re seeing an issue in production, but its hard to reproduce.  QA has requested to pull down a copy of the production data and drop it into their secured testing environment.  This is a real scenario that happens on a regular basis, mind you. While there may be concerns around customer privacy, you should be able to comply with the technical side of this request easily enough, even if (hopefully) your QA environment is detected from your production servers and vice versa.

This means that links to messaging busses, email servers, payment gateways, and all kinds of other services must be defined somewhere that is not the database, since they are all part of the definition of the QA environment that should persist beyond a simple database restoration.

Depending on your industry, your customers may also have rights to the business data stored in your database.  At no point should that imply access to your internal network topology - and vice versa. Bottom line, DevOps choices are not data.

This one can be insidious by the way.  As a random example, if you use Google Datastore then its urlsafe entity keys actually contain not only the entity and id information (convenient) but the environment also (dramatically inconvenient). Always understand the data that you're storing!

Connections vs Configuration

Oddly enough, the database is a great place to store data that is business-specific rather than technology-specific. If an organization outside of IT/Dev would need to be involved to change something, it probably belongs in the database.  First, this implies something that at some point you might want them to control, and that’s not a phrase that most people use to describe their environment. Second, it implies that the lifecycle of these changes will extend beyond that of the software deployment, and that restructuring your severs and service providers should not imply changing these values.

Dev Mode

Just don’t. If you’ve done your configuration correctly, you should be able to launch “development” versions of your system without having anything inside the code that knows that its running in that particular environment.  This will pay off in spades since you won’t have to worry about complex branching based on your system’s (possibly incorrect) understanding of where its running.

If you want, you can even extend this functionality to other systems. I recommend, as an example,  creating a new configuration of your mail sending system that rejects anything that doesn’t map to your internal domain and providing that as a configuration for outbound emails. That means that your source code doesn’t have to contain a different path to make sure that you don’t accidentally email bomb people during testing, while still providing the same level of protection.

Why not?

Hopefully this one doesn’t come as much of a shock.  If you have any good reason as to why external resource locations should be baked into a build, other than “tradition,” I’d love to hear about them.  I can usually come up with some kind of rationalization as to why someone might want to violate any particular 12 Factor directive, but this one just makes too much sense to ignore.