Python 12-factor apps with envparse

Making 12-Factor apps is all the rage these days and not without good reasons. Using the system environment to get your app's configuration is probably the most flexible way to pass configuration information once you get used to it.

One mental barrier I know I had was I didn't want to always have to pass a bunch of variables just to run my code. The ideal setup is:

Load the env vars from a file in the current directory or any directory above
Have any variables from the actual environment included and give the ability to override them on a per execution basis on the command line

Luckily, the envparse library for Python gives us this pretty easily. There are two ways you can use it. The "standard" way or more manual way. Or you can define up front a schema of variables you expect and ensure they are cast to the types you need.

Let's explore the standard way first.

from envparse import env

env.read_envfile()

That's all you need to do to load up the environment. The library will walk your file system path looking for a .env file, if it finds one it will load it. If it doesn't it will issue a warning you can safely ignore.

You're on your own for validation, however. If you go to access a variable that doesn't exist you'll get a KeyError.

Let's assume we have the following in a .env in the current directory:

AWS_ACCESS_KEY='blahblahblahblah'
AWS_SECRET_KEY='WeAreNotThatDumb'
CONSULTING_LEVEL='Pro'

If we then called our script like this:

OTHER='bar' CONSULTING_LEVEL='Expert' python envparse-example.py

Now let's look at envparse-example.py, what do you think it would output?

from envparse import env

env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER', default='foo'))

You win a cookie or something if you correctly guessed:

ACCESS = AKIAJI4KRV67OPXQZS7Q
SECRET = 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
CONSULTING_LEVEL = Expert
OTHER = bar

I want to point out two things about this. First, as you probably already understood the fact we have 'CONSULTING_LEVEL' set in both the file and on the command line, the command line version takes precedence.

The second thing I want to point out is the last line in the source above. We overrode 'OTHER' on the command line, but if we hadn't it would be set to the default of 'foo'. This helps allow you to set sane defaults in the absence of the variable being set anywhere at all.

envparse using a Schema

Now that we've covered the more manual and direct way of using envparse, let's play with the Schema support real quick.

from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER'))

Assuming the same .env file and overrides on the command line this will produce the exact same results. The different here is we import 'Env' (note the capitalization) and set the variables we expect to have.

If something is missing from our environment, envparse raises a envparse.ConfigurationError rather than getting the more generic KeyError when we go to access to a missing value.

Using a schema like this also gives us options for settings defaults as part of the definition rather than when calling for the value itself. envparse has a few other useful options for casting more complex types and giving you the ability to pre or post process the values when they are read which can be useful to help normalize values.

Using envparse with Click

I'm a huge fan of Click and while it has great support for pulling values out of the environment for you, it doesn't support using a .env file directly. Lucky for us it's straightforward to use envparse with Click. If you need options that can come from a .env file, the environment directly, or passed on the command line you can do this:

import click

from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

@click.command()
@click.option('--access-key', default=env.str('AWS_ACCESS_KEY'))
@click.option('--secret-key', default=env.str('AWS_SECRET_KEY'))
def testing(access_key, secret_key):
    print(access_key)
    print(secret_key)

if __name__ == '__main__':
    testing()

The trick here is to use envparse to read in your expected values, whether they come from a .env file or are passed on in via the environment it doesn't matter. We then use the value from envparse to set the default for Click. If we need to, we can still override this on the command line and everyone is happy.

Hope this helps you make your Python app's configuration more flexible!

python django

Pro-Tip – Python 12-factor apps with envparse

envparse using a Schema

Using envparse with Click