2 easy tips to manage environment variables in Python

Environment variables exist outside of your code as part of your server environment. They can significantly automate running your script/applications and make it more secure.

For instance, instead of manually changing the user name variable when you run a script on a different machine, you can use an environment variable that returns the corresponding value. Moreover, rather than hardcoding sensitive information like API keys and secrets directly in your code, use environment variables. That’s exactly what they are for.

Example use cases of environment variables

  • API tokens
  • private keys
  • paths to files, e.g. credentials
  • deployment/production mode option 
  • database login information, username, secret codes, and many more… 

All you need are .env and envars.py files

Let’s assume the following structure of the project:

.
├── .env
└── my_module
    ├── __init__.py
    ├── envars.py
    ├── ...
    └── my_script.py

.env file is a text file containing key-value pairs of all the environment variables your application requires. In envars.py we access them and save them as global variables. In my_script.py (or in any other script) the global variables are imported and used.

The proposed structure imposes setting environment variables in one place and accessing them in another. As a result, changing and managing environment variables becomes easy and fast. It also reduces the likelihood of naming conflicts. 

Define env vars in .env file

Let’s define some environment variables in .env file, one key-value pair per line. You can make use of already existing environment variables as well.

HOSTNAME=example_hostname
TOKEN=example_token
USER=${USERNAME}_test

How to load variable from .env file?

There are multiple ways to load variables from .env file and export them. 

Exporting a variable in the shell makes it available to all subshells and processes created by that particular shell.

1st solution: Use set Linux command

set -a
. .env
set +a

set -a causes variables defined from now on to be automatically exported, whereas set +a turns off this functionality.

2nd solution: Use dotenv.load_dotenv() Python function 

Load environment variables from .env file in Python script.

from dotenv import load_dotenv

load_dotenv()  # take environment variables from .env

The downside of the above solution is that you need to call this function in each and every script where you want to access the environment variables. 

3rd solution: Set environment variables in your IDE

If you use VSCode, open the configuration file launch.json and add envFile option. Then, run the project in the debug mode.

"envFile": "${workspaceFolder}/.env"

Python extension loads the envFile and then exports the variables.

Access environment variables in envars.py

A good practice is to load environment variables in one, dedicated Python script, called e.g. envars.py . Consequently, these global variables from envars.py can be easily imported into any script in your project. 

from os import getenv, environ

try:
    TOKEN = environ["TOKEN"]
except KeyError as exc:
    raise KeyError("Missing credentials.") from exc

HOSTNAME = getenv("HOSTNAME", "localhost")
USER = getenv("USER")

Environment variables may be accessed by using:

  • os.environ
  • os.getenv()

If a key is not present, attempting to access it through os.environ object will raise a KeyError. You can use it for environment variables that you find mandatory and without which the script won’t work (e.g. access token).

Otherwise, use os.getenv() or os.environ.get(). If a key doesn’t exist, the function returns the default value or None .

Use environment variables 

Finally, import variables from envars.py wherever you want. 

Content of sample my_script.py:

"""Script in which env variables are used."""
from envars import HOSTNAME, TOKEN, USER

print("HOSTNAME", HOSTNAME, "TOKEN", TOKEN, "USER", USER)

Output:

HOSTNAME example_hostname TOKEN example_token USER aleksandra_test

As a result, we get all imported variables since all of them were defined in .env file. But what happens when some of them are missing? 
We know that missing TOKEN environment variable will result in KeyError but what happens when we don’t set HOSTNAME and USER environment variables in .env file?

Output:

HOSTNAME localhost TOKEN example_token USER None

Assuming that only TOKEN was provided in .env file, we got the default value of HOSTNAME (defined in envars.py) and None value for USER (default value was not set for this variable). 

Recap

There are only 4 steps to make environment variables managing easier:

  1. Define env vars in .env file.
  2. Export env vars e.g. using set command. 
  3. Access env vars in envars.py script and create global variables. 
  4. Import variables from envars.py wherever you want. 

Note.env file should be included in your project locally. Do not add it to your public repository since it might contain sensitive data, e.g., tokens or private keys. Describe its content in README file instead.

2 Replies to “2 easy tips to manage environment variables in Python”

  1. Thanks for a great post!
    From my side, I recommend checking out the asdf tool (version management) and asdf-direnv plugin – it makes environment management even more comfortable.

    Links: https://asdf-vm.com/
    https://github.com/asdf-community/asdf-direnv

    1. Thank you Michał for the recommendations!

Leave a Reply