Environment variables exist outside of your code as part of your server environment. They can significantly automate running your script/applications and make it more secure.
For instance, instead of manually changing the user name variable when you run a script on a different machine, you can use an environment variable that returns the corresponding value. Moreover, rather than hardcoding sensitive information like API keys and secrets directly in your code, use environment variables. That’s exactly what they are for.
Example use cases of environment variables
- API tokens
- private keys
- paths to files, e.g. credentials
- deployment/production mode option
- database login information, username, secret codes, and many more…
All you need are .env
and envars.py
files
Let’s assume the following structure of the project:
.
├── .env
└── my_module
├── __init__.py
├── envars.py
├── ...
└── my_script.py
A .env
file is a text file containing key-value pairs of all the environment variables your application requires. In envars.py
we access them and save them as global variables. In my_script.py
(or in any other script) the global variables are imported and used.
The proposed structure imposes setting environment variables in one place and accessing them in another. As a result, changing and managing environment variables becomes easy and fast. It also reduces the likelihood of naming conflicts.
Define env vars in .env file
Let’s define some environment variables in .env
file, one key-value pair per line. You can make use of already existing environment variables as well.
HOSTNAME=example_hostname
TOKEN=example_token
USER=${USERNAME}_test
How to load variable from .env
file?
There are multiple ways to load variables from .env
file and export them.
Exporting a variable in the shell makes it available to all subshells and processes created by that particular shell.
1st solution: Use set
Linux command
set -a
. .env
set +a
set -a
causes variables defined from now on to be automatically exported, whereas set +a
turns off this functionality.
2nd solution: Use dotenv.load_dotenv()
Python function
Load environment variables from .env
file in Python script.
from dotenv import load_dotenv
load_dotenv() # take environment variables from .env
The downside of the above solution is that you need to call this function in each and every script where you want to access the environment variables.
3rd solution: Set environment variables in your IDE
If you use VSCode, open the configuration file launch.json
and add envFile
option. Then, run the project in the debug mode.
"envFile": "${workspaceFolder}/.env"
Python extension loads the envFile and then exports the variables.
Access environment variables in envars.py
A good practice is to load environment variables in one, dedicated Python script, called e.g. envars.py
. Consequently, these global variables from envars.py
can be easily imported into any script in your project.
from os import getenv, environ
try:
TOKEN = environ["TOKEN"]
except KeyError as exc:
raise KeyError("Missing credentials.") from exc
HOSTNAME = getenv("HOSTNAME", "localhost")
USER = getenv("USER")
Environment variables may be accessed by using:
os.environ
os.getenv()
If a key is not present, attempting to access it through os.environ
object will raise a KeyError
. You can use it for environment variables that you find mandatory and without which the script won’t work (e.g. access token).
Otherwise, use os.getenv()
or os.environ.get()
. If a key doesn’t exist, the function returns the default value or None
.
Use environment variables
Finally, import variables from envars.py
wherever you want.
Content of sample my_script.py
:
"""Script in which env variables are used."""
from envars import HOSTNAME, TOKEN, USER
print("HOSTNAME", HOSTNAME, "TOKEN", TOKEN, "USER", USER)
Output:
HOSTNAME example_hostname TOKEN example_token USER aleksandra_test
As a result, we get all imported variables since all of them were defined in .env
file. But what happens when some of them are missing?
We know that missing TOKEN
environment variable will result in KeyError
but what happens when we don’t set HOSTNAME
and USER
environment variables in .env
file?
Output:
HOSTNAME localhost TOKEN example_token USER None
Assuming that only TOKEN
was provided in .env
file, we got the default value of HOSTNAME
(defined in envars.py
) and None
value for USER
(default value was not set for this variable).
Recap
There are only 4 steps to make environment variables managing easier:
- Define env vars in
.env
file. - Export env vars e.g. using
set
command. - Access env vars in
envars.py
script and create global variables. - Import variables from
envars.py
wherever you want.
Note: .env
file should be included in your project locally. Do not add it to your public repository since it might contain sensitive data, e.g., tokens or private keys. Describe its content in README file instead.
Full-stack data scientist. Aleksandra writes about topics that she finds interesting for software developers and scientists who are keen on technology. Fan of Polish dumplings with blueberries.
Thanks for a great post!
From my side, I recommend checking out the asdf tool (version management) and asdf-direnv plugin – it makes environment management even more comfortable.
Links: https://asdf-vm.com/
https://github.com/asdf-community/asdf-direnv
Thank you Michał for the recommendations!