Skip to content

Virtual Environments for Python

Virtual Environments are essential for making reproducable code. Whether you want to move your process to a different platform, or have collaborators run the same code, virtual environments will make sure everyone’s on the same page.

In python, there are two main virtual environment managers:

  • Anaconda
  • Virtualenv

Anaconda

For starters, if you do not have conda installed on the yens, see this page for more info.

Create a virtual environment for your process to run

To create a conda environment, run the following command on one of the Yens:

conda create --name my_conda_env

This will make an environemnt named “my_conda_env”. You can also specify which python version you want:

conda create --name my_conda_env python=3.6

Activate your environment

First, to use your environment, you need to activate your environment:

conda activate my_conda_env

You will know your environment is activated because it will show up on your command line:

(my_conda_env) yourSUNetID@yenX:~$

Install any packages needed in that environment

Then, you can install any packages you need using conda or pip:

conda install numpy
pip install torch

If your environment is activated, conda keeps track of which packages are installed where. Some packages will require pip to install - your new conda environment has it’s own pip install.

I’ve made an environment - now what?

Using your environment is very simple - as long as your environment is activated, you can run python normally:

(my_conda_env) yourSUNetID@yenX:~$ python my_script.py

The python command will be specific to your environment. You can troubleshoot this with the which command:

(my_conda_env) yourSUNetID@yenX:~$ which python
/path/to/your/conda/envs/my_conda_env/bin/python

You can see that this python is specifically in the my_conda_env directory for your environment.

Deactivate your environment when you are finished

When you are finished using your environment, or need to switch to a different environment, you can deactivate it with:

Check out the conda docs for many other useful features!

Saving and moving your environment

One of the big advantages of virtual environments is sharing and moving the environments. This is done by saving the environment to a file. In conda, you can save your environment by running:

(my_conda_env) yourSUNetID@yenX:~$conda env export > my_conda_env.yml

You will now have a file named my_conda_env.yml with all the necessary information for conda to build your environment. If you want to load this environment on a new server, you can run the following command:

conda env create -f my_conda_env.yml

Virtualenv

Create a virtual environment for your process to run

To create a virtualenv environment, run the following command on one of the Yens:

virtualenv --python=/usr/bin/python3.6 TEST

This will make an environemnt named “TEST”.

Install any packages needed in that environment

First, you need to activate your environment using source:

Then, you can install any packages you need using pip:

I’ve made an environment - now what?

Using your environment is very simple - as long as your environment is activated, you can run python normally:

(TEST) yourSUNetID@yenX:~$ python my_script.py

The python command will be specific to your environment. You can troubleshoot this with the which command:

(TEST) yourSUNetID@yenX:~$ which python
/path/to/env/TEST/bin/python

Saving and moving your environment

One of the big advantages of virtual environments is sharing and moving the environments. This is done by saving the environment to a file. In virtualenv, you can save your environment by running:

(TEST) yourSUNetID@yenX:~$ pip freeze > requirements.txt

You will now have a file named requirements.txt with all the necessary information for pip to build your environment. If you want to load this environment on a new server, you can run the following command:

$ source <env_name>/bin/activate
(<env_name>)$ pip install -r path/to/requirements.txt

Deactivate your environment when you are finished

When you are finished using your environment, or need to switch to a different environment, you can deactivate it with:

The virtualenv docs have more information on virtualenv