EDIT: Don't read this page, there is a new version with more recent material.
After a new week during which a friend's system Python installation was destroyed by yet another unfortunate update, I felt like it was a great time to do some research about the tooling that was designed to prevent that from happening.
In this article, I'm introducing two widely used tools to control potential damage from accidental destruction of Python installations, through isolation:
pyenv: isolates system Python version from development Python versions.
pipsi: isolates command-line tools from system Python version
Some key ideas behind pyenv and pipsi are that you should not be working
directly with your system Python installation, and that your
interpreter and his friends (
pip, packages, CLIs) should live in your
$HOME directory rather than
If you are not sure why this isolation is helpful, here are a few examples:
pip3in cohabitation. You just have one
pippair, which points to a single version of Python at a time. You switch between versions by simply pointing your global Python to the version you need for your current task.
Note: This tutorial was written on an Ubuntu machine. If you are using macOS,
consider replacing some of the instructions by high-level calls to
brew to take
advantage of existing Homebrew formulas. For this, install the
Xcode Command Line Tools (
xcode-select --install) and Homebrew.
pyenv is a command-line tool that enables you to manage Python installations in isolation to the Python installation of your operating system.
The installation of pyenv requires some dependencies, so please install those first:
apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev
brew install openssl readline xz
As pyenv is independent of your OS Python, it does not use Python for
installation, and relies on a Bash script instead. This Bash script
bin/pyenv-installer) is provided in the pyenv-installer
repo. Install it using the following command:
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
After downloading pyenv and installing it to
~/.pyenv, the Bash script
will ask you to add a few initialization lines to your interactive Shell config
~/.bashrc) or your
login Shell config (
.bash_profile). If you don't know which one to
~/.zshrc if you use ZSH and
~/.bashrc if you use Bash.
export PATH="$HOME/.pyenv/bin:$PATH" eval "$(pyenv init -)" eval "$(pyenv virtualenv-init -)"
These initialization lines include a modification of your
enforces the precedence of pyenv over the other registered directories.
trick enables pyenv to intercept Python-related commands.
Pitfall: When adding new locations to the
$PATH in the future for new commands,
always make sure pyenv keeps precedence so it can continue to intercept
the Python-related commands.
After updating your config, exit your user session and login again. Once you're logged back in, open a new terminal.
When the new terminal shows up, the
pyenv command should be available,
and you should be able to run
pyenv versions. This command will show
you all the Python environments installed on your computer. As no new
environment was installed yet, you should just see the Python of your OS:
➜ ~ pyenv versions * system (set by /home/pierre/.pyenv/version)
Now, let's check which Python versions are available, and install one of them:
➜ ~ pyenv install --list | grep 3.6 3.3.6 3.6.0 3.6-dev 3.6.1 3.6.2 3.6.3 3.6.4 ➜ ~ pyenv install 3.6.4 Downloading Python-3.6.4.tar.xz... -> https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz Installing Python-3.6.4... Installed Python-3.6.4 to /home/pierre/.pyenv/versions/3.6.4 ➜ ~ pyenv versions * system (set by /home/pierre/.pyenv/version) 3.6.4
As you can see, our new Python version was installed successfully in
~/.pyenv/versions. There might be a few warnings for some extensions, but
you can fix those later by installing the relevant libraries using
your system's package manager.
In the output of the
pyenv versions command, the little star next
to the version name indicates the current global version of Python,
which is initially
Thanks to pyenv, your system Python is still fully functional
and was not destroyed by your new
➜ ~ python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
After screaming out of joy for your preserved system, you can switch to
your new version of Python using the
pyenv global command:
➜ ~ pyenv global 3.6.4 ➜ ~ python Python 3.6.4 (default, Mar 3 2018, 15:29:45) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
Please note that you may have to close and re-open the terminal session
for the update from the
pyenv global command to come into force.
You can then switch back and forth between your versions using the same command:
➜ ~ pyenv global system ➜ ~ python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> ➜ ~ pyenv global 3.6.4 ➜ ~ python Python 3.6.4 (default, Mar 3 2018, 15:29:45) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
Finally, be aware that pyenv has the ability to select multiple global
Pythons at the same time, even though it is not required for common development
use cases. That is, you may see other tutorials on the Internet where there is more
than a single star in the output of
Pitfall: As you can see, the notions of
system Python do not
refer to the same thing in pyenv. Your
system Python is the one distributed by
your OS, while your
global Python is just a pointer to the Python currently
selected by pyenv and located in your
To avoid messing up your system Python, you should always have
your global Python pointing to a version installed using pyenv. In our case,
the global Python should be pointing to the version
3.6.4 that was installed
pyenv is self-contained, so it can be removed fairly easily. You just need to:
rm -rf ~/.pyenv
pipsi is a command-line tool that installs Python-based command-line tools in separated virtualenvs, which are isolated from your global Python installation and from each other.
That is, pipsi enables you to destroy your Python command-line tools without destroying your global Python installation.
Before running the pipsi installation script, make sure your global Python
points to a specific version of Python such as
3.6.4, and not to
Indeed, pipsi is installed with a Python script (
get-pipsy.py), which makes
it dependent on your global Python installation, and you don't want to
ruin your system installation. Moreover, pipsi will install all tools
using the version of the pyenv it was itself installed with, even if you
change the global Python to point to another version.
As pipsi makes extensive use of
virtualenv, make sure this package is
installed by running
pip install virtualenv.
These caveats being said, here is the command to install pipsi:
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
Pitfall: Depending on whether your operating system has
$PATH by default or not,
get-pipsi.py may warn you that
pipsi command is not in your
$PATH yet. If it's not the case, update
your config to add it manually.
Once pipsi is installed, you can use it to install all kinds of Python command line tools, such as Pipenv:
pipsi install pipenv ... # See how Pipenv was installed in a virtualenv by pipsi ➜ ~ ls -al $(which pipenv) ... /home/pierre/.local/bin/pipenv -> /home/pierre/.local/venvs/pipenv/bin/pipenv
As shown by the
ls command, pipsi installs all tools within
~/.local/venvs directory and symlinks them to
To remove pipsi, you can use pipsi itself:
pipsi uninstall pipsi
In this last section, you will find a very brief overview of Pipenv which is a natural extension of the isolation ideas of pyenv and pipsi to Python project dependency management.
After this overview, you will find a list of links related to this article.
Pipenv is a command-line tool that simplifies Python project dependency management by combining Pip with virtualenv, in such a way that all your projects are isolated from each other and from your global Python.
That is, it enables you to destroy the installation of one of your Python
projects without affecting your other Python projects. At the same time, it
also provides a simple framework to specify package dependencies
in the Pipfile, which takes ideas from
composer.json for PHP.
As Pipenv introduces many more features than pyenv and pipsi, its usage goes
beyond the scope of this blog post. For the time being, you can just install
pipsi install pipenv, and read the
docs of Pipenv for more details. If you just want
a summary, there is a great tutorial by
Alexander VanTol on RealPython.
Here is a list of blog posts related to pyenv and pipsi that you may find interesting:
Installing things the right way with pipsi and pipenv by Craig Loftus, who shows how much pipsi and pipenv simplify the Python development workflow compared to older tools.
PyEnv is the new Conda by Bastian Bechtold, who points out the advantages of standard tools compared to the monolithic Conda.
Pyenv, Python Version Management Made Easier by Senthil Kumar, who gives a more detailed introduction to pyenv.
If you run into issues with the
$PATH manipulations of pyenv, these
StackOverflow answers might help you: