It has been 1.5 years since I published my first tutorial about "sane Python installation management". However, I still hear about Python environment issues on a regular basis from coworkers, particularly regarding local project dependency management, and I felt like an update to the original tutorial would be beneficial.

This new "2020" version of the tutorial will re-use a good amount of material from the original one, but will try to put each tool being used into a broader context, and explain how all tools relate to each other. This new version will walk you through the way to a sane Python development environment, covering the following topics in separate posts:

Part 1: Motivating the the case for isolation in the context of Python environments. You are currently reading this article.
Part 2: How to manage your Python interpreter versions with pyenv, including shielding your system Python
Part 3: How to isolate your Python-based command-line tools with pipx, so they don't break each other when upgraded
Part 4: How to isolate your Python projects development environments with pyenv virtualenv

EDIT from Tuesday, Nov 12, 2019: Interestingly enough, Jacob Kaplan-Moss actually published an updated version of his article from 2011 at the same time as I published my own update. You can check it out here: My Python development environment, 2020 edition. It also prompted some debates on HackerNews.

EDIT from Tuesday, Jan 7, 2020: The commands and the console output of this tutorial have been updated with Python 3.8.1 (was 3.6.4 before), and some installation steps have been clarified.

EDIT from Saturday, Feb 22, 2020: I recently discovered pip-tools, which is an awesome program to manage requirements.txt files. pip-tools is much closer to the "vanilla" pip experience, and therefore much more stable and less prone to usability problems. Combined with pyenv, it solved all my use cases for project dependency management.

The key idea behind all tools described above is environment isolation. In this first part, we will explain why isolation is so important in Python based on common issues encountered by developers, and then formalize the points of contact that require isolation.

Common issues with Python development environments

If you have done any serious Python development that involves installing Python packages from the PyPI, you may recognize yourself in some of these situations:

your package manager (e.g. apt or Brew) decided to update your operating system Python and broke many command-line tools you were using, some of which you did not even suspect to be using Python. Obviously, all your Python projects were also broken.
after running pip install cool_new_tool, your innocent old_boring_tool stopped working because cool_new_tool upgraded some dependencies that old_boring_tool was using.
pip install cool_library somehow threw an error so you ran it as root with sudo pip install cool_library. Then, pip install started throwing permission errors whenever it was used without sudo, which made installing some other dependencies impossible. Moreover, you were at a serious security risk because you were downloading random packages from the Internet and letting them run as root.
you and your coworker were working on the same machine on different accounts and different projects. Your coworker upgraded a Python package to assist his own project, and your project broke.
you were still maintaining legacy_project in Python 2, but the new_cool_project based on Python 3 popped up at your company and you had to work on both at the same time. While working on new_cool_project, you accidentally ran pip install insted of pip3 install and you screwed up the development environment of legacy_project.

What is the common problem to all these situations? The answer is the lack of isolation. That is, whenever you make a modification anywhere inside the Python ecosystem of your machine, you are taking the risk to set off a fire someplace else.

Points of contact requiring isolation

When looking at the common issues described in the previous section, we can identify important points of contact where isolation should be enforced:

your user <=> other users: the Python interpreters on your regular user account should not be able to affect any other interpreters belonging to other users of the same machine (or the root user). Particularly, they should not write to any root-owned path on the system (e.g. /usr) that is common to all users.
interpeters <=> interpreters: if you are using multiple versions of the Python interpreter (typically Python 2.7 and Python 3.6+), they should be isolated from each other.
tooling <=> projects: the Python tools you are using in your daily development such as flake8 or black should be isolated from each other and from your Python projects under development. Moreover, they should be installed only once per user, and not once per project.
projects <=> projects: multiple Python projects under development should be isolated from each other.

To enforce isolation for each of these points of contact, tools have been developed by the Python community:

pyenv addresses 1. and 2. by managing your Python environments and interpreter versions inside a directory in your $HOME.
pipx addresses 3. by installing each tool into a separate directory in your $HOME, including all its dependencies.
pyenv virtualenv addresses 4. by installing each project into a separate directory in your $HOME, including all its dependencies.

Each of these tools will be described in a separate part of this tutorial. Feel free to jump to Part 2 about pyenv if you want to follow the tutorial in order.