Installing Python

What is it? #

Every Linux distribution ships a system Python that operating system tools depend on. Modifying it, or installing packages into it, can break those tools.

The rule is simple: never install project packages into the system Python, and never remove or replace it.

If the system version is recent enough, use it as the base for a virtual environment. If your application needs a newer version, install that alongside from an additional repository.

Each project then gets its own virtual environment, which keeps its packages isolated from the system and from other projects.

Think of it like this #

The building's electrical system runs the lifts and the fire alarm. You do not rewire it to charge your laptop.

You plug into a socket. The virtual environment is the socket: connected to the same supply, isolated from everything that depends on it.

Simple example #

The server has Python 3.10 as the system version and your application needs 3.12. You install 3.12 alongside, create a virtual environment from it under the application directory, and install dependencies there.

Code #

# What is already here?
python3 --version
which python3
ls /usr/bin/python3*        # several versions can coexist

# Essentials for building packages with native extensions
sudo apt install python3-venv python3-dev build-essential libpq-dev

# A newer version alongside the system one (Ubuntu)
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.12 python3.12-venv python3.12-dev

python3.12 --version
# The system python3 is untouched and still points at the distribution version.

# A virtual environment owned by the application user
sudo -u appuser python3.12 -m venv /srv/app/.venv
sudo -u appuser /srv/app/.venv/bin/pip install --upgrade pip
sudo -u appuser /srv/app/.venv/bin/pip install -r /srv/app/requirements.txt

# Absolute paths, so no activation step is needed in scripts or units
/srv/app/.venv/bin/python --version
/srv/app/.venv/bin/gunicorn --version

# systemd uses the venv binary directly — no activation, no shell profile
[Service]
User=appuser
WorkingDirectory=/srv/app
ExecStart=/srv/app/.venv/bin/gunicorn app.main:app --workers 4 --bind 127.0.0.1:8000
Environment=PYTHONUNBUFFERED=1

Things that break the system Python

sudo pip install <anything>        installs into the system interpreter
sudo apt remove python3            removes tools that depend on it
replacing /usr/bin/python3         breaks apt itself on some distributions

Modern distributions actively refuse a pip install outside a virtual
environment for exactly this reason.

How it works #

Distribution tools are written against the system Python and its distribution-packaged libraries. Installing or upgrading packages into it with pip can replace a library with an incompatible version and break apt itself.

Installing 3.12 alongside adds a separate interpreter at /usr/bin/python3.12. The python3 symlink still points at the distribution version, so nothing that depends on it changes.

python3.12 -m venv creates an environment containing its own interpreter and site-packages. Everything installed there is isolated.

Activation is a convenience for interactive shells. Scripts and systemd units should use the absolute path to the environment's binary, because systemd does not load shell profiles and activation would never happen.

Creating the environment as the application user means the files are owned correctly from the start, avoiding permission problems when the service later writes or when deployments update dependencies.

build-essential and the development headers are needed for packages with native extensions such as database drivers. Their absence produces long compilation error messages that are confusing until you recognise them.

Newer distributions mark the system Python as externally managed and refuse pip installs into it, which is the packaging ecosystem enforcing the rule described here.

Real-world use #

Every Python deployment uses a virtual environment. The alternative — system-wide packages — fails as soon as two applications need different versions of anything.

Requirements should be pinned with exact versions, produced by pip freeze from a working environment or by a tool such as pip-tools. Unpinned requirements mean a deployment can pick up a breaking release without any change on your side.

Rebuilding the environment from scratch on each deployment is common and takes seconds for most projects. It guarantees the environment matches the requirements file rather than accumulating drift.

Tools such as Poetry and uv manage environments and lockfiles with better ergonomics. They are worth adopting, and the underlying principle is identical.

Containers make the whole question easier by pinning the interpreter in the image, which is again an argument for the Docker track.

Common mistakes #

• Running sudo pip install and breaking distribution tools.
• Removing or replacing the system Python.
• Relying on shell activation in scripts and systemd units.
• Creating the virtual environment as root, then hitting permission errors.
• Unpinned requirements, so an upstream release breaks the next deployment.

Practice #

Install a Python version newer than the system one, create a virtual environment for an application under a service account, and install pinned requirements into it. Then write the systemd ExecStart line using absolute paths and confirm it runs without activation.