Package Indexes¶
This guide explains how to work with Python package indexes in Pipenv, including using alternative indexes, private repositories, and security considerations.
Understanding Package Indexes¶
A package index is a repository of Python packages that can be installed using pip or Pipenv. The default and most widely used index is the Python Package Index (PyPI), but you may need to use alternative or additional indexes in certain situations.
Configuring Package Sources¶
Default PyPI Source¶
By default, Pipenv uses PyPI as the source for packages. This is defined in your Pipfile
:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
Adding Additional Sources¶
You can add additional package sources to your Pipfile
:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "https://custom-index.example.com/simple"
verify_ssl = true
name = "custom"
Each source must have:
A unique
name
identifierA valid
url
pointing to a package repositoryA
verify_ssl
setting (set tofalse
only if absolutely necessary)
Index-Restricted Packages¶
Starting with Pipenv version 2022.3.23
, all packages are mapped to a single package index for security reasons. This means that each package must be explicitly associated with a specific index.
Specifying Package Index¶
To install a package from a specific index, you must match the name of the index:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "https://download.pytorch.org/whl/cu113/"
verify_ssl = true
name = "pytorch"
[packages]
requests = "*" # From default PyPI index
torch = {version = "*", index = "pytorch"} # From PyTorch index
Installing from a Specific Index¶
You can specify the index when installing a package:
$ pipenv install torch --index=pytorch
You can also specify the index by URL, and Pipenv will add it to the Pipfile
with a generated name (or reuse an existing name if the URL already exists):
$ pipenv install torch --index=https://download.pytorch.org/whl/cu113/
Note
In prior versions of Pipenv, you could use --extra-index-urls
to search multiple indexes without specifying which package came from which index. This functionality was deprecated in favor of index-restricted packages for security reasons.
Security Considerations¶
Dependency Confusion Attacks¶
The index restriction feature was implemented to protect against dependency confusion attacks. This type of attack occurs when:
A private package is hosted on a private index
An attacker publishes a malicious package with the same name on a public index
The package manager searches the public index before the private one
The malicious package is installed instead of the legitimate private package
By requiring explicit index specification for each package, Pipenv prevents this attack vector.
Using Private Indexes Securely¶
When using private package repositories:
Always use HTTPS URLs with SSL verification enabled
Use environment variables for authentication credentials
Explicitly specify which packages come from which index
Consider using a private mirror of PyPI that includes both public and private packages
Using PyPI Mirrors¶
If you need to use a mirror of PyPI (for example, due to network restrictions or performance reasons), you can use the --pypi-mirror
option:
$ pipenv install --pypi-mirror https://mirrors.aliyun.com/pypi/simple/
This will replace the default PyPI URL with the specified mirror for that command.
You can also set a default mirror using an environment variable:
$ export PIPENV_PYPI_MIRROR=https://mirrors.aliyun.com/pypi/simple/
$ pipenv install requests
Alternative Default Index¶
If you want to use an alternative index as your default (instead of PyPI), simply omit PyPI from your sources:
[[source]]
url = "https://custom-index.example.com/simple"
verify_ssl = true
name = "custom"
Warning
When omitting PyPI, your alternative index must contain all the packages you need, including all dependencies. If a package or dependency is not available on your custom index, installation will fail.
Working with Private Repositories¶
Basic Authentication¶
For private repositories that require authentication, you can include credentials in the URL:
[[source]]
url = "https://username:password@private-repo.example.com/simple"
verify_ssl = true
name = "private"
However, it’s better to use environment variables for credentials:
[[source]]
url = "https://${USERNAME}:${PASSWORD}@private-repo.example.com/simple"
verify_ssl = true
name = "private"
Set the environment variables before running Pipenv:
$ export USERNAME=myuser
$ export PASSWORD=mypassword
$ pipenv install
Token Authentication¶
For repositories that use token authentication:
[[source]]
url = "https://${API_TOKEN}@private-repo.example.com/simple"
verify_ssl = true
name = "private"
Self-Signed Certificates¶
If your private repository uses a self-signed SSL certificate, you have two options:
Add the certificate to your system’s trusted certificates (recommended)
Disable SSL verification (use with caution):
[[source]]
url = "https://private-repo.example.com/simple"
verify_ssl = false
name = "private"
Warning
Disabling SSL verification is a security risk and should only be done in controlled environments where you trust the network and the repository.
Multi-Source Installation¶
The install_search_all_sources
Option¶
In some organizations, not everyone has access to all package sources. For these cases, Pipenv provides an option to search all configured sources when installing from a lock file:
[pipenv]
install_search_all_sources = true
With this option enabled, pipenv install
and pipenv sync
will search all configured sources for packages, but only during installation. The lock file will still require each package to be resolved from a single index.
Note
This feature should be used with caution, as it bypasses some of the security benefits of index-restricted packages. Only use it when necessary and in trusted environments.
Advanced Index Configuration¶
Specifying Index in Pipfile¶
You can specify which packages come from which index directly in your Pipfile
:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "https://private-repo.example.com/simple"
verify_ssl = true
name = "private"
[packages]
requests = "*" # From PyPI
private-package = {version = "*", index = "private"} # From private repo
Using Multiple Private Repositories¶
You can use multiple private repositories in the same project:
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "https://repo1.example.com/simple"
verify_ssl = true
name = "repo1"
[[source]]
url = "https://repo2.example.com/simple"
verify_ssl = true
name = "repo2"
[packages]
requests = "*" # From PyPI
package1 = {version = "*", index = "repo1"} # From repo1
package2 = {version = "*", index = "repo2"} # From repo2
Troubleshooting¶
Package Not Found¶
If a package can’t be found:
Verify the package name is correct
Check that the package exists on the specified index
Ensure you have proper authentication for private repositories
Try installing with verbose output for more information:
$ pipenv install package-name --verbose
Authentication Issues¶
If you’re having authentication problems:
Check that your credentials are correct
Ensure environment variables are properly set
Verify that your credentials have access to the repository
Check for special characters in your credentials that might need URL encoding
SSL Certificate Issues¶
If you encounter SSL certificate errors:
Update your CA certificates
Verify the repository’s SSL certificate is valid
If using a self-signed certificate, add it to your trusted certificates
Only as a last resort, consider disabling SSL verification
Best Practices¶
Use index restrictions for all packages to prevent dependency confusion attacks
Use environment variables for credentials instead of hardcoding them in your
Pipfile
Always enable SSL verification when possible
Specify explicit versions for packages from private repositories to ensure reproducibility
Consider using a private PyPI mirror that includes both public and private packages
Regularly audit your dependencies and their sources for security vulnerabilities
Document your custom indexes in your project’s README or documentation
Conclusion¶
Properly configuring package indexes in Pipenv is essential for security and reproducibility. By understanding how to work with multiple indexes and private repositories, you can safely manage dependencies while protecting your project from supply chain attacks.