Whl python что это
Как установить пакет Python с помощью a.WHL файл?
У меня возникли проблемы с установкой пакета Python (в частности, JPype1 0.5.7) на моей машине Windows, и я хотел бы установить его с двоичными файлами окна Кристофа Голке. (Что, по моему опыту, облегчило большую часть суеты для многих других установок пакетов.)
14 ответов
Я просто использовал следующее, что было довольно просто. Сначала откройте консоль, затем компакт-диск, где вы загрузили свой файл, как какой-то пакет.колесо и использовать
Примечание: если pip.exe не распознается, вы можете найти его в каталоге «Скрипты», откуда был установлен python. Если pip не установлен, эта страница может помочь: Как установить pip на Windows?
чтобы установить с колеса, дайте ему каталог, в который загружается колесо. Например, установить package_name.whl :
убедитесь, что вы обновили pip, чтобы включить поддержку колес:
Я нахожусь в той же лодке, что и ОП.
С помощью командной строки Windows, из каталога:
вы должны запустить pip.exe из командной строки на моем компьютере. Я печатаю C:/Python27/Scripts/pip2.exe install numpy
на сайте великого Кристофа Гольке есть несколько версий файлов.
чтобы вы знали, какую версию вы должны установить для своего компьютера. Выбор неправильной версии может привести к сбою установки пакета (особенно если вы не используете правильный тег CPython, например, cp27).
проверьте pip версия:
если он показывает 6.x серия, поддержка колеса.
только тогда вы можете установить пакет колес, как это:
чтобы иметь возможность устанавливать файлы колес с простым двойным щелчком на них, вы можете сделать следующее:
1) Выполните две команды в командной строке под правами администратора:
2) Кроме того, они могут быть скопированы в wheel.bat file и выполняется с флажком «Запуск от имени администратора»в свойствах.
PS pip.предполагается, что exe находится в пути.
обновление:
(1) Те можно совместить в одном строка:
также его выход можно сделать более подробным:
посмотреть мои блоге для сведения.
чтобы избежать необходимости загружать такие файлы, вы можете попробовать:
единственный способ, которым мне удалось установить NumPy, был следующим:
Модуль
выполнение команды из пути установки Python в PowerShell
PS. Я установил его на Windows 10.
конечно, вы также можете установить пакеты из PyPI таким образом, например,
в случае, если вы не можете установить конкретный пакет напрямую с помощью PIP.
на MacOS, с pip установлен через MacPorts в MacPorts python2.7, мне пришлось использовать решение @Dunes:
здесь python был заменен MacPorts python в моем случае, который является python2.7 или python3.5 для меня.
(я ранее работать sudo port install py27-pip py27-wheel установить pip и wheel сначала в мою установку python 2.7.)
Python на колёсах
Инфраструктура системы пакетов для Python долго подвергалась критике как от разработчиков, так и от системных администраторов. Долгое время даже само комьюнити не могло прийти к соглашению, какие именно инструменты использовать в каждом конкретном случае. Уже существуют distutils, setuptools, distribute, distutils2 в качестве базовых механизмов распространения и virtualenv, buildout, easy_install и pip в качестве высокоуровневых инструментов управления всем этим беспорядком.
Несколько лет назад PJE попытался исправить эту проблему, предоставив смесь из setuptools и pkg_resources для улучшения distutils и добавления метаданных в Python-пакеты. В дополнение к этому он написал утилиту easy_install для их установки. По причине отсутствия формата распространения, поддерживающего метаданные, был предоставлен формат ‘яиц’ [egg].
Python eggs – обычные zip-архивы, содержащие python-пакет и необходимые метаданные. Хотя многие люди, вероятно, никогда намеренно не собирали egg’и, их формат метаданных до сих пор жив-здоров. И все разворачивают свои проекты с использованием setuptools.
Потом прошло еще немного времени, и отказ от бинарных пакетов стал доставлять неудобства. Люди всё больше и больше стали деплоить на облачные сервера, а необходимость перекомпиляции C-шных библиотек на каждой машине не слишком радует. Так как ‘яйца’ на тот момент были малопонятны (я так полагаю), их переделали в новых PEP-ах, и назвали ‘колёсами’ [wheels].
В дальнейшем предполагается, что все действия происходят в virtualenv-окружении.
Что за колесо?
Начнём с простого. Что представляют собой ‘колёса’ и чем они отличаются от ‘яиц’? Оба формата являются zip-файлами. Главная разница в том, что egg можно импортировать без распаковки, wheel же придётся распаковать. Хотя нет никаких технических причин, делающих ‘колёса’ неимпортируемыми, поддержка их прямого импорта никогда даже не планировалась.
Другое различие в том, что ‘яйца’ содержат скомпилированные байткод, а ‘колёса’ – нет. Главное преимущество этого в том, что нет необходимости создавать отдельные wheel’ы для каждой версии Python до тех пор, пока не придётся распространять слинкованные через libpython модули. Хотя в новых версиях Python 3 при использовании стабильного ABI даже это уже можно провернуть.
Однако wheel-формат тоже не лишен проблем, некоторые из которых он наследует от ‘яиц’. Например, бинарные дистрибутивы под Linux до сих пор неприемлемы для большинства из-за двух недостатков: Python сам по себе компилируется под Linux в разных формах, и модули линкуются с разными системными библиотеками. Первая проблема вызвана сосуществованием несовместимых версий Python 2: USC2 и USC4. В зависимости от режима компиляции меняется ABI. В настоящее время wheel (насколько я могу судить) не содержит информации о том, с каким режимом Unicode связана библиотека. Отдельная проблема в том, что дистрибутивы Linux меньше совместимы между собой, чем хотелось бы, и обстоятельства могут сложиться так, что сборка, скомпилированная под один дистрибутив, не будет работать на остальных.
Всё это выливается в то, что, вообще говоря, на данный момент бинарные ‘колёса’ нельзя загружать на PyPI как несовместимые с различными системами.
В дополнение ко всему этому wheel сейчас знает только две крайности: бинарные пакеты и пакеты, содержащие чистый python-код. Бинарные пакеты специфичны для Python ветки 2.x. Сейчас это не кажется большой проблемой, потому что цикл 2.x подходит к концу, и пакетов, собранных только для 2.7, хватит надолго. Но если бы вдруг речь пошла про Python 2.8, была бы интересна возможность заявить, что этот пакет не зависит от версии Python, но он содержит бинарники, поэтому он не может не зависеть от архитектуры.
Единственный случай, оправдывающий существование такого пакета – это когда он содержит распределенные библиотеки, загружаемые с ctypes из CFFI. Такие библиотеки не связаны через libpython и не зависимы от реализации языка (их можно использовать даже с pypy).
Но есть и светлая сторона: ничто не запрещает использовать бинарные wheel’ы в своих собственных однородных инфраструктурах.
Сборка колеса
Решение этой проблемы – вручную реализовать Distribution из setuptools, скинув флаг чистоты в false :
Установка колеса
С использованием свежей версии pip ‘колесо’ ставится следующим образом:
Но что с зависимостями? Тут появляются некоторые сложности. Обычно одним из требований к пакету является возможность его установки даже без подключения к интернету. К счастью, pip позволяет отключать загрузку из индекса и устанавливать директорию, содержащую всё необходимое для установки. Если у нас есть wheel’ы для всех зависимостей необходимых версий, можно сделать следующее:
Таким образом будет установлена версия 1.0 пакета package в наше виртуальное окружение.
Колёса для зависимостей
Эта команда выгрузит все пакеты, от которых зависит наш пакет, в указанную папку. Но есть пара проблем.
Первая состоит в том, что в команде в настоящий момент есть баг, который не выгружает зависимости, которые уже являются ‘колёсами’. Так что если зависимость уже доступна на PyPI в wheel-формате, она не будет загружена.
Это временно решается shell-скриптом, который вручную перемещает из кэша скачанные wheel’ы.
Сборка пакетов c использованием DevPI
Такое временное решение проблемы зависимостей вполне применимо в простых ситуациях, но что делать, если есть множество внутренних python-пакетов, зависящих друг от друга? Такая конструкция быстро разваливается.
К счастью, в прошлом году Holker Krekel создал решение этой беды под названием DevPI, который по существу является хаком, эмулирующим работу pip с PyPI. После установки на компьютер DevPI работает как прозрачный прокси перед PyPI и позволяет pip-у устанавливать пакеты из локального репозитория. К тому же все пакеты, скачанные с PyPI, автоматически кэшируются, так что даже если отключить сеть, эти пакеты будут доступны для установки. И, в конце концов, появляется возможность загрузки своих собственных пакетов на локальный сервер, чтобы ссылаться на них так же, как и на хранящиеся в публичном индексе.
После запуска его необходимо единожды проинициализировать:
Так как я использую DevPI ‘для себя’, имена пользователя DevPI и системного пользователя совпадают. На последнем шаге создаётся индекс по имени проекта (при необходимости можно создать несколько).
Для перенаправления pip на локальный репозиторий можно сделать экспорт переменной окружения:
Я размешаю эту команду в скрипт postactivate моего virtualenv для предотвращения случайной загрузки из неверного индекса.
Для размещения собственных wheel’ов в локальном DevPI используется утилита devpi :
Заворачиваем
Теперь всё готово для начала использования внутренних зависимостей и сборки собственных ‘колёс’. Как только они появятся, их можно заархивировать, загрузить на другой сервер и установить в отдельный virtualenv.
Весь процесс станет чуть проще, когда pip wheel перестанет игнорировать существующие wheel-пакеты. А пока приведенный выше shell-скрипт – не худшее решение.
В сравнении с ‘яйцами’
Сейчас wheel-формат более притягателен, чем egg. Его разработка активнее, PyPI начал добавлять его поддержку и, так как с ним начинают работать утилиты, он похож на лучшее решение. ‘Яйца’ пока что поддерживаются только easy_install, хотя большинство давно перешло на pip.
Итак, теперь он у вас есть. Python на колёсах. И это вроде даже работает, и, возможно, стоит потраченного времени.
What Are Python Wheels and Why Should You Care?
Wheels are a component of the Python ecosystem that helps to make package installs just work. They allow for faster installations and more stability in the package distribution process. In this tutorial, you’ll dive into what wheels are, what good they serve, and how they’ve gained traction and made Python even more of a joy to work with.
In this tutorial, you’ll learn:
You’ll see examples using popular open source Python packages from both the user’s and the developer’s perspective.
Free Bonus: Click here to get a Python Cheat Sheet and learn the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.
Setup
That’s all you need to experiment with installing and building wheels!
Python Packaging Made Better: An Intro to Python Wheels
Before you learn how to package a project into a wheel, it helps to know what using one looks like from the user’s side. It may sound backward, but a good way to learn how wheels work is to start by installing something that isn’t a wheel.
You can start this experiment by installing a Python package into your environment just as you might normally do. In this case, install uWSGI version 2.0.x:
To fully install uWSGI, pip progresses through several distinct steps:
Note: If you see an error with the uWSGI installation, you may need to install the Python development headers.
A source distribution contains source code. That includes not only Python code but also the source code of any extension modules (usually in C or C++) bundled with the package. With source distributions, extension modules are compiled on the user’s side rather than the developer’s.
Source distributions also contain a bundle of metadata sitting in a directory called
From the developer’s perspective, a source distribution is what gets created when you run the following command:
Now try installing a different package, chardet:
You can see a noticeably different output than the uWSGI install.
From the developer’s side, a wheel is the result of running the following command:
Why does uWSGI hand you a source distribution while chardet provides a wheel? You can see the reason for this by taking a look at each project’s page on PyPI and navigating to the Download files area. This section will show you what pip actually sees on the PyPI index server:
To avoid these types of compatibility issues, some packages offer multiple wheels, with each wheel geared toward a specific Python implementation and underlying operating system.
Wheels Make Things Go Fast
It’s almost guaranteed that the chardet install occurred in a fraction of the time required for uWSGI. However, that’s arguably an unfair apples-to-oranges comparison since chardet is a significantly smaller and less complex package. With a different command, you can create a more direct comparison that will demonstrate just how much of a difference wheels make.
This command times the installation of the cryptography package, telling pip to use a source distribution even if a suitable wheel is available. Including :all: makes the rule apply to cryptography and all of its dependencies.
On my machine, this takes around thirty-two seconds from start to finish. Not only does the install take a long time, but building cryptography also requires that you have the OpenSSL development headers present and available to Python.
This option takes just over four seconds, or one-eighth the time that it took when using only source distributions for cryptography and its dependencies.
What Is a Python Wheel?
A wheel is a type of built distribution. In this case, built means that the wheel comes in a ready-to-install format and allows you to skip the build stage required with source distributions.
A wheel filename is broken down into parts separated by hyphens:
Each section in
Here’s an illustrative example using a cryptography wheel:
cryptography distributes multiple wheels. Each wheel is a platform wheel, meaning it supports only specific combinations of Python versions, Python ABIs, operating systems, and machine architectures. You can break down the naming convention into parts:
cryptography is the package name.
cp35 is the Python tag and denotes the Python implementation and version that the wheel demands. The cp stands for CPython, the reference implementation of Python, while the 35 denotes Python 3.5. This wheel wouldn’t be compatible with Jython, for instance.
abi3 is the ABI tag. ABI stands for application binary interface. You don’t really need to worry about what it entails, but abi3 is a separate version for the binary compatibility of the Python C API.
macosx_10_9_x86_64 is the platform tag, which happens to be quite a mouthful. In this case it can be broken down further into sub-parts:
Now let’s turn to a different example. Here’s what you saw in the above case for chardet:
You can break this down into its tags:
Another example is the jinja2 templating engine. If you navigate to the downloads page for the Jinja 3.x alpha release, then you’ll see the following wheel:
Notice the lack of py2 here. This is a pure-Python project that will work on any Python 3.x version, but it’s not a universal wheel because it doesn’t support Python 2. Instead, it’s called a pure-Python wheel.
Note: In 2020, a number of projects are also dropping support for Python 2, which reached end-of-life (EOL) on January 1, 2020. Jinja version 3.x dropped Python 2 support in February 2020.
Wheel | What It Is |
---|---|
PyYAML-5.3.1-cp38-cp38-win_amd64.whl | PyYAML for CPython 3.8 on Windows with AMD64 (x86-64) architecture |
numpy-1.18.4-cp38-cp38-win32.whl | NumPy for CPython 3.8 on Windows 32-bit |
scipy-1.4.1-cp36-cp36m-macosx_10_6_intel.whl | SciPy for CPython 3.6 on macOS 10.6 SDK with fat binary (multiple instruction sets) |
Now that you have a thorough understanding of what wheels are, it’s time to talk about what good they serve.
Advantages of Python Wheels
Here’s a testament to wheels from the Python Packaging Authority (PyPA):
A fuller description is that wheels benefit both users and maintainers of Python packages alike in a handful of ways:
Wheels install faster than source distributions for both pure-Python packages and extension modules.
Wheels are smaller than source distributions. For example, the six wheel is about one-third the size of the corresponding source distribution. This differential becomes even more important when you consider that a pip install for a single package may actually kick off downloading a chain of dependencies.
There’s no need for a compiler to install wheels that contain compiled extension modules. The extension module comes included with the wheel targeting a specific platform and Python version.
Wheels provide consistency by cutting many of the variables involved in installing a package out of the equation.
You can use a project’s Download files tab on PyPI to view the different distributions that are available. For example, pandas distributes a wide array of wheels.
Telling pip What to Download
You download the six module with several flags:
six is a special case: it’s actually a single Python module rather than a complete package. Wheel files can also be significantly more complex, as you’ll see later on.
The corresponding wheel is broken. This is an irony of wheels. They’re designed to make things break less often, but in some cases a wheel can be misconfigured. In this case, downloading and building the source distribution for yourself may be a working alternative.
You want to apply a small change or patch file to the project and then install it. This is an alternative to cloning the project from its version control system URL.
In this section, you got a glimpse of how to fine-tune the distribution types that pip install will use. While a regular pip install should work with no options, it’s helpful to know these options for special cases.
The manylinux Wheel Tag
If you’re writing a C/C++ extension, then this could create a problem. A source file written in C and compiled on Ubuntu Linux isn’t guaranteed to be executable on a CentOS machine or an Arch Linux distribution. Do you need to build a separate wheel for each and every Linux variant?
Luckily, the answer is no, thanks to a specially designed set of tags called the manylinux platform tag family. There are currently three variations:
manylinux1 is the original format specified in PEP 513.
manylinux2010 is an update specified in PEP 571 that upgrades to CentOS 6 as the underlying OS on which the Docker images are based. The rationale is that CentOS 5.11, which is where the list of allowed libraries in manylinux1 comes from, reached EOL in March 2017 and stopped receiving security patches and bug fixes.
manylinux2014 is an update specified in PEP 599 that upgrades to CentOS 7 since CentOS 6 is scheduled to reach EOL in November 2020.
You can find an example of manylinux distributions within the pandas project. Here are two (out of many) from the list of available pandas downloads from PyPI:
In this case, pandas has built manylinux1 wheels for CPython 3.7 supporting both x86-64 and i686 architectures.
Note: The term allowed indicates a low-level library that is assumed to be present by default on almost all Linux systems. The idea is that the dependency should exist on the base operating system without the need for an additional install.
As of mid-2020, manylinux1 is still the predominant manylinux tag. One reason for this might just be habit. Another might be that support on the client (user) side for manylinux2010 and above is limited to more recent versions of pip :
Tag | Requirement |
---|---|
manylinux1 | pip 8.1.0 or later |
manylinux2010 | pip 19.0 or later |
manylinux2014 | pip 19.3 or later |
In other words, if you’re a package developer building manylinux2010 wheels, then someone using your package will need pip 19.0 (released in January 2019) or later to let pip find and install manylinux2010 wheels from PyPI.
All that is to say, if you’re installing Python packages on a Linux host, then consider yourself fortunate if the package maintainer has gone out of their way to create manylinux wheels. This will almost guarantee a hassle-free installation of the package regardless of your specific Linux variant or version.
Security Considerations With Platform Wheels
One feature of wheels worth considering from a user security standpoint is that wheels are potentially subject to version rot because they bundle a binary dependency rather than allowing that dependency to be updated by your system package manager.
If you’re developing in an environment with heightened security precautions, this feature of some platform wheels is something to be mindful of.
Calling All Developers: Build Your Wheels
The title of this tutorial asks, “Why Should You Care?” As a developer, if you plan to distribute a Python package to the community, then you should care immensely about distributing wheels for your project because they make the installation process cleaner and less complex for end users.
The more target platforms that you can support with compatible wheels, the fewer GitHub issues you’ll see titled something like “Installation broken on Platform XYZ.” Distributing wheels for your Python package makes it objectively less likely that users of the package will encounter issues during installation.
The next few sections will walk you through building wheels for a variety of different scenarios.
Different Types of Wheels
As touched on throughout this tutorial, there are several different variations of wheels, and the wheel’s type is reflected in its filename:
A platform wheel supports a specific Python version and platform. It contains segments indicating a specific Python version, ABI, operating system, or architecture.
The differences between wheel types are determined by which version(s) of Python they support and whether they target a specific platform. Here’s a condensed summary of the differences between wheel variations:
Wheel Type | Supports Python 2 and 3 | Supports Every ABI, OS, and Platform |
---|---|---|
Universal | ✓ | ✓ |
Pure-Python | ✓ | |
Platform |
As you’ll see next, you can build universal wheels and pure-Python wheels with relatively little setup, but platform wheels may require a few additional steps.
Building a Pure-Python Wheel
You can build a pure-Python wheel or a universal wheel for any project using setuptools with just a single command:
Here’s the result of building both types of distributions for the HTTPie package:
You can combine the sdist and bdist_wheel steps into one because setup.py can take multiple subcommands:
Specifying a Universal Wheel
A universal wheel is a wheel for a pure-Python project that supports both Python 2 and 3. There are multiple ways to tell setuptools and distutils that a wheel should be universal.
Option 1 is to specify the option in your project’s setup.cfg file:
Option 3 is to tell setup() itself about the flag using its options parameter:
While any of these three options should work, the first two are used most frequently. You can see an example of this in the chardet setup configuration. After that, you can use the bdist_wheel command as shown previously:
The resulting wheel will be equivalent no matter which option you choose. The choice largely comes down to developer preference and which workflow is best for you.
Building a Platform Wheel (macOS and Windows)
Binary distributions are a subset of built distributions that contain compiled extensions. Extensions are non-Python dependencies or components of your Python package.
Usually, that means your package contains an extension module or depends on a library written in a statically typed language such as C, C++, Fortran, or even Rust or Go. Platform wheels exist to target individual platforms primarily because they contain or depend on extension modules.
With all that said, it’s high time to build a platform wheel!
Depending on your existing development environment, you may need to go through an additional prerequisite step or two to build platform wheels. The steps below will help you to get set up for building C and C++ extension modules, which are by far the most common types.
On macOS, you’ll need the command-line developer tools available through xcode :
On Windows, you’ll need to install Microsoft Visual C++:
With that squared away, you’re ready to build a platform wheel for UltraJSON ( ujson ), a JSON encoder and decoder written in pure C with Python 3 bindings. Using ujson is a good toy example because it covers a few bases:
) but is not otherwise overly complicated. ujson is designed to do one thing and do it well, which is to read and write JSON!
You can clone the project from GitHub, navigate into its directory, and build it:
You should see a whole lot of output. Here’s a trimmed version on macOS, where the Clang compiler driver is used:
The lines starting with clang show the actual call to the compiler complete with a trove of compilation flags. You might also see tools such as MSVC (Windows) or gcc (Linux) depending on the operating system.
If you run into a fatal error after executing the above code, don’t worry. You can expand the box below to learn how to deal with this problem.
Fixing «fatal error: Python.h file not found» Show/Hide
This setup.py bdist_wheel call for ujson requires the Python development header files because ujson.c pulls in
. If you don’t have them in a searchable location, then you may see an error such as this one:
To compile extension modules, you’ll need the development headers saved somewhere your compiler can find them.
If not, then you may see an error indicating that a header file can’t be found:
This tells you that Python development headers are located in the directory shown, which you can now use with python setup.py bdist_wheel :
More generally, you can pass any path you need:
On Linux, you may also need to install the headers separately:
Note that the name may vary based on your platform. For example, you’d see win_amd64.whl on 64-bit Windows.
You can peek into the wheel file and see that it contains the compiled extension:
Linux: Building manylinux Wheels
As a package developer, you’ll rarely want to build wheels for a single Linux variant. Linux wheels demand a specialized set of conventions and tools so that they can work across different Linux environments.
Unlike wheels for macOS and Windows, wheels built on one Linux variant have no guarantee of working on another Linux variant, even one with the same machine architecture. In fact, if you build a wheel on an out-of-the-box Linux container, then PyPI won’t even accept that wheel if you try to upload it!
If you want your package to be available across a range of Linux clients, then you want a manylinux wheel. A manylinux wheel is a particular type of a platform wheel that is accepted by most Linux variants. It must be built in a specific environment, and it requires a tool called auditwheel that renames the wheel file to indicate that it’s a manylinux wheel.
Note: Even if you’re approaching this tutorial from the developer rather than the user perspective, make sure that you’ve read the section on the manylinux wheel tag before continuing with this section.
Building a manylinux wheel allows you to target a wider range of user platforms. PEP 513 specifies a particular (and archaic) version of CentOS with an array of Python versions available. The choice between CentOS and Ubuntu or any other distribution doesn’t carry any special distinction. The point is for the build environment to consist of a stock Linux operating system with a limited set of external shared libraries that are common to different Linux variants.
Thankfully, you don’t have to do this yourself. PyPA provides a set of Docker images that give you this environment with a few mouse clicks:
The Docker images are provided for the different manylinux flavors:
manylinux Tag | Architecture | Docker Image |
---|---|---|
manylinux1 | x86-64 | quay.io/pypa/manylinux1_x86_64 |
manylinux1 | i686 | quay.io/pypa/manylinux1_i686 |
manylinux2010 | x86-64 | quay.io/pypa/manylinux2010_x86_64 |
manylinux2010 | i686 | quay.io/pypa/manylinux2010_i686 |
manylinux2014 | x86-64 | quay.io/pypa/manylinux2014_x86_64 |
manylinux2014 | i686 | quay.io/pypa/manylinux2014_i686 |
manylinux2014 | aarch64 | quay.io/pypa/manylinux2014_aarch64 |
manylinux2014 | ppc64le | quay.io/pypa/manylinux2014_ppc64le |
manylinux2014 | s390x | quay.io/pypa/manylinux2014_s390x |
To get started, PyPA also provides an example repository, python-manylinux-demo, which is a demo project for building manylinux wheels in conjunction with Travis-CI.
While it’s common to build wheels as a part of a remote-hosted CI solution, you can also build manylinux wheels locally. To do so, you’ll need Docker installed. Docker Desktop is available for macOS, Windows, and Linux.
First, clone the demo project:
Next, define a few shell variables for the manylinux1 Docker image and platform, respectively:
Now you can pull the Docker image and run the wheel-builder script within the container:
In a few short commands, you have a set of manylinux1 wheels for CPython 2.7 through 3.8. A common practice is also to iterate over different architectures. For instance, you could repeat this process for the quay.io/pypa/manylinux1_i686 Docker image. This would build manylinux1 wheels targeting 32-bit (i686) architecture.
If you’d like to dive deeper into building wheels, then a good next step is to learn from the best. Start at the Python Wheels page, pick a project, navigate to its source code (on a place like GitHub, GitLab, or Bitbucket), and see for yourself how it builds wheels.
Many of the projects on the Python Wheels page are pure-Python projects and distribute universal wheels. If you’re looking for more complex cases, then keep an eye out for packages that use extension modules. Here are two examples to whet your appetite:
Both of these are reputable projects that offer a great examples to learn from if you’re interested in building manylinux wheels.
Bundling Shared Libraries
In that case, several solutions come to the rescue:
Conveniently, auditwheel is present on the manylinux Docker images. Using auditwheel and delocate takes just one command. Just tell them about the wheel file(s) and they’ll do the rest:
This will detect the needed external libraries through your project’s setup.py and bundle them in to the wheel as if they were part of the project.
auditwheel and delocate know to include libprotobuf because setup.py tells them to through the libraries argument:
This means that auditwheel and delocate save users the trouble of installing protobuf as long as they’re installing from a platform and Python combination that has a matching wheel.
If you’re distributing a package that has external dependencies like this, then you can do your users a favor by using auditwheel or delocate to save them the extra step of installing the dependencies themselves.
Building Wheels in Continuous Integration
An alternative to building wheels on your local machine is to build them automatically within your project’s CI pipeline.
There are myriad CI solutions that integrate with the major code hosting services. Among them are Appveyor, Azure DevOps, BitBucket Pipelines, Circle CI, GitLab, GitHub Actions, Jenkins, and Travis CI, to name just a few.
The purpose of this tutorial isn’t to render a judgment as to which CI service is best for building wheels, and any listing of which CI services support which containers would quickly become outdated given the speed at which CI support is evolving. However, this section can help to get you started.
If you’re developing a pure-Python package, the bdist_wheel step is a blissful one-liner: it’s largely irrelevant which container OS and platform you build the wheel on. Virtually all major CI services should enable you to do this in a no-frills fashion by defining steps in a special YAML file within the project.
For example, here’s the syntax you could use with GitHub Actions:
In this configuration file, you build a wheel using the following steps:
However, if you have a complex project (maybe one with C extensions or Cython code) and you’re working to craft a CI/CD pipeline to automatically build wheels, then there will likely be additional steps involved. Here are a few projects through which you can learn by example:
Many projects roll their own CI configuration. However, some solutions have emerged for reducing the amount of code specified in configuration files to build wheels. You can use the cibuildwheel tool directly on your CI server to cut down on the lines of code and configuration that it takes to build multiple platform wheels. There’s also multibuild, which provides a set of shell scripts for assisting with building wheels on Travis CI and AppVeyor.
Making Sure Your Wheels Spin Right
One check that you can use after bdist_wheel is the check-wheel-contents tool. It looks for common problems such as the package directory having an abnormal structure or the presence of duplicate files:
Another way to confirm that the wheel you’ve built has the right stuff is to use TestPyPI. First, you can upload the package there:
Then, you can download the same package for testing as if it were the real thing:
This allows you to test your wheel by uploading and then downloading your own project.
Uploading Python Wheels to PyPI
Now it’s time to upload your Python package. Since a sdist and a wheel both get put in the dist/ directory by default, you can upload them both using the twine tool, which is a utility for publishing packages to PyPI:
Since both sdist and bdist_wheel output to dist/ by default, you can safely tell twine to upload everything under dist/ using a shell wildcard ( dist/* ).
Conclusion
Understanding the pivotal role that wheels play in the Python ecosystem can make your life easier as both a user and developer of Python packages. Furthermore, increasing your Python literacy when it comes to wheels will help you to better understand what’s happening when you install a package and when, in increasingly rare cases, that operation goes awry.
In this tutorial, you learned:
You now have a solid understanding of wheels from both a user’s and a developer’s perspective. You’re well equipped to build you own wheels and make your project’s installation process quick, convenient, and stable.
See the section below for some additional reading to dive deeper into the rapidly-expanding wheel ecosystem.
Resources
The Python Wheels page is dedicated to tracking support for wheels among the 360 most downloaded packages on PyPI. The adoption rate is pretty respectable at the time of this tutorial, at 331 out of 360, or around 91 percent.
There have been a number of Python Enhancement Proposals (PEPs) that have helped with the specification and evolution of the wheel format:
Here’s a shortlist of the various wheel packaging tools mentioned in this tutorial:
The Python documentation has several articles covering wheels and source distributions:
Finally, here are a few more useful links from PyPA:
Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.
About Brad Solomon
Brad is a software engineer and a member of the Real Python Tutorial Team.
Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:
Master Real-World Python Skills With Unlimited Access to Real Python
Join us and get access to hundreds of tutorials, hands-on video courses, and a community of expert Pythonistas:
Master Real-World Python Skills
With Unlimited Access to Real Python
Join us and get access to hundreds of tutorials, hands-on video courses, and a community of expert Pythonistas:
Real Python Comment Policy: The most useful comments are those written with the goal of learning from or helping out other readers—after reading the whole article and all the earlier comments. Complaints and insults generally won’t make the cut here.
What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.
Related Tutorial Categories: intermediate python