Project:Python/distutils-r1

The distutils-r1 eclass is the modern eclass aimed at packages using distutils, setuptools or a compatible build system.

Description

The distutils-r1 eclass is suited for installing packages that use the distutils build system. It provides metadata, environment setup, and phase functions suited for this task. The Python dependencies, REQUIRED_USE and phase functions are exported by default. If your package has only USE-conditional use for distutils, you can disable this using DISTUTILS_OPTIONAL.

distutils-r1 eclass can use either of two backends. By default, the python-r1 eclass is used as a backend, providing possibility to build the package against multiple Python implementations. However, DISTUTILS_SINGLE_IMPL may be set to enforce building for a single implementation only, and in this case python-single-r1 eclass is used.

The ebuilds can safely assume that distutils-r1 eclass will inherit either of the above eclasses. To ease switching, developers are encouraged to rely only upon the common parts of their API rather than individual functions. Similarly to the both backend eclasses, one needs to set PYTHON_COMPAT and possibly PYTHON_REQ_USE.

All of ebuild's operations should be done in sub-phases as described in the phase functions section. Overriding standard phase functions is discouraged since it doesn't provide proper Python build environment and should be only used when absolutely necessary.

The default sub-phases perform a standard build and install using setup.py. By default, an out-of-source build is performed using a shared copy of sources; this can be disabled using DISTUTILS_IN_SOURCE_BUILD.

Phase functions

As a build system eclass, distutils-r1 provides implementations for most of phase functions. Each of the phases is split into two sub-phases as outlined in the following table:

Phase function	Per-implementation sub-phase	Common sub-phase
src_prepare	python_prepare	python_prepare_all (*)
src_configure	python_configure	python_configure_all
src_compile	python_compile (*)	python_compile_all
src_test	python_test	python_test_all
src_install	python_install (*)	python_install_all (*)

The distutils-r1 standard phase functions just call sub-phases appropriately. The per-implementation sub-phases are called first, with the call being repeated for each of the enabled Python implementations. Then, the common sub-phase is called once, with the build environment of last enabled Python implementation.

In other words, the per-implementation sub-phases such as python_compile are used to perform tasks that need to be repeated for each of the enabled implementations, such as building and installing the Python modules. The common sub-phases such as python_compile_all are used to perform tasks that need being done only once such as building and installing documentation.

Each sub-phase is run with complete Python build environment, having EPYTHON, PYTHON, BUILD_DIR, and PYTHONPATH values set properly for distutils build, and Python executables and pkg-config files wrapped. Sub-phases are run in the source directory (${S}).

Similarly to regular phase functions, some of the sub-phases have default implementations. The default implementations are named alike phase functions in eclasses. For example, the default for python_compile is distutils-r1_python_compile. The sub-phases which have default implementation have been highlighted and marked with an asterisk in the phase function table.

API reference

Pre-inherit variables set by ebuilds

PYTHON_COMPAT

Obligatory. Must be set above the inherit line.

A list of all Python implementations supported by ebuild. This variable is described more thoroughly in the PYTHON_COMPAT article.

CODE Example PYTHON_COMPAT

PYTHON_COMPAT=( python{2_6,2_7,3_2,3_3} pypy2_0 )

PYTHON_REQ_USE

Note
Please do not confuse this with PYTHON_REQUIRED_USE. REQ stands for requested.

Optional, defaults to none. Must be set above the inherit line.

USE-dependency that will be applied to all Python interpreters pulled in as dependencies. Takes the form of EAPI 4 USE dependency string, must apply cleanly to all supported Python implementations. See Project:Python/Implementation USE flags for reference.

CODE Example for PYTHON_REQ_USE

PYTHON_REQ_USE="ssl(+)?,xml(+)"

DISTUTILS_SINGLE_IMPL

Boolean. Enabled if non-empty, defaults to disabled. Must be set above the inherit line.

Enables the single implementation mode. If enabled, the eclass will use python-single-r1 backend. The package will be built with support for a single implementation only but it will be able to depend on libraries that do not support multiple implementations. Otherwise (the default), the eclass uses python-r1 backend and the package is built with support for multiple implementations of choice.

Note
DISTUTILS_SINGLE_IMPL should be used when there's no possibility of supporting multiple implementations; otherwise, it generally shouldn't be used.

DISTUTILS_OPTIONAL

Boolean. Enabled if non-empty, defaults to disabled. Must be set above the inherit line.

Enables optional distutils mode. In this mode, no phase functions are exported, dependencies and REQUIRED_USE are not set by default. User is expected to define his own phase functions and call the eclass implementations properly, and to use the PYTHON_DEPS and PYTHON_REQUIRED_USE variables.

CODE Example snippet using DISTUTILS_OPTIONAL

DISTUTILS_OPTIONAL=1
inherit distutils-r1

IUSE="python"
RDEPEND="python? ( ${PYTHON_DEPS} )"
DEPEND="${RDEPEND}"
REQUIRED_USE="python? ( ${PYTHON_REQUIRED_USE} )"

src_prepare() {
    #...
    if use python; then
        pushd bindings/python >/dev/null || die
        distutils-r1_src_prepare
        popd >/dev/null || die
    fi
}

Other global variables set by ebuilds

DISTUTILS_ALL_SUBPHASE_IMPLS

Array. Defaults to unset/empty.

A list of wildcards restricting allowed implementations in *_all() sub-phases. If unset, all implementations are allowed. Otherwise, the *_all() phases will be run using one of the enabled implementations matching one of the patterns.

The value of this variable usually needs to be set in a regular phase function, preferably before any of the *_all() implementations are called (e.g. in pkg_setup). If the value changes at run-time, the following *_all() sub-phases may be called with a different Python interpreter.

Please remember to set appropriate REQUIRED_USE constraints before using DISTUTILS_ALL_SUBPHASE_IMPLS. Otherwise, the build may fail being unable to find a matching Python implementation.

CODE Example use of DISTUTILS_ALL_SUBPHASE_IMPLS

REQUIRED_USE="doc? ( || ( $(python_gen_useflags 'python2*' pypy) ) )"

pkg_setup() {
    use doc && DISTUTILS_ALL_SUBPHASE_IMPLS=( 'python2*' pypy )
}

DISTUTILS_IN_SOURCE_BUILD

Boolean. Enabled if non-empty, defaults to disabled.

Enables in-source builds. In this mode, a separate copy of package sources is created for each of the enabled Python implementations and the build is performed within this copy. Otherwise (the default), an out-of-source build is performed, with sources being shared by all builds and output being placed in separate build trees.

Use as last resort only. If an out-of-source build fails, it is recommended to find the underlying issue and fix it instead.

PATCHES

Array. Defaults to unset/empty.

A list of patches that will be applied by the distutils-r1_python_prepare_all function. The patches need to be specified by full path. If unspecified, no ebuild-specific patches are applied.

CODE Example PATCHES setting

python_prepare_all() {
    local PATCHES=(
        "${FILESDIR}"/${P}-foo.patch
        "${FILESDIR}"/${P}-bar.patch
    )

    distutils-r1_python_prepare_all
}

DOCS

Array or scalar. Defaults to unset.

A list of documentation files that will be installed by distutils-r1_python_install_all. The files can be specified using absolute paths or paths relative to source directory (${S}).

If unset, the implementation will find documentation files matching standard names (as described in EAPI 4 src_install) and install them. If set to an empty list, no documentation will be installed. Setting it to any value disables the automatic search.

CODE Example DOCS setting

DOCS=( README NEWS )

HTML_DOCS

Array or scalar. Defaults to unset/empty.

A list of HTML documentation files that will be installed by distutils-r1_python_install_all. The files can be specified using absolute paths or paths relative to source directory (${S}). If unset or empty, no HTML documentation will be installed.

CODE Example conditional HTML_DOCS setting

python_install_all() {
    use doc && local HTML_DOCS=( doc/_build/html/. )

    distutils-r1_python_install_all
}

EXAMPLES

Array. Defaults to unset/empty.

A list of example files/directories that will be installed by distutils-r1_python_install_all. The files can be specified using absolute paths or paths relative to source directory (${S}). If unset or empty, no examples will be installed.

The examples will be installed in 'examples' subdirectory of docdir, and they will not be explicitly compressed.

CODE Example conditional EXAMPLES setting

python_install_all() {
    use examples && local EXAMPLES=( samples/. )

    distutils-r1_python_install_all
}

Variables exported by eclass

PYTHON_DEPS

Contains the dependency string on Python interpreters and auxiliary tools (dev-python/python-exec).

It should be used only with DISTUTILS_OPTIONAL enabled. Otherwise, the dependencies are placed in RDEPEND and DEPEND automatically.

If the Python dependency is conditional to a USE flag, the reference should be placed in appropriate USE-conditional block.

CODE Example use of PYTHON_DEPS

RDEPEND="python? ( ${PYTHON_DEPS} )"
DEPEND="${RDEPEND}"

PYTHON_REQUIRED_USE

Contains the REQUIRED_USE constraint requiring at least one Python implementation to be selected.

It should be used only with DISTUTILS_OPTIONAL enabled. Otherwise, the constraint is placed in REQUIRED_USE automatically.

If the Python dependency is conditional to a USE flag, the reference should be placed in appropriate USE-conditional block.

CODE Example use of PYTHON_REQUIRED_USE

REQUIRED_USE="python? ( ${PYTHON_REQUIRED_USE} )"

PYTHON_USEDEP

Contains a USE dependency string that can be used to enforce matching Python implementations on package dependencies.

It should be used on package dependencies which are using the python-r1 eclass. If DISTUTILS_SINGLE_IMPL is enabled, it can be used against python-single-r1 packages as well.

CODE Example use of PYTHON_USEDEP

RDEPEND="dev-python/jinja[${PYTHON_USEDEP}]"
DEPEND="${RDEPEND}
    dev-python/setuptools[${PYTHON_USEDEP}]"

Sub-phase functions

distutils-r1_python_prepare_all

Default implementation of python_prepare_all. If you override python_prepare_all, you are required to call the default at the end of it.

The default implementation does the following, in order:

1. Applies patches from the PATCHES variable.
2. Applies user patches.
3. Performs Gentoo-specific cleanups on the sources/build system:
   • Disables auto-downloading of setuptools.
4. Creates source copies for multiple implementations if DISTUTILS_IN_SOURCE_BUILD is enabled.

distutils-r1_python_compile

Default implementation of python_compile. If you override python_compile, you are advised to call it.

The default implementation does the following, in order:

1. Performs Gentoo-specific cleanups which need to be done per-implementation:
   • Creates pydistutils.cfg file setting build directories for distutils,
   • Copies bundled egg-info files to build tree (as required for out-of-source build).
2. Calls setup.py to build the sources.

distutils-r1_python_install

Default implementation of python_install. If you override python_install, you are advised to call it.

The default implementation does the following, in order:

1. Enables byte-code compilation (which is disabled by default in ebuild scope).
2. Calls setup.py to install the package in a dedicated fake root.
3. Wraps Python scripts as necessary.
4. Merges the package to the ebuild installation root (${D}).

distutils-r1_python_install_all

Default implementation of python_install_all. If you override python_install_all, you are required to call the default in it.

The default implementation does the following, in order:

1. Installs documentation listed in the DOCS and HTML_DOCS variables.
2. Installs examples listed in the EXAMPLES variable.

Other functions

esetup.py

Usage: esetup.py [<args>...]

Run setup.py using the current Python interpreter and standard configuration arguments. The following arguments are passed to setup.py, in order:

1. mydistutilsargs,
2. Additional arguments passed to the command.

It should be noted that distutils-r1_python_compile influences the behavior of setup.py by setting build-dirs. esetup.py invocations preceding it will not respect build-dir.

distutils_install_for_testing

Usage: distutils_install_for_testing [<args>...]

Run esetup.py to install the package to a temporary directory suitable for running tests against the installed package. The install layout will be specifically adjusted to suit packages using Python namespaces.

This function introduces TEST_DIR variable that specifies the directory where the package was installed. The Python modules are installed to ${TEST_DIR}/lib, and the Python scripts are installed to ${TEST_DIR}/scripts. The former directory is added to PYTHONPATH.

CODE Example use of distutils_install_for_testing

python_test() {
    distutils_install_for_testing
    cd "${TEST_DIR}"/lib || die
    pytest -v || die "Tests fail with ${EPYTHON}"
}