This is Gentoo's testing wiki. It is a non-operational environment and its textual content is outdated.
Please visit our production wiki at https://wiki.gentoo.org
Project:Quality Assurance/Automagic dependencies
This guide aim to describe the problem of "automagic dependencies", describing the reason why they are problematic and how to handle them in most common cases.
Introduction
What are automagic dependencies?
The so-called automagic dependencies are shallow dependencies of a software recognized at build or runtime and changes the way the software works. The name automagic is a pun referred to the use of GNU autotools, that produces most of the cases of automagic dependencies .
Software usually has two kinds of dependencies: mandatory dependencies and optional dependencies. The first kind of dependencies are needed to use the software (that might be a library or a program), and cannot be missing in the system while building or running the program (depending whether they are build or runtime dependencies). Optional dependencies are the ones that can be disabled, usually at buildtime (but sometimes at runtime, too).
Optional dependencies are usually up to the user (or the builder) to enable or disable, the classical example is brought by the --enable-foo or --with-bar options at ./configure call (those parameters are used to enable dependencies that are off by default, but there are cases when the dependencies are on by default so you have --disable-foo and --without-bar ).
But with build systems that try to understand what is present in the system they are building it, sometimes dependencies get automagic . This means that the build system doesn't give the builder a way to decide if he wants something enabled, so the dependency added, but they just enable it when they find it. This is the wrong behavior.
Why automagic dependencies are wrong
In the case of binary-based distributions, like RPM or DEB based ones, automagic dependencies does not change anything: if the user has something installed and is building by hand, it's usually what he wants to enable, while if it's the maintainer, he'll just have to add a dependency over the packages required to run the binaries he has created.
Different is for source-based distributions like Gentoo Linux (and variants). As source-based distributions doesn't detach user and devel packages, the build systems might find more stuff than the user required, and will try to link to all it know about. This is the major cause of binary linking breaking after a depclean.
To simplify, when an automagic dependency is not stated as mandatory in an ebuild, but rather has a flag that just adds or remove the dependencies on a given package, if this package is present in the system building the software with automagic dependencies, but then it's removed, the software will break, requiring to run revdep-rebuild to fix the linking. It's also possible that an user really don't want some dependency enabled because he know it's likely to break from time to time, or because he's going to create a binary package for another machine where the dependency might not be present (or might not work at all).
When a package has automagic dependencies there are only two things that can be done: the first is to state the dependency as mandatory, no matter what the users put in their USE variable, but that might mean that some support that people don't want is always enable and its dependencies pulled in; the other is to fix the build system to be able to disable at build time the dependency also if it's present on the system.
Most of the time upstream developers don't really think of adding support for disabling automagic dependencies as they don't feel them as actual problems: they do have all of them installed, or the ones they need, and they usually build with all of them. Luckily, most of upstream developers also don't mind adding options to disable them if patches are provided (sometimes also without patches, but of course it might be more welcome if already prepared patches are sent), but that's not always the case, for example Wine's upstream don't want to add support for enabling or disabling features in ./configure call as they want the software to always use as much as options as possible.
Fixing automagic dependencies
Autotools
Most of the automagic dependencies, like the name suggests, are due to (bad) use of GNU autotools ( autoconf to be exact). There are two main cases where automagic dependencies are brought in: the first is the "lazy devs" case, where the dependencies does not have a ./configure parameter at all, they are just checked with AC_CHECK_LIB or the pkg-config macro PKG_CHECK_MODULES , that allows to run specific code when a library (or a package) is present or not; the other case is the "silly argument" case, where a --disable-foo or --without-bar parameter is actually accepted by ./configure , but it's not checked correctly.
The first case is actually simple to fix, it's just matter of adding a AC_ARG_WITH (or AC_ARG_ENABLE ) call and then check for the corresponding variable before doing the test. It's useful to know that the first parameter passed to the above macro actually names a variable that gets loaded by autoconf without having to add the extra parameters for action to execute when the parameter is present and when it's not, the variable is named $enable_foo or $with_bar , depending on which of the two macros are called.
For the patches to be accepted by upstream, it's usually suggested not to change the default behavior, when
./configure is called without parameters; for this reason, here will be listed two methods to make non-automagic the dependencies, one for enabled-by-default deps and one for disabled-by-default deps.AC_ARG_WITH([foo], AS_HELP_STRING([--without-foo], [Build without foo library (default: test)])) AS_IF([test "x$with_foo" != "xno"], [ PKG_CHECK_MODULES([FOO], [foo >= 0.1]) ])
AC_ARG_WITH([foo], AS_HELP_STRING([--with-foo], [Build with foo library (default: disabled)])) AS_IF([test "x$with_foo" = "xyes"], [ PKG_CHECK_MODULES([FOO], [foo >= 0.1]) ])
When the parameter is present but it's not honored, it might be simple as well as complex to fix the dependency. It might just be a test that's not properly written, so it has to be changed in something alike to the tests above, or it might be a complete screw-up in calls of AC_ARG_WITH macros. In those cases, it's better to check the code carefully and contact upstream if a screw up seems likely.
Often (almost always when a package is using automake) automagic dependencies are coupled with AM_CONDITIONAL calls. It's very important that those calls are put outside the if/fi block, or
./configure call will bomb out.While it is possible to work around the issue of automagic dependencies without patching configure.ac , by messing with the autoconf cache values, this method is not recommended. This does not fix the original issue, and cannot be sent upstream for integration in new versions, and can actually conflict when tests are slightly different between environments.
CMake
Automagic dependencies may occur in CMake-based build systems where PKG_CHECK_MODULES is called without the REQUIRED parameter unconditionally. Fixing this is quite easy, as it only involves introducing an option to build system and executing PKG_CHECK_MODULES , depending on its value.
OPTION(ENABLE_FOO "Enable foo library" ON) ... IF (ENABLE_FOO) PKG_CHECK_MODULES (FOO foo>=0.1) ENDIF (ENABLE_FOO) ... IF (ENABLE_FOO) IF (FOO_FOUND) ... ELSE (FOO_FOUND) ... ENDIF (FOO_FOUND) ENDIF (ENABLE_FOO)
Set the default in OPTION according to the original behavior.
Other build systems
Please expand this guide ;) Notes about other non-custom build systems such as
scons are welcome, if the build system has ways to define automagic dependencies, it should have a way to fix them, too.Automagic dependencies can be created also with custom build systems that are used by some software. Unfortunately, being custom, those build systems are usually difficult to tweak, and there's no way to describe a general approach to fix them.
This page is based on a document formerly found on our main website gentoo.org.
The following people contributed to the original document: Diego Elio Pettenò, Serkan Kaba
They are listed here because wiki history does not allow for any external attribution. If you edit the wiki article, please do not add yourself here; your contributions are recorded on each article's associated history page.