This is Gentoo's testing wiki. It is a non-operational environment and its textual content is outdated.
Please visit our production wiki at https://wiki.gentoo.org
User:Goverp/MasterSlaveGentoo
Gentoo is a meta-distribution - you configure your own distribution through a combination of profile, USE flag settings, /etc/portage/make.conf settings such as CHOST and packages. Many Gentoo users run the result on just one computer, but it can be easy to use it on several computers using binary packages. This can be done in several ways; this document describes one of them. It may or may not be the most appropriate for any particular situation.
Concepts and Facilities
This document describes one way to use a binary package host. The Binary package guide contains reference details of the Gentoo facilities being used.
One computer, the master does the majority of the "heavy lifting" of compiling the packages. In most cases the others, the slaves, just install the compiled code. portage will compile and install different versions of the code if the master version is unsuitable for a particular slave.
In the approach described, slaves use nfs to access the binary packages on the master. Other access methods such as http are possible: refer to the binary package guide for details.
Matching processor architectures
The master and slaves must have the same CHOST. For best code sharing, their CFLAGS should be similar. For example, the master could be an old but powerful desktop computer with a 4-way AMD Phenom processor, lots of memory and a large RAID disk array, while a slave might be a laptop with a later but smaller AMD a9-9420 processor. It turns out that in most cases code for the Phenom runs well on the a9-9420, and magically, Gentoo can identify the cases where it doesn't and this approach will lead to the slave compiling its own custom code.
It might not work if the master has a later version of the similar architecture; for example, an a9-9420 master and a Phenom slave, unless you compromise by restricting code to the older architecture. This combination is untried; it might work just as well as the other way round, or not. For more details, see #Compiler --march settings for a discussion.
64-bit and 32-bit architectures are dissimilar; 64-bit code won't run on 32-bit processors at all. You could use a 32-bit master for a compatible 64-bit slave, but you'll only be running the slave at half power.
Matching implementation profiles and USE flags
Portage can handle differing profiles and USE flags settings for master and slaves. A slave to use master code for packages with USE settings the same on both machines, while compiling different code where the settings differ.
Different profiles mean many different USE settings between the machines. This may mean the majority of packages have to be recompiled for the slave, negating the benefits of the master/slave approach.
Since Gentoo does not include compiled kernels, this master/slave approach does nothing. The slave will need its own kernel; it may be possible to share a common configuration or compiled code, but that is beyond the scope of this approach.
Installation
The master and slave machines may share the same portage tree over NFS. This saves the slave from carrying the portage tree, and saves having to process emerge --sync. However, it's impossible to run any portage commands on the slave when not connected to the master. (This configuration works well when both DISTDIR and PKGDIR have their default values, so the one directory /usr/portage contains everything needed to build code.)
Alternatively, they may share just /usr/portage/packages (or whatever PKGDIR is set to) via NFS, and the each have their own portage tree. In this case, the slave may use emerge --sync against the master's rsyncd - see Local Mirror for details. This approach is a little more cumbersome, but allows the slave to process portage commands whether or not connected to the master.
The following works using the first arrangement under NFSv4, with /home/portage being a bind mount for /usr/portage:
/etc/exports/home -ro,no_subtree_check,fsid=root,async,crossmnt 192.168.1.0/24 /home/portage -ro,no_subtree_check,async,no_root_squash 192.168.1.0/24
/etc/fstab# <fs> <mountpoint> <type> <opts> <dump/pass> ... /usr/portage /home/portage none bind,defaults 0 0
(See NFSv4 for more information. NFSv3 needs a slightly different configuration (see NFS).
/etc/portage contains the configuration. Differing configurations prohibit simply sharing this through NFS. Instead, use rsync, and exploit portage's ability to handle configuration directories instead of files for /etc/portage/make.conf and so on.
Portage processes files in configuration directories in collating sequence order. Give the master files beginning 01... and the slaves files beginning 02.... portage will use the master files as a basis, and override them with the entries in the slave files. Keep the slave's 01 files in step with the master using rsync:
Tailor the hosts definitions and so forth to meet your network configuration and security needs.
/etc/rsyncd.conf# This line is required by the /etc/init.d/rsyncd script pid file = /run/rsyncd.pid use chroot = yes uid = nobody gid = nobody hosts allow = 192.168.1.0/24 hosts deny = 192.168.1.1 read only = yes [portage-config] path = /etc/portage comment = Portage configuration dont compress = * # Uncomment the following for your own local portage package tree mirror #[gentoo-portage] # path = /usr/portage # comment = Gentoo Portage tree # exclude = /distfiles /packages # dont compress = *
The following shell script copies all 01-prefixed files of interest. The contents of /etc/portage/env get synchronised 'in toto', as they're not themselves configuration; the relevant part of that is in /etc/portage/package.env. (Substitute master with your master system's network name or address.)
nano backup files get a '~' suffix, so they're excluded in the transfer.
syncportage.shPortage configuration synchronisation shell script#! /bin/sh echo Synchronizing /etc/portage rsync -vr --exclude '*~' --include 'package.*/' --include='01*' --exclude='*' master::portage-config /etc/portage rsync -vr --exclude '*~' --include 'make.conf/' --include='01*' --exclude='*' master::portage-config /etc/portage rsync -vr --exclude '*~' master::portage-config/env/ /etc/portage/env # Uncomment the following line to synchronize the portage package tree; adjust /etc/portage/repos.conf appropriately # echo Synchronizing portage package tree # emerge --sync -q echo Done!
Create the binary package tree
By default /usr/portage/packages, as described in Binary_package_guide#Using_quickpkg.
Use
Master configuration
As described, put all the generic configuration information in files with names prefixed by 01. Here's a couple of examples:
FEATURES includes "buildkg" to save the compiled packages in /usr/portage/packages. Some large packages are excluded by --buildpkg-exclude; note the quotes.
/etc/portage/make.conf/01master# Make.conf for Acer desktop machine - AMD Phenom 4-way USE="3dnow 3dnowext cleartype cmake corefonts declarative elogind ... sndfile sql sse sse2 sse3 sse4a suid threads vlc wallpapers wayland" LINGUAS="en_GB en en_US" L10N="en-GB en en-US" VIDEO_CARDS="radeon r600" ABI_X86="64" # From cpuinfo2cpuflags CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a" CHOST="x86_64-pc-linux-gnu" # Normal mode compilations CFLAGS="-march=native -O2 -pipe" CXXFLAGS="${CFLAGS}" FEATURES="parallel-fetch userfetch userpriv nostrip buildpkg" EMERGE_DEFAULT_OPTS="--jobs 4 --load-average 4 --quiet-build y --buildpkg-exclude 'sys-kernel/*-sources games-simulation/flightgear-data'"
The master configuration specifies hardware optimizations using CPU_FLAGS_X86 and legacy USE flags. These will be overridden in the slave's configuration.
/etc/portage/package.use/01masterapp-office/libreoffice -gtk -gstreamer app-office/libreoffice-l10n offlinehelp # Loads of other stuff suppressed
Slave configuration
Files with 02- prefixed names contain the differences between the slave and master configurations.
The majority of the magic lies in /etc/portage/make.conf. Portage executes the slave's 02... configuration after the master's 01..., so its USE above starts with ${USE} to inherit values from master before overriding them. Other settings are straight overrides.
Override DISTDIR to avoid writing to the read-only NFS shared portage tree.
Alter CPU_FLAGS_X86 and USE overriding the legacy settings to allow for differences in master and slave CPUs. For example, Phenom processors support 3dNow!, whereas a9-9420 uses newer speed-ups.
Example:
Do not override CHOST in the slave.
/etc/portage/make.conf/02slave# Changes (mostly) and additions (use) to 01acer.make USE="${USE} bluetooth pm-utils upower wireless -3dnow -3dnowext -consolekit elogind" VIDEO_CARDS="amdgpu radeonsi" INPUT_DEVICES="synaptics evdev" CFLAGS="-march=native -O2 -pipe" CXXFLAGS="${CFLAGS}" CPU_FLAGS_X86="aes avx avx2 f16c fma3 fma4 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" FEATURES="parallel-fetch userfetch nostrip binpkg-logs" MAKEOPTS="-j2 -l2" EMERGE_DEFAULT_OPTS="--jobs 2 --load-average 2 --quiet-build y" # Use a local DISTDIR for stuff not found in networked /usr/portage/distfiles DISTDIR="/var/tmp/distfiles" PORTAGE_RO_DISTDIRS="/usr/portage/distfiles"
Other configuration files such as package.use can override and add values as necessary. For example:
/etc/portage/package.use/02slave# for march=native compilation sys-kernel/gentoo-sources experimental # removes dependency on network manager and its ilk kde-plasma/powerdevil -wireless # Delete bluetooth support for libreoffice to allow use of the master's code. app-office/libreoffice -bluetooth dev-python/PyQt5 -bluetooth # needed by mesa x11-libs/libdrm video_cards_radeon
Operation
Run emerge --update .... as normal on the master.
Before running it on the slave, start rsyncd on the master, and run syncportage.sh on the slave. Then start nfs on the master. and on the slave, run:
root #mount /usr/portageroot #emerge --usepkg --ask --update --deep --changed-use --with-bdeps y @world
...output to follow...
Note the purple output, which is portage's way of saying "Here's one I prepared earlier". Notice also the green output where portage calculated that the master package's code is unsuitable for this slave.
Compiler --march settings
The configurations above use --march=native on both master and slave. This works for me. Gcc is at liberty to generate code exploiting hardware features on the master that might not exist on the slave. As it works for me, it appears that it only generates feature-sensitive code if the appropriate value is present in CPU_FLAGS_X86, and portage detects packages sensitive to these flags and compiles separately for the slave. Truly magic. I expected this to end in tears.
If your experience differs, the solution is to choose an explicit, less-specific, value of --march such as k8 for AMD processors.
The code generated, in either case, will get --mtune defaulting to the same architecture, unless overridden. Thus the slave will be running code tuned for the master, and there may be a performance impact, though probably less than for running generic IA-64 or X86 code. You could of course set --mtune to something more generic.