Raspberry Pi/Cross building

Building almost anything on the Raspberry Pi takes a very, very long time - especially when a lot of dependencies are involved.

Fortunately, it is possible to offload much of the heavy lifting to a more powerful system (such as your main Gentoo desktop/server) using distcc and crossdev (though this will only work for packages that are written in language that require compiling).

Note
The following guide works for Raspberry Pi 1, 2 and Raspberry Pi 3 B running in 32-bit mode (recommended). Raspberry Pi 3 B running in 64 bit mode is not covered by this documentation.

distcc

It is also suggested that the first package you build on the Raspberry Pi should be distcc, as it will dramatically speed up subsequent packages that require a lot of compilation.

Raspberry Pi and build server(s)

On the Raspberry Pi and on all build servers install sys-devel/distcc:

root #emerge --ask sys-devel/distcc

OpenRC

Edit the distcc config file to ensure it is on the right subnet for the network configuration. For example:

FILE /etc/conf.d/distccd

DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.1.0/24"

Systemd

When using systemd edit the config file for the systemd service:

FILE /etc/systemd/system/distccd.service.d/00gentoo.conf

Environment="ALLOWED_SERVERS=192.168.1.0/24"

.

Then register and start the distcc daemon:

root #rc-update add distccd default

root #rc-config start distccd

Or for systemd:

root #systemctl enable distccd

root #systemctl start distccd

Raspberry Pi only

Tell Portage to use distcc:

FILE /etc/portage/make.conf

FEATURES="distcc"

(Optional) Also add buildpkg to the FEATURES variable to tell the Raspberry Pi to build package files for everything it builds (if you want to use the same setup on multiple Raspberry Pis without recompiling).

Edit the distcc host file to tell your Raspberry Pi to submit compile jobs to the server:

FILE /etc/distcc/hosts

# --- /etc/distcc/hosts -----------------------
# See the "Hosts Specification" section of
# "man distcc" for the format of this file.
#
# By default, just test that it works in loopback mode.
[YOUR HOSTS HERE - first host is first preference] 127.0.0.1

Now you will need to tell distcc the specific compiler name to use instead of just "gcc":

root #cd /usr/lib/distcc/bin

root #ls -l

total 0
lrwxrwxrwx  1 root root 15 Dec 23 20:13 c++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 cc -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 g++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 gcc -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 armv6j-hardfloat-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 armv6j-hardfloat-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx  1 root root 15 Dec 23 20:13 armv6j-hardfloat-linux-gnueabi-gcc -> /usr/bin/distcc

We need to replace those symlinks with the following script:

FILE /usr/lib/distcc/bin/armv6j-hardfloat-linux-gnueabi-wrapperDistcc crossdev wrapper file

#!/bin/bash
exec /usr/lib/distcc/bin/armv6j-hardfloat-linux-gnueabi-g${0:$[-2]} "$@"

root #rm c++ g++ gcc cc

root #chmod a+x /usr/lib/distcc/bin/armv6j-hardfloat-linux-gnueabi-wrapper

root #ln -s armv6j-hardfloat-linux-gnueabi-wrapper cc

root #ln -s armv6j-hardfloat-linux-gnueabi-wrapper gcc

root #ln -s armv6j-hardfloat-linux-gnueabi-wrapper g++

root #ln -s armv6j-hardfloat-linux-gnueabi-wrapper c++

Double check that you did things right:

root #ls -l

total 4
lrwxrwxrwx 1 root root 15 Dec  8 00:38 armv6j-hardfloat-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 Dec  8 00:38 armv6j-hardfloat-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 Dec  8 00:38 armv6j-hardfloat-linux-gnueabi-gcc -> /usr/bin/distcc
-rwxr-xr-x 1 root root 85 Dec  8 01:04 armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 Dec  8 01:05 c++ -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 Dec  8 01:04 cc -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 Dec  8 01:04 g++ -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 Dec  8 01:04 gcc -> armv6j-hardfloat-linux-gnueabi-wrapper

You can now check what is being dispatched to your build machines while doing an emerge operation:

root #DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text 1

crossdev

This will setup crossdev on your build servers so that they can compile binaries compatible with the Raspberry Pi. Note that you can have multiple compilation nodes - just add them to the list of hosts on the Raspberry Pi. Distcc will decide when to distribute the compilation though, so chances are you will never be able to fully load even a single modern build server with jobs from the Raspberry Pi.

Install sys-devel/crossdev:

root #emerge --ask sys-devel/crossdev

You will need to maintain separate portage profiles for the Raspberry Pi and your server's default, so you must convert your existing profile files to folders.

Copy the following file to ~/convert-profile-to-files.sh, and then run it as root:

FILE ~/convert-profile-to-files.shConvert profile files to folders

#!/bin/bash
PROFILE_DIR="/etc/portage"

if [ ! -e ${PROFILE_DIR} ]; then
  mkdir ${PROFILE_DIR};
fi;

for PACK_DIR in package.env package.accept_keywords package.keywords package.use package.unmask package.mask; do
  CUR_DIR="${PROFILE_DIR}/${PACK_DIR}"
  if [ ! -e ${CUR_DIR} ]; then
    mkdir ${CUR_DIR}
  fi

  if [ -e ${CUR_DIR} -a ! -d ${CUR_DIR} ]; then
    mv ${CUR_DIR} ${CUR_DIR}.moving
    mkdir ${CUR_DIR}
    mv ${CUR_DIR}.moving ${CUR_DIR}/monolithic
  fi
done

echo "Completed!"

Create a cross toolchain for ARM: (drop the -S option from the command when planning to run an unstable system):

root #crossdev -S -v -t armv6j-hardfloat-linux-gnueabi

If the cross toolchain for ARM fail to build try this:

root #CFLAGS="-O2 -pipe" CXXFLAGS="${CFLAGS}" crossdev -S -v -t armv6j-hardfloat-linux-gnueabi

See also

• distcc

External resources

• DistCC Cross-compiling Guide - Provides information about cross-compiling with DistCC