This is Gentoo's testing wiki. It is a non-operational environment and its textual content is outdated.

Please visit our production wiki at https://wiki.gentoo.org

User:Mjo/GLEP:User packages

From Gentoo Wiki (test)
Jump to:navigation Jump to:search


GLEP 00: User packages
Type Informational
Status Draft
Author Michael Orlitzky <mjo@gentoo.org>
Editor
Replaces GLEP:27
Replaced by (none)
Requires
Post History

Abstract

User management in Gentoo is currently ad-hoc. We create users and groups in ebuilds, but there is no systematic way of tracking which ebuilds create which users. There is no way to coordinate users used by multiple packages, and there is no way to remove unneeded users from the system. GLEP:27 was an attempt to fix some of those issues, but it has its own problems. The main one is complexity, as evidenced by the fact that it has not been implemented in 13 years. The council unanimously adopted the following motion on 2017-02-12: "Since GLEP 27 has never been implemented, the council is open to alternative solutions and improvements on the idea."

This GLEP proposes an easier way forward.

We treat system users a lot like we treat packages. We require that they be present before some other package can be installed. We require that only one user with a given name exist on the system. We want to remove users that are no longer needed. We share users between packages, etc. This GLEP proposes that we make packages for each system user and group, and then use the existing package management tools to handle them. This is easy to implement gradually, and has a simple upgrade path. There are some hairy issues, but even if we punt on those, we wind up better off than we are now.

Motivation

The following is a short list of requirements for system user/group management:

  1. It should be possible to share users and groups reliably between packages.
  2. Insofar as it is possible, users and groups should have fixed UIDs/GIDs on the system. This is useful for people running NFS, or sharing users/groups across containers.
  3. User and group management should be subject to local overrides.
  4. If a user or group is needed at runtime only, it should not be created at build time (bug 217042).
  5. Users and groups should be removed when they are no longer needed

The first three items are addressed by GLEP:27, but the remaining items are not. None of them are met presently.

In addition, we have the major practical motivation that a new system should be easy to implement. The proposal in GLEP:27 requires a new EAPI and package manager changes before one can even consider using the new system. Ideally no such changes should be needed.

Implementation phases

The most difficult part of the proposal is the removal of unused users. There are a number of ways we could do this; for example, by using find to track down any remaining files that are owned by the user in question. If there aren't any -- just remove him. If there are, prompt the user to do something about it.

But, that's all very wishy-washy at the moment. And moreover, we can't remove users now. So to make this proposal more immediately palatable and an unquestionable improvement, the implementation will take place in two phases (the second of which may never be reached).

Phase 1

  • Shared users and groups between packages.
  • Fixed UIDs where possible.
  • The ability to override the fixed UIDs.
  • Sort out the run-time vs. build-time user issue.

In this phase, attempting to uninstall a user or group package will result in an error that can be overridden with I_KNOW_WHAT_I_AM_DOING=yes. That way, if you're sure that the system user is no longer needed (or if you're just testing something), the package can be removed.

Phase 2

Implement the removal of users and groups, however we decide to do it.

Specification

Two new categories are created, sys-user and sys-group. The former contains ebuilds for system users, and the latter for system groups.

Two new eclasses sys-user.eclass and sys-group.eclass are created to be used by packages in the sys-user and sys-group categories respectively. It is an error to inherit both eclasses in the same package.

Every "user ebuild" should specify its desired UID via the SYS_USER_UID variable. If that UID is not available, another one will be chosen, essentially randomly in the range 100-999. A typical sys-user ebuild will look like the following:

FILE sys-user/tcpdump-0.ebuild
# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$

EAPI=6

SYS_USER_UID=123
inherit sys-user

KEYWORDS="~amd64 ~x86"

Some packages may need stronger UID guarantees. For example, you might have a binary foo package that requires the foo user to have UID 222. In that case, the ebuild must indicate that its UID is important:

FILE sys-user/foo-0.ebuild
# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$

EAPI=6

SYS_USER_UID=222
SYS_USER_UID_IMPORTANT=true
SYS_USER_HOME=/opt/foo
SYS_USER_SHELL=/bin/bash
SYS_USER_GROUPS="audio games video"

inherit sys-user

KEYWORDS="~amd64 ~x86"

Group ebuilds and GIDs are handled in a similar manner. The home, shell, and groups variables all default to the behavior of enewuser in the current version of user.eclass.

sys-user.eclass implementation

Here is a proof-of-concept eclass that uses user.eclass as a backend. Most variables have sensible defaults, and most phases do nothing:

FILE sys-user.eclass
: ${HOMEPAGE:="https://www.gentoo.org/"}
: ${DESCRIPTION:="The ${PN} system user"}
: ${LICENSE:="GPL-2"}

# If you want a different username, use a different package name. This
# prevents different people from claiming the same username.
SYS_USER_NAME="${PN}"

: ${SYS_USER_GROUPS:="${PN}"}
: ${SYS_USER_HOME:=-1}
: ${SYS_USER_SHELL:=-1}
S="${WORKDIR}"

sys-user_src_unpack() { :; }
sys-user_src_compile() { :; }
sys-user_src_test() { :; }

Subslots for our desired UID can be used. This (rather imperfectly) triggers rebuilds for packages that might depend on the UID of this user:

FILE sys-user.eclass
# In many cases, if the UID of a user changes, packages depending on it
# will want to rebuild. We always use SLOT=0, because you can't install
# the same user twice. Then we use the UID as our subslot so that
# subslot deps can be used to rebuild packages when our UID changes.
SLOT="0/${SYS_USER_UID}"

Dependency entries for a user's groups can be calculated automatically:

FILE sys-user.eclass
# Depend on any groups we might need.
for _group in ${SYS_USER_GROUPS}; do
	DEPEND+=" sys-group/${_group} "
	RDEPEND+=" sys-group/${_group}:= "
done
unset _group

We perform a few sanity checks in pkg_pretend before proceeding to the meat and potatoes:

FILE sys-user.eclass
sys-user_pkg_pretend() {
	# Sanity checks that would otherwise run code in global scope.
	#
	# First ensure that the user didn't say his UID is important and
	# then fail to specify one.
	if (( "${SYS_USER_UID}" == -1 )) &&
		   [[ "${SYS_USER_UID_IMPORTANT}" == "true" ]]; then
		# Don't make no damn sense.
		die "arbitrary UID requested with SYS_USER_UID_IMPORTANT=true"
	fi

	# Next ensure that no other username owns an important UID.
	if [[ "${SYS_USER_UID_IMPORTANT}" == "true" ]]; then
		# Ok, the UID is important. Make sure nobody else has it. Or
		# rather, nobody else *with a different username* has it.
		local oldname=$(egetent passwd "${SYS_USER_UID}" | cut -f1 -d':')
		if [[ "${SYS_USER_NAME}" != "${oldname}" ]]; then
			die "important UID ${SYS_USER_UID} already belongs to ${oldname}"
		fi
	fi

	# Finally, ensure that this username doesn't already exist with
	# another UID if its UID is supposedly important.
	if [[ -n $(egetent passwd "${SYS_USER_NAME}") ]]; then
		local olduid=$(id --real --user "${SYS_USER_NAME}")
		if [[ "${SYS_USER_UID_IMPORTANT}" == "true" ]] && \
			   [[ "${SYS_USER_UID}" != "${olduid}" ]]; then
			# The UID is important and specified, but there is already a
			# system user with this name and a different UID. Halp.
			die "user ${SYS_USER_NAME} already exists with UID ${olduid}"
		fi
	fi
}

The "next available UID" function is stolen from user.eclass. Currently that eclass doesn't handle the case where you run out of UIDs, but I think it should. The new eclass dies if that happens.

FILE sys-user.eclass
sys-user_next_uid() {
	local euid;
	for (( euid = 101; euid <= 999; euid++ )); do
		[[ -z $(egetent passwd "${euid}") ]] && break
	done
	if (( "${euid}" == 999 )); then
		die "out of available UIDs!"
	else
		echo "${euid}"
	fi
}

To avoid UID collisions, every user and group package installs a placeholder file under /var/lib. This will alert developers that they have screwed up.

FILE sys-user.eclass
sys-user_src_install() {
	# Install a placeholder file to /var/lib/sys-user/$uid. This will
	# cause collisions if two packages try to install users with the
	# same UID. The same problem potentially exists with the username,
	# but as long as SYS_USER_NAME is hard-coded to $PN, that shouldn't
	# be possible.
	#
	# Beware, this only works if SYS_USER_UID is guaranteed to have a
	# real UID and not, for example, -1. That is taken care of in
	# src_configure() for now.
	touch "${T}/${SYS_USER_UID}" || die
	insinto "/var/lib/sys-user"
	doins "${T}/${SYS_USER_UID}"
}

Removal of user packages should only be possible if the user owns no files, as mentioned in the rationale. Parts of this are left as pseudocode. The pkg_prerm phase is used so that we can die() before the package is removed if the system user will not be removed.

FILE sys-user.eclass
sys-user_pkg_prerm() {
	if [[ -z $(egetent passwd "${SYS_USER_NAME}") ]]; then
		# We have successfully done nothing.
		ewarn "Tried to remove nonexistent user ${SYS_USER_NAME}."
	elif (the user owns no files)
		# Remove the user, platform-dependent...
		einfo "Removed user ${SYS_USER_NAME} from the system."
	else
		die "user ${SYS_USER_NAME} still owns files, delete them or something"
	fi
}

For "user package" upgrades, we must create the user in the pkg_postinst phase. That's not ideal, but otherwise, when the old version of the user package is removed, it will remove the user we just created!

FILE sys-user.eclass
sys-user_pkg_postinst() {
	if [[ -n "${REPLACING_VERSIONS}" ]]; then
	   # This is an upgrade from a previous version of a sys-user
	   # package. This case has to be handled carefully to make sure
	   # that the pkg_prerm() of the old version doesn't remove the user
	   # that this new version is going to add. At this point, in our
	   # pkg_postinst(), the old version's pkg_prerm() phase should have
	   # already happened.
	   if [[ -n $(egetent passwd "${SYS_USER_NAME}") ]]; then
		   # The username SHOULD NOT be here... when things go according
		   # to plan, the user is deleted and re-created with exactly the
		   # same UID, homedir, shell, etc.
		   die "User ${SYS_USER_NAME} already exists during an upgrade."
	   else
		   enewuser "${SYS_USER_NAME}" \
					"${SYS_USER_UID}" \
					"${SYS_USER_SHELL}" \
					"${SYS_USER_HOME}" \
					"${SYS_USER_GROUPS}" \
			   || die "failed to add user ${SYS_USER_NAME}"
	   fi
	fi
}

Now we're getting to the tricky stuff. The "configure" phase is what checks for a pre-existing user, and falls back to using its UID, homedir, and shell if possible. This is critical for the upgrade path.

FILE sys-user.eclass
sys-user_src_configure() {
	if [[ -n $(egetent passwd "${SYS_USER_NAME}") ]]; then
		# UPGRADE PATH: This user already exists, so if the eclass
		# consumer doesn't care about some settings, we can reuse the
		# pre-existing ones.
		#
		# This is also useful for sys-user package upgrades, because it
		# prevents us from incrementing the UID on a reinstall, and doing
		# so would break most packages that need a system user to exist.
		if [[ "${SYS_USER_UID_IMPORTANT}" != "true" ]]; then
			SYS_USER_UID=$(id --real --user "${SYS_USER_NAME}")
		fi

		if (( "${SYS_USER_HOME}" == -1 )); then
			SYS_USER_HOME=$(egethome "${SYS_USER_NAME}")
		fi

		if (( "${SYS_USER_SHELL}" == -1 )); then
			SYS_USER_SHELL=$(egetshell "${SYS_USER_NAME}")

			if [[ ${SYS_USER_SHELL} == */false ]] || \
				   [[ ${SYS_USER_SHELL} == */nologin ]]; then
				# WHYYYYY? enewuser complains if we try to set a default
				# shell explicitly.
				SYS_USER_SHELL="-1"
			fi
		fi
	elif (( "${SYS_USER_UID}" == -1 )); then
		# There is no pre-existing user (i.e. this isn't along the
		# upgrade path), and the consumer says he doesn't care about the
		# UID, so pick the next one.
		SYS_USER_UID=$(sys-user_next_uid)
	fi
}

Finally, we cone to pkg_preinst. This is where we'd like to actually create the users, and where we do so unless this is an upgrade (see the comments). For PMS compliance we also have some more sanity checks buried in this phase. Fortunately, in preinst, we can bail out in time.

FILE sys-user.eclass
sys-user_pkg_preinst() {
	if [[ -z $(egetent passwd "${SYS_USER_NAME}") ]]; then
		# The user does not already exist. This is the nice and easy
		# case because no matter how we got here, we want to go ahead
		# and create the (new) user.
		enewuser "${SYS_USER_NAME}" \
				 "${SYS_USER_UID}" \
				 "${SYS_USER_SHELL}" \
				 "${SYS_USER_HOME}" \
				 "${SYS_USER_GROUPS}" \
			|| die "failed to add user ${SYS_USER_NAME}"
	elif [[ -n "${REPLACING_VERSIONS}" ]]; then
		#
		# This case is done in pkg_postint() to avoid clobbering a
		# new user when we remove the old one.
		#
		:
	else
		# UPGRADE PATH: Ok, the user exists but this isn't an upgrade of
		# a sys-user package. This is the upgrade path from the old
		# style of user/group management to the new style. Lets see if
		# the new user is compatible with the old one; it usually will be.
		# We only bail out if there's a homedir or shell conflict.
		#
		# We should make it policy that new sys-user packages have the
		# same homedir and shell as the existing ones created by
		# ebuilds, but it can't hurt to check again here. These checks
		# are done here (and not in pkg_pretend, where they would be
		# more consistent) because the PMS states that REPLACING_VERSIONS
		# may not be defined there.
		#
		# If a homedir/shell changes during a sys-user upgrade, we don't
		# consider that a problem, because the change was knowingly made
		# by a developer who just edited an ebuild to make that change.
		local oldhome=$(egethome "${SYS_USER_NAME}")
		local oldshell=$(egetshell "${SYS_USER_NAME}")

		if [[ "${oldhome}" != "${SYS_USER_HOME}" ]]; then
			die "home directory conflict for new user ${SYS_USER_HOME}"
		fi

		if [[ "${oldhshell}" != "${SYS_USER_SHELL}" ]]; then
			die "shell conflict for new user ${SYS_USER_HOME}"
		fi

		# The user already exists, so all we have left to do is to try
		# to append SYS_USER_GROUPS to the existing groups. The "usermod"
		# tool expects a comma-separated list, so change our spaces to
		# commas. This does succeed if you append duplicates.
		usermod --append --groups "${SYS_USER_GROUPS// /,}" \
			|| die "failed to append groups to existing user ${SYS_USER_NAME}"
	fi
}

sys-group.eclass implementation

Should look a lot like the sys-user.eclass implementation.

Rationale

Sharing users/groups between packages

Currently, if two packages need the same user, they both must call enewuser with exactly the same arguments. That is bad design, and prone to errors. When users are created by packages, the same thing can be accomplished by a DEPEND entry.

Fixed UIDs

Due to backwards-compatibility concerns, we cannot simply switch to fixed UIDs -- it would break existing systems. However, by having each user ebuild specify a preferred UID, we likely arrive at fixed UIDs on new systems, where every user package should get its desired UID. Existing systems can also achieve the same thing with a little work.

Another proposal that was considered would have made all UIDs arbitrary by default. However, a significant number of users requested that there be default UIDs, so that they can synchronize those UIDs on their systems.

Local overrides

Everything in this proposal can be overridden in an overlay.

Example: new installation

If fixed UIDs are critical to you on a new system, all that needs to be done is to define SYS_USER_UID_IMPORTANT=true somewhere; in a profile, or in a copy of the eclass itself.

Example: custom UID for one user

If you run some custom software that needs UID=123 but sys-user/foo wants that UID, then in an overlay, you can create your own copy of sys-user/foo.ebuild and change SYS_USER_UID to something else.

Removal of users and groups

When users and groups are packages, you can simply uninstall them.

That oversimplifies the issue a bit, but the new proposal is still an improvement over the status quo. Under this proposal, the UID for a user is not guaranteed. That means that it is not safe to remove a user/UID while he still owns files, because a new user may be created with that same UID at a later point and gain access to those files. The least bad solution to that problem seems to be that we should check for such files in pkg_prerm, and then refuse to uninstall the user package if any exist. We can instruct the administrator to delete the files and try again. Doing so will refuse to remove a lot of users, but the status quo is that those users and junk files would be left forever anyway. At least under the new proposal they are tracked in the sense that the package manager has a record of them.

Run-time users vs. build-time users

Some users are only needed at runtime, and under the new proposal they can be added to RDEPEND. Likewise, users needed at build time can be added to DEPEND.

This should solve the issue we currently have where either pkg_setup or pkg_postinst is used (often incorrectly) to create a user, based on when it is needed. Note that the documentation on this issue has been wrong for a decade. Thinking of users as dependencies turns the problem into something already understood by everyone.

Open questions and what's missing

Home directories

When creating a new user, do we want to create its home directory in the eclass's src_install phase? The benefit to that is that it can be removed later, and of course the pitfall is that we have to avoid giving ownership of e.g. /dev/null to regular system users. The enewuser function handles this by creating the directory, but only if it does not exist.

The group eclass

The sys-group.eclass still needs to be written.

Incidental user-group creation

We can (and do) prevent useradd from creating a new group named after the user, but we can't prevent userdel from deleting it. This is bug 512220 and hopefully isn't too problematic. It's a Phase 2 problem.

Profile overrides

Only certain things can be overridden in profiles. For example, it would not be possible to change the preferred UID of single user-package in a profile. However, this is mitigated by the fact that UIDs are ultimately still random in most cases, and so the only packages where that would make sense are the ones that have SYS_USER_UID_IMPORTANT=true. With any luck, there aren't any of those with different UID needs on different platforms.

In extreme situations, if we allow the user and package name to differ, we might introduce a second package for a particular architecture (where a profile override would otherwise be used). So if sys-user/foo is wholly unsuitable on hppa, then we have the ability to introduce sys-user/foo-hppa.

Portability

In the proof-of-concept sys-user.eclass, a number of platform-dependent tools are used. For example, the "id" and "userdel" programs. The output of "egetent" is also parsed with "cut". Portable functions to do those things need to be written. A portable "usermod" would improve the upgrade procedure; see below.

Migrating existing systems

Existing systems will not be able to use the new "default" UIDs without some pain. Since chown follows links, it's dangerous to call it recursively, because the owner of a directory can swap out its contents with a symlink to e.g. /root. However, the GNU chown implementation has a --from flag that would be useful in this case. So ultimately, we may be able to provide our end users with a command that updates the UIDs on their systems. Something like,

root #find / -type f -o -type d -o -type p -o -type s -print0 | xargs -0 chown --from=old_uid new_uid

Backwards Compatibility

This proposal is completely backwards-compatible, at the cost of having predictable UIDs and GIDs only on new installations. New sys-user/* and sys-group/* packages can obviously be created without affecting anything. At that point, ebuilds can be migrated to use the new user/group packages. When those packages get installed, they will reuse existing UIDs and GIDs where possible to prevent breakage. If a sys-user/* or sys-group/* package is uninstalled, it will first check that the user/group in question owns no files. In that manner we avoid breaking packages that have not been migrated to depend on sys-user/* and sys-group/* yet.

The old and new methods of user/group management can coexist for as long as necessary.

If this proposal should turn out to be a bad idea, it is easy to revert. If it's the upgrade/removal details that are problematic, then simply drop those functions from the eclass to obtain the old "never remove or change a user" behavior. If it's the whole idea, then we can reintroduce the old form of user/group management back into the tree and delete the eclasses/categories.

Reference Implementation

The eclass described above has been written, but it is not portable. The lack of backwards-incompatibility however makes this very easy to develop against the tree.

Copyright

Copyright 1999-2017 Gentoo Foundation. Distributed under the terms of the Creative Commons Attribution-ShareAlike (CC-BY-SA) license, version 3.0.

Retrieved from "https://wikitest.gentoo.org/index.php?title=User:Mjo/GLEP:User_packages&oldid=673192"