Upstream bug: users and groups created during install of packages now respect ROOT

[edannenberg]

This commit breaks the build for uncached packages that create users/groups.

Problems:

• While the user/group is now created at the right location it will fail the build if the new user is actually used in the build to change ownership of files, as for example fowners will still look on the host for the user.
• The user eclass expects certain paths to exist at ROOT which is not always true for our use case. Not handled gracefully currently and fails the build.
  * To make matters worse the behavior is now inconsistent, as binary packages still create any new users/groups on the host.

@thesamesam Could this be reverted for now until all the edge cases are fleshed out? Right now it does more harm then good as its much harder to work around then with the old behavior.

[thesamesam]

Thanks! This sounds like ebuilds may have incorrect dependencies (it's an issue when installing from binpkgs too). Just got up though so let me have a think 🤔

Is there an example set of steps I can play with?

[thesamesam]

cc @Mord3rca

[Mord3rca]

Hey !
I see .. The only way to mitigate the fowners issue is to wrap the chown / fchown syscall... So, all commands will be affected at once.
I did not try a full test build at work, but we have this kind of wrapper to avoid setting an unknown permission + we hook getpwname to return an internal list of users read from ${ROOT}/etc/passwd so commands like chown won't crash with a user which is not from the host.
I'll see if this can correct the error. If so, maybe we can add the hooked function to libsandbox. I'll keep you informed.

[thesamesam]

We discussed this a bit in #gentoo-base on IRC and I think we need some logs to properly understand it. Could you file a bug on Gentoo's Bugzilla with that information? (inc. e.g an example ebuild, and some steps to repro).

Needing BROOT to contain things in order to do e.g. fowners is considered normal and something we've been doing for ages. I'm not against a change along the lines @Mord3rca suggests in principle, I think, but would need to see it in more detail?

Everything but point 2. should be handled by dependencies and point 2 seems like it should be handled by baselayout.

[Mord3rca]

@thesamesam Good news ! I've started a full rebuild (not of this, just to be clear) with my modification in eclasses + my hook for getpwnam / getgroupnam etc and it seems to do the trick for fchown. I'm starting to adapt my work for libsandbox so you can hopefully test it soon.

[edannenberg]

Thanks for the swift replies!

Is there an example set of steps I can play with?

Any ebuild that creates a user/group and then tries to fowners with the new user will do, provided the user doesn't exist on the host yet.

Simple test ebuild

# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=7
DESCRIPTION="A test"

inherit user vcs-snapshot
EGIT_COMMIT="be5b2e1"
SRC_URI="https://github.com/edannenberg/kubler/archive/${EGIT_COMMIT}.tar.gz -> ${P}.tar.gz"

KEYWORDS="alpha amd64 arm arm64 ia64 ppc sparc x86"
IUSE="bar"
SLOT="0"

DEPEND=""
RDEPEND=""

pkg_setup() {
	enewgroup kubler
	enewuser kubler -1 -1 /usr/share/${PN} kubler
}

src_install() {
	insinto /usr/share/${PN}
	doins -r bin/ cmd/ engine/ lib/ template/ kubler.conf kubler.sh README.md COPYING

	fowners kubler:kubler /usr/share/${PN}
}

We discussed this a bit in #gentoo-base on IRC and I think we need some logs to properly understand it. Could you file a bug on Gentoo's Bugzilla with that information? (inc. e.g an example ebuild, and some steps to repro).

Given the example ebuild above:

Original behavior:

$ ROOT=/foo emerge dev-util/foo
$ cat /etc/group
<snip>
kubler:x:249:
$ cat /etc/passwd
<snip>
kubler:x:999:249:added by portage for foo:/usr/share/foo:/bin/false
$ cat /foo/etc/passwd
cat: /foo/etc/passwd: No such file or directory

Current behavior

$ rm /packages/x86_64-pc-linux-gnu/dev-util/foo/foo-0.0.1-1.xpak
$ ROOT=/foo emerge dev-util/foo
>>> Emerging (1 of 1) dev-util/foo-0.0.1::kubler for /foo/
>>> Failed to emerge dev-util/foo-0.0.1 for /foo/, Log file:
>>>  '/var/tmp/portage/dev-util/foo-0.0.1/temp/build.log'
>>> Jobs: 0 of 1 complete, 1 failed                 Load avg: 3.80, 5.65, 3.70
* Package:    dev-util/foo-0.0.1
* Repository: kubler
* Maintainer: [email protected]
* USE:        abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU
* FEATURES:   preserve-libs sandbox userpriv usersandbox
realpath: /foo/usr/lib: No such file or directory
grep: /foo/etc/group: No such file or directory
* Adding group 'kubler' to your system ...
*  - Groupid: next available
groupadd: /foo//etc/group.153: No such file or directory
groupadd: cannot lock /foo//etc/group; try again later.
* ERROR: dev-util/foo-0.0.1::kubler failed (setup phase):
*   (no error message)
*
* Call stack:
*          ebuild.sh, line 127:  Called pkg_setup
*   foo-0.0.1.ebuild, line  29:  Called enewgroup 'kubler'
*        user.eclass, line 353:  Called die
* The specific snippet of code:
*   	        groupadd -r "${opts[@]}" "${egroup}" || die

$ mkdir -p /foo/{etc,usr/lib}
$ ROOT=/foo emerge dev-util/foo
>>> Failed to emerge dev-util/foo-0.0.1 for /foo/, Log file:
>>>  '/var/tmp/portage/dev-util/foo-0.0.1/temp/build.log'
>>> Jobs: 0 of 1 complete, 1 failed                 Load avg: 4.05, 5.74, 4.16
* Package:    dev-util/foo-0.0.1o-0.0.1::kubler for /foo/
* Repository: kubler
* Maintainer: [email protected]
* USE:        abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU
* FEATURES:   preserve-libs sandbox userpriv usersandbox
grep: /foo/etc/group: No such file or directory
* Adding group 'kubler' to your system ...
*  - Groupid: next available
grep: /foo/etc/passwd: No such file or directory
* Adding user 'kubler' to your system ...
grep: /foo/etc/passwd: No such file or directory
*  - Userid: 999

# Omitted the actual error that failed the build, the above errors are just from the
# checks to determine the next free uid/gid and don't fail the build.
# fowners don't finding the kubler user fails the build.

$ cat /foo/etc/group
kubler:x:999:
$ cat /foo/etc/passwd
kubler:!:999:999:added by portage for foo:/usr/share/foo:/bin/false

The user is created at ROOT now but as the user doesn't exist on the host fowners fails the build. Happy to file a bug if you still think that would help.

Needing BROOT to contain things in order to do e.g. fowners is considered normal and something we've been doing for ages.

Not sure I follow, could you elaborate please?

Everything but point 2. should be handled by dependencies and point 2 seems like it should be handled by baselayout.

Not sure what you mean with handled by dependencies. Point 3 was me being confused by portage's environment feature, so old binary packages still worked as before (TIL). The behavior is consistent, if you manage to create a new binary package that is. Workarounds:

• create the user on the host manually prior installation and make sure it has the same uid/gid as in ROOT
• revert the patches to the eclasses until the issue is resolved (the option I went with for now)

Point 2 is a nice to have and I can work around that fairly easy by making sure the paths enewgroup/enewuser expect actually exist. Ideally the eclass would skip the check if ROOT doesn't have a passwd/group file.

[Mord3rca]

Hello,
I'm making progress on the patch for adding shadow capabilities for libsandbox, however my patch do not work in the sandboxed context. I need to read more carefully the code to understand why. (It works when using LD_PRELOAD so .. It's something I guess)
FYI, my branch can be found here: https://github.com/Mord3rca/sandbox/commits/libsandox/shadow_hook

@edannenberg regarding your workaround for the fowners situation, I guess you can do a more elegant solution. A lot of services (postfix, opendkim, ...) have the acct-user / acct-group related to their executing in both RDEPEND & BDEPEND, so you install the user in the host & target. This can be done only if you control the ebuild code of course.

Also, for your test build, be sure to install baselayout before everything else, if not, passwd / group files won't exist.

[edannenberg]

Sorry, I meant to reply to this sooner. Another option would be to support both behaviours via some ENV, the kubler patches should make this fairly easy to implement I think.