[Request for comments] do we still need cgmanager?

Didier Spaier · 03-01-2017, 06:11 PM

So I tried you new instructions, Chris.

It worked, of course

But something puzzles me: actually I do not see a difference between what you and I did (see my previous post). On the attached pic, see on the left "your" container (release=current) and on the right "my" container (release=14.2).

To apply your method I built here (Slint64-14.2) a libcgroup package from recently upgraded source in Slackware-current.

But to start both containers as didier I had /etc/rc.cgred non executable, so I just put in rc.local:
/etc/rc.d/rc.cgconfig start && chown -R root:lxc /sys/fs/cgroup.

Of course didier is a member of the lxc group.

So my question is: is it really necessary to first build a container as root then convert it?

To be sure could you please try with the attached template?

chris.willing · 03-01-2017, 11:09 PM

Your method could be just good Didier. I can't try it out right now (maybe weekend) but looking at the diff from your previous post, it seems you're doing a similar thing - just within the template itself (with remap_userns()). The original reason for doing it separately was that mknod could only be run as root. I see you've bypassed that in the non-root user case and used bind mounts instead. I wonder if that is good enough for non-root user whether it is good enough for root as well i.e. why differentiate between root * non-root user?

Matteo, are you watching this thread? What do you think?

chris

ponce · 03-01-2017, 11:27 PM

unfortunately I am just watching, ATM, I'm a little busy at work and have to find the time to get into it and a piece of hardware where to try an install without cgmanager and unprivileged containers, hope I can do it soon.

anon074 · 03-05-2017, 02:58 AM

I've replaced cgmanager with libcgroup as described by Chris Willing. Works fine. If anybody would like to run nested unprivileged containers, Gentoo outlines how to create a user namespace manually with shell commands: https://wiki.gentoo.org/wiki/LXC#Cre...8no_systemd.29. Create the namespace in the parent container's writable sub directories of /sys/fs/cgroup.

Didier Spaier · 03-05-2017, 12:51 PM

Thanks Johannes and voegelas for your comments.

I have rebuilt lxc as stated by Johannes and built libcgroup with current's SlackBuild, I can confirm that the attached patch allows to convert lxc-slackware to a template allowing to build, then start and use an unprivileged container, typing all commands as a regular user. Of course it also allows to build, start and use a privileged container typing all commands as root. This is on a Slint64-14.2, that does not differ from Slackware (same version) in the scope of my tests as far as I know.

I have inserted all information needed on top of the file, including the settings of the config files. Maybe it would be better to include these instructions in a README?

More tests are still welcome but as far as I can tell cgmanager can safely be removed from Slackware-current (and possibly handed over to slackbuids.org if someone volunteer for that).

This post is also an official request to the lxc maintainer, aka Matteo, to update this package. Of course I am ready to help in doing that if I can.

The attached patch supersedes the previous one. I have tried to include in the config file only the commands really necessary. Adding some for a better resources control is left to the admin.

I also take the occasion to highlight that sooner or later we will have to deal with cgroups v2, not knowing how long they will coexist with v1 (see also this rant). However I don't know if lxc will be adapted to cope with v2... Incidentally, this is something systemd has dealt with in the recently released version 233. Not to say that we should invent as many new options or kernel parameters for that as they do

Slightly off topic:

Having read some Linux man pages and kernel documentation to try to understand namespaces (including user namespaces), control groups, capabilities, credentials etc., I am a bit worried seeing how these things are becoming more complex and that, with so many "Linux extensions", software built using them can hardly be portable.

For instance It is worrying to read (in man capabilities) :

Quote:

No standards govern capabilities, but the Linux capability implementation is based on the withdrawn POSIX.1e draft standard; see ⟨http://wt.tuxomania.net/publications/posix.1e/⟩

How does that help to build something portable[1]? And who, apart some Linux guru, is really able to find a way among all those capabilities, let alone use them properly? But I digress, it seems.

[1]Furthermore the link leads to a page making us assume that this file has been permanently deleted...

PS Sorry for the typos in the patch's comments.

PPS Found a link to withdrawn POSIX.1e from here but see also why FreeBSD didn't like it.

chris.willing · 03-05-2017, 10:21 PM

Quote:

Originally Posted by Didier Spaier

I have inserted all information needed on top of the file, including the settings of the config files. Maybe it would be better to include these instructions in a README?

More tests are still welcome but as far as I can tell cgmanager can safely be removed from Slackware-current (and possibly handed over to slackbuids.org if someone volunteer for that).

There's a typo in step 7 where you lxc-attach -n slackware (instead of -n myslack which is what you created).

Quote:

The attached patch supersedes the previous one. I have tried to include in the config file only the commands really necessary. Adding some for a better resources control is left to the admin.

Unfortunately there's some sort of permission problem when I try the newly patched template here.

Code:

chris@d6:/tmp$ lxc-create -n test4 -t /tmp/lxc-slackware.didier 
installpkg is /sbin/installpkg
slackpkg is /usr/sbin/slackpkg

/tmp/lxc-slackware.didier: line 278: /var/lock/subsys/lxc: Permission denied
lxc-create: lxccontainer.c: create_run_template: 1290 container creation template for test4 failed
lxc-create: lxc_create.c: main: 318 Error creating container test4

chris

Didier Spaier · 03-06-2017, 08:28 AM

Quote:

Originally Posted by chris.willing

There's a typo in step 7 where you lxc-attach -n slackware (instead of -n myslack which is what you created).

Fixed, as well as other typos (alas, certainly not all), in the new attached diff below, thanks.

Quote:

Originally Posted by chris.willing

Unfortunately there's some sort of permission problem when I try the newly patched template here.

Code:

chris@d6:/tmp$ lxc-create -n test4 -t /tmp/lxc-slackware.didier 
installpkg is /sbin/installpkg
slackpkg is /usr/sbin/slackpkg

/tmp/lxc-slackware.didier: line 278: /var/lock/subsys/lxc: Permission denied
lxc-create: lxccontainer.c: create_run_template: 1290 container creation template for test4 failed
lxc-create: lxc_create.c: main: 318 Error creating container test4

chris

This is because you had previously built a container as root, thus /var/lock/subsys/lxc that remained was owned by root (and 0644).

To fix that I replaced every occurrence of /var/lock/susbys by /var/lock/subsys-$LOGNAME. We can't use $USER or $(id -u) as we are root when running the lxc-create command (inside the container) because of the uid mapping.

NOTE. I chose $LOGNAME as POSIX says:

Quote:

LOGNAME
The system shall initialize this variable at the time of login to be the user's login name.

I prefer that over $(logname) as after 'su <newuser>' the value of LOGNAME becomes <newuser>, but then 'logname' still outputs <previous user>.

chris.willing · 03-06-2017, 04:46 PM

With v2 patch, unprivileged container is now created without any obvious problem and I can start and attach to it & run simple commands OK. However if I enable networking in the container's config file, it can no longer be started. My usual config file entries to enable networking are

Code:

#lxc.network.type = veth
#lxc.network.flags = up
#lxc.network.link = lxcbr0
#lxc.network.name = eth0
#lxc.network.hwaddr = 00:16:3e:xx:xx:xx

When I enable just the first of these (type = veth), the container fails to start leaving a bunch of "Permission denied" errors like:

Code:

chris@d6:/tmp$ cat ~/.local/share/lxc/test4/test4.log 
      lxc-start 20170307073711.123 ERROR    lxc_cgfs - cgfs.c:handle_cgroup_settings:2148 - Permission denied - failed to set memory.use_hierarchy to 1; continuing
      lxc-start 20170307073711.123 ERROR    lxc_cgfs - cgfs.c:lxc_cgroupfs_create:897 - Could not set clone_children to 1 for cpuset hierarchy in parent cgroup.
      lxc-start 20170307073711.123 ERROR    lxc_cgfs - cgfs.c:cgroup_rmdir:209 - Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/pids/sysdefault/lxc
(lots, lots more like this ...)

If I use the same v2 template to create a normal privileged container, everything (so far) seems to just work - including networking. Meanwhile, containers created the "old" way continue to start and work as expected in either privileged or unprivileged mode.

Conclusion: at the moment there are still permission issues with Didier's v2 template in unprivileged mode. Maybe that's because the "autodev = 1" configuration directive can only work with devices for which the user has appropriate permissions i.e. there are still some that an ordinary user cannot create.

chris

chris.willing · 03-06-2017, 06:58 PM

Hang on - maybe the unprivileged container network stuff (hence device handling) is OK after all. Although I was unable to set the container's network configuration from its config file, the networking was working anyway (just needed to set USE_DHCP[0]="yes" in rc.inet1.conf). That raised the question of how the networking was enabled. It turns out I already had all the necessary config options set globally in /etc/lxc/default.conf. When I remove the global defaults, I was able to set them in the individual container config files and start the previously unstartable unprivileged container.

I'll play with it some more but at the moment it looks like the v2 template is OK.

chris

Richard Cranium · 03-06-2017, 08:37 PM

Quote:

Originally Posted by Didier Spaier

NOTE. I chose $LOGNAME as POSIX says: I prefer that over $(logname) as after 'su <newuser>' the value of LOGNAME becomes <newuser>, but then 'logname' still outputs <previous user>.

I learn something new every so often. Today is one of those days!

chris.willing · 03-08-2017, 04:45 PM

Something unexplained to watch out for ...
This morning I couldn't check in some changes in my local git repo due to:

Code:

gpg: can't open `/home/chris/.gnupg/pubring.gpg'
gpg: keydb_search failed: file open error
gpg: key 13A8B322: secret key without public key - skipped
gpg: skipped "Christoph Willing <chris.willing@xxx.yyy.zz>": secret key not available
gpg: signing failed: secret key not available
error: gpg failed to sign the data
fatal: failed to write commit object

Why mention that here? Well the reason the file was not available was because its ownership had recently changed:

Code:

chris@d6:/scratch/PKG/SBo/slackbuilds$ ls -l /home/chris/.gnupg/pubring.gpg
-rw------- 1 100000 100000 311271 Mar  7 07:31 /home/chris/.gnupg/pubring.gpg

and that's relevant because my /etc/sub{u,g}id contains an entry:

Code:

chris@d6:/tmp$ grep chris /etc/subuid
chris:100000:65537

which I've been using a lot in my testing of Didier's lxc-slackware template v2 patch. Note that the id range starts at 100000 - same as the new user:group of my pubring.gpg file.

I've now restored the correct user:group of my pubring.gpg file and run lots of lxc-{create,start,attach} commands, so far without being able to replicate the bogus ownership situation. Nevertheless I'm sure there's some connection and thought I'd mention it here in case anyone else sees something similar.

chris

Didier Spaier · 03-09-2017, 08:02 AM

Line #393 of the template v2:

Code:

         slackpkg -default_answer=n update.

In /usr/sbin/slackpkg:

Code:

    update)
        # If you are using "slackpkg update gpg" OR the system
        # doesn't have Slackware GPG key, download and install
        # the key
        #
        if [ "$UPARG" = "gpg" ] || [ "$GPGFIRSTTIME" = "0" ]; then
            #
            # Creates .gnupg directory if doesn't exist
            # without this dir, gpg got an error.
            #
            if ! [ -e ~/.gnupg ]; then
                mkdir ~/.gnupg
            fi
            getfile ${SOURCE}GPG-KEY $TMPDIR/gpgkey 
            gpg --yes --batch --delete-key "$SLACKKEY" &>/dev/null
            gpg --import $TMPDIR/gpgkey &>/dev/null && \
            echo -e "\t\t\tSlackware Linux Project's GPG key added"
            if [ "$UPARG" = "gpg" ]; then
                cleanup
            fi
        fi
        echo "Updating the package lists..."
        updatefilelists
    ;;

If we create a new user, there is no ~/.gnupg yet, and GPGFIRSTTIME is set to 0 by this code snippet in /usr/share/libexec/slackpkg/core-functions.sh:

Code:

    # Check if the Slackware GPG key are found in the system
    #                                                       
    GPGFIRSTTIME="$(gpg --list-keys \"$SLACKKEY\" 2>/dev/null \
            | grep -c "$SLACKKEY")"

Thus the directory ~/.gnupg is created and populated. These files are owned by 100000:100000 because of the uid/gid mappings and because they are outside the container.

But if you recursively fix the ownership of ~/.gnupg and its content, they won't change after another lxc-create, because the GPG key is already there thus GPGFIRSTTIME is not set to 0

Maybe what allows the process to make a new dir in ~ owned by 100000:100000 (and also write regular files in it or directly in ~, as I checked), has something to do with the fact that the process is granted CAP_SYS_ADMIN in this context?[1] From the script it sees the files it created in ~ as owned by root:root. But then why can't it write in another user's $HOME (I checked that)?

I am really puzzled and do not know a good fix yet. Any clue, idea, solution is welcome.

[1]Not sure about that as "man capabilities" is rather vague (highlight mine):

Quote:

CAP_SYS_ADMIN
* Perform a range of system administration operations including: quotactl(2), mount(2), umount(2), swapon(2), swapoff(2), sethostname(2), and setdomainname(2);
<snip>

chris.willing · 03-09-2017, 03:46 PM

I think the generality of your discussion about contents of ~/.gnupg is correct i.e. the issue will be due to something like that. With a fresh user account having no .gnupg directory, running lxc-create with v2 template creates

Code:

drwx------ 2 100000 100000 4096 Mar 10 07:28 .gnupg/

and, after chowning that directory to new user, its contents are:

Code:

-rw------- 1 100000 100000 9188 Mar 10 07:28 gpg.conf
-rw------- 1 100000 100000  926 Mar 10 07:28 pubring.gpg
-rw------- 1 100000 100000    0 Mar 10 07:28 pubring.gpg~
-rw------- 1 100000 100000    0 Mar 10 07:28 secring.gpg
-rw------- 1 100000 100000 1200 Mar 10 07:28 trustdb.gpg

However in my case the ~/.gnupg directory already existed for a long time previously and only the pubring.gpg file was affected; all other contents of ~/gnupg (and ~/.gnupg itself) still had correct ownership properties.

As for:

Quote:

From the script it sees the files it created in ~ as owned by root:root. But then why can't it write in another user's $HOME (I checked that)?

maybe because it's root.root in the container but still just ordinary user in the wider environment, where that particular user probably doesn't have write permission in another user's $HOME. Perhaps that restriction doesn't hold if both users have the same sub[u,g]id mapping though.

Didier Spaier · 03-09-2017, 04:23 PM

Quote:

Originally Posted by chris.willing

However in my case the ~/.gnupg directory already existed for a long time previously and only the pubring.gpg file was affected; all other contents of ~/gnupg (and ~/.gnupg itself) still had correct ownership properties.

Yes. This is expected as in this case only the commands "gpg --delete-key ..." and "gpg --import ..." are executed and they only modify pubring.gpg.

Didier Spaier · 03-10-2017, 05:21 AM

Found a fix, which is to export GNUPGHOME as $cache/.gnupg before running slackpkg. Does not hurt for privileged containers either. This should have come to my mind yesterday but my brain is slow, sorry.

PS v3.1 has just cosmetic changes (remaining typos).