[Request for comments] do we still need cgmanager?

Richard Cranium · 02-15-2017, 09:34 PM

Once upon a time, docker really did need cgmanager to run.

http://www.linuxquestions.org/questi...nt-4175580217/

Didier Spaier · 02-18-2017, 08:54 AM

Having read the posts #11 through #16, I feel the need to bring some clarification about how the control groups are dealt with in Slackware.

The control groups are created by the kernel, some very early during startup:

Code:

didier[~]$ grep -n cgroup_init /usr/src/linux-4.4.38/init/main.c 
516:    cgroup_init_early();
666:    cgroup_init();
didier[~]$

You can check this in the log:

Code:

dmesg|grep 'Initializing cgroup susbsys'

Then /etc/rc.S mounts the cgroup filesystem interface:

Code:

# Mount Control Groups filesystem interface:
    if grep -wq cgroup /proc/filesystems ; then
      if [ -d /sys/fs/cgroup ]; then
        # See linux-*/Documentation/cgroups/cgroups.txt (section 1.6)
        # Check if we have some tools to autodetect the available cgroup controllers
        if [ -x /bin/cut -a -x /bin/tail ]; then
          # Mount a tmpfs as the cgroup filesystem root
          mount -t tmpfs -o mode=0755 cgroup_root /sys/fs/cgroup
          # Autodetect available controllers and mount them in subfolders
          controllers="$(/bin/cut -f 1 /proc/cgroups | /bin/tail -n +2)"
          for i in $controllers; do
            mkdir /sys/fs/cgroup/$i
            mount -t cgroup -o $i $i /sys/fs/cgroup/$i
          done
          unset i controllers
        else
          # We can't use autodetection so fall back mounting them all together
          mount -t cgroup cgroup /sys/fs/cgroup
        fi
      else
        mkdir -p /dev/cgroup
        mount -t cgroup cgroup /dev/cgroup
      fi
    fi

Result:

Code:

didier[~]$ df -h|grep cgroup
cgroup_root        3,3G       0  3,3G   0% /sys/fs/cgroup
didier[~]$ ls /sys/fs/cgroup
blkio      cpu      cpuset   freezer  net_cls   perf_event
cgmanager  cpuacct  devices  memory   net_prio  pids
didier[~]$

From then it is possible to create and manage cgroups, attach them to controllers, do resouces management setting the tunable parameters in /sys/fs/cgroup/<sub system>/*., just using commands like "mkdir" to create cgroups and "echo" to set the parameters. This doesn't need that a daemon be running so this doesn't need that any /etc/rc.d/rc.cg* script be executable.

Now about the startup scripts (the name of the package that provides it is in parenthesis):

rc.cgconfig (libcgroups) runs the cgconfigparser utility that parses /etc/cgconfig.conf to setup the control group hierarchy, including setting the permissions (UID and GID) of the groups and possibly tunable parameters of the controllers. This just alleviates doing that ourselves at every startup, for instance running a separate shell script. This script is executable by default but anyway won't do anything until /etc/cgconfig.conf be edited (all lines are commented out as shipped)

rc.cgred (libcgroup) starts the cgroups rules engine daemon that automatically distributes to appropriate control groups the processes that changes their effective UID or GID, applying the rules found in /etc/cgrules.conf. This allows to continue apply resources control resources on such processes that could otherwise escape it. This script is non executable by default but anyway won't do anything until /etc/cgrules.conf be edited (all lines are commented out as shipped)

rc.cgmanager (cgmanager) starts the cgmanager daemon that provides an alternate way to manage the control group hierarchy through D-Bus requests possibly through the cgm front-end utility. Then, among other tasks, cgmfs is mounted on directory populated by controllers:
Code:
```
didier[~]$ df -h|grep cgmfs
cgmfs              100K       0  100K   0% /run/cgmanager/fs
didier[~]$ ls /run/cgmanager/fs
blkio  cpuacct  devices  memory   net_prio    pids
cpu    cpuset   freezer  net_cls  perf_event
didier[~]$
```
... Which is somehow redundant with what we see in /sys/fs/cgroups.
This daemon should be started for the cgm command to work, but as said in the first post not for ConsoleKit2 to work, and anyway cgmanager is an optional dependency of ConsoleKit2.

rc.cgproxy (cgmanager) starts the cgproxy daemon, that allows programs and users in a container to make cgroup administration requests using DBus calls, sending the user-ids or group-ids as SCM credentials to the cgmanager.

According to the README, "one of the driving goals (of cgmanager was) to enable nested lxc as simply and safely as possible, allowing to remove a large chunk of code from lxc. But as the devs of lxc themselves have declared cgmanager obsolete (probably as using the features of cgroup-v1 or v2 is good enough), this main driving goal disappeared.

So to answer some posts:

Quote:

Originally Posted by atelszewski

Can you name something that should fail to work if ConsoleKit2 fails to work?

Yes: rebooting or halting the system graphically as a regular user, e.g. using wm-logout.

Quote:

Originally Posted by atelszewski

I have all rc.cg* disabled since day 1:

I explained above why: if you don't set the associated config files rc.config and rc.cgred do nothing but still the control groups are fully usable. Similarly, if you don't make cgmanager executable the directory /run/cgmanager/fs won't be populated, but that doesn't prevent e.g. ConsoleKit2 to work.

Quote:

Originally Posted by upnort

@Didier Spaier: I have no comment about your original post. I only am adding that I concur with atelszewski. I never had any of the 4 cg* rc.d scripts enabled.

As stated above, this doesn't matter in most use cases, but does in some, e.g. if one wants to use the cgm command, see post #9 from Chris. In other words I agree with Chris' answer in post #13. And I also agree with what he said in post #15: non-use by some (even many) users should not, by itself, be the reason for removing a package. Else most Slackware packages should be removed

Quote:

Originally Posted by Richard Cranium

Once upon a time, docker really did need cgmanager to run.

http://www.linuxquestions.org/questi...nt-4175580217/

I will look at that, thanks.

Other input are still welcome (we are not in a hurry to get to a conclusion), and incidentally I am still eager to get real-life examples of multiseat usage.

BratPit · 02-18-2017, 10:33 AM

I think for normal usage /desktop/ all cg* demons can be off.

There is an option in kernel /default on in Slackware/ named

Quote:

SCHED_AUTOGROUP

This will automatically configure the Linux feature (cgroups) to group together related processes so that, under heavy load, certain processes still remain performant under the kernel scheduler.

but if someone wants to manage it and change it from this are those demons.
For me is too complicated . For systems with a lot containers with many processors for better performance and balance but for simple things autogroup is enough IMHO.

GazL · 02-20-2017, 09:18 AM

A word of caution on CONFIG_SCHED_AUTOGROUP: Process 'nice' values are effective only within the scope of a CGROUP: so if like me you're old-school and use nice -n 19 to run non-interactive workloads, it will no longer work if you enable AUTOGROUP.

BratPit · 02-20-2017, 10:40 AM

Yes

Quote:

Under group scheduling, a thread's nice value has an effect for scheduling decisions only relative to other threads in the same task group. This has some surprising consequences in terms of the traditional semantics of the nice value on UNIX systems. In particular, if autogrouping is enabled, then employing nice(1) on a process has an effect only for scheduling relative to other processes executed in the same session (typically: the same terminal window).
Conversely, for two processes that are (for example) the sole CPU-bound processes in different sessions (e.g., different terminal windows, each of whose jobs are tied to different autogroups), modifying the nice value of the process in one of the sessions has no effect in terms of the scheduler's decisions relative to the process in the other session.

Mixing things that do similar job provides unpredictable results .Better choose one.

So if you want to prevent autogrouping interfering with the traditional nice behavior you can disable the feature

Quote:

echo 0 > /proc/sys/kernel/sched_autogroup_enabled

Be aware though that this will also have the effect of disabling the benefits for desktop interactivity that the autogroup feature was intended to provide

chris.willing · 02-23-2017, 08:13 PM

I've been playing with trying to make unprivileged containers work without help of cgmanager since, as pointed out by Didier, cgmanager has been deprecated upstream. After rebooting without rc.cgmanager and instead running rc.cgconfig and rc.cgred startup scripts (having added appropriate stuff in their configuration files /etc/cg{config,rules}.conf), I was able this morning to run a previously created unprivileged container based on the default lxc-slackware template i.e. success.

As well as some further testing, I'd like to investigate some more sophisticated configurations before writing it all up into a howto but, in principle at least, I'm happy to drop my objection to removal of cgmanager boot time startup if it can be replaced by rc.cgconfig & rc.cgred being started at boot instead (which would be the logical thing to do if deprecating cgmanager in favour of libcgroup functionality).

chris

Didier Spaier · 02-24-2017, 09:16 AM

Thanks a lot for working at it Chris.

Meanwhile, I will see if if I can take inspiration from the download template to create the container directly as a regular user (needs to have <username>:100000:65537 in /etc/{subuid,subgid}) but I need to be able to access the jenkins server as indicated here and understand how all that works. Don't hold your breath

atelszewski · 02-24-2017, 09:33 AM

Hi,

Quote:

Originally Posted by Didier Spaier

but I need to be able to access the jenkins server

Why would you need Jenkins to test LXC?

--
Best regards,
Andrzej Telszewski

chris.willing · 02-24-2017, 03:28 PM

Quote:

Originally Posted by atelszewski

Why would you need Jenkins to test LXC?

That server provides premade unprivileged containers of various types that can be used immediately by ordinary users. The reason they are provided is because there are some features (from memory, device nodes & maybe other things) that cannot be created as a normal user. That's why I use a two step method to create an unprivileged container for Slackware; first create an ordinary privileged container, then convert it to unprivileged. I can do that on my own machine, whereas the image server caters to strictly ordinary users without root permissions (as well as those who don't want to start from absolute zero).

chris

atelszewski · 02-24-2017, 03:54 PM

Hi,

So what you actually need is the images served there, not Jenkins itself?

--
Best regards,
Andrzej Telszewski

chris.willing · 02-24-2017, 04:10 PM

Quote:

Originally Posted by atelszewski

So what you actually need is the images served there, not Jenkins itself?

That's right, although it looks like there are also image generation scripts there for a jenkins server to produce the actual download images.

chris

franzen · 02-25-2017, 03:04 PM

Hi,

i'm using systemwide(created by root) unpriviledged lxc-containers started by /etc/rc.d/rc.lxc start, see here.
I recompiled lxc on slackware 14.2 with cgmanager disabled, no problems so far.
The direct elf dependencies of lxc 2.0.7 without cgmanager are
glibc-solibs,gmp,gnutls,libcap,libffi,libidn,libunistring,nettle,p11-kit,zlib
By the way, with --enable-gnutls=no, direct elf dependencies are reduced to
glibc-solibs,libcap

Here is my diff to the official lxc buildscript from slackware 14.2.

Code:

Only in ../lxc.orig: lxc-2.0.1.tar.xz
Only in .: lxc-2.0.7.tar.gz
diff -ru ../lxc.orig/lxc.SlackBuild ./lxc.SlackBuild
--- ../lxc.orig/lxc.SlackBuild	2016-06-13 00:00:00.000000000 +0200
+++ ./lxc.SlackBuild	2017-02-25 21:04:04.936941017 +0100
@@ -24,7 +24,7 @@
 
 PKGNAM=lxc
 VERSION=${VERSION:-$(echo $PKGNAM-*.tar.?z* | rev | cut -f 3- -d . | rev | cut -f 2- -d -)}
-BUILD=${BUILD:-4}
+BUILD=${BUILD:-1}
 
 # Automatically determine the architecture we're building on:
 if [ -z "$ARCH" ]; then
@@ -96,6 +96,7 @@
   --infodir=/usr/info \
   --with-global-conf=/etc/lxc/lxc.conf \
   --with-rootfs-path=/var/lib/rootfs-lxc \
+  --enable-cgmanager=no \
   $python \
   --build=$ARCH-slackware-linux
 
@@ -139,7 +140,7 @@
 # Add a documentation directory:
 mkdir -p $PKG/usr/doc/${PKGNAM}-$VERSION
 cp -a \
-  AUTHORS CONTRIBUTING COPYING* INSTALL MAINTAINERS NEWS README* TODO \
+  AUTHORS CONTRIBUTING COPYING* INSTALL MAINTAINERS NEWS README* \
   doc/FAQ.txt \
   $PKG/usr/doc/${PKGNAM}-$VERSION
 
diff -ru ../lxc.orig/rc.lxc ./rc.lxc
--- ../lxc.orig/rc.lxc	2016-04-26 00:00:00.000000000 +0200
+++ ./rc.lxc	2017-02-21 16:43:44.000000000 +0100
@@ -17,11 +17,7 @@
     if [ "$(lxc-info -n $CONTAIN -c lxc.start.auto)" = "lxc.start.auto = 1"  ]; then
       if [ "$(/usr/bin/lxc-info -s -n $CONTAIN | grep STOPPED$)" ]; then
         echo "Starting LXC container ${CONTAIN}."
-        if [ -x /usr/bin/screen ]; then
-          /usr/bin/screen -dmS init-${CONTAIN} /usr/bin/lxc-start -n $CONTAIN
-        else
-          /usr/bin/lxc-start -n $CONTAIN -d
-        fi
+        /usr/bin/lxc-start -n $CONTAIN -d
         /usr/bin/lxc-wait -n $CONTAIN -s RUNNING
         if [ $? -gt 0 ]; then
           return 2

chris.willing · 02-25-2017, 04:59 PM

Quote:

Originally Posted by franzen

Hi,

i'm using systemwide(created by root) unpriviledged lxc-containers started by /etc/rc.d/rc.lxc start, see here.
I recompiled lxc on slackware 14.2 with cgmanager disabled, no problems so far.

What I have been concerned about so far is the case of 'unprivileged container started by ordinary user' but it's good to know that the 'unprivileged container started by root' case which you are describing is already working.

chris

chris.willing · 02-26-2017, 10:48 PM

I've done a write up about setting up to run unprivileged containers as ordinary user using libcgroup facilities rather than cgmanager (with the idea that cgmanager, now deprecated upstream, can eventually be disabled). It's at http://www.darlo.tv/lxc/setup-unpriv-slackware.html. Please note that it's basically a rewrite of the old cgmanager based article, all the important business being in Part 1. I'll look at parts 2 & 3 later but I don't expect them to change much - it's the first part that has the important changes.

Please try it if you have time and pass on any errors or problems. If you don't want the work of creating a container by hand (as in Part 2) then, having completed part 1, you should be able to download a premade container using a command like:

Code:

lxc-create -t download -n p1 -- -d ubuntu -r trusty -a amd64

then run it with

Code:

lxc-start -n p1

and then

Code:

lxc-attach -n p1

chris

Didier Spaier · 02-28-2017, 06:02 PM

Thanks Chris, will try tomorrow. Meanwhile I have modified /usr/share/lxc/templates/lxc-slackware to allow creating a container privileged or unprivileged as you like, only depending on being root or not when typing the lxc-{create,start,stop,attach,console,destroy...} commands.

I borrowed code snippets from lxc-busybox. It works, with some limitations, as I couldn't have all devices owned by root, see the pic attached. At least it starts and stop nicely. Instructions for use: see on top of the patch.