It seems that in future Linux kernel itself will force the use of systemd
 

From: Lennart Poettering <lennart <at> poettering.net>
Subject: [HEADSUP] cgroup changes
Newsgroups: gmane.comp.sysutils.systemd.devel
Date: 2013-06-21 17:36:03 GMT (19 weeks, 5 days, 5 hours and 28 minutes ago)

Heya,

On monday I posted this mail:

http://lists.freedesktop.org/archive...ne/011388.html

Here's an update and a bit on the bigger picture:

Half of what I mentioned there is now in place. There's now a new
"slice" unit type in place in git, and everything is hooked up to
it. logind will now also keep track of running containers/VMs. The
various container/VM managers have to register with logind now. This
serves the purpose of better integration of containers/VMs everywhere
(so that "ps" can show for each process where it belongs to). However,
the main reason for this is that this is eventually going to be the only
way how containers/VMs can get a cgroup of their own.

So, in that context, a bit of the bigger picture:

It took us a while to realize the full extent how awfully unusable
cgroups currently are. The attributes have way more interdependencies
than people might think and it is trivial to create non-sensical
configurations...

Of course, understanding how awful the status quo is a good first
step. But we really needed to figure out what we can do about this to
clean this up in the long run, and how we can get to something useful
quickly. So, after much discussion between Tejun (the kernel cgroup
maintainer) and various other folks here's the new scheme that we want
to go for:

1) In the long run there's only going to be a single kernel cgroup
hierarchy, the per-controller hierarchies will go away. The single
hierarchy will allow controllers to be individually enabled for each
cgroup. The net effect is that the hierarchies the controllers see are
not orthogonal anymore, they are always subtrees of the full single
hierarchy.

2) This hierarchy becomes private property of systemd. systemd will set
it up. Systemd will maintain it. Systemd will rearrange it. Other
software that wants to make use of cgroups can do so only through
systemd's APIs. This single-writer logic is absolutely necessary, since
interdependencies between the various controllers, the various
attributes, the various cgroups are non-obvious and we simply cannot
allow that cgroup users alter the tree independently of each other
forever. Due to all this: The "Pax Cgroup" document is a thing of the
past, it is dead.

3) systemd will hide the fact that cgroups are internally used almost
entirely. In fact, we will take away the unit configuration options
ControlGroup=, ControlGroupModify=, ControlGroupPersistent=,
ControlGroupAttribute= in their entirety. The high-level options
CPUShares=, MemoryLimit=, .. and so on will continue to exist and we'll
add additional ones like them. The system.conf setting
DefaultControllers=cpu will go away too. Basically, you'll get more
high-level settings, but all the low level bits will go away without
replacement. We will take away the ability for the admin to set
arbitrary low-level attributes, to arrange things in completely
arbitrary cgroup trees or to enable arbitrary controllers for a service.

4) systemd git introduced a new unit type called "slice" (see
above). This is for partitioning up resources of the system into
slices. Slices are hierarchial, and other units (such as services, but
also containers/VMs and logged in users) can then be assigned to these
slices. Slices internally map to cgroups, but they are a very high-level
construct. Slices will expose the same CPUShares=, MemoryLimit=
properties as the other units do. This means resource management will
become a first-class, built-in functionality of systemd. You can create
slices for your customers, and in them subslices for their departments,
and then run services, users, vms in them. In the long run these will by
dynamically moveable even (while they are running), but that'll take
more kernel work. By default there will three slices: "system.slice"
(where all system services are located by default), "user.slice" (where
all logged in users are located by default), "machine.slice" (where all
running VMs/containers are located by default). However, the admin will
have full freedom to create arbitary slices and then move the other
units into them.

5) systemd's logind daemon already kept track of logged in
users/sessions. It is now extended to also keep track of virtual
machines/containers. In fact, this is how libvirt/nspawn and friends
will now get their own cgroups. They register as a machine, which means
passing a bit of meta info to systemd, and getting a cgroup assigned in
response. This registration ensures that "ps" and friends can show to
which VM/container a process belongs, but easily allows other tools to
query container/VM info too, so that we'll be able to provide an
integration level of containers/VMs like solaris zones can do it in the
long run.

So, this all together sounds like an awful lot of change. #1 and #2 are
long term changes. However #3, #4, #5 are something we can do now and
should do now, as prepartion for the single-writer, unified cgroup
tree. We really, really shouldn't ship the cgroup mess for longer, so
that people make use of the current systemd APIs that expose way too
many internal guts, stuff that we *know* right now is broken and will
cease to exist. We don't want to expose low-level details we already
know *now* we cannot support for long.

Even though #3, #4, #5 sound like major work they are not. In fact #4
and #5 are fully implemented on the systemd side already now upstream. I
am working on #3. I am confident that I'll have this finished in a few
days too, since this is really actually just about deleting code more
than writing code.

With #3, #4, #5 we have something in place that should do the basic
things and first and foremost will hide all the lower-level details of
cgroups. This has the big benefit of allowing us to rearrange these
details later without having to break the user or
programming interfaces, and that's what I really care about here.

Now, what does this mean for other projects using cgroups? So basically,
since we won't implement #1 + #2 immediately the cgroup tree stays
relatively open for other cgroup users. They can continue to fiddle with
it for now, but it must be clear that this is temporary, and that they
don't attempt too fancy things. Direct access to the cgroup tree is on
is way out and that must be clear to everybody.

More specifically: libcgroup is out of the game with
this. libvirt/openshift/lxc/.. can continue to do what they do for now,
however they should be updated sooner rather than later to do things the
systemd way, i.e. rely on systemd VM/container registration and user
cgroup management.

And to make one last thing clear: this time, it's not Kay and me who are
taking away the cgroup tree from everybody else, it's actually all
Tejun's fault as the kernel cgroup maintainer... ;-) He wants a unified,
single-writer hierarchy, and it took us a while to agree to that, but
we're now fully on the same page with him.

If you are using non-trivial cgroup setups with systemd right now, then
things will change for you. We will provide you with similar
functionality as before, but things will be different and less
low-level. As long as you only used the high-level options such as
CPUShares, MemoryLimit and so on you should be on the safe side.

I hope this makes some sense,

Lennart

--
Lennart Poettering - Red Hat, Inc.

Not Again
 

When one looks at all the commotion about systemd on this SLACKWARE forum an observer might conclude that systemd is a core component of Slackware ... but it's not.

In the future, the Linux Kernel will do whatever Linus Torvalds wants it to do.

I hope so too.

You have to consider the source of the message.

blancamolinos,
Would you care to point out where Pottering says "Linux kernel itself will force the use of systemd"?
Because there is no mention of "Linux" anywhere in his post.
Thanks.

The thing is: Pottering can say whatever he wants. It is not his decision to make.

It is interesting how many of the systemd OP's have less than 10 posts...

Lennart can claim all he wants but the Linux foundation and Linus Torvalds gives final authority over what does what.

The Linux kernel is just that... a kernel. Nothing more or less, and it can be used with any software that can be coded for it, which happens to be at the moment, The GNU Operating System, or the Android Operating System.

I was thinking the same thing just yesterday. Maybe Lennart has deployed systemd bots throughout the web to find these threads and respond/spread FUD.

Wouldn't surprise me, but then again, what really does anymore?

Just because it says systemd does not mean that's the only software that can handle the role. It seems to be assumed that systemd is the only available that can do it, and maybe that is the case for now. But it's a role to manage the cgroups, not a dependency. It just want to have ONE user space writer and some people seem to conclude it will be systemd.

I'm still hoping to rid of udev, soon.

You don't use bots for that. You set up google alerts.

Yeah, for sure...

And the kernel would be forked then.

It seems to be very important to someone, that Slackware adopts systemd. Maybe the whole "switch all distributions over" world domination plan gets destroyed by Slackware not switching. ;) Says something about the relevance of this distribution, which gets not very much media coverage.

Seeing this astroturfing, I trust the 0pointer code even less. Maybe someone wants to place a backdoor on every Linux box...

please read his mail list before posting.

he only says that the kernel cgroup architecture is changing.
and that systems running with systemd, needs to use the systemd API to change the value's.
otherwise systemd will not allow you to do so.

on non systemd systems, you can still make your own API

Not again. This post has absolutely nothing to do with Slackware...why is it that people who just want to troll systemd, pulseaudio or Lennart come to the Slackware forum to do it?

The important points are #1 and #2

As indicated by Skaperen, systemd is not strictly speaking a dependency of cgroups. Another piece of software can implement the necesary API, but someone will have to make and maintain this software.

Hi,

unfortunately, systemd has a lot to do with Linux distros (and soon kernel subsystems ?).
You just have to be vigilant about the nature of the evil you call light in the tunnel ...

http://www.linux.com/news/featured-b...roups-redesign

Quote:
"Systemd and cgroup developers are working together to turn systemd into a global cgroup manager that creates higher-level control knobs and prevents direct access to the kernel."

Btw, there is one picture missing there !

So, should we congratulate Debian to successful transition to systemd, and then ourselves ?
jb

yes, this

I bet most of you are not even using cgroups.

The kernel wouldn't just get forked, Linus himself could be put on blast with every instance of social media as to why he let it happen, and the Linux Foundation would be severely ridiculed for why they allowed this. It could become a PR nightmare.

The only people who would win would be Red Hat who would attempt to step in and seize control of Linux entirely, Microsoft who would laugh and claim Linux, GNU, and open source are all more fractured than ever, and FreeBSD who would be opening their doors to the flood of new users.

*Sarcastically* That's it! I declare that Lennart Poeterring be hit in the face with a banana creme custard pie upon sight!

I configure them out of my kernels to reduce scheduling overhead, along with the virtualisation and namespace stuff I simply don't need. The resource management they provide is very useful on large systems running multiple virtual machines/containers and hosting a large number of other disparate services, but they're simply not needed on a desktop / workstation machine.

Or, at least weren't needed until that which must not be named came along and subverted them for the purpose of tracking child processes.

What kind of improvements do you notice? All of my systems here are desktop workstations. One system runs VirtualBox VMs but only as needed, not 24/7.

Related, what kind of degradations would such a user notice with cgroups enabled in the kernel but not being knowingly used in any way?

Hi,

Here is one more good overview of the topic:
https://lwn.net/Articles/557082/
where the essence is described in first two paragraphs, ending with this
description of desired solution:
"...; the hope is to have a single, privileged process handling all of the cgroup management tasks. That process will, in turn, provide some sort of higher-level interface to the rest of the system."

Enter systemd ???
Our nimble incarnation of system init process, which provides a replacement for sysvinit, but also for pm-utils, inetd, acpid, syslog, watchdog, cron and atd, and obsoletes
ConsoleKit.
A wunderwaffe ! Why bother to search for God Particle under Alps ?

Yes, cgroups have a specific purpose, namely to control allocation of resources to groups
of processes, but not (courtesy of systemd) that ...
"Or, at least weren't needed until that which must not be named came along and subverted them for the purpose of tracking child processes."
@GazL: Well said, Sir !

jb

Isn't slack run bu one really smart dude? That one really smart dud will tell you what your init will be when he decides what that will be. It seems overly dramatic to worry that anyone else does no matter whi they are and what they say. You live in a benevolent dictatorship much like Mac users don't worry until that SC says so.

I don't think there would really be any discernible difference unless there are actually active cgroups to manage, but it seems logical to believe that the less the kernel has to think about the more efficient it should be, and removing the cgroup, virtulisation and namespace code from the execution path ought to save a few cycles here and there due to the simplified logic. Perhaps when the system is at full load it might amount to a percentage point or two, but that's pure speculation and I've got no hard facts to back up that guess. You'd need to benchmark this stuff to get a accurate understand of its true impact, and that's over my head.

What I can say is that when I tried CONFIG_SCHED_AUTOGROUP which puts things in their own cgroups, I wasn't happy with the results, and I found sensible use of 'nice' priorities and 'chrt' far more effective, but again I have no evidence to offer on that.

"I didn't need it so I turned it off", is probably all one should take from my comments. ;)