[SOLVED] Trouble with kernel 5.4.65 and docker. Unable to find "memory" in controller set.

Pigi_102 · 09-24-2020, 07:14 AM

Hi all,
after migration to Pi4 and upgrade the kernel, to make it possible, I started to get :

Code:

docker: Error response from daemon: unable to find "memory" in controller set: unknown.

If it happens to you, the motivation is that in kernel is set to yes the

Code:

CONFIG_MEMCG=y

but the cgroup memory is disabled in kernel code, as per
https://github.com/raspberrypi/linux...ment-156697067
The fix took me some our to find, but is rather an easy one:
add cgroup_enable=memory in /boot/cmdline.txt and reboot.

I write this here as it may helps others.

Cheers

Pigi

Exaga · 09-25-2020, 02:59 AM

Quote:

Originally Posted by Pigi_102

Hi all,
after migration to Pi4 and upgrade the kernel, to make it possible, I started to get :

Code:

docker: Error response from daemon: unable to find "memory" in controller set: unknown.

If it happens to you, the motivation is that in kernel is set to yes the

Code:

CONFIG_MEMCG=y

but the cgroup memory is disabled in kernel code, as per
https://github.com/raspberrypi/linux...ment-156697067
The fix took me some our to find, but is rather an easy one:
add cgroup_enable=memory in /boot/cmdline.txt and reboot.

Hi Pigi, I can't say I've ever come across this problem.

The 5.4.65 kernel .config CONFIG_CGROUPS and CONFIG_MEMCG settings are:

Code:

#
# Scheduler features
#
# CONFIG_UCLAMP_TASK is not set
# end of Scheduler features

CONFIG_CGROUPS=y     <<< set
CONFIG_PAGE_COUNTER=y
CONFIG_MEMCG=y     <<< set
# CONFIG_MEMCG_SWAP is not set     <<< unset
CONFIG_MEMCG_KMEM=y     <<< set

Pigi_102 · 09-25-2020, 03:35 AM

Yes, Exaga, and that's fine !

The problem is that in the kernel code there is:

Code:

@@ -5388,6 +5388,7 @@ struct cgroup_subsys memory_cgrp_subsys = {
	.dfl_cftypes = memory_files,
	.legacy_cftypes = mem_cgroup_legacy_files,
	.early_init = 0,
        .disabled = 1,
};

This means that when you boot the kernel, the option is provided, so you find them in :

Code:

# cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
...
memory  0       1       0

...

Thus dockerd try to use them but fail.

There are two option to workaround this problem:
1) unset the option and rebuild the kernel ( too much time consuming and must be repeated for every kernel upgrade )

2) add the cgroup_enable=memory in the cmdline.txt and enable the presence and use of memory cgroups in the running kernel, which is what I did.

with this option enabled, now I have:

Code:

# cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
...
memory  5       4       1
...

And that's why I have created this post, and marked as SOLVED

I've searched a lot but find really few about my problem so I decided to write the post in case others have the same problem.
Not sure how much are using docker on Pi, and if someone would use "your" new kernel can have the same issue and find here the fix

Pigi