[SOLVED] Default kernel preemption mode revisited.

JayByrd · 01-25-2023, 12:46 PM

Some of you may recall a "debate" here several months back about Pat's decision to make the default preemption mode "voluntary." From the Slackware 15.0 ChangeLog:

Quote:

...the default in the
kernel sources is "full" (which is probably not a good default)...

Consequently, shortly thereafter, we see this in the ChangeLog:

Quote:

The primary differences with the previous (5.14.15) kernel:
The default preemption mode is changed to "voluntary".

Now, the way this change was implemented by the kernel-source.SlackBuild was via an in-line sed substitution:

Code:

CONFIG_PREEMPT_DEFAULT_MODE=${CONFIG_PREEMPT_DEFAULT_MODE:-voluntary}

...<snip>...

echo "Setting default PREEMPT mode: $CONFIG_PREEMPT_DEFAULT_MODE"
sed -i "s/^int preempt_dynamic_mode = preempt_dynamic_full;$/int preempt_dynamic_mode = preempt_dynamic_$CONFIG_PREEMPT_DEFAULT_MODE;/g" kernel/sched/core.c

This worked as intended to change the default to "voluntary."

What I have subsequently discovered while building 6.1+ kernels, is that this sed is no longer having any effect. Starting with linux-5.16.1, the file core.c now reads:

Quote:

int preempt_dynamic_mode = preempt_dynamic_undefined;

Thus, Pat's sed line in the SlackBuild doesn't match the pattern, and so no substitution is taking place.

And what is "undefined"? Scrolling down a few lines in core.c, it seems that "undefined" actually defaults to "full," same as ever.

So, from my admittedly amateurish perspective, if we want to ensure a default of "voluntary," the sed line in question perhaps should be changed to something like:

Code:

sed -i "s/^int preempt_dynamic_mode = preempt_dynamic_.*;$/int preempt_dynamic_mode = preempt_dynamic_$CONFIG_PREEMPT_DEFAULT_MODE;/g" kernel/sched/core.c

Thoughts?

volkerdi · 01-25-2023, 01:09 PM

Thanks for spotting this. I'll be applying your change to kernel-source.SlackBuild.

JayByrd · 01-25-2023, 01:26 PM

Wow, that was quick!

For the record, I went a little more elaborate for my own use and put the info into the echo, so I could stay apprised of any further changes from Linus and team

Code:

KSRC_MODE="$( grep "int preempt_dynamic_mode = preempt_dynamic_" kernel/sched/core.c | rev | cut -f1 -d '_' | cut -f2 -d ';' | rev )"
echo "Changing default PREEMPT mode from \"$KSRC_MODE\" to: $CONFIG_PREEMPT_DEFAULT_MODE"
sed -i "s/^int preempt_dynamic_mode = preempt_dynamic_$KSRC_MODE;$/int preempt_dynamic_mode = preempt_dynamic_$CONFIG_PREEMPT_DEFAULT_MODE;/g" kernel/sched/core.c

drumz · 01-25-2023, 02:06 PM

Instead of patching the source why not just enable

Code:

CONFIG_PREEMPT_VOLUNTARY=y

?

From kernel/sched/core.c:

Code:

static void __init preempt_dynamic_init(void)
{
	if (preempt_dynamic_mode == preempt_dynamic_undefined) {
		if (IS_ENABLED(CONFIG_PREEMPT_NONE)) {
			sched_dynamic_update(preempt_dynamic_none);
		} else if (IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)) {
			sched_dynamic_update(preempt_dynamic_voluntary);
		} else {
			/* Default static call setting, nothing to do */
			WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT));
			preempt_dynamic_mode = preempt_dynamic_full;
			pr_info("Dynamic Preempt: full\n");
		}
	}
}

I build my kernels using a very lightly modified Slackware .config and pristine upstream source (I wasn't even aware Pat's build script did this patching!). But I have "preempt=voluntary" in my boot cmdline to accomplish the same thing:

Code:

# grep -i preempt /var/log/dmesg
[    0.000000] Linux version 6.1.7-etr (erich@Thelio-PC) (gcc (GCC) 11.2.0, GNU ld version 2.37-slack15) #1 SMP PREEMPT_DYNAMIC Wed Jan 18 21:28:49 CST 2023
[    0.000000] Command line: BOOT_IMAGE=dev002:\EFI\SLACKWARE\bz617e.img root=/dev/mapper/luksnvme0n1p5  vga=normal preempt=voluntary delayacct mitigations=off nvidia-drm.modeset=1 ro
[    1.165976] Kernel command line: BOOT_IMAGE=dev002:\EFI\SLACKWARE\bz617e.img root=/dev/mapper/luksnvme0n1p5  vga=normal preempt=voluntary delayacct mitigations=off nvidia-drm.modeset=1 ro
[    1.167171] Dynamic Preempt: voluntary
[    2.622867] rcu: Preemptible hierarchical RCU implementation.

If I were to set CONFIG_PREEMPT_VOLUNTARY I could remove "preempt=voluntary".

JayByrd · 01-25-2023, 02:19 PM

@drumz

As you say, if you set CONFIG_PREEMPT_VOLUNTARY in the config, you no longer need to specify on the command line.

However, my suspicion is that by setting this in the config, "voluntary" mode is made obligatory--i.e., you also lose the ability to override it on the command line with "preempt=full"...?

In any event, I think Pat's approach to it works just fine:

Code:

# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y
CONFIG_PREEMPT_DYNAMIC=y
CONFIG_SCHED_CORE=y

in the config, coupled with "voluntary" default, override-able on the kernel command line...

drumz · 01-25-2023, 02:52 PM

Quote:

Originally Posted by JayByrd

@drumz

As you say, if you set CONFIG_PREEMPT_VOLUNTARY in the config, you no longer need to specify on the command line.

However, my suspicion is that by setting this in the config, "voluntary" mode is made obligatory--i.e., you also lose the ability to override it on the command line with "preempt=full"...?

In any event, I think Pat's approach to it works just fine:

Code:

# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y
CONFIG_PREEMPT_DYNAMIC=y
CONFIG_SCHED_CORE=y

in the config, coupled with "voluntary" default, override-able on the kernel command line...

To me it looks like it doesn't matter which order preempt_dynamic_init() and setup_preempt_mode() are called in, the command-line option will take priority:

Code:

static int __init setup_preempt_mode(char *str)
{
	int mode = sched_dynamic_mode(str);
	if (mode < 0) {
		pr_warn("Dynamic Preempt: unsupported mode: %s\n", str);
		return 0;
	}

	sched_dynamic_update(mode);
	return 1;
}
__setup("preempt=", setup_preempt_mode);

static void __init preempt_dynamic_init(void)
{
	if (preempt_dynamic_mode == preempt_dynamic_undefined) {
		if (IS_ENABLED(CONFIG_PREEMPT_NONE)) {
			sched_dynamic_update(preempt_dynamic_none);
		} else if (IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)) {
			sched_dynamic_update(preempt_dynamic_voluntary);
		} else {
			/* Default static call setting, nothing to do */
			WARN_ON_ONCE(!IS_ENABLED(CONFIG_PREEMPT));
			preempt_dynamic_mode = preempt_dynamic_full;
			pr_info("Dynamic Preempt: full\n");
		}
	}
}

setup_preempt_mode gets called based on the command-line parameter. It overwrites the preemption mode with the command-line mode (assuming it's valid).

preempt_dynamic_init takes care to make sure preempt_dynamic_mode isn't left in the preempt_dynamic_undefined state. If preempt_dynamic_mode is set to anything else, it has no effect.

So if the call order is:
1. setup_preempt_mode (boot command-line parameter)
2. preempt_dynamic_init
Then:
1. Gets set to boot command-line parameter
2. NOOP

If the call order is:
1. preempt_dynamic_init
2. setup_preempt_mode (boot command-line parameter)
Then:
1. a. Without Pat's kernel source patch: Set based on kernel CONFIG.
1. b. With Pat's kernel source patch: NOOP.
2. Set based on boot command-line parameter (overrides previous setting).

So actually Pat's patch makes the kernel ignore the CONFIG_PREEMPT/CONFIG_PREEMPT_NONE/CONFIG_PREEMPT_VOLUNTARY setting!

Edit to add: the source code I'm showing is from kernel 6.1.8.

Aeterna · 01-25-2023, 02:57 PM

This is (I think) solved by selecting
Preempt dynamic.
If not modified it will default to full preemption. To change it to voluntary one needs to append it in boot loader. Setting preempt dynamic will let you add your choice of preemption in boot loader.

JayByrd · 01-25-2023, 03:42 PM

Quote:

Originally Posted by drumz

...So actually Pat's patch makes the kernel ignore the CONFIG_PREEMPT/CONFIG_PREEMPT_NONE/CONFIG_PREEMPT_VOLUNTARY setting!...

True.

Then again, in Pat's config, CONFIG_PREEMPT_NONE/CONFIG_PREEMPT_VOLUNTARY are already disabled, so there's nothing to ignore.

From the end-user's perspective, whether the default is set in the config or by patching core.c, the result for the official Slackware kernel packages is the same: "voluntary" is the preemption mode. (unless overridden on the command line.)

But I guess if one is compiling the kernel oneself and altering Pat's config file in the process, then your method of just setting CONFIG_PREEMPT_VOLUNTARY=y in the config is probably preferable to patching the kernel source.

drumz · 01-25-2023, 03:49 PM

Quote:

Originally Posted by JayByrd

But I guess if one is compiling the kernel oneself and altering Pat's config file in the process, then your method of just setting CONFIG_PREEMPT_VOLUNTARY=y in the config is probably preferable to patching the kernel source.

Sorry yes this was my (implicit) goal.