LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-01-2020, 07:59 AM   #1
c01ahib
LQ Newbie
 
Registered: May 2020
Posts: 10

Rep: Reputation: Disabled
Setting Kernel space to non-cacheable on x86_64 using IA_PAT MSR or page table walks


I am trying to disable the caching for the Linux operating system only, i.e., Kernel space. I figured out that there are two ways to do this:

1- Using MTRR: This turned out to be infeasible as MTRR allow disabling caching for ranges of physical addresses. I am using a 64bit Tinycore Linux on x86_64, which means that kernel space can be mapped to the whole RAM (unlike 32bit systems where space maps to low physical memory).

2- Using PAT: This seems to be the better option as it works on virtual pages. However I am not sure whether I should use the IA32_PAT register and how? or modify the page table entries for kernel pages and how?


Could anyone provide full code for disabling cache for kernel space only or at least instructions from using IA32_PAT or modifying page table entries?

I found this question:
https://stackoverflow.com/questions/...le-entry-flags
but does not include the full code and it is not answered.

I also found these questions for page table walks:
https://stackoverflow.com/questions/...age-table-walk, and https://stackoverflow.com/questions/...ory-management

Thank you!

Last edited by c01ahib; 05-05-2020 at 09:04 AM.
 
Old 05-02-2020, 02:50 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,964

Rep: Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265
Sounds like yet another xyproblem.
How about you tell us about the actual problem you think you have that you perceive needs fixing ?.
 
Old 05-02-2020, 08:14 AM   #3
c01ahib
LQ Newbie
 
Registered: May 2020
Posts: 10

Original Poster
Rep: Reputation: Disabled
problem

The problem is that I want the linux kernel not to use the cache while applications still use the cache. I figured I should set kernel space to non-cacheable and now I need help with that.
Do you know how to do that?
My question includes the things I thought of.
 
Old 05-02-2020, 04:08 PM   #4
IlyaK
Member
 
Registered: Jun 2017
Location: Northwest Russia
Distribution: Slackware 14.2
Posts: 82

Rep: Reputation: 61
>The problem is that I want the linux kernel not to use the cache
Why?

I am not kernel hacker, but from what I know kernel device driver authors disable caching when some device use MMIO: so, instead of "real" RAM they access device, hence reading same address several times may lead to different results.
I do not think it is not possible to disable cache for all cases without of kernel patching.

See
https://lwn.net/Articles/282250/
https://www.kernel.org/doc/Documentation/x86/pat.txt
 
Old 05-03-2020, 10:40 AM   #5
c01ahib
LQ Newbie
 
Registered: May 2020
Posts: 10

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by IlyaK View Post
>The problem is that I want the linux kernel not to use the cache
Why?

A research project: I am intensively testing the cache from user and I do not want to crash the kernel.

I am not kernel hacker, but from what I know kernel device driver authors disable caching when some device use MMIO: so, instead of "real" RAM they access device, hence reading same address several times may lead to different results.
I do not think it is not possible to disable cache for all cases without of kernel patching.

See
https://lwn.net/Articles/282250/
https://www.kernel.org/doc/Documentation/x86/pat.txt

I have been through these links. They basically show you how to allocate non-cacheable memory. What I need to do is mark already allocated kernel memory as uncacheable. I know this can be done through MTRR and PAT. I am not sure how to do this though.
 
Old 05-05-2020, 03:12 AM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,964

Rep: Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265Reputation: 3265
Quote:
Originally Posted by c01ahib View Post
The problem is that I want the linux kernel not to use the cache while applications still use the cache.
There is no "the cache" - from that sentence I'm guessing you're referring to the page cache. The kernel storage is allocated in the early code and is not page-able.
 
Old 05-05-2020, 05:15 AM   #7
IlyaK
Member
 
Registered: Jun 2017
Location: Northwest Russia
Distribution: Slackware 14.2
Posts: 82

Rep: Reputation: 61
>from that sentence I'm guessing you're referring to the page cache.
I assume c01ahib talks about RAM/CPU cache.

When CPU access RAM (because of MOV instruction of example) it stores data in cache (small static memory cache resides directly on CPU).
Actually, cache controller prefetches data from RAM, so in many cases data is already in the cache.

CPU uses an interesting strategy to invalidate cache, btw.

But some regions must never be cached because they are not real RAM, but device MMIO:
* You can't prefetch data from device
* If you read several times from the same region of RAM, you should have same value. But this is not true for devices.

So, there should be some way for software to tell CPU "please, never cache this region and never prefetch from it".

There are two tools for that:
* PAT: Each page may have "do not cache" attribute
* MTRR: model-specific register where you can store address of region not to be cached.

Driver developers use these tools because they know which regions are MMIO.

I think it is possible to patch kernel to mark all pages used by kernel as "do not cache", but it will make everything VERY VERY slow. Intel added caches 30 years ago for the good reason.
And it is not something you could do with some tool. You would need to rewrite some parts of kernel.

Talk to some kernel hacker who knows how memory subsystem works. Your question is not "Newbie"))
 
Old 05-05-2020, 08:55 AM   #8
c01ahib
LQ Newbie
 
Registered: May 2020
Posts: 10

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by IlyaK View Post
>from that sentence I'm guessing you're referring to the page cache.
I assume c01ahib talks about RAM/CPU cache.

When CPU access RAM (because of MOV instruction of example) it stores data in cache (small static memory cache resides directly on CPU).
Actually, cache controller prefetches data from RAM, so in many cases data is already in the cache.

CPU uses an interesting strategy to invalidate cache, btw.

But some regions must never be cached because they are not real RAM, but device MMIO:
* You can't prefetch data from device
* If you read several times from the same region of RAM, you should have same value. But this is not true for devices.

So, there should be some way for software to tell CPU "please, never cache this region and never prefetch from it".

There are two tools for that:
* PAT: Each page may have "do not cache" attribute
* MTRR: model-specific register where you can store address of region not to be cached.

Driver developers use these tools because they know which regions are MMIO.

I think it is possible to patch kernel to mark all pages used by kernel as "do not cache", but it will make everything VERY VERY slow. Intel added caches 30 years ago for the good reason.
And it is not something you could do with some tool. You would need to rewrite some parts of kernel.

Talk to some kernel hacker who knows how memory subsystem works. Your question is not "Newbie"))

Thanks for the clarification. That's exactly what I was aiming for. I was hoping I could do this without having to patch the kernel. I assumed writing a kernel module that modifies the page table entries for the kernel pages (https://stackoverflow.com/questions/...le-entry-flags) would be sufficient. However, I do not have the necessary experience for doing so and I thought someone could help.

I am using a lightweight linux which works fine when the CPU cache is fully disabled. Full-brown desktop linux freezes when the cache is disabled.

Last edited by c01ahib; 05-05-2020 at 09:00 AM. Reason: Lightweight linux
 
Old 05-05-2020, 08:56 AM   #9
c01ahib
LQ Newbie
 
Registered: May 2020
Posts: 10

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
There is no "the cache" - from that sentence I'm guessing you're referring to the page cache. The kernel storage is allocated in the early code and is not page-able.
I simply need the Kernel to stop using the CPU cache for it's own execution.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Failed access perfctr msr (MSR c0010004 is 0) dimasar2 Linux - Laptop and Netbook 5 04-08-2019 03:18 AM
wmlongrun needs CPUID/MSR drivers in kernel masuch Linux - Newbie 0 11-03-2011 04:10 PM
Making a memory block as non cacheable by the processor Jason Souza Linux - Newbie 4 08-04-2008 03:11 AM
Configuring alien memory as Cacheable/ Non Cacheable linuxguy_72 Linux - Kernel 0 05-04-2007 01:25 AM
how to allocate non-cacheable buffer? mehuljv Programming 2 05-21-2005 01:08 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:05 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration