LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 07-12-2010, 01:04 PM   #1
adjutrix
LQ Newbie
 
Registered: May 2009
Posts: 6

Rep: Reputation: 0
kernel q re: user-space memory handling


This is probably a q for kernel cognoscenti.

Gist: "top" summary line for Mem shows "used" constantly rises to
near-total, then drops, then rises again, repeat;
meanwhile, application being monitored gets slower & slower.

I'm trying to track down cause of slowdown in app; behavior suggests
quadratic algorithm in that the larger the database is creates, the
slower it gets, however I think I've ruled that out and was suspecting
heap fragmentation, so...

I'm running top -p <app process id> while the app is running, I thought
I'd see if its mem consumption grows as its progress report slows down.

But:
VIRT, RES, SHR, CODE, DATA %MEM don't seem to change.
Meanwhile, in the Mem summary line, I see the "used" number
slowly but consistently, monotonically increase until it reaches
something very close to the 11.5G total. Then it drops down
and repeats this behavior.
(Also, nFLT and nDRT remain 0 all the time.)

I can't find any documentation on the meaning of the fields in that
summary line, for one thing.
I'm guessing that (perhaps since I have little else running on this
machine (8 cores with hyper-threading -> 16 threads total), the
kernel is willing to allocate more and more phys memory to the process
until it reaches a threshold, perhaps?

Can a kernel guru tell me what this data from top might indicate is
going on in the kernel?


Another thing I noticed is that as time goes on, more and more often
top shows that all threads in the process are sleeping but one,
which is in log_wait.
The only thing I have found about log_wait is that it's a
synchro variable for syslog, but none of *my* code is using
syslog. Aside from the possibility that some library code is using
syslog (no messages show up in the logs, by the way), is there
something in the kernel that would necessarily wait on log_wait
for an app that does almost entirely these kernel- or near-kernel
operations:
pread, write
pthread_* operations

I haven't yet found any wikis or tutorials that would shed light
on the matter, but if anyone knows of anything that might, please send
a pointer my way.

Thanks for any ideas...
 
Old 07-15-2010, 02:46 AM   #2
WildPossum
Member
 
Registered: Feb 2004
Location: Sydney - Australia
Distribution: OpenSUSE, Ubuntu, Mythbuntu, iMedia, Embedded Linux
Posts: 44

Rep: Reputation: 18
Smile Top :: Memory field breakdown

The following fields are:
VIRT = Virtual Memory available = Real Memory + Swap Area.
RES = Residential Memory = Amount of memory existing in core memory.
SHR = Shared Memory = Code data that is shared (read only) across more than one process.
CODE = Code Memory = code/executable memory.
DATA = Data Memory = memory holding variables.
%MEM = % of (total usable) memory currently available.

So, I wouldn't expect VIRT, RES,SHR,CODE,or %MEM to change much at all for only one type of CPU bound process. Maybe DATA would move minimally as new variables are acquired/used.

You can fins more information on these labels/meaning by looking into the top source code, inside the linux doc system "aka: man top", as well as in /usr/src/kernel/Docs, or goto URL http://linux.die.net/man/1/top

As you correctly pointed out, the linux kernel will usually use all of available free memory rather than waste time cleaning up (freeing memory allocation tables) or initialing (possibly zeroing) used memory - it is more efficient this way (as measured by total execution time) for each process/thread execution.

You can see how the memory management will handle memory requests by the memory size block allocations as per slab in /proc fs. Have a look i.e. cat /proc/slabinfo and cat /proc/meminfo for more information of memory relationships.

Hope this assists you.
Cheers. Grahame
 
Old 07-16-2010, 11:51 AM   #3
adjutrix
LQ Newbie
 
Registered: May 2009
Posts: 6

Original Poster
Rep: Reputation: 0
Thanks; I had already exhausted the doc for "top", alas.
What didn't make sense to me was to see the summary info for memory grow
while the %MEM for the process didn't.


As for the wchan, I figured out that it was truncating the symbol, so it wasn't
really log_wait; it was log_wait_something_else_that_i_dont_recall
in fact.
Arg.

In the old days, I'd have just read the kernel source, but it's not as simple
as it was in 1983...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Division of Logical Memory Space in to User Space and Kernel Space shreshtha Linux - Newbie 2 01-14-2010 09:59 AM
mapping contiguous kernel memory to user space jp03833 Linux - Kernel 0 08-29-2009 08:54 AM
how to call socket prog code written in user space from kernel space???HELP kurt2 Programming 2 07-15-2009 09:56 PM
Sharing Memory between driver space and user space ian b davidson Programming 1 11-10-2008 06:35 AM
mapping contiguous kernel memory to user space nili Linux - Kernel 4 01-29-2008 01:43 PM


All times are GMT -5. The time now is 06:42 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration