LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-29-2010, 11:03 AM   #1
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Rep: Reputation: 39
HTOP and PID's


Hi All,

Just trying to figure out some stuff with a broken process.
A java app seems to sometime get stuck on a loop or something and i'm trying to find out what's causing it using just the following #sysadmin tools at my disposal.

Things like:-
htop - find the PID thats causing the High CPU cycles.
I'd then want to use /proc/[PID] or lsof -p [PID] or strace [PID] etc...

But the PID doesn't exist in 'ps -ef' output so I think htop must be showing me kernel level thread PIDs?

Not sure about the PID's HTOP is actually giving me? I know that some of them are the real PID's that can be accessed through /proc/[pid] etc but others are not but are i assume child processes or more likely threads as child processes are normally shown in a default ps output anyway.

Is someone able to help distinguish to me about what exactly all these other PID's are that I can't manipulate or find apart from when using HTOP.

Cheers,
M
 
Old 12-29-2010, 01:39 PM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Can you please post an example of htop's output?

Alternatively, use ps ...
Code:
ps -e -o pid,pcpu,comm|sort -k2,2n


Cheers,
Tink
 
Old 12-29-2010, 02:38 PM   #3
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
OK - Cheers for the reply. First off...

HTOP
Code:
  1  [|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%]     Tasks: 199 total, 2 running
  2  [|                                                                                                      0.7%]     Load average: 1.92 2.02 1.98 
  Mem[||||||||||||||||||||||||||||||||||||||||||                                                       685/7680MB]     Uptime: 91 days, 12:41:17
  Swp[                                                                                                      0/0MB]

  PID USER     PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command                                                                                                                                                                      
31095 root      20   0 2510M  288M  9712 R 99.0  3.7 98h25:07 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar.jar 13676 8443                     
 9493 root      20   0 19880  1760  1104 R  1.0  0.0  0:00.48 htop
31059 root      20   0 2510M  288M  9712 S  0.0  3.7  1h41:58 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
31008 root      20   0 2510M  288M  9712 S  0.0  3.7  3h28:18 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
14021 syslog    20   0  297M  2672  1304 S  0.0  0.0  2:31.55 rsyslogd -c4
 9511 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.03 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9512 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.02 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
  527 zabbix    25   5 22344  1208   952 S  0.0  0.0  1h27:44 /opt/zabbix/sbin/zabbix_agentd
31140 root      20   0 2510M  288M  9712 S  0.0  3.7  1h38:41 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9508 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.03 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9509 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.03 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
31148 root      20   0 2510M  288M  9712 S  0.0  3.7  1h54:11 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
14020 syslog    20   0  297M  2672  1304 S  0.0  0.0  2:53.56 rsyslogd -c4
 9502 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.04 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9505 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.05 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9506 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.05 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9501 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.03 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9503 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.03 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
 9504 root      20   0 2510M  288M  9712 S  0.0  3.7  0:00.02 /usr/bin/java -Xms2g -Xmx2g -Xmn150m -XX:GCTimeRatio=2 -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:MaxGCPauseMillis=2000 -jar name.jar 13676 8443
  526 zabbix    25   5 18192   944   752 S  0.0  0.0  1h18:34 /opt/zabbix/sbin/zabbix_agentd
    1 root      20   0 23704  1916  1272 S  0.0  0.0  0:16.42 /sbin/init
    2 root      20   0     0     0     0 S  0.0  0.0  0:00.00 kthreadd
    3 root      RT   0     0     0     0 S  0.0  0.0  0:01.60 migration/0
    4 root      20   0     0     0     0 S  0.0  0.0  0:26.26 ksoftirqd/0
    5 root      RT   0     0     0     0 S  0.0  0.0  0:00.52 watchdog/0
    6 root      20   0     0     0     0 S  0.0  0.0  4:08.46 events/0
    7 root      20   0     0     0     0 S  0.0  0.0  0:00.00 cpuset
    8 root      20   0     0     0     0 S  0.0  0.0  0:00.00 khelper
    9 root      20   0     0     0     0 S  0.0  0.0  0:00.00 netns
   10 root      20   0     0     0     0 S  0.0  0.0  0:00.00 async/mgr
   11 root      20   0     0     0     0 S  0.0  0.0  0:00.00 xenwatch
   12 root      20   0     0     0     0 S  0.0  0.0  0:00.00 xenbus
   14 root      RT   0     0     0     0 S  0.0  0.0  0:01.65 migration/1
   15 root      20   0     0     0     0 S  0.0  0.0  0:12.64 ksoftirqd/1
   16 root      RT   0     0     0     0 S  0.0  0.0  0:00.64 watchdog/1
   17 root      20   0     0     0     0 S  0.0  0.0  3:44.78 events/1
   18 root      20   0     0     0     0 S  0.0  0.0  0:07.86 sync_supers
PS
Code:
# ps -e -o pid,pcpu,comm|sort -k2,2n
   10  0.0 async/mgr
10024  0.0 apache2
    1  0.0 init
   11  0.0 xenwatch
   12  0.0 xenbus
   14  0.0 migration/1
   15  0.0 ksoftirqd/1
   16  0.0 watchdog/1
   17  0.0 events/1
  171  0.0 udevd
  172  0.0 udevd
   18  0.0 sync_supers
   19  0.0 bdi-default
   20  0.0 kintegrityd/0
    2  0.0 kthreadd
   21  0.0 kintegrityd/1
21337  0.0 apache2
   22  0.0 kblockd/0
   23  0.0 kblockd/1
   24  0.0 kseriod
   27  0.0 khungtaskd
   29  0.0 kswapd0
  297  0.0 dhclient3
   30  0.0 aio/0
    3  0.0 migration/0
30995  0.0 apache2
   31  0.0 aio/1
31223  0.0 apache2
   32  0.0 jfsIO
  325  0.0 flush-8:1
   33  0.0 jfsCommit
   34  0.0 jfsCommit
   35  0.0 jfsSync
   36  0.0 xfs_mru_cache
   37  0.0 xfslogd/0
  376  0.0 kjournald
   38  0.0 xfslogd/1
   39  0.0 xfsdatad/0
   40  0.0 xfsdatad/1
    4  0.0 ksoftirqd/0
  404  0.0 sshd
  410  0.0 dbus-daemon
   41  0.0 xfsconvertd/0
   42  0.0 xfsconvertd/1
   43  0.0 glock_workqueue
  438  0.0 avahi-daemon
  439  0.0 avahi-daemon
   44  0.0 glock_workqueue
   45  0.0 delete_workqueu
   46  0.0 delete_workqueu
  461  0.0 auditd
  463  0.0 audispd
  464  0.0 kauditd
   47  0.0 kslowd000
   48  0.0 kslowd001
   49  0.0 crypto/0
   50  0.0 crypto/1
    5  0.0 watchdog/0
  511  0.0 atd
  512  0.0 cron
 5187  0.0 apache2
 5194  0.0 apache2
 5195  0.0 apache2
 5197  0.0 apache2
 5198  0.0 apache2
 5245  0.0 apache2
  525  0.0 zabbix_agentd
  526  0.0 zabbix_agentd
  527  0.0 zabbix_agentd
  528  0.0 zabbix_agentd
  529  0.0 zabbix_agentd
  530  0.0 zabbix_agentd
   53  0.0 net_accel/0
   54  0.0 net_accel/1
  545  0.0 mysqld
   55  0.0 sfc_netfront/0
   56  0.0 sfc_netfront/1
   57  0.0 kjournald
    6  0.0 events/0
 6020  0.0 apache2
  661  0.0 getty
  671  0.0 console-kit-dae
    7  0.0 cpuset
 7474  0.0 apache2
    8  0.0 khelper
   85  0.0 upstart-udev-br
   87  0.0 udevd
    9  0.0 netns
 9297  0.0 sshd
 9361  0.0 sshd
 9362  0.0 bash
 9401  0.0 su
 9408  0.0 bash
 9739  0.0 ps
 9740  0.0 sort
  PID %CPU COMMAND
11675  0.1 rsyslogd
30996 22.4 java
So htop when in tree view would show this PID as a child of Java process 30996, which is stuck in a loop or something seeing as its 99% CPU. Easy for me to fix I can just re-start the Java App and everything would be working again. But i'm using this as a little exercise to see if I can find what's going on - but more importantly try to understand why htop shows more results than a PS output. My assumption is still that htop is able to show and work out the threads that a process has or something, but i'm still yet unsure of why i would be unable to diagnose these further...

Thanks
 
Old 12-29-2010, 02:50 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
I'd stop using htop; it's obviously dreaming up PIDs. Use top instead.

I verified that it's being stupid on my machine (installed 0.9 on my
machine from source, just out of curiosity). It's showing processes
that clearly don't exist (/proc/<PID> is authoritative), and no, xmms
(htop showed my system to have 3 xmms processes) doesn't have any
phantom kernel level threads ;} ...



Cheers,
Tink

Last edited by Tinkster; 12-29-2010 at 02:53 PM.
 
Old 12-29-2010, 04:08 PM   #5
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
Cheers Tink - perhaps yo can explain this area and post a little further..

Sorry to Link to other forum ;-)

http://serverfault.com/questions/241...-ps-aux-doesnt

And then this command in particular and others:
Quote:
To highlight the difference between processes and threads in htop:

1. press F2
2. select "Display options"
3. enable "Display threads in a different color"
Most people talking of the ability for Htop to pick-up on Kernel or UserLand Threads. Maybe its just something HTOP does or something but doesn't really mean much? Any ideas?

I mean what Htop is showing me in output is very related to what i'd expect to see to an extent.. Its just confusing as i say that i'm not able to diagnose further the so called 'threads' how you would a normal PID...

Cheers,
M
 
Old 12-29-2010, 04:18 PM   #6
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
Also -

cd /proc/30996

cat status

Shows in their :

Threads : 66

When i went into htop and tree view and manually counted them... i counted 69 threads, based on the point the number of threads is always changing but would probably stay around this number, I make the assumption the two numbers are closely related?

I'm now looking about the proc/30996 files etc to see if i can find anything more pertaining to these so called 'threads'...

EDIT:
Code:
ps -eLF
Shows all the same threads I can see with htop...

According to man page for ps...
-L = "Show threads, possibly with LWP and NLWP columns"
lwp (light weight process, or thread) ID of the lwp being reported. (alias spid, tid).
nlwp NLWP number of lwps (threads) in the process. (alias thcount).

OK, think i'm getting somewhere now.. would still be interesting to know if there's anyway to diagnose further the LWP then with strace/lsof/proc etc? Or i guess it has to be done from the PID- something to do with 2.6 kernel not showing everything as a single process like the 2.4 kernel does perhaps. (noting i've only ever been using Linux with a 2.6 kernel)

Last edited by helptonewbie; 12-29-2010 at 04:31 PM.
 
Old 12-29-2010, 05:14 PM   #7
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Interesting ... I just learnt something about htop ;}

So what htop outputs is the ID of the LWP (thread). The relevant
info in /proc for those can be found under
/proc/PID/task/LWP/ ...


Won't help w/ lsof or strace, I'm afraid. Just use the actual
PID w/ those, rather than the LWP ID.


Cheers,
Tink
 
1 members found this post helpful.
Old 12-30-2010, 01:42 AM   #8
helptonewbie
Member
 
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
Thanks Tink. That is good info cheers.

I'm also going to post a link to this site:-
http://www.fromdev.com/2008/12/debug...-favorite.html

Which has useful stuff related more to Java process debugging using the methods I was interested in to start with.

Cheers for everything.
M

Last edited by helptonewbie; 12-30-2010 at 01:45 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
error while try to install htop-0.8 Elnegro8 Linux - Software 5 08-26-2013 02:36 PM
Htop vs 'top' vs 'free' disparity schachwizard Ubuntu 2 09-22-2010 04:33 PM
htop execute and exit... chamila1986 Linux - Newbie 2 01-16-2010 03:21 PM
Looking for direction with reagarding a top/htop like app energyAlchemist Programming 3 09-22-2009 08:39 PM
need help to iterate through PID's mcnkevin Programming 7 11-24-2007 05:08 PM


All times are GMT -5. The time now is 09:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration