Linux - EnterpriseThis forum is for all items relating to using Linux in the Enterprise.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Ive got an Enterprise Linux 4 system here and we are trying to tweak it to optimize performance forour Internet Monitoring system.Now we found out that the IRQ's were being distributed a bit unevenly in /proc/interrupts.So I decided to try and evn things out a bit :-
For eg.
Interrupt 0 was for all CPU's ..took it off and did an echo 1 > smp_affinity
..
Similarly I repeated teh procedure for various interrupts.The process seemed to work well enough until I rechecked everything again.I found that the entries had automatically changed to 4 from 1 or 1 to 2 and so on.Howz this happening? Is it because of the irqbalance process and is it safe to turn it off completely?
Please advise...All info is appreciated...Im looking into the Kernel documentation for guidelines so it should be correct stuff Im following..
The kernel is a 2.6.9.5 , 4 processor , 3 NIC's(1 disabled) EL4 system...
I really don't think that you have to be at all concerned about IRQ sharing because the data is sharing the same data bus anyway. What do you imagine that you will gain?
Hey,
Thnx 4 replying...Im doing this for the first time so I was just following t he kernel documentation in proc.txt. It says that its possible to assign processes to CPU's.
The reason why:
My Etherenet card (eth2) is tied to IRQ 169 and IRQ169 seems to be tied to Processor 2 all the time and we were dropping a few packets (we could see it in our compiled version of Snort) so I was just thinking ; its because the card is getting so many interrupts ( over 2000000 were listed in /proc/interrupts) for IRQ 169...if I managed to rid CPU 2 of its duties in handling all other interrupts except 169 I could see a slight performance raise...
That was my logic...I did manage to disable irqbalance and set stuff individually but today morning when I cam back to work everything was back to ffffffff even though Irqbalance was off.
Let me know if the logic Im using has a hole in it and why?
I'll look into it and edit this post with any information that I find. Now I know what you expected to achieve. My first impression is that you may be right. I haven't heard of a hardware device being assigned to a CPU.
Thnx dude..appreciate the effort...just 1 more pointer though..I ws speaking to my boss 2day about this ...and he was of the logic that if ppl were given that much control to play around with Linux they might end up wrecking it ; so these tweaks which we do in /proc...btw I tweaked a lot of virtual memory today...might be when we've almost reached a target and we want to do some final changes to squeeze that little extra out of the system...sounded logical...coz the changes I made to the memory dont seem to add too much...
But still do check out stuff n let me know whenever u hav the time...
Cheers
Arvind
p.s....My Boss loves Windows .. so we had a nice discussion abt how mucxh bettr Linux was...futile though ..ppl never do understand...
I've had a bit of fun looking for information on this subject. Once I figured out a good Google search phrase I found that most of the listings referred to only two separate pieces of information. One of these references talks about balancing network cards over multiple CPUs. This makes me think that you've already read this. Nevertheless here are the two documents that I found.
Thnx stress...will definitely take a look at the two articles you posted...just FYI..coz this is interesting..I turned irqbalance off....
as in chkconfig --level 35 irqbalance off...so its gone ...
and then we ran our performance tests again and guess what ...the performance improved straight away dramatically !! ... I looked into /proc/irq/0 etc and every single smp_affinity file had set itself to ffffffff ...strange?
If i do a cat /proc/interrupts...I see that everything (all interrupts) have bound tehmselves to processor 0 and the other procesors are all having a day off...but the performance is better...dramatically better...
Am doing a test right now to check if I was wrong , have turned irqbalance back on...will keep this post going till I understand whats going on...
and then we ran our performance tests again and guess what ...the performance improved straight away dramatically !! ... I looked into /proc/irq/0 etc and every single smp_affinity file had set itself to ffffffff ...strange?
Nah, that's expected. Think of /proc/irq/X/smp_affinity as kind of like a netmask... the numbers you put in there specify which processors the IRQ is allowed to run on. The default setting of ffffffff means "this IRQ may be handled by any of the processors in this machine".
As for the irqbalance mystery, yeah, it was responsible for the changing IRQ masks. That's what it does... randomly rolls the masks around in order to make sure that each proc spends a "fair" amount of time handling interrupts. Personally, I'd rather tie an interrupt to a processor and not worry about it anymore.
Also, as a personal opinion, I'd reccomend checking your system to make sure that you're not running Intel's Hyperthreading. I know some folks swear it improves performance, but I've never had any luck with it on a Linux server and I've found the best thing to do is just disable it. I don't have any experience with real dual procs, but from what I've read it sounds to me like those should be ok.
My Etherenet card (eth2) is tied to IRQ 169 and IRQ169 seems to be tied to Processor 2 all the time and we were dropping a few packets (we could see it in our compiled version of Snort) so I was just thinking ; its because the card is getting so many interrupts ( over 2000000 were listed in /proc/interrupts) for IRQ 169...if I managed to rid CPU 2 of its duties in handling all other interrupts except 169 I could see a slight performance raise...
That was my logic...I did manage to disable irqbalance and set stuff individually but today morning when I cam back to work everything was back to ffffffff even though Irqbalance was off.
Are you sure your talking about IRQ's? IRQ numbers only go up to 15, starting with 0, how are you getting an IRQ of 169 would be my question..
Are you sure your talking about IRQ's? IRQ numbers only go up to 15, starting with 0, how are you getting an IRQ of 169 would be my question..
Through the (black) magic of APIC.
Here's some technical info on it that should give you a good idea of what's up without having to plod through Intel tech references (sorry for the obfuscation... the forums say my post count is too low to put up a link)
http :// osdev.berlios.de / pic.html
I don't fully understand it as I'm not a hardware dude, but APIC is basically necessary for handling interrupt addressing on multiple CPU systems and allowing the CPUs to talk to each other. Somewhere along the way, the comp engineer geeks found some other tricky stuff they could use APIC to do on uniprocessor machines and so by now it's pretty much become a standard on any modern mobo made today, and thus we get stuck with those annoying but harmless "spurious interrupt" messages.
APIC is basically a 24 line table that holds 64 bits each and uses bitmasks to identify which processor the interrupt is associated with. So, if you have a scsi card on a 4 proc system that happens to "belong" to CPU2 and it's IRQ is set to 15, then APIC will map that out to 63 so that CPU0, CPU1, and CPU3 can all reference it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.