LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 09-26-2008, 12:19 PM   #1
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Squid proxy server only using 1 CPU


Hello,

I have a dedicated squid proxy server. Since I've implemented the proxy server users have been complaining about "internet slowness".

I have nothing else running on this server other than squid. I checked the RAM and I'm only using 2GB out of 8GB (no problem there); so I ran top and it seems that squid is the top user (no surprises there...it's the only service running)

Then this is what I got on my mpstat...
Code:
[root@camel:~<116>]# mpstat -P ALL 1 5
Linux 2.6.18-92.1.10.el5 (camel)        09/26/2008

10:07:17 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
10:07:18 AM  all   25.00    0.00    0.00    0.00    0.00    0.00    0.00   75.00   1144.00
10:07:18 AM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1019.00
10:07:18 AM    1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
10:07:18 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    122.00
10:07:18 AM    3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00

10:07:18 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
10:07:19 AM  all   24.94    0.00    0.00    0.00    0.00    0.00    0.00   75.06   1143.00
10:07:19 AM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1002.00
10:07:19 AM    1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
10:07:19 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    142.00
10:07:19 AM    3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00

10:07:19 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
10:07:20 AM  all   25.00    0.00    0.00    0.00    0.00    0.00    0.00   75.00   1127.00
10:07:20 AM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1020.00
10:07:20 AM    1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
10:07:20 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    106.00
10:07:20 AM    3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00

10:07:20 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
10:07:21 AM  all   25.00    0.00    0.00    0.00    0.00    0.00    0.00   75.00   1145.00
10:07:21 AM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1001.00
10:07:21 AM    1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
10:07:21 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    144.00
10:07:21 AM    3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00

10:07:21 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
10:07:22 AM  all   24.94    0.00    0.00    0.00    0.00    0.25    0.00   74.81   1172.73
10:07:22 AM    0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1058.59
10:07:22 AM    1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
10:07:22 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    114.14
10:07:22 AM    3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
It seems that squid is only using 1 processor! Now see the arerage...
Code:
Average:     CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
Average:     all   24.98    0.00    0.00    0.00    0.00    0.05    0.00   74.98   1146.29
Average:       0    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00   1020.04
Average:       1    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
Average:       2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    125.65
Average:       3    0.00    0.00    0.00    0.00    0.00    0.00    0.00  100.00      0.00
It seems that the other three processors remain "untouched"...everything that I've read suggests that squid is "multithreaded".

What am I missing? Is there any settings/tuning I can do that will allow squid to use multipule processors?

Here is some information about my sys...
Code:
[root@camel:~<118>]# uname -a
Linux camel 2.6.18-92.1.10.el5 #1 SMP Tue Aug 5 07:42:41 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux
[root@camel:~<119>]# 
[root@camel:~<119>]# cat /etc/redhat-release 
CentOS release 5.2 (Final)
[root@camel:~<120>]#
Any help would be great! Thanks!

-C
 
Old 09-26-2008, 12:55 PM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 23,134

Rep: Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464
Quote:
Originally Posted by custangro View Post
Hello,

I have a dedicated squid proxy server. Since I've implemented the proxy server users have been complaining about "internet slowness".

Any help would be great! Thanks!
Did you install this from a pre-compiled package, or via source code? I'm not in front of my box right now, but if you got it from package, it may not have been built for a multi-processor/SMP kernel. If you compile from source, run "./configure --help", to see the available options. There may be one in there for SMP. Just my first thought...
 
Old 09-26-2008, 12:58 PM   #3
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Quote:
Originally Posted by TB0ne View Post
Did you install this from a pre-compiled package, or via source code? I'm not in front of my box right now, but if you got it from package, it may not have been built for a multi-processor/SMP kernel. If you compile from source, run "./configure --help", to see the available options. There may be one in there for SMP. Just my first thought...
No, I installed it VIA yum.

Do you know if they have a "multiprocessor" version?

FWIW I have the "64bit" version installed...
Code:
[root@camel:~<126>]# yum -y list | grep squid
squid.x86_64                             7:2.6.STABLE6-5.el5_1. installed
-C

Last edited by custangro; 09-26-2008 at 01:00 PM.
 
Old 09-26-2008, 01:34 PM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 23,134

Rep: Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464
Quote:
Originally Posted by custangro View Post
No, I installed it VIA yum.

Do you know if they have a "multiprocessor" version?

FWIW I have the "64bit" version installed...
Code:
[root@camel:~<126>]# yum -y list | grep squid
squid.x86_64                             7:2.6.STABLE6-5.el5_1. installed
-C
Not sure, and I'm not even sure if that IS an option. However, since you have the 64 bit version on an SMP box, I would definitely install it from source code. Since you're running the SMP kernel, it will probably compile with that enabled, even if there isn't a command-line configure option for it.

You wouldn't have to reconfigure anything either...just compile from source, then bounce the service.
 
Old 09-26-2008, 02:06 PM   #5
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Quote:
Originally Posted by TB0ne View Post
Not sure, and I'm not even sure if that IS an option. However, since you have the 64 bit version on an SMP box, I would definitely install it from source code. Since you're running the SMP kernel, it will probably compile with that enabled, even if there isn't a command-line configure option for it.

You wouldn't have to reconfigure anything either...just compile from source, then bounce the service.
Thanks.

Is there anyway to see how the "packaged version" was compiled? That way I can use the same statement but just add the option i want...

-C
 
Old 09-26-2008, 02:14 PM   #6
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 23,134

Rep: Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464Reputation: 6464
Quote:
Originally Posted by custangro View Post
Thanks.

Is there anyway to see how the "packaged version" was compiled? That way I can use the same statement but just add the option i want...

-C
Same statement?? What do you mean? You'd have to look at the package details, to see what options it had, but sometimes it doesn't specify.

And this isn't about running Squid....you've got to build it with the option enabled, or build it on a system running the SMP kernel. You'll use the same Squid config file no matter what...squid will just run differently 'under the hood'.
 
Old 09-26-2008, 04:51 PM   #7
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Quote:
Originally Posted by TB0ne View Post
Same statement?? What do you mean? You'd have to look at the package details, to see what options it had, but sometimes it doesn't specify.

And this isn't about running Squid....you've got to build it with the option enabled, or build it on a system running the SMP kernel. You'll use the same Squid config file no matter what...squid will just run differently 'under the hood'.
I mean the config statmement...i.e
Code:
root@hot# ./configure --path=/path/to/squid --opt=more-threads...
I want to see if I can extract the configure statement from the RPM installation...that way I can use it for the one I compile myself...

Quote:
Originally Posted by TB0ne View Post
You'll use the same Squid config file no matter what...
I know that silly!

Last edited by custangro; 09-26-2008 at 04:53 PM.
 
Old 09-27-2008, 02:24 AM   #8
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
How many users do you have? OK, it is an issue that you have 4 cpus and are only managing to use 1, but, even so, that level of cpu usage seems high, unless you have many users.

My concern is that something else about the configuration may be sub-optimal (screwed up, in normal English) and that is leading to the need to use lots of cpu.
 
Old 09-27-2008, 06:55 PM   #9
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
To get the configure options which were used just do a:
Code:
squid -v
 
Old 09-27-2008, 09:44 PM   #10
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Thanks win32sux

salasi,

I have about 300 users that rely heavily on network user (our entire job tracking system is web based...and we have a call center).

Nothing else is running on that machine (I disabled X and all services that I don't need...I default to runlevel 3). I'm just surprised that there is not more information about making squid "friendly" with multiprocessors...

-C

Last edited by custangro; 09-27-2008 at 09:45 PM.
 
Old 09-27-2008, 10:01 PM   #11
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
I quote from the installation section of the wiki-powered FAQ:
Quote:
Can Squid benefit from SMP systems?

Squid is a single process application and can not make use of SMP. If you want to make Squid benefit from a SMP system you will need to run multiple instances of Squid and find a way to distribute your users on the different Squid instances just as if you had multiple Squid boxes.

Having two CPUs is indeed nice for running other CPU intensive tasks on the same server as the proxy, such as if you have a lot of logs and need to run various statistics collections during peak hours.

The authentication and group helpers barely use any CPU and does not benefit from dual-CPU configuration.
There's also interesting information on that page if you're using RAID.

Last edited by win32sux; 09-27-2008 at 10:03 PM.
 
Old 09-28-2008, 06:15 PM   #12
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
Quote:
Originally Posted by win32sux View Post
I quote from the installation section of the wiki-powered FAQ:
There's also interesting information on that page if you're using RAID.
That's unfortunate...so basically I have to run multiple instances of squid on different ports and distribute users?

This is where I say that the websense solution is starting to look more attractive...

Too bad...I was really hoping on saving the company money

Oh well...you can't be the hero ALL the time

Thanks to everyone for all their help!

-C
 
Old 09-28-2008, 08:02 PM   #13
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
[QUOTE=custangro;3293831]Thanks win32sux

salasi,

I have about 300 users that rely heavily on network user (our entire job tracking system is web based...and we have a call center).
[QUOTE]

Sorry, had to check, but with that kind of profile, the total cpu loading does seem a bit more reasonable than if you had only four users who rarely used the web...

If this is a purely internal web tracking system, should you really be caching it? Maybe, if you make this bit uncachable, it takes the load off a bit.

And, if you do need/want to cache this, shouldn't you be running a separate instance in httpd accelerator mode (as it sounds like this app is a bit 'database-y' in nature, it may well be pointless caching its output).

Its still slightly unusual though; normally lack of squid performance is down to lack of memory (physical memory, virtual memory or squid.conf not set up to use what you've got).

Quote:
Nothing else is running on that machine
Maybe caching name resolutions would be a good thing. You've certainly got the spare CPU to try it without starving squid of cpu.
 
Old 09-29-2008, 12:21 PM   #14
custangro
Senior Member
 
Registered: Nov 2006
Location: California
Distribution: Fedora , CentOS , RHEL
Posts: 1,978

Original Poster
Blog Entries: 1

Rep: Reputation: 209Reputation: 209Reputation: 209
[QUOTE=salasi;3294535]
Quote:
Originally Posted by custangro View Post

Maybe caching name resolutions would be a good thing. You've certainly got the spare CPU to try it without starving squid of cpu.
That's actually not a bad idea...

I am bypassing the internal sites already...(as you stated...there is really no need to cache internal sites).

I guess I could try installing a caching NS on the machine...then point it to our internal DNS server (for our internal sites).

I think I should do some more homework about what to cache and what not to cache...as of right now we are "bypassing" our internal site via the browser settings...maybe I can also do something at the "squid level"...

On to more reading the docs...

**EDIT**

Actually, I don't really care if Squid is caching. All I'm using it for is for content filtering. Is there any way to "disable" the caching part? All I really care about it the content filtering aspect of it...

-C

Last edited by custangro; 09-29-2008 at 01:54 PM.
 
Old 09-30-2008, 03:20 AM   #15
win32sux
LQ Guru
 
Registered: Jul 2003
Location: Los Angeles
Distribution: Ubuntu
Posts: 9,870

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Quote:
Originally Posted by custangro View Post
Actually, I don't really care if Squid is caching. All I'm using it for is for content filtering. Is there any way to "disable" the caching part? All I really care about it the content filtering aspect of it...
I used to do this by setting the disk cache directory to /dev/null. There were many times when I wanted only RAM cache to be used but not disk cache. Maybe the new Squid versions have an actual configuration option for this sort of thing but I'm not sure. Either way it's pretty simple.

Last edited by win32sux; 09-30-2008 at 03:21 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
squid proxy server configuration & distribution of internet without proxy gaurav_gupta082 Linux From Scratch 2 07-31-2010 11:25 AM
Using ISA Server as Parent Proxy and want to setup Squid as dwonstream proxy tauseef1 Red Hat 1 04-09-2008 01:03 AM
Squid/Proxy Server metallica1973 Linux - Networking 2 10-22-2007 06:16 PM
squid(proxy) server pankajkarde Linux - Server 2 03-08-2007 02:04 AM
Squid Proxy server help win2Linux Linux - Server 3 09-03-2006 09:53 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 02:19 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration