LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 02-17-2017, 06:21 AM   #1
lol123
LQ Newbie
 
Registered: Feb 2017
Posts: 2

Rep: Reputation: 0
Need help optimizing squid cluster


Hi there,
I am running some squid proxy servers inside a college where there are systems with two configurations and are used to give access to students. The system is used solely for proxy.
The systems are running squid 3.5.20
and centos 7 in all of them.
There are 10 proxies of 5 systems each
1)
i7-3770k
8 GB RAM
1 500gb 7200rpm hdd seagate barracuda

2)
i7-4770k
12GB RAM
1 500 gb 7200rpm hdd seagate barracuda.

There are two main partitions-
/cache 220 GB ext4
/logs 100 GB ext4

The one with the 8GB RAM run on a 300Mbps line (shared among proxies) while the ones with 12GB ram run on 1Gbps line (shared among proxies).

The problem I am having is that they are very slow (even with few users) and often go down whenever traffic reaches > 150 users.
I have tried almost everything and cannot seem to figure out the problem.
We also use authentication.


Code:
acl localnet src 172.16.0.0/12	# RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines


acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

#allow cachemgr access from localhost
http_access allow localhost manager


# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost

###################################################
#		BASIC CONFIGURATION
#
###################################################


acl pghostel src 172.31.144.0/22



acl allow_ip src 172.31.80.39 172.31.80.117 
acl proxies src "/etc/squid/proxy_list"
http_access allow proxies
http_access allow allow_ip

#allow cachemgr access from admin ip only
http_access allow allow_ip manager
http_access deny manager

dns_nameservers 172.31.100.6 8.8.8.8  

#sets up external authenticator which uses the passwd file to authenticate the users
auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd  

# prompt while asking for the authentication
auth_param basic realm PROXY  

# proxy authentication is needed to use net  
acl authenticated proxy_auth REQUIRED

# allows only authenticated users
http_access allow authenticated            

#Authentication parameters
auth_param basic children 10              
auth_param basic credentialsttl 2 hours   
auth_param basic casesensitive off
authenticate_cache_garbage_interval 1 hour
authenticate_ttl 1 hour
authenticate_ip_ttl 0 seconds



#####################################################
##CHANGE HERE A/C TO PROXY
#####################################################


#visible to external world 
visible_hostname  proxyXXX

#available to peers 
unique_hostname proxyXXX

#only listen on internal interface
http_port 172.31.10X.XX:3128


#http_access deny pghostel


#####################################################
##CHANGE ABOVE A/C TO PROXY
#####################################################

logformat accesslog  %ts.%03tu %6dt %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
#logformat squid      %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt

access_log /logs/squid/access.log accesslog
cache_log /logs/squid/cache.log 
cache_store_log none

#access_log none
#cache_log /dev/null

#30 rotations allowed
logfile_rotate 30

##TEST NEW FEATURES
pipeline_prefetch 90

#Maximum number of DNS IP cache entries.
ipcache_size 10240

log_icp_queries off

client_db off

buffered_logs on

#Maximum number of FQDN cache entries.
fqdncache_size 10240

#Whether to lookup the EUI or MAC address of a connected client.
eui_lookup off
##


coredump_dir /cache/squid/swap 
            
# 190 GB cache with 64 first level dir and 512 second level dir     
cache_dir  aufs /cache/squid  190000 64 512

maximum_object_size 300 MB            


#allow icp access only to proxies
icp_access allow proxies
icp_access deny all

icp_port 3130

# denies caching of local web servers
acl lan_dst dst 172.16.0.0/12
cache deny lan_dst			

# direct connection to lan servers
always_direct allow lan_dst 

# denies caching of cgi content
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY	

#cache digest has 6 bits
digest_bits_per_entry 6


           
acl proxy014 src 172.31.100.14
acl proxy025 src 172.31.100.25
acl proxy026 src 172.31.100.26
acl proxy027 src 172.31.100.27
acl proxy028 src 172.31.100.28
acl proxy029 src 172.31.100.29
acl proxy030 src 172.31.100.30
acl proxy214 src 172.31.102.14
acl proxy229 src 172.31.102.29
acl proxy329 src 172.31.103.29



#cache_peer 172.31.100.14 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.100.25 sibling 3128 3130 proxy-only login=edcguest:edcguest 
#cache_peer 172.31.100.26 sibling 3128 3130 proxy-only login=edcguest:edcguest 
#cache_peer 172.31.100.27 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.100.28 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.100.29 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.100.30 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.102.14 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.102.29 sibling 3128 3130 proxy-only login=edcguest:edcguest
#cache_peer 172.31.103.29 sibling 3128 3130 proxy-only login=edcguest:edcguest


#never_direct allow all


#cache_peer_access 172.31.100.14 deny proxy014
#cache_peer_access 172.31.100.25 deny proxy025
#cache_peer_access 172.31.100.26 deny proxy026
#cache_peer_access 172.31.100.27 deny proxy027
#cache_peer_access 172.31.100.28 deny proxy028
#cache_peer_access 172.31.100.29 deny proxy029
#cache_peer_access 172.31.100.30 deny proxy030
#cache_peer_access 172.31.102.14 deny proxy214
#cache_peer_access 172.31.102.29 deny proxy229
#cache_peer_access 172.31.103.29 deny proxy329

#cache deny all





###################################################
#		PEERS AND CACHE
#		  ENDS
###################################################

# allowing access from local network
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all


refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320

What I want is that all proxies share their cache so that we dont have to go to the internet to fetch data.
How can I optimize the proxy servers?

Thank You

Sorry If I did some posting mistake, I am new here

Last edited by lol123; 02-17-2017 at 09:03 AM.
 
Old 03-16-2017, 11:28 AM   #2
nblu
LQ Newbie
 
Registered: Oct 2012
Posts: 3

Rep: Reputation: Disabled
First try turning off all things related to logging and have the squid files on a file system with noatime mount.

Then we work it up further than that.

Basically your proxy should do most io and processing only for caching nothing else!! Is that clear sir??
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
setup high availibility cluster for Squid Proxy cccc Debian 3 05-09-2015 06:24 AM
cman service stucks during booting of cluster node (RedHat Cluster Suite) zama Linux - Software 0 07-09-2012 08:50 AM
[SOLVED] Converting Windows node cluster to Linux Cent0S 5.5 cluster do I kickstart or image ? cdrolet Linux - Newbie 1 11-18-2011 10:44 AM
Optimizing linux for Squid-cache, network and USB DSL driver mbayeb Linux - Software 0 07-02-2003 08:39 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 05:36 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration