[SOLVED] Download limits in Squid3

mandyapenguin · 12-06-2011, 07:08 AM

Hi..everybody,
I am running squid-3.1.11 on ubuntu 11.04 as transparent proxy. I need rules in which there is no restrictions in size for some authorized PCs even during office hours and there is no restrictions in size to others after office hours. First I tried

Code:

reply_body_max_size 100 MB

This is restricting also authorized IPs while downloading who are already able to access restricted sites. Then I commented out above rule and tried

Code:

acl officehours time 09:00-18:00
reply_body_max_size 104857600 allow officehours

Now none of sites(even google) are opening from the clients. So please help me to add rules to restrict downloads to everyone apart from some authorized IPs which is more than 100 MB.

deep27ak · 12-06-2011, 08:01 AM

try this

Code:

acl official_hours time 09:00-18:00
reply_body_max_size none
http_access allow official_hours

if you want to give access to few machines you can use their MAC address

Code:

acl M1 arp 01:02:03:04:05:06
acl M2 arp 11:12:13:14:15:16
http_access allow M1
http_access allow M2

or their IP address

Code:

acl our_networks src 192.168.0.
http_access allow our_networks

mandyapenguin · 12-06-2011, 09:14 PM

Hi..Deepak, Thanks for the reply.

Code:

acl official_hours time 09:00-18:00
reply_body_max_size none
http_access allow official_hours

If I put above rule, now everyone is able to download even large files and there is no limitation to any of the user.
I don't want this, I want 2 rules
1) I want to allow download upto only 100 MB for everyone during 9AM to 6PM. After 6PM to 9AM everyone should be able download large files even it is more than 100MB in size.
2) There should not be any download limitation for some authorized PCs even during office hours and these authorized PCs should be able to download even more than 100MB files at any time.
If I try this below

Code:

acl official_hours time 09:00-18:00
acl my_net src 192.168.0.0/24
acl auth_IP src 192.168.0.227
reply_body_max_size 104857600 allow my_net !auth_IP

restarted the service and got this error

Code:

service squid3 restart
 * Starting Squid HTTP Proxy 3.x squid3                                         2011/12/07 11:48:51| aclIpParseIpData: WARNING: Netmask masks away part of the specified IP in '192.168.0.1/24'
2011/12/07 11:48:51| WARNING: Unknown bytes unit 'allow'
FATAL: Bungled squid.conf line 1040: reply_body_max_size 104857600 allow my_net !auth_IP
Squid Cache (Version 3.1.11): Terminated abnormally.
CPU Usage: 0.000 seconds = 0.000 user + 0.000 sys
Maximum Resident Size: 16576 KB
Page faults with physical i/o: 0
                                                                         [fail]

I also tried deny instead of allow, but still none of the sites are opening from the clients. Then I tried

Code:

acl official_hours time 09:00-18:00
acl auth_IP src 192.168.0.227
reply_body_max_size 100 MB
http_access deny official_hours auth_IP
http_access allow official_hours

Now time and auth_IP rule is not applied and everyone including auth_IP is able to download upto only 100 MB at anytime.
So please help me to add 2 rules that I have requested in the beginning.

deep27ak · 12-07-2011, 10:50 PM

for business hours, users will have 100 MB download limit and important users will have full access to download

Code:

acl official_hours time 09:00-18:00
http_access allow official_hours
acl imp_users src 192.168.0.100
http_access allow imp_users
reply_body_max_size 0 allow imp_users
reply_body_max_size 104857600 allow official_hours

^^^^^if the given syntax doesn't works or returns with error then try 100 MB instead of 104857600

for non business hours no download limit

Code:

acl non_official_hours time 18:01-08:59
http_access allow non_official_hours
reply_body_max_size none allow non_official_hours

^^^^^if the given syntax doesn't works or returns with error then try 0 instead of none
users having access

Code:

acl imp_users src 192.168.0.81
http_access allow our_networks
http_access deny all

mandyapenguin · 12-08-2011, 09:24 AM

Thanks for the reply deepak,

Code:

acl official_hours time 09:00-18:00
http_access allow official_hours
acl imp_users src 192.168.0.100
http_access allow imp_users
reply_body_max_size 0 allow imp_users
reply_body_max_size 104857600 allow official_hours

The above rule are not applying and I am getting the error as below

Code:

#service squid3 restart
 * Restarting Squid HTTP Proxy 3.x squid3                                       2011/12/08 20:42:55| aclIpParseIpData: WARNING: Netmask masks away part of the specified IP in '192.168.0.1/24'
2011/12/08 20:42:55| aclParseAclList: ACL name 'allow' not found.
FATAL: Bungled squid.conf line 1062: reply_body_max_size 0 allow imp_users
Squid Cache (Version 3.1.11): Terminated abnormally.
CPU Usage: 0.000 seconds = 0.000 user + 0.000 sys
Maximum Resident Size: 16544 KB
Page faults with physical i/o: 0                                 [fail]

The problem is, none of the rule is applying with reply_body_max_size line. If we add as only

Code:

reply_body_max_size 100 MB

then the rule can apply but it is affecting to all including important users for all time.
So I also tried by putting "!imp_users", 100 MB instead of 104857600, 0 instead of none. but none of the rule is applying. I request you to check it once with squid-3.1.11 and please let me know if those two rules are applying in your machine.

deep27ak · 12-11-2011, 09:59 PM

sorry for replying late, I was on leave from office

You can try the following syntax as the mistake is again mine
I never checked your distro as the syntax which I was using works on RHEL

In ubuntu allow syntax is not recognized as you can see the error by yourself

Code:

2011/12/08 20:42:55| aclParseAclList: ACL name 'allow' not found.

try this

Code:

acl official_hours time 09:00-18:00
http_access allow official_hours
acl imp_users src 192.168.0.100
http_access allow imp_users
reply_body_max_size none imp_users
reply_body_max_size 100 MB official_hours

Code:

acl non_official_hours time 18:01-08:59
http_access allow non_official_hours
reply_body_max_size none non_official_hours

linuxmen · 12-12-2011, 06:37 AM

go through this tutorial for squid download size limiting...
http://servercomputing.blogspot.com/...xy-server.html

mandyapenguin · 12-13-2011, 11:54 AM

Code:

acl official_hours time 09:00-18:00
http_access allow official_hours
acl imp_users src 192.168.0.100
http_access allow imp_users
reply_body_max_size none imp_users
reply_body_max_size 100 MB official_hours

Code:

acl non_official_hours time 18:00-23:59
acl non_official_hours time 00:01-09:00
http_access allow non_official_hours
reply_body_max_size none non_official_hours

Hi..Deepak,
Thanks a lot. The above rules are working fine. I tried also with a file and mentioned imp_users IPs. Now those imp_users IPs which are listed in /etc/squid3/.imp_users file can download unlimited size at any time while others can only upto 100 MB during office hours. After office hours others also can download unlimited size. This is what I had expected from Linux guru's. Thanks Deepak.

But still others can download using https while getting error for http using same url even it is more than 100 MB and also even I have prevented some suffixes in a file like this in squid.conf.

Code:

acl denied_suffixes url_regex "/etc/squid3/.denied_suffixes"
http_access deny denied_suffixes

Code:

cat /etc/squid3/.denied_suffixes
./*.exe$
./*.iso$
./*.mp3$
./*.mp4$
./*.avi$
./*.torrent$

and so on
For example others are prevented from download with "http://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.openoffice.org/stable/3.3.0/OOo_3.3.0_Linux_x86_install-rpm-wJRE_en-US.tar.gz"
but they can download with "https://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.openoffice.org/stable/3.3.0/OOo_3.3.0_Linux_x86_install-rpm-wJRE_en-US.tar.gz"
So it would be much better if you could post a rule to prevent downloads using https which is more than 100 MB and prevented during office hours.
I will be waiting for your kind reply

Once again thank you very much.

mandyapenguin · 12-13-2011, 12:24 PM

Quote:

Originally Posted by linuxmen

go through this tutorial for squid download size limiting...
http://servercomputing.blogspot.com/...xy-server.html

Hi..Thank you very much linuxmen.
This site is very useful. I could prevent youtube videos for unauthorized users during office hours. But not with metacafe.com. Could you please guide me or send me an url by which I will be able to prevent other site's flash videos.

I added rule to block https://facebook.com using

Code:

acl badsites dstdomain .facebook.com
http_access deny CONNECT badsites

But only windows machines never get connects to https://facebook.com while linux machine can, amazing....!!!?
I also added

Code:

http_reply_access deny badsites

But still it has blocked for only window machines, not to linux machines.
So could you please help me in this to block https://facebook.com

deep27ak · 12-14-2011, 11:44 PM

Even I was new to this problem which you mentioned.

I never noticed that squid was only blocking http and allowing https

I don't know if this would be helpful as it works in my machine
you can block the access to port 443 which is for ssl during office hours.
i.e give the access to http and https only to important users and deny to all others and after office hours everyone will have full access

Code:

acl bad_port port 443
acl no_block_port_ip src 192.168.0.100
http_access deny bad_port !no_block_port_ip
http_access allow all

don't forget to comment this line in squid.conf

Code:

#acl Safe_ports port 443                # https

to block downloads with following extensions you can also try this

Code:

#cd /etc/squid

#vi badpage.acl
\.[Exe][Xx][Ee]$
\.[Zz][Ii][Pp]$
\.[Mm][Pp]3$


#vi /etc/squid/squid.conf
acl blockpages url_regex "/etc/squid/badpage.acl"
http_access deny blockpages

mandyapenguin · 12-16-2011, 09:56 AM

Hi..Deepak, Thanks for the reply

Code:

acl bad_port port 443
acl no_block_port_ip src 192.168.0.100
http_access deny bad_port !no_block_port_ip
http_access allow all

Code:

#acl Safe_ports port 443                # https

Code:

#cd /etc/squid

#vi badpage.acl
\.[Exe][Xx][Ee]$
\.[Zz][Ii][Pp]$
\.[Mm][Pp]3$

#vi /etc/squid/squid.conf
acl blockpages url_regex "/etc/squid/badpage.acl"
http_access deny blockpages

Now the above rules are working fine and no one can download the file which is more than 100 mb apart from imp_users, but only if the users goes with browser settings and none of the https sites will open in browser settings with above rule. Fine, but I am using transparent mode with squid3.*, so still the users can download the files using https. Anyway at last you helped me for those two rules that I had requested in the beginning. Thank you very much Deepak.

deep27ak · 12-16-2011, 11:48 AM

hey really glad to help

can you help me with configuring squid as transparent proxy because sometimes I face problem with that in my machine RHEL 5.2

what are the steps needed to be followed for transparent proxy?

mandyapenguin · 12-16-2011, 11:49 PM

Hi..Deepak me too glad to help.
I followed this link. I just copied the script and executed.
Here is the my configuration

Code:

eth0(internet)
IP  192.168.1.2/24
g/w 192.168.1.1

eth1(LAN)
IP  192.168.0.1/24

cat /etc/resolv.conf
search mydomain.com
nameserver 192.168.0.1

dhcp server  192.168.0.1

for the clients
dhcp range 192.168.0.200 192.168.0.253
pdns 192.168.0.254(again this will forward to 192.168.0.1 only)
sdns 192.168.0.1

I hope, I don't want to explain with configuration files under /etc/bind directory. Because I know that you are already an expert in DNS configuration.
I just changed the server IP in script as 192.168.0.1 because I have given in squid.conf file as
acl my_lan src 192.168.0.1/24. I executed the script now everything ftp, mail clients and all working fine. I went for transparent mode because I struggled a lot to enable mail client access with IPTable rules. If the mail clients are working fine for you please help me how to with IPTable rules in non transparent mode.
In RedHat based O/S for transparent mode you have to put these 4 lines

Code:

httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

I have tried in ubuntu-11.04/11.10 without above 4 rules, the users are able to access internet without browser settings. I think in centO/S-6 also you are not required above 4 rules. I am not sure in this. But make sure the line as below in squid.conf file

Code:

http_port 3128 transparent

Please find my squid.conf file which is in attachment which is working fine including mail client, ftp access. This attachment from ubuntu-11.10. And please let me know if it is possible to enable access to mail clients with non transparent mode.

deep27ak · 12-18-2011, 11:18 PM

Thanks for your help I will try your suggestion, as these for my personal practice so once I am through with my work I'll give a update

I am quite confused with the term mail client+squid

by mail client do you mean outlook, Thunderbird ?

do you face problem using these mail clients with non transparent squid ?
if you can tell me where you face errors or what sort of errors as I had connected thunderbird with my sendmail and was working fine but no idea about using squid and mail clients

If you can be little more specific I might help

mandyapenguin · 12-19-2011, 09:44 AM

Code:

I am quite confused with the term mail client+squid
by mail client do you mean outlook, Thunderbird ?
do you face problem using these mail clients with non transparent squid ?
if you can tell me where you face errors or what sort of errors as I had connected thunderbird with my sendmail and was working fine but no idea about using squid and mail clients
If you can be little more specific I might help:)

Yes, mail client is nothing but thunderbird and outlook only. This will be working fine because of nat configuration for transparent mode. For example if I configure my gmail account with tunderbird/outlook while I am on transparent mode, I will be able to send/receive mails.
But I am not able send/recieve while I am on non transparent mode. Just think, for non transparent I'll just install squid and configure the squid.conf file and doing nothing with IPTable rules and the client machine can access internet only then if they configure browser settings. But in this condition the mail client does not work. I request you to check it once by configuring your gmail/yahoomail account with thunderbir/outlook without trasparent mode. If you are able send/receive mails then please let me know that how you could do it.
As I found in the google search there is nothing to do with squid.conf for mail client access since squid does not proxy pop3, imap and smtp. So I think these protocol should masquerade in the nat table. But I dont have much experience with IPTable rules. I need such setup that the client machine can access internet only then if they configure browser settings, and also mail client should works fine. I will be waiting for your kind reply.