Slackware-14.1-Squid 3.4.11 Native ACL URL Filtering
Squid is a Caching Proxy Server that reduces bandwidth, improves response times by caching and reusing frequently-requested web pages so you have a faster Web Experience. Normally, if you wish to do URL Filtering, in other words, blocking by Web Address, you should use SquidGuard or UfdbGuard or DansGuardian to do so since they are specifically made to do URL Filtering more efficiently than the native Squid ACL. However, for those who can't or won't use the other applications, Squid can be configured to do Native Access Control URL Filtering.
With thanks to www.basicconfig.com and Arch Wiki
If you will be using the default user "nobody", you can go ahead and run the script as is. You may want to create a new user and group for squid since this user and group functions like a sandbox for Squid and will not have a functioning shell. If you want to run squid as the user "squid", then create the user and group "squid". The group ID number I am using is 317 for uid/gid "squid" which does not interfere with the SlackBuilds.org UID/GID list.
# groupadd -g 317 squid
# useradd -d /dev/null -s /bin/false -u 317 -g 317 squid
The version currently available at SlackBuilds.org is Squid-3.4.10. I am using Squid-3.4.11. The changes to the SlackBuild are: 1) Change the Version Number 2) Change the Squid User/Group from "nobody" to "squid" and 3) add the Configuration Option --with-default-user=squid
Here are the snippets for the SlackBuild revised for Squid-3.4.11:
VERSION=${VERSION:-3.4.11}
SQUIDUSER=${SQUIDUSER:-squid}
SQUIDGROUP=${SQUIDGROUP:-squid}
CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
--prefix=/usr \
--libdir=/usr/lib${LIBDIRSUFFIX} \
--sysconfdir=/etc/squid \
--localstatedir=/var/log/squid \
--datadir=/usr/share/squid \
--with-pidfile=/var/run/squid \
--mandir=/usr/man \
--with-logdir=/var/log/squid \
--with-default-user=squid \
--enable-snmp \
--enable-auth \
--enable-auth-basic \
--enable-linux-netfilter \
--enable-async-io \
--build=$ARCH-slackware-linux \
--disable-strict-error-checking
After installing the package, if you need to start squid at boot, make sure /etc/rc.d/rc.squid is executable
# chmod +x /etc/rc.d/rc.squid
and add the following to /etc/rc.d/rc.local:
# Start Squid Cache Server
if [ -x /etc/rc.d/rc.squid ]; then
/etc/rc.d/rc.squid start
fi
Save the file /etc/rc.d/rc.local and exit.
Now, we will edit the configuration file. You will need your IP Address and computer name. You can get the IP Address by using the command ifconfig and look for eth0 (wired network) or wlan0 (wireless network). Here is a portion of the response to ifconfig:
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.120 netmask 255.255.255.0 broadcast 192.168.1.255
Inet has the IP Address, in this case 192.168.1.120
You will also need your computer name. You can get it by running the following in a Terminal:
$ cat /etc/HOSTNAME
banana.linux.net
My computer name is banana
BLACKLIST/WHITELIST SETUP
We will also be adding the ACL and blocking functions based on the Shallalist URL Blacklists. Shallalist provides free blacklists for personal use and are generated daily. There are about 1.7 Million Entries. The untarred shallalist.tar.gz tarball will leave a folder named "BL" which contains a bunch of sub-folders with the blacklised domains. You need to decide where the Blacklists will reside since we have to put the file location in the Squid Configuration File. For this tutorial, I will keep the blacklists in /var/lib/squid/blacklists
Note - SquidGuard keeps the blacklists in /var/lib/squidGuard/db
UfdbGuard keeps the lists in /var/ufdbguard/blacklists/
DansGuardian keeps its blacklists in /etc/dansguardian/lists/blacklists/
Place the personal website entries you want whitelisted in the "alwaysallow/domains" file and the websites you want blacklisted in the "alwaysdeny/domains" file. These are your personal entries. You will of course need to create the alwaysallow and alwaysdeny folders along with a blank "domains" text file in each.
http://www.shallalist.de/
# mkdir -p /var/lib/squid/blacklists
# cp -R /location/of/untarred/blacklists/BL/* /var/lib/squid/blacklists
# mkdir /var/lib/squid/blacklists/alwaysallow
# mkdir /var/lib/squid/blacklists/alwaysdeny
# touch /var/lib/squid/blacklists/alwaysallow/domains
# touch /var/lib/squid/blacklists/alwaysdeny/domains
# chown -R squid:squid /var/lib/squid
# chmod 0750 /var/lib/squid
FILE EXTENSION BLOCKING
Blocking Downloading of certain files based on File Extension. Create the file /etc/squid/blocked.files.acl and put the file extensions that you do not want users to download. The following list shows the format for *.exe, *.avi, *.mpg, *.mpeg, *.mp3, *.mp4, *.flv, and *.msi
# vi /etc/squid/blocked.files.acl
\.[Ee][Xx][Ee]$
\.[Aa][Vv][Ii]$
\.[Mm][Pp][Gg]$
\.[Mm][Pp][Ee][Gg]$
\.[Mm][Pp]3$
\.[Mm][Pp]4$
\.[Ff][Ll][Vv]$
\.[Mm][Ss][Ii]$
Save the file and exit.
ERROR TEMPLATES
Now, create the ERR_BLOCKED_FILES and ERR_WEBSITE_DENIED html file since they will be referenced in the configuration file. Just copy over /usr/share/squid/errors/templates/ERR_ACCESS_DENIED to your home directory, edit the text using Geany or Bluefish, save the file as ERR_BLOCKED_FILES and ERR_WEBSITE_DENIED, then check the file by opening it with Konqueror/Firefox to see that it looks OK, then I copy it over to /usr/share/squid/errors/templates/
SQUID.CONF SETUP
Edit the file /etc/squid/squid.conf. I placed most of the configuration file below. The missing part is from all the blacklist folders that must be entered. The comments are in my previous tutorial for Squid or they are in the default Squid Configuration File.
# vi /etc/squid/squid.conf
acl blockfiles urlpath_regex -i "/etc/squid/blocked.files.acl"
deny_info ERR_BLOCKED_FILES blockfiles
http_access deny blockfiles
acl Alwaysallow dstdomain "/var/lib/squid/blacklists/alwaysallow/domains"
http_access allow Alwaysallow
acl Alwaysdeny dstdomain "/var/lib/squid/blacklists/alwaysdeny/domains"
deny_info ERR_WEBSITE_DENIED Alwaysdeny
http_access deny Alwaysdeny
acl Adv dstdomain "/var/lib/squid/blacklists/adv/domains"
deny_info ERR_WEBSITE_DENIED Adv
http_access deny Adv
Fill in the configuration file with all the other blacklist categories. Be mindful of sub-directories within the main folder.
acl Webmail dstdomain "/var/lib/squid/blacklists/webmail/domains"
deny_info ERR_WEBSITE_DENIED Webmail
http_access deny Webmail
acl Webphone dstdomain "/var/lib/squid/blacklists/webphone/domains"
deny_info ERR_WEBSITE_DENIED Webphone
http_access deny Webphone
acl Webradio dstdomain "/var/lib/squid/blacklists/webradio/domains"
deny_info ERR_WEBSITE_DENIED Webradio
http_access deny Webradio
acl Webtv dstdomain "/var/lib/squid/blacklists/webtv/domains"
deny_info ERR_WEBSITE_DENIED Webtv
http_access deny Webtv
dns_nameservers 8.8.8.8
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.1.0/24
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow !Safe_ports
http_access allow CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet
http_access allow localhost
http_access deny all
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
http_port 3128
cache_dir ufs /var/cache/squid/ 5000 16 256
access_log /var/log/squid/access.log squid
cache_store_log /var/log/squid/store.log
logfile_rotate 0
pid_filename /var/run/squid/squid.pid
cache_log /var/log/squid/cache.log
coredump_dir /var/log/squid/cache
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
cache_effective_user squid
cache_effective_group squid
visible_hostname banana
STARTING SQUID
Run squid once to create the directories Squid needs.
# /usr/sbin/squid -z
2014/09/02 20:09:47 kid1| Set Current Directory to /var/log/squid/cache
2014/09/02 20:09:47 kid1| Creating missing swap directories
2014/09/02 20:09:47 kid1| /var/cache/squid/ exists
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//00
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//01
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//02
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//03
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//04
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//05
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//06
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//07
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//08
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//09
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0A
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0B
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0C
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0D
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0E
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0F
<ENTER>
Run squid again and check for errors. There were none in my configuration.
# /usr/sbin/squid
To strip all the comments out of the configuration file, run the following after making all the changes (Be SURE to back up the old file as it has all the comments!):
# sed -i "/^#/d;/^ *$/d" /etc/squid/squid.conf
DIRECTORY PERMISSIONS
Squid stores the cache directories locally. Since we have a squid user and group with limited rights, change the permissions so only the squid user has read/write/execute access to these directories. The Sans Institute Paper on securing Squid recommends setting the sticky bit in the directories’ permissions. Setting the sticky bit on these directories will prevent data from being deleted or altered by any other user or system process. Squid and SquidGuard are very picky with regard to permissions!
# chown -R squid:squid /var/cache/squid
# chmod 1750 /var/cache/squid
# ls -al /var/cache
drwxr-x--T 18 squid squid 4096 Aug 4 11:33 squid/
# chown -R squid:squid /var/log/squid
# chmod 750 /var/log/squid
# ls -al /var/log
drwxr-x--- 4 squid squid 4096 Aug 3 20:37 squid/
Reboot your machine to start Squid (dmesg will show "Starting Squid /usr/sbin/squid -F") and run the "netstat -lundt" command when you login:
root@apple:~# netstat -lundt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp6 0 0 :::3128 :::* LISTEN
Check that Squid is indeed running as the user "squid"
# ps aufx | grep squid
squid 889 0.4 2.1 48680 45228 ? S 17:26 0:15 \_ (squid-1) -F
Now, configure your Web Browsers to use Squid.
SQUID MIDORI CONFIGURATION
Click Menu > Preferences > Network Tab
Proxy Server: HTTP Proxy Server
URI: 192.168.1.120
Port: 3128
Web Cache: 0 MB
Identify as: Automatic
SQUID CHROMIUM CONFIGURATION
To setup Chromium so that all requests are sent to Squid, you need to do start Chromium from a Terminal with the switch "proxy-server":
$ /usr/bin/chromium --proxy-server=192.168.1.120:3128
SQUID FIREFOX CONFIGURATION
To setup Mozilla so that all requests are sent to Squid, you need to do the following:
Go to Tools > Preferences > Advanced Button > Network Tab > Connection Section > Settings Button
The Default is * Use system proxy settings
Change to the following:
Note - HTTP Proxy needs to be set to the IP Address of your Local Machine, in my case, 192.168.1.120
Connection--Settings
Configure Proxies To Access The Internet
CHECK Manual proxy configuration
HTTP Proxy: 192.168.1.120 Port: 3128
UNCHECK Use this proxy server for all protocols
Cached Web Content--Clear Now
CHECK Override Automatic Cache Management
Limit cache to 0 MB of space
Click "OK" to exit the dialog box. Close Firefox and restart. If you get your Home Page, it worked!
TESTING EXTENSION BLOCKING
When you try to download a file with an extension that is listed in the /etc/squid/blocked.files.acl file, you will get a Squid Error Message, in this case, the one we copied over as ERR_BLOCKED_FILES. If you want to change the message text of that file, you can edit the HTML with Geany or Bluefish or any Text Editor and just change the text to anything you want.
ERROR
The requested URL has been blocked. The following error was encountered while trying to retrieve the URL: http://donate.libreoffice.org/home/d....0_Win_x86.msi
Access Denied.
Access control blocks your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
Your cache administrator is webmaster.
LOG ERROR MESSAGES
If anything goes wrong with Squid, try looking at the log messages:
# cat /var/log/squid/cache.log
UFDBGUARD FILTER
If you are using UfdbGuard for Squid, then add these two lines to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
url_rewrite_program /usr/sbin/ufdbgclient -l /var/log/ufdbguard
url_rewrite_children 64
SQUIDGUARD
If you are using SquidGuard for Squid, then add this line to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
PRIVOXY
If you are using Privoxy with Squid, then add these lines to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
cache_peer localhost parent 8118 0 default no-query no-digest no-netdb-exchange
never_direct allow all
SECURING SQUID
These are additional /etc/squid/squid.conf options that you can add to the beginning of the Squid Configuration File. These options reduce the amount of information leakage from your browser. I am placing them here since they are handy to know. If you are using Privoxy with Squid, then the configuration of Privoxy can basically do the same thing as these Squid options. Use Privoxy to control the headers. These particular options violate the HTTP Standard.
This group of options was called the "http_anonymizer standard"
via off
forwarded_for off
request_header_access From deny all
request_header_access Referer deny all
request_header_access User-Agent deny all
This group of options was called the "http_anonymizer paranoid"
via off
forwarded_for off
request_header_access Authorization allow all
request_header_access Proxy-Authorization allow all
request_header_access Cache-Control allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Connection allow all
request_header_access All deny all
WEB CACHING HIGH PERFORMANCE TIPS
Source - High Performance Web Caching With Squid By Joe Cooper
1. If you are able update your physical platter hard drive to a Solid-State Hard Drive. The read and writes are up to 10 times faster than the physical platter hard drives since there are no moving parts on a Solid-State Hard Drive.
2. Upgrade the RAM. Maxing out the RAM will improve your system performance. Squid uses more RAM as the cache gets bigger. A number you can use is 10MB of RAM for every 1GB of cache space.
3. If you have a separate /var partition for your Web Cache, add "noatime" to /etc/fstab so that inode access times are not updated on this filesystem (e.g., for faster access on the news spool to speed up news servers). It disables access time updates on files, saving one write for every read.
/dev/sdb5 /var ext4 defaults,noatime 1 2
4. Use ReiserFS as the File System for the cache partition. In Squid's case, it is /var. ReiserFS is a journalling file system that is very fast when dealing with many small files. Squid produces many small files in the cache. r5 is a modified version of the rupasov hash. It is used by default in ReiserFS and is the best choice unless the filesystem has huge directories and unusual file-name patterns. By default, Reiserfs stores small files and 'file tails' directly into its tree to save disk space. This confuses some utilities such as LILO(8) and adds to overhead. The 'notail' option is used to disable packing of files into the tree, thus increasing speed.
Your /etc/fstab will look like the following:
/dev/sdb5 /var reiserfs notail,noatime 1 2
Source - https://sites.google.com/site/easylinuxtipsproject/ssd and wikipedia
Source - https://wiki.archlinux.org/index.php/Solid_State_Drives
For an SSD, choose ext4. After the installation, add the "noatime" option to any ext4 file system except the swap partition! It stops "access time stamp" that the Operating System stamps on a file, thus saving writes to the SSD. Here is what the entry looks like in /etc/fstab:
/dev/sda2 / ext4 defaults,noatime 1 1
Modern SSD's support the trim function. Trim allows your Operating System to tell the SSD Drive which blocks are no longer in use and can be wiped internally. The normal operations that are performed by Operating Systems on rotating platter Hard Drives result in a significant performance degradation when done to SSD's. Trim was developed to solve this problem.
From Linux Kernel 3.7 onwards, the following file systems support Trim: btrfs, ext4, jfs, xfs. You can verify that your SSD supports trim by running the following command:
# hdparm -I /dev/sda | grep TRIM
* Data Set Management TRIM supported (limit 8 blocks)
You can change your file system mount options to get the benefits of trim. If you use the discard option, you do not need to run the fstrim command!
# vi /etc/fstab
/dev/sda1 / ext4 defaults,noatime,discard 0 1
/dev/sda2 /home ext4 defaults,noatime,discard 0 2
Save the file and exit.
If you want to run the trim command periodically, you don't need the discard file system option. You run the "fstrim" on each mount point, not on the device (i.e. /dev/sda1) Here are the commands for my setup:
# fstrim -v /
# fstrim -v /tmp
# fstrim -v /usr
# fstrim -v /var
# fstrim -v /home
Reiser4 supports TRIM and the "discard" file system option from the patches for Kernel 3.16.2 onwards. If you are using Slackware -Current Kernel 3.14.29, you can use the config file to merge with the new kernel or you can get the 3.18.1 configuration file from the /testing directory in Slackware -Current. The patch for Reiser4 is for Kernel 3.18.6. You will need to get the reiser4progs-1.0.9 and libaal-1.0.6 tarball source since these have the TRIM functionality and the SlackBuild Framework from SlackBuilds.org to build the tools.
5. File Descriptors
Source - http://slackwiki.com/Resource_Limits and Squid-Cache Wiki
If Squid gives you "Too many open files" error message, you are most likely running out of file descriptors. You may have also hit the system limit on total file descriptors for all processes. Normally, the User Limits (Soft) are one-fourth of the System Limits (Hard)
You can check the Soft Limit (User Limits) and the Hard Limit (System Limits) with the following commands:
# ulimit -Sn
1024
# ulimit -Hn
4096
To change the Soft Limit to 8192 and the Hard Limit to 32768, do the following:
# cp /sbin/initscript.sample /etc/initscript
Edit the file /etc/initscript and make it look like this:
# Set umask to safe level, and enable core dumps.
# umask 022
# ulimit -c 2097151
# PATH=/bin:/sbin:/usr/bin:/usr/sbin
# export PATH
# Raise the Hard Limits
ulimit -Hn 32768
# Raise the Soft Limits
ulimit -Sn 8192
# Execute the program.
eval exec "$4"
Save the file and exit. Slackware will automatically run /etc/initscript. You MUST leave the last line as "eval exec "$4"" or your system will lock up and you will have to use a Rescue CD to erase the /etc/initscript file. When you reboot, you can check the soft and hard limits again to make sure it worked.
# ulimit -Sn
8192
# ulimit -Hn
32768
6. Sysctl Kernel Tunables
http://www.squid-cache.org/mail-arch...1101/0042.html
The ip_local_port_range defines the local port range that is used by TCP and UDP Traffic. Here are the default listings and some suggested tubing parameters.
# cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000
The tcp_max_syn_backlog increases the number of outstanding syn requests allowed.
# cat /proc/sys/net/ipv4/tcp_max_syn_backlog
128
Edit the file /etc/sysctl.conf and add the following for both the parameters shown.
# vi /etc/sysctl.conf
# Increase system IP port limits
net.ipv4.ip_local_port_range = 1024 65535
# Increase Syn Backlog
net.ipv4.tcp_max_syn_backlog = 4096
With thanks to www.basicconfig.com and Arch Wiki
If you will be using the default user "nobody", you can go ahead and run the script as is. You may want to create a new user and group for squid since this user and group functions like a sandbox for Squid and will not have a functioning shell. If you want to run squid as the user "squid", then create the user and group "squid". The group ID number I am using is 317 for uid/gid "squid" which does not interfere with the SlackBuilds.org UID/GID list.
# groupadd -g 317 squid
# useradd -d /dev/null -s /bin/false -u 317 -g 317 squid
The version currently available at SlackBuilds.org is Squid-3.4.10. I am using Squid-3.4.11. The changes to the SlackBuild are: 1) Change the Version Number 2) Change the Squid User/Group from "nobody" to "squid" and 3) add the Configuration Option --with-default-user=squid
Here are the snippets for the SlackBuild revised for Squid-3.4.11:
VERSION=${VERSION:-3.4.11}
SQUIDUSER=${SQUIDUSER:-squid}
SQUIDGROUP=${SQUIDGROUP:-squid}
CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
--prefix=/usr \
--libdir=/usr/lib${LIBDIRSUFFIX} \
--sysconfdir=/etc/squid \
--localstatedir=/var/log/squid \
--datadir=/usr/share/squid \
--with-pidfile=/var/run/squid \
--mandir=/usr/man \
--with-logdir=/var/log/squid \
--with-default-user=squid \
--enable-snmp \
--enable-auth \
--enable-auth-basic \
--enable-linux-netfilter \
--enable-async-io \
--build=$ARCH-slackware-linux \
--disable-strict-error-checking
After installing the package, if you need to start squid at boot, make sure /etc/rc.d/rc.squid is executable
# chmod +x /etc/rc.d/rc.squid
and add the following to /etc/rc.d/rc.local:
# Start Squid Cache Server
if [ -x /etc/rc.d/rc.squid ]; then
/etc/rc.d/rc.squid start
fi
Save the file /etc/rc.d/rc.local and exit.
Now, we will edit the configuration file. You will need your IP Address and computer name. You can get the IP Address by using the command ifconfig and look for eth0 (wired network) or wlan0 (wireless network). Here is a portion of the response to ifconfig:
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.120 netmask 255.255.255.0 broadcast 192.168.1.255
Inet has the IP Address, in this case 192.168.1.120
You will also need your computer name. You can get it by running the following in a Terminal:
$ cat /etc/HOSTNAME
banana.linux.net
My computer name is banana
BLACKLIST/WHITELIST SETUP
We will also be adding the ACL and blocking functions based on the Shallalist URL Blacklists. Shallalist provides free blacklists for personal use and are generated daily. There are about 1.7 Million Entries. The untarred shallalist.tar.gz tarball will leave a folder named "BL" which contains a bunch of sub-folders with the blacklised domains. You need to decide where the Blacklists will reside since we have to put the file location in the Squid Configuration File. For this tutorial, I will keep the blacklists in /var/lib/squid/blacklists
Note - SquidGuard keeps the blacklists in /var/lib/squidGuard/db
UfdbGuard keeps the lists in /var/ufdbguard/blacklists/
DansGuardian keeps its blacklists in /etc/dansguardian/lists/blacklists/
Place the personal website entries you want whitelisted in the "alwaysallow/domains" file and the websites you want blacklisted in the "alwaysdeny/domains" file. These are your personal entries. You will of course need to create the alwaysallow and alwaysdeny folders along with a blank "domains" text file in each.
http://www.shallalist.de/
# mkdir -p /var/lib/squid/blacklists
# cp -R /location/of/untarred/blacklists/BL/* /var/lib/squid/blacklists
# mkdir /var/lib/squid/blacklists/alwaysallow
# mkdir /var/lib/squid/blacklists/alwaysdeny
# touch /var/lib/squid/blacklists/alwaysallow/domains
# touch /var/lib/squid/blacklists/alwaysdeny/domains
# chown -R squid:squid /var/lib/squid
# chmod 0750 /var/lib/squid
FILE EXTENSION BLOCKING
Blocking Downloading of certain files based on File Extension. Create the file /etc/squid/blocked.files.acl and put the file extensions that you do not want users to download. The following list shows the format for *.exe, *.avi, *.mpg, *.mpeg, *.mp3, *.mp4, *.flv, and *.msi
# vi /etc/squid/blocked.files.acl
\.[Ee][Xx][Ee]$
\.[Aa][Vv][Ii]$
\.[Mm][Pp][Gg]$
\.[Mm][Pp][Ee][Gg]$
\.[Mm][Pp]3$
\.[Mm][Pp]4$
\.[Ff][Ll][Vv]$
\.[Mm][Ss][Ii]$
Save the file and exit.
ERROR TEMPLATES
Now, create the ERR_BLOCKED_FILES and ERR_WEBSITE_DENIED html file since they will be referenced in the configuration file. Just copy over /usr/share/squid/errors/templates/ERR_ACCESS_DENIED to your home directory, edit the text using Geany or Bluefish, save the file as ERR_BLOCKED_FILES and ERR_WEBSITE_DENIED, then check the file by opening it with Konqueror/Firefox to see that it looks OK, then I copy it over to /usr/share/squid/errors/templates/
SQUID.CONF SETUP
Edit the file /etc/squid/squid.conf. I placed most of the configuration file below. The missing part is from all the blacklist folders that must be entered. The comments are in my previous tutorial for Squid or they are in the default Squid Configuration File.
# vi /etc/squid/squid.conf
acl blockfiles urlpath_regex -i "/etc/squid/blocked.files.acl"
deny_info ERR_BLOCKED_FILES blockfiles
http_access deny blockfiles
acl Alwaysallow dstdomain "/var/lib/squid/blacklists/alwaysallow/domains"
http_access allow Alwaysallow
acl Alwaysdeny dstdomain "/var/lib/squid/blacklists/alwaysdeny/domains"
deny_info ERR_WEBSITE_DENIED Alwaysdeny
http_access deny Alwaysdeny
acl Adv dstdomain "/var/lib/squid/blacklists/adv/domains"
deny_info ERR_WEBSITE_DENIED Adv
http_access deny Adv
Fill in the configuration file with all the other blacklist categories. Be mindful of sub-directories within the main folder.
acl Webmail dstdomain "/var/lib/squid/blacklists/webmail/domains"
deny_info ERR_WEBSITE_DENIED Webmail
http_access deny Webmail
acl Webphone dstdomain "/var/lib/squid/blacklists/webphone/domains"
deny_info ERR_WEBSITE_DENIED Webphone
http_access deny Webphone
acl Webradio dstdomain "/var/lib/squid/blacklists/webradio/domains"
deny_info ERR_WEBSITE_DENIED Webradio
http_access deny Webradio
acl Webtv dstdomain "/var/lib/squid/blacklists/webtv/domains"
deny_info ERR_WEBSITE_DENIED Webtv
http_access deny Webtv
dns_nameservers 8.8.8.8
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.1.0/24
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow !Safe_ports
http_access allow CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet
http_access allow localhost
http_access deny all
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
http_port 3128
cache_dir ufs /var/cache/squid/ 5000 16 256
access_log /var/log/squid/access.log squid
cache_store_log /var/log/squid/store.log
logfile_rotate 0
pid_filename /var/run/squid/squid.pid
cache_log /var/log/squid/cache.log
coredump_dir /var/log/squid/cache
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
cache_effective_user squid
cache_effective_group squid
visible_hostname banana
STARTING SQUID
Run squid once to create the directories Squid needs.
# /usr/sbin/squid -z
2014/09/02 20:09:47 kid1| Set Current Directory to /var/log/squid/cache
2014/09/02 20:09:47 kid1| Creating missing swap directories
2014/09/02 20:09:47 kid1| /var/cache/squid/ exists
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//00
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//01
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//02
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//03
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//04
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//05
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//06
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//07
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//08
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//09
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0A
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0B
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0C
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0D
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0E
2014/09/02 20:09:47 kid1| Making directories in /var/cache/squid//0F
<ENTER>
Run squid again and check for errors. There were none in my configuration.
# /usr/sbin/squid
To strip all the comments out of the configuration file, run the following after making all the changes (Be SURE to back up the old file as it has all the comments!):
# sed -i "/^#/d;/^ *$/d" /etc/squid/squid.conf
DIRECTORY PERMISSIONS
Squid stores the cache directories locally. Since we have a squid user and group with limited rights, change the permissions so only the squid user has read/write/execute access to these directories. The Sans Institute Paper on securing Squid recommends setting the sticky bit in the directories’ permissions. Setting the sticky bit on these directories will prevent data from being deleted or altered by any other user or system process. Squid and SquidGuard are very picky with regard to permissions!
# chown -R squid:squid /var/cache/squid
# chmod 1750 /var/cache/squid
# ls -al /var/cache
drwxr-x--T 18 squid squid 4096 Aug 4 11:33 squid/
# chown -R squid:squid /var/log/squid
# chmod 750 /var/log/squid
# ls -al /var/log
drwxr-x--- 4 squid squid 4096 Aug 3 20:37 squid/
Reboot your machine to start Squid (dmesg will show "Starting Squid /usr/sbin/squid -F") and run the "netstat -lundt" command when you login:
root@apple:~# netstat -lundt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp6 0 0 :::3128 :::* LISTEN
Check that Squid is indeed running as the user "squid"
# ps aufx | grep squid
squid 889 0.4 2.1 48680 45228 ? S 17:26 0:15 \_ (squid-1) -F
Now, configure your Web Browsers to use Squid.
SQUID MIDORI CONFIGURATION
Click Menu > Preferences > Network Tab
Proxy Server: HTTP Proxy Server
URI: 192.168.1.120
Port: 3128
Web Cache: 0 MB
Identify as: Automatic
SQUID CHROMIUM CONFIGURATION
To setup Chromium so that all requests are sent to Squid, you need to do start Chromium from a Terminal with the switch "proxy-server":
$ /usr/bin/chromium --proxy-server=192.168.1.120:3128
SQUID FIREFOX CONFIGURATION
To setup Mozilla so that all requests are sent to Squid, you need to do the following:
Go to Tools > Preferences > Advanced Button > Network Tab > Connection Section > Settings Button
The Default is * Use system proxy settings
Change to the following:
Note - HTTP Proxy needs to be set to the IP Address of your Local Machine, in my case, 192.168.1.120
Connection--Settings
Configure Proxies To Access The Internet
CHECK Manual proxy configuration
HTTP Proxy: 192.168.1.120 Port: 3128
UNCHECK Use this proxy server for all protocols
Cached Web Content--Clear Now
CHECK Override Automatic Cache Management
Limit cache to 0 MB of space
Click "OK" to exit the dialog box. Close Firefox and restart. If you get your Home Page, it worked!
TESTING EXTENSION BLOCKING
When you try to download a file with an extension that is listed in the /etc/squid/blocked.files.acl file, you will get a Squid Error Message, in this case, the one we copied over as ERR_BLOCKED_FILES. If you want to change the message text of that file, you can edit the HTML with Geany or Bluefish or any Text Editor and just change the text to anything you want.
ERROR
The requested URL has been blocked. The following error was encountered while trying to retrieve the URL: http://donate.libreoffice.org/home/d....0_Win_x86.msi
Access Denied.
Access control blocks your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
Your cache administrator is webmaster.
LOG ERROR MESSAGES
If anything goes wrong with Squid, try looking at the log messages:
# cat /var/log/squid/cache.log
UFDBGUARD FILTER
If you are using UfdbGuard for Squid, then add these two lines to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
url_rewrite_program /usr/sbin/ufdbgclient -l /var/log/ufdbguard
url_rewrite_children 64
SQUIDGUARD
If you are using SquidGuard for Squid, then add this line to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
PRIVOXY
If you are using Privoxy with Squid, then add these lines to the top of the Squid Configuration File. The Browser will be configured to send the Web Traffic to Squid 192.168.1.120:3128 as shown above.
cache_peer localhost parent 8118 0 default no-query no-digest no-netdb-exchange
never_direct allow all
SECURING SQUID
These are additional /etc/squid/squid.conf options that you can add to the beginning of the Squid Configuration File. These options reduce the amount of information leakage from your browser. I am placing them here since they are handy to know. If you are using Privoxy with Squid, then the configuration of Privoxy can basically do the same thing as these Squid options. Use Privoxy to control the headers. These particular options violate the HTTP Standard.
This group of options was called the "http_anonymizer standard"
via off
forwarded_for off
request_header_access From deny all
request_header_access Referer deny all
request_header_access User-Agent deny all
This group of options was called the "http_anonymizer paranoid"
via off
forwarded_for off
request_header_access Authorization allow all
request_header_access Proxy-Authorization allow all
request_header_access Cache-Control allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Connection allow all
request_header_access All deny all
WEB CACHING HIGH PERFORMANCE TIPS
Source - High Performance Web Caching With Squid By Joe Cooper
1. If you are able update your physical platter hard drive to a Solid-State Hard Drive. The read and writes are up to 10 times faster than the physical platter hard drives since there are no moving parts on a Solid-State Hard Drive.
2. Upgrade the RAM. Maxing out the RAM will improve your system performance. Squid uses more RAM as the cache gets bigger. A number you can use is 10MB of RAM for every 1GB of cache space.
3. If you have a separate /var partition for your Web Cache, add "noatime" to /etc/fstab so that inode access times are not updated on this filesystem (e.g., for faster access on the news spool to speed up news servers). It disables access time updates on files, saving one write for every read.
/dev/sdb5 /var ext4 defaults,noatime 1 2
4. Use ReiserFS as the File System for the cache partition. In Squid's case, it is /var. ReiserFS is a journalling file system that is very fast when dealing with many small files. Squid produces many small files in the cache. r5 is a modified version of the rupasov hash. It is used by default in ReiserFS and is the best choice unless the filesystem has huge directories and unusual file-name patterns. By default, Reiserfs stores small files and 'file tails' directly into its tree to save disk space. This confuses some utilities such as LILO(8) and adds to overhead. The 'notail' option is used to disable packing of files into the tree, thus increasing speed.
Your /etc/fstab will look like the following:
/dev/sdb5 /var reiserfs notail,noatime 1 2
Source - https://sites.google.com/site/easylinuxtipsproject/ssd and wikipedia
Source - https://wiki.archlinux.org/index.php/Solid_State_Drives
For an SSD, choose ext4. After the installation, add the "noatime" option to any ext4 file system except the swap partition! It stops "access time stamp" that the Operating System stamps on a file, thus saving writes to the SSD. Here is what the entry looks like in /etc/fstab:
/dev/sda2 / ext4 defaults,noatime 1 1
Modern SSD's support the trim function. Trim allows your Operating System to tell the SSD Drive which blocks are no longer in use and can be wiped internally. The normal operations that are performed by Operating Systems on rotating platter Hard Drives result in a significant performance degradation when done to SSD's. Trim was developed to solve this problem.
From Linux Kernel 3.7 onwards, the following file systems support Trim: btrfs, ext4, jfs, xfs. You can verify that your SSD supports trim by running the following command:
# hdparm -I /dev/sda | grep TRIM
* Data Set Management TRIM supported (limit 8 blocks)
You can change your file system mount options to get the benefits of trim. If you use the discard option, you do not need to run the fstrim command!
# vi /etc/fstab
/dev/sda1 / ext4 defaults,noatime,discard 0 1
/dev/sda2 /home ext4 defaults,noatime,discard 0 2
Save the file and exit.
If you want to run the trim command periodically, you don't need the discard file system option. You run the "fstrim" on each mount point, not on the device (i.e. /dev/sda1) Here are the commands for my setup:
# fstrim -v /
# fstrim -v /tmp
# fstrim -v /usr
# fstrim -v /var
# fstrim -v /home
Reiser4 supports TRIM and the "discard" file system option from the patches for Kernel 3.16.2 onwards. If you are using Slackware -Current Kernel 3.14.29, you can use the config file to merge with the new kernel or you can get the 3.18.1 configuration file from the /testing directory in Slackware -Current. The patch for Reiser4 is for Kernel 3.18.6. You will need to get the reiser4progs-1.0.9 and libaal-1.0.6 tarball source since these have the TRIM functionality and the SlackBuild Framework from SlackBuilds.org to build the tools.
5. File Descriptors
Source - http://slackwiki.com/Resource_Limits and Squid-Cache Wiki
If Squid gives you "Too many open files" error message, you are most likely running out of file descriptors. You may have also hit the system limit on total file descriptors for all processes. Normally, the User Limits (Soft) are one-fourth of the System Limits (Hard)
You can check the Soft Limit (User Limits) and the Hard Limit (System Limits) with the following commands:
# ulimit -Sn
1024
# ulimit -Hn
4096
To change the Soft Limit to 8192 and the Hard Limit to 32768, do the following:
# cp /sbin/initscript.sample /etc/initscript
Edit the file /etc/initscript and make it look like this:
# Set umask to safe level, and enable core dumps.
# umask 022
# ulimit -c 2097151
# PATH=/bin:/sbin:/usr/bin:/usr/sbin
# export PATH
# Raise the Hard Limits
ulimit -Hn 32768
# Raise the Soft Limits
ulimit -Sn 8192
# Execute the program.
eval exec "$4"
Save the file and exit. Slackware will automatically run /etc/initscript. You MUST leave the last line as "eval exec "$4"" or your system will lock up and you will have to use a Rescue CD to erase the /etc/initscript file. When you reboot, you can check the soft and hard limits again to make sure it worked.
# ulimit -Sn
8192
# ulimit -Hn
32768
6. Sysctl Kernel Tunables
http://www.squid-cache.org/mail-arch...1101/0042.html
The ip_local_port_range defines the local port range that is used by TCP and UDP Traffic. Here are the default listings and some suggested tubing parameters.
# cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000
The tcp_max_syn_backlog increases the number of outstanding syn requests allowed.
# cat /proc/sys/net/ipv4/tcp_max_syn_backlog
128
Edit the file /etc/sysctl.conf and add the following for both the parameters shown.
# vi /etc/sysctl.conf
# Increase system IP port limits
net.ipv4.ip_local_port_range = 1024 65535
# Increase Syn Backlog
net.ipv4.tcp_max_syn_backlog = 4096
Total Comments 0
