Saturday, June 30, 2012

SQUID / LUSCA DI UBUNTU

1#  Installing SQUID / LUSCA in UBUNTU

1mkdir /temp
2cd /temp
4tar xzvf LUSCA_HEAD-r14809.tar.gz
Update & Install some tools for ubuntu
1sudo apt-get update
2sudo apt-get install gcc build-essential sharutils ccze libzip-dev automake1.9
3cd LUSCA_HEAD-r14809
Now compile LUSCA with following options
1./configure '--prefix=/usr/local/squid' '--enable-removal-policies=heap,lru' '--disable-dependency-tracking' '--disable-arp-acl' '--disable-cache-digests' '--enable-cachemgr-hostname=localhost' '--disable-delay-pools' '--enable-epoll' '--enable-external-acl-helpers=ip_user' '--disable-ident-lookups' '--enable-linux-netfilter' '--disable-referer-log' '--enable-removal-policies=heap,lru' '--disable-snmp' '--disable-ssl' '--enable-storeio=aufs,null,coss' '--disable-useragent-log' '--disable-wccpv2' '--with-aio' '--with-maxfd=1048576' '--with-dl' '--with-pthreads' 'build_alias=i686-redhat-linux-gnu' 'host_alias=i686-redhat-linux-gnu' 'targe_alias=i686-redhat-linux-gnu''--enable-truncate' '--disable-unlinkd' '--with-large-files' '--disable-htcp'
2sudo make all
3sudo make install
All of Lusca/Squid configuration files can be found at
/usr/local/squid/etc/
and squid executable can be found at
/usr/local/squid/sbin/
Now We will edit squid.conf file to make it customize according to our requirements by . . .
1nano /usr/local/squid/etc/squid.conf
Delete all previously lines , and paste the following lines.
Remember following squid.conf is not very neat and clean , you will find many un necessary junk entries in it, but as I didn’t had time to clean them all, so you may clean them as per your targets and goals.
001# SQUID 2.7/ LUSCA TEST CONFIG FILE
002# Email: aacable@hotmail.com
004 
005# PORT and Transparent Option
006http_port 8080 transparent
007server_http11 on
008icp_port 0
009 
010# Cache Directory , modify it according to your system.
011# but first create directory in root by mkdir /cache1
012# and then issue this command  chown proxy:proxy /cache1
013# [for ubuntu user is proxy, in Fedora user is SQUID]
014# I have set 500 MB for caching reserved just for caching ,
015# adjust it according to your need.
016# My recommendation is to have one cache_dir per drive. zzz
017 
018#store_dir_select_algorithm round-robin
019cache_dir aufs /cache1 500 16 256
020cache_replacement_policy heap LFUDA
021memory_replacement_policy heap LFUDA
022 
023# If you want to enable DATE time n SQUID Logs,use following
024emulate_httpd_log on
025logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
026log_fqdn off
027 
028# How much days to keep users access web logs
029# You need to rotate your log files with a cron job. For example:
030# 0 0 * * * /usr/local/squid/bin/squid -k rotate
031logfile_rotate 14
032debug_options ALL,1
033cache_access_log /var/log/squid/access.log
034cache_log /var/log/squid/cache.log
035cache_store_log /var/log/squid/store.log
036 
037#I used DNSAMSQ service for fast dns resolving
038#so install by using "apt-get install dnsmasq" first
039dns_nameservers 127.0.0.1 221.132.112.8
040ftp_user anonymous@
041ftp_list_width 32
042ftp_passive on
043ftp_sanitycheck on
044 
045#ACL Section
046acl all src 0.0.0.0/0.0.0.0
047acl manager proto cache_object
048acl localhost src 127.0.0.1/255.255.255.255
049acl to_localhost dst 127.0.0.0/8
050acl SSL_ports port 443 563 # https, snews
051acl SSL_ports port 873 # rsync
052acl Safe_ports port 80 # http
053acl Safe_ports port 21 # ftp
054acl Safe_ports port 443 563 # https, snews
055acl Safe_ports port 70 # gopher
056acl Safe_ports port 210 # wais
057acl Safe_ports port 1025-65535 # unregistered ports
058acl Safe_ports port 280 # http-mgmt
059acl Safe_ports port 488 # gss-http
060acl Safe_ports port 591 # filemaker
061acl Safe_ports port 777 # multiling http
062acl Safe_ports port 631 # cups
063acl Safe_ports port 873 # rsync
064acl Safe_ports port 901 # SWAT
065acl purge method PURGE
066acl CONNECT method CONNECT
067http_access allow manager localhost
068http_access deny manager
069http_access allow purge localhost
070http_access deny purge
071http_access deny !Safe_ports
072http_access deny CONNECT !SSL_ports
073http_access allow localhost
074http_access allow all
075http_reply_access allow all
076icp_access allow all
077 
078#==========================
079# Administrative Parameters
080#==========================
081 
082# I used UBUNTU so user is proxy, in FEDORA you may use use squid
083cache_effective_user proxy
084cache_effective_group proxy
085cache_mgr aacable@hotmail.com
086visible_hostname proxy.aacable.net
087unique_hostname aacable@hotmail.com
088 
089#=============
090# ACCELERATOR
091#=============
092half_closed_clients off
093quick_abort_min 0 KB
094quick_abort_max 0 KB
095vary_ignore_expire on
096reload_into_ims on
097log_fqdn off
098memory_pools off
099cache_swap_low 98
100cache_swap_high 99
101max_filedescriptors 65536
102fqdncache_size 16384
103retry_on_error on
104offline_mode off
105pipeline_prefetch on
106# If you want to hide your proxy machine from being detected at various site use following
107via off
108 
109#============================================
110# OPTIONS WHICH AFFECT THE CACHE SIZE / zaib
111#============================================
112# If you have 4GB memory in Squid box, we will use formula of 1/3
113# You can adjust it according to your need. IF squid is taking too much of RAM
114# Then decrease it to 128 MB or even less.
115 
116cache_mem 8 MB
117minimum_object_size 0 bytes
118maximum_object_size 100 MB
119maximum_object_size_in_memory 128 KB
120 
121#============================================================$
122# SNMP , if you want to generate graphs for SQUID via MRTG
123#============================================================$
124#acl snmppublic snmp_community gl
125#snmp_port 3401
126#snmp_access allow snmppublic all
127#snmp_access allow all
128 
129#============================================================
130# ZPH , To enable cache content to be delivered at full lan speed,
131# To bypass the queue at MT.
132#============================================================
133tcp_outgoing_tos 0x30 all
134zph_mode tos
135zph_local 0x30
136zph_parent 0
137zph_option 136
138 
139# Caching Youtube
140acl videocache_allow_url url_regex -i \.youtube\.com\/get_video\?
141acl videocache_allow_url url_regex -i \.youtube\.com\/videoplayback \.youtube\.com\/videoplay \.youtube\.com\/get_video\?
142acl videocache_allow_url url_regex -i \.youtube\.[a-z][a-z]\/videoplayback \.youtube\.[a-z][a-z]\/videoplay \.youtube\.[a-z][a-z]\/get_video\?
143acl videocache_allow_url url_regex -i \.googlevideo\.com\/videoplayback \.googlevideo\.com\/videoplay \.googlevideo\.com\/get_video\?
144acl videocache_allow_url url_regex -i \.google\.com\/videoplayback \.google\.com\/videoplay \.google\.com\/get_video\?
145acl videocache_allow_url url_regex -i \.google\.[a-z][a-z]\/videoplayback \.google\.[a-z][a-z]\/videoplay \.google\.[a-z][a-z]\/get_video\?
146acl videocache_allow_url url_regex -i proxy[a-z0-9\-][a-z0-9][a-z0-9][a-z0-9]?\.dailymotion\.com\/
147acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/
148acl videocache_allow_url url_regex -i [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\.xtube\.com\/(.*)flv
149acl videocache_allow_url url_regex -i \.vimeo\.com\/(.*)\.(flv|mp4)
150acl videocache_allow_url url_regex -i va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
151acl videocache_allow_url url_regex -i \.youporn\.com\/(.*)\.flv
152acl videocache_allow_url url_regex -i \.msn\.com\.edgesuite\.net\/(.*)\.flv
153acl videocache_allow_url url_regex -i \.tube8\.com\/(.*)\.(flv|3gp)
154acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
155acl videocache_allow_url url_regex -i \.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
156acl videocache_allow_url url_regex -i \.apniisp\.com\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v)
157acl videocache_allow_url url_regex -i \.break\.com\/(.*)\.(flv|mp4)
158acl videocache_allow_url url_regex -i redtube\.com\/(.*)\.flv
159acl videocache_allow_dom dstdomain .mccont.com .metacafe.com .cdn.dailymotion.com
160acl videocache_deny_dom  dstdomain .download.youporn.com .static.blip.tv
161acl dontrewrite url_regex redbot\.org \.php
162acl getmethod method GET
163 
164storeurl_access deny dontrewrite
165storeurl_access deny !getmethod
166storeurl_access deny videocache_deny_dom
167storeurl_access allow videocache_allow_url
168storeurl_access allow videocache_allow_dom
169storeurl_access deny all
170 
171storeurl_rewrite_program /etc/squid/storeurl.pl
172storeurl_rewrite_children 7
173storeurl_rewrite_concurrency 100
174 
175acl store_rewrite_list urlpath_regex -i \/(get_video\?|videodownload\?|videoplayback.*id)
176acl store_rewrite_list urlpath_regex -i \.flv$ \.mp3$ \.mp4$ \.swf$ \
177storeurl_access allow store_rewrite_list
178storeurl_access deny all
179 
180refresh_pattern -i \.flv$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
181refresh_pattern -i \.mp3$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
182refresh_pattern -i \.mp4$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
183refresh_pattern -i \.swf$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
184refresh_pattern -i \.gif$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
185refresh_pattern -i \.jpg$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private ignore-auth
186refresh_pattern -i \.jpeg$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private  ignore-auth
187refresh_pattern -i \.exe$ 10080 80% 10080  override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache  ignore-private  ignore-auth
188 
189# 1 year = 525600 mins, 1 month = 10080 mins, 1 day = 1440
190refresh_pattern (get_video\?|videoplayback\?|videodownload\?|\.flv?)    10080 80% 10080 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
191refresh_pattern (get_video\?|videoplayback\?id|videoplayback.*id|videodownload\?|\.flv?)    10080 80% 10080 ignore-no-cache  ignore-private override-expire override-lastmod reload-into-ims
192refresh_pattern \.(ico|video-stats) 10080 80% 10080 override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod  negative-ttl=10080
193refresh_pattern \.etology\?                                     10080 80% 10080 override-expire ignore-reload ignore-no-cache
194refresh_pattern galleries\.video(\?|sz)                         10080 80% 10080 override-expire ignore-reload ignore-no-cache
195refresh_pattern brazzers\?                                      10080 80% 10080 override-expire ignore-reload ignore-no-cache
196refresh_pattern \.adtology\?                                    10080 80% 10080 override-expire ignore-reload ignore-no-cache
197refresh_pattern ^.*(utm\.gif|ads\?|rmxads\.com|ad\.z5x\.net|bh\.contextweb\.com|bstats\.adbrite\.com|a1\.interclick\.com|ad\.trafficmp\.com|ads\.cubics\.com|ad\.xtendmedia\.com|\.googlesyndication\.com|advertising\.com|yieldmanager|game-advertising\.com|pixel\.quantserve\.com|adperium\.com|doubleclick\.net|adserving\.cpxinteractive\.com|syndication\.com|media.fastclick.net).* 10080 20% 10080 ignore-no-cache  ignore-private override-expire ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
198refresh_pattern ^.*safebrowsing.*google  10080 80% 10080 override-expire ignore-reload ignore-no-cache ignore-private ignore-auth  negative-ttl=10080
199refresh_pattern ^http://((cbk|mt|khm|mlt)[0-9]?)\.google\.co(m|\.uk)    10080 80% 10080 override-expire ignore-reload ignore-private  negative-ttl=10080
200refresh_pattern ytimg\.com.*\.jpg                                       10080 80% 10080 override-expire ignore-reload
201refresh_pattern images\.friendster\.com.*\.(png|gif)                    10080 80% 10080 override-expire ignore-reload
202refresh_pattern garena\.com                                             10080 80% 10080 override-expire reload-into-ims
203refresh_pattern photobucket.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)          10080 80% 10080 override-expire ignore-reload
204refresh_pattern vid\.akm\.dailymotion\.com.*\.on2\?                     10080 80% 10080 ignore-no-cache override-expire override-lastmod
205refresh_pattern mediafire.com\/images.*\.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10080 80% 10080 reload-into-ims override-expire ignore-private
206refresh_pattern ^http:\/\/images|pics|thumbs[0-9]\.                     10080 80% 10080 reload-into-ims ignore-no-cache  ignore-reload override-expire
207refresh_pattern ^http:\/\/www.onemanga.com.*\/                          10080 80% 10080 reload-into-ims ignore-no-cache  ignore-reload override-expire
208refresh_pattern ^http://v\.okezone\.com/get_video\/([a-zA-Z0-9]) 10080 80% 10080 override-expire ignore-reload ignore-no-cache  ignore-private ignore-auth override-lastmod  negative-ttl=10080
209 
210#images facebook
211refresh_pattern -i \.facebook.com.*\.(jpg|png|gif)                      10080 80% 10080 ignore-reload override-expire ignore-no-cache
212refresh_pattern -i \.fbcdn.net.*\.(jpg|gif|png|swf|mp3)                 10080 80% 10080 ignore-reload override-expire ignore-no-cache
213refresh_pattern  static\.ak\.fbcdn\.net*\.(jpg|gif|png)                 10080 80% 10080 ignore-reload override-expire ignore-no-cache
214refresh_pattern ^http:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)        10080 80% 10080 ignore-reload override-expire ignore-no-cache
215 
216#All File
217refresh_pattern -i \.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt)       10080 80% 10080 ignore-no-cache   override-expire override-lastmod reload-into-ims
218refresh_pattern -i \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar)  10080 80% 10080 ignore-no-cache   override-expire override-lastmod reload-into-ims
219refresh_pattern -i \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|dat|ad|txt|dll)         10080 80% 10080 ignore-no-cache   override-expire override-lastmod reload-into-ims
220refresh_pattern -i \.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob) 10080 80% 10080 ignore-no-cache   override-expire override-lastmod reload-into-ims
221refresh_pattern -i \.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv) 10080 80% 10080 ignore-no-cache   override-expire override-lastmod reload-into-ims
222 
223refresh_pattern -i (/cgi-bin/|\?)  0  0%  0
224refresh_pattern ^gopher:        1440    0%      1440
225refresh_pattern ^ftp:           10080   95% 10080 override-lastmod reload-into-ims
226refresh_pattern .               1440    95% 10080 override-lastmod reload-into-ims
Now create cache dir and logs files , and assign them necessary permissions.
1mkdir /cache1
2chown proxy:proxy /cache1
3mkdir /var/log/squid
4chmod 777 /var/log/squid
Now initialize cache dir by
1/usr/local/squid/sbin/squid -z

SOTEURL.PL

Now We have to create an important file name storeurl.pl , which is very important and actually it does the
main job to pull video from cache.
1mkdir /etc/squid
2touch /etc/squid/storeurl.pl
3chmod +x /etc/squid/storeurl.pl
4nano /etc/squid/storeurl.pl
Now paste the following lines, then Save and exit.
001#!/usr/bin/perl
002# This script is NOT written or modified by me, I only copy pasted it from the internet.
003# It was First originally Writen by chudy_fernandez@yahoo.com
004# & Have been modified by various persons over the net to fix/add various functions.
005# For Example this ver was modified by member of comstuff.net to satisfy common and dynamic content.
006# th30nly @comstuff.net a.k.a invisible_theater ,
008$|=1;
009while (<>) {
010@X = split;
011#       $X[1] =~ s/&sig=.*//;
012$x = $X[0] . " ";
013$_ = $X[1];
014$u = $X[1];
015 
016#speedtest
017if (m/^http:\/\/(.*)\/speedtest\/(.*\.(jpg|txt))\?(.*)/) {
019 
020#mediafire
021}elsif (m/^http:\/\/199\.91\.15\d\.\d*\/\w{12}\/(\w*)\/(.*)/) {
022print $x . "http://www.mediafire.com.SQUIDINTERNAL/" . $1 ."/" . $2 . "\n";
023 
024#fileserve
025}elsif (m/^http:\/\/fs\w*\.fileserve\.com\/file\/(\w*)\/[\w-]*\.\/(.*)/) {
026print $x . "http://www.fileserve.com.SQUIDINTERNAL/" . $1 . "./" . $2 . "\n";
027 
028#filesonic
029}elsif (m/^http:\/\/s[0-9]*\.filesonic\.com\/download\/([0-9]*)\/(.*)/) {
030print $x . "http://www.filesonic.com.SQUIDINTERNAL/" . $1 . "\n";
031 
032#4shared
033}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/download\/(.*)\/(.*\..*)\?.*/) {
035 
036#4shared preview
037}elsif (m/^http:\/\/[a-zA-Z]{2}\d*\.4shared\.com(:8080|)\/img\/(\d*)\/\w*\/dlink__2Fdownload_2F(\w*)_3Ftsid_3D[\w-]*\/preview\.mp3\?sId=\w*/) {
039 
040#photos-X.ak.fbcdn.net where X a-z
041}elsif (m/^http:\/\/photos-[a-z](\.ak\.fbcdn\.net)(\/.*\/)(.*\.jpg)/) {
042print $x . "http://photos" . $1 . "/" . $2 . $3  . "\n";
043 
044#YX.sphotos.ak.fbcdn.net where X 1-9, Y a-z
045} elsif (m/^http:\/\/[a-z][0-9]\.sphotos\.ak\.fbcdn\.net\/(.*)\/(.*)/) {
046print $x . "http://photos.ak.fbcdn.net/" . $1  ."/". $2 . "\n";
047 
048#maps.google.com
049} elsif (m/^http:\/\/(cbk|mt|khm|mlt|tbn)[0-9]?(.google\.co(m|\.uk|\.id).*)/) {
050print $x . "http://" . $1  . $2 . "\n";
051 
052# compatibility for old cached get_video?video_id
053} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?(videoplayback\?id=.*?|video_id=.*?)\&(.*?)/) {
054$z = $2; $z =~ s/video_id=/get_video?video_id=/;
055print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $z . "\n";
056 
057# youtube fix
058} elsif (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/videoplayback\?(.*)/) {
059$p_str = $2;
060$tag = "";
061$alg = "";
062$id = "";
063$range = "";
064if ($p_str =~ m/(itag=[0-9]*)/){$tag = "&".$1}
065if ($p_str =~ m/(algorithm=[a-z]*\-[a-z]*)/){$alg = "&".$1}
066if ($p_str =~ m/(id=[a-zA-Z0-9]*)/){$id = "&".$1}
067if ($p_str =~ m/(range=[0-9\-]*)/){$range = "&".$1; $range =~ s/-//; $range =~ s/range=//; }
068print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $tag . "&" . $alg . "&" . $id . "&" . $range . "\n";
069 
070} elsif (m/^http:\/\/www\.google-analytics\.com\/__utm\.gif\?.*/) {
072 
073#Cache High Latency Ads
074} elsif (m/^http:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|\.googlesyndication\.com|yieldmanager|cpxinteractive)(.*)/) {
075$y = $3;$z = $2;
076for ($y) {
077s/pixel;.*/pixel/;
078s/activity;.*/activity/;
079s/(imgad[^&]*).*/\1/;
080s/;ord=[?0-9]*//;
081s/;&timestamp=[0-9]*//;
082s/[&?]correlator=[0-9]*//;
083s/&cookie=[^&]*//;
084s/&ga_hid=[^&]*//;
085s/&ga_vid=[^&]*//;
086s/&ga_sid=[^&]*//;
087# s/&prev_slotnames=[^&]*//
088# s/&u_his=[^&]*//;
089s/&dt=[^&]*//;
090s/&dtd=[^&]*//;
091s/&lmt=[^&]*//;
092s/(&alternate_ad_url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
093s/(&url=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
094s/(&ref=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
095s/(&cookie=http%3A%2F%2F[^(%2F)]*)[^&]*/\1/;
096s/[;&?]ord=[?0-9]*//;
097s/[;&]mpvid=[^&;]*//;
098s/&xpc=[^&]*//;
099# yieldmanager
100s/\?clickTag=[^&]*//;
101s/&u=[^&]*//;
102s/&slotname=[^&]*//;
103s/&page_slots=[^&]*//;
104}
105print $x . "http://" . $1 . $2 . $y . "\n";
106 
107#cache high latency ads
108} elsif (m/^http:\/\/(.*?)\/(ads)\?(.*?)/) {
109print $x . "http://" . $1 . "/" . $2  . "\n";
110 
111# spicific servers starts here....
112} elsif (m/^http:\/\/(www\.ziddu\.com.*\.[^\/]{3,4})\/(.*?)/) {
113print $x . "http://" . $1 . "\n";
114 
115#cdn, varialble 1st path
116} elsif (($u =~ /filehippo/) && (m/^http:\/\/(.*?)\.(.*?)\/(.*?)\/(.*)\.([a-z0-9]{3,4})(\?.*)?/)) {
117@y = ($1,$2,$4,$5);
118$y[0] =~ s/[a-z0-9]{2,5}/cdn./;
119print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
120 
121#rapidshare
122} elsif (($u =~ /rapidshare/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?)([a-z]*\.[^\/]{3}\/[a-z]*\/[0-9]*)\/(.*?)\/([^\/\?\&]{4,})$/)) {
123print $x . "http://cdn." . $3 . "/SQUIDINTERNAL/" . $5 . "\n";
124 
125} elsif (($u =~ /maxporn/) && (m/^http:\/\/([^\/]*?)\/(.*?)\/([^\/]*?)(\?.*)?$/)) {
126print $x . "http://" . $1 . "/SQUIDINTERNAL/" . $3 . "\n";
127 
128#like porn hub variables url and center part of the path, filename etention 3 or 4 with or without ? at the end
129} elsif (($u =~ /tube8|pornhub|xvideos/) && (m/^http:\/\/(([A-Za-z]+[0-9-.]+)*?(\.[a-z]*)?)\.([a-z]*[0-9]?\.[^\/]{3}\/[a-z]*)(.*?)((\/[a-z]*)?(\/[^\/]*){4}\.[^\/\?]{3,4})(\?.*)?$/)) {
130print $x . "http://cdn." . $4 . $6 . "\n";
131#...spicific servers end here.
132 
133#photos-X.ak.fbcdn.net where X a-z
134} elsif (m/^http:\/\/photos-[a-z].ak.fbcdn.net\/(.*)/) {
135print $x . "http://photos.ak.fbcdn.net/" . $1  . "\n";
136 
137#for yimg.com video
138} elsif (m/^http:\/\/(.*yimg.com)\/\/(.*)\/([^\/\?\&]*\/[^\/\?\&]*\.[^\/\?\&]{3,4})(\?.*)?$/) {
139print $x . "http://cdn.yimg.com//" . $3 . "\n";
140 
141#for yimg.com doubled
142} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*?)\.yimg\.com\/(.*?)\?(.*)/) {
143print $x . "http://cdn.yimg.com/"  . $3 . "\n";
144 
145#for yimg.com with &sig=
146} elsif (m/^http:\/\/(.*?)\.yimg\.com\/(.*)/) {
147@y = ($1,$2);
148$y[0] =~ s/[a-z]+[0-9]+/cdn/;
149$y[1] =~ s/&sig=.*//;
150print $x . "http://" . $y[0] . ".yimg.com/"  . $y[1] . "\n";
151 
152#youjizz. We use only domain and filename
153} elsif (($u =~ /media[0-9]{2,5}\.youjizz/) && (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\/([^\/\?\&]*)\.([^\/\?\&]{3,4})((\?|\%).*)?$/)) {
154@y = ($1,$2,$4,$5);
155$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
156print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
157 
158#general purpose for cdn servers. add above your specific servers.
159} elsif (m/^http:\/\/([0-9.]*?)\/\/(.*?)\.(.*)\?(.*?)/) {
160print $x . "http://squid-cdn-url//" . $2  . "." . $3 . "\n";
161 
162#generic http://variable.domain.com/path/filename."ex" "ext" or "exte" with or withour "? or %"
163} elsif (m/^http:\/\/(.*)(\.[^\.\-]*?\..*?)\/(.*)\.([^\/\?\&]{2,4})((\?|\%).*)?$/) {
164@y = ($1,$2,$3,$4);
165$y[0] =~ s/(([a-zA-A]+[0-9]+(-[a-zA-Z])?$)|(.*cdn.*)|(.*cache.*))/cdn/;
166print $x . "http://" . $y[0] . $y[1] . "/" . $y[2] . "." . $y[3] . "\n";
167 
169} elsif (m/^http:\/\/(([A-Za-z]+[0-9-]+)*?|.*cdn.*|.*cache.*)\.(.*?)\.(.*?)\/(.*)$/) {
170print $x . "http://cdn." . $3 . "." . $4 . "/" . $5 .  "\n";
171 
172# spicific extention that ends with ?
173} elsif (m/^http:\/\/(.*?)\/(.*?)\.(jp(e?g|e|2)|gif|png|tiff?|bmp|ico|flv|on2)(.*)/) {
174print $x . "http://" . $1 . "/" . $2  . "." . $3 . "\n";
175 
176# all that ends with ;
177} elsif (m/^http:\/\/(.*?)\/(.*?)\;(.*)/) {
178print $x . "http://" . $1 . "/" . $2  . "\n";
179 
180} else {
181print $x . $_ . "sucks\n";
182}
183}
Now start SQUIDServer by
1/usr/local/squid/sbin/squid
TIP:
To start SQUID Server in Debug mode, to check any erros, use

1/usr/local/squid/sbin/squid -d1                             
Read more ...

subnetting pembelajaran smk tkj

 SUBNETTING
Nomor IP terdiri dari 32 bit yang didalamnya terdapat bit untuk NETWORK ID
(NetID) dan HOST ID (HostID). Secara garis besar berikut inilah pembagian kelas IP
secara default
8 bit 24 bit
32 bit
Net ID Host ID
16 bit 16 bit
Net ID Host ID
CLASS A
CLASS B
24 bit 8 bit
Net ID Host ID
CLASS C
Gambar Pembagian kelas IP
Netmask
Ketika kita berhubungan dengan komputer lain pada suatu jaringan, selain IP yang
dibutuhkan adalah netmask. Misal kita pada IP 10.252.102.12 ingin berkirim data pada
10.252.102.135 bagaimana komputer kita memutuskan apakah ia berada pada satu
jaringan atau lain jaringan? Maka yang dilakukan adalah mengecek dulu netmask
komputer kita karena kombinasi IP dan netmask menentukan range jaringan kita.
Jika netmask kita 255.255.255.0 maka range terdiri dari atas semua IP yang
memiliki 3 byte pertama yang sama. Misal jika IP saya 10.252.102.12 dan netmask saya
255.255.255.0 maka range jaringan saya adalah 10.252.102.0-10.252.102.255 sehingga
kita bisa secara langsung berkomunukasi pada mesin yang diantara itu, jadi
10.252.102.135 berada pada jaringan yang sama yaitu 10.252.102 (lihat yang angkaangka
tercetak tebal menunjukkan dalam satu jaringan karena semua sama).
Dalam suatu organisasi komersial biasanya terdiri dari beberapa bagian, misalnya
bagian personalia/HRD, Marketing, Produksi, Keuangan, IT dsb. Setiap bagian di
perusahaan tentunya mempunyai kepentingan yang berbeda-beda. Dengan beberapa
alasan maka setiap bagian bisa dibuatkan jaringan lokal sendiri – sendiri dan antar bagian
bisa pula digabungkan jaringannya dengan bagian yang lain.
Ada beberapa alasan yang menyebabkan satu organisasi membutuhkan lebih dari
satu jaringan lokal (LAN) agar dapat mencakup seluruh organisasi :
 Teknologi yang berbeda. Dalam suatu organisasi dimungkinkan menggunakan
bermacam teknologi dalam jaringannya. Semisal teknologi ethernet akan mempunyai
LAN yang berbeda dengan teknologi FDDI.
 Sebuah jaringan mungkin dibagi menjadi jaringan yang lebih kecil karena
masalah performanasi. Sebuah LAN dengan 254 host akan memiliki performansi
yang kurang baik dibandingkan dengan LAN yang hanya mempunyai 62 host.
Semakin banyak host yang terhubung dalam satu media akan menurunkan performasi
dari jaringan. Pemecahan yang paling sedherhana adalah memecah menjadi 2 LAN.
 Departemen tertentu membutuhkan keamanan khusus sehingga solusinya memecah
menjadi jaringan sendiri.
Pembagian jaringan besar ke dalam jaringan yang kecil-kecil inilah yang disebut
sebagai subnetting. Pemecehan menggunakan konsep subnetting. Membagi jaringan
besar tunggal ke dalam sunet-subnet (sub-sub jaringan). Setiap subnet ditentukan
dengan menggunakan subnet mask bersama-sama dengan no IP.
Pada subnetmask dalam biner, seluruh bit yang berhubungan dengan netID diset 1,
sedangkan bit yang berhubungan dengan hostID diset 0.
Dalam subnetting, proses yang dilakukan ialah memakai sebagian bit hostID untuk
membentuk subnetID. Dengan demikian jumlah bit yang digunakan untuk HostID
menjadi lebih sedikit. Semakin panjang subnetID, jumlah subnet yang dibentuk semkain
banyak, namun jumlah host dalam tiap subnet menjadi semakin sedikit.
NetID HostID
NetID SubNetID HostID
NetIDBaru
Lama
Baru
HostIDBaru
Gambar pembentukan subnet
Cara Pembentukan Subnet
Misal jika jaringan kita adalah 192.168.0.0 dalm kelas B (kelas B memberikan
range 192.168.0.0 – 192.168.255.255). Ingat kelas B berarti 16 bit pertama menjadi
NetID yang dalam satu jaringan tidak berubah (dalam hal ini adalah 192.168) dan bit
selanjutya sebagai Host ID (yang merupakan nomor komputer yang terhubung ke dan
setiap komputer mempunyai no unik mulai dari 0.0 – 255.255). Jadi
netmasknya/subnetmasknya adalah 255.255.0.0
Kita dapat membagi alokasi jaringan diatas menjadi jaringan yang kebih kecil
dengan cara mengubha subnet yang ada.
Ada dua pendekatan dalam melakukan pembentukan subnet yaitu :
1. Berdasarkan jumlah jaringan yang akan dibentuk
2. Berdasarkan jumlah host yang dibentuk dalam jaringan.
Cara perhitungan subnet berdasarkan jumlah jaringan yang dibutuhkan.
1. Menentukan jumlah jaringan yang dibutuhkan dan merubahnya menjadi biner.
Misalkan kita ingin membuat 255 jaringan kecil dari nomor jaringan yang sudah
ditentukan. 255  11111111
2. Menghitung jumlah bit dari nomor 1. Dan jumlah bit inilah yang disebut sebagai
subnetID
Dari 255  11111111  jumlah bitnya adalah 8
3. Jumlah bit hostID baru adalah HosiID lama dikurangi jumlah bit nomor 2.
Misal dari contoh diatas hostIDbaru: 16 bit – 8 bit = 8 bit.
4. Isi subnetID dengan 1 dan jumlahkan dengan NetIDLama.
Jadi NetID baru kita adalah NetIDlama + SubNetID :
 11111111.11111111.11111111.00000000 (24 bit bernilai 1 biasa ditulis /24)
Berkat perhitungan di atas maka kita mempunyai 256 jaringan baru yaitu :
192.168.0.xxx, 192.168.1.xxx, 192.168.2.xxx, 192.168.3.xxx hingga
192.168.255.xxx dengan netmash 255.255.255.0.
xxx  menunjukkan hostID antara 0-255
Biasa ditulis dengan 192.168.0/24  192.168.0 menunjukkan NetID dan 24
menunjukkan subnetmask (jumlah bit yang bernilai 1 di subnetmask).
Dengan teknik ini kita bisa mengalokasikan IP address kelas B menjadi sekian
banyak jaringan yang berukuran sama.
Cara perhitungan subnet berdasarkan jumlah host adalah sebagai berikut :
1. Ubah IP dan netmask menjadi biner
IP : 192.168.1.0  11000000.10101000.00000000.00000000
Netmask : 255.255.255.0  11111111.11111111. 11111111.00000000
Panjang hostID kita adalah yang netmasknya semua 0  16 bit.
2. Memilih jumlah host terbanyak dalam suatu jaringan dan rubah menjadi biner.
Misal dalam jaringan kita membutuhkan host 25 maka menjadi 11001.
3. Hitung jumlah bit yang dibutuhkan angka biner pada nomor 1. Dan angka inilah nanti
sebagai jumlah host dalam jaringan kita.
Jumlah host 25 menjadi biner 11001 dan jumlah bitnya adalah 5.
4. Rubah netmask jaringan kita dengan cara menyisakan angka 0 sebanyak jumlah
perhitungan nomor 3.
Jadi netmasknya baru adalah 11111111.11111111.11111111.11100000
Identik dengan 255.255.255.224 jika didesimalkan.
Jadi netmask jaringan berubah dan yang awalnya hanya satu jaringan dengan range IP
dari 1 -254 menjadi 8 jaringan, dengan setiap jaringan ada 30 host/komputer
Alokasi Range IP
1 192.168.1.0 – 192.168.1.31
2 192.168.1.32 – 192.168.1.63
3 192.168.1.64 – 192.168.1.95
4 192.168.1.96 – 192.168.1.127
5 192.168.1.128 – 192.168.1.159
6 192.168.1.160 – 192.168.1.191
7 192.168.1.192 – 192.168.1.223
8 192.168.1.224 – 192.168.1.255
Nomor IP awal dan akhir setiap subnet tidak bisa dipakai. Awal dipakai ID Jaringan
(NetID) dan akhir sebagai broadcast.
Misal jaringan A 192.168.1.0 sebagai NetID dan 192.168.1.31 sebagai broadcast dan
range IP yang bisa dipakai 192.168.1.1-192.168.1.30.

jangan lupa di pahami dan banyak latihan ya
Read more ...