TransWikia.com

Video and file caching with squid lusca?

Super User Asked by Badr uz Zaman on December 27, 2020

I have configured squid lusca on Ubuntu 11.04 and also configured the video caching but the problem is the squid cannot configure the video more than 2 min long and the file of size up to 5.xx MB only.

Here is my config, please guide me how can I cache the long videos and files with squid:

# PORT and Transparent Option http_port 8080 transparent server_http11 on icp_port 0

# Cache Directory , modify it according to your system.
# but first create directory in root by mkdir /cache1
# and then issue this command  chown proxy:proxy /cache1
# [for ubuntu user is proxy, in Fedora user is SQUID]
# I have set 500 MB for caching reserved just for caching ,
# adjust it according to your need.
# My recommendation is to have one cache_dir per drive. zzz

#store_dir_select_algorithm round-robin cache_dir aufs /cache1 500 16 256 cache_replacement_policy heap LFUDA memory_replacement_policy heap
LFUDA

# If you want to enable DATE time n SQUID Logs,use following emulate_httpd_log on logformat squid %tl %6tr %>a %Ss/%03Hs %<st %rm
%ru %un %Sh/%<A %mt log_fqdn off

# How much days to keep users access web logs
# You need to rotate your log files with a cron job. For example:
# 0 0 * * * /usr/local/squid/bin/squid -k rotate logfile_rotate 14 debug_options ALL,1 cache_access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log cache_store_log
/var/log/squid/store.log

#I used DNSAMSQ service for fast dns resolving
#so install by using "apt-get install dnsmasq" first dns_nameservers 127.0.0.1 101.11.11.5 ftp_user anonymous@ ftp_list_width 32 ftp_passive on ftp_sanitycheck on

#ACL Section acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl
to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 563 # https, snews
acl SSL_ports port 873 # rsync acl Safe_ports port 80 # http acl
Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl
Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port
280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port
591 # filemaker acl Safe_ports port 777 # multiling http acl
Safe_ports port 631 # cups acl Safe_ports port 873 # rsync acl
Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method
CONNECT http_access allow manager localhost http_access deny manager
http_access allow purge localhost http_access deny purge http_access
deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow
localhost http_access allow all http_reply_access allow all icp_access
allow all

#==========================
# Administrative Parameters
#==========================

# I used UBUNTU so user is proxy, in FEDORA you may use use squid cache_effective_user proxy cache_effective_group proxy cache_mgr
[email protected] visible_hostname proxy.aacable.net unique_hostname
[email protected]

#=============
# ACCELERATOR
#============= half_closed_clients off quick_abort_min 0 KB quick_abort_max 0 KB vary_ignore_expire on reload_into_ims on log_fqdn
off memory_pools off

# If you want to hide your proxy machine from being detected at various site use following via off

#============================================
# OPTIONS WHICH AFFECT THE CACHE SIZE / zaib
#============================================
# If you have 4GB memory in Squid box, we will use formula of 1/3
# You can adjust it according to your need. IF squid is taking too much of RAM
# Then decrease it to 128 MB or even less.

cache_mem 256 MB minimum_object_size 512 bytes maximum_object_size 500
MB maximum_object_size_in_memory 128 KB

#============================================================$
# SNMP , if you want to generate graphs for SQUID via MRTG
#============================================================$
#acl snmppublic snmp_community gl
#snmp_port 3401
#snmp_access allow snmppublic all
#snmp_access allow all

#============================================================
# ZPH , To enable cache content to be delivered at full lan speed,
# To bypass the queue at MT.
#============================================================ tcp_outgoing_tos 0x30 all zph_mode tos zph_local 0x30 zph_parent 0
zph_option 136

# Caching Youtube acl videocache_allow_url url_regex -i .youtube.com/get_video? acl videocache_allow_url url_regex -i
.youtube.com/videoplayback .youtube.com/videoplay
.youtube.com/get_video? acl videocache_allow_url url_regex -i
.youtube.[a-z][a-z]/videoplayback .youtube.[a-z][a-z]/videoplay
.youtube.[a-z][a-z]/get_video? acl videocache_allow_url url_regex
-i .googlevideo.com/videoplayback .googlevideo.com/videoplay .googlevideo.com/get_video? acl videocache_allow_url url_regex -i
.google.com/videoplayback .google.com/videoplay
.google.com/get_video? acl videocache_allow_url url_regex -i
.google.[a-z][a-z]/videoplayback .google.[a-z][a-z]/videoplay
.google.[a-z][a-z]/get_video? acl videocache_allow_url url_regex
-i proxy[a-z0-9-][a-z0-9][a-z0-9][a-z0-9]?.dailymotion.com/ acl videocache_allow_url url_regex -i vid.akm.dailymotion.com/ acl
videocache_allow_url url_regex -i
[a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?.xtube.com/(.*)flv acl
videocache_allow_url url_regex -i .vimeo.com/(.*).(flv|mp4) acl
videocache_allow_url url_regex -i
va.wrzuta.pl/wa[0-9][0-9][0-9][0-9]? acl videocache_allow_url
url_regex -i .youporn.com/(.*).flv acl videocache_allow_url
url_regex -i .msn.com.edgesuite.net/(.*).flv acl
videocache_allow_url url_regex -i .tube8.com/(.*).(flv|3gp) acl
videocache_allow_url url_regex -i .mais.uol.com.br/(.*).flv acl
videocache_allow_url url_regex -i
.blip.tv/(.*).(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v) acl
videocache_allow_url url_regex -i
.apniisp.com/(.*).(flv|avi|mov|mp3|m4v|mp4|wmv|rm|ram|m4v) acl
videocache_allow_url url_regex -i .break.com/(.*).(flv|mp4) acl
videocache_allow_url url_regex -i redtube.com/(.*).flv acl
videocache_allow_dom dstdomain .mccont.com .metacafe.com
.cdn.dailymotion.com acl videocache_deny_dom  dstdomain
.download.youporn.com .static.blip.tv acl dontrewrite url_regex
redbot.org .php acl getmethod method GET

storeurl_access deny dontrewrite storeurl_access deny !getmethod
storeurl_access deny videocache_deny_dom storeurl_access allow
videocache_allow_url storeurl_access allow videocache_allow_dom
storeurl_access deny all

storeurl_rewrite_program /etc/squid/storeurl.pl
storeurl_rewrite_children 7 storeurl_rewrite_concurrency 10

acl store_rewrite_list urlpath_regex -i
/(get_video?|videodownload?|videoplayback.*id) acl
store_rewrite_list urlpath_regex -i .flv$ .mp3$ .mp4$ .swf$ 
storeurl_access allow store_rewrite_list storeurl_access deny all

refresh_pattern -i .flv$ 10080 80% 10080  override-expire
override-lastmod reload-into-ims ignore-reload ignore-no-cache
ignore-private ignore-auth refresh_pattern -i .mp3$ 10080 80% 10080
override-expire override-lastmod reload-into-ims ignore-reload
ignore-no-cache  ignore-private ignore-auth refresh_pattern -i .mp4$
10080 80% 10080  override-expire override-lastmod reload-into-ims
ignore-reload ignore-no-cache  ignore-private ignore-auth
refresh_pattern -i .swf$ 10080 80% 10080  override-expire
override-lastmod reload-into-ims ignore-reload ignore-no-cache
ignore-private ignore-auth refresh_pattern -i .gif$ 10080 80% 10080
override-expire override-lastmod reload-into-ims ignore-reload
ignore-no-cache  ignore-private ignore-auth refresh_pattern -i .jpg$
10080 80% 10080  override-expire override-lastmod reload-into-ims
ignore-reload ignore-no-cache  ignore-private ignore-auth
refresh_pattern -i .jpeg$ 10080 80% 10080  override-expire
override-lastmod reload-into-ims ignore-reload ignore-no-cache
ignore-private  ignore-auth refresh_pattern -i .exe$ 10080 80% 10080
override-expire override-lastmod reload-into-ims ignore-reload
ignore-no-cache  ignore-private  ignore-auth

# 1 year = 525600 mins, 1 month = 10080 mins, 1 day = 1440 refresh_pattern (get_video?|videoplayback?|videodownload?|.flv?)
10080 80% 10080 ignore-no-cache  ignore-private override-expire
override-lastmod reload-into-ims refresh_pattern
(get_video?|videoplayback?id|videoplayback.*id|videodownload?|.flv?)
10080 80% 10080 ignore-no-cache  ignore-private override-expire
override-lastmod reload-into-ims refresh_pattern .(ico|video-stats)
10080 80% 10080 override-expire ignore-reload ignore-no-cache
ignore-private ignore-auth override-lastmod  negative-ttl=10080
refresh_pattern .etology?                                     10080
80% 10080 override-expire ignore-reload ignore-no-cache
refresh_pattern galleries.video(?|sz)                         10080
80% 10080 override-expire ignore-reload ignore-no-cache
refresh_pattern brazzers?                                      10080
80% 10080 override-expire ignore-reload ignore-no-cache
refresh_pattern .adtology?                                    10080
80% 10080 override-expire ignore-reload ignore-no-cache
refresh_pattern
^.*(utm.gif|ads?|rmxads.com|ad.z5x.net|bh.contextweb.com|bstats.adbrite.com|a1.interclick.com|ad.trafficmp.com|ads.cubics.com|ad.xtendmedia.com|.googlesyndication.com|advertising.com|yieldmanager|game-advertising.com|pixel.quantserve.com|adperium.com|doubleclick.net|adserving.cpxinteractive.com|syndication.com|media.fastclick.net).*
10080 20% 10080 ignore-no-cache  ignore-private override-expire
ignore-reload ignore-auth   negative-ttl=40320 max-stale=10
refresh_pattern ^.*safebrowsing.*google  10080 80% 10080
override-expire ignore-reload ignore-no-cache ignore-private
ignore-auth  negative-ttl=10080 refresh_pattern
^http://((cbk|mt|khm|mlt)[0-9]?).google.co(m|.uk)    10080 80%
10080 override-expire ignore-reload ignore-private  negative-ttl=10080
refresh_pattern ytimg.com.*.jpg
10080 80% 10080 override-expire ignore-reload refresh_pattern
images.friendster.com.*.(png|gif)                    10080 80%
10080 override-expire ignore-reload refresh_pattern garena.com
10080 80% 10080 override-expire reload-into-ims refresh_pattern
photobucket.*.(jp(e?g|e|2)|tiff?|bmp|gif|png)          10080 80%
10080 override-expire ignore-reload refresh_pattern
vid.akm.dailymotion.com.*.on2?                     10080 80%
10080 ignore-no-cache override-expire override-lastmod refresh_pattern
mediafire.com/images.*.(jp(e?g|e|2)|tiff?|bmp|gif|png)    10080 80%
10080 reload-into-ims override-expire ignore-private refresh_pattern
^http://images|pics|thumbs[0-9].                     10080 80%
10080 reload-into-ims ignore-no-cache  ignore-reload override-expire
refresh_pattern ^http://www.onemanga.com.*/
10080 80% 10080 reload-into-ims ignore-no-cache  ignore-reload
override-expire refresh_pattern
^http://v.okezone.com/get_video/([a-zA-Z0-9]) 10080 80% 10080
override-expire ignore-reload ignore-no-cache  ignore-private
ignore-auth override-lastmod  negative-ttl=10080

#images facebook refresh_pattern -i .facebook.com.*.(jpg|png|gif)                      10080 80% 10080 ignore-reload override-expire ignore-no-cache
refresh_pattern -i .fbcdn.net.*.(jpg|gif|png|swf|mp3)
10080 80% 10080 ignore-reload override-expire ignore-no-cache
refresh_pattern  static.ak.fbcdn.net*.(jpg|gif|png)
10080 80% 10080 ignore-reload override-expire ignore-no-cache
refresh_pattern ^http://profile.ak.fbcdn.net*.(jpg|gif|png)
10080 80% 10080 ignore-reload override-expire ignore-no-cache

#All File refresh_pattern -i .(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt)
10080 80% 10080 ignore-no-cache   override-expire override-lastmod
reload-into-ims refresh_pattern -i
.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar)
10080 80% 10080 ignore-no-cache   override-expire override-lastmod
reload-into-ims refresh_pattern -i
.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|dat|ad|txt|dll)
10080 80% 10080 ignore-no-cache   override-expire override-lastmod
reload-into-ims refresh_pattern -i
.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob)
10080 80% 10080 ignore-no-cache   override-expire override-lastmod
reload-into-ims refresh_pattern -i
.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv)
10080 80% 10080 ignore-no-cache   override-expire override-lastmod
reload-into-ims

refresh_pattern -i (/cgi-bin/|?)  0  0%  0 refresh_pattern ^gopher:
1440    0%      1440 refresh_pattern ^ftp:           10080   95% 10080
override-lastmod reload-into-ims refresh_pattern .               1440
95% 10080 override-lastmod reload-into-ims

One Answer

The solution is probably to specify the maximum_object_size to something bigger than the default. You can also have a look at my configuration (for Squid 3) which should also do the job.

Answered by ypid on December 27, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP