Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

adblock: 0.60.1

Browse source 
- fix possible race condition during startup
- fix duplicate logging during startup
- fix wget parms to prevent partitial downloads
- fix iptables rules to meet openwrt user chains
- added a rule in output chain to reject local ad related requests as
well
- changed default IPv4/IPv6 blackhole ip address to fix routing issues
with windows clients

Signed-off-by: Dirk Brenken <openwrt@brenken.org>
This commit is contained in:
Dirk Brenken 2016-02-03 19:19:29 +01:00
parent 75b0ded581
commit 1b8358b1ab
5 changed files with 180 additions and 124 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
net/adblock/Makefile
View file

@ -7,7 +7,7 @@
include $(TOPDIR)/rules.mk include $(TOPDIR)/rules.mk
PKG_NAME:=adblock PKG_NAME:=adblock
PKG_VERSION:=0.60.0 PKG_VERSION:=0.60.1
PKG_RELEASE:=1 PKG_RELEASE:=1
PKG_LICENSE:=GPL-3.0+ PKG_LICENSE:=GPL-3.0+
PKG_MAINTAINER:=Dirk Brenken <openwrt@brenken.org> PKG_MAINTAINER:=Dirk Brenken <openwrt@brenken.org>

12
net/adblock/files/README.md
View file

@ -42,7 +42,7 @@ When the dns server on your router receives dns requests, you will sort out quer
* overall duplicate removal in separate adblock lists (will be automatically disabled on low memory systems) * overall duplicate removal in separate adblock lists (will be automatically disabled on low memory systems)
* adblock source list parsing by fast & flexible regex rulesets * adblock source list parsing by fast & flexible regex rulesets
* additional white- and blacklist support for manual overrides * additional white- and blacklist support for manual overrides
* quality checks during/after update of adblock lists to ensure a reliable dnsmasq service * quality checks during & after update of adblock lists to ensure a reliable dnsmasq service
* wan update check, to wait for an active wan uplink before update * wan update check, to wait for an active wan uplink before update
* basic adblock statistics via iptables packet counters * basic adblock statistics via iptables packet counters
* status & error logging to stdout and syslog * status & error logging to stdout and syslog
@ -53,9 +53,9 @@ When the dns server on your router receives dns requests, you will sort out quer
## Prerequisites ## Prerequisites
* [openwrt](https://openwrt.org), tested with latest stable release (Chaos Calmer 15.05) and with current trunk (Designated Driver > r47025) * [openwrt](https://openwrt.org), tested with latest stable release (Chaos Calmer 15.05) and with current trunk (Designated Driver > r47025)
* required software packages: * usual openwrt setup with 'iptables' & 'uhttpd', additional required software packages:
* wget * wget
* optional for IPv6 support: kmod-ipt-nat6 * optional: 'kmod-ipt-nat6' for IPv6 support
* the above dependencies and requirements will be checked during package installation & script runtime, please check console output or *logread -e "adblock"* for errors * the above dependencies and requirements will be checked during package installation & script runtime, please check console output or *logread -e "adblock"* for errors
## Usage ## Usage
@ -70,7 +70,7 @@ When the dns server on your router receives dns requests, you will sort out quer
* => see [openwrt wiki](https://wiki.openwrt.org/doc/uci/fstab) for further details * => see [openwrt wiki](https://wiki.openwrt.org/doc/uci/fstab) for further details
* add static, personal domain white- or blacklist entries, one domain per line (wildcards & regex are not allowed!), by default both lists are located in */etc/adblock* * add static, personal domain white- or blacklist entries, one domain per line (wildcards & regex are not allowed!), by default both lists are located in */etc/adblock*
* enable the backup/restore feature, to restore automatically the latest, stable backup of your adblock lists in case of any processing error * enable the backup/restore feature, to restore automatically the latest, stable backup of your adblock lists in case of any processing error
* enable the debug log feature for continuous logfile writing to monitor the adblock runs over a longer period * enable the logging feature for continuous logfile writing to monitor the adblock runs over a longer period
## Distributed samples ## Distributed samples
* all sample configuration files stored in */etc/adblock/samples* * all sample configuration files stored in */etc/adblock/samples*
@ -82,8 +82,8 @@ When the dns server on your router receives dns requests, you will sort out quer
## Background ## Background
This adblock package is a dns/dnsmasq based adblock solution for openwrt. This adblock package is a dns/dnsmasq based adblock solution for openwrt.
Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6. Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6.
For that purpose adblock uses an ip address from the old reserved [class 'E'](https://tools.ietf.org/html/rfc1700) subnet (254.0.0.1 / ::ffff:fe00:0001) by default. For that purpose adblock uses an ip address from the private 'TEST-NET-1' subnet (192.0.2.1 / ::ffff:c000:0201) by default.
Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to internal adblock pixel server (in PREROUTING chain) or rejected (in FORWARD chain). Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to internal adblock pixel server (in PREROUTING chain) or rejected (in FORWARD and OUTPUT chain).
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files. All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other openwrt related config files.
## Removal ## Removal

239
net/adblock/files/adblock-helper.sh
View file

@ -47,19 +47,14 @@ f_envload()
# #
f_envparse() f_envparse()
{ {
# set the C locale, characters are single bytes, the charset is ASCII
# speeds up sort, grep etc.
#
LC_ALL=C
# set initial defaults, # set initial defaults,
# may be overwritten by setting appropriate adblock config options in global section of /etc/config/adblock # may be overwritten by setting appropriate adblock config options in global section of /etc/config/adblock
# #
adb_wanif="wan" adb_wanif="wan"
adb_lanif="lan" adb_lanif="lan"
adb_port="65535" adb_port="65535"
adb_nullipv4="254.0.0.1" adb_nullipv4="192.0.2.1"
adb_nullipv6="::ffff:fe00:0001" adb_nullipv6="::ffff:c000:0201"
adb_maxtime="60" adb_maxtime="60"
adb_maxloop="20" adb_maxloop="20"
adb_blacklist="/etc/adblock/adblock.blacklist" adb_blacklist="/etc/adblock/adblock.blacklist"
@ -136,14 +131,21 @@ f_envparse()
# set more script defaults (can't be overwritten by adblock config options) # set more script defaults (can't be overwritten by adblock config options)
# #
adb_minspace="20000" adb_cnt=0
adb_unique="1" adb_minspace=20000
adb_unique=1
adb_tmpfile="$(mktemp -tu 2>/dev/null)" adb_tmpfile="$(mktemp -tu 2>/dev/null)"
adb_tmpdir="$(mktemp -p /tmp -d 2>/dev/null)" adb_tmpdir="$(mktemp -p /tmp -d 2>/dev/null)"
adb_dnsdir="/tmp/dnsmasq.d" adb_dnsdir="/tmp/dnsmasq.d"
adb_dnsprefix="adb_list" adb_dnsprefix="adb_list"
unset adb_srcfind adb_prechain_ipv4="prerouting_rule"
unset adb_revsrcfind adb_fwdchain_ipv4="forwarding_rule"
adb_outchain_ipv4="output_rule"
adb_prechain_ipv6="PREROUTING"
adb_fwdchain_ipv6="forwarding_rule"
adb_outchain_ipv6="output_rule"
unset adb_srclist
unset adb_revsrclist
# set adblock source ruleset definitions # set adblock source ruleset definitions
# #
@ -164,21 +166,34 @@ f_envparse()
rset_yoyo="${rset_start} | sed 's/,/\n/g' | ${rset_end}" rset_yoyo="${rset_start} | sed 's/,/\n/g' | ${rset_end}"
rset_zeus="${rset_start} | ${rset_end}" rset_zeus="${rset_start} | ${rset_end}"
# get logical wan update interfaces # get logical wan update interfaces (with default route) and their device names
# #
network_find_wan adb_wanif4 2>/dev/null while [ $((adb_cnt)) -le $((adb_maxloop)) ]
network_find_wan6 adb_wanif6 2>/dev/null do
if [ "${adb_wanif4}" = "${adb_lanif}" ] || [ "${adb_wanif6}" = "${adb_lanif}" ] network_find_wan adb_wanif4 2>/dev/null
then network_find_wan6 adb_wanif6 2>/dev/null
rc=125 if [ -z "${adb_wanif4}" ] && [ -z "${adb_wanif6}" ]
f_log "LAN only (${adb_lanif}) network, no valid IPv4/IPv6 wan update interface found" "${rc}" then
f_exit network_flush_cache
elif [ -z "${adb_wanif4}" ] && [ -z "${adb_wanif6}" ] elif [ "${adb_wanif4}" = "${adb_lanif}" ] || [ "${adb_wanif6}" = "${adb_lanif}" ]
then then
rc=125 rc=125
f_log "no valid IPv4/IPv6 wan update interface found" "${rc}" f_log "LAN only (${adb_lanif}) network, no valid IPv4/IPv6 wan update interface found" "${rc}"
f_exit f_exit
fi else
network_get_device adb_wandev4 "${adb_wanif4}" 2>/dev/null
network_get_device adb_wandev6 "${adb_wanif6}" 2>/dev/null
break
fi
if [ $((adb_cnt)) -ge $((adb_maxloop)) ]
then
rc=125
f_log "no valid IPv4/IPv6 wan update interface found" "${rc}"
f_exit
fi
adb_cnt=$((adb_cnt + 1))
sleep 1
done
# get lan ip addresses # get lan ip addresses
# #
@ -203,6 +218,7 @@ f_envcheck()
{ {
# check general package dependencies # check general package dependencies
# #
f_depend "uhttpd"
f_depend "wget" f_depend "wget"
f_depend "iptables" f_depend "iptables"
f_depend "kmod-ipt-nat" f_depend "kmod-ipt-nat"
@ -231,9 +247,9 @@ f_envcheck()
check="$(printf "${pkg_list}" | grep "^ca-certificates -" 2>/dev/null)" check="$(printf "${pkg_list}" | grep "^ca-certificates -" 2>/dev/null)"
if [ -z "${check}" ] if [ -z "${check}" ]
then then
wget_parm="--no-config --no-check-certificate --quiet" wget_parm="--no-config --no-check-certificate --quiet --tries=1 --no-cache --no-cookies --max-redirect=0"
else else
wget_parm="--no-config --quiet" wget_parm="--no-config --quiet --tries=1 --no-cache --no-cookies --max-redirect=0"
fi fi
# check adblock blacklist/whitelist configuration # check adblock blacklist/whitelist configuration
@ -254,10 +270,11 @@ f_envcheck()
# #
if [ -n "${adb_tmpdir}" ] && [ -d "${adb_tmpdir}" ] if [ -n "${adb_tmpdir}" ] && [ -d "${adb_tmpdir}" ]
then then
f_space "${adb_tmpdir}" "please supersize your /tmp directory" f_space "${adb_tmpdir}"
if [ "${space_ok}" = "false" ] if [ "${space_ok}" = "false" ]
then then
rc=140 rc=140
f_log "not enough space in '${adb_tmpdir}', please supersize your temp directory" "${rc}"
f_exit f_exit
fi fi
else else
@ -274,19 +291,21 @@ f_envcheck()
if [ $((mem_total)) -le 64000 ] && [ $((swap_total)) -eq 0 ] if [ $((mem_total)) -le 64000 ] && [ $((swap_total)) -eq 0 ]
then then
adb_unique=0 adb_unique=0
f_log "overall sort/unique processing will be disabled," f_log "not enough memory, overall sort/unique processing will be disabled"
f_log "please consider adding an external swap device to supersize your /tmp directory (total: ${mem_total}, free: ${mem_free}, swap: ${mem_swap})" f_log "please consider adding an external swap device to supersize your temp directory (total: ${mem_total}, free: ${mem_free}, swap: ${mem_swap})"
fi fi
# check backup configuration # check backup configuration
# #
if [ -n "${adb_backupdir}" ] && [ -d "${adb_backupdir}" ] if [ -n "${adb_backupdir}" ] && [ -d "${adb_backupdir}" ]
then then
f_space "${adb_backupdir}" "backup/restore will be disabled" f_space "${adb_backupdir}"
if [ "${space_ok}" = "false" ] if [ "${space_ok}" = "false" ]
then then
f_log "backup/restore will be disabled"
backup_ok="false" backup_ok="false"
else else
f_log "backup/restore will be enabled"
backup_ok="true" backup_ok="true"
fi fi
else else
@ -294,41 +313,45 @@ f_envcheck()
f_log "backup/restore will be disabled" f_log "backup/restore will be disabled"
fi fi
# check debug log configuration # check log configuration
# #
adb_logdir="${adb_logfile%/*}" adb_logdir="${adb_logfile%/*}"
if [ -n "${adb_logdir}" ] && [ -d "${adb_logdir}" ] if [ -n "${adb_logdir}" ] && [ -d "${adb_logdir}" ]
then then
f_space "${adb_logdir}" "debug logging will be disabled" f_space "${adb_logdir}"
if [ "${space_ok}" = "false" ] if [ "${space_ok}" = "false" ]
then then
f_log "not enough space in '${adb_logdir}', logging will be disabled"
log_ok="false" log_ok="false"
else else
f_log "logging will be enabled"
log_ok="true" log_ok="true"
fi fi
else else
log_ok="false" log_ok="false"
f_log "debug logging will be disabled" f_log "logging will be disabled"
fi fi
# check ipv4/iptables configuration # check ipv4/iptables configuration
# #
if [ -n "${adb_wanif4}" ] if [ -n "${adb_wanif4}" ]
then then
f_firewall "IPv4" "nat" "I" "PREROUTING" "adb-nat: tcp, port 80, DNAT" "-p tcp -d ${adb_nullipv4} --dport 80 -j DNAT --to-destination ${adb_ipv4}:${adb_port}" f_firewall "IPv4" "nat" "A" "${adb_prechain_ipv4}" "adblock-pre" "-p tcp -d ${adb_nullipv4} --dport 80 -j REDIRECT --to-ports ${adb_port}"
f_firewall "IPv4" "nat" "A" "PREROUTING" "adb-dns: udp, port 53, DNAT" "-p udp --dport 53 -j DNAT --to-destination ${adb_ipv4}" f_firewall "IPv4" "nat" "A" "${adb_prechain_ipv4}" "adblock-dns" "-p udp --dport 53 -j REDIRECT"
f_firewall "IPv4" "nat" "A" "PREROUTING" "adb-dns: tcp, port 53, DNAT" "-p tcp --dport 53 -j DNAT --to-destination ${adb_ipv4}" f_firewall "IPv4" "nat" "A" "${adb_prechain_ipv4}" "adblock-dns" "-p tcp --dport 53 -j REDIRECT"
f_firewall "IPv4" "filter" "I" "FORWARD" "adb-rej: all protocols, all ports, REJECT" "-d ${adb_nullipv4} -j REJECT" f_firewall "IPv4" "filter" "A" "${adb_fwdchain_ipv4}" "adblock-fwd" "-d ${adb_nullipv4} -j REJECT"
f_firewall "IPv4" "filter" "A" "${adb_outchain_ipv4}" "adblock-out" "-d ${adb_nullipv4} -j REJECT"
fi fi
# check ipv6/ip6tables configuration # check ipv6/ip6tables configuration
# #
if [ -n "${adb_wanif6}" ] if [ -n "${adb_wanif6}" ]
then then
f_firewall "IPv6" "nat" "I" "PREROUTING" "adb-nat: tcp, port 80, DNAT" "-p tcp -d ${adb_nullipv6} --dport 80 -j DNAT --to-destination [${adb_ipv6}]:${adb_port}" f_firewall "IPv6" "nat" "A" "${adb_prechain_ipv6}" "adblock-pre" "-p tcp -d ${adb_nullipv6} --dport 80 -j REDIRECT --to-ports ${adb_port}"
f_firewall "IPv6" "nat" "A" "PREROUTING" "adb-dns: udp, port 53, DNAT" "-p udp --dport 53 -j DNAT --to-destination ${adb_ipv6}" f_firewall "IPv6" "nat" "A" "${adb_prechain_ipv6}" "adblock-dns" "-p udp --dport 53 -j REDIRECT"
f_firewall "IPv6" "nat" "A" "PREROUTING" "adb-dns: tcp, port 53, DNAT" "-p tcp --dport 53 -j DNAT --to-destination ${adb_ipv6}" f_firewall "IPv6" "nat" "A" "${adb_prechain_ipv6}" "adblock-dns" "-p tcp --dport 53 -j REDIRECT"
f_firewall "IPv6" "filter" "I" "FORWARD" "adb-rej: all protocols, all ports, REJECT" "-d ${adb_nullipv6} -j REJECT" f_firewall "IPv6" "filter" "A" "${adb_fwdchain_ipv6}" "adblock-fwd" "-d ${adb_nullipv6} -j REJECT"
f_firewall "IPv6" "filter" "A" "${adb_outchain_ipv6}" "adblock-out" "-d ${adb_nullipv6} -j REJECT"
fi fi
# check volatile adblock uhttpd instance configuration # check volatile adblock uhttpd instance configuration
@ -374,19 +397,18 @@ f_envcheck()
# wait for active wan update interface # wait for active wan update interface
# #
cnt=0 while [ $((adb_cnt)) -le $((adb_maxloop)) ]
while [ $((cnt)) -le $((adb_maxloop)) ]
do do
for interface in ${adb_wanif} for interface in ${adb_wanif}
do do
network_get_device adb_wandev "${interface}" 2>/dev/null network_get_device adb_wandev "${interface}" 2>/dev/null
if [ -z "${adb_wandev}" ] || [ ! -d "/sys/class/net/${adb_wandev}" ] if [ -z "${adb_wandev}" ] || [ ! -d "/sys/class/net/${adb_wandev}" ]
then then
if [ -n "${adb_wanif4}" ] if [ -n "${adb_wandev4}" ]
then then
network_get_device adb_wandev "${adb_wanif4}" 2>/dev/null adb_wandev="${adb_wandev4}"
else else
network_get_device adb_wandev "${adb_wanif6}" 2>/dev/null adb_wandev="${adb_wandev6}"
fi fi
if [ -z "${adb_wandev}" ] || [ ! -d "/sys/class/net/${adb_wandev}" ] if [ -z "${adb_wandev}" ] || [ ! -d "/sys/class/net/${adb_wandev}" ]
then then
@ -398,15 +420,15 @@ f_envcheck()
check="$(cat /sys/class/net/${adb_wandev}/operstate 2>/dev/null)" check="$(cat /sys/class/net/${adb_wandev}/operstate 2>/dev/null)"
if [ "${check}" = "up" ] if [ "${check}" = "up" ]
then then
f_log "get active wan update interface/device (${adb_wanif}/${adb_wandev}) after ${cnt} loops" f_log "get active wan update interface/device (${adb_wanif}/${adb_wandev})"
break 2 break 2
elif [ $((cnt)) -eq $((adb_maxloop)) ] elif [ $((adb_cnt)) -eq $((adb_maxloop)) ]
then then
rc=145 rc=145
f_log "wan update interface/device not running (${adb_wanif}/${adb_wandev}) after ${cnt} loops" "${rc}" f_log "wan update interface/device not running (${adb_wanif}/${adb_wandev})" "${rc}"
f_restore f_restore
fi fi
cnt=$((cnt + 1)) adb_cnt=$((adb_cnt + 1))
sleep 1 sleep 1
done done
done done
@ -415,7 +437,6 @@ f_envcheck()
# #
if [ -n "${adb_ntpsrv}" ] if [ -n "${adb_ntpsrv}" ]
then then
cnt=0
unset ntp_pool unset ntp_pool
for srv in ${adb_ntpsrv} for srv in ${adb_ntpsrv}
do do
@ -425,11 +446,9 @@ f_envcheck()
rc=${?} rc=${?}
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
ntp_ok="true"
f_log "get ntp time sync" f_log "get ntp time sync"
else else
rc=0 rc=0
ntp_ok="false"
f_log "ntp time sync failed" f_log "ntp time sync failed"
fi fi
fi fi
@ -457,14 +476,13 @@ f_envcheck()
# #
f_depend() f_depend()
{ {
local rc_func
local package="${1}" local package="${1}"
check="$(printf "${pkg_list}" | grep "^${package} -" 2>/dev/null)" check="$(printf "${pkg_list}" | grep "^${package} -" 2>/dev/null)"
if [ -z "${check}" ] if [ -z "${check}" ]
then then
rc_func=150 rc=150
f_log "package '${package}' not found" "${rc_func}" f_log "package '${package}' not found" "${rc}"
f_exit f_exit
fi fi
} }
@ -474,7 +492,6 @@ f_depend()
# #
f_firewall() f_firewall()
{ {
local rc_func
local ipt local ipt
local iptv4="/usr/sbin/iptables" local iptv4="/usr/sbin/iptables"
local iptv6="/usr/sbin/ip6tables" local iptv6="/usr/sbin/ip6tables"
@ -496,16 +513,16 @@ f_firewall()
# check whether iptables rule already applied and proceed accordingly # check whether iptables rule already applied and proceed accordingly
# #
rc_func="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1; printf ${?})" rc="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1; printf ${?})"
if [ $((rc_func)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
"${ipt}" -w -t "${table}" -"${ctype}" "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1 "${ipt}" -w -t "${table}" -"${ctype}" "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1
rc_func=${?} rc=${?}
if [ $((rc_func)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
f_log "created volatile ${proto} firewall rule in '${chain}' chain (${notes})" f_log "created volatile ${proto} firewall rule in '${chain}' chain (${notes})"
else else
f_log "failed to initialize volatile ${proto} firewall rule in '${chain}' chain (${notes})" "${rc_func}" f_log "failed to initialize volatile ${proto} firewall rule in '${chain}' chain (${notes})" "${rc}"
f_restore f_restore
fi fi
fi fi
@ -530,8 +547,12 @@ f_log()
log_rc=", rc: ${log_rc}" log_rc=", rc: ${log_rc}"
log_msg="${log_msg}${log_rc}" log_msg="${log_msg}${log_rc}"
fi fi
/usr/bin/logger -s -t "adblock[${pid}] ${class}" "${log_msg}" /usr/bin/logger -t "adblock[${pid}] ${class}" "${log_msg}"
if [ "${log_ok}" = "true" ] && [ "${ntp_ok}" = "true" ] if [ -t 1 ]
then
printf "%s\n" "adblock[${pid}] ${class}: ${log_msg}"
fi
if [ "${log_ok}" = "true" ]
then then
printf "%s\n" "$(/bin/date "+%d.%m.%Y %H:%M:%S") adblock[${pid}] ${class}: ${log_msg}" >> "${adb_logfile}" printf "%s\n" "$(/bin/date "+%d.%m.%Y %H:%M:%S") adblock[${pid}] ${class}: ${log_msg}" >> "${adb_logfile}"
fi fi
@ -543,9 +564,7 @@ f_log()
# #
f_space() f_space()
{ {
local rc_func
local mp="${1}" local mp="${1}"
local notes="${2}"
# check relevant mount points in a subshell # check relevant mount points in a subshell
# #
@ -558,22 +577,23 @@ f_space()
av_space="${available}" av_space="${available}"
if [ $((av_space)) -eq 0 ] if [ $((av_space)) -eq 0 ]
then then
rc_func=155 rc=155
f_log "no space left on device/not mounted (${mp}), ${notes}" f_log "no space left on device/not mounted (${mp})"
exit ${rc_func} exit ${rc}
elif [ $((av_space)) -lt $((adb_minspace)) ] elif [ $((av_space)) -lt $((adb_minspace)) ]
then then
rc_func=155 rc=155
f_log "not enough space left on device (${mp}), ${notes}" f_log "not enough space left on device (${mp})"
exit ${rc_func} exit ${rc}
fi fi
done done
# subshell return code handling, set space trigger accordingly # subshell return code handling, set space trigger accordingly
# #
rc_func=${?} rc=${?}
if [ $((rc_func)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
rc=0
space_ok="false" space_ok="false"
fi fi
fi fi
@ -584,7 +604,6 @@ f_space()
# #
f_restore() f_restore()
{ {
local rc_func
local removal_done local removal_done
local restore_done local restore_done
@ -592,11 +611,11 @@ f_restore()
# #
if [ -n "${adb_revsrclist}" ] if [ -n "${adb_revsrclist}" ]
then then
find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrcfind} \) -exec rm -f "{}" \; 2>/dev/null find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -exec rm -f "{}" \; 2>/dev/null
rc_func=${?} rc=${?}
if [ $((rc_func)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
f_log "error during removal of bogus adblock lists" "${rc_func}" f_log "error during removal of bogus adblock lists" "${rc}"
f_exit f_exit
else else
removal_done="true" removal_done="true"
@ -612,10 +631,10 @@ f_restore()
do do
filename="${file##*/}" filename="${file##*/}"
cp -pf "${file}" "${adb_dnsdir}" 2>/dev/null cp -pf "${file}" "${adb_dnsdir}" 2>/dev/null
rc_func=${?} rc=${?}
if [ $((rc_func)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
f_log "error during restore of adblock list (${filename})" "${rc_func}" f_log "error during restore of adblock list (${filename})" "${rc}"
f_exit f_exit
fi fi
restore_done="true" restore_done="true"
@ -635,10 +654,16 @@ f_restore()
if [ -n "${dns_status}" ] if [ -n "${dns_status}" ]
then then
rc=0 rc=0
if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ]
then
adb_count="$(($(head -qn -4 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | wc -l) / 2))"
else
adb_count="$(head -qn -4 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | wc -l)"
fi
f_log "adblock lists with overall ${adb_count} domains loaded"
else else
rc=160 rc=160
f_log "dnsmasq restart failed, please check 'logread' output" "${rc}" f_log "dnsmasq restart failed, please check 'logread' output" "${rc}"
f_restore
fi fi
fi fi
f_exit f_exit
@ -650,10 +675,14 @@ f_restore()
# #
f_exit() f_exit()
{ {
local ipv4_nat local ipv4_prerouting
local ipv4_rej local ipv4_forward
local ipv6_nat local ipv4_output
local ipv6_rej local ipv6_prerouting
local ipv6_forward
local ipv6_output
local iptv4="/usr/sbin/iptables"
local iptv6="/usr/sbin/ip6tables"
# delete temporary files & directories # delete temporary files & directories
# #
@ -670,23 +699,35 @@ f_exit()
# #
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
if [ -n "${adb_wanif4}" ]
then
ipv4_prerouting="$(${iptv4} -t nat -vnL | grep -F "adblock-pre" | grep -Eo "[0-9]+" | head -n1)"
ipv4_forward="$(${iptv4} -vnL | grep -F "adblock-fwd" | grep -Eo "[0-9]+" | head -n1)"
ipv4_output="$(${iptv4} -vnL | grep -F "adblock-out" | grep -Eo "[0-9]+" | head -n1)"
fi
if [ -n "${adb_wanif6}" ]
then
ipv6_prerouting="$(${iptv6} -t nat -vnL | grep -F "adblock-pre" | grep -Eo "[0-9]+" | head -n1)"
ipv6_forward="$(${iptv6} -vnL | grep -F "adblock-fwd" | grep -Eo "[0-9]+" | head -n1)"
ipv6_output="$(${iptv6} -vnL | grep -F "adblock-out" | grep -Eo "[0-9]+" | head -n1)"
fi
if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ] if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ]
then then
ipv4_nat="$(iptables -t nat -vnL | grep -F "adb-nat" | grep -Eo "[0-9]+" | head -n1)" f_log "adblock firewall statistics (IPv4/IPv6):"
ipv4_rej="$(iptables -vnL | grep -F "adb-rej" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv4_prerouting}/${ipv6_prerouting} packets redirected in PREROUTING chain"
ipv6_nat="$(ip6tables -t nat -vnL | grep -F "adb-nat" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv4_forward}/${ipv6_forward} packets rejected in FORWARD chain"
ipv6_rej="$(ip6tables -vnL | grep -F "adb-rej" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv4_output}/${ipv6_output} packets rejected in OUTPUT chain"
f_log "adblock firewall statistics (IPv4/IPv6): ${ipv4_nat}/${ipv6_nat} packets redirected in PREROUTING chain, ${ipv4_rej}/${ipv6_rej} packets rejected in FORWARD chain"
elif [ -n "${adb_wanif4}" ] elif [ -n "${adb_wanif4}" ]
then then
ipv4_nat="$(iptables -t nat -vnL | grep -F "adb-nat" | grep -Eo "[0-9]+" | head -n1)" f_log "adblock firewall statistics (IPv4):"
ipv4_rej="$(iptables -vnL | grep -F "adb-rej" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv4_prerouting} packets redirected in PREROUTING chain"
f_log "adblock firewall statistics (IPv4): ${ipv4_nat} packets redirected in PREROUTING chain, ${ipv4_rej} packets rejected in FORWARD chain" f_log "${ipv4_forward} packets rejected in FORWARD chain"
f_log "${ipv4_output} packets rejected in OUTPUT chain"
elif [ -n "${adb_wanif6}" ] elif [ -n "${adb_wanif6}" ]
then then
ipv6_nat="$(ip6tables -t nat -vnL | grep -F "adb-nat" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv6_prerouting} packets redirected in PREROUTING chain"
ipv6_rej="$(ip6tables -vnL | grep -F "adb-rej" | grep -Eo "[0-9]+" | head -n1)" f_log "${ipv6_forward} packets rejected in FORWARD chain"
f_log "adblock firewall statistics (IPv6): ${ipv6_nat} packets redirected in PREROUTING chain, ${ipv6_rej} packets rejected in FORWARD chain" f_log "${ipv6_output} packets rejected in OUTPUT chain"
fi fi
f_log "domain adblock processing finished successfully (${adb_version}, ${openwrt_version}, $(/bin/date "+%d.%m.%Y %H:%M:%S"))" f_log "domain adblock processing finished successfully (${adb_version}, ${openwrt_version}, $(/bin/date "+%d.%m.%Y %H:%M:%S"))"
else else

48
net/adblock/files/adblock-update.sh
View file

@ -23,9 +23,14 @@
# environment # # environment #
############### ###############
# set the C locale, characters are single bytes, the charset is ASCII
# speeds up things like sort, grep etc.
#
LC_ALL=C
# set script version # set script version
# #
adb_version="0.60.0" adb_version="0.60.1"
# get current pid, script directory and openwrt version # get current pid, script directory and openwrt version
# #
@ -48,9 +53,9 @@ fi
# main program # # main program #
################ ################
# call restore function on trap signals (HUP, INT, QUIT, BUS, SEGV, TERM) # call exit function on trap signals (HUP, INT, QUIT, BUS, SEGV, TERM)
# #
trap "rc=255; f_log 'trap error' '${rc}'; f_restore" 1 2 3 10 11 15 trap "rc=255; f_log 'error signal received/trapped' '${rc}'; f_exit" 1 2 3 10 11 15
# start logging # start logging
# #
@ -94,7 +99,7 @@ then
fi fi
if [ -z "${list_time}" ] || [ "${list_time}" != "${shalla_time}" ] if [ -z "${list_time}" ] || [ "${list_time}" != "${shalla_time}" ]
then then
wget ${wget_parm} --timeout="${adb_maxtime}" --tries=1 --output-document="${shalla_archive}" "${adb_arc_shalla}" 2>/dev/null wget ${wget_parm} --timeout="${adb_maxtime}" --output-document="${shalla_archive}" "${adb_arc_shalla}" 2>/dev/null
rc=${?} rc=${?}
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
@ -126,7 +131,7 @@ then
rc=0 rc=0
fi fi
else else
adb_srcfind="! -name ${adb_dnsprefix}.${src_name}" adb_srclist="! -name ${adb_dnsprefix}.${src_name}"
f_log " source archive doesn't change, no update required" f_log " source archive doesn't change, no update required"
fi fi
fi fi
@ -152,11 +157,11 @@ do
# prepare find statement with active adblock list sources # prepare find statement with active adblock list sources
# #
if [ -z "${adb_srcfind}" ] if [ -z "${adb_srclist}" ]
then then
adb_srcfind="! -name ${adb_dnsprefix}.${src_name}" adb_srclist="! -name ${adb_dnsprefix}.${src_name}"
else else
adb_srcfind="${adb_srcfind} -a ! -name ${adb_dnsprefix}.${src_name}" adb_srclist="${adb_srclist} -a ! -name ${adb_dnsprefix}.${src_name}"
fi fi
# only download adblock list with newer/updated timestamp # only download adblock list with newer/updated timestamp
@ -187,7 +192,7 @@ do
tmp_domains="$(cat "${shalla_file}" 2>/dev/null)" tmp_domains="$(cat "${shalla_file}" 2>/dev/null)"
rc=${?} rc=${?}
else else
tmp_domains="$(wget ${wget_parm} --timeout="${adb_maxtime}" --tries=1 --output-document=- "${url}" 2>/dev/null)" tmp_domains="$(wget ${wget_parm} --timeout="${adb_maxtime}" --output-document=- "${url}" 2>/dev/null)"
rc=${?} rc=${?}
fi fi
else else
@ -212,8 +217,8 @@ do
f_log " empty source download finished" f_log " empty source download finished"
continue continue
else else
f_log " source download failed"
rc=0 rc=0
f_log " source download failed"
continue continue
fi fi
@ -233,11 +238,11 @@ do
# prepare find statement with revised adblock list sources # prepare find statement with revised adblock list sources
# #
if [ -z "${adb_revsrcfind}" ] if [ -z "${adb_revsrclist}" ]
then then
adb_revsrcfind="-name ${adb_dnsprefix}.${src_name}" adb_revsrclist="-name ${adb_dnsprefix}.${src_name}"
else else
adb_revsrcfind="${adb_revsrcfind} -o -name ${adb_dnsprefix}.${src_name}" adb_revsrclist="${adb_revsrclist} -o -name ${adb_dnsprefix}.${src_name}"
fi fi
# write preliminary adblock list footer # write preliminary adblock list footer
@ -267,9 +272,9 @@ done
# remove old adblock lists and their backups # remove old adblock lists and their backups
# #
if [ -n "${adb_srcfind}" ] if [ -n "${adb_srclist}" ]
then then
adb_rmfind="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srcfind} \) -print -exec rm -f "{}" \; 2>/dev/null)" adb_rmfind="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -print -exec rm -f "{}" \; 2>/dev/null)"
if [ $((rc)) -eq 0 ] && [ -n "${adb_rmfind}" ] if [ $((rc)) -eq 0 ] && [ -n "${adb_rmfind}" ]
then then
f_log "no longer used adblock lists removed" "${rc}" f_log "no longer used adblock lists removed" "${rc}"
@ -280,7 +285,7 @@ then
fi fi
if [ "${backup_ok}" = "true" ] if [ "${backup_ok}" = "true" ]
then then
find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srcfind} \) -exec rm -f "{}" \; 2>/dev/null find "${adb_backupdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}.*" \( ${adb_srclist} \) -exec rm -f "{}" \; 2>/dev/null
if [ $((rc)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
f_log "error during removal of old backups" "${rc}" f_log "error during removal of old backups" "${rc}"
@ -302,7 +307,7 @@ fi
# #
if [ $((adb_unique)) -eq 1 ] if [ $((adb_unique)) -eq 1 ]
then then
if [ -n "${adb_revsrcfind}" ] if [ -n "${adb_revsrclist}" ]
then then
f_log "remove duplicates in separate adblock lists" f_log "remove duplicates in separate adblock lists"
@ -352,10 +357,17 @@ else
adb_count="$(head -qn -4 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | wc -l)" adb_count="$(head -qn -4 "${adb_dnsdir}/${adb_dnsprefix}."* 2>/dev/null | wc -l)"
fi fi
# restore adblock lists if overall count is null (i.e. all downloads failed)
#
if [ "${backup_ok}" = "true" ] && [ $((adb_count)) -eq 0 ]
then
f_restore
fi
# restart dnsmasq with newly generated or deleted adblock lists, # restart dnsmasq with newly generated or deleted adblock lists,
# check dnsmasq startup afterwards # check dnsmasq startup afterwards
# #
if [ -n "${adb_revsrcfind}" ] || [ -n "${adb_rmfind}" ] if [ -n "${adb_revsrclist}" ] || [ -n "${adb_rmfind}" ]
then then
/etc/init.d/dnsmasq restart >/dev/null 2>&1 /etc/init.d/dnsmasq restart >/dev/null 2>&1
sleep 2 sleep 2

3
net/adblock/files/adblock.conf
View file

@ -1,3 +1,6 @@
# adblock configuration, for further information
# please see /etc/adblock/samples/adblock.conf.sample
# and /etc/adblock/README.md
config adblock "global" config adblock "global"
option adb_blacklist "/etc/adblock/adblock.blacklist" option adb_blacklist "/etc/adblock/adblock.blacklist"