Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

adblock: release 3.0

Browse source 
* add kresd & turris omnia support
* add dnscrypt-proxy support
* change start priority to 30, to fix possible trigger issues on slow
booting hardware
* simplify suspend/resume handling (no longer use a hideout directory)
* enhanced LuCI frontend
* many small changes & improvements
* default config change (please update your config!), adblock is now
disabled by default
* documentation update

Signed-off-by: Dirk Brenken <dev@brenken.org>
This commit is contained in:
Dirk Brenken 2017-09-09 17:35:42 +02:00
parent 7cdb6c7031
commit 206c847bb5
5 changed files with 238 additions and 188 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
net/adblock/Makefile
View file

@ -6,7 +6,7 @@
include $(TOPDIR)/rules.mk
PKG_NAME:=adblock
PKG_VERSION:=2.8.5
PKG_VERSION:=3.0.0
PKG_RELEASE:=1
PKG_LICENSE:=GPL-3.0+
PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org>

104
net/adblock/files/README.md
View file

@ -29,6 +29,8 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* => daily updates, approx. 150 entries
* [reg_cn](https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt)
* => regional blocklist for China, daily updates, approx. 1.600 entries
* [reg_id](https://easylist-downloads.adblockplus.org/abpindo+easylist.txt)
* => regional blocklist for Indonesia, daily updates, approx. 800 entries
* [reg_nl](https://easylist-downloads.adblockplus.org/easylistdutch+easylist.txt)
* => regional blocklist for the Netherlands, weekly updates, approx. 1300 entries
* [reg_pl](http://adblocklist.org)
@ -57,12 +59,12 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* => daily updates, approx. 440 entries
* zero-conf like automatic installation & setup, usually no manual changes needed
* simple but yet powerful adblock engine: adblock does not use error prone external iptables rulesets, http pixel server instances and things like that
* automatically selects dnsmasq, unbound, bind or kresd (experimental!) as dns backend.
* supports five different dns backends / block list formats: dnsmasq, unbound, named (bind), kresd and dnscrypt-proxy
* automatically selects uclient-fetch or wget as download utility (other tools like curl or aria2c are supported as well)
* support http only mode (without installed ssl library) for all non-SSL blocklist sources
* automatically supports a wide range of router modes, even AP modes are supported
* provides 'http only' mode without installed ssl library for all non-SSL block list sources
* supports a wide range of router modes, even AP modes are supported
* full IPv4 and IPv6 support
* supports tld compression (top level domain compression), this feature removes thousands of needless host entries from the block list and lowers the memory footprint for the dns backends
* provides top level domain compression ('tld compression'), this feature removes thousands of needless host entries from the block list and lowers the memory footprint for the dns backends
* each block list source will be updated and processed separately
* block list source parsing by fast & flexible regex rulesets
* overall duplicate removal in central block list (adb_list.overall)
@ -70,10 +72,11 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* quality checks during block list update to ensure a reliable dns backend service
* minimal status & error logging to syslog, enable debug logging to receive more output
* procd based init system support (start/stop/restart/reload/suspend/resume/query/status)
* procd based hotplug support, the adblock start will be solely triggered by network interface triggers
* procd network interface trigger support or classic time based startup
* suspend & resume adblock actions temporarily without block list reloading
* runtime information available via LuCI & via 'status' init command
* output comprehensive runtime information via LuCI or via 'status' init command
* query function to quickly identify blocked (sub-)domains, e.g. for whitelisting
* strong LuCI support
* optional: force dns requests to local resolver
* optional: force overall sort / duplicate removal for low memory devices (handle with care!)
* optional: 'manual mode' to re-use blocklist backups during startup, get fresh lists only via manual reload or restart action
@ -85,16 +88,16 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* a usual setup with an enabled dns backend at minimum - dump AP modes without a working dns backend are _not_ supported
* a download utility:
* to support all blocklist sources a full version (with ssl support) of 'wget', 'uclient-fetch' with one of the 'libustream-*' ssl libraries, 'aria2c' or 'curl' is required
* for limited devices with real memory constraints, adblock provides also a plain http option and supports wget-nossl and uclient-fetch (without libustream-ssl), too
* for limited devices with real memory constraints, adblock provides also a 'http only' option and supports wget-nossl and uclient-fetch (without libustream-ssl) as well
* for more configuration options see examples below
## LEDE trunk Installation & Usage
* install 'adblock' (_opkg install adblock_) and that's it - the adblock start will be automatically triggered by procd interface trigger
## Installation & Usage
* install 'adblock' (_opkg install adblock_)
* at minimum configure the appropriate dns backend ('dnsmasq' by default) and enable the adblock service in _/etc/config/adblock_
* control the adblock service manually with _/etc/init.d/adblock_ start/stop/restart/reload/suspend/resume/status or use the LuCI frontend
* enable/disable your favored block list sources in _/etc/config/adblock_ - 'adaway', 'disconnect' and 'yoyo' are enabled by default
## LuCI adblock companion package
* for easy management of the various block list sources and all other adblock options you can also use a nice & efficient LuCI frontend
* for easy management of the various block list sources and all other adblock options you should use the provided LuCI frontend
* install 'luci-app-adblock' (_opkg install luci-app-adblock_)
* the application is located in LuCI under 'Services' menu
@ -105,7 +108,7 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* **add white- / blacklist entries:** add domain white- or blacklist entries to always-allow or -deny certain (sub) domains, by default both lists are empty and located in _/etc/adblock_. Please add one domain per line - ip addresses, wildcards & regex are _not_ allowed (see example below)
* **backup & restore block lists:** enable this feature, to restore automatically the latest compressed backup of your block lists in case of any processing error (e.g. a single block list source is not available during update). Please use an (external) solid partition and _not_ your volatile router temp directory for this
* **scheduled list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below)
* **restrict procd interface trigger:** restrict the procd interface trigger to a (list of) certain interface(s) (default: wan). To disable it at all, remove all entries
* **change startup behaviour:** by default the startup will be triggered by the 'wan' procd interface trigger. Choose 'none' to disable automatic startups, 'timed' to use a classic timeout (default 30 sec.) or select another trigger interface.
* **suspend & resume adblocking:** to quickly switch the adblock service 'on' or 'off', simply use _/etc/init.d/adblock [suspend|resume]_
* **domain query:** to query the active block list for a specific domain, please run _/etc/init.d/adblock query `<DOMAIN>`_ (see example below)
* **add new list sources:** you could add new block list sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
@ -113,12 +116,14 @@ A lot of people already use adblocker plugins within their desktop browsers, but
## Further adblock config options
* usually the pre-configured adblock setup works quite well and no manual config overrides are needed, all listed options apply to the 'global' config section:
* adb\_enabled => main switch to enable/disable adblock service (default: '1', enabled)
* adb\_enabled => main switch to enable/disable adblock service (default: '0', disabled)
* adb\_debug => enable/disable adblock debug output (default: '0', disabled)
* adb\_iface => set the procd interface trigger to a (list of) lan / wan interface(s) (default: 'wan')
* adb\_fetch => full path to a different download utility, see example below (default: not set, use wget)
* adb\_fetchparm => options for the download utility, see example below (default: not set, use wget options)
* adb\_triggerdelay => additional trigger delay in seconds before adblock processing starts (default: '2')
* adb\_dns => select the dns backend for your environment: 'dnsmasq', 'unbound', 'named', 'kresd' or 'dnscrypt-proxy' (default: 'dnsmasq')
* adb\_dnsdir => target directory for the generated blocklist 'adb_list.overall' (default: not set, use dns backend default)
* adb\_trigger => set the startup trigger to a certain interface, to 'timed' or to 'none' (default: 'wan')
* adb\_triggerdelay => additional trigger delay in seconds before adblock processing begins (default: '1')
* adb\_fetch => full path to a dedicated download utility, see example below (default: not set, use wget default)
* adb\_fetchparm => options for the download utility, see example below (default: not set, use wget default options)
* adb\_forcedns => force dns requests to local resolver (default: '0', disabled)
* adb\_forcesrt => force overall sort on low memory devices with less than 64 MB RAM (default: '0', disabled)
* adb\_manmode => do not automatically update block lists during startup, use backups instead (default: '0', disabled)
@ -132,7 +137,7 @@ If you use manual configuration for unbound, then just include the following lin
include: "/var/lib/unbound/adb_list.overall"
</code></pre>
**change default dns backend to 'bind':**
**change default dns backend to 'named' (bind):**
<pre><code>
Adblock deposits the sorted and filtered block list (adb_list.overall) in '/var/lib/bind' where bind can find them.
To use the block list please modify the following bind configuration files:
@ -159,9 +164,9 @@ create the new file '/etc/bind/db.rpz' and add:
**change default dns backend to 'kresd':**
<pre><code>
The knot-resolver (kresd) support is only available to turris omnia users. At this stage there's no package for kresd in the official LEDE / OpenWrt package repository.
Adblock deposits the sorted and filtered block list (adb_list.overall) in '/tmp/kresd' where kresd can find them.
To use the block list please modify the following kresd configuration files (experimental / untested!):
The knot-resolver (kresd) is only available on turris omnia devices. Currently there's no package for kresd in the official LEDE / OpenWrt package repository.
Adblock deposits the sorted and filtered block list (adb_list.overall) in '/etc/kresd' where kresd can find them.
To use the block list please create/modify the following kresd configuration files:
edit '/etc/config/resolver' and uncomment the following option:
option include_config '/etc/kresd/custom.conf'
@ -169,25 +174,27 @@ edit '/etc/config/resolver' and uncomment the following option:
in the same file change the 'forward_upstream' option like that:
forward_upstream '0'
edit '/etc/kresd/custom.conf' and add:
policy.add(policy.rpz(policy.DENY, '/etc/kresd/db.rpz'))
create '/etc/kresd/custom.conf' and add:
policy.add(policy.rpz(policy.DENY, '/etc/kresd/adb_list.overall'))
policy.add(policy.all(policy.FORWARD('8.8.8.8')))
policy.add(policy.all(policy.FORWARD('8.8.4.4')))
</code></pre>
**change default dns backend to 'dnscrypt-proxy':**
<pre><code>
Adblock deposits the sorted and filtered block list (adb_list.overall) by default in '/tmp' where DNSCrypt-Proxy can find them.
The blacklist option is not supported by default, because DNSCrypt-Proxy is compiled without plugins support.
Take a custom LEDE build with plugins support to use this feature:
create the new file '/etc/kresd/db.rpz' and add:
$TTL 2h
$ORIGIN rpz.
@ SOA localhost. root.localhost. (1 6h 1h 1w 2h)
NS localhost.
$INCLUDE /tmp/kresd/adb_list.overall
edit '/etc/config/dnscrypt-proxy' and add the following option per dnscrypt-proxy instance:
list blacklist 'domains:/tmp/adb_list.overall'
</code></pre>
**configuration for different download utilities:**
<pre><code>
wget (default):
option adb_fetch="/usr/bin/wget"
option adb_fetchparm="--quiet --no-cache --no-cookies --max-redirect=0 --timeout=10 --no-check-certificate -O"
option adb_fetch '/usr/bin/wget'
option adb_fetchparm '--quiet --no-cache --no-cookies --max-redirect=0 --timeout=10 --no-check-certificate -O'
aria2c:
option adb_fetch '/usr/bin/aria2c'
@ -206,13 +213,13 @@ curl:
<pre><code>
/etc/init.d/adblock status
::: adblock runtime information
status : active
adblock_version : 2.8.0
blocked_domains : 122827
fetch_info : wget (built-in)
dns_backend : dnsmasq
last_rundate : 26.06.2017 17:00:27
system : LEDE Reboot SNAPSHOT r4434-b91a38d647
+ adblock_status : enabled
+ adblock_version : 3.0.0
+ blocked_domains : 37406
+ fetch_utility : wget (built-in)
+ dns_backend : kresd (/etc/kresd)
+ last_rundate : 08.09.2017 21:21:21
+ system_release : OpenWrt omnia 15.05
</code></pre>
**cronjob for a regular block list update (/etc/crontabs/root):**
@ -250,17 +257,16 @@ This entry does not remove:
**query active block list for a certain (sub-)domain, e.g. for whitelisting:**
<pre><code>
/etc/init.d/adblock query example.www.doubleclick.net
::: results for (sub-)domain 'example.www.doubleclick.net' (max. 5)
- no match
::: results for (sub-)domain 'www.doubleclick.net' (max. 5)
- no match
::: results for (sub-)domain 'doubleclick.net' (max. 5)
+ doubleclick.net
+ feedads.g.doubleclick.net
+ survey.g.doubleclick.net
root@turris:~# /etc/init.d/adblock query example.www.doubleclick.net
::: max. ten results for domain 'example.www.doubleclick.net'
- no match
::: max. ten results for domain 'www.doubleclick.net'
- no match
::: max. ten results for domain 'doubleclick.net'
+ doubleclick.net
The query function checks against the submitted (sub-)domain and recurses automatically to the upper top level domain(s).
For every (sub-)domain it returns the first five relevant results.
The query function checks against the submitted (sub-)domain and recurses automatically to the upper top level domain.
For every (sub-)domain it returns the first ten relevant results.
In the example above whitelist "doubleclick.net" to free the submitted domain.
</code></pre>

12
net/adblock/files/adblock.conf
View file

@ -2,12 +2,12 @@
# see 'https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md'
config adblock 'global'
option adb_enabled '1'
option adb_enabled '0'
option adb_debug '0'
option adb_forcesrt '0'
option adb_forcedns '0'
option adb_iface 'wan'
option adb_triggerdelay '2'
option adb_dns 'dnsmasq'
option adb_trigger 'wan'
option adb_whitelist '/etc/adblock/adblock.whitelist'
option adb_whitelist_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}'
option adb_backup '0'
@ -86,6 +86,12 @@ config source 'reg_cn'
option adb_src_rset '{FS=\"[|^]\"} \$0 ~/^\|\|([A-Za-z0-9_-]+\.){1,}[A-Za-z]+\^$/{print tolower(\$3)}'
option adb_src_desc 'focus on chinese ads, daily updates, approx. 1.600 entries'
config source 'reg_id'
option enabled '0'
option adb_src 'https://easylist-downloads.adblockplus.org/abpindo+easylist.txt'
option adb_src_rset '{FS=\"[|^]\"} \$0 ~/^\|\|([A-Za-z0-9_-]+\.){1,}[A-Za-z]+\^$/{print tolower(\$3)}'
option adb_src_desc 'focus on indonesian ads plus generic easylist additions, weekly updates, approx. 800 entries'
config source 'reg_nl'
option enabled '0'
option adb_src 'https://easylist-downloads.adblockplus.org/easylistdutch+easylist.txt'

26
net/adblock/files/adblock.init
View file

@ -1,7 +1,7 @@
#!/bin/sh /etc/rc.common
#
START=50
START=30
USE_PROCD=1
EXTRA_COMMANDS="suspend resume query status"
@ -16,7 +16,6 @@ adb_script="/usr/bin/adblock.sh"
boot()
{
adb_boot=1
ubus -t 30 wait_for network.interface 2>/dev/null
rc_procd start_service
}
@ -26,7 +25,11 @@ start_service()
then
if [ -n "${adb_boot}" ]
then
return 0
local trigger="$(uci_get adblock.global.adb_trigger)"
if [ "${trigger}" != "timed" ]
then
return 0
fi
fi
procd_open_instance "adblock"
procd_set_param command "${adb_script}" "${@}"
@ -74,14 +77,13 @@ status()
service_triggers()
{
local iface="$(uci -q get adblock.global.adb_iface)"
local delay="$(uci -q get adblock.global.adb_triggerdelay)"
local trigger="$(uci_get adblock.global.adb_trigger)"
local delay="$(uci_get adblock.global.adb_triggerdelay)"
PROCD_RELOAD_DELAY=$((${delay:=2} * 1000))
for name in ${iface}
do
procd_add_interface_trigger "interface.*.up" "${name}" "${adb_init}" start
done
PROCD_RELOAD_DELAY=1000
procd_add_config_trigger "config.change" "adblock" "${adb_init}" start
if [ "${trigger}" != "none" ] && [ "${trigger}" != "timed" ]
then
PROCD_RELOAD_DELAY=$((${delay:=1} * 1000))
procd_add_interface_trigger "interface.*.up" "${trigger}" "${adb_init}" start
fi
procd_add_reload_trigger "adblock"
}

282
net/adblock/files/adblock.sh
View file

@ -10,33 +10,31 @@
#
LC_ALL=C
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
adb_ver="2.8.5"
adb_ver="3.0.0"
adb_sysver="$(ubus -S call system board | jsonfilter -e '@.release.description')"
adb_enabled=1
adb_enabled=0
adb_debug=0
adb_minfree=2
adb_manmode=0
adb_forcesrt=0
adb_forcedns=0
adb_triggerdelay=0
adb_backup=0
adb_backupdir="/mnt"
adb_whitelist="/etc/adblock/adblock.whitelist"
adb_whitelist_rset="\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}"
adb_fetch="/usr/bin/wget"
adb_fetchparm="--quiet --no-cache --no-cookies --max-redirect=0 --timeout=10 --no-check-certificate -O"
adb_dnslist="dnsmasq unbound named kresd"
adb_dns="dnsmasq"
adb_dnsprefix="adb_list"
adb_dnsfile="${adb_dnsprefix}.overall"
adb_rtfile="/tmp/adb_runtime.json"
adb_sources=""
adb_src_cat_shalla=""
adb_action="${1}"
adb_action="${1:-"start"}"
adb_cnt=0
adb_rc=0
# f_envload: load adblock environment
#
f_envload()
{
local services dns_up cnt=0
local dns_up cnt=0
# source in system libraries
#
@ -89,55 +87,60 @@ f_envload()
# set dns backend environment
#
while [ ${cnt} -le 20 ]
case "${adb_action}" in
start|restart|reload)
> "${adb_rtfile}"
if [ "${adb_action}" = "start" ] && [ "${adb_trigger}" = "timed" ]
then
sleep ${adb_triggerdelay}
fi
;;
esac
while [ ${cnt} -le 30 ]
do
services="$(ubus -S call service list 2>/dev/null)"
if [ -n "${services}" ]
dns_up="$(ubus -S call service list "{\"name\":\"${adb_dns}\"}" 2>/dev/null | jsonfilter -l1 -e "@[\"${adb_dns}\"].instances.*.running" 2>/dev/null)"
if [ "${dns_up}" = "true" ]
then
for dns in ${adb_dnslist}
do
dns_up="$(printf "%s" "${services}" | jsonfilter -l1 -e "@.${dns}.instances.*.running")"
if [ "${dns_up}" = "true" ]
then
case "${dns}" in
dnsmasq)
adb_dns="${dns}"
adb_dnsdir="${adb_dnsdir:="/tmp/dnsmasq.d"}"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsformat="awk '{print \"local=/\"\$0\"/\"}'"
break 2
;;
unbound)
adb_dns="${dns}"
adb_dnsdir="${adb_dnsdir:="/var/lib/unbound"}"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsformat="awk '{print \"local-zone: \042\"\$0\"\042 static\"}'"
break 2
;;
named)
adb_dns="${dns}"
adb_dnsdir="${adb_dnsdir:="/var/lib/bind"}"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsformat="awk '{print \"\"\$0\" IN CNAME .\n*.\"\$0\" IN CNAME .\"}'"
break 2
;;
kresd)
adb_dns="${dns}"
adb_dnsdir="${adb_dnsdir:="/tmp/kresd"}"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsformat="awk '{print \"\"\$0\" CNAME .\n*.\"\$0\" CNAME .\"}'"
break 2
;;
esac
fi
done
case "${adb_dns}" in
dnsmasq)
adb_dnsuser="${adb_dns}"
adb_dnsdir="${adb_dnsdir:-"/tmp/dnsmasq.d"}"
adb_dnsformat="awk '{print \"local=/\"\$0\"/\"}'"
break
;;
unbound)
adb_dnsuser="${adb_dns}"
adb_dnsdir="${adb_dnsdir:-"/var/lib/unbound"}"
adb_dnsformat="awk '{print \"local-zone: \042\"\$0\"\042 static\"}'"
break
;;
named)
adb_dnsuser="bind"
adb_dnsdir="${adb_dnsdir:-"/var/lib/bind"}"
adb_dnsformat="awk '{print \"\"\$0\" IN CNAME .\n*.\"\$0\" IN CNAME .\"}'"
break
;;
kresd)
adb_dnsuser="root"
adb_dnsdir="${adb_dnsdir:-"/etc/kresd"}"
adb_dnsformat="awk '{print \"\"\$0\" CNAME .\n*.\"\$0\" CNAME .\"}'"
adb_dnsheader="\$TTL 2h"$'\n'"@ IN SOA localhost. root.localhost. (2 6h 1h 1w 2h)"$'\n'" IN NS localhost."
break
;;
dnscrypt-proxy)
adb_dnsuser="nobody"
adb_dnsdir="${adb_dnsdir:-"/tmp"}"
adb_dnsformat="awk '{print \$0}'"
break
;;
esac
fi
sleep 1
cnt=$((cnt+1))
done
if [ -z "${adb_dns}" ] || [ -z "${adb_dnsformat}" ] || [ ! -x "$(command -v ${adb_dns})" ] || [ ! -d "${adb_dnsdir}" ]
then
f_log "error" "no active/supported DNS backend found"
f_log "error" "'${adb_dns}' not running, DNS backend not found"
fi
# force dns to local resolver
@ -180,6 +183,7 @@ f_envcheck()
f_rmdns
f_dnsrestart
fi
f_jsnupdate
f_log "info " "adblock is currently disabled, please set adb_enabled to '1' to use this service"
exit 0
fi
@ -192,7 +196,8 @@ f_envcheck()
if [ "$(readlink -fn "${adb_fetch}")" = "/usr/bin/wget-nossl" ]
then
adb_fetchparm="--quiet --no-cache --no-cookies --max-redirect=0 --timeout=10 -O"
elif [ "$(readlink -fn "/bin/wget")" = "/bin/busybox" ] || [ "$(readlink -fn "${adb_fetch}")" = "/bin/busybox" ]
elif [ "$(readlink -fn "${adb_fetch}")" = "/bin/busybox" ] ||
([ "$(readlink -fn "/bin/wget")" = "/bin/busybox" ] && [ "$(readlink -fn "${adb_fetch}")" != "/usr/bin/wget" ])
then
adb_fetch="/bin/busybox"
adb_fetchparm="-q -O"
@ -217,16 +222,6 @@ f_envcheck()
fi
adb_fetchinfo="${adb_fetch##*/} (${ssl_lib})"
# create dns hideout directory
#
if [ ! -d "${adb_dnshidedir}" ]
then
mkdir -p -m 660 "${adb_dnshidedir}"
chown -R "${adb_dns}":"${adb_dns}" "${adb_dnshidedir}" 2>/dev/null
else
rm -f "${adb_dnshidedir}/${adb_dnsprefix}"*
fi
# create adblock temp file/directory
#
adb_tmpload="$(mktemp -tu)"
@ -235,7 +230,7 @@ f_envcheck()
# prepare whitelist entries
#
if [ -s "${adb_whitelist}" ]
if [ -s "${adb_whitelist}" ] && [ -n "${adb_whitelist_rset}" ]
then
awk "${adb_whitelist_rset}" "${adb_whitelist}" > "${adb_tmpdir}/tmp.whitelist"
fi
@ -259,10 +254,10 @@ f_rmdns()
{
if [ -n "${adb_dns}" ]
then
rm -f "${adb_dnsdir}/${adb_dnsprefix}"*
rm -f "${adb_backupdir}/${adb_dnsprefix}"*.gz
rm -rf "${adb_dnshidedir}"
> "${adb_dnsdir}/${adb_dnsfile}"
> "${adb_rtfile}"
rm -f "${adb_dnsdir}/.${adb_dnsfile}"
rm -f "${adb_backupdir}/${adb_dnsprefix}"*.gz
fi
}
@ -270,19 +265,15 @@ f_rmdns()
#
f_dnsrestart()
{
local dns_up mem_free cnt=0
local dns_up cnt=0
"/etc/init.d/${adb_dns}" restart >/dev/null 2>&1
while [ ${cnt} -le 10 ]
do
dns_up="$(ubus -S call service list "{\"name\":\"${adb_dns}\"}" | jsonfilter -l1 -e "@.${adb_dns}.instances.*.running")"
dns_up="$(ubus -S call service list "{\"name\":\"${adb_dns}\"}" | jsonfilter -l1 -e "@[\"${adb_dns}\"].instances.*.running")"
if [ "${dns_up}" = "true" ]
then
mem_free="$(awk '/^MemFree/ {print int($2/1000)}' "/proc/meminfo")"
if [ ${mem_free} -ge ${adb_minfree} ]
then
return 0
fi
return 0
fi
cnt=$((cnt+1))
sleep 1
@ -334,6 +325,11 @@ f_list()
else
eval "${adb_dnsformat}" "${adb_tmpdir}/${adb_dnsfile}" > "${adb_dnsdir}/${adb_dnsfile}"
fi
if [ -n "${adb_dnsheader}" ]
then
printf '%s\n' "${adb_dnsheader}" | cat - "${adb_dnsdir}/${adb_dnsfile}" > "${adb_tmpdir}/${adb_dnsfile}"
cat "${adb_tmpdir}/${adb_dnsfile}" > "${adb_dnsdir}/${adb_dnsfile}"
fi
adb_rc=${?}
;;
esac
@ -358,25 +354,24 @@ f_switch()
{
local source target status mode="${1}"
if [ -d "${adb_dnshidedir}" ]
if [ -s "${adb_dnsdir}/${adb_dnsfile}" ] && [ "${mode}" = "suspend" ]
then
if [ -s "${adb_dnsdir}/${adb_dnsfile}" ] && [ "${mode}" = "suspend" ]
then
source="${adb_dnsdir}/${adb_dnsfile}"
target="${adb_dnshidedir}"
status="suspended"
elif [ -s "${adb_dnshidedir}/${adb_dnsfile}" ] && [ "${mode}" = "resume" ]
then
source="${adb_dnshidedir}/${adb_dnsfile}"
target="${adb_dnsdir}"
status="resumed"
fi
if [ -n "${status}" ]
then
mv -f "${source}"* "${target}"
f_dnsrestart
f_log "info " "adblock processing ${status}"
fi
source="${adb_dnsdir}/${adb_dnsfile}"
target="${adb_dnsdir}/.${adb_dnsfile}"
status="suspended"
elif [ -s "${adb_dnsdir}/.${adb_dnsfile}" ] && [ "${mode}" = "resume" ]
then
source="${adb_dnsdir}/.${adb_dnsfile}"
target="${adb_dnsdir}/${adb_dnsfile}"
status="resumed"
fi
if [ -n "${status}" ]
then
cat "${source}" > "${target}"
> "${source}"
f_dnsrestart
f_jsnupdate
f_log "info " "adblock processing ${status}"
fi
}
@ -393,21 +388,74 @@ f_query()
printf "%s\n" "::: no active block list found, please start / resume adblock first"
elif [ -z "${domain}" ] || [ "${domain}" = "${tld}" ]
then
printf "%s\n" "::: invalid domain input, please submit a specific (sub-)domain, e.g. 'www.abc.xyz'"
printf "%s\n" "::: invalid domain input, please submit a single domain, e.g. 'doubleclick.net'"
else
cd "${adb_dnsdir}"
while [ "${domain}" != "${tld}" ]
do
search="${domain//./\.}"
result="$(grep -Hm5 "[/\"\.]${search}[/\"]" "${adb_dnsfile}" | awk -F ':|=|/|\"' '{printf(" + %s\n",$4)}')"
printf "%s\n" "::: results for (sub-)domain '${domain}' (max. 5)"
printf "%s\n" "${result:=" - no match"}"
if [ "${adb_dns}" = "dnsmasq" ] || [ "${adb_dns}" = "unbound" ]
then
result="$(awk -F '/|\"' "/[\/\"\.]${search}/{i++;{printf(\" + %s\n\",\$2)};if(i>9){exit}}" "${adb_dnsfile}")"
else
result="$(awk "/(^[^\*][a-z]*[\.]+${search}|^${search})/{i++;{printf(\" + %s\n\",\$1)};if(i>9){exit}}" "${adb_dnsfile}")"
fi
printf "%s\n" "::: max. ten results for domain '${domain}'"
printf "%s\n" "${result:-" - no match"}"
domain="${tld}"
tld="${domain#*.}"
done
fi
}
# f_jsnupdate: update runtime information
#
f_jsnupdate()
{
local status rundate="$(/bin/date "+%d.%m.%Y %H:%M:%S")"
if [ ${adb_rc} -gt 0 ]
then
status="error"
elif [ ${adb_enabled} -ne 1 ]
then
status="disabled"
elif [ -s "${adb_dnsdir}/.${adb_dnsfile}" ]
then
status="paused"
else
status="enabled"
if [ -s "${adb_dnsdir}/${adb_dnsfile}" ]
then
if [ "${adb_dns}" = "named" ] || [ "${adb_dns}" = "kresd" ]
then
adb_cnt="$(( ( $(wc -l < "${adb_dnsdir}/${adb_dnsfile}") - $(printf "%s" "${adb_dnsheader}" | grep -c "^") ) / 2 ))"
else
adb_cnt="$(( $(wc -l < "${adb_dnsdir}/${adb_dnsfile}") - $(printf "%s" "${adb_dnsheader}" | grep -c "^") ))"
fi
fi
fi
if [ -z "${adb_fetchinfo}" ] && [ -s "${adb_rtfile}" ]
then
json_load "$(cat "${adb_rtfile}" 2>/dev/null)"
json_select data
json_get_var adb_fetchinfo "fetch_utility"
fi
json_init
json_add_object "data"
json_add_string "adblock_status" "${status}"
json_add_string "adblock_version" "${adb_ver}"
json_add_string "blocked_domains" "${adb_cnt}"
json_add_string "fetch_utility" "${adb_fetchinfo}"
json_add_string "dns_backend" "${adb_dns} (${adb_dnsdir})"
json_add_string "last_rundate" "${rundate}"
json_add_string "system_release" "${adb_sysver}"
json_close_object
json_dump > "${adb_rtfile}"
}
# f_status: output runtime information
#
f_status()
@ -416,21 +464,14 @@ f_status()
if [ -s "${adb_rtfile}" ]
then
if [ -s "${adb_dnsdir}/${adb_dnsfile}" ]
then
value="active"
else
value="no domains blocked"
fi
printf "%s\n" "::: adblock runtime information"
printf " %-15s : %s\n" "status" "${value}"
json_load "$(cat "${adb_rtfile}" 2>/dev/null)"
json_select data
json_get_keys keylist
for key in ${keylist}
do
json_get_var value "${key}"
printf " %-15s : %s\n" "${key}" "${value}"
printf " + %-15s : %s\n" "${key}" "${value}"
done
fi
}
@ -453,6 +494,8 @@ f_log()
f_rmdns
f_dnsrestart
fi
adb_rc=1
f_jsnupdate
exit 1
fi
fi
@ -462,7 +505,7 @@ f_log()
#
f_main()
{
local src_name src_rset shalla_archive enabled url hash_old hash_new cnt=0
local src_name src_rset shalla_archive enabled url hash_old hash_new
local mem_total="$(awk '/^MemTotal/ {print int($2/1000)}' "/proc/meminfo")"
f_log "info " "start adblock processing ..."
@ -488,7 +531,7 @@ f_main()
# manual / backup mode
#
if [ ${adb_manmode} -eq 1 ] && [ -z "${adb_action}" ] && [ "${src_name}" != "blacklist" ]
if [ ${adb_manmode} -eq 1 ] && [ "${adb_action}" = "start" ] && [ "${src_name}" != "blacklist" ]
then
f_list restore
if [ ${adb_rc} -eq 0 ] && [ -s "${adb_tmpfile}" ]
@ -536,7 +579,10 @@ f_main()
if [ -s "${adb_tmpfile}" ]
then
f_tldcompression "${adb_tmpfile}"
f_list backup
if [ "${src_name}" != "blacklist" ]
then
f_list backup
fi
else
f_list restore
fi
@ -562,7 +608,7 @@ f_main()
#
if [ -f "${adb_dnsdir}/${adb_dnsfile}" ]
then
hash_old="$(sha256sum "${adb_dnsdir}/${adb_dnsfile}" | awk '{print $1}')"
hash_old="$(sha256sum "${adb_dnsdir}/${adb_dnsfile}" 2>/dev/null | awk '{print $1}')"
fi
if [ -s "${adb_tmpdir}/${adb_dnsfile}" ]
then
@ -574,30 +620,20 @@ f_main()
else
> "${adb_dnsdir}/${adb_dnsfile}"
fi
hash_new="$(sha256sum "${adb_dnsdir}/${adb_dnsfile}" | awk '{print $1}')"
cnt="$(wc -l < "${adb_dnsdir}/${adb_dnsfile}")"
chown "${adb_dnsuser}":"${adb_dnsuser}" "${adb_dnsdir}/${adb_dnsfile}" 2>/dev/null
f_rmtemp
# conditional restart of the dns backend and runtime information export
#
chown "${adb_dns}":"${adb_dns}" "${adb_dnsdir}/${adb_dnsfile}" 2>/dev/null
f_rmtemp
if [ "${hash_old}" != "${hash_new}" ]
hash_new="$(sha256sum "${adb_dnsdir}/${adb_dnsfile}" 2>/dev/null | awk '{print $1}')"
if [ -z "${hash_old}" ] || [ -z "${hash_new}" ] || [ "${hash_old}" != "${hash_new}" ]
then
f_dnsrestart
fi
if [ ${?} -eq 0 ]
then
json_init
json_add_object "data"
json_add_string "adblock_version" "${adb_ver}"
json_add_string "blocked_domains" "${cnt}"
json_add_string "fetch_info" "${adb_fetchinfo}"
json_add_string "dns_backend" "${adb_dns}"
json_add_string "last_rundate" "$(/bin/date "+%d.%m.%Y %H:%M:%S")"
json_add_string "system" "${adb_sysver}"
json_close_object
json_dump > "${adb_rtfile}"
f_log "info " "block list with overall ${cnt} domains loaded successfully (${adb_sysver})"
f_jsnupdate "${adb_cnt}"
f_log "info " "block list with overall ${adb_cnt} domains loaded successfully (${adb_sysver})"
else
f_log "error" "dns backend restart with active block list failed"
fi