Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

adblock: update 1.4.4

Browse source 
Update for-15.05 adblock from 1.2.1 to 1.4.4

Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>

Original commit messages of 1.2.5-1.4.4 commits:

adblock: update 1.2.5
* restructured sources
* fix logical glitches in config handling
* many corner case fixes & cosmetics
* show runtime errors in LuCI (in lastrun section)

adbock: update 1.2.6
* small addition in case of a failed list download

adblock: update 1.2.7
* provide adblock statistics as a separate function
  (/etc/init.d/adblock stats)

adblock: update 1.2.8
* fw rule changes:
	force_dns now supports multiple lan devices
	disable needless force_dns- & forward/output-rules in 'ap mode'
	check return codes during adblock chain creation
* simplified the test for a running firewall
* documentation update

adblock: release 1.3.0
* revised hotplug script
* remove wget package dependency
* support uclient-fetch or wget with ssl support
* documentation update

adblock: update 1.3.1
* fix uclient-fetch detection
* cosmetics

adblock: update 1.3.2
* only a few more fixes

adblock: update 1.3.3
* enable automatic restore on empty source downloads

adblock: release 1.4.0
* rework/speed up overall sort
* simplified dns error handling

adblock: update 1.4.1
* fix two possible overflows in adblock statistics

adblock: update 1.4.2
* ad broad blocklist source 'hphosts' https://hosts-file.net

adblock: update 1.4.3
* fix race condition in restricted mode
* cosmetics

adblock: update 1.4.4
* filter non-printable characters/binary data in input stream
* fix IPv4 adblock statistics in CC
This commit is contained in:
Dirk Brenken 2016-08-14 19:23:04 +03:00 committed by Hannu Nyman
parent 01c323c293
commit 7ea2bf5899
7 changed files with 550 additions and 505 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
net/adblock/Makefile
View file

@ -7,7 +7,7 @@
include $(TOPDIR)/rules.mk include $(TOPDIR)/rules.mk
PKG_NAME:=adblock PKG_NAME:=adblock
PKG_VERSION:=1.2.1 PKG_VERSION:=1.4.4
PKG_RELEASE:=1 PKG_RELEASE:=1
PKG_LICENSE:=GPL-3.0+ PKG_LICENSE:=GPL-3.0+
PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org> PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org>
@ -18,13 +18,12 @@ define Package/$(PKG_NAME)
SECTION:=net SECTION:=net
CATEGORY:=Network CATEGORY:=Network
TITLE:=Powerful adblock script to block ad/abuse domains TITLE:=Powerful adblock script to block ad/abuse domains
DEPENDS:=+wget
PKGARCH:=all PKGARCH:=all
endef endef
define Package/$(PKG_NAME)/description define Package/$(PKG_NAME)/description
Powerful adblock script to block ad/abuse domains. Powerful adblock script to block ad/abuse domains.
Currently the script supports 19 domain blacklist sites plus manual black- and whitelist overrides. Currently the script supports 20 domain blacklist sites plus manual black- and whitelist overrides.
Please see https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md for further information. Please see https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md for further information.
endef endef

26
net/adblock/files/README.md
View file

@ -15,6 +15,8 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* => daily updates, approx. 4.500 entries * => daily updates, approx. 4.500 entries
* [feodotracker](https://feodotracker.abuse.ch) * [feodotracker](https://feodotracker.abuse.ch)
* => daily updates, approx. 0-10 entries * => daily updates, approx. 0-10 entries
* [hphosts](https://hosts-file.net)
* => monthly updates, approx. 390.000 entries
* [malwaredomains](http://malwaredomains.com) * [malwaredomains](http://malwaredomains.com)
* => daily updates, approx. 16.000 entries * => daily updates, approx. 16.000 entries
* [malwaredomainlist](http://www.malwaredomainlist.com) * [malwaredomainlist](http://www.malwaredomainlist.com)
@ -58,9 +60,9 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* status & error logging to stdout and syslog * status & error logging to stdout and syslog
* use two dynamic uhttpd instances as adblock pixel server, separated for ads delivered on port 80 and on port 443 * use two dynamic uhttpd instances as adblock pixel server, separated for ads delivered on port 80 and on port 443
* use dynamic iptables chains/rulesets for adblock related redirects/rejects * use dynamic iptables chains/rulesets for adblock related redirects/rejects
* init system support (start/stop/restart/reload/toggle) * init system support (start/stop/restart/reload/toggle/stats/cfgup)
* hotplug support, the adblock start will be triggered by wan 'ifup' event * hotplug support, the adblock start will be triggered by wan 'ifup' event
* adblock toggle to quickly switch adblocking 'on' or 'off' * toggle to quickly switch adblock 'on' or 'off'
* optional: automatic adblock list backup/restore, backups will be (de-)compressed on the fly (disabled by default) * optional: automatic adblock list backup/restore, backups will be (de-)compressed on the fly (disabled by default)
* optional: add new adblock sources via uci config (see example below) * optional: add new adblock sources via uci config (see example below)
@ -69,7 +71,7 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* [LEDE project](https://www.lede-project.org), tested with trunk > r98 * [LEDE project](https://www.lede-project.org), tested with trunk > r98
* usual setup with enabled 'iptables', 'dnsmasq' and 'uhttpd' - dump AP modes without these basics are _not_ supported! * usual setup with enabled 'iptables', 'dnsmasq' and 'uhttpd' - dump AP modes without these basics are _not_ supported!
* additional required software packages: * additional required software packages:
* wget * a download utility: 'uclient-fetch' and 'wget' (full versions with ssl support) are supported. Normally you should use 'wget', it's quite stable and supports the online timestamp checks. If you need a smaller memory footprint try 'uclient-fetch' without openssl dependency. The default ustream ssl backend 'libustream-polarssl' has issues with certain https sites and is currently not supported. To change the ssl backend see example below.
* optional: 'kmod-ipt-nat6' for IPv6 support * optional: 'kmod-ipt-nat6' for IPv6 support
* the above dependencies and requirements will be checked during package installation & script runtime * the above dependencies and requirements will be checked during package installation & script runtime
@ -89,8 +91,8 @@ A lot of people already use adblocker plugins within their desktop browsers, but
## Chaos Calmer installation notes ## Chaos Calmer installation notes
* 'adblock' and 'luci-app-adblock' are _not_ available as .ipk packages in the Chaos Calmer download repository * 'adblock' and 'luci-app-adblock' are _not_ available as .ipk packages in the Chaos Calmer download repository
* download both packages from a development snapshot package directory: * download both packages from a development snapshot package directory:
* for 'adblock' look [here](https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/packages/) * for 'adblock' look [here](https://downloads.lede-project.org/snapshots/packages/x86_64/packages/)
* for 'luci-app-adblock' look [here](https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/luci/) * for 'luci-app-adblock' look [here](https://downloads.lede-project.org/snapshots/packages/x86_64/luci/)
* manually transfer the packages to your routers temp directory (with tools like _sshfs_ or _winscp_) * manually transfer the packages to your routers temp directory (with tools like _sshfs_ or _winscp_)
* install the packages with _opkg install <...>_ as described above * install the packages with _opkg install <...>_ as described above
@ -100,9 +102,10 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* **backup/restore:** enable the backup/restore feature, to restore automatically the latest compressed backup of your adblock lists in case of any processing error (i.e. a single blocklist source is down). Please use an (external) solid partition and _not_ your volatile router temp directory for this * **backup/restore:** enable the backup/restore feature, to restore automatically the latest compressed backup of your adblock lists in case of any processing error (i.e. a single blocklist source is down). Please use an (external) solid partition and _not_ your volatile router temp directory for this
* **list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below) * **list updates:** for a scheduled call of the adblock service add an appropriate crontab entry (see example below)
* **new list sources:** you could add new blocklist sources on your own via uci config, all you need is a source url and an awk one-liner (see example below) * **new list sources:** you could add new blocklist sources on your own via uci config, all you need is a source url and an awk one-liner (see example below)
* **AP mode:** in AP mode adblock uses automatically the local router ip as nullip address. To make sure that your LuCI interface will be still accessible, please change the local uhttpd instance to ports <> 80/443 (see example below) * **AP mode:** in 'AP mode' adblock uses automatically the local router ip as nullip address. To make sure that your LuCI interface will be still accessible, you have to change the local uhttpd instance to ports <> 80/443 (see example below)
* **restricted mode:** to disable flash writes with adblock status information to the adblock config file (used by LuCI frontend), please set 'adb\_restricted' to '1' * **restricted mode:** to disable flash writes with adblock status information to the adblock config file (used by LuCI frontend), please set 'adb\_restricted' to '1'
* **adblock toggle:** to quickly switch adblocking 'on' or 'off', simply use _/etc/init.d/adblock toggle_ * **adblock toggle:** to quickly switch adblocking 'on' or 'off', simply use _/etc/init.d/adblock toggle_
* **adblock statistics:** to update only the adblock statistics (without updating the block lists as well), please run _/etc/init.d/adblock stats_
* **configuration update:** to update an outdated adblock config file with the current default version, please run _/etc/init.d/adblock cfgup_, make your individual changes and start the adblock service again * **configuration update:** to update an outdated adblock config file with the current default version, please run _/etc/init.d/adblock cfgup_, make your individual changes and start the adblock service again
* **debugging:** for script debugging please set the 'adb\_debug' variable in the header of _/etc/init.d/adblock_ to '1' * **debugging:** for script debugging please set the 'adb\_debug' variable in the header of _/etc/init.d/adblock_ to '1'
* **disable active dns probing in windows:** to prevent a possible yellow exclamation mark on your internet connection icon (which wrongly means connected, but no internet), please change the following registry key/value from "1" to "0" _HKLM\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet\EnableActiveProbing_ * **disable active dns probing in windows:** to prevent a possible yellow exclamation mark on your internet connection icon (which wrongly means connected, but no internet), please change the following registry key/value from "1" to "0" _HKLM\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet\EnableActiveProbing_
@ -116,12 +119,19 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* adb\_nullportssl => port of the adblock uhttpd instance used for ads delivered on port 443 (default: '65535') * adb\_nullportssl => port of the adblock uhttpd instance used for ads delivered on port 443 (default: '65535')
* adb\_nullipv4 => IPv4 blackhole ip address (default: '198.18.0.1', in AP mode: local router ip) * adb\_nullipv4 => IPv4 blackhole ip address (default: '198.18.0.1', in AP mode: local router ip)
* adb\_nullipv6 => IPv6 blackhole ip address (default: '::ffff:c612:0001', in AP mode: local router ip) * adb\_nullipv6 => IPv6 blackhole ip address (default: '::ffff:c612:0001', in AP mode: local router ip)
* adb\_forcedns => redirect all local DNS queries to the local dnsmasq resolver (default: '1', enabled) * adb\_forcedns => redirect all local DNS queries to the local dnsmasq resolver (default: '1', enabled / always disabled in 'AP mode')
* adb\_fetchttl => set the timeout for list downloads (default: '5' seconds) * adb\_fetchttl => set the timeout for list downloads (default: '5' seconds)
* adb\_restricted => disable updates of the adblock config file (no flash writes) during runtime (default: '0', disabled) * adb\_restricted => disable updates of the adblock config file (no flash writes) during runtime (default: '0', disabled)
## Examples ## Examples
**example to change the ssl backend for 'uclient-fetch':**
<pre><code>
opkg update
opkg remove --force-depends libustream-polarssl
opkg install libustream-mbedtls
</code></pre>
**example cronjob for a regular block list update:** **example cronjob for a regular block list update:**
<pre><code> <pre><code>
# configuration found in /etc/crontabs/root # configuration found in /etc/crontabs/root
@ -223,7 +233,7 @@ If your awk one-liner works quite well, add a new source section in adblock conf
## Background ## Background
This adblock package is a dns/dnsmasq based adblock solution. This adblock package is a dns/dnsmasq based adblock solution.
Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6. For that purpose adblock uses an ip address from the private 'Benchmark Test' subnet (198.18.0.1 / ::ffff:c612:0001) by default (in AP mode the local router ip address will be used). Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to two uhttpd instances, separated for ads delivered on port 80 and on port 443 (in PREROUTING chain) or rejected (in FORWARD or OUTPUT chain). Queries to ad/abuse domains are never forwarded and always replied with a local IP address which may be IPv4 or IPv6. For that purpose adblock uses an ip address from the private 'Benchmark Test' subnet (198.18.0.1 / ::ffff:c612:0001) by default (in AP mode the local router ip address will be used). Furthermore all ad/abuse queries will be filtered by ip(6)tables and redirected to two uhttpd instances, separated for ads delivered on port 80 and on port 443 (in PREROUTING chain) or rejected (in FORWARD or OUTPUT chain). In 'AP mode' only the uhttpd related rules in PREROUTING chain are enabled.
All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other system related config files. There is _no_ adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date. All iptables and uhttpd related adblock additions are non-destructive, no hard-coded changes in 'firewall.user', 'uhttpd' config or any other system related config files. There is _no_ adblock background daemon running, the (scheduled) start of the adblock service keeps only the adblock lists up-to-date.

565
net/adblock/files/adblock-helper.sh
View file

@ -2,6 +2,27 @@
# function library used by adblock-update.sh # function library used by adblock-update.sh
# written by Dirk Brenken (dev@brenken.org) # written by Dirk Brenken (dev@brenken.org)
# set initial defaults
#
LC_ALL=C
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
adb_lanif="lan"
adb_nullport="65534"
adb_nullportssl="65535"
adb_nullipv4="198.18.0.1"
adb_nullipv6="::ffff:c612:0001"
adb_whitelist="/etc/adblock/adblock.whitelist"
adb_whitelist_rset="\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}"
adb_dnsdir="/tmp/dnsmasq.d"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsprefix="adb_list"
adb_count=0
adb_minspace=12000
adb_forcedns=1
adb_fetchttl=5
adb_restricted=0
adb_uci="$(which uci)"
# f_envload: load adblock environment # f_envload: load adblock environment
# #
f_envload() f_envload()
@ -12,7 +33,7 @@ f_envload()
then then
. "/lib/functions.sh" . "/lib/functions.sh"
else else
rc=-1 rc=-10
f_log "system function library not found, please check your installation" f_log "system function library not found, please check your installation"
f_exit f_exit
fi fi
@ -23,26 +44,21 @@ f_envload()
then then
. "/lib/functions/network.sh" . "/lib/functions/network.sh"
else else
rc=-1 rc=-10
f_log "system network library not found, please check your installation" f_log "system network library not found, please check your installation"
f_exit f_exit
fi fi
# set initial defaults, # check opkg availability
# may be overwritten by setting appropriate adblock config options in global section of /etc/config/adblock
# #
adb_lanif="lan" if [ -f "/var/lock/opkg.lock" ]
adb_nullport="65534" then
adb_nullportssl="65535" rc=-10
adb_nullipv4="198.18.0.1" f_log "adblock installation finished successfully, 'opkg' currently locked by package installer"
adb_nullipv6="::ffff:c612:0001" f_exit
adb_whitelist="/etc/adblock/adblock.whitelist" fi
adb_whitelist_rset="\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}"
adb_forcedns=1
adb_fetchttl=5
adb_restricted=0
# function to parse global section by callback # uci function to parse global section by callback
# #
config_cb() config_cb()
{ {
@ -60,14 +76,11 @@ f_envload()
fi fi
} }
# function to parse 'service' and 'source' sections # uci function to parse 'service' and 'source' sections
# #
parse_config() parse_config()
{ {
local value opt section="${1}" options="adb_dir adb_src adb_src_rset adb_src_cat" local value opt section="${1}" options="enabled adb_dir adb_src adb_src_rset adb_src_cat"
config_get switch "${section}" "enabled"
if [ "${switch}" = "1" ]
then
if [ "${section}" != "backup" ] if [ "${section}" != "backup" ]
then then
eval "adb_sources=\"${adb_sources} ${section}\"" eval "adb_sources=\"${adb_sources} ${section}\""
@ -80,49 +93,28 @@ f_envload()
eval "${opt}_${section}=\"${value}\"" eval "${opt}_${section}=\"${value}\""
fi fi
done done
fi
} }
# check opkg availability
#
if [ -r "/var/lock/opkg.lock" ]
then
rc=-1
f_log "adblock installation finished successfully, 'opkg' currently locked by package installer"
f_exit
fi
# get list with all installed packages
#
pkg_list="$(opkg list-installed)"
if [ -z "${pkg_list}" ]
then
rc=-1
f_log "empty package list, please check your installation"
f_exit
fi
# load adblock config and start parsing functions # load adblock config and start parsing functions
# #
config_load adblock config_load adblock
config_foreach parse_config service config_foreach parse_config service
config_foreach parse_config source config_foreach parse_config source
# set more script defaults (can't be overwritten by adblock config options) # get network basics
# #
adb_minspace=12000 network_get_ipaddr adb_ipv4 "${adb_lanif}"
adb_tmpfile="$(mktemp -tu)" network_get_ipaddr6 adb_ipv6 "${adb_lanif}"
adb_tmpdir="$(mktemp -p /tmp -d)" network_get_device adb_landev "${adb_lanif}"
adb_dnsdir="/tmp/dnsmasq.d" network_find_wan adb_wanif4
adb_dnshidedir="${adb_dnsdir}/.adb_hidden" network_find_wan6 adb_wanif6
adb_dnsprefix="adb_list" }
adb_iptv4="$(which iptables)"
adb_iptv6="$(which ip6tables)" # f_envcheck: check/set environment prerequisites
adb_uhttpd="$(which uhttpd)" #
adb_fetch="$(which wget)" f_envcheck()
adb_uci="$(which uci)" {
adb_date="$(which date)" local check
unset adb_srclist adb_revsrclist
# check 'enabled' & 'version' config options # check 'enabled' & 'version' config options
# #
@ -133,33 +125,38 @@ f_envload()
f_exit f_exit
elif [ "${adb_cfgver#*.}" != "${adb_mincfgver#*.}" ] elif [ "${adb_cfgver#*.}" != "${adb_mincfgver#*.}" ]
then then
outdate_ok="true" outdated_ok="true"
fi fi
if [ $((adb_enabled)) -ne 1 ] if [ "${adb_enabled}" != "1" ]
then then
rc=-1 rc=-10
f_log "adblock is currently disabled, please set adblock.global.adb_enabled=1' to use this service" f_log "adblock is currently disabled, please set adblock.global.adb_enabled=1' to use this service"
f_exit f_exit
fi fi
# get list with all installed packages
#
pkg_list="$(opkg list-installed)"
if [ -z "${pkg_list}" ]
then
rc=-1
f_log "empty 'opkg' package list, please check your installation"
f_exit
fi
adb_sysver="$(printf "${pkg_list}" | grep "^base-files -")"
adb_sysver="${adb_sysver##*-}"
# get lan ip addresses # get lan ip addresses
# #
network_get_ipaddr adb_ipv4 "${adb_lanif}"
network_get_ipaddr6 adb_ipv6 "${adb_lanif}"
if [ -z "${adb_ipv4}" ] && [ -z "${adb_ipv6}" ] if [ -z "${adb_ipv4}" ] && [ -z "${adb_ipv6}" ]
then then
rc=-1 rc=-1
f_log "no valid IPv4/IPv6 configuration found (${adb_lanif}), please set 'adb_lanif' manually" f_log "no valid IPv4/IPv6 configuration found (${adb_lanif}), please set 'adb_lanif' manually"
f_exit f_exit
else
network_get_device adb_landev4 "${adb_lanif}"
network_get_device adb_landev6 "${adb_lanif}"
fi fi
# check logical update interfaces (with default route) # check logical update interfaces (with default route)
# #
network_find_wan adb_wanif4
network_find_wan6 adb_wanif6
if [ -z "${adb_wanif4}" ] && [ -z "${adb_wanif6}" ] if [ -z "${adb_wanif4}" ] && [ -z "${adb_wanif6}" ]
then then
adb_wanif4="${adb_lanif}" adb_wanif4="${adb_lanif}"
@ -171,18 +168,18 @@ f_envload()
then then
adb_nullipv4="${adb_ipv4}" adb_nullipv4="${adb_ipv4}"
adb_nullipv6="${adb_ipv6}" adb_nullipv6="${adb_ipv6}"
if [ "$(${adb_uci} -q get uhttpd.main.listen_http | grep -Fo "80")" = "80" ] || if [ -n "$(${adb_uci} -q get uhttpd.main.listen_http | grep -Fo "80")" ] ||
[ "$(${adb_uci} -q get uhttpd.main.listen_https | grep -Fo "443")" = "443" ] [ -n "$(${adb_uci} -q get uhttpd.main.listen_https | grep -Fo "443")" ]
then then
rc=-1 rc=-1
f_log "AP mode detected, set local LuCI instance to ports <> 80/443" f_log "AP mode detected, please set local LuCI instance to ports <> 80/443"
f_exit f_exit
elif [ -z "$(pgrep -f "dnsmasq")" ] elif [ -z "$(pgrep -f "dnsmasq")" ]
then then
rc=-1 rc=-1
f_log "please enable the local dnsmasq instance to use adblock" f_log "please enable the local dnsmasq instance to use adblock"
f_exit f_exit
elif [ -z "$(${adb_iptv4} -w -vnL | grep -Fo "DROP")" ] elif [ ! -f "/var/run/fw3.state" ]
then then
rc=-1 rc=-1
f_log "please enable the local firewall to use adblock" f_log "please enable the local firewall to use adblock"
@ -191,34 +188,93 @@ f_envload()
apmode_ok="true" apmode_ok="true"
fi fi
else else
apmode_ok="false"
check="$(${adb_uci} -q get bcp38.@bcp38[0].enabled)" check="$(${adb_uci} -q get bcp38.@bcp38[0].enabled)"
if [ $((check)) -eq 1 ] if [ "${check}" = "1" ]
then then
check="$(${adb_uci} -q get bcp38.@bcp38[0].match | grep -Fo "${adb_nullipv4%.*}")" if [ -n "$(${adb_uci} -q get bcp38.@bcp38[0].match | grep -Fo "${adb_nullipv4%.*}")" ]
if [ -n "${check}" ]
then then
rc=-1 rc=-1
f_log "please whitelist '${adb_nullipv4}' in your bcp38 configuration to use your adblock null-ip" f_log "please whitelist '${adb_nullipv4}' in your bcp38 configuration to use adblock"
f_exit f_exit
fi fi
fi fi
fi fi
# get system release level # check general package dependencies
# #
adb_sysver="$(printf "${pkg_list}" | grep "^base-files -")" f_depend "busybox"
adb_sysver="${adb_sysver##*-}" f_depend "uci"
} f_depend "uhttpd"
f_depend "iptables"
f_depend "kmod-ipt-nat"
# f_envcheck: check/set environment prerequisites # check ipv6 related package dependencies
# #
f_envcheck() if [ -n "${adb_wanif6}" ]
{ then
local check f_depend "ip6tables" "true"
if [ "${package_ok}" = "false" ]
then
f_log "package 'ip6tables' not found, IPv6 support will be disabled"
unset adb_wanif6
else
f_depend "kmod-ipt-nat6" "true"
if [ "${package_ok}" = "false" ]
then
f_log "package 'kmod-ipt-nat6' not found, IPv6 support will be disabled"
unset adb_wanif6
fi
fi
fi
# check uclient-fetch/wget dependencies
#
f_depend "uclient-fetch" "true"
if [ "${package_ok}" = "true" ]
then
f_depend "libustream-polarssl" "true"
if [ "${package_ok}" = "false" ]
then
f_depend "libustream-\(mbedtls\|openssl\|cyassl\)" "true"
if [ "${package_ok}" = "true" ]
then
adb_fetch="$(which uclient-fetch)"
fetch_parm="-q --timeout=${adb_fetchttl}"
response_parm="--spider"
fi
fi
fi
if [ -z "${adb_fetch}" ]
then
f_depend "wget" "true"
if [ "${package_ok}" = "true" ]
then
adb_fetch="$(which wget)"
fetch_parm="--no-config --quiet --tries=1 --no-cache --no-cookies --max-redirect=0 --dns-timeout=${adb_fetchttl} --connect-timeout=${adb_fetchttl} --read-timeout=${adb_fetchttl}"
response_parm="--spider --server-response"
else
rc=-1
f_log "please install 'uclient-fetch' or 'wget' with ssl support to use adblock"
f_exit
fi
fi
# check ca-certificate package and set fetch parm accordingly
#
f_depend "ca-certificates" "true"
if [ "${package_ok}" = "false" ]
then
fetch_parm="${fetch_parm} --no-check-certificate"
fi
# start normal processing/logging
#
f_log "domain adblock processing started (${adb_scriptver}, ${adb_sysver}, $(/bin/date "+%d.%m.%Y %H:%M:%S"))"
# log partially outdated config # log partially outdated config
# #
if [ "${outdate_ok}" = "true" ] if [ "${outdated_ok}" = "true" ]
then then
f_log "partially outdated adblock config (${adb_mincfgver} vs. ${adb_cfgver}), please run '/etc/init.d/adblock cfgup' to update your configuration" f_log "partially outdated adblock config (${adb_mincfgver} vs. ${adb_cfgver}), please run '/etc/init.d/adblock cfgup' to update your configuration"
fi fi
@ -230,42 +286,14 @@ f_envcheck()
f_log "AP mode enabled" f_log "AP mode enabled"
fi fi
# set & log restricted mode # set/log restricted mode
# #
if [ $((adb_restricted)) -eq 1 ] if [ "${adb_restricted}" = "1" ]
then then
adb_uci="$(which true)" adb_uci="$(which true)"
f_log "Restricted mode enabled" f_log "Restricted mode enabled"
fi fi
# check general package dependencies
#
f_depend "busybox"
f_depend "uci"
f_depend "uhttpd"
f_depend "wget"
f_depend "iptables"
f_depend "kmod-ipt-nat"
# check ipv6 related package dependencies
#
if [ -n "${adb_wanif6}" ]
then
check="$(printf "${pkg_list}" | grep "^ip6tables -")"
if [ -z "${check}" ]
then
f_log "package 'ip6tables' not found, IPv6 support will be disabled"
unset adb_wanif6
else
check="$(printf "${pkg_list}" | grep "^kmod-ipt-nat6 -")"
if [ -z "${check}" ]
then
f_log "package 'kmod-ipt-nat6' not found, IPv6 support will be disabled"
unset adb_wanif6
fi
fi
fi
# check dns hideout directory # check dns hideout directory
# #
if [ -d "${adb_dnshidedir}" ] if [ -d "${adb_dnshidedir}" ]
@ -275,17 +303,10 @@ f_envcheck()
mkdir -p -m 660 "${adb_dnshidedir}" mkdir -p -m 660 "${adb_dnshidedir}"
fi fi
# check ca-certificates package and set fetch parms accordingly
#
fetch_parm="--no-config --quiet --tries=1 --no-cache --no-cookies --max-redirect=0 --dns-timeout=${adb_fetchttl} --connect-timeout=${adb_fetchttl} --read-timeout=${adb_fetchttl}"
check="$(printf "${pkg_list}" | grep "^ca-certificates -")"
if [ -z "${check}" ]
then
fetch_parm="${fetch_parm} --no-check-certificate"
fi
# check adblock temp directory # check adblock temp directory
# #
adb_tmpfile="$(mktemp -tu)"
adb_tmpdir="$(mktemp -p /tmp -d)"
if [ -n "${adb_tmpdir}" ] && [ -d "${adb_tmpdir}" ] if [ -n "${adb_tmpdir}" ] && [ -d "${adb_tmpdir}" ]
then then
f_space "${adb_tmpdir}" f_space "${adb_tmpdir}"
@ -294,15 +315,15 @@ f_envcheck()
if [ $((av_space)) -le 2000 ] if [ $((av_space)) -le 2000 ]
then then
rc=105 rc=105
f_log "not enough free space in '${adb_tmpdir}' (avail. ${av_space} kb)" "${rc}" f_log "not enough free space in '${adb_tmpdir}' (avail. ${av_space} kb)"
f_exit f_exit
else else
f_log "not enough free space to handle all adblock list sources at once in '${adb_tmpdir}' (avail. ${av_space} kb)" f_log "not enough free space to handle all block list sources at once in '${adb_tmpdir}' (avail. ${av_space} kb)"
fi fi
fi fi
else else
rc=110 rc=110
f_log "temp directory not found" "${rc}" f_log "temp directory not found"
f_exit f_exit
fi fi
@ -321,7 +342,7 @@ f_envcheck()
# check backup configuration # check backup configuration
# #
if [ -n "${adb_dir_backup}" ] && [ -d "${adb_dir_backup}" ] if [ "${enabled_backup}" = "1" ] && [ -d "${adb_dir_backup}" ]
then then
f_space "${adb_dir_backup}" f_space "${adb_dir_backup}"
if [ "${space_ok}" = "false" ] if [ "${space_ok}" = "false" ]
@ -353,51 +374,45 @@ f_envcheck()
# #
if [ -n "${adb_wanif4}" ] if [ -n "${adb_wanif4}" ]
then then
check="$(${adb_iptv4} -w -vnL | grep -Fo "adb-")" if [ "${apmode_ok}" = "false" ]
if [ -z "${check}" ]
then then
if [ $((adb_forcedns)) -eq 1 ] && [ -n "${adb_landev4}" ] if [ "${adb_forcedns}" = "1" ] && [ -n "${adb_landev}" ]
then then
f_firewall "IPv4" "nat" "prerouting_rule" "prerouting_rule" "0" "dns" "-i ${adb_landev4} -p udp --dport 53 -j DNAT --to-destination ${adb_ipv4}:53" f_firewall "IPv4" "nat" "prerouting_rule" "adb-dns" "1" "dns" "-p udp --dport 53 -j DNAT --to-destination ${adb_ipv4}:53"
f_firewall "IPv4" "nat" "prerouting_rule" "prerouting_rule" "0" "dns" "-i ${adb_landev4} -p tcp --dport 53 -j DNAT --to-destination ${adb_ipv4}:53" f_firewall "IPv4" "nat" "prerouting_rule" "adb-dns" "2" "dns" "-p tcp --dport 53 -j DNAT --to-destination ${adb_ipv4}:53"
fi fi
f_firewall "IPv4" "nat" "prerouting_rule" "adb-nat" "1" "nat" "-p tcp --dport 80 -j DNAT --to-destination ${adb_ipv4}:${adb_nullport}"
f_firewall "IPv4" "nat" "prerouting_rule" "adb-nat" "2" "nat" "-p tcp --dport 443 -j DNAT --to-destination ${adb_ipv4}:${adb_nullportssl}"
f_firewall "IPv4" "filter" "forwarding_rule" "adb-fwd" "1" "fwd" "-p tcp -j REJECT --reject-with tcp-reset" f_firewall "IPv4" "filter" "forwarding_rule" "adb-fwd" "1" "fwd" "-p tcp -j REJECT --reject-with tcp-reset"
f_firewall "IPv4" "filter" "forwarding_rule" "adb-fwd" "2" "fwd" "-j REJECT --reject-with icmp-host-unreachable" f_firewall "IPv4" "filter" "forwarding_rule" "adb-fwd" "2" "fwd" "-j REJECT --reject-with icmp-host-unreachable"
f_firewall "IPv4" "filter" "output_rule" "adb-out" "1" "out" "-p tcp -j REJECT --reject-with tcp-reset" f_firewall "IPv4" "filter" "output_rule" "adb-out" "1" "out" "-p tcp -j REJECT --reject-with tcp-reset"
f_firewall "IPv4" "filter" "output_rule" "adb-out" "2" "out" "-j REJECT --reject-with icmp-host-unreachable" f_firewall "IPv4" "filter" "output_rule" "adb-out" "2" "out" "-j REJECT --reject-with icmp-host-unreachable"
fi fi
f_firewall "IPv4" "nat" "prerouting_rule" "adb-nat" "1" "nat" "-p tcp --dport 80 -j DNAT --to-destination ${adb_ipv4}:${adb_nullport}"
f_firewall "IPv4" "nat" "prerouting_rule" "adb-nat" "2" "nat" "-p tcp --dport 443 -j DNAT --to-destination ${adb_ipv4}:${adb_nullportssl}"
fi fi
if [ -n "${adb_wanif6}" ] if [ -n "${adb_wanif6}" ]
then then
check="$(${adb_iptv6} -w -vnL | grep -Fo "adb-")" if [ "${apmode_ok}" = "false" ]
if [ -z "${check}" ]
then then
if [ $((adb_forcedns)) -eq 1 ] && [ -n "${adb_landev6}" ] if [ "${adb_forcedns}" = "1" ] && [ -n "${adb_landev}" ]
then then
f_firewall "IPv6" "nat" "PREROUTING" "PREROUTING" "0" "dns" "-i ${adb_landev6} -p udp --dport 53 -j DNAT --to-destination [${adb_ipv6}]:53" f_firewall "IPv6" "nat" "PREROUTING" "adb-dns" "1" "dns" "-p udp --dport 53 -j DNAT --to-destination [${adb_ipv6}]:53"
f_firewall "IPv6" "nat" "PREROUTING" "PREROUTING" "0" "dns" "-i ${adb_landev6} -p tcp --dport 53 -j DNAT --to-destination [${adb_ipv6}]:53" f_firewall "IPv6" "nat" "PREROUTING" "adb-dns" "2" "dns" "-p tcp --dport 53 -j DNAT --to-destination [${adb_ipv6}]:53"
fi fi
f_firewall "IPv6" "nat" "PREROUTING" "adb-nat" "1" "nat" "-p tcp --dport 80 -j DNAT --to-destination [${adb_ipv6}]:${adb_nullport}"
f_firewall "IPv6" "nat" "PREROUTING" "adb-nat" "2" "nat" "-p tcp --dport 443 -j DNAT --to-destination [${adb_ipv6}]:${adb_nullportssl}"
f_firewall "IPv6" "filter" "forwarding_rule" "adb-fwd" "1" "fwd" "-p tcp -j REJECT --reject-with tcp-reset" f_firewall "IPv6" "filter" "forwarding_rule" "adb-fwd" "1" "fwd" "-p tcp -j REJECT --reject-with tcp-reset"
f_firewall "IPv6" "filter" "forwarding_rule" "adb-fwd" "2" "fwd" "-j REJECT --reject-with icmp6-addr-unreachable" f_firewall "IPv6" "filter" "forwarding_rule" "adb-fwd" "2" "fwd" "-j REJECT --reject-with icmp6-addr-unreachable"
f_firewall "IPv6" "filter" "output_rule" "adb-out" "1" "out" "-p tcp -j REJECT --reject-with tcp-reset" f_firewall "IPv6" "filter" "output_rule" "adb-out" "1" "out" "-p tcp -j REJECT --reject-with tcp-reset"
f_firewall "IPv6" "filter" "output_rule" "adb-out" "2" "out" "-j REJECT --reject-with icmp6-addr-unreachable" f_firewall "IPv6" "filter" "output_rule" "adb-out" "2" "out" "-j REJECT --reject-with icmp6-addr-unreachable"
fi fi
f_firewall "IPv6" "nat" "PREROUTING" "adb-nat" "1" "nat" "-p tcp --dport 80 -j DNAT --to-destination [${adb_ipv6}]:${adb_nullport}"
f_firewall "IPv6" "nat" "PREROUTING" "adb-nat" "2" "nat" "-p tcp --dport 443 -j DNAT --to-destination [${adb_ipv6}]:${adb_nullportssl}"
fi fi
if [ "${fw_done}" = "true" ] if [ "${firewall_ok}" = "true" ]
then then
f_log "created volatile firewall rulesets" f_log "created volatile firewall rulesets"
fw_done="false"
fi fi
# check volatile uhttpd instance configuration # check volatile uhttpd instance configuration
# #
check="$(pgrep -f "uhttpd -h /www/adblock")"
if [ -z "${check}" ]
then
if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ] if [ -n "${adb_wanif4}" ] && [ -n "${adb_wanif6}" ]
then then
f_uhttpd "adbIPv4+6_80" "1" "-p ${adb_ipv4}:${adb_nullport} -p [${adb_ipv6}]:${adb_nullport}" f_uhttpd "adbIPv4+6_80" "1" "-p ${adb_ipv4}:${adb_nullport} -p [${adb_ipv6}]:${adb_nullport}"
@ -410,11 +425,9 @@ f_envcheck()
f_uhttpd "adbIPv6_80" "1" "-p [${adb_ipv6}]:${adb_nullport}" f_uhttpd "adbIPv6_80" "1" "-p [${adb_ipv6}]:${adb_nullport}"
f_uhttpd "adbIPv6_443" "0" "-p [${adb_ipv6}]:${adb_nullportssl}" f_uhttpd "adbIPv6_443" "0" "-p [${adb_ipv6}]:${adb_nullportssl}"
fi fi
if [ "${uhttpd_done}" = "true" ] if [ "${uhttpd_ok}" = "true" ]
then then
f_log "created volatile uhttpd instances" f_log "created volatile uhttpd instances"
uhttpd_done="false"
fi
fi fi
# check whitelist entries # check whitelist entries
@ -424,7 +437,7 @@ f_envcheck()
awk "${adb_whitelist_rset}" "${adb_whitelist}" > "${adb_tmpdir}/tmp.whitelist" awk "${adb_whitelist_rset}" "${adb_whitelist}" > "${adb_tmpdir}/tmp.whitelist"
fi fi
# remove no longer used opkg package list # remove temporary package list
# #
unset pkg_list unset pkg_list
} }
@ -435,12 +448,17 @@ f_depend()
{ {
local check local check
local package="${1}" local package="${1}"
local check_only="${2}"
package_ok="true"
check="$(printf "${pkg_list}" | grep "^${package} -")" check="$(printf "${pkg_list}" | grep "^${package} -")"
if [ -z "${check}" ] if [ "${check_only}" = "true" ] && [ -z "${check}" ]
then then
rc=115 package_ok="false"
f_log "package '${package}' not found" "${rc}" elif [ -z "${check}" ]
then
rc=-1
f_log "package '${package}' not found"
f_exit f_exit
fi fi
} }
@ -449,7 +467,7 @@ f_depend()
# #
f_firewall() f_firewall()
{ {
local ipt="${adb_iptv4}" local ipt="iptables"
local nullip="${adb_nullipv4}" local nullip="${adb_nullipv4}"
local proto="${1}" local proto="${1}"
local table="${2}" local table="${2}"
@ -463,7 +481,7 @@ f_firewall()
# #
if [ "${proto}" = "IPv6" ] if [ "${proto}" = "IPv6" ]
then then
ipt="${adb_iptv6}" ipt="ip6tables"
nullip="${adb_nullipv6}" nullip="${adb_nullipv6}"
fi fi
@ -474,26 +492,32 @@ f_firewall()
then then
"${ipt}" -w -t "${table}" -N "${chain}" "${ipt}" -w -t "${table}" -N "${chain}"
"${ipt}" -w -t "${table}" -A "${chain}" -m comment --comment "${notes}" -j RETURN "${ipt}" -w -t "${table}" -A "${chain}" -m comment --comment "${notes}" -j RETURN
if [ "${chain}" = "adb-dns" ]
then
"${ipt}" -w -t "${table}" -A "${chsrc}" -i "${adb_landev}+" -m comment --comment "${notes}" -j "${chain}"
else
"${ipt}" -w -t "${table}" -A "${chsrc}" -d "${nullip}" -m comment --comment "${notes}" -j "${chain}" "${ipt}" -w -t "${table}" -A "${chsrc}" -d "${nullip}" -m comment --comment "${notes}" -j "${chain}"
fi fi
rc=${?}
if [ $((rc)) -ne 0 ]
then
f_log "failed to initialize volatile ${proto} firewall chain '${chain}'"
f_exit
fi
fi
# check whether iptables rule already exist # check whether iptables rule already exist
# #
rc="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1; printf ${?})" rc="$("${ipt}" -w -t "${table}" -C "${chain}" -m comment --comment "${notes}" ${rules} >/dev/null 2>&1; printf ${?})"
if [ $((rc)) -ne 0 ] if [ $((rc)) -ne 0 ]
then then
if [ $((chpos)) -eq 0 ]
then
"${ipt}" -w -t "${table}" -A "${chain}" -m comment --comment "${notes}" ${rules}
else
"${ipt}" -w -t "${table}" -I "${chain}" "${chpos}" -m comment --comment "${notes}" ${rules} "${ipt}" -w -t "${table}" -I "${chain}" "${chpos}" -m comment --comment "${notes}" ${rules}
fi
rc=${?} rc=${?}
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
fw_done="true" firewall_ok="true"
else else
f_log "failed to initialize volatile ${proto} firewall rule '${notes}'" "${rc}" f_log "failed to initialize volatile ${proto} firewall rule '${notes}'"
f_exit f_exit
fi fi
fi fi
@ -503,56 +527,32 @@ f_firewall()
# #
f_uhttpd() f_uhttpd()
{ {
local check
local realm="${1}" local realm="${1}"
local timeout="${2}" local timeout="${2}"
local ports="${3}" local ports="${3}"
"${adb_uhttpd}" -h "/www/adblock" -N 25 -T "${timeout}" -r "${realm}" -k 0 -t 0 -R -D -S -E "/index.html" ${ports}
check="$(pgrep -f "uhttpd -h /www/adblock -N 25 -T ${timeout} -r ${realm}")"
if [ -z "${check}" ]
then
uhttpd -h "/www/adblock" -N 25 -T "${timeout}" -r "${realm}" -k 0 -t 0 -R -D -S -E "/index.html" ${ports}
rc=${?} rc=${?}
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
uhttpd_done="true" uhttpd_ok="true"
else else
f_log "failed to initialize volatile uhttpd instance (${realm})" "${rc}" f_log "failed to initialize volatile uhttpd instance (${realm})"
f_exit f_exit
fi fi
}
# f_log: log messages to stdout and syslog
#
f_log()
{
local log_parm
local log_msg="${1}"
local log_rc="${2}"
local class="info "
# check for terminal session
#
if [ -t 1 ]
then
log_parm="-s"
fi
# log to different output devices and set log class accordingly
#
if [ -n "${log_msg}" ]
then
if [ $((log_rc)) -gt 0 ]
then
class="error"
log_rc=", rc: ${log_rc}"
log_msg="${log_msg}${log_rc}"
fi
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] ${class}" "${log_msg}" 2>&1
fi fi
} }
################################################
# f_space: check mount points/space requirements # f_space: check mount points/space requirements
# #
f_space() f_space()
{ {
local mp="${1}" local mp="${1}"
space_ok="true"
if [ -d "${mp}" ] if [ -d "${mp}" ]
then then
@ -570,7 +570,6 @@ f_cntconfig()
{ {
local src_name local src_name
local count=0 local count=0
local count_sum=0
for src_name in $(ls -ASr "${adb_dnsdir}/${adb_dnsprefix}"*) for src_name in $(ls -ASr "${adb_dnsdir}/${adb_dnsprefix}"*)
do do
@ -581,79 +580,181 @@ f_cntconfig()
count=$((count / 2)) count=$((count / 2))
fi fi
"${adb_uci}" -q set "adblock.${src_name}.adb_src_count=${count}" "${adb_uci}" -q set "adblock.${src_name}.adb_src_count=${count}"
count_sum=$((count_sum + count)) adb_count=$((adb_count + count))
done done
"${adb_uci}" -q set "adblock.global.adb_overall_count=${count_sum}" "${adb_uci}" -q set "adblock.global.adb_overall_count=${adb_count}"
} }
# f_rmconfig: remove counters & timestamps in given config sections # f_rmconfig: remove volatile config entries
# #
f_rmconfig() f_rmconfig()
{ {
local src_name local opt
local rm_done="${1}" local options="adb_src_timestamp adb_src_count"
local section="${1}"
for src_name in ${rm_done} "${adb_uci}" -q delete "adblock.global.adb_overall_count"
"${adb_uci}" -q delete "adblock.global.adb_dnstoggle"
"${adb_uci}" -q delete "adblock.global.adb_percentage"
"${adb_uci}" -q delete "adblock.global.adb_lastrun"
for opt in ${options}
do do
src_name="${src_name#*.}" "${adb_uci}" -q delete "adblock.${section}.${opt}"
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_timestamp"
done done
} }
# f_exit: delete (temporary) files, generate statistics and exit # f_rmdns: remove dns block lists and backups
# #
f_exit() f_rmdns()
{
rm_dns="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print -exec rm -f "{}" \;)"
if [ -n "${rm_dns}" ]
then
rm -rf "${adb_dnshidedir}"
if [ "${enabled_backup}" = "1" ] && [ -d "${adb_dir_backup}" ]
then
rm -f "${adb_dir_backup}/${adb_dnsprefix}"*.gz
fi
/etc/init.d/dnsmasq restart
fi
}
# f_rmuhttpd: remove uhttpd instances
#
f_rmuhttpd()
{
rm_uhttpd="$(pgrep -f "uhttpd -h /www/adblock")"
if [ -n "${rm_uhttpd}" ]
then
for pid in ${rm_uhttpd}
do
kill -9 "${pid}"
done
fi
}
# f_rmfirewall: remove firewall rulsets
#
f_rmfirewall()
{
rm_fw="$(iptables -w -t nat -vnL | grep -Fo "adb-")"
if [ -n "${rm_fw}" ]
then
iptables-save -t nat | grep -Fv -- "adb-" | iptables-restore
iptables-save -t filter | grep -Fv -- "adb-" | iptables-restore
if [ -n "$(lsmod | grep -Fo "ip6table_nat")" ]
then
ip6tables-save -t nat | grep -Fv -- "adb-" | ip6tables-restore
ip6tables-save -t filter | grep -Fv -- "adb-" | ip6tables-restore
fi
fi
}
# f_log: log messages to stdout and syslog
#
f_log()
{
local log_parm
local log_msg="${1}"
local class="info "
# check for terminal session
#
if [ -t 1 ]
then
log_parm="-s"
fi
# log to different output devices and set log class accordingly
#
if [ -n "${log_msg}" ]
then
if [ $((rc)) -gt 0 ]
then
class="error"
fi
logger ${log_parm} -t "adblock[${adb_pid}] ${class}" "${log_msg}" 2>&1
fi
}
# f_statistics: adblock runtime statistics
f_statistics()
{ {
local ipv4_blk=0 ipv4_all=0 ipv4_pct=0 local ipv4_blk=0 ipv4_all=0 ipv4_pct=0
local ipv6_blk=0 ipv6_all=0 ipv6_pct=0 local ipv6_blk=0 ipv6_all=0 ipv6_pct=0
local lastrun="$(${adb_date} "+%d.%m.%Y %H:%M:%S")"
# delete temporary files & directories if [ -n "${adb_wanif4}" ]
#
if [ -f "${adb_tmpfile}" ]
then then
rm -f "${adb_tmpfile}" ipv4_blk="$(iptables -t nat -vxnL adb-nat | awk '$3 ~ /^DNAT$/ {sum += $1} END {printf sum}')"
ipv4_all="$(iptables -t nat -vxnL PREROUTING | awk '$3 ~ /^(delegate_prerouting|prerouting_rule)$/ {sum += $1} END {printf sum}')"
if [ $((ipv4_all)) -gt 0 ] && [ $((ipv4_blk)) -gt 0 ] && [ $((ipv4_all)) -gt $((ipv4_blk)) ]
then
ipv4_pct="$(printf "${ipv4_blk}" | awk -v all="${ipv4_all}" '{printf( "%5.2f\n",$1/all*100)}')"
elif [ $((ipv4_all)) -lt $((ipv4_blk)) ]
then
iptables -t nat -Z adb-nat
fi fi
if [ -d "${adb_tmpdir}" ] fi
if [ -n "${adb_wanif6}" ]
then then
ipv6_blk="$(ip6tables -t nat -vxnL adb-nat | awk '$3 ~ /^DNAT$/ {sum += $1} END {printf sum}')"
ipv6_all="$(ip6tables -t nat -vxnL PREROUTING | awk '$3 ~ /^(adb-nat|DNAT)$/ {sum += $1} END {printf sum}')"
if [ $((ipv6_all)) -gt 0 ] && [ $((ipv6_blk)) -gt 0 ] && [ $((ipv6_all)) -gt $((ipv6_blk)) ]
then
ipv6_pct="$(printf "${ipv6_blk}" | awk -v all="${ipv6_all}" '{printf( "%5.2f\n",$1/all*100)}')"
elif [ $((ipv6_all)) -lt $((ipv6_blk)) ]
then
ip6tables -t nat -Z adb-nat
fi
fi
"${adb_uci}" -q set "adblock.global.adb_percentage=${ipv4_pct}%/${ipv6_pct}%"
f_log "firewall statistics (IPv4/IPv6): ${ipv4_pct}%/${ipv6_pct}% of all packets in prerouting chain are ad related & blocked"
}
# f_exit: delete temporary files, generate statistics and exit
#
f_exit()
{
local lastrun="$(date "+%d.%m.%Y %H:%M:%S")"
if [ "${adb_restricted}" = "1" ]
then
adb_uci="$(which true)"
fi
# delete temp files & directories
#
rm -f "${adb_tmpfile}"
rm -rf "${adb_tmpdir}" rm -rf "${adb_tmpdir}"
# tidy up on error
#
if [ $((rc)) -lt 0 ] || [ $((rc)) -gt 0 ]
then
f_rmdns
f_rmuhttpd
f_rmfirewall
config_foreach f_rmconfig source
if [ $((rc)) -eq -1 ]
then
"${adb_uci}" -q set "adblock.global.adb_lastrun=${lastrun} => runtime error, please check the log!"
fi
fi fi
# final log message and iptables statistics # final log message and iptables statistics
# #
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
if [ -n "${adb_wanif4}" ] f_statistics
then
ipv4_blk="$(${adb_iptv4} -t nat -vnL adb-nat | awk '$3 ~ /^DNAT$/ {sum += $1} END {printf sum}')"
ipv4_all="$(${adb_iptv4} -t nat -vnL PREROUTING | awk '$3 ~ /^prerouting_rule$/ {sum += $1} END {printf sum}')"
if [ $((ipv4_all)) -gt 0 ] && [ $((ipv4_blk)) -gt 0 ] && [ $((ipv4_all)) -gt $((ipv4_blk)) ]
then
ipv4_pct="$(printf "${ipv4_blk}" | awk -v all="${ipv4_all}" '{printf( "%5.2f\n",$1/all*100)}')"
fi
fi
if [ -n "${adb_wanif6}" ]
then
ipv6_blk="$(${adb_iptv6} -t nat -vnL adb-nat | awk '$3 ~ /^DNAT$/ {sum += $1} END {printf sum}')"
ipv6_all="$(${adb_iptv6} -t nat -vnL PREROUTING | awk '$3 ~ /^(adb-nat|DNAT)$/ {sum += $1} END {printf sum}')"
if [ $((ipv6_all)) -gt 0 ] && [ $((ipv6_blk)) -gt 0 ] && [ $((ipv6_all)) -gt $((ipv6_blk)) ]
then
ipv6_pct="$(printf "${ipv6_blk}" | awk -v all="${ipv6_all}" '{printf( "%5.2f\n",$1/all*100)}')"
fi
fi
"${adb_uci}" -q set "adblock.global.adb_percentage=${ipv4_pct}%/${ipv6_pct}%"
"${adb_uci}" -q set "adblock.global.adb_lastrun=${lastrun}" "${adb_uci}" -q set "adblock.global.adb_lastrun=${lastrun}"
"${adb_uci}" -q commit "adblock"
f_log "firewall statistics (IPv4/IPv6): ${ipv4_pct}%/${ipv6_pct}% of all packets in prerouting chain are ad related & blocked"
f_log "domain adblock processing finished successfully (${adb_scriptver}, ${adb_sysver}, ${lastrun})" f_log "domain adblock processing finished successfully (${adb_scriptver}, ${adb_sysver}, ${lastrun})"
elif [ $((rc)) -gt 0 ] elif [ $((rc)) -gt 0 ]
then then
"${adb_uci}" -q revert "adblock"
f_log "domain adblock processing failed (${adb_scriptver}, ${adb_sysver}, ${lastrun})" f_log "domain adblock processing failed (${adb_scriptver}, ${adb_sysver}, ${lastrun})"
else else
rc=0 rc=0
fi fi
"${adb_uci}" -q commit "adblock"
rm -f "${adb_pidfile}" rm -f "${adb_pidfile}"
exit ${rc} exit ${rc}
} }

222
net/adblock/files/adblock-update.sh
View file

@ -6,112 +6,103 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# set the C locale # prepare environment
#
LC_ALL=C
# set pid & logger
# #
adb_pid="${$}" adb_pid="${$}"
adb_pidfile="/var/run/adblock.pid" adb_pidfile="/var/run/adblock.pid"
adb_log="$(which logger)" adb_scriptver="1.4.4"
adb_mincfgver="2.3"
adb_scriptdir="${0%/*}"
if [ -r "${adb_pidfile}" ] if [ -r "${adb_pidfile}" ]
then then
rc=255 rc=255
"${adb_log}" -s -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile}))" logger -s -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile}))"
exit ${rc} exit ${rc}
else else
printf "${adb_pid}" > "${adb_pidfile}" printf "${adb_pid}" > "${adb_pidfile}"
fi
# get current directory and set script/config version
#
adb_scriptdir="${0%/*}"
adb_scriptver="1.2.1"
adb_mincfgver="2.2"
# source in adblock function library
#
if [ -r "${adb_scriptdir}/adblock-helper.sh" ] if [ -r "${adb_scriptdir}/adblock-helper.sh" ]
then then
. "${adb_scriptdir}/adblock-helper.sh" . "${adb_scriptdir}/adblock-helper.sh"
f_envload
else else
rc=254 rc=254
"${adb_log}" -s -t "adblock[${adb_pid}] error" "adblock function library not found" logger -s -t "adblock[${adb_pid}] error" "adblock function library not found"
rm -f "${adb_pidfile}" rm -f "${adb_pidfile}"
exit ${rc} exit ${rc}
fi fi
fi
# call trap function on error signals (HUP, INT, QUIT, BUS, SEGV, TERM) # call trap function on error signals (HUP, INT, QUIT, BUS, SEGV, TERM)
# #
trap "rc=250; f_log 'error signal received/trapped' '${rc}'; f_exit" 1 2 3 10 11 15 trap "rc=250; f_log 'error signal received/trapped'; f_exit" 1 2 3 10 11 15
# load environment
#
f_envload
# start logging
#
f_log "domain adblock processing started (${adb_scriptver}, ${adb_sysver}, $(/bin/date "+%d.%m.%Y %H:%M:%S"))"
# check environment # check environment
# #
f_envcheck f_envcheck
# loop through active adblock domain sources, # main loop for all block list sources
# download sources, prepare output and store all extracted domains in temp file
# #
for src_name in ${adb_sources} for src_name in ${adb_sources}
do do
# check disabled sources
#
eval "enabled=\"\${enabled_${src_name}}\""
if [ "${enabled}" = "0" ]
then
if [ -r "${adb_dnsdir}/${adb_dnsprefix}.${src_name}" ]
then
rm -f "${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
if [ "${backup_ok}" = "true" ] && [ -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ]
then
rm -f "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz"
fi
rm_done="true"
f_log "=> disabled source '${src_name}' removed"
fi
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_timestamp"
continue
fi
f_log "=> processing source '${src_name}'"
eval "url=\"\${adb_src_${src_name}}\"" eval "url=\"\${adb_src_${src_name}}\""
eval "src_rset=\"\${adb_src_rset_${src_name}}\"" eval "src_rset=\"\${adb_src_rset_${src_name}}\""
eval "list_time=\"\${CONFIG_${src_name}_adb_src_timestamp}\""
adb_dnsfile="${adb_dnsdir}/${adb_dnsprefix}.${src_name}" adb_dnsfile="${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
list_time="$(${adb_uci} -q get "adblock.${src_name}.adb_src_timestamp")"
f_log "=> processing adblock source '${src_name}'"
# check 'url' and 'src_rset' values # check 'url' and 'src_rset' values
# #
if [ -z "${url}" ] || [ -z "${src_rset}" ] if [ -z "${url}" ] || [ -z "${src_rset}" ]
then then
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=broken config" "${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=broken config"
f_log " broken source configuration, check 'adb_src' and 'adb_src_rset' in config" f_log " broken source configuration, skipped"
continue continue
fi fi
# prepare find statement with active adblock list sources
#
if [ -z "${adb_srclist}" ]
then
adb_srclist="! -name ${adb_dnsprefix}.${src_name}*"
else
adb_srclist="${adb_srclist} -a ! -name ${adb_dnsprefix}.${src_name}*"
fi
# download only block list with newer/updated timestamp # download only block list with newer/updated timestamp
# #
if [ "${src_name}" = "blacklist" ] if [ "${src_name}" = "blacklist" ]
then then
url_time="$(date -r "${url}")" url_time="$(date -r "${url}")"
else else
url_time="$(${adb_fetch} ${fetch_parm} --server-response --spider "${url}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}')" url_time="$(${adb_fetch} ${fetch_parm} ${response_parm} "${url}" 2>&1 | awk '$0 ~ /Last-Modified/ {printf substr($0,18)}')"
fi fi
if [ -z "${url_time}" ] if [ -z "${url_time}" ]
then then
url_time="$(date)" url_time="$(date)"
f_log " no online timestamp received" f_log " no online timestamp"
fi fi
if [ -z "${list_time}" ] || [ "${list_time}" != "${url_time}" ] || [ ! -r "${adb_dnsfile}" ] ||\ if [ -z "${list_time}" ] || [ "${list_time}" != "${url_time}" ] || [ ! -r "${adb_dnsfile}" ] ||\
([ "${backup_ok}" = "true" ] && [ ! -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ]) ([ "${backup_ok}" = "true" ] && [ ! -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ])
then then
if [ "${src_name}" = "blacklist" ] if [ "${src_name}" = "blacklist" ]
then then
tmp_domains="$(cat "${url}")" tmp_domains="$(cat "${url}" | strings -n 1)"
elif [ "${src_name}" = "shalla" ] elif [ "${src_name}" = "shalla" ]
then then
shalla_archive="${adb_tmpdir}/shallalist.tar.gz" shalla_archive="${adb_tmpdir}/shallalist.tar.gz"
shalla_file="${adb_tmpdir}/shallalist.txt" shalla_file="${adb_tmpdir}/shallalist.txt"
"${adb_fetch}" ${fetch_parm} --output-document="${shalla_archive}" "${url}" "${adb_fetch}" ${fetch_parm} -O "${shalla_archive}" "${url}"
rc=${?} rc=${?}
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
@ -126,17 +117,17 @@ do
break break
fi fi
done done
tmp_domains="$(cat "${shalla_file}")" tmp_domains="$(cat "${shalla_file}" | strings -n 1)"
rm -rf "${adb_tmpdir}/BL" rm -rf "${adb_tmpdir}/BL"
rm -f "${shalla_archive}" rm -f "${shalla_archive}"
rm -f "${shalla_file}" rm -f "${shalla_file}"
fi fi
else else
tmp_domains="$(${adb_fetch} ${fetch_parm} --output-document=- "${url}")" tmp_domains="$(${adb_fetch} ${fetch_parm} -O- "${url}" | strings -n 1)"
fi fi
rc=${?} rc=${?}
else else
f_log " source doesn't change, no update required" f_log " source doesn't change, skipped"
continue continue
fi fi
@ -154,9 +145,23 @@ do
unset tmp_domains unset tmp_domains
elif [ $((rc)) -eq 0 ] && [ -z "${tmp_domains}" ] elif [ $((rc)) -eq 0 ] && [ -z "${tmp_domains}" ]
then then
if [ "${backup_ok}" = "true" ] && [ -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ]
then
gunzip -cf "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" > "${adb_tmpfile}"
count="$(wc -l < "${adb_tmpfile}")"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=list restored"
f_log " empty source download, restored (${count} entries)"
else
if [ -r "${adb_dnsdir}/${adb_dnsprefix}.${src_name}" ]
then
rm -f "${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
rm_done="true"
fi
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=empty download" "${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=empty download"
f_log " empty source download finished" f_log " empty source download, skipped"
continue continue
fi
else else
rc=0 rc=0
if [ "${backup_ok}" = "true" ] && [ -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ] if [ "${backup_ok}" = "true" ] && [ -r "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" ]
@ -164,10 +169,16 @@ do
gunzip -cf "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" > "${adb_tmpfile}" gunzip -cf "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" > "${adb_tmpfile}"
count="$(wc -l < "${adb_tmpfile}")" count="$(wc -l < "${adb_tmpfile}")"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=list restored" "${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=list restored"
f_log " source download failed, list restored (${count} entries)" f_log " source download failed, restored (${count} entries)"
else else
if [ -r "${adb_dnsdir}/${adb_dnsprefix}.${src_name}" ]
then
rm -f "${adb_dnsdir}/${adb_dnsprefix}.${src_name}"
rm_done="true"
fi
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=download failed" "${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=download failed"
f_log " source download failed" f_log " source download failed, skipped"
continue continue
fi fi
fi fi
@ -184,17 +195,9 @@ do
sort -u "${adb_tmpfile}" | eval "${adb_dnsformat}" > "${adb_dnsfile}" sort -u "${adb_tmpfile}" | eval "${adb_dnsformat}" > "${adb_dnsfile}"
fi fi
rc=${?} rc=${?}
# finish domain processing, prepare find statement with revised adblock list source
#
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
if [ -z "${adb_revsrclist}" ] rev_done="true"
then
adb_revsrclist="-name ${adb_dnsprefix}.${src_name}"
else
adb_revsrclist="${adb_revsrclist} -o -name ${adb_dnsprefix}.${src_name}"
fi
f_log " domain merging finished" f_log " domain merging finished"
else else
rc=0 rc=0
@ -203,7 +206,9 @@ do
then then
rm -f "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" rm -f "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz"
fi fi
f_log " domain merging failed, list removed" "${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=domain merging failed"
f_log " domain merging failed, skipped"
continue continue
fi fi
else else
@ -212,99 +217,58 @@ do
then then
rm -f "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz" rm -f "${adb_dir_backup}/${adb_dnsprefix}.${src_name}.gz"
fi fi
"${adb_uci}" -q delete "adblock.${src_name}.adb_src_count"
"${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=empty domain input" "${adb_uci}" -q set "adblock.${src_name}.adb_src_timestamp=empty domain input"
f_log " empty domain input received, list removed" f_log " empty domain input, skipped"
continue continue
fi fi
done done
# remove disabled adblock lists and their backups # overall sort, make block list entries unique
# #
if [ -n "${adb_srclist}" ] if [ "${rev_done}" = "true" ] && [ "${mem_ok}" = "true" ]
then then
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_srclist} \) -print -exec rm -f "{}" \;)" f_log "remove duplicates in separate block lists"
if [ "${backup_ok}" = "true" ] && [ -n "${rm_done}" ]
then
find "${adb_dir_backup}" -maxdepth 1 -type f \( ${adb_srclist} \) -exec rm -f "{}" \;
fi
else
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print -exec rm -f "{}" \;)"
if [ "${backup_ok}" = "true" ]
then
find "${adb_dir_backup}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -exec rm -f "{}" \;
fi
fi
if [ -n "${rm_done}" ]
then
f_rmconfig "${rm_done}"
f_log "disabled adblock lists removed"
fi
# make separate adblock lists entries unique
#
if [ "${mem_ok}" = "true" ] && [ -n "${adb_revsrclist}" ]
then
f_log "remove duplicates in separate adblock lists"
# generate a unique overall block list
#
sort -u "${adb_dnsdir}/${adb_dnsprefix}."* > "${adb_tmpdir}/blocklist.overall"
# loop through all separate lists, ordered by size (ascending)
#
for list in $(ls -ASr "${adb_dnsdir}/${adb_dnsprefix}"*) for list in $(ls -ASr "${adb_dnsdir}/${adb_dnsprefix}"*)
do do
# check overall block list vs. separate block list,
# write all duplicate entries to separate list
#
list="${list/*./}" list="${list/*./}"
sort "${adb_tmpdir}/blocklist.overall" "${adb_dnsdir}/${adb_dnsprefix}.${list}" | uniq -d > "${adb_tmpdir}/tmp.${list}" if [ -s "${adb_tmpdir}/blocklist.overall" ]
mv -f "${adb_tmpdir}/tmp.${list}" "${adb_dnsdir}/${adb_dnsprefix}.${list}" then
sort "${adb_tmpdir}/blocklist.overall" "${adb_tmpdir}/blocklist.overall" "${adb_dnsdir}/${adb_dnsprefix}.${list}" | uniq -u > "${adb_tmpdir}/tmp.blocklist"
# write all unique entries back to overall block list cat "${adb_tmpdir}/tmp.blocklist" > "${adb_dnsdir}/${adb_dnsprefix}.${list}"
# fi
sort "${adb_tmpdir}/blocklist.overall" "${adb_dnsdir}/${adb_dnsprefix}.${list}" | uniq -u > "${adb_tmpdir}/tmp.overall" cat "${adb_dnsdir}/${adb_dnsprefix}.${list}" >> "${adb_tmpdir}/blocklist.overall"
mv -f "${adb_tmpdir}/tmp.overall" "${adb_tmpdir}/blocklist.overall"
done done
rm -f "${adb_tmpdir}/blocklist.overall"
fi fi
# restart & check dnsmasq with newly generated set of adblock lists # restart & check dnsmasq with generated set of block lists
# #
f_cntconfig if [ "${rev_done}" = "true" ] || [ "${rm_done}" = "true" ] || [ -n "${mv_done}" ]
adb_count="$(${adb_uci} -q get "adblock.global.adb_overall_count")"
if [ -n "${adb_revsrclist}" ] || [ -n "${rm_done}" ]
then then
"${adb_uci}" -q set "adblock.global.adb_dnstoggle=on" "${adb_uci}" -q delete "adblock.global.adb_dnstoggle"
/etc/init.d/dnsmasq restart
sleep 1
check="$(pgrep -f "dnsmasq")"
if [ -n "${check}" ]
then
f_log "adblock lists with overall ${adb_count} domains loaded"
else
f_log "dnsmasq restart failed, retry without newly generated block lists"
rm_done="$(find "${adb_dnsdir}" -maxdepth 1 -type f \( ${adb_revsrclist} \) -print -exec rm -f "{}" \;)"
if [ -n "${rm_done}" ]
then
f_log "bogus adblock lists removed"
f_rmconfig "${rm_done}"
/etc/init.d/dnsmasq restart /etc/init.d/dnsmasq restart
sleep 1 sleep 1
check="$(pgrep -f "dnsmasq")" check="$(pgrep -f "dnsmasq")"
if [ -n "${check}" ] if [ -n "${check}" ]
then then
f_cntconfig f_cntconfig
f_log "adblock lists with overall ${adb_count} domains loaded" f_log "block lists with overall ${adb_count} domains loaded"
else else
f_rmdns
sleep 1
check="$(pgrep -f "dnsmasq")"
if [ -n "${check}" ]
then
f_log "dnsmasq restart without block lists succeeded, please check your configuration"
else
f_log "dnsmasq restart without block lists failed, please check your configuration"
fi
rc=100 rc=100
f_log "dnsmasq restart failed, please check 'logread' output" "${rc}"
f_exit f_exit
fi fi
fi
fi
else else
f_log "adblock lists with overall ${adb_count} domains are still valid, no update required" f_cntconfig
f_log "block lists with overall ${adb_count} domains are still valid, no update required"
fi fi
# remove temporary files and exit # remove temporary files and exit

8
net/adblock/files/adblock.conf
View file

@ -3,7 +3,7 @@
config adblock 'global' config adblock 'global'
option adb_enabled '1' option adb_enabled '1'
option adb_cfgver '2.2' option adb_cfgver '2.3'
option adb_whitelist '/etc/adblock/adblock.whitelist' option adb_whitelist '/etc/adblock/adblock.whitelist'
option adb_whitelist_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}' option adb_whitelist_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}'
option adb_forcedns '1' option adb_forcedns '1'
@ -42,6 +42,12 @@ config source 'feodo'
option adb_src_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\$1)}' option adb_src_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\$1)}'
option adb_src_desc 'focus on feodo botnet domains, daily updates, approx. 0-10 entries' option adb_src_desc 'focus on feodo botnet domains, daily updates, approx. 0-10 entries'
config source 'hphosts'
option enabled '0'
option adb_src 'https://hosts-file.net/download/hosts.txt'
option adb_src_rset '\$0 ~/^127\.0\.0\.1[ \t]+([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\$2)}'
option adb_src_desc 'broad blocklist against ad-, tracking- and other malicious hosts, monthly updates, approx. 390.000 entries'
config source 'malware' config source 'malware'
option enabled '0' option enabled '0'
option adb_src 'https://mirror.cedia.org.ec/malwaredomains/justdomains' option adb_src 'https://mirror.cedia.org.ec/malwaredomains/justdomains'

9
net/adblock/files/adblock.hotplug
View file

@ -2,20 +2,19 @@
# #
adb_pid="${$}" adb_pid="${$}"
adb_helper="/usr/bin/adblock-helper.sh"
adb_pidfile="/var/run/adblock.pid" adb_pidfile="/var/run/adblock.pid"
adb_logger="/usr/bin/logger"
if [ -f "${adb_pidfile}" ] || [ "${ACTION}" != "ifup" ] if [ -f "${adb_pidfile}" ] || [ "${ACTION}" != "ifup" ]
then then
exit 0 exit 0
fi fi
. /lib/functions/network.sh . "${adb_helper}"
network_find_wan adb_wanif4 f_envload
network_find_wan6 adb_wanif6
if [ "${INTERFACE}" = "${adb_wanif4}" ] || [ "${INTERFACE}" = "${adb_wanif6}" ] if [ "${INTERFACE}" = "${adb_wanif4}" ] || [ "${INTERFACE}" = "${adb_wanif6}" ]
then then
/etc/init.d/adblock start /etc/init.d/adblock start
"${adb_logger}" -t "adblock[${adb_pid}] info " "adblock service started due to '${ACTION}' of '${INTERFACE}' interface" f_log "adblock service started due to '${ACTION}' of '${INTERFACE}' interface"
fi fi

142
net/adblock/files/adblock.init
View file

@ -2,48 +2,36 @@
# #
START=99 START=99
PATH="/usr/sbin:/usr/bin:/sbin:/bin" EXTRA_COMMANDS="toggle stats cfgup"
EXTRA_COMMANDS="toggle cfgup" EXTRA_HELP=" toggle Toggle adblock 'on' or 'off'
EXTRA_HELP=" toggle Toggle adblocking 'on' or 'off' stats Update adblock statistics
cfgup Update the adblock configuration file" cfgup Update adblock configuration file"
adb_debug=0 adb_debug=0
adb_pid="${$}"
adb_script="/usr/bin/adblock-update.sh"
adb_helper="/usr/bin/adblock-helper.sh"
adb_pidfile="/var/run/adblock.pid"
bg_parm="&"
if [ $((adb_debug)) -eq 0 ] if [ $((adb_debug)) -eq 0 ]
then then
exec 2>/dev/null exec 2>/dev/null
fi fi
adb_pid="${$}"
adb_script="/usr/bin/adblock-update.sh"
adb_dnsdir="/tmp/dnsmasq.d"
adb_dnshidedir="${adb_dnsdir}/.adb_hidden"
adb_dnsprefix="adb_list"
adb_pidfile="/var/run/adblock.pid"
adb_log="$(which logger)"
adb_uci="$(which uci)"
if [ -t 1 ]
then
log_parm="-s"
unset bg_parm
else
unset log_parm
bg_parm="&"
fi
if [ -r "${adb_pidfile}" ] if [ -r "${adb_pidfile}" ]
then then
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile}))" 2>&1 logger -s -t "adblock[${adb_pid}] error" "adblock service already running ($(cat ${adb_pidfile}))" 2>&1
exit 255 exit 255
fi fi
rm_config() . "${adb_helper}"
{ f_envload
local value opt section="${1}" options="adb_src_timestamp adb_src_count"
for opt in ${options} if [ "${adb_restricted}" = "1" ]
do then
"${adb_uci}" -q delete "adblock.${section}.${opt}" adb_uci="$(which true)"
done fi
}
boot() boot()
{ {
@ -52,103 +40,80 @@ boot()
start() start()
{ {
if [ -t 1 ]
then
unset bg_parm
fi
eval "${adb_script}" ${bg_parm} eval "${adb_script}" ${bg_parm}
return 0 return 0
} }
restart() restart()
{ {
restart="true"
stop stop
start start
} }
reload() reload()
{ {
restart="true" reload="true"
stop stop
start start
} }
stop() stop()
{ {
cfg_check="$(${adb_uci} -q get "adblock.global.adb_overall_count")" f_rmdns
if [ -n "${cfg_check}" ] && [ -z "${restart}" ] f_rmuhttpd
config_foreach f_rmconfig source
if [ -z "${reload}" ]
then then
. "/lib/functions.sh" f_rmfirewall
config_load adblock
config_foreach rm_config source
"${adb_uci}" -q delete "adblock.global.adb_dnstoggle"
"${adb_uci}" -q delete "adblock.global.adb_overall_count"
"${adb_uci}" -q delete "adblock.global.adb_percentage"
"${adb_uci}" -q delete "adblock.global.adb_lastrun"
"${adb_uci}" -q commit "adblock"
fi fi
fw_check="$(iptables -w -vnL | grep -Fo "adb-")" if [ -n "${rm_dns}" ] || [ -n "${rm_uhttpd}" ] || [ -n "${rm_fw}" ] || [ -n "$(${adb_uci} -q changes adblock)" ]
if [ -n "${fw_check}" ] && [ -z "${restart}" ]
then then
iptables-save -t nat | grep -Fv -- "adb-" | iptables-restore "${adb_uci}" -q commit adblock
iptables-save -t filter | grep -Fv -- "adb-" | iptables-restore f_log "all adblock related services stopped"
if [ -n "$(lsmod | grep -F "ip6table_nat")" ]
then
ip6tables-save -t nat | grep -Fv -- "adb-" | ip6tables-restore
ip6tables-save -t filter | grep -Fv -- "adb-" | ip6tables-restore
fi
fi
if [ -d "${adb_dnshidedir}" ]
then
find "${adb_dnshidedir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -exec mv -f "{}" "${adb_dnsdir}" \;
fi
dns_check="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print -exec rm -f "{}" \;)"
if [ -n "${dns_check}" ]
then
rm -rf "${adb_dnshidedir}"
/etc/init.d/dnsmasq restart
fi
www_check="$(pgrep -f "uhttpd -h /www/adblock")"
if [ -n "${www_check}" ]
then
for pid in ${www_check}
do
kill -9 "${pid}"
done
fi
if [ -n "${cfg_check}" ] || [ -n "${fw_check}" ] || [ -n "${dns_check}" ] || [ -n "${www_check}" ]
then
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] info " "all adblock related services stopped" 2>&1
fi fi
return 0 return 0
} }
toggle() toggle()
{ {
if [ "$(${adb_uci} -q get "adblock.global.adb_restricted")" = "1" ]
then
adb_uci="$(which true)"
fi
if [ -d "${adb_dnshidedir}" ] if [ -d "${adb_dnshidedir}" ]
then then
list_dns="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print)" list_dns="$(find "${adb_dnsdir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print)"
list_dnshide="$(find "${adb_dnshidedir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print)" list_dnshide="$(find "${adb_dnshidedir}" -maxdepth 1 -type f -name "${adb_dnsprefix}*" -print)"
if [ -n "${list_dns}" ] if [ -n "${list_dns}" ]
then then
mv -f "${adb_dnsdir}/${adb_dnsprefix}"* "${adb_dnshidedir}" source="${adb_dnsdir}/${adb_dnsprefix}"
"${adb_uci}" -q set "adblock.global.adb_dnstoggle=off" target="${adb_dnshidedir}"
"${adb_uci}" -q commit "adblock" pos="off"
/etc/init.d/dnsmasq restart
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] info " "toggle for adblocking switched 'off'" 2>&1
elif [ -n "${list_dnshide}" ] elif [ -n "${list_dnshide}" ]
then then
mv -f "${adb_dnshidedir}/${adb_dnsprefix}"* "${adb_dnsdir}" source="${adb_dnshidedir}/${adb_dnsprefix}"
"${adb_uci}" -q set "adblock.global.adb_dnstoggle=on" target="${adb_dnsdir}"
"${adb_uci}" -q commit "adblock" pos="on"
fi
if [ -n "${list_dns}" ] || [ -n "${list_dnshide}" ]
then
mv -f "${source}"* "${target}"
/etc/init.d/dnsmasq restart /etc/init.d/dnsmasq restart
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] info " "toggle for adblocking switched 'on'" 2>&1 "${adb_uci}" -q set "adblock.global.adb_dnstoggle=${pos}"
"${adb_uci}" -q commit "adblock"
f_log "adblock toggle switched '${pos}'"
fi fi
fi fi
return 0 return 0
} }
stats()
{
f_statistics
"${adb_uci}" -q commit "adblock"
return 0
}
cfgup() cfgup()
{ {
stop stop
@ -156,8 +121,9 @@ cfgup()
rc=$? rc=$?
if [ $((rc)) -eq 0 ] if [ $((rc)) -eq 0 ]
then then
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] info " "default adblock configuration applied, please check the settings in '/etc/config/adblock'" 2>&1 f_log "default adblock configuration applied, please check the settings in '/etc/config/adblock'"
else else
"${adb_log}" ${log_parm} -t "adblock[${adb_pid}] info " "default adblock configuration not found, please re-install the package via 'opkg install adblock --force-maintainer'" 2>&1 f_log "default adblock configuration not found, please re-install the package via 'opkg install adblock --force-maintainer'"
fi fi
return 0
} }