Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #4199 from dibdot/adblock

Browse source 
adblock: update 2.5.0
This commit is contained in:
Hannu Nyman 2017-03-26 14:14:23 +03:00 committed by GitHub
commit 3ce7ce72bd
5 changed files with 79 additions and 115 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
net/adblock/Makefile
View file

@ -6,7 +6,7 @@
include $(TOPDIR)/rules.mk include $(TOPDIR)/rules.mk
PKG_NAME:=adblock PKG_NAME:=adblock
PKG_VERSION:=2.4.0 PKG_VERSION:=2.5.0
PKG_RELEASE:=2 PKG_RELEASE:=2
PKG_LICENSE:=GPL-3.0+ PKG_LICENSE:=GPL-3.0+
PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org> PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org>

86
net/adblock/files/README.md
View file

@ -64,8 +64,8 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* additional whitelist for manual overrides, located by default in /etc/adblock/adblock.whitelist * additional whitelist for manual overrides, located by default in /etc/adblock/adblock.whitelist
* quality checks during block list update to ensure a reliable dns backend service * quality checks during block list update to ensure a reliable dns backend service
* minimal status & error logging to syslog, enable debug logging to receive more output * minimal status & error logging to syslog, enable debug logging to receive more output
* procd based init system support (start/stop/restart/reload/suspend/resume) * procd based init system support (start/stop/restart/reload/suspend/resume/query)
* procd based hotplug support, the adblock start will be triggered by interface triggers * procd based hotplug support, the adblock start will be solely triggered by network interface triggers
* suspend & resume adblock actions temporarily without block list reloading * suspend & resume adblock actions temporarily without block list reloading
* runtime statistics via ubus service call * runtime statistics via ubus service call
* query function to quickly identify blocked (sub-)domains, e.g. for whitelisting * query function to quickly identify blocked (sub-)domains, e.g. for whitelisting
@ -73,8 +73,7 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* optional: add new adblock sources on your own via uci config * optional: add new adblock sources on your own via uci config
## Prerequisites ## Prerequisites
* [openwrt](https://openwrt.org), tested with latest stable release (Chaos Calmer) and with current trunk (Designated Driver) * [LEDE project](https://www.lede-project.org), tested with latest stable release (LEDE 17.01) and with current LEDE snapshot
* [LEDE project](https://www.lede-project.org), tested with trunk > r98
* a usual setup with an enabled dns backend at minimum - dump AP modes without a working dns backend are _not_ supported * a usual setup with an enabled dns backend at minimum - dump AP modes without a working dns backend are _not_ supported
* a download utility: full versions (with ssl support) of 'wget', 'uclient-fetch', 'aria2c' or 'curl' are supported - the Chaos Calmer built-in busybox wget is not * a download utility: full versions (with ssl support) of 'wget', 'uclient-fetch', 'aria2c' or 'curl' are supported - the Chaos Calmer built-in busybox wget is not
* Chaos Calmer: download & install the external 'wget' package * Chaos Calmer: download & install the external 'wget' package
@ -87,19 +86,11 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* enable/disable your favored block list sources in _/etc/config/adblock_ - 'adaway', 'disconnect' and 'yoyo' are enabled by default * enable/disable your favored block list sources in _/etc/config/adblock_ - 'adaway', 'disconnect' and 'yoyo' are enabled by default
## LuCI adblock companion package ## LuCI adblock companion package
* for easy management of the various block list sources and options you can also use a nice & efficient LuCI frontend * for easy management of the various block list sources and all other adblock options you can also use a nice & efficient LuCI frontend
* install 'luci-app-adblock' (_opkg install luci-app-adblock_) * install 'luci-app-adblock' (_opkg install luci-app-adblock_)
* the application is located in LuCI under 'Services' menu * the application is located in LuCI under 'Services' menu
* _Thanks to Hannu Nyman for this great adblock LuCI frontend!_ * _Thanks to Hannu Nyman for this great adblock LuCI frontend!_
## Chaos Calmer installation notes
* 'adblock' and 'luci-app-adblock' are _not_ available as .ipk packages in the Chaos Calmer download repository
* download both packages from a development snapshot package directory:
* for 'adblock' look [here](https://downloads.lede-project.org/snapshots/packages/x86_64/packages/)
* for 'luci-app-adblock' look [here](https://downloads.lede-project.org/snapshots/packages/x86_64/luci/)
* manually transfer the packages to your routers temp directory (with tools like _sshfs_ or _winscp_)
* install the packages with _opkg install <...>_ as described above
## Tweaks ## Tweaks
* **status/runtime statistics:** the adblock status and runtime statistics are available via ubus service call (see example below) * **status/runtime statistics:** the adblock status and runtime statistics are available via ubus service call (see example below)
* **debug logging:** for script debugging please set the config option 'adb\_debug' to '1' and check the runtime output with _logread -e "adblock"_ * **debug logging:** for script debugging please set the config option 'adb\_debug' to '1' and check the runtime output with _logread -e "adblock"_
@ -118,10 +109,9 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* usually the pre-configured adblock setup works quite well and no manual config overrides are needed, all listed options apply to the 'global' config section: * usually the pre-configured adblock setup works quite well and no manual config overrides are needed, all listed options apply to the 'global' config section:
* adb\_enabled => main switch to enable/disable adblock service (default: '1', enabled) * adb\_enabled => main switch to enable/disable adblock service (default: '1', enabled)
* adb\_debug => enable/disable adblock debug output (default: '0', disabled) * adb\_debug => enable/disable adblock debug output (default: '0', disabled)
* adb\_iface => restrict the procd interface trigger to a (list of) certain wan interface(s) or disable it at all (default: not set, disabled) * adb\_iface => set the procd interface trigger to a (list of) lan/wan interface(s) (default: 'wan wwan lan')
* adb\_fetch => full path to a different download utility, see example below (default: not set, use wget) * adb\_fetch => full path to a different download utility, see example below (default: not set, use wget)
* adb\_fetchparm => options for the download utility, see example below (default: not set, use wget options) * adb\_fetchparm => options for the download utility, see example below (default: not set, use wget options)
* adb\_tldcomp => enable/disable tld compression (default: '1', enabled)
## Examples ## Examples
@ -155,50 +145,42 @@ curl:
**receive adblock statistics via ubus:** **receive adblock statistics via ubus:**
<pre><code> <pre><code>
ubus call service list '{"name":"adblock_stats"}' ubus call service get_data '{"name":"adblock"}
This will output the active block lists and other runtime information as JSON, e.g.: This will output the active block lists and other runtime information as JSON, e.g.:
{ {
"adblock_stats": { "adblock": {
"instances": { "adblock": {
"statistics": {
"running": false,
"command": [
""
],
"data": {
"active_lists": [ "active_lists": [
{ {
"palevo": "14", "palevo": "14",
"blacklist": "144", "blacklist": "147",
"winspy": "168", "winspy": "138",
"zeus": "422", "adaway": "378",
"adaway": "408", "zeus": "397",
"rolist": "649", "rolist": "652",
"malwarelist": "1219", "malwarelist": "1157",
"ransomware": "1495", "yoyo": "2272",
"ruadlist": "1791", "ruadlist": "1793",
"yoyo": "2304", "ransomware": "1670",
"openphish": "2139", "dshield": "190",
"dshield": "154", "openphish": "2672",
"disconnect": "3176", "disconnect": "2910",
"spam404": "6251", "spam404": "5715",
"adguard": "11081", "whocares": "8900",
"whocares": "11575", "winhelp": "8528",
"winhelp": "10574", "adguard": "10790",
"malware": "13854", "securemecca": "5080",
"sysctl": "8539", "sysctl": "7906",
"securemecca": "9262", "malware": "14617",
"shalla": "25358", "hphosts": "12450",
"hphosts": "36256" "shalla": "23926"
} }
], ],
"adblock_version": "2.3.0", "adblock_version": "2.5.0",
"blocked_domains": "146833", "blocked_domains": "112302",
"dns_backend": "dnsmasq", "dns_backend": "dnsmasq",
"last_rundate": "04.02.2017 21:10:31", "last_rundate": "25.03.2017 21:09:12",
"system": "LEDE Reboot SNAPSHOT r3286-c980147527" "system": "LEDE Reboot SNAPSHOT r3867-313197d707"
}
}
} }
} }
} }
@ -295,7 +277,7 @@ If your awk one-liner works quite well, add a new source section in adblock conf
</code></pre> </code></pre>
## Support ## Support
Please join the adblock discussion in this [forum thread](https://forum.openwrt.org/viewtopic.php?id=59803) or contact me by mail <dev@brenken.org> Please join the adblock discussion in this [forum thread](https://forum.lede-project.org/t/adblock-2-x-support-thread/507) or contact me by mail <dev@brenken.org>
## Removal ## Removal
* stop all adblock related services with _/etc/init.d/adblock stop_ * stop all adblock related services with _/etc/init.d/adblock stop_

1
net/adblock/files/adblock.conf
View file

@ -4,6 +4,7 @@
config adblock 'global' config adblock 'global'
option adb_enabled '1' option adb_enabled '1'
option adb_debug '0' option adb_debug '0'
option adb_iface 'wan wwan lan'
option adb_whitelist '/etc/adblock/adblock.whitelist' option adb_whitelist '/etc/adblock/adblock.whitelist'
option adb_whitelist_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}' option adb_whitelist_rset '\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}'
option adb_backup '0' option adb_backup '0'

40
net/adblock/files/adblock.init
View file

@ -9,32 +9,24 @@ EXTRA_HELP=" suspend Suspend adblock processing
resume Resume adblock processing resume Resume adblock processing
query <DOMAIN> Query active blocklists for specific domain" query <DOMAIN> Query active blocklists for specific domain"
adb_init="/etc/init.d/adblock"
adb_script="/usr/bin/adblock.sh" adb_script="/usr/bin/adblock.sh"
boot() boot()
{ {
local wanif4 wanif6 cnt=0 max_cnt=30 local list iface="$(uci -q get adblock.global.adb_iface)"
. "/lib/functions/network.sh" for name in ${iface}
while [ ${cnt} -le ${max_cnt} ]
do do
network_find_wan wanif4 list="${list} network.interface.${name}"
network_find_wan6 wanif6
if [ -n "${wanif4}" ] || [ -n "${wanif6}" ] || [ ${cnt} -eq ${max_cnt} ]
then
rc_procd start_service
return 0
else
sleep 1
cnt=$((cnt+1))
network_flush_cache
fi
done done
ubus -t 60 wait_for network.interface ${list}
rc_procd start_service
} }
start_service() start_service()
{ {
if [ $(/etc/init.d/adblock enabled; printf ${?}) -eq 0 ] if [ $("${adb_init}" enabled; printf ${?}) -eq 0 ]
then then
procd_open_instance "adblock" procd_open_instance "adblock"
procd_set_param command "${adb_script}" "${@}" procd_set_param command "${adb_script}" "${@}"
@ -44,11 +36,6 @@ start_service()
fi fi
} }
reload_service()
{
rc_procd start_service reload
}
stop_service() stop_service()
{ {
rc_procd "${adb_script}" stop rc_procd "${adb_script}" stop
@ -61,12 +48,12 @@ restart()
suspend() suspend()
{ {
rc_procd start_service suspend rc_procd "${adb_script}" suspend
} }
resume() resume()
{ {
rc_procd start_service resume rc_procd "${adb_script}" resume
} }
query() query()
@ -78,14 +65,9 @@ service_triggers()
{ {
local iface="$(uci -q get adblock.global.adb_iface)" local iface="$(uci -q get adblock.global.adb_iface)"
if [ -z "${iface}" ]
then
procd_add_raw_trigger "interface.*.up" 1000 /etc/init.d/adblock start
else
for name in ${iface} for name in ${iface}
do do
procd_add_interface_trigger "interface.*.up" "${name}" /etc/init.d/adblock start procd_add_interface_trigger "interface.*.up" "${name}" "${adb_init}" start
done done
fi procd_add_config_trigger "config.change" "adblock" "${adb_init}" start
procd_add_config_trigger "config.change" "adblock" /etc/init.d/adblock start
} }

49
net/adblock/files/adblock.sh
View file

@ -10,11 +10,11 @@
# #
LC_ALL=C LC_ALL=C
PATH="/usr/sbin:/usr/bin:/sbin:/bin" PATH="/usr/sbin:/usr/bin:/sbin:/bin"
adb_ver="2.4.0-2" adb_ver="2.5.0"
adb_sysver="$(ubus -S call system board | jsonfilter -e '@.release.description')"
adb_enabled=1 adb_enabled=1
adb_debug=0 adb_debug=0
adb_backup=0 adb_backup=0
adb_tldcomp=1
adb_backupdir="/mnt" adb_backupdir="/mnt"
adb_whitelist="/etc/adblock/adblock.whitelist" adb_whitelist="/etc/adblock/adblock.whitelist"
adb_whitelist_rset="\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}" adb_whitelist_rset="\$1 ~/^([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\"^\"\$1\"\\\|[.]\"\$1)}"
@ -40,7 +40,7 @@ f_envload()
# set dns backend environment # set dns backend environment
# #
while [ ${cnt} -le 10 ] while [ ${cnt} -le 20 ]
do do
for dns in ${adb_dnslist} for dns in ${adb_dnslist}
do do
@ -188,7 +188,6 @@ f_rmdns()
rm -f "${adb_backupdir}/${adb_dnsprefix}"*.gz rm -f "${adb_backupdir}/${adb_dnsprefix}"*.gz
rm -rf "${adb_dnshidedir}" rm -rf "${adb_dnshidedir}"
fi fi
ubus call service delete "{\"name\":\"adblock_stats\",\"instances\":\"statistics\"}" 2>/dev/null
} }
# f_dnsrestart: restart the dns backend # f_dnsrestart: restart the dns backend
@ -203,7 +202,7 @@ f_dnsrestart()
adb_dnsup="$(ubus -S call service list "{\"name\":\"${adb_dns}\"}" | jsonfilter -l1 -e "@.${adb_dns}.instances.*.running")" adb_dnsup="$(ubus -S call service list "{\"name\":\"${adb_dns}\"}" | jsonfilter -l1 -e "@.${adb_dns}.instances.*.running")"
if [ "${adb_dnsup}" = "true" ] if [ "${adb_dnsup}" = "true" ]
then then
return 0 break
fi fi
cnt=$((cnt+1)) cnt=$((cnt+1))
sleep 1 sleep 1
@ -323,7 +322,7 @@ f_log()
logger -t "adblock-[${adb_ver}] ${class}" "${log_msg}" logger -t "adblock-[${adb_ver}] ${class}" "${log_msg}"
if [ "${class}" = "error" ] if [ "${class}" = "error" ]
then then
logger -t "adblock-[${adb_ver}] ${class}" "Please check the online documentation 'https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md'" logger -t "adblock-[${adb_ver}] ${class}" "Please check 'https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md' (${adb_sysver})"
f_rmtemp f_rmtemp
if [ "$(ls -dA "${adb_dnsdir}/${adb_dnsprefix}"* >/dev/null 2>&1)" ] if [ "$(ls -dA "${adb_dnsdir}/${adb_dnsprefix}"* >/dev/null 2>&1)" ]
then then
@ -340,8 +339,7 @@ f_log()
f_main() f_main()
{ {
local enabled url cnt sum_cnt=0 mem_total=0 local enabled url cnt sum_cnt=0 mem_total=0
local src_name src_rset shalla_archive list active_lists local src_name src_rset shalla_archive list active_lists active_triggers
local sysver="$(ubus -S call system board | jsonfilter -e '@.release.description')"
mem_total="$(awk '$1 ~ /^MemTotal/ {printf $2}' "/proc/meminfo" 2>/dev/null)" mem_total="$(awk '$1 ~ /^MemTotal/ {printf $2}' "/proc/meminfo" 2>/dev/null)"
f_log "info " "start adblock processing ..." f_log "info " "start adblock processing ..."
@ -398,17 +396,12 @@ f_main()
# #
if [ ${adb_rc} -eq 0 ] && [ -s "${adb_tmpload}" ] if [ ${adb_rc} -eq 0 ] && [ -s "${adb_tmpload}" ]
then then
awk "${src_rset}" "${adb_tmpload}" > "${adb_tmpfile}" awk "${src_rset}" "${adb_tmpload}" 2>/dev/null > "${adb_tmpfile}"
if [ -s "${adb_tmpfile}" ] if [ -s "${adb_tmpfile}" ]
then then
if [ ${adb_tldcomp} -eq 1 ] awk -F "." '{for(f=NF;f > 1;f--) printf "%s.", $f;print $1}' "${adb_tmpfile}" 2>/dev/null | sort -u > "${adb_tmpload}"
then awk '{if(NR==1){tld=$NF};while(getline){if($NF !~ tld"\\."){print tld;tld=$NF}}print tld}' "${adb_tmpload}" 2>/dev/null > "${adb_tmpfile}"
awk -F "." '{for(f=NF;f > 1;f--) printf "%s.", $f;print $1}' "${adb_tmpfile}" | sort -u > "${adb_tmpload}" awk -F "." '{for(f=NF;f > 1;f--) printf "%s.", $f;print $1}' "${adb_tmpfile}" 2>/dev/null > "${adb_tmpload}"
awk '{if(NR==1){tld=$NF};while(getline){if($NF !~ tld"\\."){print tld;tld=$NF}}print tld}' "${adb_tmpload}" > "${adb_tmpfile}"
awk -F "." '{for(f=NF;f > 1;f--) printf "%s.", $f;print $1}' "${adb_tmpfile}" > "${adb_tmpload}"
else
sort -u "${adb_tmpfile}" > "${adb_tmpload}"
fi
mv -f "${adb_tmpload}" "${adb_tmpfile}" mv -f "${adb_tmpload}" "${adb_tmpfile}"
f_list backup f_list backup
else else
@ -424,9 +417,9 @@ f_main()
then then
if [ -s "${adb_tmpdir}/tmp.whitelist" ] if [ -s "${adb_tmpdir}/tmp.whitelist" ]
then then
grep -vf "${adb_tmpdir}/tmp.whitelist" "${adb_tmpfile}" | eval "${adb_dnsformat}" > "${adb_dnsfile}" grep -vf "${adb_tmpdir}/tmp.whitelist" "${adb_tmpfile}" 2>/dev/null | eval "${adb_dnsformat}" > "${adb_dnsfile}"
else else
cat "${adb_tmpfile}" | eval "${adb_dnsformat}" > "${adb_dnsfile}" cat "${adb_tmpfile}" 2>/dev/null | eval "${adb_dnsformat}" > "${adb_dnsfile}"
fi fi
adb_rc=${?} adb_rc=${?}
if [ ${adb_rc} -ne 0 ] if [ ${adb_rc} -ne 0 ]
@ -470,18 +463,24 @@ f_main()
f_dnsrestart f_dnsrestart
if [ "${adb_dnsup}" = "true" ] if [ "${adb_dnsup}" = "true" ]
then then
f_log "info " "block lists with overall ${sum_cnt} domains loaded successfully (${sysver})" f_log "info " "block lists with overall ${sum_cnt} domains loaded successfully (${adb_sysver})"
ubus call service set "{\"name\":\"adblock_stats\", for name in ${adb_iface}
\"instances\":{\"statistics\":{\"command\":[\"\"], do
active_triggers="${active_triggers}[\"interface.*.up\",[\"if\",[\"eq\",\"interface\",\"${name}\"],[\"run_script\",\"/etc/init.d/adblock\",\"start\"],1000]],"
done
active_triggers="${active_triggers}[\"config.change\",[\"if\",[\"eq\",\"package\",\"adblock\"],[\"run_script\",\"/etc/init.d/adblock\",\"start\"],1000]]"
ubus call service set "{\"name\":\"adblock\",
\"instances\":{\"adblock\":{\"command\":[\"/usr/bin/adblock.sh\"],
\"data\":{\"active_lists\":[{${active_lists}}], \"data\":{\"active_lists\":[{${active_lists}}],
\"adblock_version\":\"${adb_ver}\", \"adblock_version\":\"${adb_ver}\",
\"blocked_domains\":\"${sum_cnt}\", \"blocked_domains\":\"${sum_cnt}\",
\"dns_backend\":\"${adb_dns}\", \"dns_backend\":\"${adb_dns}\",
\"last_rundate\":\"$(/bin/date "+%d.%m.%Y %H:%M:%S")\", \"last_rundate\":\"$(/bin/date "+%d.%m.%Y %H:%M:%S")\",
\"system\":\"${sysver}\"}}}}" \"system\":\"${adb_sysver}\"}}},
return 0 \"triggers\":[${active_triggers}]}"
else
f_log "error" "dns backend restart with active block lists failed"
fi fi
f_log "error" "dns backend restart with active block lists failed (${sysver})"
} }
# handle different adblock actions # handle different adblock actions