Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

simple-adblock: dnsmasq.ipset option support, better handling of IDNs, updated README

Browse source 
Signed-off-by: Stan Grishin <stangri@melmac.net>
This commit is contained in:
Stan Grishin 2019-09-24 09:11:45 -07:00
parent 1f81686c0e
commit c6beee74f9
4 changed files with 420 additions and 229 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
net/simple-adblock/Makefile
View file

@ -5,8 +5,8 @@
include $(TOPDIR)/rules.mk include $(TOPDIR)/rules.mk
PKG_NAME:=simple-adblock PKG_NAME:=simple-adblock
PKG_VERSION:=1.8.0 PKG_VERSION:=1.8.1
PKG_RELEASE:=4 PKG_RELEASE:=7
PKG_MAINTAINER:=Stan Grishin <stangri@melmac.net> PKG_MAINTAINER:=Stan Grishin <stangri@melmac.net>
PKG_LICENSE:=GPL-3.0-or-later PKG_LICENSE:=GPL-3.0-or-later

40
net/simple-adblock/files/README.md
View file

@ -5,24 +5,39 @@ A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
## Features ## Features
- Super-fast due to the nature of supported block lists and parallel downloading/processing of the blacklists. - Super-fast due to the nature of supported block lists and parallel downloading/processing of the blacklists.
- Supports both hosts files and domains lists for blocking (to keep it lean and fast). - Supports hosts files and domains lists for blocking.
- Everything is configurable from Web UI. - Everything is configurable from Web UI.
- Allows you to easily add your own domains to whitelist or blacklist. - Allows you to easily add your own domains to whitelist or blacklist.
- Allows you to easily add URLs to your own blocked hosts or domains lists to block/whitelist (just put whitelisted domains one per line). - Allows you to easily add URLs to your own blocked hosts or domains lists to block/whitelist (just put whitelisted domains one per line in the file you're linking).
- Requires no configuration for the download utility wherever you want to use wget/libopenssl or uclient-fetch/libustream-mbedtls. - Supports multiple modes of AdBlocking implementations with DNSMASQ and Unbound.
- Installs dependencies automatically. - Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 (depending on settings) reply for blocked domains.
- As some of the default lists are using https, reliably works with either wget/libopenssl, uclient-fetch/libustream-mbedtls or curl. - As some of the default lists are using https, reliably works with either wget/libopenssl, uclient-fetch/libustream-mbedtls or curl.
- Very lightweight and easily hackable, the whole script is just one ```/etc/init.d/simple-adblock``` file. - Very lightweight and easily hackable, the whole script is just one ```/etc/init.d/simple-adblock``` file.
- Retains the downloaded/sorted AdBlocking list on service stop and reuses it on service start (use ```dl``` command if you want to force re-download of the list). - Retains the downloaded/sorted AdBlocking list on service stop and reuses it on service start (use ```dl``` command if you want to force re-download of the list).
- Has an option to store a compressed copy of the AdBlocking list in persistent memory which survives reboots.
- Blocks ads served over https (unlike PixelServ-derived solutions). - Blocks ads served over https (unlike PixelServ-derived solutions).
- Blocks ads inside browsers with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html) -- with the ```dnsmasq.ipset``` option.
- Proudly made in Canada, using locally-sourced electrons. - Proudly made in Canada, using locally-sourced electrons.
If you want a more robust AdBlocking, supporting free memory detection and complex block lists, supporting IDN, check out [net/adblock](https://github.com/openwrt/packages/tree/master/net/adblock/files). If you want a more robust AdBlocking, supporting free memory detection and complex block lists, supporting IDN, check out [net/adblock](https://github.com/openwrt/packages/tree/master/net/adblock/files).
## Screenshot (luci-app-simple-adblock) ## Screenshots (luci-app-simple-adblock)
![screenshot](https://raw.githubusercontent.com/stangri/openwrt_packages/master/screenshots/simple-adblock/screenshot07.png "screenshot") Service Status
![screenshot](https://raw.githubusercontent.com/stangri/openwrt_packages/master/screenshots/simple-adblock/screenshot08-status.png "Service Status")
Configuration - Basic Configuration
![screenshot](https://raw.githubusercontent.com/stangri/openwrt_packages/master/screenshots/simple-adblock/screenshot08-config-basic.png "Configuration - Basic Configuration")
Configuration - Advanced Configuration
![screenshot](https://raw.githubusercontent.com/stangri/openwrt_packages/master/screenshots/simple-adblock/screenshot08-config-advanced.png "Configuration - Advanced Configuration")
Whitelist and Blocklist Management
![screenshot](https://raw.githubusercontent.com/stangri/openwrt_packages/master/screenshots/simple-adblock/screenshot08-lists.png "Whitelist and Blocklist Management")
## Requirements ## Requirements
@ -30,6 +45,14 @@ This service requires the following packages to be installed on your router: ```
To satisfy the requirements for connect to your router via ssh and run the following commands: To satisfy the requirements for connect to your router via ssh and run the following commands:
### How to use DNSMASQ ipset
The ```dnsmasq.ipset``` option requires you to install ```ipset``` and ```dnsmasq-full``` instead of the ```dnsmasq```. To do that, connect to your router via ssh and run the following command:
```sh
opkg update; opkg remove dnsmasq; opkg install dnsmasq-full ipset;
```
### OpenWrt 15.05.1 Requirements ### OpenWrt 15.05.1 Requirements
```sh ```sh
@ -126,6 +149,7 @@ Currently supported options are:
| --- | --- | | --- | --- |
|```dnsmasq.addnhosts```|Creates the DNSMASQ additional hosts file ```/var/run/simple-adblock.addnhosts``` and modifies DNSMASQ settings, so that DNSMASQ resolves all blocked domains to "local machine": 127.0.0.1. This option doesn't allow block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), so it results in a much larger block-list file, but, unlike other DNSMASQ-based options, it has almost no effect on the DNS look up speed. This option also allows quick reloads of DNSMASQ on block-list updates.| |```dnsmasq.addnhosts```|Creates the DNSMASQ additional hosts file ```/var/run/simple-adblock.addnhosts``` and modifies DNSMASQ settings, so that DNSMASQ resolves all blocked domains to "local machine": 127.0.0.1. This option doesn't allow block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), so it results in a much larger block-list file, but, unlike other DNSMASQ-based options, it has almost no effect on the DNS look up speed. This option also allows quick reloads of DNSMASQ on block-list updates.|
|```dnsmasq.conf```|Creates the DNSMASQ config file ```/var/dnsmasq.d/simple-adblock``` so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat.| |```dnsmasq.conf```|Creates the DNSMASQ config file ```/var/dnsmasq.d/simple-adblock``` so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat.|
|```dnsmasq.ipset```|Creates the DNSMASQ ipset file ```/var/dnsmasq.d/simple-adblock.ipset``` and the firewall rule to reject the matching requests. This is the only option for AdBlocking if you're using a browser with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html). This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option requires you install ```dnsmasq-full``` and ```ipset``` [as described here](#how-to-use-dnsmasq-ipset).<br/>PLEASE NOTE, that unlike other options which are truly domain name based blocking, this is essentially an IP address based blocking, ie: if you try to block ```google-analytics.com``` with this option, it may also block/break things like YouTube, Hangouts and other Google services if they share IP address(es) with ```google-analytics.com```.|
|```dnsmasq.servers```|Creates the DNSMASQ servers file ```/var/run/simple-adblock.servers``` and modifies DNSMASQ settings so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat. This is a default setting as it results in the smaller block-file and allows quick reloads of DNSMASQ.| |```dnsmasq.servers```|Creates the DNSMASQ servers file ```/var/run/simple-adblock.servers``` and modifies DNSMASQ settings so that DNSMASQ replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file. This option will slow down DNS look up speed somewhat. This is a default setting as it results in the smaller block-file and allows quick reloads of DNSMASQ.|
|```unbound.adb_list```|Creates the Unbound config file ```/var/lib/unbound/adb_list.simple-adblock``` so that Unbound replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file.| |```unbound.adb_list```|Creates the Unbound config file ```/var/lib/unbound/adb_list.simple-adblock``` so that Unbound replies with NXDOMAIN: "domain not found". This option allows the block-list optimization (by removing secondary level domains if the top-level domain is also in the block-list), resulting in the smaller block-list file.|
@ -143,4 +167,4 @@ Please head to [OpenWrt Forum](https://forum.openwrt.org/t/simple-adblock-fast-l
## Thanks ## Thanks
I'd like to thank everyone who helped create, test and troubleshoot this service. Special thanks to [@hnyman](https://github.com/hnyman) for general package/luci guidance, [@dibdot](https://github.com/dibdot) for general guidance and block-list optimization code, [@ckuethe](https://github.com/ckuethe) for the curl support, non-ASCII filtering and compressed cache code, [@EricLuehrsen](https://github.com/EricLuehrsen) for the Unbound support information and [@phasecat](https://forum.openwrt.org/u/phasecat/summary) for submitting bugs and testing. I'd like to thank everyone who helped create, test and troubleshoot this service. Special thanks to [@hnyman](https://github.com/hnyman) for general package/luci guidance, [@dibdot](https://github.com/dibdot) for general guidance and block-list optimization code, [@ckuethe](https://github.com/ckuethe) for the curl support, non-ASCII filtering and compressed cache code, [@EricLuehrsen](https://github.com/EricLuehrsen) for the Unbound support information, [@mushoz]( https://github.com/mushoz) for performance testing and [@phasecat](https://forum.openwrt.org/u/phasecat/summary) for submitting various bugs and testing.

2
net/simple-adblock/files/simple-adblock.conf
View file

@ -1,6 +1,7 @@
config simple-adblock 'config' config simple-adblock 'config'
option enabled '0' option enabled '0'
option dns 'dnsmasq.servers' option dns 'dnsmasq.servers'
option dns_instance '0'
option verbosity '2' option verbosity '2'
option force_dns '1' option force_dns '1'
option led 'none' option led 'none'
@ -9,7 +10,6 @@ config simple-adblock 'config'
option curl_retry '3' option curl_retry '3'
option parallel_downloads '1' option parallel_downloads '1'
option debug '0' option debug '0'
option allow_non_ascii '0'
option compressed_cache '0' option compressed_cache '0'
list whitelist_domain 'raw.githubusercontent.com' list whitelist_domain 'raw.githubusercontent.com'
# list blacklist_hosts_url 'http://support.it-mate.co.uk/downloads/hosts.txt' # list blacklist_hosts_url 'http://support.it-mate.co.uk/downloads/hosts.txt'

567
net/simple-adblock/files/simple-adblock.init
View file

@ -8,12 +8,12 @@ export START=94
export USE_PROCD=1 export USE_PROCD=1
export LC_ALL=C export LC_ALL=C
export EXTRA_COMMANDS="check dl killcache status" export EXTRA_COMMANDS='check dl killcache status'
export EXTRA_HELP=" check Checks if specified domain is found in current blacklist export EXTRA_HELP=' check Checks if specified domain is found in current blacklist
dl Force-redownloads all the list dl Force-redownloads all the list
status Shows the service last-run status" status Shows the service last-run status'
readonly packageName="simple-adblock" readonly packageName='simple-adblock'
readonly serviceName="$packageName $PKG_VERSION" readonly serviceName="$packageName $PKG_VERSION"
readonly addnhostsFile="/var/run/${packageName}.addnhosts" readonly addnhostsFile="/var/run/${packageName}.addnhosts"
readonly addnhostsCache="/var/run/${packageName}.addnhosts.cache" readonly addnhostsCache="/var/run/${packageName}.addnhosts.cache"
@ -24,6 +24,10 @@ readonly dnsmasqFile="/var/dnsmasq.d/${packageName}"
readonly dnsmasqCache="/var/run/${packageName}.dnsmasq.cache" readonly dnsmasqCache="/var/run/${packageName}.dnsmasq.cache"
readonly dnsmasqGzip="/etc/${packageName}.dnsmasq.gz" readonly dnsmasqGzip="/etc/${packageName}.dnsmasq.gz"
readonly dnsmasqOutputFilter='s|^|local=/|;s|$|/|' readonly dnsmasqOutputFilter='s|^|local=/|;s|$|/|'
readonly ipsetFile="/var/dnsmasq.d/${packageName}.ipset"
readonly ipsetCache="/var/run/${packageName}.ipset.cache"
readonly ipsetGzip="/etc/${packageName}.ipset.gz"
readonly ipsetOutputFilter='s|^|ipset=/|;s|$|/adb|'
readonly serversFile="/var/run/${packageName}.servers" readonly serversFile="/var/run/${packageName}.servers"
readonly serversCache="/var/run/${packageName}.servers.cache" readonly serversCache="/var/run/${packageName}.servers.cache"
readonly serversGzip="/etc/${packageName}.servers.gz" readonly serversGzip="/etc/${packageName}.servers.gz"
@ -55,17 +59,26 @@ readonly statusForceReloading='Force-Reloading'
readonly statusProcessing='Processing' readonly statusProcessing='Processing'
readonly statusStopped='Stopped' readonly statusStopped='Stopped'
readonly sharedMemoryError="/dev/shm/$packageName-error" readonly sharedMemoryError="/dev/shm/$packageName-error"
readonly sharedMemoryOutput="/dev/shm/$packageName-output"
create_lock() { [ -e "$PIDFile" ] && return 1; touch "$PIDFile"; } create_lock() { [ -e "$PIDFile" ] && return 1; touch "$PIDFile"; }
remove_lock() { [ -e "$PIDFile" ] && rm -f "$PIDFile"; } remove_lock() { [ -e "$PIDFile" ] && rm -f "$PIDFile"; }
trap remove_lock EXIT trap remove_lock EXIT
output_ok() { output 1 "$_OK_"; output 2 "$__OK__\\n"; } output_ok() { output 1 "$_OK_"; output 2 "$__OK__\\n"; }
output_okn() { output 1 "$_OK_\\n"; output 2 "$__OK__\\n"; } output_okn() { output 1 "$_OK_\\n"; output 2 "$__OK__\\n"; }
output_fail() { output 1 "$_FAIL_\\n"; output 2 "$__FAIL__\\n"; } output_fail() { output 1 "$_FAIL_"; output 2 "$__FAIL__\\n"; }
output_failn() { output 1 "$_FAIL_"; output 2 "$__FAIL__\\n"; } output_failn() { output 1 "$_FAIL_\\n"; output 2 "$__FAIL__\\n"; }
str_replace() { echo "$1" | sed -e "s/$2/$3/g"; } str_replace() { echo "$1" | sed -e "s/$2/$3/g"; }
str_contains() { [ "$1" != "$(str_replace "$1" "$2" "")" ]; } str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
readonly sharedMemoryOutput="/dev/shm/$packageName-output" compare_versions() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
is_chaos_calmer() { ubus -S call system board | grep -q 'Chaos Calmer'; }
is_ipset_procd() { compare_versions "$(sed -ne 's/^Version: //p' /usr/lib/opkg/info/firewall.control)" "2019-09-18"; }
led_on(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo 'default-on' > "${1}/trigger" 2>&1; fi; }
led_off(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo 'none' > "${1}/trigger" 2>&1; fi; }
dnsmasq_kill() { killall -q -HUP dnsmasq; }
dnsmasq_restart() { /etc/init.d/dnsmasq restart >/dev/null 2>&1; }
unbound_restart() { /etc/init.d/unbound restart >/dev/null 2>&1; }
output() { output() {
# Can take a single parameter (text) to be output at any verbosity # Can take a single parameter (text) to be output at any verbosity
# Or target verbosity level and text to be output at specifc verbosity # Or target verbosity level and text to be output at specifc verbosity
@ -83,53 +96,39 @@ output() {
printf "%b" "$msg" >> "$sharedMemoryOutput" printf "%b" "$msg" >> "$sharedMemoryOutput"
fi fi
} }
led_on(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo "default-on" > "${1}/trigger" 2>&1; fi; }
led_off(){ if [ -n "${1}" ] && [ -e "${1}/trigger" ]; then echo "none" > "${1}/trigger" 2>&1; fi; }
export serviceEnabled export serviceEnabled forceDNS parallelDL debug allowIDN compressedCache
export forceDNS export targetDNS bootDelay dlTimeout curlRetry verbosity led dnsInstance
export parallelDL export whitelist_domains blacklist_domains
export debug export whitelist_domains_urls blacklist_domains_urls blacklist_hosts_urls
export allowNonAscii
export compressedCache
export targetDNS
export bootDelay
export dlTimeout
export curlRetry
export verbosity
export led
export whitelist_domains
export blacklist_domains
export whitelist_domains_urls
export blacklist_domains_urls
export blacklist_hosts_urls
export wan_if wan_gw wanphysdev dl_command serviceStatus dl_flag export wan_if wan_gw wanphysdev dl_command serviceStatus dl_flag
export outputFilter outputFilterIPv6 outputFile outputGzip outputCache ipv6Enabled export outputFilter outputFilterIPv6 outputFile outputGzip outputCache ipv6Enabled
load_package_config() { load_package_config() {
config_load "$packageName" config_load "$packageName"
config_get_bool serviceEnabled "config" "enabled" 1 config_get_bool serviceEnabled 'config' 'enabled' 1
config_get_bool forceDNS "config" "force_dns" 1 config_get_bool forceDNS 'config' 'force_dns' 1
config_get_bool parallelDL "config" "parallel_downloads" 1 config_get_bool parallelDL 'config' 'parallel_downloads' 1
config_get_bool debug "config" "debug" 0 config_get_bool debug 'config' 'debug' 0
config_get_bool allowNonAscii "config" "allow_non_ascii" 0 config_get_bool compressedCache 'config' 'compressed_cache' 0
config_get_bool compressedCache "config" "compressed_cache" 0 config_get_bool ipv6Enabled 'config' 'ipv6_enabled' 0
config_get_bool ipv6Enabled "config" "ipv6_enabled" 0 config_get bootDelay 'config' 'boot_delay' '120'
config_get bootDelay "config" "boot_delay" "120" config_get dlTimeout 'config' 'download_timeout' '20'
config_get dlTimeout "config" "download_timeout" "20" config_get curlRetry 'config' 'curl_retry' '3'
config_get curlRetry "config" "curl_retry" "3" config_get verbosity 'config' 'verbosity' '2'
config_get verbosity "config" "verbosity" "2" config_get led 'config' 'led'
config_get led "config" "led" config_get targetDNS 'config' 'dns' 'dnsmasq.servers'
config_get targetDNS "config" "dns" "dnsmasq.servers" config_get dnsInstance 'config' 'dns_instance' '0'
config_get whitelist_domains "config" "whitelist_domain" config_get whitelist_domains 'config' 'whitelist_domain'
config_get blacklist_domains "config" "blacklist_domain" config_get blacklist_domains 'config' 'blacklist_domain'
config_get whitelist_domains_urls "config" "whitelist_domains_url" config_get whitelist_domains_urls 'config' 'whitelist_domains_url'
config_get blacklist_domains_urls "config" "blacklist_domains_url" config_get blacklist_domains_urls 'config' 'blacklist_domains_url'
config_get blacklist_hosts_urls "config" "blacklist_hosts_url" config_get blacklist_hosts_urls 'config' 'blacklist_hosts_url'
if [ "$targetDNS" != "dnsmasq.addnhosts" ] && [ "$targetDNS" != "dnsmasq.conf" ] && \ if [ "$targetDNS" != 'dnsmasq.addnhosts' ] && [ "$targetDNS" != 'dnsmasq.conf' ] && \
[ "$targetDNS" != "dnsmasq.servers" ] && [ "$targetDNS" != "unbound.adb_list" ]; then [ "$targetDNS" != 'dnsmasq.servers' ] && [ "$targetDNS" != 'unbound.adb_list' ] && \
targetDNS="dnsmasq.servers" [ "$targetDNS" != 'dnsmasq.ipset' ] ; then
targetDNS='dnsmasq.servers'
fi fi
case "$targetDNS" in case "$targetDNS" in
@ -140,6 +139,7 @@ load_package_config() {
outputGzip="$addnhostsGzip" outputGzip="$addnhostsGzip"
[ "$ipv6Enabled" -gt 0 ] && outputFilterIPv6="$addnhostsOutputFilterIPv6" [ "$ipv6Enabled" -gt 0 ] && outputFilterIPv6="$addnhostsOutputFilterIPv6"
rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip" rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
rm -f "$serversFile" "$serversCache" "$serversGzip" rm -f "$serversFile" "$serversCache" "$serversGzip"
rm -f "$unboundFile" "$unboundCache" "$unboundGzip" rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
;; ;;
@ -149,6 +149,17 @@ load_package_config() {
outputCache="$dnsmasqCache" outputCache="$dnsmasqCache"
outputGzip="$dnsmasqGzip" outputGzip="$dnsmasqGzip"
rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip" rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
rm -f "$serversFile" "$serversCache" "$serversGzip"
rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
;;
dnsmasq.ipset)
outputFilter="$ipsetOutputFilter"
outputFile="$ipsetFile"
outputCache="$ipsetCache"
outputGzip="$ipsetGzip"
rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
rm -f "$serversFile" "$serversCache" "$serversGzip" rm -f "$serversFile" "$serversCache" "$serversGzip"
rm -f "$unboundFile" "$unboundCache" "$unboundGzip" rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
;; ;;
@ -159,6 +170,7 @@ load_package_config() {
outputGzip="$serversGzip" outputGzip="$serversGzip"
rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip" rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip" rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
rm -f "$unboundFile" "$unboundCache" "$unboundGzip" rm -f "$unboundFile" "$unboundCache" "$unboundGzip"
;; ;;
unbound.adb_list) unbound.adb_list)
@ -166,8 +178,9 @@ load_package_config() {
outputFile="$unboundFile" outputFile="$unboundFile"
outputCache="$unboundCache" outputCache="$unboundCache"
outputGzip="$unboundGzip" outputGzip="$unboundGzip"
rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip" rm -f "$addnhostsFile" "$addnhostsCache" "$addnhostsGzip"
rm -f "$dnsmasqFile" "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$ipsetFile" "$ipsetCache" "$ipsetGzip"
rm -f "$serversFile" "$serversCache" "$serversGzip" rm -f "$serversFile" "$serversCache" "$serversGzip"
;; ;;
esac esac
@ -207,6 +220,31 @@ is_enabled() {
return 1 return 1
fi fi
case $targetDNS in
dnsmasq.addnhosts | dnsmasq.conf | dnsmasq.ipset | dnsmasq.servers)
if dnsmasq -v 2>/dev/null | grep -q 'no-IDN' || ! dnsmasq -v 2>/dev/null | grep -q -w 'IDN'; then
allowIDN=0
else
allowIDN=1
fi
;;
unbound.adb_list)
allowIDN=1;;
esac
case $targetDNS in
dnsmasq.ipset)
if dnsmasq -v 2>/dev/null | grep -q 'no-ipset' || ! dnsmasq -v 2>/dev/null | grep -q -w 'ipset'; then
output "$_ERROR_: DNSMASQ ipset support is enabled in $packageName, but DNSMASQ is either not installed or installed DNSMASQ does not support ipsets!\\n"
targetDNS='dnsmasq.servers'
fi
if ! ipset help hash:net >/dev/null 2>&1; then
output "$_ERROR_: DNSMASQ ipset support is enabled in $packageName, but ipset is either not installed or installed ipset does not support 'hash:net' type!\\n"
targetDNS='dnsmasq.servers'
fi
;;
esac
[ ! -d "${outputFile%/*}" ] && mkdir -p "${outputFile%/*}" [ ! -d "${outputFile%/*}" ] && mkdir -p "${outputFile%/*}"
[ ! -d "${outputCache%/*}" ] && mkdir -p "${outputFile%/*}" [ ! -d "${outputCache%/*}" ] && mkdir -p "${outputFile%/*}"
[ ! -d "${outputGzip%/*}" ] && mkdir -p "${outputFile%/*}" [ ! -d "${outputGzip%/*}" ] && mkdir -p "${outputFile%/*}"
@ -216,9 +254,6 @@ is_enabled() {
output "$_ERROR_: $serviceName failed to discover WAN gateway.\\n"; return 1; output "$_ERROR_: $serviceName failed to discover WAN gateway.\\n"; return 1;
} }
dnsmasq_kill() { killall -q -HUP dnsmasq; }
dnsmasq_restart() { /etc/init.d/dnsmasq restart >/dev/null 2>&1; }
unbound_restart() { /etc/init.d/unbound restart >/dev/null 2>&1; }
reload_resolver() { reload_resolver() {
local param output_text local param output_text
case $1 in case $1 in
@ -231,37 +266,58 @@ reload_resolver() {
fi fi
case "$targetDNS" in case "$targetDNS" in
dnsmasq.addnhosts) dnsmasq.addnhosts)
uci -q del_list dhcp.@dnsmasq[0].addnhosts="$addnhostsFile" if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
uci add_list dhcp.@dnsmasq[0].addnhosts="$addnhostsFile" uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
if [ -n "$(uci changes dhcp)" ]; then
uci commit dhcp
param=dnsmasq_restart
output_text="Restarting DNSMASQ"
else
param=dnsmasq_kill
output_text="Reloading DNSMASQ"
fi fi
if ! uci -q get dhcp.@dnsmasq["$dnsInstance"].addnhosts | grep -q "$addnhostsFile"; then
uci add_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
fi
param=dnsmasq_kill
output_text='Reloading DNSMASQ'
;; ;;
dnsmasq.conf) dnsmasq.conf)
uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
fi
param=dnsmasq_restart param=dnsmasq_restart
output_text="Restarting DNSMASQ" output_text='Restarting DNSMASQ'
;;
dnsmasq.ipset)
uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
fi
param=dnsmasq_restart
output_text='Restarting DNSMASQ'
;; ;;
dnsmasq.servers) dnsmasq.servers)
if [ "$(uci -q get dhcp.@dnsmasq[0].serversfile)" != "$serversFile" ]; then uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
uci set dhcp.@dnsmasq[0].serversfile="$serversFile" if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" != "$serversFile" ]; then
uci commit dhcp uci set dhcp.@dnsmasq["$dnsInstance"].serversfile="$serversFile"
param=dnsmasq_restart
output_text="Restarting DNSMASQ"
else
param=dnsmasq_kill
output_text="Reloading DNSMASQ"
fi fi
param=dnsmasq_kill
output_text='Reloading DNSMASQ'
;; ;;
unbound.adb_list) unbound.adb_list)
uci -q del_list dhcp.@dnsmasq["$dnsInstance"].addnhosts="$addnhostsFile"
if [ "$(uci -q get dhcp.@dnsmasq["$dnsInstance"].serversfile)" = "$serversFile" ]; then
uci -q del dhcp.@dnsmasq["$dnsInstance"].serversfile
fi
param=unbound_restart param=unbound_restart
output_text="Restarting Unbound" output_text='Restarting Unbound'
;; ;;
esac esac
if [ -n "$(uci changes dhcp)" ]; then
uci commit dhcp
if [ "$param" = 'unbound_restart' ]; then
param='dnsmasq_restart; unbound_restart;'
output_text='Restarting Unbound/DNSMASQ'
else
param=dnsmasq_restart
output_text='Restarting DNSMASQ'
fi
fi
output 1 "$output_text " output 1 "$output_text "
output 2 "$output_text " output 2 "$output_text "
tmpfs set message "$output_text" tmpfs set message "$output_text"
@ -281,26 +337,29 @@ reload_resolver() {
cacheOps 'create' cacheOps 'create'
case "$targetDNS" in case "$targetDNS" in
dnsmasq.addnhosts | dnsmasq.servers) dnsmasq.addnhosts | dnsmasq.servers)
if [ -n "$(uci changes dhcp)" ]; then
uci -q commit dhcp
param=dnsmasq_restart
else
param=dnsmasq_kill param=dnsmasq_kill
fi
;; ;;
dnsmasq.conf) dnsmasq.conf | dnsmasq.ipset)
param=dnsmasq_restart param=dnsmasq_restart
;; ;;
unbound.adb_list) unbound.adb_list)
param=unbound_restart param=unbound_restart
;; ;;
esac esac
if [ -n "$(uci changes dhcp)" ]; then
uci -q commit dhcp
if [ "$param" = 'unbound_restart' ]; then
param='dnsmasq_restart; unbound_restart;'
else
param=dnsmasq_restart
fi
fi
eval "$param" eval "$param"
return $? return $?
;; ;;
quiet) quiet)
case "$targetDNS" in case "$targetDNS" in
dnsmasq.addnhosts | dnsmasq.servers | dnsmasq.conf) dnsmasq.addnhosts | dnsmasq.conf | dnsmasq.ipset | dnsmasq.servers)
param=dnsmasq_restart param=dnsmasq_restart
;; ;;
unbound.adb_list) unbound.adb_list)
@ -337,12 +396,12 @@ tmpfs() {
stats) stats)
echo "$stats"; return;; echo "$stats"; return;;
triggers) triggers)
curReload="$allowNonAscii $parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS" curReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
curRestart="$compressedCache $forceDNS $led" curRestart="$compressedCache $forceDNS $led"
if [ "$curReload" != "$readReload" ]; then if [ "$curReload" != "$readReload" ]; then
ret="download" ret='download'
elif [ "$curRestart" != "$readRestart" ]; then elif [ "$curRestart" != "$readRestart" ]; then
ret="restart" ret='restart'
fi fi
echo "$ret" echo "$ret"
return;; return;;
@ -362,6 +421,12 @@ tmpfs() {
;; ;;
del) del)
case "$instance" in case "$instance" in
all)
unset status;
unset message;
unset error;
unset stats;
;;
status) status)
unset status;; unset status;;
message) message)
@ -385,14 +450,14 @@ tmpfs() {
stats) stats)
stats="$value";; stats="$value";;
triggers) triggers)
readReload="$allowNonAscii $parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS" readReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
readRestart="$compressedCache $forceDNS $led" readRestart="$compressedCache $forceDNS $led"
;; ;;
esac esac
;; ;;
esac esac
json_init json_init
json_add_object "data" json_add_object 'data'
json_add_string version "$PKG_VERSION" json_add_string version "$PKG_VERSION"
json_add_string status "$status" json_add_string status "$status"
json_add_string message "$message" json_add_string message "$message"
@ -445,34 +510,94 @@ cacheOps() {
esac esac
} }
is_chaos_calmer() { ubus -S call system board | grep -q "Chaos Calmer"; } fw3Ops() {
local action="$1" param="$2" _restart
remove_fw3_redirect() { case "$action" in
local name reload) /etc/init.d/firewall reload >/dev/null 2>&1;;
config_get name "$1" "name" restart) /etc/init.d/firewall restart >/dev/null 2>&1;;
# shellcheck disable=SC2154 remove)
if [ -n "$name" ] && str_contains "$name" "simple-adblock"; then case "$param" in
uci -q del "firewall.$1" dns_redirect) uci -q del firewall.simple_adblock_dns_redirect;;
ipset) uci -q del firewall.simple_adblock_ipset
uci -q del firewall.simple_adblock_ipset_rule;;
*)
uci -q del firewall.simple_adblock_dns_redirect
uci -q del firewall.simple_adblock_ipset
uci -q del firewall.simple_adblock_ipset_rule
;;
esac
;;
insert)
case "$param" in
dns_redirect)
if ! uci -q get firewall.simple_adblock_dns_redirect >/dev/null; then
uci -q set firewall.simple_adblock_dns_redirect=redirect
uci -q set firewall.simple_adblock_dns_redirect.name=simple_adblock_dns_hijack
uci -q set firewall.simple_adblock_dns_redirect.target=DNAT
uci -q set firewall.simple_adblock_dns_redirect.src=lan
uci -q set firewall.simple_adblock_dns_redirect.proto=tcpudp
uci -q set firewall.simple_adblock_dns_redirect.src_dport=53
uci -q set firewall.simple_adblock_dns_redirect.dest_port=53
fi fi
} ;;
ipset)
fw3_setup() { if ! uci -q get firewall.simple_adblock_ipset >/dev/null; then
config_load "firewall" uci -q set firewall.simple_adblock_ipset=ipset
config_foreach remove_fw3_redirect "redirect" uci -q set firewall.simple_adblock_ipset.name=adb
if [ "$1" = "start" ]; then uci -q set firewall.simple_adblock_ipset.match=dest_net
uci -q add firewall redirect >/dev/null 2>&1 uci -q set firewall.simple_adblock_ipset.storage=hash
uci -q set firewall.@redirect[-1].name="simple_adblock_dns_hijack" uci -q set firewall.simple_adblock_ipset.enabled=1
uci -q set firewall.@redirect[-1].target="DNAT" _restart=1
uci -q set firewall.@redirect[-1].src="lan"
uci -q set firewall.@redirect[-1].proto="tcpudp"
uci -q set firewall.@redirect[-1].src_dport="53"
uci -q set firewall.@redirect[-1].dest_port="53"
uci -q set firewall.@redirect[-1].dest_ip="$2"
uci -q set firewall.@redirect[-1].reflection="0"
fi fi
if ! uci -q get firewall.simple_adblock_ipset_rule >/dev/null; then
uci -q set firewall.simple_adblock_ipset_rule=rule
uci -q set firewall.simple_adblock_ipset_rule.name=simple_adblock_ipset_rule
uci -q set firewall.simple_adblock_ipset_rule.ipset=adb
uci -q set firewall.simple_adblock_ipset_rule.src=lan
uci -q set firewall.simple_adblock_ipset_rule.dest='*'
uci -q set firewall.simple_adblock_ipset_rule.proto=tcpudp
uci -q set firewall.simple_adblock_ipset_rule.target=REJECT
uci -q set firewall.simple_adblock_ipset_rule.enabled=1
fi
;;
*)
if ! uci -q get firewall.simple_adblock_dns_redirect >/dev/null; then
uci -q set firewall.simple_adblock_dns_redirect=redirect
uci -q set firewall.simple_adblock_dns_redirect.name=simple_adblock_dns_hijack
uci -q set firewall.simple_adblock_dns_redirect.target=DNAT
uci -q set firewall.simple_adblock_dns_redirect.src=lan
uci -q set firewall.simple_adblock_dns_redirect.proto=tcpudp
uci -q set firewall.simple_adblock_dns_redirect.src_dport=53
uci -q set firewall.simple_adblock_dns_redirect.dest_port=53
fi
if ! uci -q get firewall.simple_adblock_ipset >/dev/null; then
uci -q set firewall.simple_adblock_ipset=ipset
uci -q set firewall.simple_adblock_ipset.name=adb
uci -q set firewall.simple_adblock_ipset.match=dest_net
uci -q set firewall.simple_adblock_ipset.storage=hash
uci -q set firewall.simple_adblock_ipset.enabled=1
_restart=1
fi
if ! uci -q get firewall.simple_adblock_ipset_rule >/dev/null; then
uci -q set firewall.simple_adblock_ipset_rule=rule
uci -q set firewall.simple_adblock_ipset_rule.name=simple_adblock_ipset_rule
uci -q set firewall.simple_adblock_ipset_rule.ipset=adb
uci -q set firewall.simple_adblock_ipset_rule.src=lan
uci -q set firewall.simple_adblock_ipset_rule.dest='*'
uci -q set firewall.simple_adblock_ipset_rule.proto=tcpudp
uci -q set firewall.simple_adblock_ipset_rule.target=REJECT
uci -q set firewall.simple_adblock_ipset_rule.enabled=1
fi
;;
esac
esac
if [ -n "$(uci changes firewall)" ]; then if [ -n "$(uci changes firewall)" ]; then
uci -q commit firewall uci -q commit firewall
/etc/init.d/firewall restart >/dev/null 2>&1 if [ -z "$_restart" ]; then
fw3Ops 'reload'
else
fw3Ops 'restart'
fi
fi fi
} }
@ -480,15 +605,15 @@ process_url() {
local label type D_TMP R_TMP local label type D_TMP R_TMP
if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then return 1; fi if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then return 1; fi
label="${1##*//}"; label="${label%%/*}"; label="${1##*//}"; label="${label%%/*}";
if [ "$2" = "hosts" ]; then if [ "$2" = 'hosts' ]; then
label="Hosts: $label"; filter="$hostsFilter"; label="Hosts: $label"; filter="$hostsFilter";
else else
label="Domains: $label"; filter="$domainsFilter"; label="Domains: $label"; filter="$domainsFilter";
fi fi
if [ "$3" = "blocked" ]; then if [ "$3" = 'blocked' ]; then
type="Blocked"; D_TMP="$B_TMP"; type='Blocked'; D_TMP="$B_TMP";
else else
type="Allowed"; D_TMP="$A_TMP"; type='Allowed'; D_TMP="$A_TMP";
fi fi
while [ -z "$R_TMP" ] || [ -e "$R_TMP" ]; do while [ -z "$R_TMP" ] || [ -e "$R_TMP" ]; do
R_TMP="$(mktemp -u -q -t ${packageName}_tmp.XXXXXXXX)" R_TMP="$(mktemp -u -q -t ${packageName}_tmp.XXXXXXXX)"
@ -496,13 +621,13 @@ process_url() {
if ! $dl_command "$1" $dl_flag "$R_TMP" 2>/dev/null || [ ! -s "$R_TMP" ]; then if ! $dl_command "$1" $dl_flag "$R_TMP" 2>/dev/null || [ ! -s "$R_TMP" ]; then
output 1 "$_FAIL_" output 1 "$_FAIL_"
output 2 "[DL] $type $label $__FAIL__\\n" output 2 "[DL] $type $label $__FAIL__\\n"
printf "%b" "Error: downloading '${1}'.\\n" >> "$sharedMemoryError" echo "Error: downloading '${1}'." >> "$sharedMemoryError"
else else
sed -i "$filter" "$R_TMP" sed -i "$filter" "$R_TMP"
if [ ! -s "$R_TMP" ]; then if [ ! -s "$R_TMP" ]; then
output 1 "$_FAIL_" output 1 "$_FAIL_"
output 2 "[DL] $type $label $__FAIL__\\n" output 2 "[DL] $type $label $__FAIL__\\n"
printf "%b" "Error: parsing '${1}'.\\n" >> "$sharedMemoryError" echo "Error: parsing '${1}'." >> "$sharedMemoryError"
else else
cat "${R_TMP}" >> "$D_TMP" cat "${R_TMP}" >> "$D_TMP"
output 1 "$_OK_" output 1 "$_OK_"
@ -519,7 +644,7 @@ download_lists() {
tmpfs set message "${statusDownloading}..." tmpfs set message "${statusDownloading}..."
rm -f "$A_TMP" "$B_TMP" "$outputFile" "$outputCache" "$outputGzip" rm -f "$A_TMP" "$B_TMP" "$outputFile" "$outputCache" "$outputGzip"
if [ "$(awk '/^MemFree/ {print int($2/1000)}' "/proc/meminfo")" -lt 32 ]; then if [ "$(awk '/^MemFree/ {print int($2/1000)}' "/proc/meminfo")" -lt 32 ]; then
output 3 "Low free memory, restarting resolver... " output 3 'Low free memory, restarting resolver... '
if reload_resolver 'quiet'; then if reload_resolver 'quiet'; then
output_okn output_okn
else else
@ -527,148 +652,158 @@ download_lists() {
fi fi
fi fi
touch $A_TMP; touch $B_TMP; touch $A_TMP; touch $B_TMP;
output 1 "Downloading lists " output 1 'Downloading lists '
rm -f "$sharedMemoryError" rm -f "$sharedMemoryError"
if [ -n "$blacklist_hosts_urls" ]; then if [ -n "$blacklist_hosts_urls" ]; then
for hf in ${blacklist_hosts_urls}; do for hf in ${blacklist_hosts_urls}; do
if [ "$parallelDL" -gt 0 ]; then if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" "hosts" "blocked" & process_url "$hf" 'hosts' 'blocked' &
else else
process_url "$hf" "hosts" "blocked" process_url "$hf" 'hosts' 'blocked'
fi fi
done done
fi fi
if [ -n "$blacklist_domains_urls" ]; then if [ -n "$blacklist_domains_urls" ]; then
for hf in ${blacklist_domains_urls}; do for hf in ${blacklist_domains_urls}; do
if [ "$parallelDL" -gt 0 ]; then if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" "domains" "blocked" & process_url "$hf" 'domains' 'blocked' &
else else
process_url "$hf" "domains" "blocked" process_url "$hf" 'domains' 'blocked'
fi fi
done done
fi fi
if [ -n "$whitelist_domains_urls" ]; then if [ -n "$whitelist_domains_urls" ]; then
for hf in ${whitelist_domains_urls}; do for hf in ${whitelist_domains_urls}; do
if [ "$parallelDL" -gt 0 ]; then if [ "$parallelDL" -gt 0 ]; then
process_url "$hf" "domains" "allowed" & process_url "$hf" 'domains' 'allowed' &
else else
process_url "$hf" "domains" "allowed" process_url "$hf" 'domains' 'allowed'
fi fi
done done
fi fi
wait wait
[ -s "$sharedMemoryError" ] && tmpfs add error "$(cat "$sharedMemoryError")" [ -s "$sharedMemoryError" ] && tmpfs add error "$(cat "$sharedMemoryError")"
rm -f "$sharedMemoryError" rm -f "$sharedMemoryError"
output 1 "\\n" output 1 '\n'
[ -n "$blacklist_domains" ] && for hf in ${blacklist_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done [ -n "$blacklist_domains" ] && for hf in ${blacklist_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
whitelist_domains="${whitelist_domains} whitelist_domains="${whitelist_domains}
$(cat $A_TMP)" $(cat $A_TMP)"
[ -n "$whitelist_domains" ] && for hf in ${whitelist_domains}; do hf=$(echo "$hf" | sed 's/\./\\./g'); w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done [ -n "$whitelist_domains" ] && for hf in ${whitelist_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
[ ! -s "$B_TMP" ] && return 1 [ ! -s "$B_TMP" ] && return 1
output 1 "Processing downloads " output 1 'Processing downloads '
output 2 "Sorting combined list " output 2 'Sorting combined list '
tmpfs set message "$statusProcessing: sorting combined list" tmpfs set message "$statusProcessing: sorting combined list"
if [ "$allowNonAscii" -gt 0 ]; then if [ "$allowIDN" -gt 0 ]; then
if sort "$B_TMP" | uniq > "$A_TMP"; then if sort -u "$B_TMP" > "$A_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Sorting error." tmpfs add error 'Error: Sorting error.'
fi fi
else else
if sort "$B_TMP" | uniq | grep -E -v '[^a-zA-Z0-9=/.-]' > "$A_TMP"; then if sort -u "$B_TMP" | grep -E -v '[^a-zA-Z0-9=/.-]' > "$A_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Sorting error." tmpfs add error 'Error: Sorting error.'
fi fi
fi fi
if [ "$targetDNS" = "dnsmasq.conf" ] || \ if [ "$targetDNS" = 'dnsmasq.conf' ] || \
[ "$targetDNS" = "dnsmasq.servers" ] || \ [ "$targetDNS" = 'dnsmasq.ipset' ] || \
[ "$targetDNS" = "unbound.adb_list" ]; then [ "$targetDNS" = 'dnsmasq.servers' ] || \
[ "$targetDNS" = 'unbound.adb_list' ]; then
# TLD optimization written by Dirk Brenken (dev@brenken.org) # TLD optimization written by Dirk Brenken (dev@brenken.org)
output 2 "Optimizing combined list " output 2 'Optimizing combined list '
tmpfs set message "$statusProcessing: optimizing combined list" tmpfs set message "$statusProcessing: optimizing combined list"
# sed -E 'G;:t;s/(.*)(\.)(.*)(\n)(.*)/\1\4\5\2\3/;tt;s/(.*)\n(\.)(.*)/\3\2\1/' is actually slower than awk
if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$A_TMP" > "$B_TMP"; then if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$A_TMP" > "$B_TMP"; then
if sort "$B_TMP" > "$A_TMP"; then if sort "$B_TMP" > "$A_TMP"; then
if awk '{if(NR=1){tld=$NF};while(getline){if($NF!~tld"\\."){print tld;tld=$NF}}print tld}' "$A_TMP" > "$B_TMP"; then if awk '{if(NR=1){tld=$NF};while(getline){if($NF!~tld"\\."){print tld;tld=$NF}}print tld}' "$A_TMP" > "$B_TMP"; then
if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$B_TMP" > "$A_TMP"; then if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$B_TMP" > "$A_TMP"; then
if sort "$A_TMP" | uniq > "$B_TMP"; then if sort -u "$A_TMP" > "$B_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Data file optimization." tmpfs add error 'Error: Data file optimization.'
mv "$A_TMP" "$B_TMP" mv "$A_TMP" "$B_TMP"
fi fi
else else
output_failn output_failn
tmpfs add error "Error: Data file optimization." tmpfs add error 'Error: Data file optimization.'
fi fi
else else
output_failn output_failn
tmpfs add error "Error: Data file optimization." tmpfs add error 'Error: Data file optimization.'
mv "$A_TMP" "$B_TMP" mv "$A_TMP" "$B_TMP"
fi fi
else else
output_failn output_failn
tmpfs add error "Error: Data file optimization." tmpfs add error 'Error: Data file optimization.'
fi fi
else else
output_failn output_failn
tmpfs add error "Error: Data file optimization." tmpfs add error 'Error: Data file optimization.'
mv "$A_TMP" "$B_TMP" mv "$A_TMP" "$B_TMP"
fi fi
else else
mv "$A_TMP" "$B_TMP" mv "$A_TMP" "$B_TMP"
fi fi
output 2 "Whitelisting domains " output 2 'Whitelisting domains '
tmpfs set message "$statusProcessing: whitelisting domains" tmpfs set message "$statusProcessing: whitelisting domains"
if sed -i "$w_filter" "$B_TMP"; then if sed -i "$w_filter" "$B_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Whitelist processing." tmpfs add error 'Error: Whitelist processing.'
fi fi
output 2 "Formatting merged file " output 2 'Formatting merged file '
tmpfs set message "$statusProcessing: formatting merged file" tmpfs set message "$statusProcessing: formatting merged file"
if [ -z "$outputFilterIPv6" ]; then if [ -z "$outputFilterIPv6" ]; then
if sed "$outputFilter" "$B_TMP" > "$A_TMP"; then if sed "$outputFilter" "$B_TMP" > "$A_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Data file formatting." tmpfs add error 'Error: Data file formatting.'
fi fi
else else
case "$targetDNS" in
dnsmasq.addnhosts)
if sed "$outputFilter" "$B_TMP" > "$A_TMP" && \ if sed "$outputFilter" "$B_TMP" > "$A_TMP" && \
sed "$outputFilterIPv6" "$B_TMP" >> "$A_TMP"; then sed "$outputFilterIPv6" "$B_TMP" >> "$A_TMP"; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: Data file formatting." tmpfs add error 'Error: Data file formatting.'
fi fi
;;
esac
fi fi
case "$targetDNS" in case "$targetDNS" in
dnsmasq.addnhosts) dnsmasq.addnhosts)
output 2 "Creating DNSMASQ addnhosts file " output 2 'Creating DNSMASQ addnhosts file '
tmpfs set message "$statusProcessing: creating DNSMASQ addnhosts file" tmpfs set message "$statusProcessing: creating DNSMASQ addnhosts file"
;; ;;
dnsmasq.conf) dnsmasq.conf)
output 2 "Creating DNSMASQ config file " output 2 'Creating DNSMASQ config file '
tmpfs set message "$statusProcessing: creating DNSMASQ config file" tmpfs set message "$statusProcessing: creating DNSMASQ config file"
;; ;;
dnsmasq.ipset)
output 2 'Creating DNSMASQ ipset file '
tmpfs set message "$statusProcessing: creating DNSMASQ ipset file"
;;
dnsmasq.servers) dnsmasq.servers)
output 2 "Creating DNSMASQ servers file " output 2 'Creating DNSMASQ servers file '
tmpfs set message "$statusProcessing: creating DNSMASQ servers file" tmpfs set message "$statusProcessing: creating DNSMASQ servers file"
;; ;;
unbound.adb_list) unbound.adb_list)
output 2 "Creating Unbound adb_list file " output 2 'Creating Unbound adb_list file '
tmpfs set message "$statusProcessing: creating Unbound adb_list file" tmpfs set message "$statusProcessing: creating Unbound adb_list file"
;; ;;
esac esac
@ -679,27 +814,27 @@ $(cat $A_TMP)"
tmpfs add error "Error: moving data file '${A_TMP}' to '${outputFile}'." tmpfs add error "Error: moving data file '${A_TMP}' to '${outputFile}'."
fi fi
if [ "$compressedCache" -gt 0 ]; then if [ "$compressedCache" -gt 0 ]; then
output 2 "Creating compressed cache " output 2 'Creating compressed cache '
tmpfs set message "$statusProcessing: creating compressed cache" tmpfs set message "$statusProcessing: creating compressed cache"
if cacheOps 'createGzip'; then if cacheOps 'createGzip'; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: creating compressed cache." tmpfs add error 'Error: creating compressed cache.'
fi fi
else else
rm -f "$outputGzip" rm -f "$outputGzip"
fi fi
output 2 "Removing temporary files " output 2 'Removing temporary files '
tmpfs set message "$statusProcessing: removing temporary files" tmpfs set message "$statusProcessing: removing temporary files"
rm -f "/tmp/${packageName}_tmp.*" "$A_TMP" "$B_TMP" "$outputCache" || j=1 rm -f "/tmp/${packageName}_tmp.*" "$A_TMP" "$B_TMP" "$outputCache" || j=1
if [ $j -eq 0 ]; then if [ $j -eq 0 ]; then
output_ok output_ok
else else
output_failn output_failn
tmpfs add error "Error: removing temporary files." tmpfs add error 'Error: removing temporary files.'
fi fi
output 1 "\\n" output 1 '\n'
} }
boot() { boot() {
@ -713,37 +848,66 @@ boot() {
start_service() { start_service() {
is_enabled 'on_start' || return 1 is_enabled 'on_start' || return 1
local ip action status error message stats local action status error message stats
if create_lock; then if create_lock; then
procd_open_instance "main" if is_chaos_calmer || ! is_ipset_procd; then
if [ "$forceDNS" -ne 0 ]; then
fw3Ops 'insert' 'dns_redirect'
else
fw3Ops 'remove' 'dns_redirect'
fi
if [ "$targetDNS" = 'dnsmasq.ipset' ]; then
fw3Ops 'insert' 'ipset'
else
fw3Ops 'remove' 'ipset'
fi
procd_open_instance 'main'
procd_set_param command /bin/true procd_set_param command /bin/true
procd_set_param stdout 1 procd_set_param stdout 1
procd_set_param stderr 1 procd_set_param stderr 1
network_get_ipaddr ip "lan" procd_close_instance
# shellcheck disable=SC2154
if [ "$forceDNS" -ne 0 ] && [ -n "$ip" ]; then
if is_chaos_calmer; then
fw3_setup "start" "$ip"
else else
procd_open_instance 'main'
procd_set_param command /bin/true
procd_set_param stdout 1
procd_set_param stderr 1
procd_open_data procd_open_data
json_add_array firewall json_add_array firewall
json_add_object "" if [ "$forceDNS" -ne 0 ]; then
json_add_object ''
json_add_string type redirect json_add_string type redirect
json_add_string target "DNAT" json_add_string name simple_adblock_dns_redirect
json_add_string src "lan" json_add_string target DNAT
json_add_string dest "lan" json_add_string src lan
json_add_string proto "tcpudp" json_add_string proto tcpudp
json_add_string src_dport "53" json_add_string src_dport 53
json_add_string dest_port "53" json_add_string dest_port 53
json_add_string dest_ip "$ip" json_add_string reflection 0
json_add_string name "simple-adblock-dns-hijack"
json_add_string reflection "0"
json_close_object json_close_object
fi
if [ "$targetDNS" = 'dnsmasq.ipset' ]; then
json_add_object ''
json_add_string type ipset
json_add_string name adb
json_add_string match dest_net
json_add_string storage hash
json_add_string enabled 1
json_close_object
json_add_object ''
json_add_string type rule
json_add_string name simple_adblock_ipset_rule
json_add_string ipset adb
json_add_string src lan
json_add_string dest '*'
json_add_string proto tcpudp
json_add_string target REJECT
json_add_string enabled 1
json_close_object
fi
json_close_array json_close_array
procd_close_data procd_close_data
fi
fi
procd_close_instance procd_close_instance
fi
status="$(tmpfs get status)" status="$(tmpfs get status)"
error="$(tmpfs get error)" error="$(tmpfs get error)"
@ -752,27 +916,24 @@ start_service() {
action="$(tmpfs get triggers)" action="$(tmpfs get triggers)"
case "$1" in case "$1" in
download) action="download";; download) action='download';;
restart|*) restart|*)
if [ "$1" != "restart" ] && [ -s "$outputFile" ] && [ -n "$status" ]; then if [ -s "$outputFile" ] && [ -n "$status" ]; then
status status
exit 0 exit 0
elif [ ! -s "$outputFile" ] && ! cacheOps 'test' && ! cacheOps 'testGzip'; then elif [ ! -s "$outputFile" ] && ! cacheOps 'test' && ! cacheOps 'testGzip'; then
action="download" action='download'
elif cacheOps 'test' || cacheOps 'testGzip'; then elif cacheOps 'test' || cacheOps 'testGzip'; then
action="start" action='start'
fi fi
if [ -n "$error" ]; then if [ -n "$error" ]; then
action="download" action='download'
fi fi
action="${action:-$1}" action="${action:-$1}"
;; ;;
esac esac
tmpfs del status tmpfs del all
tmpfs del error
tmpfs del message
tmpfs del stats
tmpfs set triggers tmpfs set triggers
case $action in case $action in
@ -790,7 +951,7 @@ start_service() {
reload_resolver 'on_start' reload_resolver 'on_start'
;; ;;
restart|start) restart|start)
if [ "$action" = "restart" ]; then if [ "$action" = 'restart' ]; then
output 0 "Restarting $serviceName... " output 0 "Restarting $serviceName... "
output 3 "Restarting $serviceName...\\n" output 3 "Restarting $serviceName...\\n"
tmpfs set status "$statusRestarting" tmpfs set status "$statusRestarting"
@ -800,20 +961,20 @@ start_service() {
tmpfs set status "$statusStarting" tmpfs set status "$statusStarting"
fi fi
if cacheOps 'testGzip' && ! cacheOps 'test' && [ ! -s "$outputFile" ]; then if cacheOps 'testGzip' && ! cacheOps 'test' && [ ! -s "$outputFile" ]; then
output 3 "Found compressed cache file, unpacking it " output 3 'Found compressed cache file, unpacking it '
tmpfs set message "found compressed cache file, unpacking it." tmpfs set message 'found compressed cache file, unpacking it.'
if cacheOps 'unpackGzip'; then if cacheOps 'unpackGzip'; then
output_okn output_okn
else else
output_fail output_fail
output "$_ERROR_: $serviceName failed to unpack compressed cache!\\n" output "$_ERROR_: $serviceName failed to unpack compressed cache!\\n"
tmpfs add error "Error: Failed to unpack compressed cache." tmpfs add error 'Error: Failed to unpack compressed cache.'
return 1 return 1
fi fi
fi fi
if cacheOps 'test' && [ ! -s "$outputFile" ]; then if cacheOps 'test' && [ ! -s "$outputFile" ]; then
output 3 "Found cache file, reusing it " output 3 'Found cache file, reusing it '
tmpfs set message "found cache file, reusing it." tmpfs set message 'found cache file, reusing it.'
if cacheOps 'restore'; then if cacheOps 'restore'; then
output_okn output_okn
else else
@ -828,7 +989,7 @@ start_service() {
if [ -s "$outputFile" ] && [ "$(tmpfs get status)" != "$statusFail" ]; then if [ -s "$outputFile" ] && [ "$(tmpfs get status)" != "$statusFail" ]; then
output 0 "$__OK__\\n"; output 0 "$__OK__\\n";
c="$(wc -l < "$outputFile")" c="$(wc -l < "$outputFile")"
output 3 "$serviceName is blocking $c domains "; output_okn output 3 "$serviceName is blocking $c domains (with ${targetDNS}) "; output_okn
tmpfs del message tmpfs del message
tmpfs set status "$statusSuccess: $c domains blocked (with ${targetDNS})." tmpfs set status "$statusSuccess: $c domains blocked (with ${targetDNS})."
error="$(tmpfs get error)" error="$(tmpfs get error)"
@ -838,7 +999,7 @@ start_service() {
else else
output 0 "$__FAIL__\\n"; output 0 "$__FAIL__\\n";
tmpfs set status "$statusFail" tmpfs set status "$statusFail"
tmpfs add error "Error: Failed to create blocklist." tmpfs add error 'Error: Failed to create blocklist.'
fi fi
remove_lock remove_lock
else else
@ -847,17 +1008,18 @@ start_service() {
fi fi
} }
service_started() { procd_set_config_changed firewall; } service_started() { is_ipset_procd && procd_set_config_changed firewall; }
service_stopped() { procd_set_config_changed firewall; } service_stopped() { is_ipset_procd && procd_set_config_changed firewall; }
restart_service() { rc_procd start_service "restart"; } restart_service() { rc_procd start_service 'restart'; }
reload_service() { restart_service; } reload_service() { restart_service; }
restart() { restart_service; } restart() { restart_service; }
reload() { restart_service; } reload() { restart_service; }
dl() { rc_procd start_service "download"; } dl() { rc_procd start_service 'download'; }
killcache() { killcache() {
rm -f "$addnhostsCache" "$addnhostsGzip" rm -f "$addnhostsCache" "$addnhostsGzip"
rm -f "$dnsmasqCache" "$dnsmasqGzip" rm -f "$dnsmasqCache" "$dnsmasqGzip"
rm -f "$ipsetCache" "$ipsetGzip"
rm -f "$serversCache" "$serversGzip" rm -f "$serversCache" "$serversGzip"
rm -f "$unboundCache" "$unboundGzip" rm -f "$unboundCache" "$unboundGzip"
return 0 return 0
@ -874,9 +1036,7 @@ status() {
stop_service() { stop_service() {
load_package_config load_package_config
if is_chaos_calmer; then fw3Ops 'remove' 'all'
fw3_setup 'stop'
fi
if [ -s "$outputFile" ]; then if [ -s "$outputFile" ]; then
output "Stopping $serviceName... " output "Stopping $serviceName... "
tmpfs del triggers tmpfs del triggers
@ -897,17 +1057,24 @@ stop_service() {
check() { check() {
load_package_config load_package_config
local string="$1" local string="$1"
local c="$(grep -c "$string" "$outputFile")"
if [ ! -f "$outputFile" ]; then if [ ! -f "$outputFile" ]; then
echo "No blacklist ('$outputFile') found." echo "No blacklist ('$outputFile') found."
elif [ -z "$string" ]; then elif [ -z "$string" ]; then
echo "Usage: /etc/init.d/${packageName} check domain" echo "Usage: /etc/init.d/${packageName} check string"
elif grep -m1 -q "$string" "$outputFile"; then elif [ "$c" -gt 0 ]; then
echo "Found $(grep -c "$string" "$outputFile") matches for '$string' in '$outputFile':" if [ "$c" -gt 1 ]; then
echo "Found $c matches for '$string' in '$outputFile':"
else
echo "Found 1 match for '$string' in '$outputFile':"
fi
case "$targetDNS" in case "$targetDNS" in
dnsmasq.addnhosts) dnsmasq.addnhosts)
grep "$string" "$outputFile" | sed 's|^127.0.0.1 ||;s|^:: ||;';; grep "$string" "$outputFile" | sed 's|^127.0.0.1 ||;s|^:: ||;';;
dnsmasq.conf) dnsmasq.conf)
grep "$string" "$outputFile" | sed 's|local=/||;s|/$||;';; grep "$string" "$outputFile" | sed 's|local=/||;s|/$||;';;
dnsmasq.ipset)
grep "$string" "$outputFile" | sed 's|ipset=/||;s|/adb$||;';;
dnsmasq.servers) dnsmasq.servers)
grep "$string" "$outputFile" | sed 's|server=/||;s|/$||;';; grep "$string" "$outputFile" | sed 's|server=/||;s|/$||;';;
unbound.adb_list) unbound.adb_list)