* remove dumb list cache
* start adblock processing after adding/removing
list sources via CLI
* add regional list source for france
Signed-off-by: Dirk Brenken <dev@brenken.org>
(cherry picked from commit ab44d8146a)
* new package dependencies: coreultis-sort and
a download util with SSL support
* focus on speed (multicore-support) to handle quite big lists
* include more than 40 pre-configured blocklist sources in a compressed
json file (/etc/adblock/adblock.sources.gz)
* dynamic SafeSearch support for google, bing, duckduckgo,
yandex, youtube and pixabay (CNAME (bind) & IP (dnsmaq, unbound))
* DNS backend autodetection
* Download Utility autodetection
* Report Interface autodetection
* Easy cron wrapper to set an adblock related auto-timer for
automatic blocklist updates
* raw domain/blocklist support (e.g. for dnscrypt support)
* re-add restrictive Jaillist support
* rework online doc
* Complete LuCI rewrite (migrated to client side JS)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove 'reg_cz' list (abandoned)
* add 'notracking' list (provided by @rcarmo)
* 19.07-only: fix/bring back status message
Signed-off-by: Dirk Brenken <dev@brenken.org>
* more startup tweaks
* re-use f_log function in helper scripts
* small fixes / polish up for forthcoming 19.07 release
Signed-off-by: Dirk Brenken <dev@brenken.org>
(cherry picked from commit 2c3cb6f1d1)
* use raw procd interface trigger as last resort, if the
adblock config is not available during startup
* fix selective subdomain whitelisting for dnsmasq
* fix a kresd restart issue with 'DNS File Reset'
* fix a suspend/resume cornercase
* disable the tld compression, if the number of blocked domains
is greater than 'adb_maxtld' (default: 100000)
* made the fw portlist configurable (default '53 853 5353')
* preliminary support for inotify-like autoload features
of dns backends like kresd in future Turris OS. If 'adb_dnsinotify'
is set to 'true', all adblock related restarts and the
'DNS File Reset' will be disabled
Signed-off-by: Dirk Brenken <dev@brenken.org>
(cherry picked from commit 45cb0e1023)
* add support for 'DNS File Reset', where the final DNS blockfile
will be purged after DNS backend loading (save storage space).
A small background service will be started to trace/handle
dns backend reloads/restarts
* add support for the 'null' blocking variant in dnsmasq
(via addn-hosts), which may provide better response times
in dnsmasq
* enhance the report & search engine to support
the new blocking variants. Search now includes
backups & black-/whitelist as well
* compressed source list backups are now mandatory (default to '/tmp')
* speed up TLD compression
* E-Mail notification setup is now integrated in UCI/LuCI
* update the LuCI frontend to reflect all changes (separate PR)
* drop preliminary dnscrypt-proxy-support (use dnsmasq instead)
* drop additional 'dnsjail' blocklist support (not used by anyone)
* procd cleanups in init
* various shellcheck cleanups
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
(cherry picked from commit 504412ccdb)
* the DNS Report now displays the hostname, MAC-Address or
client IP (CLI & LuCI)
* Filter the DNS Query result set for a particular domain, client or
time frame (CLI & LuCI)
* remove needless XHR.Poll-Events from Reporting page in LuCI
* remove needless 'force sort' option in LuCI
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adblock dns query reporting via tcpdump (see readme for details)
* fix tld compression on low memory systems (< 64 MB)
* fix various small issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add low priority mode (nice level 10), disabled by default
* enhance 'Force DNS' to redirect ports 53, 853 and 5353
frontend:
* switch to dynamic XHR polling for runtime information and logfile
viewing
* add new 'Refresh' button to reload blocklists
* various cleanups & small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enable code to support Turris Omnia forthcoming upstream change
(new kresd 'keep_cache' option) to preserve kresd DNS cache
* fix a 'status' race condition while the adblock process
is running in parallel
* various small speed improvements
* rework debug output
* refine blacklist handling
* enable the (empty) blacklist source in the default config
* email notification supports mstmp, even without sendmail symlink
* email notification writes minimal status to log (one-liner)
* LuCI: refine logfile search term
* LuCI: Textarea 'autoscroll down' in logfile view
* LuCI: Left-align blocklist source table plus a more compact design
Signed-off-by: Dirk Brenken <dev@brenken.org>
* preserve DNS cache after adblock processing,
- 'unbound' and 'named' support this (please check readme)
- 'dnsmasq' now uses the 'servers-file' directive
to minimize the reload disruption,
even though the dns cache will be cleared after SIGHUP
- 'kresd' dns cache is persistent by upstream default, anyway
Turris Omnia devices need a small upstream software change
which is not accepted/implemented yet
* email notification in case of an error or domain count < n
(default 0, check readme)
* removed securemecca from default config (service has been closed)
* new separate functions for hash compare and list/overall count
* add missing package dependencies
* various clean-ups
* update documentation
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add kresd & turris omnia support
* add dnscrypt-proxy support
* change start priority to 30, to fix possible trigger issues on slow
booting hardware
* simplify suspend/resume handling (no longer use a hideout directory)
* enhanced LuCI frontend
* many small changes & improvements
* default config change (please update your config!), adblock is now
disabled by default
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* various small fixes & optimizations
LuCI frontend (see luci repo):
* Limit Blacklist/Whitelist Online editing to max. 512 KB, approx.
20.000 domains per list
* Automatically refresh the overview page after button onclick event,
e.g. 'Suspend/Resume' or 'Save & Apply'
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* change start priority to get all interface trigger events, even on
fast hardware
* made default trigger delay more conservative to fix possible start up
issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
* no longer misuse ubus/procd service object for adblock runtime
information, now save all required information directly
in a JSON file/format (/tmp/adb_runtime.json)
* new 'status' init command to print runtime information
* add a configurable interface trigger timeout for
nested or slow modem/router setups,
set 'adb_triggerdelay' accordingly (default 1 second)
* add support for pure http download utilities like wget-nossl
or uclient-fetch without libustream-ssl (http donwloads only!)
* fix stop action
* fix enabled/disabled action
* fix country code in regional list for china
* LuCI update to reflect all changes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* re-order all regional lists in adblocklist/easylist format
* add polish blocklist
* add chinese blocklist
* lists called now 'reg_<ch|pl|ro|ru>'
* changed interface trigger default in config to 'wan' only
* update / sync LuCI frontend (see luci repo for details)
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* revamp LuCI GUI (see luci repo for details)
* integrated runtime statistics in normal adblock ubus service instance
(see readme)
* simplified boot/init
* no longer use raw interface trigger
* use only network interface trigger, pre-configured for
'wan/wwan/lan' interfaces (see readme)
* fixed a reload issue
* removed switch to disable tld compression (seems to be pretty stable)
* muted awk parser if source list contains incorrect records
* raise dns backend timeout to 20 seconds
* documentation update
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add tld compression,
this new "top level domain compression" removes up to 40 thousand
needless host entries from the block lists and
lowers the memory footprint for the dns backends by 8-10 MByte
* optimize restart behavior in case of an error
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* various optimizations & corner case fixes
* removed no longer needed debug information
* polished up for forthcoming LEDE release ;-)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* automatically selects dnsmasq or unbound as dns backend
* add the new 'adguard' source, a combined/quite effective block list
* remove needless dns backend restarts
* optimize adblock restart behavior
* optimize block list processing on inotify enabled filesystems
* better return code checking on block list download
* fix boot function/startup on Chaos Calmer
* fix a bug in blocklist removal function
* add more (optional) debug output
* move backup options to global config
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fixed dnsmasq check if multiple instances are present
* bring back query function on highly demand
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fixed a dnsmasq restart issue (udhcpc error)
* fixed a long standing corner case bug in "disabled" state (does not
remove active block lists!)
* simplified overall sort, removed needless 'for loop'
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* changed complete dnsmasq handling
- no longer redirect ad requests to internal web server via firewall
rules etc., in fact send back a simple NXDOMAIN for all
ad related (sub-)domains
- smaller memory footprint for block lists
- removed needless uhttpd instances
- removed needless firewall rules/redirects
* init/hotplug system migrated to procd
- removed hotplug.d script, now using procd interface trigger
* reduced code size/complexity
- removed needless internal pre-checks & function blocks,
no longer rely on a separate helper library
- removed flash writes to adblock config
* support different download tools like wget (default), aria2c,
uclient-fetch, curl (see online doc)
* adblock status/statistics via ubus call (see online doc)
* various bug fixes
* documentation update
* changed makefile copyright notice
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add new 'envchk'function to check adblock environment only,
i.e. check volatile firewall rules or uhttpd instances
without list updates
* add new optional parm 'adb_loglevel',
set it to "0" to mute output (print only errors)
* set hotplug priority to '90' as well (missed in the last commit)
* documentation update
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enhance the new query function:
change the regex to find only the relevant blocklist entries
add a recursive tld search to quickly identify domains for
whitelisting (see documentation)
better result preparation
* add securemecca as new blocklist source
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add a query function to search the active blocklists for a specific
domain (/etc/init.d/adblock query <DOMAIN>)
* fix bug in ap mode/uhttpd port detection
* check general firewall and dnsmasq package dependencies and remove
redundant checks in ap mode
Signed-off-by: Dirk Brenken <dev@brenken.org>
* restructured sources
* fix logical glitches in config handling
* many corner case fixes & cosmetics
* show runtime errors in LuCI (in lastrun section)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* rework download/backup/restore handling
* rework firewall handling
* refine whitelist parsing
* changed default null-ip to 198.18.0.1 to prevent conflicts with bcp38
* spread traffic from ad servers (port 80/443) on two uhttpd instances,
this fixes timeouts on sites with many ssl ad server connections, i.e.
youtube.com
* provide statistics & last runtime for LuCI interface
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new config option 'adb_restricted' to disable updates of the adblock
config during runtime (no flash writes, disabled by default)
* new update function, to apply latest configuration changes if needed
(/etc/init.d/adblock cfgup)
* fix dns redirect rule
* various optimizations
Signed-off-by: Dirk Brenken <dev@brenken.org>
* support more router modes, as long as the firewall and the DNS server
are enabled
* new source sites can be added & changed via awk ruleset in uci config
* source domain count, last update time & overall count will be stored
in uci config
* added 3 new source sites:
ransomware tracker
rolist/easylist
winspy
* switch to minimal inline base64 encoded 1×1 GIF for pixel server
(separate png image no longer needed)
* simplified uci parser
* source download & domain sort optimization
* add whitelist parser with wildcard support
* reduced code size & various cleanups
* updated documentation
Signed-off-by: Dirk Brenken <openwrt@brenken.org>
