* made the reporting/top statistics flexible, see "top_count" parm in CLI or in LuCI (default 10), fixes#19622
* added the new blocklist source cpbl (provided by PascalCoffeeLake@gmail.com)
* added/separated Easylist/Easyprivacy blocklist sources (provided by PascalCoffeeLake@gmail.com)
* added reg_jp blocklist_source (provided by PascalCoffeeLake@gmail.com)
* removed the easylist addons from the other regional lists
* removed the second/obsolete pl regional list and renamed the first one to "reg_pl"
* updated the readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* dnsmasq upstream has changed the code for domain handling
and recommends the 'local' syntax for large blocklists
* remove pipefail command, see #19043 for reference
* removed the unused 'adb_dnsinotify' parameter
* removed the 'adb_maxqueue' parameter,
the queue size will be automatically set by the number of cpu cores
* various cleanups, mostly shellcheck related
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add new 'hblock' compilation source (XL, see https://hblock.molinero.dev for reference)
* print runtime/date information in ISO-8601 standard format
* minor cleanups
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add a tcpdump option to resolve IPs in adblock reporting,
set 'adb_represolve' accordingly (disabled by default). If enabled
tcpdump will perform a reverse DNS (PTR) lookup for each IP address
* add 'stalkerware' source (provided by @astryzia)
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix a small json syntax issue in adblock.sources
* add easylist addon to reg_fr source
* add switch 'adb_fetchinsecure' to allow insecure downloads
without certificate check (disabled by default)
* better explain 'adb_fetchparm' in readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* support the RPZ trigger 'RPZ-CLIENT-IP' to always allow/block certain
clients based on their IP (currently only supported by bind!)
* avoid promiscuous mode in tcpdump setup for adblock reporting
* speed up dns report preparation
* support dns report mailing (/etc/init.d/adblock report mail)
* fix bind autodetection
* update LuCI-frontend (separate PR)
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add a restrictive "jail mode only" variant, just point your
jail directory to your primary dns directory
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adguard_tracking source (list with cname trackers)
* optimize/sort output of active sources in status
* optimize log output in EMails
Signed-off-by: Dirk Brenken <dev@brenken.org>
* major source changes:
* split oisd.nl in basic and full variant
* add swedish regional list
* made archive categories for shallalist and utcapitole selectable
via LuCI
* made all list variants of energized and stevenblack selectable
via LuCI
* removed dns filereset mode
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove 'dshield' and 'sysctl' (discontinued)
* switch 'malwaredomains', 'shallalist' and 'winhelp' to https
* add a second regional list for poland (provided by matx1002)
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add anti_ad blocklist source
* made SafeSearch provider configurable, you can limit
SafeSearch to certain providers
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add regional list source for czech/slovak
* add regional list source for korea
* adapt oisd_nl changes, switch to adb-syntax domains
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove dumb list cache
* start adblock processing after adding/removing
list sources via CLI
* add regional list source for france
Signed-off-by: Dirk Brenken <dev@brenken.org>
* removed 'hphosts' from sources (discontinued)
* fixed a "out of range" bug and another small issue
in the f_dnsup function
* add three new sources: 'anudeep', 'stopforumspam' and 'youtube'
* changed 'list' behaviour, the source file has now
a higher precedence than the archive file (see readme)
* update readme, added missing parameters & more
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fix dependency issue
* fix query timeouts in web frontend (seen with many selected lists),
now the query comes back latest after 30 seconds, to prevent any
timeouts, with all results to this point.
* add missing parameter in readme.md
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new package dependencies: coreultis-sort and
a download util with SSL support
* focus on speed (multicore-support) to handle quite big lists
* include 38 pre-configured blocklist sources in a compressed
json file (/etc/adblock/adblock.sources.gz)
* dynamic SafeSearch support for google, bing, duckduckgo,
yandex, youtube and pixabay (CNAME (bind) & IP (dnsmaq, unbound))
* DNS backend autodetection
* Download Utility autodetection
* Report Interface autodetection
* Easy cron wrapper to set an adblock related auto-timer for
automatic blocklist updates
* raw domain/blocklist support (e.g. for dnscrypt support)
* re-add restrictive Jaillist support
* rework online doc
* Complete LuCI rewrite (migrated to client side JS)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove 'ransomware' blocklist by abbuse.ch (discontinued)
from default adblock config
* fix/switch 'someonewhocares' config to https only
* fix curl download parameters to follow redirects and
suppress needless output
* made the tmp directory of sort operations configurable,
set 'adb_sorttmp' accordingly (only supported by 'coreutils-sort')
Signed-off-by: Dirk Brenken <dev@brenken.org>
* use raw procd interface trigger as last resort, if the
adblock config is not available during startup
* fix selective subdomain whitelisting for dnsmasq
* fix a kresd restart issue with 'DNS File Reset'
* fix a suspend/resume cornercase
* disable the tld compression, if the number of blocked domains
is greater than 'adb_maxtld' (default: 100000)
* made the fw portlist configurable (default '53 853 5353')
* preliminary support for inotify-like autoload features
of dns backends like kresd in future Turris OS. If 'adb_dnsinotify'
is set to 'true', all adblock related restarts and the
'DNS File Reset' will be disabled
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add support for 'DNS File Reset', where the final DNS blockfile
will be purged after DNS backend loading (save storage space).
A small background service will be started to trace/handle
dns backend reloads/restarts
* add support for the 'null' blocking variant in dnsmasq
(via addn-hosts), which may provide better response times
in dnsmasq
* enhance the report & search engine to support
the new blocking variants. Search now includes
backups & black-/whitelist as well
* compressed source list backups are now mandatory (default to '/tmp')
* speed up TLD compression
* E-Mail notification setup is now integrated in UCI/LuCI
* update the LuCI frontend to reflect all changes (separate PR)
* drop preliminary dnscrypt-proxy-support (use dnsmasq instead)
* drop additional 'dnsjail' blocklist support (not used by anyone)
* procd cleanups in init
* various shellcheck cleanups
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
- fix reporting for bogus hostnames with underscores
- no longer accidently overwrite existing 'serversfile' entries in dhcp
config which reference to the adblock jail list
- remove needless 'no_mail' flag
- refined log message regarding tcpdump requirement for reporting
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enhance the query function to search in adblock backups as well,
to get back the set of blocking lists sources for a certain domain
* add "Latest DNS Queries" report to commandline version as well
(already in LuCI)
* made the tld compression (the error handling) more robust,
remove the needless 'adb_forcesrt' option
* removed abandoned 'feodo' list source
* updated readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* report engine supports multiple listening ports, set
'adb_replisten' to a space separated list of ports,
default '53'
* report engine supports multiple interfaces, set 'adb_repiface'
to 'any'
* small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adblock dns query reporting via tcpdump (see readme for details)
* fix tld compression on low memory systems (< 64 MB)
* fix various small issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add generic blocklist archive support
* add support for blacklist archive from Toulouse 1 University Capitole
* add support for urlhaus RPZ domains by abuse.ch
* small bugfixes & enhancements
luci:
* archive sub-categories (shalla & ut_capitole) are now configurable
via LuCI CBI template
* small bugfixes & enhancements
Signed-off-by: Dirk Brenken <dev@brenken.org>
* major performance boost: add a flexible 'Download Queue' to handle
downloads & list processing in parallel, default queue size is '4',
you can raise this e.g. to '8' or '16' to get it really fast
* replace former 'whitelist mode': the new 'Jail' option
builds an additional 'adb_list.jail' list in parallel
which can be used manually for guest wifi or kidsafe configurations
* regex parser & query function now fully support IDN domains
with non-ASCII characters
* add error handling in tld compression,
to handle OOM conditions better
* adblock.notify sends now html emails,
to get a better look & feel, even on mobile devices
* add czech regional blocklist maintained by turris omnia users
* LuCI: Support new 'Download Queue' & 'Jail' options
* LuCI: fix field width in "Runtime Information" section
Signed-off-by: Dirk Brenken <dev@brenken.org>
* preserve DNS cache after adblock processing,
- 'unbound' and 'named' support this (please check readme)
- 'dnsmasq' now uses the 'servers-file' directive
to minimize the reload disruption,
even though the dns cache will be cleared after SIGHUP
- 'kresd' dns cache is persistent by upstream default, anyway
Turris Omnia devices need a small upstream software change
which is not accepted/implemented yet
* email notification in case of an error or domain count < n
(default 0, check readme)
* removed securemecca from default config (service has been closed)
* new separate functions for hash compare and list/overall count
* add missing package dependencies
* various clean-ups
* update documentation
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new function to set/delete options in external uci config files
* kresd: automated 'rpz_file' handling in /etc/config/resolver
* firewall: automated 'force_dns' handling if you
enable or disable adblock
* support sha256sum (default) and md5sum for blocklist
comparison & conditional dns restarts
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* please note: config file update required!
* add 'whitelist only' mode, block access to all domains
except those explicitly listed in the whitelist file
* rework awk regex for all blocklist sources
* include 'third-party' domains for all regional lists
* change adguard url and refine filter ruleset
* use POSIX character classes
* fix regex for whitelist preparation
* fix corner case parsing issues
* fix enable/disable behavior
* various other small fixes
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
