Difuse/packages
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

banip: release 0.8.9-1

Browse source 
* added HTTP ETag or entity tag support to download only ressources that have been updated on the server side,
  to save bandwith and speed up banIP reloads
* added 4 new feeds: binarydefense, bruteforceblock, etcompromised, ipblackhole (see readme)
* updated the readme

Signed-off-by: Dirk Brenken <dev@brenken.org>
This commit is contained in:
Dirk Brenken 2023-07-07 18:28:21 +02:00
parent 381a5515c7
commit 68cdc3952d
No known key found for this signature in database
GPG key ID: 9D71CD547BFAE684
4 changed files with 108 additions and 36 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
net/banip/Makefile
View file

@ -5,8 +5,8 @@
include $(TOPDIR)/rules.mk
PKG_NAME:=banip
PKG_VERSION:=0.8.8
PKG_RELEASE:=2
PKG_VERSION:=0.8.9
PKG_RELEASE:=1
PKG_LICENSE:=GPL-3.0-or-later
PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org>

17
net/banip/files/README.md
View file

@ -17,7 +17,9 @@ IP address blocking is commonly used to protect against brute force attacks, pre
| antipopads | antipopads IPs | | | x | [Link](https://github.com/dibdot/banIP-IP-blocklists) |
| asn | ASN IPs | | | x | [Link](https://asn.ipinfo.app) |
| backscatterer | backscatterer IPs | x | x | | [Link](https://www.uceprotect.net/en/index.php) |
| binarydefense | binary defense banlist | x | x | | [Link](https://iplists.firehol.org/?ipset=bds_atif) |
| bogon | bogon prefixes | x | x | | [Link](https://team-cymru.com) |
| bruteforceblock | bruteforceblocker IPs | x | x | | [Link](https://danger.rulez.sk/index.php/bruteforceblocker/) |
| country | country blocks | x | x | | [Link](https://www.ipdeny.com/ipblocks) |
| cinsscore | suspicious attacker IPs | x | x | | [Link](https://cinsscore.com/#list) |
| darklist | blocks suspicious attacker IPs | x | x | | [Link](https://darklist.de) |
@ -26,6 +28,7 @@ IP address blocking is commonly used to protect against brute force attacks, pre
| drop | spamhaus drop compilation | x | x | | [Link](https://www.spamhaus.org) |
| dshield | dshield IP blocklist | x | x | | [Link](https://www.dshield.org) |
| edrop | spamhaus edrop compilation | x | x | | [Link](https://www.spamhaus.org) |
| etcompromised | ET compromised hosts | x | x | | [Link](https://iplists.firehol.org/?ipset=et_compromised) |
| feodo | feodo tracker | x | x | x | [Link](https://feodotracker.abuse.ch) |
| firehol1 | firehol level 1 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level1) |
| firehol2 | firehol level 2 compilation | x | x | | [Link](https://iplists.firehol.org/?ipset=firehol_level2) |
@ -34,6 +37,7 @@ IP address blocking is commonly used to protect against brute force attacks, pre
| greensnow | suspicious server IPs | x | x | | [Link](https://greensnow.co) |
| iblockads | Advertising IPs | | | x | [Link](https://www.iblocklist.com) |
| iblockspy | Malicious spyware IPs | x | x | | [Link](https://www.iblocklist.com) |
| ipblackhole | blackhole IPs | x | x | | [Link](https://ip.blackhole.monster) |
| ipthreat | hacker and botnet TPs | x | x | | [Link](https://ipthreat.net) |
| myip | real-time IP blocklist | x | x | | [Link](https://myip.ms) |
| nixspam | iX spam protection | x | x | | [Link](http://www.nixspam.org) |
@ -72,7 +76,8 @@ IP address blocking is commonly used to protect against brute force attacks, pre
* Per feed it can be defined whether the wan-input chain, the wan-forward chain or the lan-forward chain should be blocked (default: all chains)
* Automatic blocklist backup & restore, the backups will be used in case of download errors or during startup
* Automatically selects one of the following download utilities with ssl support: aria2c, curl, uclient-fetch or full wget
* Supports an 'allowlist only' mode, this option restricts internet access from/to a small number of secure websites/IPs
* Provides HTTP ETag or entity tag support to download only ressources that have been updated on the server side, to save bandwith and speed up banIP reloads
* Supports an 'allowlist only' mode, this option restricts internet access from/to a given number of secure websites/IPs
* Deduplicate IPs accross all Sets (single IPs only, no intervals)
* Provides comprehensive runtime information
* Provides a detailed Set report
@ -86,7 +91,7 @@ IP address blocking is commonly used to protect against brute force attacks, pre
## Prerequisites
* **[OpenWrt](https://openwrt.org)**, latest stable release or a snapshot with nft/firewall 4 and logd/logread support
* A download utility with SSL support: 'aria2c', 'curl', full 'wget' or 'uclient-fetch' with one of the 'libustream-*' SSL libraries
* A download utility with SSL support: 'aria2c', 'curl', full 'wget' or 'uclient-fetch' with one of the 'libustream-*' SSL libraries, the latter one doesn't provide support for ETag HTTP header
* A certificate store like 'ca-bundle', as banIP checks the validity of the SSL certificates of all download sites by default
* For E-Mail notifications you need to install and setup the additional 'msmtp' package
@ -145,7 +150,7 @@ Available commands:
| ban_autoblocklist | option | 1 | add suspicious attacker IPs and resolved domains automatically to the local blocklist (not only to the Sets) |
| ban_autoblocksubnet | option | 0 | add entire subnets to the blocklist Sets based on an additional RDAP request with the suspicious IP |
| ban_autoallowuplink | option | subnet | limit the uplink autoallow function to: 'subnet', 'ip' or 'disable' it at all |
| ban_allowlistonly | option | 0 | restrict the internet access from/to a small number of secure websites/IPs |
| ban_allowlistonly | option | 0 | restrict the internet access from/to a given number of secure websites/IPs |
| ban_basedir | option | /tmp | base working directory while banIP processing |
| ban_reportdir | option | /tmp/banIP-report | directory where banIP stores the report files |
| ban_backupdir | option | /tmp/banIP-backup | directory where banIP stores the compressed backup files |
@ -292,6 +297,9 @@ Depending on the options 'ban_autoallowlist' and 'ban_autoallowuplink' the uplin
Furthermore, you can reference external Allowlist URLs with additional IPv4 and IPv6 feeds (see 'ban_allowurl').
Both local lists also accept domain names as input to allow IP filtering based on these names. The corresponding IPs (IPv4 & IPv6) will be extracted and added to the Sets. You can also start the domain lookup separately via /etc/init.d/banip lookup at any time.
**allowlist-only mode**
banIP supports an "allowlist only" mode. This option restricts the internet access from/to a small number of secure MACs, IPs or domains, and block access from/to the rest of the internet. All IPs and Domains which are _not_ listed in the allowlist (plus the external Allowlist URLs) are blocked.
**MAC/IP-binding**
banIP supports concatenation of local MAC addresses with IPv4/IPv6 addresses, e.g. to enforce dhcp assignments. Following notations in the local allow and block lists are allowed:
```
@ -313,9 +321,6 @@ C8:C2:9B:F7:80:12 192.168.1.10 => this will be populated to
C8:C2:9B:F7:80:12 => this will be populated to v6MAC-Set with the IP-wildcard ::/0
```
**allowlist-only mode**
banIP supports an "allowlist only" mode. This option restricts the internet access from/to a small number of secure MACs, IPs or domains, and block access from/to the rest of the internet. All IPs and Domains which are _not_ listed in the allowlist are blocked.
**redirect Asterisk security logs to lodg/logread**
banIP only supports logfile scanning via logread, so to monitor attacks on Asterisk, its security log must be available via logread. To do this, edit '/etc/asterisk/logger.conf' and add the line 'syslog.local0 = security', then run 'asterisk -rx reload logger' to update the running Asterisk configuration.

103
net/banip/files/banip-functions.sh
View file

@ -79,6 +79,7 @@ ban_fetchparm=""
ban_fetchinsecure=""
ban_fetchretry="5"
ban_rdapparm=""
ban_etagparm=""
ban_cores=""
ban_memory=""
ban_packages=""
@ -332,25 +333,28 @@ f_getfetch() {
[ "${ban_fetchinsecure}" = "1" ] && insecure="--check-certificate=false"
ban_fetchparm="${ban_fetchparm:-"${insecure} --timeout=20 --retry-wait=10 --max-tries=${ban_fetchretry} --max-file-not-found=${ban_fetchretry} --allow-overwrite=true --auto-file-renaming=false --log-level=warn --dir=/ -o"}"
ban_rdapparm="--timeout=5 --allow-overwrite=true --auto-file-renaming=false --dir=/ -o"
ban_etagparm="--timeout=5 --allow-overwrite=true --auto-file-renaming=false --dir=/ --dry-run --log -"
;;
"curl")
[ "${ban_fetchinsecure}" = "1" ] && insecure="--insecure"
ban_fetchparm="${ban_fetchparm:-"${insecure} --connect-timeout 20 --retry-delay 10 --retry ${ban_fetchretry} --retry-all-errors --fail --silent --show-error --location -o"}"
ban_fetchparm="${ban_fetchparm:-"${insecure} --connect-timeout 20 --retry-delay 10 --retry ${ban_fetchretry} --retry-max-time $((ban_fetchretry * 20)) --retry-all-errors --fail --silent --show-error --location -o"}"
ban_rdapparm="--connect-timeout 5 --silent --location -o"
ban_etagparm="--connect-timeout 5 --silent --location --head"
;;
"wget")
[ "${ban_fetchinsecure}" = "1" ] && insecure="--no-check-certificate"
ban_fetchparm="${ban_fetchparm:-"${insecure} --no-cache --no-cookies --timeout=20 --waitretry=10 --tries=${ban_fetchretry} --retry-connrefused -O"}"
ban_rdapparm="--timeout=5 -O"
ban_etagparm="--timeout=5 --spider --server-response"
;;
"uclient-fetch")
[ "${ban_fetchinsecure}" = "1" ] && insecure="--no-check-certificate"
ban_fetchparm="${ban_fetchparm:-"${insecure} --timeout=20 -O"}"
ban_rdapparm="--timeout=5 -O"
;;
"wget")
[ "${ban_fetchinsecure}" = "1" ] && insecure="--no-check-certificate"
ban_fetchparm="${ban_fetchparm:-"${insecure} --no-cache --no-cookies --timeout=20 --waitretry=10 --tries=${ban_fetchretry} --retry-connrefused -O"}"
ban_rdapparm="--timeout=5 -O"
;;
esac
f_log "debug" "f_getfetch ::: auto/update: ${ban_autodetect}/${update}, cmd: ${ban_fetchcmd:-"-"}, fetch_parm: ${ban_fetchparm:-"-"}, rdap_parm: ${ban_rdapparm:-"-"}"
f_log "debug" "f_getfetch ::: auto/update: ${ban_autodetect}/${update}, cmd: ${ban_fetchcmd:-"-"}, fetch_parm: ${ban_fetchparm:-"-"}, rdap_parm: ${ban_rdapparm:-"-"}, etag_parm: ${ban_etagparm:-"-"}"
}
# get wan interfaces
@ -462,7 +466,7 @@ f_getuplink() {
for ip in ${ban_uplink}; do
if ! "${ban_grepcmd}" -q "${ip} " "${ban_allowlist}"; then
if [ "${update}" = "0" ]; then
"${ban_sedcmd}" -i '/# uplink added on /d' "${ban_allowlist}"
"${ban_sedcmd}" -i "/# uplink added on /d" "${ban_allowlist}"
fi
printf "%-42s%s\n" "${ip}" "# uplink added on $(date "+%Y-%m-%d %H:%M:%S")" >>"${ban_allowlist}"
f_log "info" "add uplink '${ip}' to local allowlist"
@ -471,7 +475,7 @@ f_getuplink() {
done
ban_uplink="${ban_uplink%%?}"
elif [ "${ban_autoallowlist}" = "1" ] && [ "${ban_autoallowuplink}" = "disable" ]; then
"${ban_sedcmd}" -i '/# uplink added on /d' "${ban_allowlist}"
"${ban_sedcmd}" -i "/# uplink added on /d" "${ban_allowlist}"
update="1"
fi
@ -502,6 +506,31 @@ f_getelements() {
[ -s "${file}" ] && printf "%s" "elements={ $("${ban_catcmd}" "${file}" 2>/dev/null) };"
}
# handle etag http header
#
f_etag() {
local http_head http_code etag_id etag_rc out_rc="4" feed="${1}" feed_url="${2}" feed_suffix="${3}"
if [ -n "${ban_etagparm}" ]; then
[ ! -f "${ban_backupdir}/banIP.etag" ] && : >"${ban_backupdir}/banIP.etag"
http_head="$("${ban_fetchcmd}" ${ban_etagparm} "${feed_url}" 2>&1)"
http_code="$(printf "%s" "${http_head}" | "${ban_awkcmd}" 'tolower($0)~/^http\/[0123\.]+ /{printf "%s",$2}')"
etag_id="$(printf "%s" "${http_head}" | "${ban_awkcmd}" '{FS="\""}tolower($0)~/^[[:space:]]*etag: /{printf "%s",$2}')"
etag_rc="${?}"
if [ "${http_code}" = "404" ] || { [ "${etag_rc}" = "0" ] && [ -n "${etag_id}" ] && "${ban_grepcmd}" -q "^${feed}${feed_suffix}.*${etag_id}\$" "${ban_backupdir}/banIP.etag"; }; then
out_rc="0"
elif [ "${etag_rc}" = "0" ] && [ -n "${etag_id}" ] && ! "${ban_grepcmd}" -q "^${feed}${feed_suffix}.*${etag_id}\$" "${ban_backupdir}/banIP.etag"; then
"${ban_sedcmd}" -i "/^${feed}${feed_suffix}/d" "${ban_backupdir}/banIP.etag"
printf "%-20s%s\n" "${feed}${feed_suffix}" "${etag_id}" >>"${ban_backupdir}/banIP.etag"
out_rc="2"
fi
fi
f_log "debug" "f_etag ::: feed: ${feed}, suffix: ${feed_suffix:-"-"}, http_code: ${http_code:-"-"}, etag_id: ${etag_id:-"-"} , etag_rc: ${etag_rc:-"-"}, rc: ${out_rc}"
return "${out_rc}"
}
# build initial nft file with base table, chains and rules
#
f_nftinit() {
@ -547,13 +576,13 @@ f_nftinit() {
feed_rc="${?}"
f_log "debug" "f_nftinit ::: devices: ${ban_dev}, priority: ${ban_nftpriority}, policy: ${ban_nftpolicy}, loglevel: ${ban_nftloglevel}, rc: ${feed_rc:-"-"}, log: ${feed_log:-"-"}"
return ${feed_rc}
return "${feed_rc}"
}
# handle downloads
#
f_down() {
local log_input log_forwardwan log_forwardlan start_ts end_ts tmp_raw tmp_load tmp_file split_file ruleset_raw handle
local log_input log_forwardwan log_forwardlan start_ts end_ts tmp_raw tmp_load tmp_file split_file ruleset_raw handle rc etag_rc="0"
local cnt_set cnt_dl restore_rc feed_direction feed_rc feed_log feed="${1}" proto="${2}" feed_url="${3}" feed_rule="${4}" feed_flag="${5}"
start_ts="$(date +%s)"
@ -616,12 +645,34 @@ f_down() {
} >"${tmp_flush}"
fi
# restore local backups during init
# restore local backups
#
if { [ "${ban_action}" != "reload" ] || [ "${feed_url}" = "local" ]; } && [ "${feed%v*}" != "allowlist" ] && [ "${feed%v*}" != "blocklist" ]; then
f_restore "${feed}" "${feed_url}" "${tmp_load}"
restore_rc="${?}"
feed_rc="${restore_rc}"
if { [ "${ban_action}" != "reload" ] || [ "${feed_url}" = "local" ] || [ -n "${ban_etagparm}" ]; } && [ "${feed%v*}" != "allowlist" ] && [ "${feed%v*}" != "blocklist" ]; then
if [ -n "${ban_etagparm}" ] && [ "${feed_url}" != "local" ]; then
if [ "${feed%v*}" = "country" ]; then
for country in ${ban_country}; do
f_etag "${feed}" "${feed_url}${country}-aggregated.zone" ".${country}"
rc="${?}"
[ "${rc}" = "4" ] && break
etag_rc="$((etag_rc + rc))"
done
elif [ "${feed%v*}" = "asn" ]; then
for asn in ${ban_asn}; do
f_etag "${feed}" "${feed_url}AS${asn}" ".{asn}"
rc="${?}"
[ "${rc}" = "4" ] && break
etag_rc="$((etag_rc + rc))"
done
else
f_etag "${feed}" "${feed_url}"
etag_rc="${?}"
fi
fi
if [ "${etag_rc}" = "0" ] || [ "${ban_action}" != "reload" ] || [ "${feed_url}" = "local" ]; then
f_restore "${feed}" "${feed_url}" "${tmp_load}" "${etag_rc}"
restore_rc="${?}"
feed_rc="${restore_rc}"
fi
fi
# prepare local allowlist
@ -781,10 +832,7 @@ f_down() {
"gz")
feed_log="$("${ban_fetchcmd}" ${ban_fetchparm} "${tmp_raw}" "${feed_url}" 2>&1)"
feed_rc="${?}"
if [ "${feed_rc}" = "0" ]; then
"${ban_zcatcmd}" "${tmp_raw}" 2>/dev/null >"${tmp_load}"
feed_rc="${?}"
fi
[ "${feed_rc}" = "0" ] && "${ban_zcatcmd}" "${tmp_raw}" 2>/dev/null >"${tmp_load}"
rm -f "${tmp_raw}"
;;
esac
@ -898,7 +946,7 @@ f_down() {
rm -f "${tmp_split}" "${tmp_nft}"
end_ts="$(date +%s)"
f_log "debug" "f_down ::: name: ${feed}, cnt_dl: ${cnt_dl:-"-"}, cnt_set: ${cnt_set:-"-"}, split_size: ${ban_splitsize:-"-"}, time: $((end_ts - start_ts)), rc: ${feed_rc:-"-"}, log: ${feed_log:-"-"}"
f_log "debug" "f_down ::: feed: ${feed}, cnt_dl: ${cnt_dl:-"-"}, cnt_set: ${cnt_set:-"-"}, split_size: ${ban_splitsize:-"-"}, time: $((end_ts - start_ts)), rc: ${feed_rc:-"-"}, log: ${feed_log:-"-"}"
}
# backup feeds
@ -909,24 +957,23 @@ f_backup() {
gzip -cf "${feed_file}" >"${ban_backupdir}/banIP.${feed}.gz"
backup_rc="${?}"
f_log "debug" "f_backup ::: name: ${feed}, source: ${feed_file##*/}, target: banIP.${feed}.gz, rc: ${backup_rc}"
return ${backup_rc}
f_log "debug" "f_backup ::: feed: ${feed}, file: banIP.${feed}.gz, rc: ${backup_rc}"
return "${backup_rc}"
}
# restore feeds
#
f_restore() {
local tmp_feed restore_rc="1" feed="${1}" feed_url="${2}" feed_file="${3}" feed_rc="${4:-"0"}"
local tmp_feed restore_rc="4" feed="${1}" feed_url="${2}" feed_file="${3}" in_rc="${4}"
[ "${feed_rc}" != "0" ] && restore_rc="${feed_rc}"
[ "${feed_url}" = "local" ] && tmp_feed="${feed%v*}v4" || tmp_feed="${feed}"
if [ -f "${ban_backupdir}/banIP.${tmp_feed}.gz" ]; then
if [ -s "${ban_backupdir}/banIP.${tmp_feed}.gz" ]; then
"${ban_zcatcmd}" "${ban_backupdir}/banIP.${tmp_feed}.gz" 2>/dev/null >"${feed_file}"
restore_rc="${?}"
fi
f_log "debug" "f_restore ::: name: ${feed}, source: banIP.${tmp_feed}.gz, target: ${feed_file##*/}, in_rc: ${feed_rc}, rc: ${restore_rc}"
return ${restore_rc}
f_log "debug" "f_restore ::: feed: ${feed}, file: banIP.${tmp_feed}.gz, in_rc: ${in_rc:-"-"}, rc: ${restore_rc}"
return "${restore_rc}"
}
# remove disabled Sets

20
net/banip/files/banip.feeds
View file

@ -40,6 +40,11 @@
"descr": "backscatterer IPs",
"flag": "gz"
},
"binarydefense":{
"url_4": "https://iplists.firehol.org/files/bds_atif.ipset",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
"descr": "binary defense banlist"
},
"bogon":{
"url_4": "https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt",
"url_6": "https://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt",
@ -47,6 +52,11 @@
"rule_6": "/^(([0-9A-f]{0,4}:){1,7}[0-9A-f]{0,4}:?(\\/(1?[0-2][0-8]|[0-9][0-9]))?)$/{printf \"%s,\\n\",$1}",
"descr": "bogon prefixes"
},
"bruteforceblock":{
"url_4": "https://danger.rulez.sk/projects/bruteforceblocker/blist.php",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)[[:space:]]/{printf \"%s,\\n\",$1}",
"descr": "bruteforceblocker IPs"
},
"cinsscore":{
"url_4": "https://cinsscore.com/list/ci-badguys.txt",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
@ -95,6 +105,11 @@
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)[[:space:]]/{printf \"%s,\\n\",$1}",
"descr": "spamhaus edrop compilation"
},
"etcompromised":{
"url_4": "https://iplists.firehol.org/files/et_compromised.ipset",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
"descr": "ET compromised hosts"
},
"feodo":{
"url_4": "https://feodotracker.abuse.ch/downloads/ipblocklist.txt",
"rule_4": "BEGIN{RS=\"\\r\\n\"}/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
@ -137,6 +152,11 @@
"descr": "malicious spyware IPs",
"flag": "gz"
},
"ipblackhole":{
"url_4": "https://ip.blackhole.monster/blackhole-today",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf \"%s,\\n\",$1}",
"descr": "blackhole IP blocklist"
},
"ipthreat":{
"url_4": "https://lists.ipthreat.net/file/ipthreat-lists/threat/threat-30.txt",
"rule_4": "/^(([0-9]{1,3}\\.){3}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\\/(1?[0-9]|2?[0-9]|3?[0-2]))?)[-[:space:]]?/{printf \"%s,\\n\",$1}",