From 43d5a8c43a0982480c64b3c97ff6a5666f0698f3 Mon Sep 17 00:00:00 2001 From: Jez Caudle Date: Thu, 1 Feb 2024 15:15:42 +0000 Subject: [PATCH] Getting the Read.me done --- .pf-badhost.sh.kate-swp | Bin 0 -> 99 bytes README.md | 75 ++++++++++++++++------------------------ pf-badhost.sh | 4 +-- 3 files changed, 32 insertions(+), 47 deletions(-) create mode 100644 .pf-badhost.sh.kate-swp diff --git a/.pf-badhost.sh.kate-swp b/.pf-badhost.sh.kate-swp new file mode 100644 index 0000000000000000000000000000000000000000..cf7765d7cd14a6adcddbb6537cb96293dd27559d GIT binary patch literal 99 zcmZQzU=Z?7EJ;-eE>A2_aLdd|RWQ;sU|?VnDKZFJ5SHt6`&8qmw1dS#_nP!R2?l!t eWkiA4ABY(hT!TY_w3usf5Rd{12LN#(Iu`)`p%qvF literal 0 HcmV?d00001 diff --git a/README.md b/README.md index d6412dc..0cf0be6 100644 --- a/README.md +++ b/README.md @@ -10,8 +10,7 @@ In loving memory of Ron Sather Copyright (c) Jez Caudle 2024 onwards with the same license as above. -# Fresh Installation Guide - +# Fresh Installation Guide Create a new user (we'll call ours "_pfbadhost"). The user should be created with a default shell of "nologin", home folder set to /var/empty/ with no password specified (disables password logins) @@ -67,7 +66,7 @@ Most options can also be configured via command line flags. See man page for mor See the "Notes" section below for more info on installing optional utilities. -Run pfbadhost as user "_pfbadhost" using the "-O openbsd" argument: +Run pfbadhost as user "_pfbadhost": ``` doas -u _pfbadhost pf-badhost ``` @@ -86,7 +85,7 @@ Edit _pfbadhost users crontab to run pf-badhost every night: ``` crontab -u _pfbadhost -e ... -~ 0~1 * * * -s pf-badhost -O openbsd +~ 0~1 * * * -s pf-badhost ... ``` @@ -96,57 +95,43 @@ With the nightly cron job, the list will be regularly updated with the latest known bad hosts. Please read the man page for information on how to configure pf-badhost. -The manpage can be found here: https://www.geoghegan.ca/pub/pf-badhost/0.5/man/man.txt - -To receive notification of new pf-badhost releases and updates please send an email -to 'announce@geoghegan.ca' with a subject line and body of "subscribe pf-badhost" - +The manpage can be found here: https://git.sr.ht/~jezcaudle/pfbadhost-fork/tree/main/item/man.txt ## Post Install Notes: - -To add custom rules or enable features, or add alternate blocklists, see the "User Configuration Area" located at the top of the script. This area serves as a built in config file, so please feel free to edit it and experiment with all the features available within. Note: Most options can also be configured from the command line +To add custom rules or enable features, or add alternate blocklists, see the "User Configuration Area" located at the top of the script. This area serves as a built in config file, so please feel free to edit it and experiment with all the features available within. Note: Most options can also be configured from the command line. Regarding Cron Jobs - over the past year I've noticed a number of list host servers going down at midnight in populated timezones (ie West Coast, East Coast and Western Europe). To be respectful (and to avoid overloading list providers servers) we have cron jobs scheduled to run at a random time within a defined interval. With the new default cron job, pf-badhost will be run every night at some point between midnight and 2AM, and thus distributing the load of thousands of queries from numerous users over a 2 hour period rather than a matter of seconds. +Subnet aggregation is used to reduce the number of entries in a PF table by taking the individual IP addresses and translating them into CIDR notation such as 82.221.99.224/28 -The script is able to detect which (if any) subnet aggregation utilities are installed and will try to "Do The Right Thing(tm)" and fallback to the best available option. If no subnet aggregation utility is found, the script will fallback to using a pure Perl IPv4 aggregator if Perl is installed. Despite its name, "aggregate6" supports both IPv4 and IPv6 addresses and is written in Python, whereas the "aggregate" utility supports only IPv4 addresses and is written in C and uses significantly less memory but runs much slower. For greatly improved performance, aggregate6 can be run with Pypy. If both utilities are installed, the C based "aggregate" utility will be preferred for IPv4 aggregation, but the script will happily function if only one or the other is installed (or neither). +You can see what the limits are for various things in pf with the +``` +pfctl -sm + +states hard limit 100000 +src-nodes hard limit 10000 +frags hard limit 65536 +tables hard limit 1000 +table-entries hard limit 200000 +pktdelay-pkts hard limit 10000 +anchors hard limit 512 +``` + +The script is able to detect which (if any) subnet aggregation utilities are installed and will try to "Do The Right Thing(tm)" and fallback to the best available option. If no subnet aggregation utility is found, the script will fallback to using a pure Perl IPv4 aggregator if Perl is installed. + +Despite its name, "aggregate6" supports both IPv4 and IPv6 addresses and is written in Python, whereas the "aggregate" utility supports only IPv4 addresses and is written in C and uses significantly less memory but runs much slower. For greatly improved performance, aggregate6 can be run with Pypy. If both utilities are installed, the C based "aggregate" utility will be preferred for IPv4 aggregation, but the script will happily function if only one or the other is installed (or neither). Note: Subnet aggregation can be enabled with the '-A' switch on the commandline. - * "aggregate" can be installed via: # pkg_add aggregate - * "aggregate6" can be installed via: # pkg_add aggregate6 + - "aggregate" can be installed via: # pkg_add aggregate + - "aggregate6" can be installed via: # pkg_add aggregate6 - * For greatly improved aggregation performance, run aggregate6 with Pypy: - # pkg_add pypy - # pypy -m pip install wheel aggregate6 - # ln -s /usr/local/pypy/bin/aggregate6 /usr/local/bin/aggregate6 - - * The experimental aggregator "aggy" can be installed like so: - ... - # pkg_add go - $ ftp https://geoghegan.ca/pub/aggy/0.1/aggy.go - $ go build aggy.go - # install -m 755 -o root -g bin aggy /usr/local/bin/aggy - ... - - -2) If you would like to update pf-badhost blocklists at a more frequent interval - you can use a modified cron job: - --- - It is essential that you make use of the crontab randomization feature to - ensure use of pf-badhost remains respecful of blocklist provider resources. - --- - The following example runs pf-badhost every 3 hours at a random minute - (this avoids flooding the server with traffic at exactly XX:00) - --- - Run pf-badhost every 3 hours Example: (you can change '/3' to '/4' to run every 4 hours) - Note: Use of the '-s' switch prevents 2 instances of pf-badhost from being run - at the same time - which is a good thing. - - # crontab -u _pfbadhost -e - ... - ~ */3 * * * -s pf-badhost -O openbsd - ... + - For greatly improved aggregation performance, run aggregate6 with Pypy: + ``` +pkg_add pypy +pypy -m pip install wheel aggregate6 +ln -s /usr/local/pypy/bin/aggregate6 /usr/local/bin/aggregate6 +``` diff --git a/pf-badhost.sh b/pf-badhost.sh index 8f983f0..723052a 100644 --- a/pf-badhost.sh +++ b/pf-badhost.sh @@ -17,7 +17,7 @@ # Copyright (c) Jez Caudle 2024 onwards -# The original version worked across different BSDs. This is only tested on OpenBSD. All the compatability code has been removed as I don't have the resources to test anywhere else. +# The original version worked across different BSDs. This is only tested on OpenBSD. All the comparability code has been removed as I don't have the resources to test anywhere else. # This script downloads some of the most popular IP Blocklists, but you can add # any lists you like. @@ -57,7 +57,7 @@ set -ef #-o pipefail # This is only tested with OpenBSD and 'ksh' # HTTP user agent override (Pretend to be Firefox on Win10 by default) -_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0" +_AGENT="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0" # Enable Logging to /var/log/pf-badhost/ _LOG=1