Recent Changes

Remove start.sh to track EPEL repository, use daemonize as replacement in RPMs using pymilter. Allow ACCEPT as untrapped exception policy. Support optional dir for getaddrset and getaddrdict in Milter.config to reduce config clutter. Show registered milter name in untrapped exception message. Include selinux subpackage (FIXME: selinux subpackage should not care about pymilter version). Support sqlite3 for greylisting, and provide Milter.greylist export and Milter.greysql import to migrate data.

0.9.8

Add Milter.test module for unit testing milters. Fix typo that prevented setsymlist from being active. Change untrapped exception message to: "pymilter: untrapped exception in milter app"

0.9.7

Raise RuntimeError when result != CONTINUE for @noreply and @nocallback decorators. Remove redundant table in miltermodule (low level change).

0.9.6

Raise ValueError on unescaped '%' passed to setreply (setreply arg is ultimately passed to printf by libmilter, usually resulting in a coredump if it contains % escapes).

0.9.5

Print milter.error for invalid callback return type. (Since stacktrace is empty, the TypeError exception is confusing.) Fix milter-template.py. It is not in the test suite and stopped working at some point - not good for example code.

0.9.4

Handle IP6 in Milter.utils.iniplist(). Support (and require for RPM packages) python-2.6.

0.9.3

Handle source routes in Milter.util.parse_addr(). Fix unitialized optional arg in chgfrom(). Disable negotiate callback when libmilter < 8.14.3 (runtime API version 1.0.1)

0.9.2

Change result of @noreply callbacks to NOREPLY when so negotiated. Cache callback negotiation. Add new callback support: data,negotiate,unknown. Auto-negotiate protocol steps. Fix missing address of optional param to addrcpt().

0.9.0

Spec file change for Fedora: stop using INSTALLED_FILES to make Fedora happy, remove config flag from start.sh glue, own /var/log/milter, use _localstatedir.

pymilter-0.9.0 is the first version after separating milter and pymilter. This will allow easier reuse by other projects using pymilter to wrap libmilter. In addition, we now support chgfrom and addrcpt_par in the milter API. NS records are now supported by Milter.dns. I suspect that it might be useful to track reputation by nameserver to fight throwaway domains.

Recent Changes for the `milter` project

0.8.18-2

Default logdir to datadir for compatibility with old configs. Add logdir to sample config.

0.8.18-1

test cases and bug fixes for spfmilter.
configure untrapped_exception policy for spfmilter.
reject numeric HELO for spfmilter.
from_words from file feature for bms milter
ban mailbox, not entire domain, for configured email_providers
Use pysqlite (included in python) for greylist database
banned_domains and ips moved back to logdir for bms milter
straighten out datadir vs logdir for bms milter

0.8.17-2

Include logrotate for dkim-milter.

0.8.17-1

report keysize of DKIM signatures.
simple DKIM milter as another sample
basic DKIM signing support
Implement DKIM policy in access file.
Parse Authentication-Results header to get dkim result for feedback mail.
Let DKIM confirm domain for missing or neutral SPF result.

0.8.16

Experimental DKIM support. Reference templated URLs in error messages.

0.8.15

Support (and require for RPM packages) Python2.6.

0.8.14

Ignore zero length keywords (from_words, porn_words) - a disastrous typo. Ban generic domains for common subdomains. Allow illegal HELO from internal network for braindead copiers. Don't ban for multiple anonymous MFROM. Trust localhost not to be a zombie - sendmail sends from queue on localhost. Ban domains on best_guess pass.

0.8.13

Default internal_policy to off. Experimental banned domain list. Block DSN from internal connections, except for listed internal MTAs. BAN policy in access file bans connect IP. Use DATA callback to improve SRS check.

0.8.12

Use the pid file in the initscript. Fix bugs with greylisting config and adjust demerits for HELO fail. Add an SPF Pass policy. Can be used to ban a domain.

0.8.11

Greylisting is now supported. Messages from the 'vacation' program are now recognized as autoreplies. IPs of trusted relays (secondary MXes, for instance) are never banned. Added ban2zone.py to convert banned IP lists to BIND zonefile data.

0.8.10

SRS rejections now log the recipient. I have finally implemented plain CBV (no DSN). The CBV policy will do a plain CBV from now on, and the DSN policy is required if you want to send a DSN. I started checking the MAIL FROM fullname (human readable part of an email) for porn keywords. There is now a banned IP database. IPs are banned for too many bad MAIL FROMs or RCPT TOs, and remain banned for 7 days.

0.8.9

I use the %ifarch hack to build milter and milter-spf packages as noarch, while pymilter is built as native. I removed the spf dependency from dsn.py, so pymilter can be used without installing pyspf, and added a Milter.dns module to let python milters do general DNS lookups without loading pyspf.

0.8.8

Programs do not belong in the /var/log directory. I moved the milter apps to /usr/lib/pymilter. Since having the programs and data in the same directory is convenient for debugging, it will still use an executable present in the datadir. Several general utility classes and functions are now in the Milter package for possible use by other python milters. In addition to the trivial example milter, a simple SPF only milter is included as a realistic example. The spec file now build 3 RPMs:

pymilter is the milter module and Milter package for use by all python milters.
milter is the all-singing, all-dancing python milter application, with supporting /etc/init.d, logrotate and other scripts.
milter-spf is the simple SPF only milter application.

0.8.7

The spf module has been moved to the pyspf package. Download here.

0.8.6

Python milter has been moved to pymilter Sourceforge project for development and release downloads.

0.8.5

Release 0.8.5 fixes some build bugs reported by Stephen Figgins. It fixes many small things, like not auto-whitelisting recipients of outgoing mail when the subject contains "autoreply:". There is a simple trusted forwarder implementation. If you have more than 2 or so forwarders, we will need a way to "compile" SPF records into an IP set and TTL for it to be efficient (like libspf2 does).

GOSSiP

An alpha release of pygossip has been commited to CVS, module pygossip. A version of the bms.py milter has been commited to CVS which supports calling GOSSiP to track domain reputation in a local database.

New website design

Hey, I'm no artist, so I just used the ht2html package by Barry Warsaw. The mascot is by Christian Hafner, or maybe his wife. I chose Maxwell's daemon because it tirelessly and invisibly sorts molecules, just as milters sort mail. Christian has also provided a fun simulation that lets you try your hand at sorting molecules.

0.8.4

Release 0.8.4 makes configuring SPF policy via access.db actually work. The honeypot idea is enhanced by auto-whitelisting recipients of email sent from selected domains. Whitelisted messages are then used to train the honeypot. This makes the honeypot screener entirely self training. The smfi_progress() API is now automatically supported when present. An optional idx parameter to milter.addheader() invokes smfi_insheader().

0.8.3

Release 0.8.3 uses the standard logging module, and supports configuring more detailed SPF policy via the sendmail access map. SMTP AUTH connections are considered INTERNAL. Preventing forgery between internal domains is just a matter of specifying the user-domain map - I'll define something for the next version. We now send DSNs when mail is quarantined (rejecting if DSN fails) and for SPF syntax errors (PermError). There is an experimental option to add a Sender header when it is missing and the From domain doesn't match the MAIL FROM domain. Next release, we may start renaming and replacing an existing Sender header when neither it nor the From domain matches MAIL FROM. Since bogus MAIL FROMs are rejected (to varying degrees depending on the configured SPF policy), and both Sender and From and displayed by default in many email clients, this provides some phishing protection without rejecting mail based on headers.

0.8.2

Release 0.8.2 has changes to SPF to bring it in line with the newly official RFC. It adds SES support (the original SES without body hash) for pysrs-0.30.10, and honeypot support for pydspam-1.1.9. There is a new method in the base milter module. milter.set_exception_policy(i) lets you choose a policy of CONTINUE, REJECT, or TEMPFAIL (default) for untrapped exceptions encountered in a milter callback.

0.8.0

Release 0.8.0 is the first Sourceforge release. It supports Python-2.4, and provides an option to accept mail that gets an SPF softfail or fails the 3 strikes rule, provided the alleged sender accepts a DSN explaining the problem. Python-2.3 is no longer supported by the reworked mime.py module, although API changes could be backported. There are too many incompatible changes to the python email package.

Older Releases

Release 0.7.2 tightens the authentication screws with a "3 strikes and you're out" policy. A sender must have a valid PTR, HELO, or SPF record to send email. Specific senders can be whitelisted using the "delegate" option in the spf configuration section by adding a default SPF record for them. The PTR and HELO are required by RFC anyway, so this is not an unreasonable requirement. There is now a coherent policy for an SPF softfail result. A softfail is accepted if there is a valid PTR or HELO, or if the domain is listed in the "accept_softfail" option of the spf configuration section. A neutral result is accepted by default if there is a valid PTR or HELO, (and the SPF record was not guessed), unless the domain is listed in the "reject_neutral" option. Common forms of PTR records for dynamic IPs are recognized, and do not count as a valid PTR. This does not prevent anyone from sending mail from a dynamic IP - they just need to configure a valid HELO name or publish an SPF record.

As SPF adoption continues to rise, forged spam is not getting through. So spammers are publishing their SPF records as predicted. The 0.7.2 RPM now provides the rhsbl sendmail hack so that spammer domains can be blacklisted. With the RPM installed, add a line like the following to your sendmail.mc.

HACK(rhsbl,`blackholes.example.com',"550 Rejected: " $&{RHS} " has been spamming our customers.")dnl

Of course, spammers are now starting to register throwaway domains. The next thing we need is a custom DNS server, in Python, that can recognize patterns. For instance, one spammer registers ded304.com, ded305.com, ded306.com, etc. We also need the custom DNS server to let SPF classic clients check SES (which will be part of pysrs). The Twisted Python framework provides a custom DNS server - but I would like a smaller implementation for our use.

The RPM for release 0.7.0 moves the config file and socket locations to /etc/mail and /var/run/milter respectively. We now parse Microsoft CID records - but only hotmail.com uses them. They seem to have applied for a patent on the brilliant idea of examining the mail headers to see who the message is from. We aren't doing that here, so not to worry - but I am not a lawyer, so if you are worried, change spf.py around line 626 to return None instead of calling CIDParser(). There is a new option to reject mail with no PTR and no SPF.

Microsoft is pushing an anti-opensource license for their pending patent along with their sender-ID proposal before the IETF. It is royalty free - but requires anyone distributing a binary they've compiled from source to sign a license agreement. The Apache Software Foundation explains the problem with sender-ID, and Debian concurs. Since the Microsoft license is incompatible with free software in general and the GPL in particular, Python milter will not be able to implement sender-ID in its current form. This was, no doubt, Microsoft's intent all along.

Sender-ID attempts to do for RFC2822 headers what SPF does for RFC2821 headers. Unlike SPF, it has never been tried, and is encumbered by a stupid patent. I recommend ignoring it and continuing to implement and improve SPF until a working and unencumbered proposal for RFC2822 headers surfaces.

Release 0.6.6 adds support for SPF, a protocol to prevent forging of the envelope from address. SPF support requires pydns. The included spf.py module is an updated version of the original 1.6 version at wayforward.net. The updated version tracks the draft RFC and test suite.

The FAQ addresses how to get started with SPF.

Release 0.6.1 adds a full milter based dspam application.

I have selected the dspam bayes filter project and packaged it for python. Release 0.6.0 offers a simple application of dspam I call "header triage", which rejects messages with spammy headers. To use header triage, you must have DSPAM installed, and select a dictionary that is well moderated by someone who gets lots of spam. That dictionary can be used to block spam that is obvious from the headers (e.g. X-Mailer and Subject) before it ties up any more resources. I have yet to see any false positives from this approach (check the milter log), but if there are, the sender will get a REJECT with the message "Your message looks spammy."

Recent Changes for the pymilter project

1.0.2

1.0