CVS commit: pkgsrc/mail/spamdyke

["Amitai Schlair" <schmonz%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   schmonz
Date:           Wed Jan 29 20:22:20 UTC 2014

Modified Files:
        pkgsrc/mail/spamdyke: Makefile PLIST distinfo

Log Message:
Update to 5.0.0. From the changelog:

Rearranged the test scripts to put them in folders by category.  This just
  makes the directory listing a little more manageable.
Corrected some typos in the README file.  Thanks to John Mendoza for reporting
  those.
Fixed a very obscure bug in spamdyke_log(): on Linux systems (possibly only
  64-bit systems), vsyslog() occasionally will not print all the variable
  arguments.  One way was found to trigger this behavior -- when the
  rdns-blacklist-dir filter is activated from a configuration directory.
Fixed a bug in find_domain() that could cause segfaults when parsing certain
  invalid formats.  Thanks to Gary Gendel for reporting this one.
Added a backup/restore feature to the "run" script in the "tests" folder to
  save a copy of the most critical system and qmail files before running any
  scripts.  This is needed because some of the scripts alter those files and,
  if they don't run correctly or are cancelled, the originals are lost.
Added a "-skipcompile" flag to the "run" script in the "tests" folder to skip
  reconfiguring and recompiling all of the binaries when the script is run.
Changed the "run" script in the "tests" folder to empty qmail's queue before
  and after the tests are run.
Changed the "run" script in the "tests" folder to compare the current system
  and qmail configuration files to the latest backup after every script
  finishes.  If they don't match, the latest backup is restored.  If they
  still don't match, the script stops with an error.
Changed nihdns_query() to accept an optional "preferred" type of response.  If
  multiple types are queried, it will wait for at least one timeout period for
  an answer of that type to arrive instead of always accepting the first
  answer to arrive.  It will accept a saved answer before resending the
  queries, however.
NOT BACKWARDS COMPATIBLE: Changed nihdns_mx() to prefer an MX record over an A
  record, if both exist.  Given the choice, the MX record will be checked for
  validity and the A record will be ignored.  Thanks to Bruce Schreiber for
  suggesting this one.
Fixed filter_level() and smtp_filter() to disregard whitelisting and require
  authentication if the "filter-level" option is set to "require-auth", as the
  documentation says it should.  Thanks to Arne for reporting this one.
Changed nihdns_create_packet() to strip trailing dots from names before using
  them in DNS queries.  A trailing dot is the traditional way to tell libc's
  resolver not to append the local domain name and many sysadmins expect to
  have to use it.  Since spamdyke never appends the local domain and doesn't
  use libc's resolver, it isn't necessary and causes lookups to fail.  Thanks
  to Dossy Shiobara for reporting this one.
Changed middleman() to always send a "STARTTLS" response to "EHLO" as a
  continuation, never as the last line (only when spamdyke is inserting
  "STARTTLS").  This works around a bug in the Android mail client, which only
  looks for "STARTTLS" as a continuation.  Thanks to Jonas Pasche for writing
  about how to work around this bug on his blog.
NOT BACKWARDS COMPATIBLE: Changed the meaning of "whitelisted" to only exempt
  the connection from spamdyke's spam filters; whitelisting no longer allows
  the connection to relay mail.  This means spamdyke will now only set the
  RELAYCLIENT environment variable if the "relay-level" option is set to
  "allow-all".  Relaying must now be controlled through tcpserver or xinetd.
  Many thanks to Eric Shubert for suggesting and debating this with me.
NOT BACKWARDS COMPATIBLE: Removed the "access-file" and
  "rejection-text-access-denied" options because they were only needed for
  controlling relaying.  Also removed the test scripts that exercised them and
  modified many other test scripts that used them.
NOT BACKWARDS COMPATIBLE: Removed the "no-check" value from the "relay-level"
  option and changed the meaning of the "normal" value to use the logic
  previously assigned to "no-check".
Added the option "reject-sender" to take multiple values.  If the value
  "not-local" is given, the sender will be rejected if the domain name is not
  hosted locally.  If the value "authentication-mismatch" is given, the sender
  will be rejected if the sender address does not exactly match the username
  given during authentication (or if the authentication username is not an
  email address, the sender username must match the authentication username).
  If the value "authentication-domain-mismatch" is given, the sender will be
  rejected if the domain name is not part of the username given during
  authentication.  Thanks to Mark Frater for suggesting this one.
Added the options "rejection-text-sender-not-local" and
  "rejection-text-sender-authentication-mismatch" to set the rejection text
  given when the "reject-sender" option's filters are triggered.
NOT BACKWARDS COMPATIBLE: Removed the option "reject-missing-sender-mx" and
  folded its filter into the "reject-sender" filter's "no-mx" option.
NOT BACKWARDS COMPATIBLE: Renamed the option
  "rejection-text-missing-sender-mx" to "rejection-text-sender-no-mx".
NOT BACKWARDS COMPATIBLE: Renamed the option
  "reject-identical-sender-recipient" to "reject-recipient" with the value
  "same-as-sender".  The functionality remains the same.
NOT BACKWARDS COMPATIBLE: Renamed the option
  "rejection-text-identical-sender-recipient" to
  "rejection-text-recipient-same-as-sender".
NOT BACKWARDS COMPATIBLE: Renamed the option "local-domains-file" to
  "qmail-rcpthosts-file".  The naming has always been confusing, since qmail
  distinguishes between domains that should be accepted by qmail-smtpd during
  SMTP (rcpthosts) and domains that are actually hosted locally with mailboxes
  on the local filesystem (locals).  These options have always meant the
  former, but now that spamdyke needs to know both lists of domains, it's time
  to rename them.  This option is also now allowed in configuration
  directories.
NOT BACKWARDS COMPATIBLE: Removed the option "local-domains-entry" because
  supplying domains that can be accepted during SMTP to spamdyke only (but
  not qmail) will cause inconsistent results during recipient validation.
  If a domain is to be accepted during SMTP, it should be added to the control
  files used by both spamdyke and qmail.
Added CDB searching code in cdb.[ch] to read DJB's "constant database" files
  during recipient validation.  The format of these files is claimed (by DJB)
  to be fast and efficient.  Don't believe the hype...
Added the option "qmail-morercpthosts-cdb" to allow CDB files to be provided
  that contain lists of domains for which mail should be accepted during SMTP.
  Does anyone actually use this qmail "feature"?
Poured over qmail's documentation and source code to figure out exactly how
  it determines where to deliver a message.  The documentation is frequently
  in error and extensive testing was required to discover the truth.  The
  resulting procedure is encapsulated in a flowchart in the documentation
  folder.
Added the "generator" program to create test scripts to check every possible
  path through the recipient validation flowchart, both with spamdyke in place
  and without (to check the flowchart is correct).  A program to generate the
  scripts was required, since there are nearly 250K possible paths to test.
Added the value "invalid" to the option "reject-recipient" to check if a local
  recipient address exists before accepting a message.  This validation
  process uses the same logic as qmail when deciding whether/where to deliver
  a message, so no extra steps are needed to make this work (e.g. maintaining
  a list of valid addresses in a separate file).  If this process determines
  a local address is valid, delivery is guaranteed.  This option should
  eliminate qmail's habit of sending backscatter spam.
Added the value "unavailable" to the option "reject-recipient" to check if a
  local recipient is accepting mail at the moment.  Probably as a holdover
  from the elder days when people actually edited .qmail files by hand, qmail
  checks file permissions on files and folders before delivering a message.
  If they are set to certain values, qmail will queue the message until the
  permissions are fixed or bounce the message if is queued too long.  In these
  enlightened times, such permissions are more likely to be due to an error or
  oversight than deliberate intent.
Added the options "qmail-assign-cdb", "qmail-defaultdelivery-file",
  "qmail-envnoathost-file", "qmail-locals-file", "qmail-me-file",
  "qmail-percenthack-file" and "qmail-virtualdomains-file" to allow spamdyke
  to use different control files than qmail.  It's very unlikely anyone will
  ever need these options (and it would be unwise to use them), but they're
  available just in case.
Added the option "rejection-text-recipient-invalid" to set the rejection text
  when the "invalid" filter on "reject-recipient" is triggered.
Added the option "rejection-text-recipient-unavailable" to set the rejection
  text when the "unavailable" filter on "reject-recipient" is triggered.
Removed the function filter_recipient_local() and moved its logic into
  filter_recipient_valid().
Removed the function filter_recipient_relay() and moved its logic into
  filter_recipient_valid().
Changed the "help" option to just show a listing of available options without
  help text.
Added the "more-help" option to show the full listing of options with all help
  text.
Added the options "ip-relay-entry", "ip-relay-file", "rdns-relay-entry" and
  "rdns-relay-file" to allow relaying from specific IPs and/or rDNS names,
  since whitelisting no longer implies the ability to relay.  If any of these
  options are matched, the RELAYCLIENT variable will be set before qmail is
  started.
Created the "create_cdb" program to generate CDB files of arbitrary size,
  filled with random data, for testing spamdyke's CDB validation routines.
  create_cdb also has the ability to corrupt the generated CDB in seven ways;
  this makes for more specific testing than simply using a file of random
  garbage.
Removed all uses of the TESTSD_* environment variables from the test scripts
  and replaced them with appropriate invocations of dnsdummy.  This allows the
  test scripts to run without potential interference from external DNS
  changes and without needing a running spamdyke server to find example
  values.
Fixed smtp_filter() and middleman() to clear the list of saved recipient
  addresses after printing the log messages.  This prevents duplicate log
  messages when multiple email messages are delivered in the same connection.
  Thanks to Teodor Milkov and David Davidov for reporting this one.
Added the "-skippatched" and "-skipunpatched" flags to the "run" scripts to
  skip any tests that require a patched or unpatched version of qmail,
  respectively.
Fixed a minor bug in find_username() that would truncate the last character
  of the username when no domain is given.  This hasn't been a problem since
  spamdyke rejects recipient addresses without domain names anyway, but one
  of the recipient validation test scripts found it.
Added the option "tls-dhparams-file" option to read DH params from a file
  for creating ephemeral keys during SSL/TLS key negotiation.  Thanks to
  Marc Gregel for suggesting this one.
Changed all error messages to output the filename, function name and line
  number that generated them, just like the debug and excessive messages.
Added a new log level, LOG_LEVEL_CONFIG_TEST, for config-test error messages.
  The level is treated much the same as LOG_LEVEL_ERROR except the filename,
  function name and line numbers are not printed.
Added a new decision level, FILTER_DECISION_AUTHENTICATED for authenticated
  connections.  The filter routines use this level to distinguish between
  connections that should be unfiltered due to authentication versus
  whitelisting.
Added a new config option type: CONFIG_TYPE_ALIAS.  Options of this type are
  aliases for other options.  This eliminates the duplication of values and
  potential for oversights in the graylist/greylist options.
Added some code to the "run" script in the "tests" directory to try to detect
  core dumps.  Some of the tests will declare success even if spamdyke
  segfaults and cuts off the output prematurely.
Removed the unused functions reset_rejection() and skip_cfws().
Discovered spamdyke cannot read all the files it needs for recipient
  validation during normal operation because they are owned by different users
  with restrictive permissions and spamdyke does not run as root.  I'm not
  sure how I missed that, but it completely moots more than a year of work.
Moved all the recipient valiation code into an external program named
  "spamdyke-qrv".  This program is meant to only perform recipient validation
  and nothing else, so it should be safe to run as root (at least safer than
  running spamdyke as root).
Removed the options "qmail-assign-cdb", "qmail-defaultdelivery-file",
  "qmail-envnoathost-file", "qmail-locals-file", "qmail-me-file" and
  "qmail-percenthack-file" from spamdyke, since the recipient validation code
  is gone.
Added the option "recipient-validation-command" for passing the path to
  spamdyke-qrv, which will be called when recipient validation is needed.


To generate a diff of this commit:
cvs rdiff -u -r1.31 -r1.32 pkgsrc/mail/spamdyke/Makefile
cvs rdiff -u -r1.7 -r1.8 pkgsrc/mail/spamdyke/PLIST
cvs rdiff -u -r1.28 -r1.29 pkgsrc/mail/spamdyke/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.