CVS commit: pkgsrc/mail/sympa

["Manuel Bouyer" <bouyer%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   bouyer
Date:           Sat Sep 26 14:56:10 UTC 2015

Modified Files:
        pkgsrc/mail/sympa: MESSAGE Makefile PLIST distinfo
        pkgsrc/mail/sympa/files: sympa.sh
        pkgsrc/mail/sympa/patches: patch-aa patch-ab patch-ac
Added Files:
        pkgsrc/mail/sympa/patches: patch-src_lib_Sympa_Session.pm
Removed Files:
        pkgsrc/mail/sympa/patches: patch-af
            patch-doc_man8_alias__manager.pod.in patch-doc_man8_archived.pod.in
            patch-doc_man8_bounced.pod.in patch-doc_man8_sympa.pod

Log Message:
Update to 6.2.7. Lots of change since 6.1.24, see
http://www.sympa.org/distribution/latest-stable/NEWS
for complete list.
Main changes are:
Tracking feature
The tracking feature is a way to request DSN or DSN + MDN when sending
a message to each subscribers. In that case, Sympa (bounced.pl) collect both
DSN and MDN and store them in a new table "notification_table".
Then, for each message, the list owner can display which subscribers has
displayed, received or not received the message. This can be used for some
important lists where list owner need to collect the proof of reception or
display of each message. This page is accessible via archive

Spools back to filesystem
Because of stability and scalability issues, spool for bulk sending no
longer depend on database tables: It is based on filesystem.

Stats
Now Sympa stores data whenever one of the following event occurs:
        a message is sent to a list;
        a user subscribed to a list;
        a user unsubscribed from a list;
        a user is added to a list by another user;
        a user is removed from a list by another user;
        a user is removed from a list by the automatic bounce management;
        a file is uploaded to the shared directory;
        a file is deleted from the shared;
        a message to a list is rejected in moderation;
        a user logs in to the web interface;
        a user logs out;
        a list is created;
        a list is deleted;
        a list is restored;
        a human user (not a harvester) hits a page.

These data are regularly aggregated by the task_manager.
The aggregated data are available to users, owners and listmaster in
their respective interfaces.

Web interface
The most visible is probably the new web interface skin.
It is nicer, simpler, responsive and retains all the customization
capabilities that make Sympa adaptable to your needs.
Exclusion table was just a display of the users excluded. This page is now
a form, similar to the review page, which allows to restore users subscriptions.

Added "Add subscriber" button to modindex page if the sender is not subscribed.
It may be useful because users need not know about SUB command and sympa
command address. The buttons appear only when Web user is the list owner.

The "Customizing" link was moved from main list admin to list config sub
section as list related text files are in a way list configuration parameters
as well.
Better password validation. When the user requests change of password via
WWSympa or SympaSOAP, new password may be checked its strength.
New parameter password_validation may be used to customize policy of password
validation. See help text of the parameter for more details.

Web services

Full VOOT 2.0 protocol support. VOOT is an opensocial subset to manage group
membership. full details on this IETF draft here: http://openvoot.org.
This feature enables Sympa to be both group provider and consumer in the VOOT
protocol.
The soap server now exposes full review with owners, editors and gecos to list masters and list owners.

Plug, plug, plug...

We added or improved several ways to plug new functionnalities in Sympa.
Template plugins

This feature allows to package code to be used in templates. It enables
foreign data integration in Sympa's UI and emails, user specific
UI enhancement and much more! Integrating user targeted data retreived through
a call to a webservice becomes possible in a simple way, automatically adding
info from a database to outgoing emails as well !
Further details here : https://www.sympa.org/manual/templates_plugins

Enhanced custom actions functionality

Custom actions are used to run specific code and/or display user defined
templates. They can be executed in list or global context (it is up to you
to decide what to do in both cases). Previously, a custom action was a
simple TT2 template added to the web interface. It could only display data,
not process them. They were improved to allow greater expressiveness.
You can now develop a perl module to process the data passed to your
custom action.

Warning: AFTER UPGRADING TO 6.2, ANY PRE-EXISTING CUSTOM ACTION MUST BE
MOVED TO THE RELEVANT CUSTOM_ACTION DIRECTORY TO KEEP WORKING.

Mail hooks

Sympa::Message::Plugin provides hook mechanism to intervene in processing by
Sympa. Each hook may modify messages or may break ordinary processing.
This functionnality is quite new and still experimental. Please read the
perldoc of Sympa::Message::Plugin for more details about how to create and enable hooks in message handling.

Custom attributes provisionning

Custom attributes can now be provisionned using external data sources,
the same way as email addresses. For now, only SQL or LDAP datasources are
supported. To use this feature, you need first to define the custom attributes
as previously. This attribute must have the same name as the fields used in
your queries.

Synchronization disabling

In several organizations, nightly maintenances make the datasources
unavailable during some period of times. a new sub-parameter in data sources
definition allow to prevent Sympa from trying to synchronize list members
with these sources during a defined time range.

A recurrent question regarding list inclusion was to be able to extract not the whole list, but only a subset. It is now possible.

include_list parameter value can now look like this:

<listname> filter <filter_condition>

ODBC is now supported for email data sources.
sql, ldap and ldap2 user inclusions now handles gecos. It uses 2nd returned
column for sql and 2nd given attribute for ldap (comma separated).
Now "host" parameter of the LDAP datasource in list config may include URLs
(ldap, ldaps and ldapi) along with hostnames. So "port" and "use_ssl"
parameters

Scenarios

Now CustomCondition can set the action to take (do_it, reject ...) by
setting $_, this allows for complex, single-module CustomConditions.
"[msg_header#field][index]" returns the value of particular field.
Index may be negative. - "[msg_header#field]" still returns list of field
values, additionally, ordering will be preserved. Fixes:
Conditions is_listmaster, is_owner, is_editor and is_subscriber can
handle multiple values.
They also parse arguments as header field values to get address parts.

New parameters

New list parameter: 'member_include'. This parameters allows to defined
.incl files to include list members, the same way it is done for list admins.
This grants extended flexibility in datasources, allowing list owners to tweak
their own data sources without accessing to the actual details of the query
made to gather email addresses.
New sympa.conf config parameter "sender_headers" to specify header fields by
which message sender is detected. This is a enhancement to
S. Shipway's improvement.
RFC: What is the reasonable default for this parameter?
Behavior of one time ticket lock-out is configurable.
Two new robot parameters were introduced:
one_time_ticket_lockout:
"one_time" won't allow access again. This is previous behavior.
"remote_addr" will lock the ticket on remote address accessed at first time.
"open" will never lock-out tickets.
one_time_ticket_lifetime: Duration before ticket expires. Default is "2d", previous hardcoded value.
Changes of list config parameters on archiving. New parameter
"process_archive" controls archiving. The default is "off":
To enable archiving, it must be set to "on" explicitly. OTOH even if
archiving is disabled by setting it "off", past archives will be kept
and accessible according to access settings.

Config files

wwsympa.conf will no longer be used: it would be migrated to sympa.conf.

Alias management uniformization

Now alias maintenance utilities other than newaliases may be used without
special configure options nor patch to alias_manager.pl. Changes:

aliaswrapper and virtualwrapper were deprecated and replaced with sympa_newaliases-wrapper.
New alias management program sympa_newaliases.pl which will typically be
called by alias_manager.pl via sympa_newaliases-wrapper.
New site configuration parameters aliases_db_type and aliases_program will
control behaviour of alias database maintenance.

openssl dependency dropped

openssl(1) utility is no longer required. Instead, a few CPAN modules are required to use S/MIME or SSL features:

Crypt::SMIME
Crypt::OpenSSL::X509

The parameter "openssl" in sympa.conf was obsoleted. Note that p12topem.pl still needs openssl: To make it work, PATH environment variable should be set appropriately.
Internationalization

i18n titles in configuration.

List families scenari

Now you can define a "scenari" directory in the lists family directory.
These scenarii will be available for lists instantiated from this family.
List families addistional files

Until now, only the config.tt2 file was parsed when the family was
instantiated, thus producing the lists' config file. As a lot of other files
can be used to configure a list (footer, header, etc.), we added the
possibility to instantiate these files as well. The list of files parsed
(in addition to config.tt2) when instantiating a family are defined in
a new (sympa.conf and robot.conf) parameter called 'parsed_family_files'.
This parameter must contain a comma-separated list of file names.
If these files exist, with the '.tt2' extension added to their name,
in a family, they will be parsed and added to the list directory.

The default value of the parameter is: 'message.footer,message.header,message.footer.mime,message.header.mime,info'.

These files are updated in existing lists when a family is instantiated.
note that it will overwrite any other customization.
On the web automatic lists

Administration
Widely extended logs when the log_module parameter is activated with the
value 'scenario': all scenario results are evaluated. Remember that extended
logs can be limited by the IP or the email of the user, using the
'log_condition' parameter. log_module and log_condition are now declared in
sympa.conf instead of wwsympa.conf. Remember these parameters syntax:

log_module: can take the value "scenario" - only.
log condition: takes a character string as value. The form is
"ip=<an.ip>,email=<an_email>". You can put ip alone, email alone or both,
with the comma-separated form.

Listmaster notifications can now be grouped. If, for a specific robot,
several notifications are issued within a short period of time with the
same operation code only the first ones are sent, the next ones are stacked.
The stacking ends when no notifications are issued anymore, or when the first
one is too old. All the stacked notifications are then sent as attachments of
a single message. Internal settings : Stack if more than 3 notifications with
the same operation code are issued for the same robot - Send stacked
notifications if no new one satisfying the condition above was received for
more than 30 seconds - Send stacked notifications if the oldest one was issued
more than 60 seconds ago

sympa.pl -health_check checks config files, database connection and structure,
and data structure version.
Daemons such as sympa.pl no longer check database structure nor data version.

File extension of lock files are changed from ".lock" to ".LOCK", because lowercase name can conflict with domain part in the future.


To generate a diff of this commit:
cvs rdiff -u -r1.4 -r1.5 pkgsrc/mail/sympa/MESSAGE
cvs rdiff -u -r1.59 -r1.60 pkgsrc/mail/sympa/Makefile
cvs rdiff -u -r1.13 -r1.14 pkgsrc/mail/sympa/PLIST
cvs rdiff -u -r1.16 -r1.17 pkgsrc/mail/sympa/distinfo
cvs rdiff -u -r1.1 -r1.2 pkgsrc/mail/sympa/files/sympa.sh
cvs rdiff -u -r1.7 -r1.8 pkgsrc/mail/sympa/patches/patch-aa
cvs rdiff -u -r1.6 -r1.7 pkgsrc/mail/sympa/patches/patch-ab
cvs rdiff -u -r1.3 -r1.4 pkgsrc/mail/sympa/patches/patch-ac
cvs rdiff -u -r1.3 -r0 pkgsrc/mail/sympa/patches/patch-af
cvs rdiff -u -r1.2 -r0 \
    pkgsrc/mail/sympa/patches/patch-doc_man8_alias__manager.pod.in \
    pkgsrc/mail/sympa/patches/patch-doc_man8_archived.pod.in \
    pkgsrc/mail/sympa/patches/patch-doc_man8_bounced.pod.in \
    pkgsrc/mail/sympa/patches/patch-doc_man8_sympa.pod
cvs rdiff -u -r0 -r1.1 \
    pkgsrc/mail/sympa/patches/patch-src_lib_Sympa_Session.pm

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.