[pkgsrc/trunk]: pkgsrc/net/rsync rsync: updated to 3.2.1

[adam <adam%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/27f469368c41
branches:  trunk
changeset: 435069:27f469368c41
user:      adam <adam%pkgsrc.org@localhost>
date:      Tue Jun 30 11:56:14 2020 +0000

description:
rsync: updated to 3.2.1

NEWS for rsync 3.2.1 (22 Jun 2020)

Protocol: 31 (unchanged)

Changes since 3.2.0:

BUG FIXES:

Fixed a potential build issue with the MD5 assembly-language code by removing some non-portable directives.

Use the preprocessor with the asm file to ensure that if the code is unneeded, it doesn't get built.

Avoid the stack getting set to executable when including the asm code.

Some improvements in the SIMD configure testing to try to avoid build issues, such as avoiding a clang++ core dump when ‑g is combined with ‑O2. Note that clang++ is quite buggy in this area, and it 
does still crash for some folks, so just use ‑‑disable-simd if you need to avoid their buggy compiler (since the configure test is apparently not finding all the compilers that will to crash and 
burn).

Fixed an issue in the md2man script when building from an alternate dir.

Disable ‑‑atimes on macOS (it apparently just ignores the atime change).

ENHANCEMENTS:

The use of ‑‑backup-dir=STR now implies ‑‑backup.

Added ‑‑zl=NUM as a short-hand for ‑‑compress-level=NUM.

Added ‑‑early-input=FILE option that allows the client to send some data to a daemon's (optional) "early exec" script on its stdin.

Mention atimes in the capabilities list that ‑‑version outputs.

Mention either "default protect-args" or "optional protect-args" in the ‑‑version capabilities depending on how rsync was configured.

Some info on optimizations is now elided from the ‑‑version capabilities since they aren't really user-facing capabilities. You can still see the info (plus the status of a couple extra 
optimizations) by repeating the ‑‑version option (e.g. ‑VV).

Updated various URLs to be https instead of http.

Some documentation improvements.

PACKAGING RELATED:

If you had to use ‑‑disable-simd for 3.2.0, you might want to try removing that and see if it will succeed or auto-disable. Some buggy clang++ compilers are still not auto disabled, though.

The MD5 asm code is now under its own configure flag (not shared with the SIMD setting), so if you have any issues compiling it, re-run configure with ‑‑disable-asm.

Merged the OLDNEWS.md file into NEWS.md.



NEWS for rsync 3.2.0 (19 Jun 2020)

Protocol: 31 (unchanged)

Changes since 3.1.3:

BUG FIXES:

Avoid a potential out-of-bounds read in daemon mode if argc can be made to become 0.

Fix the default list of skip-compress files for non-daemon transfers.

Fix xattr filter rules losing an 'x' attribute in a non-local transfer.

Avoid an error when a check for a potential fuzzy file happens to reference a directory.

Make the atomic-rsync helper script have a more consistent error-exit.

Make sure that a signal handler's use of exit_cleanup() calls _exit() instead of exit().

Various zlib fixes, including security fixes for CVE-2016-9843, CVE-2016-9842, CVE-2016-9841, and CVE-2016-9840.

Fixed an issue with ‑‑remove-source-files not removing a source symlink when combined with ‑‑copy-links.

Fixed a bug where the daemon would fail to write early fatal error messages to the client, such as refused or unknown command-line options.

Fixed the block-size validation logic when dealing with older protocols.

Some rrsync fixes and enhancements to handle the latest options.

Fixed a problem with the ‑‑link-dest|‑‑copy-dest code when ‑‑xattrs was specified along with multiple alternate-destination directories (it could possibly choose a bad file match while trying to find 
a better xattr match).

Fixed a couple bugs in the handling of files with the ‑‑sparse option.

Fixed a bug in the writing of the batch.sh file (w/‑‑write-batch) when the source & destination args were not last on the command-line.

Avoid a hang when an overabundance of messages clogs up all the I/O buffers.

Fixed a mismatch in the RSYNC_PID values put into the environment of pre-xfer exec and a post-xfer exec.

Fixed a crash in the ‑‑iconv code.

Fixed a rare crash in the popt_unalias() code.

ENHANCEMENTS:

Various checksum enhancements, including the optional use of openssl's MD4 & MD5 checksum algorithms, some x86-64 optimizations for the rolling checksum, some x86-64 optimizations for the 
(non-openssl) MD5 checksum, the addition of xxHash checksum support, and a negotiation heuristic that ensures that it is easier to add new checksum algorithms in the future. The environment variable 
RSYNC_CHECKSUM_LIST can be used to customize the preference order of the negotiation, or use ‑‑checksum-choice (‑‑cc) to force a choice.

Various compression enhancements, including the addition of zstd and lz4 compression algorithms and a negotiation heuristic that picks the best compression option supported by both sides. The 
environment variable RSYNC_COMPRESS_LIST can be used to customize the preference order of the negotiation, or use ‑‑compress-choice (‑‑zc) to force a choice.

Added a ‑‑debug=NSTR option that outputs details of the new negotiation strings (for checksums and compression). The first level just outputs the result of each negotiation on the client, level 2 
outputs the values of the strings that were sent to and received from the server, and level 3 outputs all those values on the server side too (when the server was given the debug option).

The ‑‑debug=OPTS command-line option is no longer auto-forwarded to the remote rsync which allows for the client and server to have different levels of debug specified. This also allows for newer 
debug options to be specified, such as using ‑‑debug=NSTR to see the negotiated hash result, without having the command fail if the server version is too old to handle that debug item. Use 
‑M‑‑debug=OPTS to send the options to the remote side.

Added the ‑‑atimes option based on the long-standing patch (just with some fixes that the patch has been needing).

Added ‑‑open-noatime option to open files using O_NOATIME.

Added the ‑‑write-devices option based on the long-standing patch.

Added openssl & preliminary gnutls support to the rsync-ssl script, which is now installed by default. This was unified with the old stunnel-rsync helper script to simplify packaging. Note that the 
script accepts the use of ‑‑type=gnutls for gnutls testing, but does not look for gnutls-cli on the path yet. The use of ‑‑type=gnutls will not work right until gnutls-cli no longer drops data.

Rsync was enhanced to set the RSYNC_PORT environment variable when running a daemon-over-rsh script. Its value is the user-specified port number (set via ‑‑port or an rsync:// URL) or 0 if the user 
didn't override the port.

Added the proxy protocol daemon parameter that allows your rsyncd to know the real remote IP when it is setup behind a proxy.

Added negated matching to the daemon's refuse options setting by using match strings that start with a ! (such as !compress*). This lets you refuse all options except for a particular approved list, 
for example. It also lets rsync refuse certain options by default (such as write-devices) while allowing the config to override that, as desired.

Added the early exec daemon parameter that runs a script before the transfer parameters are known, allowing some early setup based on module name.

Added status output in response to a signal (via both SIGINFO & SIGVTALRM).

Added ‑‑copy-as=USER option to give some extra security to root-run rsync commands into/from untrusted directories (such as backups and restores).

When resuming the transfer of a file in the ‑‑partial-dir, rsync will now update that partial file in-place instead of creating yet another tmp file copy. This requires both sender & receiver to be 
at least v3.2.0.

Added support for RSYNC_SHELL & RSYNC_NO_XFER_EXEC environment variables that affect the early, pre-xfer, and post-xfer exec rsync daemon parameters.

Optimize the ‑‑fuzzy ‑‑fuzzy heuristic to avoid the fuzzy directory scan until all other basis-file options are exhausted (such as ‑‑link-dest).

Have the daemon log include the normal-exit sent/received stats when the transfer exited with an error when possible (i.e. if it is the sender).

The daemon now locks its pid file (when configured to use one) so that it will not fail to start when the file exists but no daemon is running.

Various man page improvements, including some html representations (that aren't installed by default).

Made ‑V the short option for ‑‑version and improved its information.

Pass the ‑4 or ‑6 option to the ssh command, making it easier to type than ‑‑rsh='ssh ‑4' (or the ‑6 equivalent).

Added example config for rsyncd SSL proxy configs to rsyncd.conf.

More errors messages now mention if the error is coming from the sender or the receiver.

PACKAGING RELATED:

Add installed binary: /usr/bin/rsync-ssl

Add installed man page: /usr/man/man1/rsync-ssl.1

Tweak auxiliary doc file names, such as: README.md, INSTALL.md, & NEWS.md.

The rsync-ssl script wants to run openssl or stunnel4, so consider adding a dependency for one of those options (though it's probably fine to just let it complain about being unable to find the 
program and let the user decide if they want to install one or the other).

If you packaged rsync + rsync-ssl + rsync-ssl-daemon as separate packages, the rsync-ssl package is now gone (rsync-ssl should be considered to be mainstream now that Samba requires SSL for its rsync 
daemon).

Add build dependency for liblz4-dev, libxxhash-dev, libzstd-dev, and libssl-dev. These development libraries will give rsync extra compression algorithms, extra checksum algorithms, and allow use of 
openssl's crypto lib for (potentially) faster MD4/MD5 checksums.

Add build dependency for g++ or clang++ on x86_64 systems to enable the SIMD checksum optimizations.

Add build dependency for either python3-cmarkcfm or python3-commonmark to allow for patching of man pages or building a git release. This is not required for a release-tar build, since it comes with 
pre-built man pages. Note that cmarkcfm is faster than commonmark, but they generate the same data. The commonmark dependency is easiest to install since it's native python, and can even be installed 
via pip3 install ‑‑user commonmark if you want to just install it for the build user.

Remove yodl build dependency (if it was even listed before).

DEVELOPER RELATED:

Silenced some annoying warnings about major() & minor() by improving an autoconf include-file check.

Converted the man pages from yodl to markdown. They are now processed via a simple python3 script using the cmarkgfm or commonmark library. This should make it easier to package rsync, since yodl has 
gotten obscure.

Improved some configure checks to work better with strict C99 compilers.

Some perl building/packaging scripts were recoded into awk and python3.

Some defines in byteorder.h were changed into static inline functions that will help to ensure that the args don't get evaluated multiple times on "careful alignment" hosts.

Some code typos were fixed (as pointed out by a Fossies run).

diffstat:

 net/rsync/Makefile                  |  24 +++++++++++-------------
 net/rsync/PLIST                     |   6 +++---
 net/rsync/distinfo                  |  12 ++++++------
 net/rsync/patches/patch-Makefile.in |  31 +++++++++++++++++--------------
 4 files changed, 37 insertions(+), 36 deletions(-)

diffs (133 lines):

diff -r 120d1182d601 -r 27f469368c41 net/rsync/Makefile
--- a/net/rsync/Makefile        Tue Jun 30 11:10:26 2020 +0000
+++ b/net/rsync/Makefile        Tue Jun 30 11:56:14 2020 +0000
@@ -1,7 +1,6 @@
-# $NetBSD: Makefile,v 1.112 2020/01/26 17:31:55 rillig Exp $
+# $NetBSD: Makefile,v 1.113 2020/06/30 11:56:14 adam Exp $
 
-DISTNAME=      rsync-3.1.3
-PKGREVISION=   2
+DISTNAME=      rsync-3.2.1
 CATEGORIES=    net
 MASTER_SITES=  http://rsync.samba.org/ftp/rsync/
 MASTER_SITES+= http://rsync.samba.org/ftp/rsync/old-versions/
@@ -12,16 +11,13 @@
 COMMENT=       Network file distribution/synchronisation utility
 LICENSE=       gnu-gpl-v3
 
-INSTALLATION_DIRS=     bin ${PKGMANDIR}/man1 ${PKGMANDIR}/man5 share/doc/rsync
-
+USE_TOOLS+=            bash:run
 GNU_CONFIGURE=         yes
-CONFIGURE_ARGS+=       --with-included-popt
 CONFIGURE_ARGS+=       --with-rsyncd-conf=${PKG_SYSCONFDIR}/rsyncd.conf
+PKG_SYSCONFSUBDIR=     rsync
 TEST_TARGET=           test
 
-USE_TOOLS+=    perl
-
-PKG_SYSCONFSUBDIR=     rsync
+REPLACE_BASH=          rsync-ssl
 
 RCD_SCRIPTS=           rsyncd
 RCD_SCRIPT_SRC.rsyncd= files/rsyncd.sh
@@ -34,9 +30,11 @@
 SUBST_SED.paths=       -e 's|/etc/rsyncd|${PKG_SYSCONFDIR}/rsyncd|g'
 SUBST_SED.paths+=      -e 's|/usr/bin/rsync|${PREFIX}/bin/rsync|g'
 
-post-install:
-       ${INSTALL_DATA} ${WRKSRC}/README ${WRKSRC}/tech_report.tex \
-               ${DESTDIR}${PREFIX}/share/doc/rsync
-
+.include "../../archivers/lz4/buildlink3.mk"
+.include "../../archivers/zstd/buildlink3.mk"
 .include "../../converters/libiconv/buildlink3.mk"
+.include "../../devel/popt/buildlink3.mk"
+.include "../../devel/xxhash/buildlink3.mk"
+.include "../../devel/zlib/buildlink3.mk"
+.include "../../security/openssl/buildlink3.mk"
 .include "../../mk/bsd.pkg.mk"
diff -r 120d1182d601 -r 27f469368c41 net/rsync/PLIST
--- a/net/rsync/PLIST   Tue Jun 30 11:10:26 2020 +0000
+++ b/net/rsync/PLIST   Tue Jun 30 11:56:14 2020 +0000
@@ -1,6 +1,6 @@
-@comment $NetBSD: PLIST,v 1.7 2014/03/11 14:05:11 jperkin Exp $
+@comment $NetBSD: PLIST,v 1.8 2020/06/30 11:56:14 adam Exp $
 bin/rsync
+bin/rsync-ssl
+man/man1/rsync-ssl.1
 man/man1/rsync.1
 man/man5/rsyncd.conf.5
-share/doc/rsync/README
-share/doc/rsync/tech_report.tex
diff -r 120d1182d601 -r 27f469368c41 net/rsync/distinfo
--- a/net/rsync/distinfo        Tue Jun 30 11:10:26 2020 +0000
+++ b/net/rsync/distinfo        Tue Jun 30 11:56:14 2020 +0000
@@ -1,8 +1,8 @@
-$NetBSD: distinfo,v 1.48 2018/01/29 11:32:59 adam Exp $
+$NetBSD: distinfo,v 1.49 2020/06/30 11:56:14 adam Exp $
 
-SHA1 (rsync-3.1.3.tar.gz) = 82e7829c0b3cefbd33c233005341e2073c425629
-RMD160 (rsync-3.1.3.tar.gz) = 95a040e0c32e09d01f37fc7d2defd2c41a184db6
-SHA512 (rsync-3.1.3.tar.gz) = 8385f4c0ea37e7a1da3cf45794154f5bc4d1c49bc625ba3b5f85adaf3eafe6d71c15bdcb1410bde731e5d4c19aff3331606637462fa27a68dc3e13192dd78f99
-Size (rsync-3.1.3.tar.gz) = 905908 bytes
-SHA1 (patch-Makefile.in) = 14d261cc65d00864bfb0acdba1d5a8bdf989c288
+SHA1 (rsync-3.2.1.tar.gz) = b00af97012cf1cb4b567e28e89c6365a651a24c9
+RMD160 (rsync-3.2.1.tar.gz) = 5e766682e2d2ea1194cf8cb13d21caa81d3a87ba
+SHA512 (rsync-3.2.1.tar.gz) = 80d639ab56a993de8016115fe15794bb9aa7012e4a8300402954760566a821c230af9dcdb19143d70663497a830289212ce329fb0c4fd5c4b6f392d850e5fd8c
+Size (rsync-3.2.1.tar.gz) = 1051665 bytes
+SHA1 (patch-Makefile.in) = 45e47fc610428a26542abf2b0657aeaa22c27794
 SHA1 (patch-authenticate.c) = 0612fb141cea1509b882df78f1b90fa52b1092b0
diff -r 120d1182d601 -r 27f469368c41 net/rsync/patches/patch-Makefile.in
--- a/net/rsync/patches/patch-Makefile.in       Tue Jun 30 11:10:26 2020 +0000
+++ b/net/rsync/patches/patch-Makefile.in       Tue Jun 30 11:56:14 2020 +0000
@@ -1,32 +1,35 @@
-$NetBSD: patch-Makefile.in,v 1.2 2018/01/29 11:32:59 adam Exp $
+$NetBSD: patch-Makefile.in,v 1.3 2020/06/30 11:56:14 adam Exp $
 
 https://bugzilla.samba.org/show_bug.cgi?id=11594
 Fix installation.
 
---- Makefile.in.orig   2016-07-20 15:34:26.000000000 +0000
+--- Makefile.in.orig   2020-06-21 19:55:24.000000000 +0000
 +++ Makefile.in
-@@ -66,12 +66,9 @@ CHECK_OBJS=tls.o testrun.o getgroups.o g
- all: Makefile rsync$(EXEEXT) rsync-ssl stunnel-rsync stunnel-rsyncd.conf @MAKE_MAN@
- 
+@@ -76,13 +76,13 @@ all: Makefile rsync$(EXEEXT) stunnel-rsy
+ .PHONY: install
  install: all
--      -${MKDIR_P} ${DESTDIR}${bindir}
+       -${MKDIR_P} ${DESTDIR}${bindir}
 -      ${INSTALLCMD} ${INSTALL_STRIP} -m 755 rsync$(EXEEXT) ${DESTDIR}${bindir}
--      -${MKDIR_P} ${DESTDIR}${mandir}/man1
--      -${MKDIR_P} ${DESTDIR}${mandir}/man5
+-      ${INSTALLCMD} -m 755 rsync-ssl ${DESTDIR}${bindir}
++      ${BSD_INSTALL_PROGRAM} rsync$(EXEEXT) ${DESTDIR}${bindir}
++      ${BSD_INSTALL_SCRIPT} rsync-ssl ${DESTDIR}${bindir}
+       -${MKDIR_P} ${DESTDIR}${mandir}/man1
+       -${MKDIR_P} ${DESTDIR}${mandir}/man5
 -      if test -f rsync.1; then ${INSTALLMAN} -m 644 rsync.1 ${DESTDIR}${mandir}/man1; fi
+-      if test -f rsync-ssl.1; then ${INSTALLMAN} -m 644 rsync-ssl.1 ${DESTDIR}${mandir}/man1; fi
 -      if test -f rsyncd.conf.5; then ${INSTALLMAN} -m 644 rsyncd.conf.5 ${DESTDIR}${mandir}/man5; fi
-+      ${BSD_INSTALL_PROGRAM} rsync$(EXEEXT) ${DESTDIR}${bindir}
 +      if test -f rsync.1; then ${BSD_INSTALL_MAN} rsync.1 ${DESTDIR}${mandir}/man1; fi
++      if test -f rsync-ssl.1; then ${BSD_INSTALL_MAN} rsync-ssl.1 ${DESTDIR}${mandir}/man1; fi
 +      if test -f rsyncd.conf.5; then ${BSD_INSTALL_MAN} rsyncd.conf.5 ${DESTDIR}${mandir}/man5; fi
  
- install-ssl-client: rsync-ssl stunnel-rsync
-       -${MKDIR_P} ${DESTDIR}${bindir}
-@@ -209,7 +206,7 @@ proto: proto.h-tstamp
+ install-ssl-daemon: stunnel-rsyncd.conf
+       -${MKDIR_P} ${DESTDIR}/etc/stunnel
+@@ -227,7 +227,7 @@ proto: proto.h-tstamp
  proto.h: proto.h-tstamp
        @if test -f proto.h; then :; else cp -p $(srcdir)/proto.h .; fi
  
 -proto.h-tstamp: $(srcdir)/*.c $(srcdir)/lib/compat.c config.h
 +proto.h-tstamp: $(srcdir)/*.c $(srcdir)/lib/compat.c
-       perl $(srcdir)/mkproto.pl $(srcdir)/*.c $(srcdir)/lib/compat.c
+       $(AWK) -f $(srcdir)/mkproto.awk $(srcdir)/*.c $(srcdir)/lib/compat.c
  
- man: rsync.1 rsyncd.conf.5 man-copy
+ .PHONY: man