[pkgsrc/trunk]: pkgsrc/databases/redis redis: updated to 6.2.0

[adam <adam%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/d851ce852604
branches:  trunk
changeset: 447964:d851ce852604
user:      adam <adam%pkgsrc.org@localhost>
date:      Mon Mar 01 13:19:18 2021 +0000

description:
redis: updated to 6.2.0

Redis 6.2.0 GA  Released Tue Feb 22 14:00:00 IST 2021
================================================================================

Upgrade urgency: SECURITY if you use 32bit build of redis (see bellow), MODERATE
if you used earlier versions of Redis 6.2, LOW otherwise.

Integer overflow on 32-bit systems (CVE-2021-21309):
Redis 4.0 or newer uses a configurable limit for the maximum supported bulk
input size. By default, it is 512MB which is a safe value for all platforms.
If the limit is significantly increased, receiving a large request from a client
may trigger several integer overflow scenarios, which would result with buffer
overflow and heap corruption.

Bug fixes:
* Avoid 32-bit overflows when proto-max-bulk-len is set high
* Fix broken protocol in client tracking tracking-redir-broken message
* Avoid unsafe field name characters in INFO commandstats, errorstats, modules
* XINFO able to access expired keys during CLIENT PAUSE WRITE
* Fix allowed length for REPLCONF ip-address, needed due to Sentinel's support for hostnames
* Fix broken protocol in redis-benchmark when used with -a or --dbnum
* XADD counts deleted records too when considering switching to a new listpack

Bug fixes that are only applicable to previous releases of Redis 6.2:
* Fixes in GEOSEARCH bybox (accuracy and mismatch between width and height)
* Fix risk of OOM panic in HRANDFIELD, ZRANDMEMBER commands with huge negative count
* Fix duplicate replicas issue in Sentinel, needed due to hostname support
* Fix Sentinel configuration rewrite

Command behavior changes:
* SRANDMEMBER uses RESP3 array type instead of set type
* EXPIRE, EXPIREAT, SETEX, GETEX: Return error when provided expire time overflows

Other behavior changes:
* Remove ACL subcommand validation if fully added command exists.

Improvements:
* Optimize sorting in GEORADIUS / GEOSEARCH with COUNT
* Optimize HRANDFIELD and ZRANDMEMBER case 4 when ziplist encoded
* Optimize in-place replacement of elements in HSET, HINCRBY, LSET
* Remove redundant list to store pubsub patterns
* Add --insecure option to command line tools

Info fields and introspection changes:
* Add INFO fields to track progress of BGSAVE, AOFRW, replication

Modules:
* RM_ZsetRem: Delete key if empty, the bug could leave empty zset keys
* RM_HashSet: Add COUNT_ALL flag and set errno


Redis 6.2 RC3   Released Tue Feb 1 14:00:00 IST 2021
================================================================================

Upgrade urgency LOW: This is the third Release Candidate of Redis 6.2.

New commands / args:
* Add HRANDFIELD and ZRANDMEMBER commands
* Add FAILOVER command
* Add GETEX, GETDEL commands
* Add PXAT/EXAT arguments to SET command
* Add SYNC arg to FLUSHALL and FLUSHDB, and ASYNC/SYNC arg to SCRIPT FLUSH

Sentinel:
* Add hostname support to Sentinel
* Prevent file descriptors from leaking into Sentinel scripts
* Fix config file line order dependency and config rewrite sequence

New configuration options:
* Add set-proc-title config option to disable changes to the process title
* Add proc-title-template option to control what's shown in the process title
* Add lazyfree-lazy-user-flush config option to control FLUSHALL, FLUSHDB and SCRIPT FLUSH

Bug fixes:
* AOF: recover from last write error by turning on/off appendonly config
* Exit on fsync error when the AOF fsync policy is 'always'
* Avoid assertions (on older kernels) when testing arm64 CoW bug
* CONFIG REWRITE should honor umask settings
* Fix firstkey,lastkey,step in COMMAND command for some commands

Special considerations:
* Fix misleading description of the save configuration directive

Improvements:
* A way to get RDB file via replication without excessive replication buffers
* Optimize performance of clusterGenNodesDescription for large clusters

Info fields and introspection changes:
* SLOWLOG and LATENCY monitor include unblocking time of blocked commands

Modules:
* Add modules API for streams
* Add event for fork child birth and termination
* Add RM_BlockedClientMeasureTime* etc, to track background processing in commandstats
* Fix bug in v6.2, wrong value passed to the new unlink callback
* Fix bug in v6.2, modules blocked on keys unblock on commands like LPUSH


Redis 6.2 RC2   Released Tue Jan 12 16:17:20 IST 2021
================================================================================

Upgrade urgency LOW: This is the second Release Candidate of Redis 6.2.

IMPORTANT: If you're running Redis on ARM64 or a big-endian system, upgrade may
have significant implications. Please be sure to read the notes below.

New commands / args:
* Add the REV, BYLEX and BYSCORE arguments to ZRANGE, and the ZRANGESTORE command
* Add the XAUTOCLAIM command
* Add the MINID trimming strategy and the LIMIT argument to XADD and XTRIM
* Add the ANY argument to GEOSEARCH and GEORADIUS
* Add the CH, NX, XX arguments to GEOADD
* Add the COUNT argument to LPOP and RPOP
* Add the WRITE argument to CLIENT PAUSE for pausing write commands exclusively
* Change the proto-ver argument of HELLO to optional
* Add the CLIENT TRACKINGINFO subcommand

Command behavior changes:
* CLIENT TRACKING yields an error when given overlapping BCAST prefixes
* SWAPDB invalidates WATCHed keys
* SORT command behaves differently when used on a writable replica

Other behavior changes:
* Avoid propagating MULTI/EXEC for read-only transactions
* Remove the read-only flag from TIME, ECHO, ROLE, LASTSAVE
* Fix the command flags of PFDEBUG
* Tracking clients will no longer receive unnecessary key invalidation messages after FLUSHDB
* Sentinel: Fix missing updates to the config file after SENTINEL SET command

Bug fixes with compatibility implications (bugs introduced in Redis 6.0):
* Fix RDB CRC64 checksum on big-endian systems
  If you're using big-endian please consider the compatibility implications with
  RESTORE, replication and persistence.
* Fix wrong order of key/value in Lua's map response
  If your scripts use redis.setresp() or return a map (new in Redis 6.0), please
  consider the implications.

Bug fixes that are only applicable to previous releases of Redis 6.2:
* Resolve rare assertions in active defragmentation while loading

Bug fixes:
* Fix the selection of a random element from large hash tables
* Fix an issue where a forked process deletes the parent's pidfile
* Fix crashes when enabling io-threads-do-reads
* Fix a crash in redis-cli after executing cluster backup
* Fix redis-benchmark to use an IP address for the first cluster node
* Fix saving of strings larger than 2GB into RDB files

Additional improvements:
* Improve replication handshake time
* Release client tracking table memory asynchronously in cases where the DB is also freed asynchronously
* Avoid wasteful transient memory allocation in certain cases
* Handle binary string values by the 'requirepass' and 'masterauth' configs

Platform and deployment-related changes:
* Install redis-check-rdb and redis-check-aof as symlinks to redis-server
* Add a check for an ARM64 Linux kernel bug
  Due to the potential severity of this issue, Redis will refuse to run on
  affected platforms by default.

Info fields and introspection changes:
* Add the errorstats section to the INFO command
* Add the failed_calls and rejected_calls fields INFO's commandstats section
* Report child copy-on-write metrics continuously

Module API changes:
* Add the RedisModule_SendChildCOWInfo API
* Add the may-replicate command flag


Redis 6.2 RC1   Released Mon Dec 14 11:50:00 IST 2020
================================================================================

Upgrade urgency LOW: This is the first Release Candidate of Redis 6.2.

Introduction to the Redis 6.2 release
=====================================

This release is the first significant Redis release managed by the core team
under the new project governance model.

Redis 6.2 includes many new commands and improvements, but no big features. It
mainly makes Redis more complete and addresses issues that have been requested
by many users frequently or for a long time.

Many of these changes were not eligible for 6.0.x for several reasons:

1. They are not backward compatible, which is always the case with new or
   extended commands (that cannot be replicated to an older replica).
2. They require a longer release-candidate test cycle.

New commands / args:
* Add SMISMEMBER command that checks multiple members
* Add ZMSCORE command that returns an array of scores
* Add LMOVE and BLMOVE commands that pop and push arbitrarily
* Add RESET command that resets client connection state
* Add COPY command that copies keys
* Add ZDIFF and ZDIFFSTORE commands
* Add ZINTER and ZUNION commands
* Add GEOSEARCH/GEOSEARCHSTORE commands for bounding box spatial queries
* Add GET parameter to SET command, for more powerful GETSET
* Add exclusive range query to XPENDING
* Add exclusive range query to X[REV]RANGE
* Add GT and LT options to ZADD for conditional score updates
* Add CLIENT INFO and CLIENT LIST for specific ids
* Add IDLE argument to XPENDING command
* Add local address to CLIENT LIST, and a CLIENT KILL filter.
* Add NOMKSTREAM option to XADD command
* Add command introspection to Sentinel
* Add SENTINEL MYID subcommand

New features:
* Dump payload sanitization: prevent corrupt payload causing crashes
  Has flags to enable full O(N) validation (disabled by default).
* ACL patterns for Pub/Sub channels
* Support ACL for Sentinel mode
* Support getting configuration from both stdin and file at the same time
  Lets you avoid storing secrets on the disk.

New features in CLI tools:
* redis-cli RESP3 push support
* redis-cli cluster import support source and target that require auth
* redis-cli URIs able to provide user name in addition to password
* redis-cli/redis-benchmark allow specifying the prefered ciphers/ciphersuites
* redis-cli add -e option to exit with code when command execution fails

Command behavior changes:
* EXISTS should not alter LRU
  In Redis 5.0 and 6.0 it would have touched the LRU/LFU of the key.
* OBJECT should not reveal logically expired keys
  Will now behave the same TYPE or any other non-DEBUG command.
* Improve db id range check for SELECT and MOVE
  Changes the error message text on a wrong db index.
* Modify AUTH / HELLO error message
  Changes the error message text when the user isn't found or is disabled.
* BITOPS length limited to proto_max_bulk_len rather than 512MB
  The limit is now configurable like in SETRANGE, and APPEND.
* GEORADIUS[BYMEMBER] can fail with -OOM if Redis is over the memory limit

Other behavior changes:
* Optionally (default) fail to start if requested bind address is not available
  If you rely on Redis starting successfully even if one of the bind addresses
  is not available, you'll need to tune the new config.
* Limit the main db dictionaries expansion to prevent key eviction
  In the past big dictionary rehashing could result in massive data eviction.
  Now this rehashing is delayed (up to a limit), which can result in performance
  loss due to hash collisions.
* CONFIG REWRITE is atomic and safer, but requires write access to the config file's folder
  This change was already present in 6.0.9, but was missing from the release
  notes.
* A new incremental eviction mechanism that reduces latency on eviction spikes
  In pathological cases this can cause memory to grow uncontrolled and may require
  specific tuning.
* Not resetting "save" config when Redis is started with command line arguments.
  In case you provide command line arguments without "save" and count on it
  being disabled, Now the defaults "save" config will kick in.
* Update memory metrics for INFO during loading
* When "supervised" config is enabled, it takes precedence over "daemonize".
* Assertion and panic, print crash log without generating SIGSEGV
* Added crash log report on SIGABRT, instead of silently exiting
* Disable THP (Transparent Huge Pages) if enabled
  If you deliberately enabled it, you'll need to config Redis to keep it.

Bug fixes:
* Handle output buffer limits for module blocked clients
  Could result in a module sending reply to a blocked client to go beyond the
  limit.
* Fix setproctitle related crashes.
  Caused various crashes on startup, mainly on Apple M1 chips or under
  instrumentation.
* A module doing RM_Call could cause replicas to get nested MULTI
* Backup/restore cluster mode keys to slots map for repl-diskless-load=swapdb
  In cluster mode with repl-diskless-load, when loading failed, slot map
  wouldn't have been restored.
* Fix oom-score-adj-values range, and bug when used in config file
  Enabling setting this in the config file in a line after enabling it, would
  have been buggy.
* Reset average ttl when empty databases
  Just causing misleading metric in INFO
* Disable rehash when Redis has child process
  This could have caused excessive CoW during BGSAVE, replication or AOFRW.
* Further improved ACL algorithm for picking categories
  Output of ACL GETUSER is now more similar to the one provided by ACL SETUSER.
* Fix bug with module GIL being released prematurely
  Could in theory (and rarely) cause multi-threaded modules to corrupt memory.
* Fix cluster redirect for module command with no firstkey.
* Reduce effect of client tracking causing feedback loop in key eviction
* Kill disk-based fork child when all replicas drop and 'save' is not enabled
* Rewritten commands (modified for propagation) are logged as their original command
* Fix cluster access to unaligned memory (SIGBUS on old ARM)
* If diskless repl child is killed, make sure to reap the child pid
* Broadcast a PONG message when slot's migration is over, may reduce MOVED responses

Other improvements:
* TLS Support in redis-benchmark
* Accelerate diskless master connections, and general re-connections
* Run active defrag while blocked / loading
* Performance and memory reporting improvement - sds take control of its internal fragmentation
* Speedup cluster failover.

Platform / toolchain support related improvements:
* Optionally (not by default) use H/W Monotonic clock for faster time sampling
* Remove the requirements for C11 and _Atomic supporting compiler
  This would allow to more easily build and use Redis on older systems and
  compilers again.
* Fix crash log registers output on ARM.
* Raspberry build fix.
* Setting process title support for Haiku.
* DragonFlyBSD RSS memory sampling support.

New configuration options:
* Enable configuring OpenSSL using the standard openssl.cnf
* oom-score-adj-values config can now take absolute values (besides relative ones)
* TLS: Add different client cert support.
* Note that a few other changes listed above added their config options.

Info fields and introspection changes:
* Add INFO fields to track diskless and disk-based replication progress
* Add INFO field for main thread cpu time, and scrape system time.
* Add total_forks to INFO STATS
* Add maxclients and cluster_connections to INFO CLIENTS
* Add tracking bcast flag and client redirection in client list
* Fixed INFO client_recent_max_input_buffer includes argv array
* Note that a few other changes listed above added their info fields.

Module API changes:
* Add CTX_FLAGS_DENY_BLOCKING as a unified the way to know if blocking is allowed
* Add data type callbacks for lazy free effort, and unlink
* Add data type callback for COPY command
* Add callbacks for defrag support.
* Add module event for repl-diskless-load swapdb

Module related fixes:
* Moved RMAPI_FUNC_SUPPORTED so that it's usable
* Improve timer accuracy
* Allow '\0' inside of result of RM_CreateStringPrintf

diffstat:

 databases/redis/Makefile                    |   4 +-
 databases/redis/distinfo                    |  15 ++++++-----
 databases/redis/patches/patch-redis.conf    |  18 +++++++-------
 databases/redis/patches/patch-src_Makefile  |  14 +++++-----
 databases/redis/patches/patch-src_zmalloc.c |  35 +++++++++++++++++++++++++++++
 5 files changed, 61 insertions(+), 25 deletions(-)

diffs (165 lines):

diff -r e7de4768d878 -r d851ce852604 databases/redis/Makefile
--- a/databases/redis/Makefile  Mon Mar 01 13:13:21 2021 +0000
+++ b/databases/redis/Makefile  Mon Mar 01 13:19:18 2021 +0000
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.64 2021/01/13 11:30:28 adam Exp $
+# $NetBSD: Makefile,v 1.65 2021/03/01 13:19:18 adam Exp $
 
-DISTNAME=      redis-6.0.10
+DISTNAME=      redis-6.2.0
 CATEGORIES=    databases
 MASTER_SITES=  http://download.redis.io/releases/
 
diff -r e7de4768d878 -r d851ce852604 databases/redis/distinfo
--- a/databases/redis/distinfo  Mon Mar 01 13:13:21 2021 +0000
+++ b/databases/redis/distinfo  Mon Mar 01 13:19:18 2021 +0000
@@ -1,10 +1,11 @@
-$NetBSD: distinfo,v 1.56 2021/01/13 11:30:28 adam Exp $
+$NetBSD: distinfo,v 1.57 2021/03/01 13:19:18 adam Exp $
 
-SHA1 (redis-6.0.10.tar.gz) = fcdfddab2a9a0faa317148b9211af42db1d64523
-RMD160 (redis-6.0.10.tar.gz) = 590bec2469980b73e80286cfac7c23aa69d7a9a9
-SHA512 (redis-6.0.10.tar.gz) = 4ef60645316143db3c26c45e90c7dc629e15641313267fd732396212c67db9ebeb9e170d830599b2e143ad5135d1ef249b6fbc1fb4960f4cf857cfdb5c1470c8
-Size (redis-6.0.10.tar.gz) = 2271970 bytes
-SHA1 (patch-redis.conf) = 528c28e2b7483eb20ccab8f2ab9543822792f016
-SHA1 (patch-src_Makefile) = 5c5be572f50d0db6b050a9d83fc30a28e8fa9856
+SHA1 (redis-6.2.0.tar.gz) = a09b62525fc4b122de4a28f4f2324d6cf8c872b5
+RMD160 (redis-6.2.0.tar.gz) = f14cd2652ea16f1bc9ebcf48b5f6d22f65f228b8
+SHA512 (redis-6.2.0.tar.gz) = 905cdcbe42b6126c2bfb68a948cb38d232af3d04e620c51897fb80ac43b5b8da1f60e9701377dcd15db8c3eb168ab9c3605d5715d664903e0a620b464d1ac9b8
+Size (redis-6.2.0.tar.gz) = 2435539 bytes
+SHA1 (patch-redis.conf) = ee657a9d82711263ceb0fb8f7d8059ed23528fe9
+SHA1 (patch-src_Makefile) = b74e1575d423b9a4d09b6b5e3eeb355d79c27855
 SHA1 (patch-src_hyperloglog.c) = e9bdd3c630024a6fbe02c2c1d85e26131ad938cf
 SHA1 (patch-src_object.c) = 30ffaec9c7e6135e3a5576cd1a35d7bcec668299
+SHA1 (patch-src_zmalloc.c) = 6e182eb1ee22f5c8e36b5d98e221db0a6524b0a6
diff -r e7de4768d878 -r d851ce852604 databases/redis/patches/patch-redis.conf
--- a/databases/redis/patches/patch-redis.conf  Mon Mar 01 13:13:21 2021 +0000
+++ b/databases/redis/patches/patch-redis.conf  Mon Mar 01 13:19:18 2021 +0000
@@ -1,28 +1,28 @@
-$NetBSD: patch-redis.conf,v 1.1 2018/12/13 19:35:12 adam Exp $
+$NetBSD: patch-redis.conf,v 1.2 2021/03/01 13:19:19 adam Exp $
 
 Sane defaults for the config file.
 
---- redis.conf.orig    2018-12-12 12:25:58.000000000 +0000
+--- redis.conf.orig    2021-02-22 21:23:58.000000000 +0000
 +++ redis.conf
-@@ -133,7 +133,7 @@ tcp-keepalive 300
- 
+@@ -244,7 +244,7 @@ tcp-keepalive 300
  # By default Redis does not run as a daemon. Use 'yes' if you need it.
  # Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
+ # When Redis is supervised by upstart or systemd, this parameter has no impact.
 -daemonize no
 +daemonize yes
  
  # If you run Redis from upstart or systemd, Redis can interact with your
  # supervision tree. Options:
-@@ -155,7 +155,7 @@ supervised no
+@@ -276,7 +276,7 @@ daemonize no
  #
- # Creating a pid file is best effort: if Redis is not able to create it
- # nothing bad happens, the server will start and run normally.
+ # Note that on modern Linux systems "/run/redis.pid" is more conforming
+ # and should be used instead.
 -pidfile /var/run/redis_6379.pid
 +pidfile @REDIS_PIDDIR@/redis.pid
  
  # Specify the server verbosity level.
  # This can be one of:
-@@ -168,7 +168,7 @@ loglevel notice
+@@ -289,7 +289,7 @@ loglevel notice
  # Specify the log file name. Also the empty string can be used to force
  # Redis to log on the standard output. Note that if you use standard
  # output for logging but daemonize, logs will be sent to /dev/null
@@ -31,7 +31,7 @@
  
  # To enable logging to the system logger, just set 'syslog-enabled' to yes,
  # and optionally update the other syslog parameters to suit your needs.
-@@ -260,7 +260,7 @@ dbfilename dump.rdb
+@@ -441,7 +441,7 @@ rdb-del-sync-files no
  # The Append Only File will also be created inside this directory.
  #
  # Note that you must specify a directory here, not a file name.
diff -r e7de4768d878 -r d851ce852604 databases/redis/patches/patch-src_Makefile
--- a/databases/redis/patches/patch-src_Makefile        Mon Mar 01 13:13:21 2021 +0000
+++ b/databases/redis/patches/patch-src_Makefile        Mon Mar 01 13:19:18 2021 +0000
@@ -1,12 +1,12 @@
-$NetBSD: patch-src_Makefile,v 1.3 2020/10/27 10:22:34 adam Exp $
+$NetBSD: patch-src_Makefile,v 1.4 2021/03/01 13:19:19 adam Exp $
 
 Add DESTDIR support.
 Fix NetBSD support.
 
---- src/Makefile.orig  2020-10-27 07:12:01.000000000 +0000
+--- src/Makefile.orig  2021-02-22 21:23:58.000000000 +0000
 +++ src/Makefile
-@@ -30,8 +30,8 @@ WARN=-Wall -W -Wno-missing-field-initial
- OPT=$(OPTIMIZATION)
+@@ -40,8 +40,8 @@ else
+ endif
  
  PREFIX?=/usr/local
 -INSTALL_BIN=$(PREFIX)/bin
@@ -16,7 +16,7 @@
  PKG_CONFIG?=pkg-config
  
  # Default allocator defaults to Jemalloc if it's not an ARM
-@@ -130,6 +130,10 @@ ifeq ($(uname_S),NetBSD)
+@@ -152,6 +152,10 @@ ifeq ($(uname_S),NetBSD)
            FINAL_LIBS+= -lexecinfo
        endif
  else
@@ -27,11 +27,11 @@
  ifeq ($(uname_S),FreeBSD)
        # FreeBSD
        FINAL_LIBS+= -lpthread -lexecinfo
-@@ -165,6 +169,7 @@ endif
+@@ -187,6 +191,7 @@ endif
  endif
  endif
  endif
 +endif
  # Include paths to dependencies
- FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src
+ FINAL_CFLAGS+= -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -I../deps/hdr_histogram
  
diff -r e7de4768d878 -r d851ce852604 databases/redis/patches/patch-src_zmalloc.c
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/redis/patches/patch-src_zmalloc.c       Mon Mar 01 13:19:18 2021 +0000
@@ -0,0 +1,35 @@
+$NetBSD: patch-src_zmalloc.c,v 1.1 2021/03/01 13:19:19 adam Exp $
+
+Fix compile errors with no HAVE_MALLOC_SIZE.
+https://github.com/redis/redis/pull/8533/files
+
+--- src/zmalloc.c.orig 2021-03-01 13:13:04.000000000 +0000
++++ src/zmalloc.c
+@@ -32,6 +32,7 @@
+ #include <stdlib.h>
+ #include <stdint.h>
+ #include <unistd.h>
++#include <assert.h>
+ 
+ /* This function provide us access to the original libc free(). This is useful
+  * for instance to free results obtained by backtrace_symbols(). We need
+@@ -49,18 +50,14 @@ void zlibc_free(void *ptr) {
+ 
+ #ifdef HAVE_MALLOC_SIZE
+ #define PREFIX_SIZE (0)
++#define ASSERT_NO_SIZE_OVERFLOW(sz)
+ #else
+ #if defined(__sun) || defined(__sparc) || defined(__sparc__)
+ #define PREFIX_SIZE (sizeof(long long))
+ #else
+ #define PREFIX_SIZE (sizeof(size_t))
+ #endif
+-#endif
+-
+-#if PREFIX_SIZE > 0
+ #define ASSERT_NO_SIZE_OVERFLOW(sz) assert((sz) + PREFIX_SIZE > (sz))
+-#else
+-#define ASSERT_NO_SIZE_OVERFLOW(sz)
+ #endif
+ 
+ /* Explicitly override malloc/free etc when using tcmalloc. */