[pkgsrc/trunk]: pkgsrc/net/tor tor: update to 0.4.6.5.

[wiz <wiz%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/6af5dea80c54
branches:  trunk
changeset: 455073:6af5dea80c54
user:      wiz <wiz%pkgsrc.org@localhost>
date:      Sun Jun 27 21:18:38 2021 +0000

description:
tor: update to 0.4.6.5.

Changes in version 0.4.6.5 - 2021-06-14
  Tor 0.4.6.5 is the first stable release in its series. The 0.4.6.x
  series includes numerous features and bugfixes, including a significant
  improvement to our circuit timeout algorithm that should improve
  observed client performance, and a way for relays to report when they are
  overloaded.

  This release also includes security fixes for several security issues,
  including a denial-of-service attack against onion service clients,
  and another denial-of-service attack against relays. Everybody should
  upgrade to one of 0.3.5.15, 0.4.4.9, 0.4.5.9, or 0.4.6.5.

  o Major bugfixes (security):
    - Don't allow relays to spoof RELAY_END or RELAY_RESOLVED cell on
      half-closed streams. Previously, clients failed to validate which
      hop sent these cells: this would allow a relay on a circuit to end
      a stream that wasn't actually built with it. Fixes bug 40389;
      bugfix on 0.3.5.1-alpha. This issue is also tracked as TROVE-2021-
      003 and CVE-2021-34548.

  o Major bugfixes (security, defense-in-depth):
    - Detect more failure conditions from the OpenSSL RNG code.
      Previously, we would detect errors from a missing RNG
      implementation, but not failures from the RNG code itself.
      Fortunately, it appears those failures do not happen in practice
      when Tor is using OpenSSL's default RNG implementation. Fixes bug
      40390; bugfix on 0.2.8.1-alpha. This issue is also tracked as
      TROVE-2021-004. Reported by Jann Horn at Google's Project Zero.

  o Major bugfixes (security, denial of service):
    - Resist a hashtable-based CPU denial-of-service attack against
      relays. Previously we used a naive unkeyed hash function to look
      up circuits in a circuitmux object. An attacker could exploit this
      to construct circuits with chosen circuit IDs, to create
      collisions and make the hash table inefficient. Now we use a
      SipHash construction here instead. Fixes bug 40391; bugfix on
      0.2.4.4-alpha. This issue is also tracked as TROVE-2021-005 and
      CVE-2021-34549. Reported by Jann Horn from Google's Project Zero.
    - Fix an out-of-bounds memory access in v3 onion service descriptor
      parsing. An attacker could exploit this bug by crafting an onion
      service descriptor that would crash any client that tried to visit
      it. Fixes bug 40392; bugfix on 0.3.0.1-alpha. This issue is also
      tracked as TROVE-2021-006 and CVE-2021-34550. Reported by Sergei
      Glazunov from Google's Project Zero.

  o Major features (control port, onion services):
    - Add controller support for creating version 3 onion services with
      client authorization. Previously, only v2 onion services could be
      created with client authorization. Closes ticket 40084. Patch by
      Neel Chauhan.

  o Major features (directory authority):
    - When voting on a relay with a Sybil-like appearance, add the Sybil
      flag when clearing out the other flags. This lets a relay operator
      know why their relay hasn't been included in the consensus. Closes
      ticket 40255. Patch by Neel Chauhan.

  o Major features (metrics):
    - Relays now report how overloaded they are in their extrainfo
      documents. This information is controlled with the
      OverloadStatistics torrc option, and it will be used to improve
      decisions about the network's load balancing. Implements proposal
      328; closes ticket 40222.

  o Major features (relay, denial of service):
    - Add a new DoS subsystem feature to control the rate of client
      connections for relays. Closes ticket 40253.

  o Major features (statistics):
    - Relays now publish statistics about the number of v3 onion
      services and volume of v3 onion service traffic, in the same
      manner they already do for v2 onions. Closes ticket 23126.

  o Major bugfixes (circuit build timeout):
    - Improve the accuracy of our circuit build timeout calculation for
      60%, 70%, and 80% build rates for various guard choices. We now
      use a maximum likelihood estimator for Pareto parameters of the
      circuit build time distribution, instead of a "right-censored
      estimator". This causes clients to ignore circuits that never
      finish building in their timeout calculations. Previously, clients
      were counting such unfinished circuits as having the highest
      possible build time value, when in reality these circuits most
      likely just contain relays that are offline. We also now wait a
      bit longer to let circuits complete for measurement purposes,
      lower the minimum possible effective timeout from 1.5 seconds to
      10ms, and increase the resolution of the circuit build time
      histogram from 50ms bin widths to 10ms bin widths. Additionally,
      we alter our estimate Xm by taking the maximum of the top 10 most
      common build time values of the 10ms histogram, and compute Xm as
      the average of these. Fixes bug 40168; bugfix on 0.2.2.14-alpha.
    - Remove max_time calculation and associated warning from circuit
      build timeout 'alpha' parameter estimation, as this is no longer
      needed by our new estimator from 40168. Fixes bug 34088; bugfix
      on 0.2.2.9-alpha.

  o Major bugfixes (signing key):
    - In the tor-gencert utility, give an informative error message if
      the passphrase given in `--create-identity-key` is too short.
      Fixes bug 40189; bugfix on 0.2.0.1-alpha. Patch by Neel Chauhan.

  o Minor features (bridge):
    - We now announce the URL to Tor's new bridge status at
      https://bridges.torproject.org/ when Tor is configured to run as a
      bridge relay. Closes ticket 30477.

  o Minor features (build system):
    - New "make lsp" command to auto generate the compile_commands.json
      file used by the ccls server. The "bear" program is needed for
      this. Closes ticket 40227.

  o Minor features (client):
    - Clients now check whether their streams are attempting to re-enter
      the Tor network (i.e. to send Tor traffic over Tor), and close
      them preemptively if they think exit relays will refuse them for
      this reason. See ticket 2667 for details. Closes ticket 40271.

  o Minor features (command line):
    - Add long format name "--torrc-file" equivalent to the existing
      command-line option "-f". Closes ticket 40324. Patch by
      Daniel Pinto.

  o Minor features (command-line interface):
    - Add build informations to `tor --version` in order to ease
      reproducible builds. Closes ticket 32102.
    - When parsing command-line flags that take an optional argument,
      treat the argument as absent if it would start with a '-'
      character. Arguments in that form are not intelligible for any of
      our optional-argument flags. Closes ticket 40223.
    - Allow a relay operator to list the ed25519 keys on the command
      line by adding the `rsa` and `ed25519` arguments to the
      --list-fingerprint flag to show the respective RSA and ed25519
      relay fingerprint. Closes ticket 33632. Patch by Neel Chauhan.

  o Minor features (compatibility):
    - Remove an assertion function related to TLS renegotiation. It was
      used nowhere outside the unit tests, and it was breaking
      compilation with recent alpha releases of OpenSSL 3.0.0. Closes
      ticket 40399.

  o Minor features (control port, stream handling):
    - Add the stream ID to the event line in the ADDRMAP control event.
      Closes ticket 40249. Patch by Neel Chauhan.

  o Minor features (dormant mode):
    - Add a new 'DormantTimeoutEnabled' option to allow coarse-grained
      control over whether the client ever becomes dormant from
      inactivity. Most people won't need this. Closes ticket 40228.
    - Add a new 'DormantTimeoutEnabled' option for coarse-grained
      control over whether the client can become dormant from
      inactivity. Most people won't need this. Closes ticket 40228.

  o Minor features (geoip data):
    - Update the geoip files to match the IPFire Location Database, as
      retrieved on 2021/06/10.

  o Minor features (logging):
    - Edit heartbeat log messages so that more of them begin with the
      string "Heartbeat: ". Closes ticket 40322; patch
      from 'cypherpunks'.
    - Change the DoS subsystem heartbeat line format to be more clear on
      what has been detected/rejected, and which option is disabled (if
      any). Closes ticket 40308.
    - In src/core/mainloop/mainloop.c and src/core/mainloop/connection.c,
      put brackets around IPv6 addresses in log messages. Closes ticket
      40232. Patch by Neel Chauhan.

  o Minor features (logging, diagnostic):
    - Log decompression failures at a higher severity level, since they
      can help provide missing context for other warning messages. We
      rate-limit these messages, to avoid flooding the logs if they
      begin to occur frequently. Closes ticket 40175.

  o Minor features (onion services):
    - Add a warning message when trying to connect to (no longer
      supported) v2 onion services. Closes ticket 40373.

  o Minor features (performance, windows):
    - Use SRWLocks to implement locking on Windows. Replaces the
      "critical section" locking implementation with the faster
      SRWLocks, available since Windows Vista. Closes ticket 17927.
      Patch by Daniel Pinto.

  o Minor features (protocol, proxy support, defense in depth):
    - Close HAProxy connections if they somehow manage to send us data
      before we start reading. Closes another case of ticket 40017.

  o Minor features (tests, portability):
    - Port the hs_build_address.py test script to work with recent
      versions of python. Closes ticket 40213. Patch from
      Samanta Navarro.

  o Minor features (vote document):
    - Add a "stats" line to directory authority votes, to report various
      statistics that authorities compute about the relays. This will
      help us diagnose the network better. Closes ticket 40314.

  o Minor bugfixes (build):
    - The configure script now shows whether or not lzma and zstd have
      been used, not just if the enable flag was passed in. Fixes bug
      40236; bugfix on 0.4.3.1-alpha.

  o Minor bugfixes (compatibility):
    - Fix a failure in the test cases when running on the "hppa"
      architecture, along with a related test that might fail on other
      architectures in the future. Fixes bug 40274; bugfix
      on 0.2.5.1-alpha.

  o Minor bugfixes (compilation):
    - Fix a compilation warning about unused functions when building
      with a libc that lacks the GLOB_ALTDIRFUNC constant. Fixes bug
      40354; bugfix on 0.4.5.1-alpha. Patch by Daniel Pinto.

  o Minor bugfixes (consensus handling):
    - Avoid a set of bugs that could be caused by inconsistently
      preferring an out-of-date consensus stored in a stale directory
      cache over a more recent one stored on disk as the latest
      consensus. Fixes bug 40375; bugfix on 0.3.1.1-alpha.

  o Minor bugfixes (control, sandbox):
    - Allow the control command SAVECONF to succeed when the seccomp
      sandbox is enabled, and make SAVECONF keep only one backup file to
      simplify implementation. Previously SAVECONF allowed a large
      number of backup files, which made it incompatible with the
      sandbox. Fixes bug 40317; bugfix on 0.2.5.4-alpha. Patch by
      Daniel Pinto.

  o Minor bugfixes (directory authorities, voting):
    - Add a new consensus method (31) to support any future changes that
      authorities decide to make to the value of bwweightscale or
      maxunmeasuredbw. Previously, there was a bug that prevented the
      authorities from parsing these consensus parameters correctly under
      most circumstances. Fixes bug 19011; bugfix on 0.2.2.10-alpha.

  o Minor bugfixes (ipv6):
    - Allow non-SOCKSPorts to disable IPv4, IPv6, and PreferIPv4. Some
      rare configurations might break, but in this case you can disable
      NoIPv4Traffic and NoIPv6Traffic as needed. Fixes bug 33607; bugfix
      on 0.4.1.1-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (key generation):
    - Do not require a valid torrc when using the `--keygen` argument to
      generate a signing key. This allows us to generate keys on systems
      or users which may not run Tor. Fixes bug 40235; bugfix on
      0.2.7.2-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (logging, relay):
    - Emit a warning if an Address is found to be internal and tor can't
      use it. Fixes bug 40290; bugfix on 0.4.5.1-alpha.

  o Minor bugfixes (metrics port):
    - Fix a bug that made tor try to re-bind() on an already open
      MetricsPort every 60 seconds. Fixes bug 40370; bugfix
      on 0.4.5.1-alpha.

  o Minor bugfixes (onion services, logging):
    - Downgrade the severity of a few rendezvous circuit-related
      warnings from warning to info. Fixes bug 40207; bugfix on
      0.3.2.1-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (relay):
    - Reduce the compression level for data streaming from HIGH to LOW.
      This should reduce the CPU and memory burden for directory caches.
      Fixes bug 40301; bugfix on 0.3.5.1-alpha.

  o Minor bugfixes (testing, BSD):
    - Fix pattern-matching errors when patterns expand to invalid paths
      on BSD systems. Fixes bug 40318; bugfix on 0.4.5.1-alpha. Patch by
      Daniel Pinto.

  o Code simplification and refactoring:
    - Remove the orconn_ext_or_id_map structure and related functions.
      (Nothing outside of unit tests used them.) Closes ticket 33383.
      Patch by Neel Chauhan.

  o Removed features:
    - Remove unneeded code for parsing private keys in directory
      documents. This code was only used for client authentication in v2
      onion services, which are now unsupported. Closes ticket 40374.
    - As of this release, Tor no longer supports the old v2 onion
      services. They were deprecated last July for security, and support
      will be removed entirely later this year. We strongly encourage
      everybody to migrate to v3 onion services. For more information,
      see https://blog.torproject.org/v2-deprecation-timeline . Closes
      ticket 40266. (NOTE: We accidentally released an earlier version
      of the 0.4.6.1-alpha changelog without this entry. Sorry for
      the confusion!)

  o Code simplification and refactoring (metrics, DoS):
    - Move the DoS subsystem into the subsys manager, including its
      configuration options. Closes ticket 40261.

  o Documentation (manual):
    - Move the ServerTransport* options to the "SERVER OPTIONS" section.
      Closes issue 40331.
    - Indicate that the HiddenServiceStatistics option also applies to
      bridges. Closes ticket 40346.
    - Move the description of BridgeRecordUsageByCountry to the section
      "STATISTICS OPTIONS". Closes ticket 40323.

  o Removed features (relay):
    - Because DirPorts are only used on authorities, relays no longer
      advertise them. Similarly, self-testing for DirPorts has been
      disabled, since an unreachable DirPort is no reason for a relay
      not to advertise itself. (Configuring a DirPort will still work,
      for now.) Closes ticket 40282.

diffstat:

 net/tor/Makefile   |  31 +++++++++++++++++--------
 net/tor/PLIST      |   5 +--
 net/tor/distinfo   |  10 ++++----
 net/tor/options.mk |  65 ++++++++++++-----------------------------------------
 4 files changed, 43 insertions(+), 68 deletions(-)

diffs (190 lines):

diff -r b89f43446eeb -r 6af5dea80c54 net/tor/Makefile
--- a/net/tor/Makefile  Sun Jun 27 18:30:26 2021 +0000
+++ b/net/tor/Makefile  Sun Jun 27 21:18:38 2021 +0000
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.169 2021/06/14 17:46:43 wiz Exp $
+# $NetBSD: Makefile,v 1.170 2021/06/27 21:18:38 wiz Exp $
 
-DISTNAME=      tor-0.4.5.9
+DISTNAME=      tor-0.4.6.5
 CATEGORIES=    net security
 MASTER_SITES=  https://dist.torproject.org/
 
@@ -11,18 +11,24 @@
 
 USE_LANGUAGES=         c99
 USE_PKGLOCALEDIR=      yes
-USE_TOOLS+=            perl:test pkg-config
-# There are multiple reports of failure to build with BSD make, due to
-# an apparent lack of firing the rule to create micro-revision.i.
-# Work around this by using gmake, which is tiny compared to the
-# existing perl dependency, and unlikely to be needed on any given
-# system just for tor.
-USE_TOOLS+=            gmake bash:test
+USE_TOOLS+=            perl:test pkg-config bash:test
 GNU_CONFIGURE=         yes
 CONFIGURE_ARGS+=       --localstatedir=${VARBASE}
 CONFIGURE_ARGS+=       --sysconfdir=${PKG_SYSCONFBASEDIR}
 CONFIGURE_ENV+=                CPP=${CPP:Q}
+CONFIGURE_ENV+=                CONFDIR=${PKG_SYSCONFDIR}
 
+#============================================================================
+#Testsuite summary for tor 0.4.6.5
+#============================================================================
+## TOTAL: 33
+## PASS:  29
+## SKIP:  4
+## XFAIL: 0
+## FAIL:  0
+## XPASS: 0
+## ERROR: 0
+#============================================================================
 TEST_TARGET=           check
 
 TOR_USER?=             tor
@@ -56,7 +62,12 @@
 
 CHECK_PORTABILITY_SKIP+=       contrib/*
 
+REPLACE_BASH+=         scripts/maint/checkShellScripts.sh
+REPLACE_BASH+=         scripts/maint/run_check_subsystem_order.sh
+
+REPLACE_PERL+=         scripts/maint/checkOptionDocs.pl.in
 REPLACE_PERL+=         scripts/maint/checkSpace.pl
+REPLACE_PERL+=         scripts/coccinelle/test-operator-cleanup
 
 .include "options.mk"
 
@@ -66,9 +77,9 @@
 CFLAGS+=               -c99
 .endif
 
+.include "../../archivers/zstd/buildlink3.mk"
 BUILDLINK_API_DEPENDS.libevent+= libevent>=2.0
 .include "../../devel/libevent/buildlink3.mk"
 .include "../../devel/zlib/buildlink3.mk"
-.include "../../security/openssl/buildlink3.mk"
 .include "../../mk/pthread.buildlink3.mk"
 .include "../../mk/bsd.pkg.mk"
diff -r b89f43446eeb -r 6af5dea80c54 net/tor/PLIST
--- a/net/tor/PLIST     Sun Jun 27 18:30:26 2021 +0000
+++ b/net/tor/PLIST     Sun Jun 27 21:18:38 2021 +0000
@@ -1,6 +1,6 @@
-@comment $NetBSD: PLIST,v 1.15 2021/02/15 19:01:31 wiz Exp $
+@comment $NetBSD: PLIST,v 1.16 2021/06/27 21:18:38 wiz Exp $
 bin/tor
-${PLIST.gencert}bin/tor-gencert
+${PLIST.openssl}bin/tor-gencert
 bin/tor-print-ed-signing-cert
 bin/tor-resolve
 bin/torify
@@ -14,7 +14,6 @@
 ${PLIST.doc}share/doc/tor/tor-resolve.html
 ${PLIST.doc}share/doc/tor/tor.html
 ${PLIST.doc}share/doc/tor/torify.html
-share/examples/rc.d/tor
 share/examples/tor/torrc.sample
 share/tor/geoip
 share/tor/geoip6
diff -r b89f43446eeb -r 6af5dea80c54 net/tor/distinfo
--- a/net/tor/distinfo  Sun Jun 27 18:30:26 2021 +0000
+++ b/net/tor/distinfo  Sun Jun 27 21:18:38 2021 +0000
@@ -1,6 +1,6 @@
-$NetBSD: distinfo,v 1.115 2021/06/14 17:46:43 wiz Exp $
+$NetBSD: distinfo,v 1.116 2021/06/27 21:18:38 wiz Exp $
 
-SHA1 (tor-0.4.5.9.tar.gz) = adbfefa54a23c6e7418561bd042442ee6fc50bfc
-RMD160 (tor-0.4.5.9.tar.gz) = c3933bbd8ceec45813ae736ad72b65d7935a76b2
-SHA512 (tor-0.4.5.9.tar.gz) = b3c3b5cce30c881fb1e705ec6183513f625ddb9d076671b9cd6299e81a410bc12f59a30677636371c336e397211432f0831bdcb2105c9aed8dcb608eae54e2b2
-Size (tor-0.4.5.9.tar.gz) = 7840294 bytes
+SHA1 (tor-0.4.6.5.tar.gz) = 49bfc6faa704302e41cd2f577a9feedae1370db5
+RMD160 (tor-0.4.6.5.tar.gz) = e773423c05ea9557c109a85cdee15b731ce4340b
+SHA512 (tor-0.4.6.5.tar.gz) = 62bcd8d446199cdd397a688d454730a057ef20be4152e6d0632f64df27c993c70be0dbde0a2e7679a8a20850371a503b8daf777296d555760d8aae3286e48050
+Size (tor-0.4.6.5.tar.gz) = 7754823 bytes
diff -r b89f43446eeb -r 6af5dea80c54 net/tor/options.mk
--- a/net/tor/options.mk        Sun Jun 27 18:30:26 2021 +0000
+++ b/net/tor/options.mk        Sun Jun 27 21:18:38 2021 +0000
@@ -1,13 +1,15 @@
-# $NetBSD: options.mk,v 1.18 2021/02/20 09:08:58 wiz Exp $
+# $NetBSD: options.mk,v 1.19 2021/06/27 21:18:38 wiz Exp $
 
-PKG_OPTIONS_VAR=       PKG_OPTIONS.tor
-PKG_SUPPORTED_OPTIONS= doc lzma nss zstd # rust
-PKG_SUGGESTED_OPTIONS+=        doc lzma zstd
+# XXX Add rust to supported options.
+PKG_OPTIONS_VAR=               PKG_OPTIONS.tor
+PKG_SUPPORTED_OPTIONS=         doc
+PKG_OPTIONS_REQUIRED_GROUPS=   ssl
+PKG_OPTIONS_GROUP.ssl=         nss openssl
+PKG_SUGGESTED_OPTIONS+=                doc openssl
 
 .include "../../mk/bsd.options.mk"
 
-PLIST_VARS+=           doc
-PLIST_VARS+=           gencert
+PLIST_VARS+=           doc openssl
 
 ###
 ### This enables the build of manual pages. It requires asciidoc
@@ -22,54 +24,17 @@
 CONFIGURE_ARGS+=       --disable-asciidoc
 .endif
 
-### This enables building tor with rust as per
-### https://trac.torproject.org/projects/tor/wiki/RustInTor
-### XXX:
-### As of 0.4.4.6 I am not sure how this is supposed to work.
-### https://gitlab.torproject.org/tpo/core/tor/-/issues/40191
-#.if !empty(PKG_OPTIONS:Mrust)
-#CONFIGURE_ENV+=       TOR_RUST_DEPENDENCIES=${WRKDIR}/vendor
-#CONFIGURE_ARGS+=      --enable-rust
-#CARGO_CRATE_DEPENDS+= digest-0.7.2
-#CARGO_CRATE_DEPENDS+= generic-array-0.9.0
-#CARGO_CRATE_DEPENDS+= libc-0.2.39
-#CARGO_CRATE_DEPENDS+= rand-0.5.0-pre.2
-#CARGO_CRATE_DEPENDS+= rand_core-0.2.0-pre.0
-#CARGO_CRATE_DEPENDS+= typenum-1.9.0
-
-#RUST_REQ=     1.34.0
-#.include "../../lang/rust/cargo.mk"
-
-#pre-configure:
-#      cd ${WRKSRC} && ${MKDIR} -p src/rust/target/release
-
-#.PHONY: show-tor-cargo-depends
-## \todo: Maybe we should add a path option to show-cargo-depends.
-#show-tor-cargo-depends:
-#      ${RUN}${AWK} '/^\"checksum/ { print "CARGO_CRATE_DEPENDS+=\t" $$2 "-" $$3""; next } ' ${WRKSRC}/src/rust/Cargo.lock
-
-#.else
-CONFIGURE_ARGS+=       --disable-rust
-#.endif
-
-.if !empty(PKG_OPTIONS:Mlzma)
-CONFIGURE_ARGS+=       --enable-lzma
-.include "../../archivers/xz/buildlink3.mk"
-.else
-CONFIGURE_ARGS+=       --disable-lzma
+.if !empty(PKG_OPTIONS:Mopenssl)
+PLIST.openssl=         yes
+.include "../../security/openssl/buildlink3.mk"
 .endif
 
 .if !empty(PKG_OPTIONS:Mnss)
 CONFIGURE_ARGS+=       --enable-nss
 .include "../../devel/nss/buildlink3.mk"
-.else
-CONFIGURE_ARGS+=       --disable-nss
-PLIST.gencert=         yes
 .endif
 
-.if !empty(PKG_OPTIONS:Mzstd)
-CONFIGURE_ARGS+=       --enable-zstd
-.include "../../archivers/zstd/buildlink3.mk"
-.else
-CONFIGURE_ARGS+=       --disable-zstd
-.endif
+#.if !empty(PKG_OPTIONS:Mrust)
+#CONFIGURE_ARGS+=      --enable-rust
+#.include "../../lang/rust/buildlink3.mk"
+#.endif