CVS commit: pkgsrc/net/py-zeroconf

["Adam Ciarcinski" <adam%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   adam
Date:           Fri Jul  2 09:54:09 UTC 2021

Modified Files:
        pkgsrc/net/py-zeroconf: Makefile PLIST distinfo

Log Message:
py-zeroconf: updated to 0.32.0

0.32.0

This release offers 100% line and branch coverage.

Made ServiceInfo first question QU

We want an immediate response when requesting with ServiceInfo by asking a QU question; most responders will not delay the response and respond right away to our question. This also improves 
compatibility with split networks as we may not have been able to see the response otherwise. If the responder has not multicast the record recently, it may still choose to do so in addition to 
responding via unicast

Reduces traffic when there are multiple zeroconf instances running on the network running ServiceBrowsers

If we don't get an answer on the first try, we ask a QM question in the event, we can't receive a unicast response for some reason

This change puts ServiceInfo inline with ServiceBrowser which also asks the first question as QU since ServiceInfo is commonly called from ServiceBrowser callbacks

Limited duplicate packet suppression to 1s intervals

Only suppress duplicate packets that happen within the same second. Legitimate queriers will retry the question if they are suppressed. The limit was reduced to one second to be in line with rfc6762

Made multipacket known answer suppression per interface

The suppression was happening per instance of Zeroconf instead of per interface. Since the same network can be seen on multiple interfaces (usually and wifi and ethernet), this would confuse the 
multi-packet known answer supression since it was not expecting to get the same data more than once

New ServiceBrowsers now request QU in the first outgoing when unspecified

https://datatracker.ietf.org/doc/html/rfc6762#section-5.4 When we start a ServiceBrowser and zeroconf has just started up, the known answer list will be small. By asking a QU question first, it is 
likely that we have a large known answer list by the time we ask the QM question a second later (current default which is likely too low but would be a breaking change to increase). This reduces the 
amount of traffic on the network, and has the secondary advantage that most responders will answer a QU question without the typical delay answering QM questions.

IPv6 link-local addresses are now qualified with scope_id

When a service is advertised on an IPv6 address where the scope is link local, i.e. fe80::/64 (see RFC 4007) the resolved IPv6 address must be extended with the scope_id that identifies through the 
"%" symbol the local interface to be used when routing to that address. A new API parsed_scoped_addresses() is provided to return qualified addresses to avoid breaking compatibility on the existing 
parsed_addresses().

Network adapters that are disconnected are now skipped

Fixed listeners missing initial packets if Engine starts too quickly

When manually creating a zeroconf.Engine object, it is no longer started automatically. It must manually be started by calling .start() on the created object.

The Engine thread is now started after all the listeners have been added to avoid a race condition where packets could be missed at startup.

Fixed answering matching PTR queries with the ANY query

Fixed lookup of uppercase names in the registry

If the ServiceInfo was registered with an uppercase name and the query was for a lowercase name, it would not be found and vice-versa.

Fixed unicast responses from any source port

Unicast responses were only being sent if the source port was 53, this prevented responses when testing with dig:

dig -p 5353 @224.0.0.251 media-12.local
The above query will now see a response

Fixed queries for AAAA records not being answered

Removed second level caching from ServiceBrowsers

The ServiceBrowser had its own cache of the last time it saw a service that was reimplementing the DNSCache and presenting a source of truth problem that lead to unexpected queries when the two 
disagreed.

Fixed server cache not being case-insensitive

If the server name had uppercase chars and any of the matching records were lowercase, and the server would not be found

Fixed cache handling of records with different TTLs

There should only be one unique record in the cache at a time as having multiple unique records will different TTLs in the cache can result in unexpected behavior since some functions returned all 
matching records and some fetched from the right side of the list to return the newest record. Instead we now store the records in a dict to ensure that the newest record always replaces the same 
unique record, and we never have a source of truth problem determining the TTL of a record from the cache.

Fixed ServiceInfo with multiple A records

If there were multiple A records for the host, ServiceInfo would always return the last one that was in the incoming packet, which was usually not the one that was wanted.

Fixed stale unique records expiring too quickly

Records now expire 1s in the future instead of instant removal.

tools.ietf.org/html/rfc6762#section-10.2 Queriers receiving a Multicast DNS response with a TTL of zero SHOULD NOT immediately delete the record from the cache, but instead record a TTL of 1 and then 
delete the record one second later. In the case of multiple Multicast DNS responders on the network described in Section 6.6 above, if one of the responders shuts down and incorrectly sends goodbye 
packets for its records, it gives the other cooperating responders one second to send out their own response to "rescue" the records before they expire and are deleted.

Fixed exception when unregistering a service multiple times

Added an AsyncZeroconfServiceTypes to mirror ZeroconfServiceTypes to zeroconf.asyncio

Fixed interface_index_to_ip6_address not skiping ipv4 adapters

Added async_unregister_all_services to AsyncZeroconf

Fixed services not being removed from the registry when calling unregister_all_services

There was a race condition where a query could be answered for a service in the registry, while goodbye packets which could result in a fresh record being broadcast after the goodbye if a query came 
in at just the right time. To avoid this, we now remove the services from the registry right after we generate the goodbye packet

Fixed zeroconf exception on load when the system disables IPv6

Fixed the QU bit missing from for probe queries

The bit should be set per datatracker.ietf.org/doc/html/rfc6762#section-8.1

Fixed the TC bit missing for query packets where the known answers span multiple packets

Fixed packets not being properly separated when exceeding maximum size

Ensure that questions that exceed the max packet size are moved to the next packet. This fixes DNSQuestions being sent in multiple packets in violation of: 
datatracker.ietf.org/doc/html/rfc6762#section-7.2

Ensure only one resource record is sent when a record exceeds _MAX_MSG_TYPICAL datatracker.ietf.org/doc/html/rfc6762#section-17

Fixed PTR questions asked in uppercase not being answered

Added Support for context managers in Zeroconf and AsyncZeroconf

Implemented an AsyncServiceBrowser to compliment the sync ServiceBrowser

Added async_get_service_info to AsyncZeroconf and async_request to AsyncServiceInfo

Implemented allowing passing in a sync Zeroconf instance to AsyncZeroconf

Fixed IPv6 setup under MacOS when binding to ""

Fixed ZeroconfServiceTypes.find not always cancels the ServiceBrowser

There was a short window where the ServiceBrowser thread could be left running after Zeroconf is closed because the .join() was never waited for when a new Zeroconf object was created

Fixed duplicate packets triggering duplicate updates

If TXT or SRV records update was already processed and then received again, it was possible for a second update to be called back in the ServiceBrowser

Fixed ServiceStateChange.Updated event happening for IPs that already existed

Fixed RFC6762 Section 10.2 paragraph 2 compliance

Reduced length of ServiceBrowser thread name with many types

Fixed empty answers being added in ServiceInfo.request

Fixed ServiceInfo not populating all AAAA records

Use get_all_by_details to ensure all records are loaded into addresses.

Only load A/AAAA records from the cache once in load_from_cache if there is a SRV record present

Move duplicate code that checked if the ServiceInfo was complete into its own function

Fixed a case where the cache list can change during iteration

Return task objects created by AsyncZeroconf

Traffic Reduction:

Added support for handling QU questions

Implements RFC 6762 sec 5.4: Questions Requesting Unicast Responses datatracker.ietf.org/doc/html/rfc6762#section-5.4

Implemented protect the network against excessive packet flooding

Additionals are now suppressed when they are already in the answers section

Additionals are no longer included when the answer is suppressed by known-answer suppression

Implemented multi-packet known answer supression

Implements datatracker.ietf.org/doc/html/rfc6762#section-7.2

Implemented efficient bucketing of queries with known answers

Implemented duplicate question suppression

http://datatracker.ietf.org/doc/html/rfc6762#section-7.3

Technically backwards incompatible:

Update internal version check to match docs (3.6+)

Python version earlier then 3.6 were likely broken with zeroconf already, however, the version is now explicitly checked.

Update python compatibility as PyPy3 7.2 is required

Backwards incompatible:

Drop oversize packets before processing them

Oversized packets can quickly overwhelm the system and deny service to legitimate queriers. In practice, this is usually due to broken mDNS implementations rather than malicious actors.

Guard against excessive ServiceBrowser queries from PTR records significantly lowerthan recommended

We now enforce a minimum TTL for PTR records to avoid ServiceBrowsers generating excessive queries refresh queries. Apple uses a 15s minimum TTL, however, we do not have the same level of rate limit 
and safeguards, so we use 1/4 of the recommended value.

RecordUpdateListener now uses async_update_records instead of update_record

This allows the listener to receive all the records that have been updated in a single transaction such as a packet or cache expiry.

update_record has been deprecated in favor of async_update_records A compatibility shim exists to ensure classes that use RecordUpdateListener as a base class continue to have update_record called, 
however, they should be updated as soon as possible.

A new method async_update_records_complete is now called on each listener when all listeners have completed processing updates and the cache has been updated. This allows ServiceBrowsers to delay 
calling handlers until they are sure the cache has been updated as its a common pattern to call for ServiceInfo when a ServiceBrowser handler fires.

The async_ prefix was chosen to make it clear that these functions run in the eventloop and should never do blocking I/O. Before 0.32+ these functions ran in a select() loop and should not have been 
doing any blocking I/O, but it was not clear to implementors that I/O would block the loop.

Pass both the new and old records to async_update_records

Pass the old_record (cached) as the value and the new_record (wire) to async_update_records instead of forcing each consumer to check the cache since we will always have the old_record when 
generating the async_update_records call. This avoids the overhead of multiple cache lookups for each listener.


To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 pkgsrc/net/py-zeroconf/Makefile \
    pkgsrc/net/py-zeroconf/distinfo
cvs rdiff -u -r1.3 -r1.4 pkgsrc/net/py-zeroconf/PLIST

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/net/py-zeroconf/Makefile
diff -u pkgsrc/net/py-zeroconf/Makefile:1.18 pkgsrc/net/py-zeroconf/Makefile:1.19
--- pkgsrc/net/py-zeroconf/Makefile:1.18        Wed May 19 09:40:34 2021
+++ pkgsrc/net/py-zeroconf/Makefile     Fri Jul  2 09:54:09 2021
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.18 2021/05/19 09:40:34 adam Exp $
+# $NetBSD: Makefile,v 1.19 2021/07/02 09:54:09 adam Exp $
 
-DISTNAME=      zeroconf-0.31.0
+DISTNAME=      zeroconf-0.32.0
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=    net python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=z/zeroconf/}
Index: pkgsrc/net/py-zeroconf/distinfo
diff -u pkgsrc/net/py-zeroconf/distinfo:1.18 pkgsrc/net/py-zeroconf/distinfo:1.19
--- pkgsrc/net/py-zeroconf/distinfo:1.18        Wed May 19 09:40:34 2021
+++ pkgsrc/net/py-zeroconf/distinfo     Fri Jul  2 09:54:09 2021
@@ -1,6 +1,6 @@
-$NetBSD: distinfo,v 1.18 2021/05/19 09:40:34 adam Exp $
+$NetBSD: distinfo,v 1.19 2021/07/02 09:54:09 adam Exp $
 
-SHA1 (zeroconf-0.31.0.tar.gz) = 58a37b9a9494026037cfb1179aacf4c58854dae6
-RMD160 (zeroconf-0.31.0.tar.gz) = c43f46bbead0e4c5e36e9f424b7671ef907f3ae7
-SHA512 (zeroconf-0.31.0.tar.gz) = e01bf8e42ca995eaf74081b6b872f5ec5f9229219e239ea8a3b3b61eb4b908808eec48de17a011a6fc34aabf086b5b3c0924a404706b2dd6f2aa234c1f5cd95f
-Size (zeroconf-0.31.0.tar.gz) = 66836 bytes
+SHA1 (zeroconf-0.32.0.tar.gz) = db361f930253670aacf8f9d19339952d956f045d
+RMD160 (zeroconf-0.32.0.tar.gz) = c6916d756b582d7e6e847a66a7e5674c6fe93857
+SHA512 (zeroconf-0.32.0.tar.gz) = 17708ee523ae601323ed24a39ce93cf4eb0444a6b21e1d491f27764e65f1d6654945ddf61d91e97209c64f7b873a9ba59494b5e8fd4d58d47f478f10f95e170a
+Size (zeroconf-0.32.0.tar.gz) = 90945 bytes

Index: pkgsrc/net/py-zeroconf/PLIST
diff -u pkgsrc/net/py-zeroconf/PLIST:1.3 pkgsrc/net/py-zeroconf/PLIST:1.4
--- pkgsrc/net/py-zeroconf/PLIST:1.3    Thu May  6 04:49:07 2021
+++ pkgsrc/net/py-zeroconf/PLIST        Fri Jul  2 09:54:09 2021
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.3 2021/05/06 04:49:07 adam Exp $
+@comment $NetBSD: PLIST,v 1.4 2021/07/02 09:54:09 adam Exp $
 ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
 ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
 ${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
@@ -8,13 +8,70 @@ ${PYSITELIB}/${EGG_INFODIR}/top_level.tx
 ${PYSITELIB}/zeroconf/__init__.py
 ${PYSITELIB}/zeroconf/__init__.pyc
 ${PYSITELIB}/zeroconf/__init__.pyo
+${PYSITELIB}/zeroconf/_cache.py
+${PYSITELIB}/zeroconf/_cache.pyc
+${PYSITELIB}/zeroconf/_cache.pyo
+${PYSITELIB}/zeroconf/_core.py
+${PYSITELIB}/zeroconf/_core.pyc
+${PYSITELIB}/zeroconf/_core.pyo
+${PYSITELIB}/zeroconf/_dns.py
+${PYSITELIB}/zeroconf/_dns.pyc
+${PYSITELIB}/zeroconf/_dns.pyo
+${PYSITELIB}/zeroconf/_exceptions.py
+${PYSITELIB}/zeroconf/_exceptions.pyc
+${PYSITELIB}/zeroconf/_exceptions.pyo
+${PYSITELIB}/zeroconf/_handlers.py
+${PYSITELIB}/zeroconf/_handlers.pyc
+${PYSITELIB}/zeroconf/_handlers.pyo
+${PYSITELIB}/zeroconf/_history.py
+${PYSITELIB}/zeroconf/_history.pyc
+${PYSITELIB}/zeroconf/_history.pyo
+${PYSITELIB}/zeroconf/_logger.py
+${PYSITELIB}/zeroconf/_logger.pyc
+${PYSITELIB}/zeroconf/_logger.pyo
+${PYSITELIB}/zeroconf/_protocol.py
+${PYSITELIB}/zeroconf/_protocol.pyc
+${PYSITELIB}/zeroconf/_protocol.pyo
+${PYSITELIB}/zeroconf/_services/__init__.py
+${PYSITELIB}/zeroconf/_services/__init__.pyc
+${PYSITELIB}/zeroconf/_services/__init__.pyo
+${PYSITELIB}/zeroconf/_services/browser.py
+${PYSITELIB}/zeroconf/_services/browser.pyc
+${PYSITELIB}/zeroconf/_services/browser.pyo
+${PYSITELIB}/zeroconf/_services/info.py
+${PYSITELIB}/zeroconf/_services/info.pyc
+${PYSITELIB}/zeroconf/_services/info.pyo
+${PYSITELIB}/zeroconf/_services/registry.py
+${PYSITELIB}/zeroconf/_services/registry.pyc
+${PYSITELIB}/zeroconf/_services/registry.pyo
+${PYSITELIB}/zeroconf/_services/types.py
+${PYSITELIB}/zeroconf/_services/types.pyc
+${PYSITELIB}/zeroconf/_services/types.pyo
+${PYSITELIB}/zeroconf/_updates.py
+${PYSITELIB}/zeroconf/_updates.pyc
+${PYSITELIB}/zeroconf/_updates.pyo
+${PYSITELIB}/zeroconf/_utils/__init__.py
+${PYSITELIB}/zeroconf/_utils/__init__.pyc
+${PYSITELIB}/zeroconf/_utils/__init__.pyo
+${PYSITELIB}/zeroconf/_utils/asyncio.py
+${PYSITELIB}/zeroconf/_utils/asyncio.pyc
+${PYSITELIB}/zeroconf/_utils/asyncio.pyo
+${PYSITELIB}/zeroconf/_utils/name.py
+${PYSITELIB}/zeroconf/_utils/name.pyc
+${PYSITELIB}/zeroconf/_utils/name.pyo
+${PYSITELIB}/zeroconf/_utils/net.py
+${PYSITELIB}/zeroconf/_utils/net.pyc
+${PYSITELIB}/zeroconf/_utils/net.pyo
+${PYSITELIB}/zeroconf/_utils/struct.py
+${PYSITELIB}/zeroconf/_utils/struct.pyc
+${PYSITELIB}/zeroconf/_utils/struct.pyo
+${PYSITELIB}/zeroconf/_utils/time.py
+${PYSITELIB}/zeroconf/_utils/time.pyc
+${PYSITELIB}/zeroconf/_utils/time.pyo
 ${PYSITELIB}/zeroconf/asyncio.py
 ${PYSITELIB}/zeroconf/asyncio.pyc
 ${PYSITELIB}/zeroconf/asyncio.pyo
+${PYSITELIB}/zeroconf/const.py
+${PYSITELIB}/zeroconf/const.pyc
+${PYSITELIB}/zeroconf/const.pyo
 ${PYSITELIB}/zeroconf/py.typed
-${PYSITELIB}/zeroconf/test.py
-${PYSITELIB}/zeroconf/test.pyc
-${PYSITELIB}/zeroconf/test.pyo
-${PYSITELIB}/zeroconf/test_asyncio.py
-${PYSITELIB}/zeroconf/test_asyncio.pyc
-${PYSITELIB}/zeroconf/test_asyncio.pyo