[pkgsrc/pkgsrc-2019Q4]: pkgsrc/net/youtube-dl Pullup ticket #6124 - requested...

[bsiegert <bsiegert%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/49cd20ec2636
branches:  pkgsrc-2019Q4
changeset: 419726:49cd20ec2636
user:      bsiegert <bsiegert%pkgsrc.org@localhost>
date:      Mon Jan 27 11:04:38 2020 +0000

description:
Pullup ticket #6124 - requested by leot
net/youtube-dl: update

Revisions pulled up:
- net/youtube-dl/Makefile                                       1.198-1.199
- net/youtube-dl/PLIST                                          1.98
- net/youtube-dl/distinfo                                       1.180-1.181
- net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py     1.1

---
   Module Name:    pkgsrc
   Committed By:   leot
   Date:           Tue Jan 14 20:42:39 UTC 2020

   Modified Files:
           pkgsrc/net/youtube-dl: Makefile PLIST distinfo

   Log Message:
   youtube-dl: Update to 20200115

   pkgsrc changes:
    - Apply a pkgsrc patch to fix la7 extractor (shared upstream)

   Changes:
   20200115
   --------
   Extractors
   * [yourporn] Fix extraction (#21645, #22255, #23459)
   + [canvas] Add support for new API endpoint (#17680, #18629)
   * [ndr:base:embed] Improve thumbnails extraction (#23731)
   + [vodplatform] Add support for embed.kwikmotion.com domain
   + [twitter] Add support for promo_video_website cards (#23711)
   * [orf:radio] Clean description and improve extraction
   * [orf:fm4] Fix extraction (#23599)
   * [safari] Fix kaltura session extraction (#23679, #23670)
   * [lego] Fix extraction and extract subtitle (#23687)
   * [cloudflarestream] Improve extraction
       + Add support for bytehighway.net domain
       + Add support for signed URLs
       + Extract thumbnail
   * [naver] Improve extraction
       * Improve geo-restriction handling
       + Extract automatic captions
       + Extract uploader metadata
       + Extract VLive HLS formats
       * Improve metadata extraction
   - [pandatv] Remove extractor (#23630)
   * [dctp] Fix format extraction (#23656)
   + [scrippsnetworks] Add support for www.discovery.com videos
   * [discovery] Fix anonymous token extraction (#23650)
   * [nrktv:seriebase] Fix extraction (#23625, #23537)
   * [wistia] Improve format extraction and extract subtitles (#22590)
   * [vice] Improve extraction (#23631)
   * [redtube] Detect private videos (#23518)

---
   Module Name:    pkgsrc
   Committed By:   leot
   Date:           Wed Jan 15 08:22:03 UTC 2020

   Added Files:
           pkgsrc/net/youtube-dl/patches: patch-youtube__dl_extractor_la7.py

   Log Message:
   youtube-dl: Add missing patch (should be part of previous update)

   Thanks <wiz>!

---
   Module Name:    pkgsrc
   Committed By:   adam
   Date:           Fri Jan 24 08:08:24 UTC 2020

   Modified Files:
           pkgsrc/net/youtube-dl: Makefile distinfo

   Log Message:
   youtube-dl: updated to 20200124

   version 2020.01.24

   Extractors
   * [youtube] Fix sigfunc name extraction
   * [stretchinternet] Fix extraction
   * [voicerepublic] Fix extraction
   * [azmedien] Fix extraction
   * [businessinsider] Fix jwplatform id extraction
   + [24video] Add support for 24video.vip
   * [ivi:compilation] Fix entries extraction
   * [ard] Improve extraction
       * Simplify extraction
       + Extract age limit and series
       * Bypass geo-restriction
   + [nbc] Add support for nbc multi network URLs
   * [americastestkitchen] Fix extraction
   * [zype] Improve extraction
       + Extract subtitles
       + Support URLs with alternative keys/tokens
       + Extract more metadata
   * [orf:tvthek] Improve geo restricted videos detection
   * [soundcloud] Restore previews extraction

diffstat:

 net/youtube-dl/Makefile                                   |   4 +-
 net/youtube-dl/PLIST                                      |   5 +-
 net/youtube-dl/distinfo                                   |  11 +-
 net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py |  75 +++++++++++++++
 4 files changed, 84 insertions(+), 11 deletions(-)

diffs (131 lines):

diff -r c8adc8d99c01 -r 49cd20ec2636 net/youtube-dl/Makefile
--- a/net/youtube-dl/Makefile   Mon Jan 27 10:59:04 2020 +0000
+++ b/net/youtube-dl/Makefile   Mon Jan 27 11:04:38 2020 +0000
@@ -1,8 +1,8 @@
-# $NetBSD: Makefile,v 1.196 2019/12/26 22:18:18 leot Exp $
+# $NetBSD: Makefile,v 1.196.4.1 2020/01/27 11:04:38 bsiegert Exp $
 
 # XXX: VERSION_DATE can contains also an optional part that indicates
 # XXX: possible same day revisions. PKGNAME preserves that dotted part as is.
-VERSION_DATE=  2019.12.25
+VERSION_DATE=  2020.01.24
 DISTNAME=      youtube-dl-${VERSION_DATE}
 PKGNAME=       ${DISTNAME:S/.//:S/.//}
 CATEGORIES=    net
diff -r c8adc8d99c01 -r 49cd20ec2636 net/youtube-dl/PLIST
--- a/net/youtube-dl/PLIST      Mon Jan 27 10:59:04 2020 +0000
+++ b/net/youtube-dl/PLIST      Mon Jan 27 11:04:38 2020 +0000
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.97 2019/12/26 22:12:27 leot Exp $
+@comment $NetBSD: PLIST,v 1.97.4.1 2020/01/27 11:04:38 bsiegert Exp $
 bin/youtube-dl
 ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
 ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
@@ -1358,9 +1358,6 @@
 ${PYSITELIB}/youtube_dl/extractor/packtpub.py
 ${PYSITELIB}/youtube_dl/extractor/packtpub.pyc
 ${PYSITELIB}/youtube_dl/extractor/packtpub.pyo
-${PYSITELIB}/youtube_dl/extractor/pandatv.py
-${PYSITELIB}/youtube_dl/extractor/pandatv.pyc
-${PYSITELIB}/youtube_dl/extractor/pandatv.pyo
 ${PYSITELIB}/youtube_dl/extractor/pandoratv.py
 ${PYSITELIB}/youtube_dl/extractor/pandoratv.pyc
 ${PYSITELIB}/youtube_dl/extractor/pandoratv.pyo
diff -r c8adc8d99c01 -r 49cd20ec2636 net/youtube-dl/distinfo
--- a/net/youtube-dl/distinfo   Mon Jan 27 10:59:04 2020 +0000
+++ b/net/youtube-dl/distinfo   Mon Jan 27 11:04:38 2020 +0000
@@ -1,9 +1,10 @@
-$NetBSD: distinfo,v 1.178 2019/12/26 22:12:27 leot Exp $
+$NetBSD: distinfo,v 1.178.4.1 2020/01/27 11:04:38 bsiegert Exp $
 
-SHA1 (youtube-dl-2019.12.25.tar.gz) = 1658783c89b9e1a941e6fa187f9640c8eab94704
-RMD160 (youtube-dl-2019.12.25.tar.gz) = 288bf6a6dc7c31172babb1923167c900b3e642b9
-SHA512 (youtube-dl-2019.12.25.tar.gz) = 951219648597d5e19fd3bcb3c64d8641bffd63cf3c6b4b30332d9148ad8b12c1be252e4cacbcd8169ce502b0fe20b2ff7163620382769b5e83e82d155e4f38c8
-Size (youtube-dl-2019.12.25.tar.gz) = 3153090 bytes
+SHA1 (youtube-dl-2020.01.24.tar.gz) = 29d0dd957ceacb10e773ac8f865c75b9d2ed5bc3
+RMD160 (youtube-dl-2020.01.24.tar.gz) = 64b072449da28ab53728a35a46454486cd58c3ed
+SHA512 (youtube-dl-2020.01.24.tar.gz) = 8f85a218395d2932b26c7b76d5b03cbdf9a7c541415c8d1b805239b6ab1ca74b94fc497f32052a1d93fe20ac760994975a13551dd8c5319f0549af20b70a1a78
+Size (youtube-dl-2020.01.24.tar.gz) = 3159926 bytes
 SHA1 (patch-setup.py) = a67074ae7cfe5e77847c2f610337ea553eddb69b
+SHA1 (patch-youtube__dl_extractor_la7.py) = d104c32abb21f9dfe982c0f42e049aeff0f50d67
 SHA1 (patch-youtube__dl_extractor_rai.py) = ae67a6fb599c90491fd68b72bf71821659a2eca4
 SHA1 (patch-youtube__dl_postprocessor_ffmpeg.py) = f96676170a448d9205d542a7def4beca615a1490
diff -r c8adc8d99c01 -r 49cd20ec2636 net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/net/youtube-dl/patches/patch-youtube__dl_extractor_la7.py Mon Jan 27 11:04:38 2020 +0000
@@ -0,0 +1,75 @@
+$NetBSD: patch-youtube__dl_extractor_la7.py,v 1.1.2.2 2020/01/27 11:04:38 bsiegert Exp $
+
+[la7] Fix extraction
+
+Parsing `videoParams' or `videoLa7' JavaScript snippets as JSON is getting too
+for js_to_json.  Just extract the `vid' from there and use _search_og_* for all
+other data.
+
+Update 2nd test with an existent video.
+
+Closes #23323.
+
+Shared upstream via:
+
+ <https://github.com/ytdl-org/youtube-dl/pull/23641>
+
+--- youtube_dl/extractor/la7.py.orig
++++ youtube_dl/extractor/la7.py
+@@ -3,7 +3,6 @@ from __future__ import unicode_literals
+ 
+ from .common import InfoExtractor
+ from ..utils import (
+-    js_to_json,
+     smuggle_url,
+ )
+ 
+@@ -31,12 +30,16 @@ class LA7IE(InfoExtractor):
+         },
+     }, {
+         # 'src' is a dictionary
+-        'url': 'http://tg.la7.it/repliche-tgla7?id=189080',
+-        'md5': '6b0d8888d286e39870208dfeceaf456b',
++        'url': 'http://tg.la7.it/repliche-tgla7?id=300465',
++        'md5': '9440057f4531005c426421487770b756',
+         'info_dict': {
+-            'id': '189080',
++            'id': '0_i2fn53i6',
+             'ext': 'mp4',
+-            'title': 'TG LA7',
++            'title': 'Repliche Tgla7',
++            'description': 'Repliche Tgla7',
++            'uploader_id': 'kdla7pillole%iltrovatore.it@localhost',
++            'timestamp': 1578231030,
++            'upload_date': '20200105',
+         },
+     }, {
+         'url': 'http://www.la7.it/omnibus/rivedila7/omnibus-news-02-07-2016-189077',
+@@ -48,20 +51,19 @@ class LA7IE(InfoExtractor):
+ 
+         webpage = self._download_webpage(url, video_id)
+ 
+-        player_data = self._parse_json(
+-            self._search_regex(
+-                [r'(?s)videoParams\s*=\s*({.+?});', r'videoLa7\(({[^;]+})\);'],
+-                webpage, 'player data'),
+-            video_id, transform_source=js_to_json)
++        player_data = self._search_regex(
++            [r'(?s)videoParams\s*=\s*({.+?});', r'videoLa7\(({[^;]+})\);'],
++            webpage, 'player data')
++        vid = self._search_regex(r'vid\s*:\s*"(.+?)",', player_data, 'vid')
+ 
+         return {
+             '_type': 'url_transparent',
+-            'url': smuggle_url('kaltura:103:%s' % player_data['vid'], {
++            'url': smuggle_url('kaltura:103:%s' % vid, {
+                 'service_url': 'http://nkdam.iltrovatore.it',
+             }),
+             'id': video_id,
+-            'title': player_data['title'],
++            'title': self._og_search_title(webpage, default=None),
+             'description': self._og_search_description(webpage, default=None),
+-            'thumbnail': player_data.get('poster'),
++            'thumbnail': self._og_search_thumbnail(webpage, default=None),
+             'ie_key': 'Kaltura',
+         }