[pkgsrc/trunk]: pkgsrc/net/clive Update to 0.4.9:

[wiz <wiz%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/6d9ae30383b5
branches:  trunk
changeset: 541011:6d9ae30383b5
user:      wiz <wiz%pkgsrc.org@localhost>
date:      Sun Apr 13 09:58:48 2008 +0000

description:
Update to 0.4.9:

Version 0.4.9                                        [2008-04-09]
=================================================================
+new !fix ~change -remove | focus: major feature improvements

+ RSS/Atom parser (req. python newt module >= 0.51)
    Designed to be used with video host feeds. Parses and displays
    found videos available for extraction. If the required newt module
    is not found, the feature is disabled.

    Note that the python feedparser module is now part of the clive
    distribution, for more details, see the README file.

    Option:
        --rss (config: url_feed)

    When the --rss option is used, the URLs are treated as RSS/Atom feed URLs,
    for example:
        % clive --rss http://youtube.com/rss/global/recently_featured.rss

    Downstreamers: Python newt module is a new prerequisite. Note that the
    program still runs without it, only the --rss feature is disabled.

    FreeBSD: the newt library needs to be built with WITH_PYTHON,
    for example:
        % cd /usr/ports/devel/newt
        % make WITH_PYTHON=yes install clean

    Ubuntu: https://bugs.launchpad.net/ubuntu/+source/newt/+bug/110880

+ Dailymotion: Log-in support (sr #1834)
    Options:
        --dmotion-user=USERNAME (config: dmotion_id)
        --dmotion-pass=PASSWORD (config: dmotion_password)

+ Metacafe: Family filter off (sr #1956)
    The family filter is disabled for all extractions.

+ Youtube: Log-in support (sr #1953)
    Resurrected the log-in support that was previously available in 0.2.x
    but removed in 0.3.0 due to the complete program overhaul.

    Options:
        --youtube-user=USERNAME (config: youtube_id)
        --youtube-pass=PASSWORD (config: youtube_password)

! ~/.clive/config; improved parsing
    The parser is now more resistant to invalid values. clive will now also
    exit if an error occurs while parsing.

! ~/.clive/config; output_if_file_exists
    Accepts "RENAME" as it should have.

! Console; q_r_paste command
    Crash at: "UnboundLocalError: local variable 'cmd' referenced before assignment"

~ --write-conf; increased verbosity
    clive is now more verbose about found 3rd party programs.

~ --write-conf; sort commands
    The written commands are now sorted alphabetically.


Version 0.4.8                                        [2008-03-31]
=================================================================
+new !fix ~change -remove | focus: minor bugfixes

+ Youtube: HTTP/415 No media fallback
    Extract low quality (flv) video if better quality (mp4) is not available.

! Youtube: Partial support for continuing transfers
    It is now be possible to continue partial downloads as long as they are
    high-quality (mp4) videos.

    Note that this fails utterly for the flv videos. It appears urlgrabber
    appends a complete file at the end of the existing file when the host
    refuses to continue transfer. Instead of bloating the code with the
    conditions, continuing the youtube flv file transfers remains disabled.

! Dailymotion: improved media support
    clive no more assumes 512x384 is the highest available quality. The
    video page is now parsed for best available quality which clive extracts
    unless the --low-quality option is being used. It should be noted that
    most videos appear to have only the low-quality (flv) video available.

    No media warning will be displayed if high quality video is not found
    and the low quality video is extracted instead.

! Transfer rate calculations
    File length offset is now used properly with files that are being
    overwritten. This fixes the only-progress-percentage odditity is
    being displayed.

! Youtube: New error messages
    "error: country restriction/censorship"
    "error: private video"

! --check-update,-u
    Crash at: "TypeError: __init__() takes exactly 2 arguments (1 given)"

~ Reduce progress stdout flickering (sr #1944)
    Progress is now rendered in one second intervals. clive will now also
    display average transfer rate and total time elapsed when the transfer
    completes.

~ Clean up
    clive(1), INSTALL etc.


Version 0.4.7                                        [2008-03-17]
=================================================================
+new !fix ~change -remove | focus: major feature enhancements

+ GNU Autotools support
    See the INSTALL file for the updated installation details.

+ urlgrabber integration
    The urlgrabber module is now part of the source code distribution.
    For more info, please read the README file.

    Downstream maintainers: urlgrabber module is no longer a prerequisite

+ Cache
    If caching is enabled, clive will collect data about visited URLs
    to a sqlite3 database file (~/.clive/cache). Note that cache is only
    available for Python >= 2.5. Also, depending on the system, the sqlite3
    may not be installed with Python by default, e.g. on FreeBSD you need
    to install it separately (databases/py-sqlite3).

    Added also --no-cache and enable_cache. Caching is enabled by default.

+ Youtube: new default format
    clive now defaults to extract the better quality (mp4) videos from Youtube.
    The default behaviour can be overridden with the --low-quality option.

+ "Paste" option displayed at program start
    The state for "paste from xclip" is now displayed when the program starts.

! Same output filename in URL batch
    clive did not previously check the URLs in current batch for same output
    filename. For example:

        http://video.google.com/videoplay?docid=-4505462782975458603
        http://video.google.ca/videoplay?docid=-4505462782975458603

    Both URLs point to the same video, note the different TLD. If they were
    fed to clive in same URL batch the program accepted them without any
    scrutiny. This resulted clive to waste bandwidth downloading the same
    file twice and overwriting the previously extracted video file.

    clive now generates (--exist=rename) a new output filename for the last
    URL.

! Dailymotion support
    clive will now attempt to extract the higher resolution vide (mp4) _only_
    if it is available. This was causing HTTP/404 errors for some dmotion URLs
    with the previous release.

    The new option --low-quality overrides this default behaviour.

! Status/progress stdout line breaking on some system configurations
    This was caused by improper use of '\b' which would cause the lines
    to break on some terminal emulation settings.

! Calculations for resumed transfers (patch #990)
    The transfer rate calculations should no longer go berzerk for
    resumed transfers.

    "The problem is that when urlgrabber calls update(), the 'read'
    parameter is the current size of the file. When you resume, you
    need to discard the offset from the previous download or else
    the calculations will be off." -- zythmer

~ clive(1)
    The manual page was rewritten.

~ Accept-encoding is now always set to gzip
    Removed the rather useless --no-gzip option from wasting space.
    Note that compression is only used if the host supports it.

~ Configuration file format
    The configuration file format has undergone several iterations of
    changes over the last 12 months and this release introduces yet another
    format. Wait for it. This one's final. For a comparison:

    Then: {.., 'verbose':1, ..}
     Now: enable_verbose="YES"

    To get started, you can generate a "template" configuration file
    with:

        % clive --write-conf

    The created file contains comments and all supported commands. clive
    will also attempt to autodetect the 3rd party programs such as ffmpeg
    and set the paths to these programs with recommended options.

~ File name changes in ~/.clive
    config.py -> config
    last.log -> recall
    userdef.py -> custom.py

    Note that since history is no longer used, the ~/.clive/history.log is
    now obsolete and can be removed if it's still around.

~ Option --write-conf
    Previously known as --conf-write. Writes the config file with default
    program values. clive will now attempt to autodetect vlc, ffmpeg and
    xclip.

~ Option --emit
    Renamed to "--emit-csv". Standard output format was also changed.

    From:
        video: "extraction-url" "filename" "length"\n
    To:
        OK: "url","extraction-url","filename","length"\n
        FAILED: "url","error-message"\n

    Should be now more parsing-friendly.

~ Options changed
    Much of the options were either renamed, merged, removed or otherwise
    modified. Some new short options were added also.

~ Renamed --simulate
    Renamed --simulate,-s to --no-extract,-n.

~ Tweaked warnings
    Some of the warnings were rewritten.

~ Stdout overhaul
    Much of the stdout messages were changed, e.g. "status:" and similar
    were replaced with lines that make better use of the line space.

~ Options displayed at program start
    Renamed "throttle" to "rate". Renamed "exists" to "exist".

~ sys.platform instead of platform.uname
        Platform string is now constructed using the platform module.

- Logging features
    More or less replaced by the new "cache" and "recall" features.
    See also "File name changes" above.

- GUI features: --configure, --history
    Rather than transforming clive into a CLI-GUI hybrid, I've decided
    to remove the GUI elements. The code to these short-lived features
    could still be reused for a GUI-only clive configruation tool later
    if there's any demand for it.

    Downstream maintainers: Tkinter is no longer used by clive.

- Mswindows support
    Anyone familiar with the woe32 specific non-standard approach to
    everything can probably relate to this, and even if python hides
    much of the mswindows nonsense, I've decided not to bother myself
    with it anymore. So instead of bloating the code base with a set
    of conditions and exceptions for woe32, I've, hereby, removed
    all of the ms-platform specific nonsense from the project.

    Come to the unix side.

- distutils support (setup.py)
    Starting from this release, clive uses GNU Autotools.

- stage6 support
    Shut down service effective February 28, 2008
    (http://www.stage6.com/blog/107/).

diffstat:

 net/clive/Makefile |  16 +++++++++++-----
 net/clive/PLIST    |  49 ++++++++++++++++++++++++++++++++++++++++++-------
 net/clive/distinfo |   8 ++++----
 3 files changed, 57 insertions(+), 16 deletions(-)

diffs (128 lines):

diff -r 1363638401e6 -r 6d9ae30383b5 net/clive/Makefile
--- a/net/clive/Makefile        Sun Apr 13 09:57:57 2008 +0000
+++ b/net/clive/Makefile        Sun Apr 13 09:58:48 2008 +0000
@@ -1,22 +1,28 @@
-# $NetBSD: Makefile,v 1.2 2008/03/14 01:34:44 wiz Exp $
+# $NetBSD: Makefile,v 1.3 2008/04/13 09:58:48 wiz Exp $
 #
 
-DISTNAME=      clive-0.4.6
+DISTNAME=      clive-0.4.9
 CATEGORIES=    net multimedia www
 MASTER_SITES=  ${MASTER_SITE_SOURCEFORGE:=clive/} \
                http://dl.gna.org/clive/0.4/src/
+EXTRACT_SUFX=  .tar.bz2
 
 MAINTAINER=    wiz%NetBSD.org@localhost
 HOMEPAGE=      http://clive.sourceforge.net/
 COMMENT=       Command line video extraction tool for user-uploaded video hosts
 
-DEPENDS+=      urlgrabber>=2.9.9:../../www/urlgrabber
-DEPENDS+=      ${PYPKGPREFIX}-Tk-[0-9]*:../../x11/py-Tk
+DEPENDS+=      ${PYPKGPREFIX}-newt-[0-9]*:../../devel/py-newt
 
+GNU_CONFIGURE= yes
 USE_LANGUAGES= # none
 PYTHON_VERSIONS_ACCEPTED=      24 23 22
-PYDISTUTILSPKG=                        yes
 PY_PATCHPLIST=                 yes
 
+REPLACE_INTERPRETER+=  env_python
+REPLACE.env_python.old=        /usr/bin/env python
+REPLACE.env_python.new= ${PYTHONBIN}
+REPLACE_FILES.env_python=      src/clive/feedparser/feedparser.py
+REPLACE_FILES.env_python+=     src/scripts/clive run-tests.py run.py
+
 .include "../../lang/python/extension.mk"
 .include "../../mk/bsd.pkg.mk"
diff -r 1363638401e6 -r 6d9ae30383b5 net/clive/PLIST
--- a/net/clive/PLIST   Sun Apr 13 09:57:57 2008 +0000
+++ b/net/clive/PLIST   Sun Apr 13 09:58:48 2008 +0000
@@ -1,20 +1,29 @@
-@comment $NetBSD: PLIST,v 1.1.1.1 2008/03/14 01:32:48 wiz Exp $
+@comment $NetBSD: PLIST,v 1.2 2008/04/13 09:58:48 wiz Exp $
 bin/clive
 ${PYSITELIB}/clive/__init__.py
 ${PYSITELIB}/clive/__init__.pyc
 ${PYSITELIB}/clive/__init__.pyo
-${PYSITELIB}/clive/configure.py
-${PYSITELIB}/clive/configure.pyc
-${PYSITELIB}/clive/configure.pyo
+${PYSITELIB}/clive/cache.py
+${PYSITELIB}/clive/cache.pyc
+${PYSITELIB}/clive/cache.pyo
+${PYSITELIB}/clive/config.py
+${PYSITELIB}/clive/config.pyc
+${PYSITELIB}/clive/config.pyo
 ${PYSITELIB}/clive/console.py
 ${PYSITELIB}/clive/console.pyc
 ${PYSITELIB}/clive/console.pyo
 ${PYSITELIB}/clive/error.py
 ${PYSITELIB}/clive/error.pyc
 ${PYSITELIB}/clive/error.pyo
-${PYSITELIB}/clive/history.py
-${PYSITELIB}/clive/history.pyc
-${PYSITELIB}/clive/history.pyo
+${PYSITELIB}/clive/feedparser/__init__.py
+${PYSITELIB}/clive/feedparser/__init__.pyc
+${PYSITELIB}/clive/feedparser/__init__.pyo
+${PYSITELIB}/clive/feedparser/feedparser.py
+${PYSITELIB}/clive/feedparser/feedparser.pyc
+${PYSITELIB}/clive/feedparser/feedparser.pyo
+${PYSITELIB}/clive/login.py
+${PYSITELIB}/clive/login.pyc
+${PYSITELIB}/clive/login.pyo
 ${PYSITELIB}/clive/main.py
 ${PYSITELIB}/clive/main.pyc
 ${PYSITELIB}/clive/main.pyo
@@ -27,11 +36,37 @@
 ${PYSITELIB}/clive/parse.py
 ${PYSITELIB}/clive/parse.pyc
 ${PYSITELIB}/clive/parse.pyo
+${PYSITELIB}/clive/path.py
+${PYSITELIB}/clive/path.pyc
+${PYSITELIB}/clive/path.pyo
 ${PYSITELIB}/clive/progress.py
 ${PYSITELIB}/clive/progress.pyc
 ${PYSITELIB}/clive/progress.pyo
+${PYSITELIB}/clive/recall.py
+${PYSITELIB}/clive/recall.pyc
+${PYSITELIB}/clive/recall.pyo
+${PYSITELIB}/clive/rss.py
+${PYSITELIB}/clive/rss.pyc
+${PYSITELIB}/clive/rss.pyo
 ${PYSITELIB}/clive/update.py
 ${PYSITELIB}/clive/update.pyc
 ${PYSITELIB}/clive/update.pyo
+${PYSITELIB}/clive/urlgrabber/__init__.py
+${PYSITELIB}/clive/urlgrabber/__init__.pyc
+${PYSITELIB}/clive/urlgrabber/__init__.pyo
+${PYSITELIB}/clive/urlgrabber/byterange.py
+${PYSITELIB}/clive/urlgrabber/byterange.pyc
+${PYSITELIB}/clive/urlgrabber/byterange.pyo
+${PYSITELIB}/clive/urlgrabber/grabber.py
+${PYSITELIB}/clive/urlgrabber/grabber.pyc
+${PYSITELIB}/clive/urlgrabber/grabber.pyo
+${PYSITELIB}/clive/urlgrabber/keepalive.py
+${PYSITELIB}/clive/urlgrabber/keepalive.pyc
+${PYSITELIB}/clive/urlgrabber/keepalive.pyo
+${PYSITELIB}/clive/urlgrabber/sslfactory.py
+${PYSITELIB}/clive/urlgrabber/sslfactory.pyc
+${PYSITELIB}/clive/urlgrabber/sslfactory.pyo
 man/man1/clive.1
+@dirrm ${PYSITELIB}/clive/urlgrabber
+@dirrm ${PYSITELIB}/clive/feedparser
 @dirrm ${PYSITELIB}/clive
diff -r 1363638401e6 -r 6d9ae30383b5 net/clive/distinfo
--- a/net/clive/distinfo        Sun Apr 13 09:57:57 2008 +0000
+++ b/net/clive/distinfo        Sun Apr 13 09:58:48 2008 +0000
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.1.1.1 2008/03/14 01:32:48 wiz Exp $
+$NetBSD: distinfo,v 1.2 2008/04/13 09:58:48 wiz Exp $
 
-SHA1 (clive-0.4.6.tar.gz) = 94c730feaaac11bf2a0fcde14e477649ce7d895d
-RMD160 (clive-0.4.6.tar.gz) = fface2c3d8a001f4266dcdebaa673e5034c311e0
-Size (clive-0.4.6.tar.gz) = 62910 bytes
+SHA1 (clive-0.4.9.tar.bz2) = c3f71798a866bf64c68dba31faaccb2681535e2b
+RMD160 (clive-0.4.9.tar.bz2) = 369be7a7549c2195ea5390445cefec2ff9942537
+Size (clive-0.4.9.tar.bz2) = 163823 bytes