Subject: rather abysmal performance of good old unix compress on NetBSD....
To: None <current-users@NetBSD.ORG>
From: Greg A. Woods <woods@most.weird.com>
List: current-users
Date: 11/23/1997 02:19:49
I turned on savecore -z the other day to save a few megabytes of space
on my test sun3 that was panicing left right and centre.  I noted that
it seemed to take an *exceptionally* long time to compress the output
even when it was likely that most of RAM had not yet been touched since
being initialized by the PROM.

This lead me to look into the compress implementation on NetBSD.  It
seems the more "recent" improvements in compress were never returned to
the BSD tree and it sure shows!

This is with a NetBSD/sun3-97/11/17 kernel on a Sun-3/260 w/32MB RAM.

02:05 [9] $ /usr/bin/time wc /netbsd > /dev/null
        2.94 real         1.92 user         0.91 sys
02:05 [10] $ /usr/bin/time wc /netbsd > /dev/null 
        2.36 real         1.86 user         0.47 sys
02:05 [11] $ /usr/bin/time wc /netbsd > /dev/null 
        2.36 real         1.88 user         0.51 sys
02:05 [12] $ /usr/bin/time /usr/bin/compress -v < /netbsd > /dev/null
       26.25 real        25.36 user         0.76 sys
02:06 [13] $ /usr/bin/time /usr/bin/compress -v < /netbsd > /dev/null 
       26.08 real        25.37 user         0.69 sys
02:06 [14] $ /usr/bin/time /usr/bin/compress -v < /netbsd > /dev/null 
       26.29 real        25.43 user         0.70 sys
02:08 [15] $ /usr/bin/time ./compress42 -v < /netbsd > /dev/null
Compression: 34.46%
       11.39 real         9.66 user         1.56 sys
02:08 [16] $ /usr/bin/time ./compress42 -v < /netbsd > /dev/null 
Compression: 34.46%
       11.28 real         9.84 user         1.07 sys
02:08 [17] $ /usr/bin/time ./compress42 -v < /netbsd > /dev/null 
Compression: 34.46%
       10.87 real         9.72 user         1.16 sys
02:08 [18] $ ./compress42 -V                                     
Compress version: (N)compress 4.2.4, compiled: Sun Nov 23 02:07:17 EST 1997
Compile options:
        FAST, DIRENT, LSTAT, 
        REGISTERS=14 IBUFSIZ=1024, OBUFSIZ=1024, BITS=16

Author version 4.2 (Speed improvement & source cleanup):
     Peter Jannesen  (peter@ncs.nl)

Author version 4.1 (Added recursive directory compress):
     Dave Mack  (csu@alembic.acs.com)

Authors version 4.0 (World release in 1985):
     Spencer W. Thomas, Jim McKie, Steve Davies,
     Ken Turkowski, James A. Woods, Joe Orost
02:09 [19] $ /usr/bin/compress -V
compress: illegal option -- V
usage: compress [-cfv] [-b bits] [file ...]

(N)compres-4.2.4 was compiled with '-O' and with 1024 byte buffers and
with USERMEM=800000.

It's not clear from any comments exactly which version is in NetBSD's
/usr/src/usr.bin/compress/zopen.c but it seems it might even be nearly
the original 4.0.  The performance improvements in 4.2.4 (N)compress are
definitely worth having, esp. for those of us who like to avoid GNU
stuff at times!  ;-)

You'll also note that (N)compress has several other improvements such as
not spewing it's verbose output to stdout and thus not corrupting the
output file when I/O redirection is used....

Of course one could also ask the other question of why savecore doesn't
use gzip, but that is definitely an entirely separate question!

-- 
							Greg A. Woods

+1 416 443-1734      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>