>Number:         21421
>Category:       pkg
>Synopsis:       updates and fixes for benchmarks/bonnie
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu May 01 21:34:00 UTC 2003
>Closed-Date:
>Last-Modified:
>Originator:     Greg A. Woods
>Release:        pkgsrc-20030430
>Organization:
Planix, Inc.; Toronto, Ontario; Canada
>Environment:
Architecture: i386
Machine: i386
>Description:

	benchmarks/bonnie suffers from a number of problems:

	- a bad cast to lseek() offset parameter

	- inability to compare "apples to apples" with FreeBSD's port
	  which uses only 8192-byte blocks, so to keep things fair for
	  everyone the chunk size needs to be adjustable at runtime

	- ugly signed values are printed for >31-bit sizes and offsets.

	- the DESCR content is poorly worded

	- the MESSAGE is misleading

	- the default benchmark run doesn't take available RAM into account

>How-To-Repeat:

	run bonnie on a fast and high-capacity machine with '-s' > 2048

>Fix:

Index: DESCR
===================================================================
RCS file: /cvs/master/m-NetBSD/main/pkgsrc/benchmarks/bonnie/DESCR,v
retrieving revision 1.1
diff -c -u -r1.1 DESCR
--- DESCR	31 Oct 2001 23:45:08 -0000	1.1
+++ DESCR	1 May 2003 21:07:07 -0000
@@ -1,17 +1,25 @@
-Bonnie: Filesystem Benchmark Program
+Bonnie:  A simple Filesystem Benchmark Program
 
-Bonnie tests the speed of file I/O using standard C library calls.
-It does reads and writes of blocks, testing for the limit of sustained
-data rate (usually limited by the drive or controller) and updates on
-a file (better simulating normal operating conditions and quite dependent 
-on drive and OS optimisations).
-The per character read and write tests are generally limited by CPU speed
-only on current generation hardware. It takes some 35 SPECint92 to read
-or write a file at a rate of 1MB/s using getc() and putc().
-The seek tests are dependent on the buffer cache size, since the fraction
-of disk blocks that fits into the buffer cache will be found without any 
-disk operation and will contribute zero seek time readings. I.e. if the 
-buffer cache is 16MB and the Bonnie test file is 32MB in size, then the 
-seek time will come out as half its real value. The seek time includes 
-rotational delay, and will thus always come out higher than specified for 
-a drive.
+Bonnie tests the speed of sequential file I/O using standard C library
+calls.  It does reads and writes of characters using STDIO; and of
+blocks using system calls.  Then it does a sequential pass of reading
+every block, modifying one byte, and writing it back; and finally it
+does random seeks followed by reads and writes of blocks.  It reports
+the average sustained data rate (usually limited by the drive or
+controller).  It is quite dependent on hardware and OS optimisations.
+
+The per character read and write tests are generally limited by CPU
+speed except on the fastest of systems.  It takes some 35 SPECint92 to
+read or write a file at a rate of 1MB/s using getc() and putc().
+
+The input tests, particularly the block reads, are highly dependent on
+the buffer cache size.  A file size of at least four times available RAM
+is needed to effectivley mask the effects of the buffer cache.
+
+The seek tests are also dependent on the buffer cache size, since the
+fraction of disk blocks which fit into the buffer cache will be found
+without any actual disk access and will contribute little or no apparent
+seek time to the mesurements.  I.e. if the buffer cache is 16MB and the
+Bonnie test file is 32MB in size, then the seek time will come out as
+about half its real value.  The seek time includes rotational delay, and
+will thus always come out higher than the specified value for a drive.
Index: Makefile
===================================================================
RCS file: /cvs/master/m-NetBSD/main/pkgsrc/benchmarks/bonnie/Makefile,v
retrieving revision 1.19
diff -c -u -r1.19 Makefile
--- Makefile	29 Mar 2003 12:40:10 -0000	1.19
+++ Makefile	1 May 2003 20:53:50 -0000
@@ -1,16 +1,16 @@
 # $NetBSD: Makefile,v 1.19 2003/03/29 12:40:10 jmmv Exp $
-# FreeBSD Id: Makefile,v 1.5 1997/11/26 23:09:14 jseger Exp
 #
 
 DISTNAME=       bonnie
 PKGNAME=	bonnie-2.06
+PKGREVISION=	1
 WRKSRC=		${WRKDIR}
 CATEGORIES=	benchmarks
 MASTER_SITES=   http://www.textuality.com/bonnie/
 
 MAINTAINER=	root@garbled.net
 HOMEPAGE=	http://www.textuality.com/bonnie/index.html
-COMMENT=	Performance Test of Filesystem I/O
+COMMENT=	Performance Test of Sequential Filesystem I/O and STDIO
 
 NO_CONFIGURE=	yes
 
@@ -19,11 +19,13 @@
 post-extract:
 	${MV} ${WRKSRC}/Bonnie.c ${WRKSRC}/bonnie.c
 
+SYSCTL?=	/sbin/sysctl
+
 benchmark:
-	@(cd ${WRKSRC}; ./bonnie | tee bonnie.out)
+	@(cd ${WRKSRC}; ./bonnie -m nb$$(uname -r) -s $$(${EXPR} 4 \* $$(${SYSCTL} -n hw.physmem) / 1024 / 1024) | tee bonnie.out)
 
 results:
-	@${MKDIR} /tmp/benches/`domainname`
-	-@${CP} ${WRKSRC}/bonnie.out /tmp/benches/`domainname`/bonnie.`uname`-`uname -m`-`uname -r`.`hostname`
+	@${MKDIR} /tmp/benches/$$(domainname)
+	-@${CP} ${WRKSRC}/bonnie.out /tmp/benches/$$(domainname)/bonnie.$$(uname)-$$(uname -m)-$$(uname -r).$$(hostname)
 
 .include "../../mk/bsd.pkg.mk"
Index: distinfo
===================================================================
RCS file: /cvs/master/m-NetBSD/main/pkgsrc/benchmarks/bonnie/distinfo,v
retrieving revision 1.4
diff -c -u -r1.4 distinfo
--- distinfo	9 Jun 2002 00:25:12 -0000	1.4
+++ distinfo	1 May 2003 21:33:26 -0000
@@ -3,4 +3,5 @@
 SHA1 (bonnie.tar.gz) = f435ae56903883c8e3e011eb989783c8410f5ee3
 Size (bonnie.tar.gz) = 7140 bytes
 SHA1 (patch-aa) = 33ac03e1c70bba2aedf0482e23c16cf19956b47e
-SHA1 (patch-ac) = c05ff777135943cff5439010d6d3b420e9cc84a0
+SHA1 (patch-ab) = 927415c66b7c03897aa4cbee07cd54c242272148
+SHA1 (patch-ac) = 40ef6b029df4c287943bcc9dda22c54b64571445
Index: patches/patch-ab
===================================================================
RCS file: patches/patch-ab
diff -N patches/patch-ab
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ patches/patch-ab	1 May 2003 21:33:23 -0000
@@ -0,0 +1,60 @@
+# $NetBSD$
+
+--- bonnie.1.orig	Wed Aug 28 21:42:42 1996
++++ bonnie.1	Thu May  1 17:26:39 2003
+@@ -60,26 +60,43 @@
+ .fi
+ .in
+ ..
+-.TH Bonnie 1 "2.0.6" Textuality
++.TH bonnie 1 "2.0.6" Textuality
+ .SH NAME
+-Bonnie - File system benchmark
++Bonnie \- File system benchmark
+ .SH "SYNOPSIS"
+-.B Bonnie
+-.RI "[-d\ scratch-dir]"
+-.RI "[-html]"
+-.RI "[-m\ machine-label]"
+-.RI "[-s\ size-in-Mb]"
++.B bonnie
++.RI "[\-c\ block-in-bytes]"
++.RI "[\-d\ scratch-dir]"
++.RI "[\-html]"
++.RI "[\-m\ machine-label]"
++.RI "[\-s\ size-in-Mb]"
+ .SH "OPTIONS"
+ .PP
+-  -d scratch-dir   : Write scratch file in named directory (default: ".")
+-  -html            : Generate HTML output
+-  -m machine-label : use <machine-label> to label report
+-  -s size-in-Mb    : how many Mb to use for testing (default: 100)
++.TP
++\-c block-in-bytes
++how many bytes to make each block for block I/O (default: 16384)
++.TP
++\-d scratch-dir
++Write scratch file in named directory (default: ".")
++.TP
++\-html
++Generate HTML output
++.TP
++\-m machine-label
++use <machine-label> to label report
++.TP
++\-s size-in-Mb
++how many Mb to use for testing (default: 100)
+ .\"-------
+ .SH "DESCRIPTION"
+ .\"-------
+ .I Bonnie
+-measures the performance of UNIX filesystem operations.
++is a file system benchmark which attempts to study bottlenecks.
++It is named `Bonnie' after Bonnie Raitt, who knows how to use one.
++.PP
++.I Bonnie
++measures the performance of sequential filesystem operations.
++.PP
+ For details, see http://www.textuality.com/bonnie/
+ .\"-------
+ .SH "AUTHOR"
Index: patches/patch-ac
===================================================================
RCS file: /cvs/master/m-NetBSD/main/pkgsrc/benchmarks/bonnie/patches/patch-ac,v
retrieving revision 1.1
diff -c -u -r1.1 patch-ac
--- patches/patch-ac	9 Jun 2002 00:24:30 -0000	1.1
+++ patches/patch-ac	30 Apr 2003 04:10:21 -0000
@@ -1,13 +1,113 @@
 $NetBSD: patch-ac,v 1.1 2002/06/09 00:24:30 grant Exp $
 
---- bonnie.c.orig	Thu Aug 29 02:23:49 1996
-+++ bonnie.c	Sun Jun  9 10:20:02 2002
-@@ -458,7 +458,7 @@
+--- bonnie.c.orig	Wed Aug 28 12:23:49 1996
++++ bonnie.c	Tue Apr 29 23:22:39 2003
+@@ -25,6 +25,7 @@
+ 
+ #include <unistd.h>
+ #include <stdio.h>
++#include <stdlib.h>
+ #include <errno.h>
+ #include <fcntl.h>
+ #include <sys/types.h>
+@@ -49,7 +50,8 @@
+ #define Seeks (4000)
+ #define UpdateSeek (10)
+ #define SeekProcCount (3)
+-#define Chunk (16384)
++
++unsigned int Chunk = 16384;
+ 
+ /* labels for the tests, used as an array index */
+ typedef enum
+@@ -91,7 +93,7 @@
+   int    argc,
+   char * argv[])
+ {
+-  int    buf[Chunk / IntSize];
++  int    *buf;
+   int    bufindex;
+   int    chars[256];
+   int    child;
+@@ -122,6 +124,8 @@
+   for (next = 1; next < argc; next++)
+     if (strcmp(argv[next], "-d") == 0)
+       dir = argv[++next];
++    else if (strcmp(argv[next], "-c") == 0)
++      Chunk = atoi(argv[++next]);
+     else if (strcmp(argv[next], "-s") == 0)
+       size = atol(argv[++next]);
+     else if (strcmp(argv[next], "-m") == 0)
+@@ -146,7 +150,12 @@
+   /* size is in meg, rounded down to multiple of Chunk */
+   size *= (1024 * 1024);
+   size = Chunk * (size / Chunk);
+-  fprintf(stderr, "File '%s', size: %ld\n", name, size);
++  fprintf(stderr, "File '%s', size: %qu\n", name, (u_quad_t) size);
++
++  if (!(buf = (int *) malloc(Chunk / IntSize))) {
++    fprintf(stderr, "Error allocating buffer memory: %s\n", strerror(errno));
++    exit(1);
++  }     
+ 
+   /* Fill up a file, writing it a char at a time with the stdio putc() call */
+   fprintf(stderr, "Writing with putc()...");
+@@ -288,6 +297,7 @@
+     { /* child process */
+ 
+       /* set up and wait for the go-ahead */
++      close(0);					/* From FreeBSD */
+       close(seek_feedback[0]);
+       close(seek_control[1]);
+       newfile(name, &fd, &stream, 0);
+@@ -303,7 +313,13 @@
+       /* loop until we read a 0 ticket back from our parent */
+       while(seek_tickets[0])
+       { /* until Mom says stop */
+-        doseek((long) (random() % (size / Chunk)), fd,
++        off_t seekto;
++
++        if (size < ((off_t)1 << 32))		/* From FreeBSD */
++          seekto = random() % (size / Chunk);
++        else
++          seekto = ((off_t)random() << 32 + random()) % (size / Chunk);
++        doseek(seekto, fd,
+ 	  ((lseek_count++ % UpdateSeek) == 0));
+ 	if (read(seek_control[0], seek_tickets, 1) != 1)
+ 	  io_error("read ticket");
+@@ -413,7 +429,7 @@
+   printf("K/sec %%CPU K/sec %%CPU K/sec %%CPU K/sec %%CPU K/sec ");
+   printf("%%CPU  /sec %%CPU\n");
+ 
+-  printf("%-8.8s %4d ", machine, size / (1024 * 1024));
++  printf("%-8.8s %4qu ", machine, (u_quad_t) size / (1024 * 1024));
+   printf("%5d %4.1f %5d %4.1f %5d %4.1f ",
+     (int) (((double) size) / (delta[(int) Putc][Elapsed] * 1024.0)),
+     delta[(int) Putc][CPU] / delta[(int) Putc][Elapsed] * 100.0,
+@@ -458,7 +474,7 @@
  usage()
  {
    fprintf(stderr,
 -    "usage: Bonnie [-d scratch-dir] [-s size-in-Mb] [-html] [-m machine-label]\n");
-+    "usage: bonnie [-d scratch-dir] [-s size-in-Mb] [-html] [-m machine-label]\n");
++    "usage: bonnie [-c chunk-size] [-d scratch-dir] [-s size-in-Mb] [-html] [-m machine-label]\n");
+   exit(1);
+ }
+ 
+@@ -529,7 +545,7 @@
+ {
+   char buf[Chunk];
+ 
+-  sprintf(buf, "Bonnie: drastic I/O error (%s)", message);
++  sprintf(buf, "\nBonnie: drastic I/O error (%s)", message);
+   perror(buf);
    exit(1);
  }
+@@ -568,7 +584,7 @@
  
+     /* touch a word */
+     buf[((int) random() % (size/IntSize - 2)) + 1]--;
+-    if (lseek(fd, (long) probe, 0) != probe)
++    if (lseek(fd, probe, 0) != probe)
+       io_error("lseek in doseek update");
+     if (write(fd, (char *) buf, size) == -1)
+       io_error("write in doseek");
>Release-Note:
>Audit-Trail:
>Unformatted: