Subject: Re: dd and pipe question
To: None <current-users@NetBSD.ORG>
From: der Mouse <mouse@Collatz.McRCIM.McGill.EDU>
List: current-users
Date: 04/11/1996 08:14:18
> Please respond directly to me via email.
I'm sending this to both you and current-users...
> I have a compressed tar image on disk, in file "foo.tar.gz". I wish
> to write it out to a fixed block size tape (an 8mm, with a 1k block
> size). My preliminary pass is:
> gzip -dc foo.tar.gz | dd ibs=10240 obs=10240 conv=sync of=/dev/tape
> However, (probably because of the pipe) there are _multiple_ partial
> input blocks.
Yes. dd is not the right tool here. The effect you want can be
expressed as "dd ibs=1 obs=10240 ....", but that is ruinously
inefficient.
With most systems, you can use dd obs=, without specifying an ibs (or
equivalently specify ibs=512, which is the default), and it will "just
work", but this depends on (a) the pipe writer always writing in
multiples of 512 bytes and (b) the pipe never breaking a 512-byte
chunk. Neither of these is guaranteed, of course, though I think with
gzip as the writer and a NetBSD-current system doing the pipe it will
work.
Unfortunately, I know of no other tool shipped with NetBSD that is
capable of blocking out writes like this. I have one, called
"catblock", designed specifically for reading its input as a
byte-stream and writing it as a block-stream. It's small; I'll include
it below. (In the particular case of a tar file, you could also use my
tar - "gunzip ... | tar P > /dev/tape" - , but in this case it's
overkill to tear the tar archive apart and reconstruct it just to get
tape blocking right.)
Here's catblock.c. I've tried to make sure it'll compile on a stock
system; YMMV, but I'll be glad to correspond with anyone who has
trouble building it.
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <strings.h>
char **argvec;
static int blocksize;
static int padsize;
static char *buf;
static int ifill;
static void padwrite(void)
{
int nw;
if (ifill % padsize)
{ bzero(buf+ifill,padsize-(ifill%padsize));
ifill = ((ifill / padsize) + 1) * padsize;
}
nw = write(1,buf,ifill);
if (nw < 0)
{ fprintf(stderr,"%s: write error: %s\n",argvec[0],strerror(errno));
exit(1);
}
if (nw != ifill)
{ fprintf(stderr,"%s: short write (%d instead of %d)\n",argvec[0],nw,ifill);
exit(1);
}
ifill = 0;
}
int main(int, char **);
int main(int ac, char **av)
{
int nr;
argvec=av;
if ((ac < 2) || (ac > 3))
{ fprintf(stderr,"Usage: %s blocksize [padsize]\n",argvec[0]);
exit(1);
}
blocksize = atoi(av[1]);
if (blocksize < 1)
{ fprintf(stderr,"%s: invalid blocksize, must be >= 1\n",argvec[0]);
exit(1);
}
buf = malloc(blocksize);
padsize = (ac > 2) ? atoi(av[2]) : 1;
if (padsize < 1)
{ fprintf(stderr,"%s: invalid padsize, must be >= 1\n",argvec[0]);
exit(1);
}
if (padsize > blocksize)
{ fprintf(stderr,"%s: invalid padsize, must be < blocksize\n",argvec[0]);
exit(1);
}
if (blocksize % padsize)
{ fprintf(stderr,"%s: invalid padsize, must divide blocksize\n",argvec[0]);
exit(1);
}
ifill = 0;
while (1)
{ nr = read(0,buf+ifill,blocksize-ifill);
if (nr < 0)
{ fprintf(stderr,"%s: input read error: %s\n",argvec[0],strerror(errno));
if (ifill > 0) padwrite();
exit(1);
}
if (nr == 0)
{ if (ifill > 0) padwrite();
exit(0);
}
ifill += nr;
if (ifill > blocksize)
{ fprintf(stderr,"%s: BUGCHECK: ifill > blocksize\n",argvec[0]);
abort();
}
else if (ifill == blocksize)
{ padwrite();
}
}
}
der Mouse
mouse@collatz.mcrcim.mcgill.edu