On Thu, Dec 02, 2004 at 08:34:53PM -0500, Dan LaBell wrote:
> 
> On Thursday, December 2, 2004, at 01:58 PM, Vincent Stemen wrote:
> 
> >On Thu, Dec 02, 2004 at 03:25:35PM +0100, Martin Husemann wrote:
> >>On Wed, Dec 01, 2004 at 07:05:40PM -0600, Vincent Stemen wrote:
> >>>but the pointer I get back from realloc() does
> >>>not point to the beginning of the data allocated by the first 
> >>>malloc()
> >>>as it should.
> >>
> >>Why do you think it should?
> >
> >Because the manual on realloc() says it should and it would not be
> >very useful otherwise.  Either way, as pointed out by other replies, I
> >was misinterpreting the result.  That was not the problem.
> >
> >
> 
> From K&R 2nd ed.  "realloc returns a pointer to the new space or
> NULL if the request cannot be satisfied ..."
> 
> It's allowed to relocate the space,  ( in which case is copies from

Yes, I understand that.  Sorry if I didn't word that part clearly
enough.  When I said the return from realloc() did not point to the
beginning of the data, I didn't mean that I expected it to point to
the same address, but to the beginning of the data even if it was
relocated.  Since I thought you could depend on the amount of data
read from the popen() pipe, I was not advancing my pointer after the
first read when it was shorter than my specified block size, causing
the first block of data to be overwritten.  I misinterpreted that as
an incorrect pointer being returned from realloc().

So the actual difference between NetBSD and FreeBSD seems to be in
read() when reading from the pipe.  I don't understand why, on NetBSD,
it always limits the first read to 1024 bytes and not any of the
following reads, and why it does not do it when I step through it
slowly with the debugger.

Your other points are interesting though.  Thanks.


> the from old location to the new location ) or it wouldn't be useful,
> in that it would break trivially, if there is any memory fragmentation.
> Really, you only get the same address back in special cases.
> 1)  The memory allocated is at the edge of of the brk/sbrk line, when
>     malloc increases the break, it's still contiguous, so the regions 
> only
>     increases in size, you get the same pointer address back.
> 2)  Because of other free()'s,  malloc sees there is contiguous memory
>     available, and no brk/sbrk call is necessary.
> 3)  (special case of 1) Malloc actually brk/sbrk'ed more memory than it
>      needed, last malloc,  just so every malloc doesn't mean a syscall 
> to
>      sbrk()  (for efficiency ).
> 
> If it isn't a special case then realloc is almost the same as  
> malloc(newsize), and bcopy(old,new,oldsize) , free(old), return (new).  
> Basically, I consider realloc a convenience function that does 
> (malloc,bcopy,free,) except that it has "window optimizations" (if 
> that's the right term), where in some circumstances the bcopy can 
> avoided.
> 
> I don't know what freebsd uses for malloc, but implementations vary,
> gnu's will even try to use mmap for large chunks, as opposed to sbrk()  
> -- at any rate if freebsd's brk()'ed for more memory on the first call 
> than netbsd,
> that may be why.   I probably would go-ahead and use read() and 
> write(), like you are doing, to avoid any need to flush with stdio ( 
> although I think flush is more important on write than read, but I 
> would read in PIPEBUF chunks.
> As far as the other issues,  I wonder how cat is writing it out, and
> if anything changes if you run dd if=testfile bs=1028 -- I bet cat is 
> linebuffering with stdio.
> 
> Actually looking at the source, its memcpy not bcopy, and seems like we 
> are using the same malloc, as I see freebsd notices... and I'm 
> surprised how tiny PIPEBUF is, 512 bytes, is that configurable?  If its 
> posix I still think its low for atomic writes...
> I don't know if I actually helped, but I thought I'd weigh in w/ some 
> c/unix lore.

-- 
Vincent Stemen
Avoid the VeriSign/Network Solutions domain registration trap!
Read how Network Solutions (NSI) was involved in stealing our domain name.
http://www.InetAddresses.net