Re: pkg/46190: pkgsrc-2011Q4 - misc/rlwrap - Segfaults on amd64 - odd gdb results

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

The following reply was made to PR pkg/46190; it has been noted by GNATS.

From: Matthew Mondor <mm_lists%pulsar-zone.net@localhost>
To: gnats-bugs%NetBSD.org@localhost
Cc: 
Subject: Re: pkg/46190: pkgsrc-2011Q4 - misc/rlwrap - Segfaults on amd64 -
 odd gdb results
Date: Tue, 1 May 2012 04:39:13 -0400

 Someone else on IRC was having issues with BitchX on NetBSD/AMD64,
 and we both agreed it was very similar to this PR's issue.  While
 trying to come up with the smallest test case that could reproduce
 the problem, the following became obvious:
 
 On the amd64 ELF ABI, the C default return value (int) is 32-bit.
 Thus, calling any function working on pointers that is lacking a
 prototype can be fatal and result in corrupted pointers with the
 high 32-bit set to 0xffffffff.  I'm not sure if this could somehow
 be mitigated, but it might require breaking compatibility.
 
 Exemple:
 
 
 mymalloc.c:
 
 #include <stdlib.h>
 #include <err.h>
 
 void *
 my_malloc(size_t size)
 {
        void *buf = malloc(size);
 
        if (buf == NULL)
                err(EXIT_FAILURE, "malloc(%ld)", size);
 
        return buf;
 }
 
 
 main.c:
 
 #include <stdio.h>
 #include <stdint.h>
 #include <stdlib.h>
 #include <err.h>
 
 int main(void);
 
 int
 main(void)
 {
        char *str = (char *)my_malloc(2048);
 
        (void)printf("%c %p\n", (str ? '1' : '0'), str);
 
        if (str != NULL)
                free(str);
 
        return EXIT_SUCCESS;
 }
 
 
 Compile:
 
 cc -Wall -c -o mymalloc.o mymalloc.c
 cc -Wall -c -o main.o main.c
 cc -Wall -o test main.o mymalloc.o
 
 Test:
 
 ninja$ ./test 
 1 0xfffffffff7701800
 [1]   Segmentation fault (core dumped) ./test
 
 
 Conclusion:
 
 rlwrap is probably missing some prototype(s)/headers which it should
 include, and I'll post a further followup once I can patch it to
 work on amd64.
 
 
 -- 
 Matt