toolchain/40401: ld --gc-sections should not discard .note.netbsd.ident

[xkernigh%netscape.net@localhost]

>Number:         40401
>Category:       toolchain
>Synopsis:       ld --gc-sections should not discard .note.netbsd.ident
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    toolchain-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jan 15 00:10:01 +0000 2009
>Originator:     Kernigh
>Release:        NetBSD 4.0.1
>Organization:
N/A
>Environment:
System: NetBSD ghostborough.local 4.0.1 NetBSD 4.0.1 (GENERIC) #0: Tue Oct 7 
23:19:46 PDT 2008 
builds@wb27:/home/builds/ab/netbsd-4-0-1-RELEASE/macppc/200810080053Z-obj/home/builds/ab/netbsd-4-0-1-RELEASE/src/sys/arch/macppc/compile/GENERIC
 macppc
Architecture: powerpc
Machine: macppc
>Description:
Programs linked with ld --gc-sections do not contain a .note.netbsd.ident
section and do not run. Instead these programs usually print an unpredictable
error message and exit. A ktrace reveals that the 4.0.1/macppc GENERIC kernel
loads these programs using Linux emulation.

The ld --gc-sections flag discards unused input sections, so that their
contents do not appear in the linked program. The most popular use of this
flag is to compile a program with cc -fdata-sections -ffunction-sections so
that each datum or function has its own ELF section, then link the program
with cc -Wl,--gc-sections to discard all unused data and functions.

The dbus package (in pkgsrc as sysutils/dbus) uses these three cc flags to
discard all unused data and functions as a security feature. The package
uses a configure test to find the flags. NetBSD has the flags, so the test
passes. The built dbus-daemon executable does not have the .note.netbsd.ident
section and does not run on macppc (Problem Report pkg/39105). To work around
this problem, someone added BUILDLINK_TRANSFORM+=rm:-Wl,--gc-sections in
pkgsrc/sysutils/dbus/Makefile to delete that compiler flag.

If the linker has a --gc-sections flag, then someone should fix the flag so
that it works correctly.
>How-To-Repeat:
A very simple C program can reproduce this problem. I never use the
-fdata-sections nor -ffunction-sections flags, in fact the program consists
only of a main function:

$ cat simple.c
#include <stdio.h>

int
main()
{
        puts("This simple program must work.");
        return 0;
}
$

If I compile the program normally on my NetBSD system, then it works:

$ cc -o simple simple.c
$ ./simple
This simple program must work.
$

If I use --gc-sections, then I reproduce this problem:

$ cc -Wl,--gc-sections -o simple-gc simple.c
$ ./simple-gc
Cannot map anonymous memorySocket operation on non-socket: Socket operation on 
non-socket
$

The ld --gc-sections flag works by discarding input sections. Thus the output
sections would either have a smaller size or become missing. I carefully
compare 'readelf -S simple' versus 'readelf -S simple-gc' to find such output
sections. In simple-gc, four sections have a smaller size (.data, .shrstab,
.symtab, .strtab) and three sections become missing (.note.netbsd.ident,
.note.netbsd.pax, .fixup).

To verify that .note.netbsd.ident is the problem, I use objcopy -R to discard
the .note.netbsd.ident section from simple.

$ objcopy -R .note.netbsd.ident simple test3
$ ./test3
Cannot map anonymous memorySocket operation on non-socket: Socket operation on 
non-socket
$

Examination with ktrace shows that the kernel is trying to run simple-gc under
Linux emulation. This is despite the fact 

ktrace shows the kernel tries to run simple-gc under Linux emulation. Thus
every system call is wrong. (I also have not created /emul/linux nor installed
any Linux libraries on my system.)

$ ktrace ./simple-gc
Cannot map anonymous memorySocket operation on non-socket: Socket operation on 
non-socket
$ kdump
  2164      1 ktrace   EMUL  "netbsd"
  2164      1 ktrace   CALL  execve(0xffffdd2b,0xffffdc68,0xffffdc70)
  2164      1 ktrace   NAMI  "./simple-gc"
  2164      1 ktrace   NAMI  "/emul/linux/usr/libexec/ld.elf_so"
  2164      1 ktrace   NAMI  "/usr/libexec/ld.elf_so"
  2164      1 ktrace   NAMI  "/usr/libexec/ld.elf_so"
  2164      1 simple-gc EMUL  "linux"
  2164      1 simple-gc RET   execve JUSTRETURN
  2164      1 simple-gc CALL  #198 (unimplemented sys_pciconfig_read)
  2164      1 simple-gc RET   #198 (unimplemented sys_pciconfig_read) -1 errno 
38 Too many processes
  2164      1 simple-gc CALL  write(2,0xffffd968,0x1b)
  2164      1 simple-gc GIO   fd 2 wrote 27 bytes
       "Cannot map anonymous memory"
  2164      1 simple-gc RET   write 27/0x1b
  2164      1 simple-gc CALL  write(2,0xffffd9c8,0x1e)
  2164      1 simple-gc GIO   fd 2 wrote 30 bytes
       "Socket operation on non-socket"
  2164      1 simple-gc RET   write 30/0x1e
  2164      1 simple-gc CALL  write(2,0xffffd978,0x21)
  2164      1 simple-gc GIO   fd 2 wrote 33 bytes
       ": Socket operation on non-socket
       "
  2164      1 simple-gc RET   write 33/0x21
  2164      1 simple-gc CALL  exit(1)

I can conjecture what is wrong. The missing .note.netbsd.ident section contains
an ELF note that would have identified the program as a native NetBSD program
(http://www.netbsd.org/docs/kernel/elf-notes.html). Because simple-gc does
not have this ELF note, so the kernel must guess which emulation to use. The
kernel guesses wrongly. All native NetBSD executables must have the ELF note.

NetBSD uses GNU ld, so I checked two other operating systems that use GNU ld
with ELF, to check if ld --gc-sections has the same problem. I checked
OpenBSD/macppc, and I also borrowed access to a Linux/powerpc system. With
both systems, I verified that ld --gc-sections does discard the
.note.otheros.ident section, however both kernels can still run a simple
program without this section.
>Fix:
I do not have a fix for this problem, but I can guess what the fix might be.

The following workaround does NOT work:

$ objcopy -j .note.netbsd.ident -O binary simple ident
$ objcopy --add-section .note.netbsd.ident=ident simple-gc test4
$ ./test4
Cannot map anonymous memorySocket operation on non-socket: Socket operation on 
non-socket
$

I tried to dump the .note.netbsd.ident section from simple and add it to
simple-gc to create test4. Now 'readelf -S test4' shows the section, but
'readelf -l test4' verifies that the section is not in a PT_NOTE segment.
This is why test4 does not run.

I tried to write a linker script that uses KEEP() to prevent that --gc-sections
discards any .note.netbsd.ident section. I failed:

$ cat workaround.sc
SECTIONS
{
        .note.netbsd.ident : { KEEP(*(.note.netbsd.ident)) } 
}
$ cc -Wl,--gc-sections -o test5 simple.c workaround.sc
ld: test5: Not enough room for program headers (allocated 6, need 7)
ld: final link failed: Bad value
$

If workaround.sc would work, then it would still not keep any note sections
other than .note.netbsd.ident sections.

I guess that a fix would be to hack the 'ld' sources so that --gc-sections
would not discard any input section destined for a PT_NOTE segment. This hack
would fix the linker because the linker cannot know whether a PT_NOTE segment
would be unused, so it is wrong to discard any such input section.

The documentation (http://www.netbsd.org/docs/kernel/elf-notes.html) claims
that, 'When final linking is done, all sections which have names starting with
".note" and which are marked "allocate" will be put into PT_NOTE sections in
the final object file.' Therefore, the hack would need to tell --gc-sections
to discard not any section which has a name starting with ".note" and which
is marked "allocate".

A correct fix would allow the user to discard unused data and functions with
cc -Wl,--gc-sections -ffunction-sections -fdata-sections. One might test this
by adding to simple.c some unused data or functions.

Unless I am wrong.