kern/59359: static pies are broken

[campbell+netbsd%mumble.net@localhost]

>Number:         59359
>Category:       kern
>Synopsis:       static pies are broken
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Apr 26 15:25:00 +0000 2025
>Originator:     Taylor R Campbell
>Release:        current, 10, 9, ...
>Organization:
The StaticBSD Pie-in-the-sky
>Environment:
>Description:
$ cat hello.c
#include <stdio.h>

int
main(void)
{
	printf("Hello, world!\n");
	fflush(stdout);
	return ferror(stdout);
}
$ make hello.o hello DBG=-g\ -O2\ -Wall\ -Werror\ -fpie LDFLAGS=-pie\ -static
cc -g -O2 -Wall -Werror -fpie   -c hello.c -o hello.o
cc -g -O2 -Wall -Werror -fpie  -pie -static  -o hello hello.c 
$ ./hello
[1]   Segmentation fault (core dumped) ./hello

Same deal on mips and riscv.  Need to test and review on other platforms.  Presumably this happens because nothing relocates the GOT in static executables.

On mips, it crashes inside ___start (crt0-common.c) as soon as it tries to read something out of the GOT:

(gdb) x/i $pc
=> 0x2345c <___start+52>:       ld      s2,-32720(gp)
(gdb) print &_GLOBAL_OFFSET_TABLE_
$6 = (<variable (not text or data), no debug info> *) 0xf53f0
(gdb) print $gp - 0x7ff0
$8 = 0xe53f0

Note that these are off by 0x10000, the page size.  And so is $t9 (called procedure) versus the ___start symbol itself:

(gdb) print ___start
$9 = {void (void (*)(void), struct ps_strings *)} 0x23428 <___start>
(gdb) print $t9
$10 = 0x13428

But somehow control has reached the ___start symbol!

The executable itself has a LOAD command at 0, as shown by `readelf -l', which is rounded up by the kernel to the requested segment alignment, 0x10000:

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  PHDR           0x0000000000000040 0x0000000000000040 0x0000000000000040
                 0x00000000000001c0 0x00000000000001c0  R      0x8
  ABIFLAGS       0x0000000000000230 0x0000000000000230 0x0000000000000230
                 0x0000000000000018 0x0000000000000018  R      0x8
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000000cd45c 0x00000000000cd45c  R E    0x10000
  LOAD           0x00000000000cd460 0x00000000000dd460 0x00000000000dd460
                 0x0000000000009bc8 0x0000000000118700  RW     0x10000
  DYNAMIC        0x00000000000024a8 0x00000000000024a8 0x00000000000024a8
                 0x00000000000001e0 0x00000000000001e0  R      0x8
  NOTE           0x0000000000000200 0x0000000000000200 0x0000000000000200
                 0x000000000000002c 0x000000000000002c  R      0x4
  TLS            0x00000000000cd460 0x00000000000dd460 0x00000000000dd460
                 0x0000000000000a68 0x0000000000000a71  R      0x8
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x8

    128 static int
    129 elf_placedynexec(struct exec_package *epp, Elf_Ehdr *eh, Elf_Phdr *ph)
    130 {
    131 	Elf_Addr align, offset;
    132 	int i;
    133 
    134 	for (align = 1, i = 0; i < eh->e_phnum; i++)
    135 		if (ph[i].p_type == PT_LOAD && ph[i].p_align > align)
    136 			align = ph[i].p_align;
    137 
    138 	offset = (Elf_Addr)pax_aslr_exec_offset(epp, align);
    139 	if (offset < epp->ep_vm_minaddr)
    140 		offset = roundup(epp->ep_vm_minaddr, align);
    141 	if ((offset & (align - 1)) != 0) {
    142 		DPRINTF("bad offset=%#jx align=%#jx",
    143 		    (uintmax_t)offset, (uintmax_t)align);
    144 		return SET_ERROR(EINVAL);
    145 	}
    146 
    147 	for (i = 0; i < eh->e_phnum; i++)
    148 		ph[i].p_vaddr += offset;
    149 	epp->ep_entryoffset = offset;
    150 	eh->e_entry += offset;
    151 	return 0;
    152 }

https://nxr.netbsd.org/xref/src/sys/kern/exec_elf.c?r=1.107#128

How did control successfully transfer from __start (crt0.S) to ___start (crt0-common.c) with the wrong address in t9, when it works via jump to register t9?

     55 	PTR_L	t9,%call16(_C_LABEL(___start))(gp)
     56 	 move	a1, a3		/* ps_strings */
     57 	.reloc  1f,R_MIPS_JALR,___start
     58 1:	jr	t9

https://nxr.netbsd.org/xref/src/lib/csu/arch/mips/crt0.S?r=1.4#55

Disassembling crt0.o reveals why -- the assembler/linker has cleverly replaced  jr t9  by  b ___start:

0000000000013390 <__start>:
__start():
   13390:       03807825        move    t3,gp
   13394:       3c1c000e        lui     gp,0xe
   13398:       279ca050        addiu   gp,gp,-24496
   1339c:       0399e02d        daddu   gp,gp,t9
   133a0:       00a02025        move    a0,a1
   133a4:       df998020        ld      t9,-32736(gp)
   133a8:       00e02825        move    a1,a3
   133ac:       1000001e        b       13428 <___start>
   133b0:       00000000        nop

So the global offset table content is off by 0x10000, the amount by which the kernel adjusted the LOAD commands to avoid the zero page.  If I set a breakpoint on __start, gdb confirms this:

(gdb) disas
Dump of assembler code for function _start:
   0x0000000000023390 <+0>:     move    t3,gp
   0x0000000000023394 <+4>:     lui     gp,0xe
   0x0000000000023398 <+8>:     addiu   gp,gp,-24496
   0x000000000002339c <+12>:    daddu   gp,gp,t9
=> 0x00000000000233a0 <+16>:    move    a0,a1
   0x00000000000233a4 <+20>:    ld      t9,-32736(gp)
   0x00000000000233a8 <+24>:    move    a1,a3
   0x00000000000233ac <+28>:    b       0x23428 <___start>
   0x00000000000233b0 <+32>:    nop
End of assembler dump.
(gdb) print $gp
$5 = 0xfd3e0
(gdb) print &_GLOBAL_OFFSET_TABLE_
$6 = (<variable (not text or data), no debug info> *) 0xf53f0
(gdb) print $gp - 0x7ff0
$7 = 0xf53f0
(gdb) x/xg $gp - 32736
0xf5400:        0x0000000000013428
(gdb) print ___start
$8 = {void (void (*)(void), struct ps_strings *)} 0x23428 <___start>

Presumably there is nothing relocating the GOT like _rtld_start/_rtld_relocate_nonplt_self does on mips.  Likely the same issue on Alpha, but with explicit relocations instead of hard-coded logic to relocate each GOT entry; however, gdb is having more trouble on alpha, so I haven't confirmed this.

Guessing that in order to make this work, we need some CSU routine to do essentially the logic of _rtld_relocate_nonplt_self and relocate the GOT/.rel.dyn/.rela.dyn.
>How-To-Repeat:
as above
>Fix:
Yes, please!