kern/54560: PXE netboot regression

[gson%gson.org@localhost (Andreas Gustafsson)]

>Number:         54560
>Category:       kern
>Synopsis:       PXE netboot regression
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Sep 22 14:40:00 +0000 2019
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current, source date >= 2019.09.13.05.13.54
>Organization:

>Environment:
System: NetBSD
Architecture: x86_64
Machine: amd64
>Description:

I have an automated testing setup that netboots an INSTALL kernel and
performs a scripted installation of NetBSD/amd64 on physical hardware
using sysinst.

This recently stopped working.  As pxeboot starts to load the
kernel over TFTP, it restarts, leading to repeating output on the
console like this:

  >> NetBSD/x86 PXE boot, Revision 5.1 (Fri Sep 13 05:13:54 UTC 2019) (from NetBSD 9.99.11)
  >> Memory: 552/3668992 k
  Press return to boot now, any other key for boot menu
  booting netbsd - starting in 0 seconds.
  PXE BIOS Version 2.1
  Using PCI device at bus 3 device 0 function 0
  Ethernet address 98:4b:e1:67:68:98
  |
  >> NetBSD/x86 PXE boot, Revision 5.1 (Fri Sep 13 05:13:54 UTC 2019) (from NetBSD 9.99.11)
  >> Memory: 552/3668992 k
  Press return to boot now, any other key for boot menu
  booting netbsd - starting in 0 seconds.
  PXE BIOS Version 2.1
  Using PCI device at bus 3 device 0 function 0
  Ethernet address 98:4b:e1:67:68:98
  |
  >> NetBSD/x86 PXE boot, Revision 5.1 (Fri Sep 13 05:13:54 UTC 2019) (from NetBSD 9.99.11)
  >> Memory: 552/3668992 k
  Press return to boot now, any other key for boot menu
  booting netbsd - starting in 0 seconds.
  PXE BIOS Version 2.1
  Using PCI device at bus 3 device 0 function 0
  Ethernet address 98:4b:e1:67:68:98
  |

After this loop has repeated several more times, the system gives up
on netbooting and boots from its second priority boot device (in this
case, the hard disk).

A full console log is at:

  http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2019/2019.09.13.05.13.54/install.log

A packet capture of the network traffic is at:

  http://www.gson.org/netbsd/bugs/pxeboot/2019.09.13.05.13.54.pcap

The problem started around the time of manu's changes to add multiboot 2
support on September 13.

>How-To-Repeat:

Attempt to netboot an amd64 system using pxeboot.

>Fix: