tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Reproducible builds: Recent Changes & Current State



This is very interesting, thank you. I hope this ends up as a research
paper.

jhigh

On Thu, May 2, 2024, 15:53 Jan-Benedict Glaw <jbglaw%lug-owl.de@localhost> wrote:

> Hi!
>
> Some time ago, I started to look into NetBSD's success after reaching
> reproducibility for at least two ports. Being both a VAX enthusiast
> and somebody running CI builds for NetBSD current, I used that
> infrastructure to work on reproducibility for VAX, as a cross-build
> target building on a Linux host. That was extended to build all
> port/arch combinations listed in src/build.sh .  After a pile of
> additional patches, all buildable port/arch combinations could be
> build reproducible on Linux, a few patches later also on NetBSD. These
> days, I'm chasing differences between Linux- and NetBSD-built release
> artifacts.  TL;DR: There are two remaining generic issues and probably
> a small number of remaining port-specific differences.
>
>   1. Building install ISOs with `makefs` doesn't result in identical
>      filesystem images, as the ISOs contain a large number of files
>      _not_ listed in the manifest. These files retain their host's
>      user/group identity.
>      https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58196
>
>   2. Some ports use CTF debugging data, which is (due to n unknown
>      root cause) different when created with NetBSD compared to CTF
>      data generated on a Linux host. (I suspect that data is sorted
>      somewhere, but that sort isn't stable for elements considered
>      "equal".)
>      https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58197
>
> A third problem was that my local tarballs of generated release
> artifacts wasn't stable, too, but an update from GNU tar 1.34 -> 1.35
> seems to have solved this issue.
>
>
> Here is a table of current build results (NetBSD current as of a few
> days ago, plus a small number of patches that still need to be
> reviewed, submitted & merged):
>
> mostly ok:
>         If checked [x], there are no differences within the release
>                         files, except the ISO images.
>         If checked [!], there are known differences that need to be
>                         checked. (May be CTF-related, though.)
> CTF:
>         If checked [!], this arch (aarch64, i386, amd64, earm) uses
>                         non-reproducible CTF debug infos.
>         If checked [-], this arch does not use CTF debug infos.
> ISO:
>         If checked [!], the ISO file contain differences (due to
>                         ISO9660 owner/group), but the actual content
>                         files are okay.
>         If checked [-], this port does not build ISO images.
>         If checked [x], this ISO(s) are bit-identical!
> perfect:
>         If checked [x], a tarball from result artifact created on a
>                         Linux build host is identical to a tarball
>                         created from NetBSD-built artifacts.
>                         Not tested for many ports/archs as it was
>                         pointless with GNU tar 1.34.
>
>                         mst ok  CTF     ISO     perfect
> acorn32-earmv4          [!]     [!]     [!]     [ ]
> algor-mips64el          [x]     [-]     [-]     [x]
> algor-mipsel            [!]     [-]     [ ]     [ ]             Minimal
> differences in algor/binary/kernel/netbsd-P5064.gz /
> algor/binary/sets/kern-P5064.tgz
> alpha-alpha             [x]     [-]     [!]     [ ]
> amd64-x86_64            [!]     [!]     [!]     [ ]
> amiga-m68k              [x]     [-]     [!]     [ ]
> amigappc-powerpc        [x]     [-]     [-]     [ ]
> arc-mips64el            [ ]     [ ]     [ ]     [ ]             Fails to
> build: mipsel--netbsd-gcc: not found: No such file or directory (calling
> 32bit GCC while only 64bit is configured)
> arc-mipsel              [x]     [-]     [!]     [ ]
> atari-m68k              [x]     [-]     [!]     [ ]
> bebox-powerpc           [x]     [-]     [-]     [ ]
> cats-earmv4             [!]     [!]     [!]     [ ]
> cesfic-m68k             [x]     [-]     [-]     [ ]
> cobalt-mips64el         [x]     [-]     [!]     [ ]
> cobalt-mipsel           [x]     [-]     [!]     [ ]
> dreamcast-sh3el         [!]     [-]     [!]     [ ]             Minimal
> differences in dreamcast/binary/kernel/netbsd-GENERIC_MD{.bin,}.gz (like
> algor-mipsel)
> emips-mipseb            [x]     [-]     [!]     [ ]
> epoc32-earmv4           [!]     [!]     [!]     [ ]
> evbarm-aarch64          [!]     [!]     [-]     [ ]
> evbarm-aarch64eb        [!]     [!]     [-]     [ ]
> evbarm-earmv4           [!]     [!]     [-]     [ ]
> evbarm-earmv4eb         [!]     [!]     [-]     [ ]
> evbarm-earmv5           [ ]     [ ]     [ ]     [ ]             Fails to
> build
> evbarm-earmv5eb         [ ]     [ ]     [ ]     [ ]             Fails to
> build
> evbarm-earmv5hf         [ ]     [ ]     [ ]     [ ]             Fails to
> build
> evbarm-earmv5hfeb       [ ]     [ ]     [ ]     [ ]             Fails to
> build
> evbarm-earmv6           [!]     [!]     [-]     [ ]
> evbarm-earmv6eb         [!]     [!]     [-]     [ ]
> evbarm-earmv6hf         [!]     [!]     [-]     [ ]
> evbarm-earmv6hfeb       [!]     [!]     [-]     [ ]
> evbarm-earmv7           [!]     [!]     [-]     [ ]
> evbarm-earmv7eb         [!]     [!]     [-]     [ ]
> evbarm-earmv7hf         [!]     [!]     [-]     [ ]
> evbarm-earmv7hfeb       [!]     [!]     [-]     [ ]
> evbcf-coldfire          [ ]     [ ]     [ ]     [ ]             Fails to
> build: checking target system type... Invalid configuration
> `m5407--netbsdelf': machine `m5407-' not recognized
> evbmips-mips64eb        [x]     [-]     [!]     [ ]
> evbmips-mips64el        [x]     [-]     [!]     [ ]
> evbmips-mipseb          [!]     [-]     [-]     [ ]
> evbmips-mipsel          [!]     [-]     [-]     [ ]
> evbmips-mipsn64eb       [x]     [-]     [!]     [ ]
> evbmips-mipsn64el       [x]     [-]     [!]     [ ]
> evbppc-powerpc          [!]     [-]     [!]     [ ]
> evbppc-powerpc64        [x]     [-]     [!]     [ ]
> evbsh3-sh3eb            [x]     [-]     [!]     [ ]
> evbsh3-sh3el            [x]     [-]     [!]     [ ]
> ews4800mips-mipseb      [x]     [-]     [!]     [ ]
> hp300-m68k              [x]     [-]     [!]     [ ]
> hpcarm-earmv4           [!]     [!]     [-]     [ ]
> hpcmips-mipsel          [x]     [-]     [!]     [ ]
> hpcsh-sh3el             [x]     [-]     [!]     [ ]
> hppa-hppa               [!]     [-]     [!]     [ ]
> i386-i386               [!]     [!]     [!]     [ ]
> ia64-ia64               [x]     [-]     [!]     [ ]
> ibmnws-powerpc          [x]     [-]     [!]     [ ]
> iyonix-earm             [!]     [!]     [-]     [ ]
> landisk-sh3el           [x]     [-]     [-]     [ ]
> luna68k-m68k            [x]     [-]     [!]     [ ]
> mac68k-m68k             [x]     [-]     [!]     [ ]
> macppc-powerpc          [x]     [-]     [!]     [ ]
> macppc-powerpc64        [x]     [-]     [!]     [ ]
> mipsco-mipseb           [x]     [-]     [!]     [ ]
> mmeye-sh3eb             [x]     [-]     [!]     [ ]
> mvme68k-m68k            [x]     [-]     [!]     [ ]
> mvmeppc-powerpc         [x]     [-]     [!]     [ ]
> netwinder-earmv4        [!]     [!]     [!]     [ ]
> news68k-m68k            [x]     [-]     [!]     [ ]
> newsmips-mipseb         [x]     [-]     [!]     [ ]
> next68k-m68k            [x]     [-]     [!]     [ ]
> ofppc-powerpc           [x]     [-]     [!]     [ ]
> ofppc-powerpc64         [ ]     [ ]     [ ]     [ ]             Failed to
> build (fails to link 32 vs 64 bit code)
> or1k-or1k               [ ]     [ ]     [ ]     [ ]             Failed to
> build (No rule to make target
> `/var/lib/laminar/run/netbsd-or1k-or1k/31/NetBSD-src/tools/gcc/../../external/gpl3/gcc.old/dist/gcc/config/or1k/or1k.md')
> playstation2-mipsel     [ ]     [ ]     [ ]     [ ]             Failed to
> build (/root/NetBSD-src/usr.sbin/crash/obj/mips/mutex.h:50:2: error:
> unknown type name 'ipl_cookie_t')
> pmax-mips64el           [x]     [-]     [!]     [ ]
> pmax-mipsel             [x]     [-]     [!]     [ ]
> prep-powerpc            [x]     [-]     [!]     [ ]
> riscv-riscv32           [!]     [-]     [!]     [ ]             Diff in
> [base]/lib/libc.so.12.221 (.eh_frame_hdr)
> riscv-riscv64           [!]     [-]     [!]     [ ]             Diff in
> base{,32}.tgz
> rs6000-powerpc          [x]     [-]     [-]     [ ]
> sandpoint-powerpc       [x]     [-]     [!]     [ ]
> sbmips-mips64eb         [x]     [-]     [-]     [x]
> sbmips-mips64el         [x]     [-]     [-]     [x]
> sbmips-mipseb           [x]     [-]     [-]     [x]
> sbmips-mipsel           [!]     [-]     [-]     [ ]             Diff in
> sbmips/binary/kernel/netbsd-GENERIC.gz, sbmips/binary/sets/kern-GENERIC.tgz
> sgimips-mips64eb        [ ]     [ ]     [ ]     [ ]             Failed to
> build: nbmake[4]: don't know how to make netbsd-INSTALL64_IP2x.gz. Stop
> sgimips-mipseb          [x]     [-]     [!]     [ ]
> shark-earmv4            [!]     [!]     [-]     [ ]
> sparc-sparc             [!]     [-]     [!]     [ ]             Diff in
> sparc/installation/bootfs/instfs.tgz, rootfs.tgz
> sparc64-sparc64         [x]     [-]     [!]     [ ]
> sun2-m68000             [!]     [-]     [!]     [ ]             Diff in
> sun2/binary/kernel/netbsd-RAMDISK.gz
> sun3-m68k               [!]     [-]     [!]     [ ]             Diff in
> sun3/binary/kernel/netbsd-RAMDISK3X.gz, netbsd-RAMDISK.gz
> vax-vax                 [x]     [-]     [!]     [ ]
> virt68k-m68k            [x]     [x]     [-]     [x]
> x68k-m68k               [x]     [-]     [!]     [ ]
> zaurus-earm             [!]     [!]     [!]     [ ]             Diff in
> x11 (date in docs)
>
> Ignoring needed fixes for ISO and CTF generation, NetBSD is on a good
> track reaching reproducible build results even on quite different
> build operating systems! That's a nice outcome!
>
> Thanks,
>   Jan-Benedict
> PS: Special thanks to Christos and other people who had a look at my
> occasional GNATS PRs!
>
> --
>


Home | Main Index | Thread Index | Old Index