Re: Supporting compression-less release builds

[Alistair Crooks <alistaircrooks%gmail.com@localhost>]

On Sun, 12 Jan 2025 at 06:49, Julio Merino <jmmv%netbsd.org@localhost> wrote:

> Hello,
>
> I'm currently working on a NetBSD-based embedded disk image and, while
> working on this project, the compression of various artifacts during the
> build gets in the way.  Compressing the sets and kernels is very visible
> the choke point of the build even on a fast machine, and is specially so
> when xz is in use.  But the thing is: compression is completely useless
> (for me) during the development cycle.
>
> I started with a simple idea to preserve current behavior yet allow me
> to disable sets compression:
>
> 1. Add a new COMPRESS_SETS tunable that takes no/gzip/xz as values.
>    The "no" value is what allows a user to disable compression.
> 2. Deduce the default value of COMPRESS_SETS with the same logic we have
>    today based on USE_XZ_SETS/USE_PIGZGZIP.  COMPRESS_SETS=no would never
>    be set by default today.
> 3. Retire USE_XZ_SETS.
>
> This is simple enough to do and I have the change ready to go, but
> working on it opened up more questions: why focus on the release sets
> only?  There are other artifacts of the build that are compressed such
> as disk images, and those are also huge choke points of the build process.
> Plus we have inconsistencies regarding compression flags throughout the
> tree to indicate which compression levels to use.
>
> So I'm thinking to evolve this change towards something more comprehensive,
> and would like to think through it from the beginning to minimize code
> change churn.
>
> The general idea would be to replace COMPRESS_SETS (not yet even committed)
> with a more generic COMPRESS_RELEASE (or similar name) and use that value
> wherever the release build wants to produce compressed artifacts.  Then,
> unify the many -0, -9, -n... flags that appear throughout the tree for gzip
> and xz in one single place so that, when compression is indeed enabled, we
> apply it consistently.
>
> The results of this work should be mostly invisible.  Only those who
> care to set COMPRESS_RELEASE=no explicitly would see a change.  There
> might be minor size changes in the default release builds because of the
> unification of flags (e.g. applying -9 in places where it isn't applied
> today).
>
> Does anyone see a problem with this plan?
>
> I'd like to have someone help me review the intermediate changes
> because, while simple, I'm sure I may be missing something due to my
> long break from this codebase.  I expect 3-5 separate commits to get to
> the end state.  Anyone?
>
> Thanks
>
> --
> Julio Merino
>

Is there any reason that you can't just use cat(1) as a compressor?