tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Supporting compression-less release builds



Hello,

I'm currently working on a NetBSD-based embedded disk image and, while
working on this project, the compression of various artifacts during the
build gets in the way.  Compressing the sets and kernels is very visible
the choke point of the build even on a fast machine, and is specially so
when xz is in use.  But the thing is: compression is completely useless
(for me) during the development cycle.

I started with a simple idea to preserve current behavior yet allow me
to disable sets compression:

1. Add a new COMPRESS_SETS tunable that takes no/gzip/xz as values.
   The "no" value is what allows a user to disable compression.
2. Deduce the default value of COMPRESS_SETS with the same logic we have
   today based on USE_XZ_SETS/USE_PIGZGZIP.  COMPRESS_SETS=no would never
   be set by default today.
3. Retire USE_XZ_SETS.

This is simple enough to do and I have the change ready to go, but
working on it opened up more questions: why focus on the release sets
only?  There are other artifacts of the build that are compressed such
as disk images, and those are also huge choke points of the build process.
Plus we have inconsistencies regarding compression flags throughout the
tree to indicate which compression levels to use.

So I'm thinking to evolve this change towards something more comprehensive,
and would like to think through it from the beginning to minimize code
change churn.

The general idea would be to replace COMPRESS_SETS (not yet even committed)
with a more generic COMPRESS_RELEASE (or similar name) and use that value
wherever the release build wants to produce compressed artifacts.  Then,
unify the many -0, -9, -n... flags that appear throughout the tree for gzip
and xz in one single place so that, when compression is indeed enabled, we
apply it consistently.

The results of this work should be mostly invisible.  Only those who
care to set COMPRESS_RELEASE=no explicitly would see a change.  There
might be minor size changes in the default release builds because of the
unification of flags (e.g. applying -9 in places where it isn't applied
today).

Does anyone see a problem with this plan?

I'd like to have someone help me review the intermediate changes
because, while simple, I'm sure I may be missing something due to my
long break from this codebase.  I expect 3-5 separate commits to get to
the end state.  Anyone?

Thanks

-- 
Julio Merino


Home | Main Index | Thread Index | Old Index