Re: Amazon EC2 build script

[Jeff Rizzo <riz%NetBSD.org@localhost>]

Just an FYI:

I'm actively working on cleaning these scripts up and modifying them to
work with release builds (as opposed to building from scratch), and will
be committing a modified version of them to the tree at some point in
the relatively near future.

+j

On 2/5/12 10:05 AM, Jean-Yves Migeon wrote:
> Dear all,
>
> I wrote a set of shell scripts to help build and upload NetBSD images
> (see attached). I am presently using them to update the 5.1 images.
>
> I am well aware that they are ugly :) Feel free to send me
> patches/ideas to improve them, and/or commit them to localsrc.
>
> The set_ec2_env.sh.example script is a skeleton to set up environment
> for EC2 use. Edit it to fit your EC2 account.
>
> What you can do:
> - build_ec2_img.sh: compile and generate a NetBSD image, suitable for
> upload to EC2. Feel free to adapt it to your needs, I wrote a function
> for each step (building src, patching installed files, makefs, etc).
>
> Please keep the various ec2 scripts, and only modify them when you
> know what you are doing. EC2 AMIs require some automation to fetch ssh
> keys, for example.
>
> Usage:
> # ./build_ec2_img.sh <path-to-NetBSD-src> <tmp>
>
> - create_ec2_ami.sh:
>   - list NetBSD AMIs by region (when called without arguments).
>   - Upload the image and kernel built previously, and create the new AMI.
>
> Usage:
> $ ./create_ec2_ami.sh # Outputs NetBSD's AMI information
> $ ./create_ec2_ami.sh <tmp> # Upload and create a NetBSD AMI
>
> === Shortcomings and possible improvements ===
>
> - build_ec2_img.sh must be executed as root. Using mtree specs with
> makefs can avoid that, but as I am using ./build.sh install=$EC2DIR
> with specific sets, I did not find a way around this.
>
> - scripts are ugly, and make extensive use of awk.
>
> - i386 is not supported, but should be trivial to implement.
>
> - there may be some unhandled corner cases with AMI creation,
> especially when communicating via EC2 API. Amazon did not design their
> tools in a shell friendly way. Errors can be reported either by
> command's return value, or out-of-band (in the output).
>
> - the upload/creation of the AMI can take some time. It checks the SSH
> host key from the console output, and since it's buffered on Amazon's
> side upload will wait for them before starting AMI creation.
>