Re: Xen kernel broken for me

[Roger Pau Monné <roger.pau%entel.upc.edu@localhost>]

Hello,

I use Xen 4.1.1 right now, and I can confirm it works with XEN3_DOM0
(amd64) kernel 2011-06-09 01:30 UTC. The line I use to boot is:

menu=NetBSD/XEN41(serial):load /netbsd.xen console=com0;multiboot
/xen.gz dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin console=com1
com1=115200,8n1

I see that you use /xen instead of /xen.gz, are you sure this is
correct? If you don't see any output this means that not even your xen
kernel is booting, even if you have a bad netbsd kernel, you should at
least see the output of the xen kernel booting (xen kernel boots
before netbsd kernel).

Regards, Roger.

2011/7/15 Ben C. <armondbc%gmail.com@localhost>:
> It's quite possible.  I don't get any output, while it seems you get a
> plethora of output.
>
> Secondly, I actually tried backing it up ALL the way to 11/30/2010 and
> still didn't get a boot which I find very odd.  Although when running
> -current source over 6 months old, it's hard for me to complain about
> that aspect.
>
> Rodger, can you please procide me with the line you are using to boot
> in /boot.cfg and also if you are using the xen or xen-debug kernel and
> I'll see if I can't get any output?  Also, have you tried the xen4
> hypervisor kernel?
>
> Just for ref., here is almost line-by-line how I'm getting to the
> point of non-booting, straight from my recipe book:
>
> # Install pkgsrc ---------------------------------
> cd /usr
> ftp ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.gz
> tar xfvz pkgsrc.tar.gz
> rm pkgsrc.tar.gz
>
> # Install vim ----------------------------------
> echo ACCEPTABLE_LICENSES= vim-license >> /etc/pkg_install.conf
> cd /usr/pkgsrc/editors/vim/
> make install clean
>
> # Install src for custom kernel -------------------------------------------
> mkdir /root/.ssh
> echo 'anoncvs.us.netbsd.org,204.152.190.16 ssh-rsa
> AAAAB3NzaC1yc2EAAAABIwAAAIEA3QiBl8leG9fqIJpKeNov0PKq5YryFFiroMWOPUv4hDFn8R0jC07YVaR/OSBrr37CTmGX5AFceXPzoFnLlwCqWR7rXg4NR75FTlTp9CG9EBAEtU8mee27KDrUFBTZdfVl2+aRYoAI5fTXA+0vpIO68Cq843vRWUZCcwinS4cNLUU='
>> /root/.ssh/known_hosts
> cd /usr
> # this is how I was pulling old src's for example
> #cvs -z3 -d anoncvs%anoncvs.us.netbsd.org@localhost:/cvsroot co -P -D
> '2011-05-30 00:00:00' src
> # but this gets current..
> cvs -z3 -d anoncvs%anoncvs.us.netbsd.org@localhost:/cvsroot co -P src
>
> mkdir /usr/mybuild /usr/mybuild/obj /usr/mybuild/tools
> /usr/mybuild/dest /usr/mybuild/release /usr/mybuild/xsrc
>
> # Never got far enough to make this useful, but it was indeed present
> when building my kernels
> echo MKLVM=yes >> /etc/mk.conf
>
>
> # netbsd_t was my "Development Generic Kernel"
> # to compare against the xen kernels to see if it was an overall
> netbsd issue or specifically related to xen,
> # which from what I can tell it is indeed very much related to xen
>
> cd /usr/src
> ./build.sh -T /usr/mybuild/tools -D /usr/mybuild/dest -R
> /usr/mybuild/release -O /usr/mybuild/obj -X /usr/mybuild/xsrc -j8
> tools kernel=GENERIC
> cp /netbsd_t /netbsd_t.`date +%F.%s`
> cp /usr/mybuild/obj/sys/arch/amd64/compile/GENERIC/netbsd /netbsd_t
>
> # I also have other parts of the receipe that included a kernel called
> XEN3_DOM0-SPEED which removed debugging and some other things,
> # but this is indeed what was tested, never got as far as to trying to
> refine the kernel:
>
> cp /usr/src/sys/arch/amd64/conf/XEN3_DOM0
> /usr/src/sys/arch/amd64/conf/XEN3_DOM0-GENERIC
> cd /usr/src
> ./build.sh -T /usr/mybuild/tools -D /usr/mybuild/dest -R
> /usr/mybuild/release -O /usr/mybuild/obj -X /usr/mybuild/xsrc -j8 -r
> tools kernel=XEN3_DOM0-GENERIC
> cp /XEN3_DOM0-GENERIC /XEN3_DOM0-GENERIC.`date +%F.%s`
> cp /usr/mybuild/obj/sys/arch/amd64/compile/XEN3_DOM0-GENERIC/netbsd
> /XEN3_DOM0-GENERIC
>
> # Install xentools33 and xenkernel33 (hypervisor kernel) ------------------
> cd /usr/pkgsrc/sysutils/xentools33
> make install clean
> cd /usr/pkgsrc/sysutils/xenkernel33
> make install clean
> cp /usr/pkg/xen3-kernel/xen.gz /
> cp /usr/pkg/xen3-kernel/xen-debug.gz /
> gunzip /xen.gz
> gunzip /xen-debug.gz
>
> # Pretty much the rest except boot.cfg, is probably not relevant, but
> included here for completeness:
>
> cd /dev
> ./MAKEDEV xen
> sh MAKEDEV vnd4 vnd5 vnd6 vnd7 vnd8 vnd9 vnd10 vnd11 vnd12 vnd13 vnd14
> vnd15 vnd16 vnd17 vnd18 vnd19 vnd20 vnd21 vnd22 vnd23 vnd24 vnd25
> vnd26 vnd27 vnd28 vnd29 vnd30 vnd31 vnd32 vnd33 vnd34 vnd35 vnd36
> vnd37 vnd38 vnd39 vnd40 vnd41 vnd42 vnd43 vnd44 vnd45 vnd46 vnd47
> vnd48 vnd49 vnd50 vnd51 vnd52 vnd53 vnd54 vnd55 vnd56 vnd57 vnd58
> vnd59 vnd60 vnd61 vnd62 vnd63 vnd64 vnd65 vnd66 vnd67 vnd68 vnd69
> vnd70 vnd71 vnd72 vnd73 vnd74 vnd75 vnd76 vnd77 vnd78 vnd79 vnd80
> vnd81 vnd82 vnd83 vnd84 vnd85 vnd86 vnd87 vnd88 vnd89 vnd90 vnd91
> vnd92 vnd93 vnd94 vnd95 vnd96 vnd97 vnd98 vnd99 vnd100 vnd101 vnd102
> vnd103 vnd104 vnd105 vnd106 vnd107 vnd108 vnd109 vnd110 vnd111 vnd112
> vnd113 vnd114 vnd115 vnd116 vnd117
>
> echo 'sshd=YES
> xend=YES
> xenbackendd=YES
> xendomains=""
> ' >> /etc/rc.conf
>
> echo 'menu=Boot normally:boot netbsd
> menu=Boot single user:boot netbsd -s
> menu=Disable ACPI:boot netbsd -2
> menu=Drop to boot prompt:prompt
> menu=Development GENERIC kernel:boot netbsd_t
> menu=Boot Xen with 768MB GENERIC XEN and inc. loglvl:load
> /XEN3_DOM0-GENERIC console=com0;multiboot /xen dom0_mem=768M
> loglvl=all guest_loglvl=all com1=9600,8n1 console=com1
> menu=Boot Xen-debug with 768MB GENERIC XEN and inc. loglvl:load
> /XEN3_DOM0-GENERIC console=com0;multiboot /xen-debug dom0_mem=768M
> loglvl=all guest_loglvl=all com1=9600,8n1 console=com1
> default=6
> timeout=8
> clear=1
> ' > /boot.cfg
>
> reboot
>
>
> ----
>
> I would also note a few modifications:
> 1) I tried shortening the filename of the kernel at one point.  I
> don't think netbsd would have a problem even with extremely long
> kernel names, but just in case, it was tried on /some/ of the kernels
> I compiled throughout the last few days.
>
> 2) At several points I tried increasing the memory from 768MB of RAM
> to 2GB (dom0_mem=2048M)
>
> Thanks, Ben
>
> On Thu, Jul 14, 2011 at 3:06 PM, Roger Pau Monné
> <roger.pau%entel.upc.edu@localhost> wrote:
>> I've filled a pr the past month, some of the recently merged branches
>> broke the dom0 xen kernel, here is the bug report:
>>
>> http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=45062
>>
>> Take a look to see if it's related.
>>
>> Regards, Roger.
>>
>> 2011/7/14 Ben C. <armondbc%gmail.com@localhost>:
>>> Couldn't decide if it was more appropriate for ports-xen or
>>> current-users.  I figured current-users since it's more to do with the
>>> -current branch than xen, maybe.  I'm not sure, so apologies if anyone
>>> feels this is in the wrong place.
>>>
>>> on July 9th, I ran cvsup on my src tree, and rebuilt the XEN3-DOM0
>>> kernel.  At the time I was using a slightly customized kernel
>>> configuration based on the stock XEN3-DOM0 kernel config.  Mine just
>>> turned off some debugging and tracing.  The server never came back up
>>> after rebooting the server with the new kernel and, the majority of
>>> the events with that are irrelevant at this point.  Eventually, I got
>>> a hold of some new hardware and have been attempting to install
>>> XEN3-DOM0 on it for about 2 days.
>>>
>>> I've ran cvsup's on July 9th, 10th, & 11th and essentially the
>>>     /usr/src/sys/arch/amd64/conf/GENERIC kernel seems working on my
>>> system, just for a comparison.  It works fine.
>>>     /usr/src/sys/arch/amd64/conf/XEN3_DOM0-GENERIC kernel is broken.
>>> I can't provide much debugging information.  All I can tell you is the
>>> datacenter says "The screen shows a white underscore (_) blinking on
>>> the screen, and nothing ever happens."
>>>
>>> I backed up the NetBSD src up a bit and June 25th was also showing the
>>> same behavior.  At this point, I ditched my real hardware (I'm pretty
>>> sure at this point my datacenter was ready to throw me to the curb
>>> after 10-15 manual reboots and 3 reinstalls in a matter of days).  I
>>> setup a NetBSD domu running HVM, and then tried a variety of Xen Dom0
>>> kernels (under this HVM).  I tracked it down to something between 5/30
>>> and 6/05 that "broke" my hardware.  I could be wrong, but this seems
>>> to be the case.
>>>
>>> That is, the kernel that came with the source from 6/05, 6/10, 6/25,
>>> 6/30, 7/9, 7/10, 7/11 and HEAD as of sometimes about 9AM today, the
>>> 14th, is non-working for me.  They all have the same behavior as
>>> mentioend above: GENERIC work, the XEN3-DOM0 kernel does not.  Once
>>> again, so far all the datacenter techs can tell me is "it shows a
>>> blinking underscore (_)" and never boots. -- this was all on the i7
>>>
>>> The i5 is running right now on a kernel from 5/30 or so.  I'm waiting
>>> for the datacenter to reboot my box into the generic kernel again and
>>> I'm going to test the i7 with the source from 5/30.
>>>
>>> Is anyone else experiencing issues?
>>>
>>> My original hardware was 2.8GHz i5 760 with 8GB of RAM.
>>> My newer hardware is a 2.93GHz i7 870 with 16GB of RAM.
>>>
>>> Please excuse any ignorance encountered in the above.  Thanks,
>>>
>>> Ben
>>>
>>
>