Archive for category Xen

Specifying dom0 RAM in Xen – update

Another thing from the xen-users mailing list:

Apparently the way to specify the dom0 RAM at boot in Xen has changed. You should now specify the maximum memory – dom0_mem=2G,max:2G instead of dom0_mem=2G.

Also, a new addition to the Xen Best Practices suggests preventing Xen from auto-ballooning the dom0 memory down by changing autoballoon in /etc/xen/xl.conf to 0.


No Comments

Defining the dom0 memory limit in GRUB2

Before I upgraded my dom0 to Fedora 16 (and by association Grub2), I had the dom0 memory limit fixed as an argument in grub.conf. When I upgraded to Grub2, I initially added it manually to the Xen boot command line, but eventually stopped doing that.

Today while looking through the xen-users list, I found an email that mentioned how to get grub2-mkconfig to automatically add it for you – add GRUB_CMDLINE_XEN=”dom0_mem=1024M” to /etc/default/grub. It seems to be the same on both Debian and Fedora, so I don’t see why it wouldn’t work on dom0s built with other distros.


No Comments

Fedora 16 domUs – Update

So, I’ve wiped my test system with the intention of doing a tutorial on installing F16 + Xen. As part of this, I looked into installing a F16 domU with virt-install. When I tried it previously, I ran into this bug, and I ended up having to vnc into the dom0 and use virt-viewer to do the install.

Recently, there was some activity on the bug, centering around using “rd_NO_PLYMOUTH” to bypass the plymouth boot screen (the progress bar that scrolls across the screen, apparently – though why it’s in the installer I have no clue).

In any case, it seems to work – my revised install command of virt-install -p -r 1024 -n rawhide -f /dev/domU/rawhide -l -x "ks=\"http://hydrogen/~kyl191/ks/ks.php?hostname=rawhide\" rd_NO_PLYMOUTH serial" seems to do the trick – it gets into the console, at the very least.

That said, I get a nasty error using the partitioning scheme in my Kickstart file – grub2 wants a BIOS boot partition, a 1MB partition for storing files, and installation fails.

So, either I add the extra partition to the kickstart partitioning stanza, or I pass the installer an extra nogpt option when installing.

Guess which I chose. The BIOS boot partition makes no sense for Xen, because we’re booting with pygrub – we only care about grub.cfg, not what GRUB files are present or even if the bootloader is installed (which leads me to a question – can we get away with not specifying a bootloader to be installed in anaconda? I imagine not, since that probably means grub2 won’t be installed, but it might be something to look into.)

Though, it does seem to have completely fallen over – pygrub’s not returning any data on a (F14) Xen 4.0.2 system. xm create rawhide.cfg just returns an unhelpful “Error: Boot loader didn’t return any data!”, but trying pygrub directly gives a bit more detail – “RuntimeError: couldn’t find bootloader config file in the image provided.”

Trying with my F16 testbed now…

Also, apparently, radeontool, yum-pluginfastestmirror & ntpdate aren’t present in the F16 repos, so I have to remove them from my Kickstart file.

, ,

No Comments

Fedora 16 dom0 – update

I’m trying out F16 as a dom0 from the beginning. No upgrades from F15 this time. So far, it’s generally ok.

The install (from the netinstall ISO) was straightforward. Only issue was that F16 strongly recommends that a BIOS boot partition be present. (Read: Requires it unless you pass a nogpt option to the kernel boot line when installing.) Unlike in the F16 domU, I need grub2 to be working so I can boot, so I added the partition.

However, it’s subject to some strict size limits – between 1-2MB. 1MB is recommended, though I put 2MB and it seems to boot fine. If you try anything larger though, you get this error: (Why it says allowable size between 0MB-2MB is a mystery though. 1MB is the minimum size for all partitions.)

So, other than that install issue, there was nothing of note. Now, in the working system, I did a yum install xen, and yum pulled down xen & the dependencies and installed them. As part of the install, it also regenerated grub.cfg. So don’t press Ctrl+C even though it looks like yum’s frozen. It isn’t.

One annoyance: The generated file is filled with Xen entries. You can either ignore them, and select Xen 4.1.2, or patch GRUB2 as specified here to get rid of the spurious entries. (No idea when the patched version will go into the repos.)

Next is networking. On my desktop, I have a strange Ethernet driver – it shows up as p34p1. No clue why, and it’s given me no end of trouble, particularly where NetworkManager attempts to seize control despite being specified as “NM_CONTROLLED=NO”.

But that’s neither here nor there. On the testbed, getting networking is trivial, just a matter of creating the bridge definition, and modifying the existing network adapter definition.

In /etc/sysconfig/network-scripts/, the existing file is named ifcfg-<network_adaptor>. (The one other than ifcfg-lo.) In my case, it was ifcfg-em1. If you have special requirements, I’d recommend Google (which threw up this as the first result, and it seems pretty good), but if you’re using DHCP (if you don’t know what you’re using, chances are you are using DHCP), you just need to add the following to the new file ifcfg-br0 with your favourite text editor:


And append BRIDGE=br0 to your respective network adaptor definition file (i.e. ifcfg-em1). (With Vim or echo >> file, it doesn’t matter.)

Reboot, and the system seems to be up and running Xen. (Verify with a xl info)



Because this is worth repeating

I could have sworn I did this a while ago. But I got the error “‘Out of memory’, “xc_dom_boot_mem_init: can’t allocate low memory for domain\n “” when I tried to do a virt-install today.

The problem? I was installing a 64bit domU on a 32bit dom0. Which is a no-no.

Didn’t realise until I found this post on someone else’s blog detailing the error.

Now hopefully I don’t repeat this again…

, ,

No Comments

Dangers of not thinking about what you’re typing

Just upgraded a domU to Fedora 15 from F14. As part of the upgrade, init essentially dies, and you have to hard reboot the VM. So I thought “Oh hey, why not use the Magic Sysrq keys to trigger a reboot. After all, I just learnt about xm sysrq.”

The first command, xm sysrq 0 r went ok. People who know the syntax for xm/xl commands will know what I did wrong there. 0. I sent the sysrq command to dom0, the VM host!

I didn’t think about this. I got used to just typing in 0 after the command because I was fooling around with block-attack/detach/list. So I didn’t notice what I’d done. Until I typed in xm sysrq 0 e and took down my SSH session, along with virtually all the daemons on the dom0.

Surprisingly, my domUs were still up. Mostly. Jumping over to the dom0 console, I brought sshd, xend, xenstored and xenconsoled back up. None of the xm commands were giving anything, so I restarted xend. I suppose that brought the domUs down, because none of my domUs were accessible by SSH.

Tried init 3 to see if that’d bring the daemons back up. No dice. Ok… init 5, and back to init 3. “Oh wait, it’s frozen now.”

So I had to reset the dom0 to get stuff back up. At least this is giving my startup scripts a good workout!


No Comments

How I learnt to love xl block-attach

Originally, I exported one block device to each VM, then partitioned it. No fuss, everything self-contained in the one file/logical volume.

Then I had a few incidences where the VM crashed, and I needed to get a file or two off the virtual drive. It was a complicated process, as detailed in one of my earlier posts. So I get fed up and gave each VM 4 virtual disks – boot, root, home & swap, so I could shuffle the respective disks around when necessary.

Now I’ve discovered an easier way to use a single logical volume with each VM:

xl block-attach 0 phy:/dev/domU/fedora13pv.1 xvda w

Simply put, xl block-attach (and xm block-attach) creates a new virtual block device in the specified domain. There’s no man page for xl (yet), but it’s supposed to be a drop-in replacement for xm block-attach, which does have a manual entry. That command up there attached a domU’s logical volume to my dom0, where pvs shows the new physical volume appearing as /dev/xvda2, and the domU’s volume group appearing, ready to be mounted.

Using xl block-attach

Usage: xl [-v] block-attach <Domain> <BackDev> <FrontDev> [<Mode>] [BackDomain]

Points to note about using block-attach:

  • domain should be the domain ID, get it from xl list
  • backdev needs to include ‘phy:’ or ‘file:’ if you use LVM/raw partitions or disk images respectively
  • frontdev doesn’t appear to need /dev, like the config file
  • mode should be ‘w’ or ‘r’

Essentially, block-attach uses the VM config file syntax – does disk = [ "phy:/dev/domU/fedora13pv.1,xvda,w" ] look familiar?

Using xl block-detach

Now, using block-attach to recover data (or running fsck on a disk) is simple. Detaching the drive from your VM? Yeah… easier said than done, unfortunately. Trying xl block-detach to get the syntax gives you this:

Usage: xl [-v] block-detach <Domain> <DevId>

So… try xl block-detach 0 xvda, since we attached the block device to the domain’s xvda.

Error: Device xvda not connected. Oops.

Turns out, DevId doesn’t mean the front end or back end device names. What it needs is something from xl block-list. Running xl block-list 0 (since I want the block devices attached to dom0) gave me this output:

Vdev  BE  handle state evt-ch ring-ref BE-path
51712 0   0      1     -1     -1       /local/domain/0/backend/vbd/0/51712

Look at the Vdev number. That’s what you want.

Substitute it into the command to get xl block-detach 0 51712, run it, and your block device is detached.

As a point of note, it doesn’t print any message on success though.

And I’m prefix this last bit with saying that this is unconfirmed in that I haven’t investigated why, but xm block-detach doesn’t like the Vdev number when you try to use it. I don’t know if it’s just something up with xm, or my install.

I’d bet my install.


Xen 4.1.2 on Fedora 16 Beta – My experience

Duplicate of my mail to the Xen-users list: Read the rest of this entry »


No Comments

Revised Fedora virt-install command

I got tired of having to muck around with kpartx and such whenever a domU went down.

So I decided to just go with 4 LVs per domU – root, home, swap and boot.

The command to create the LVs is just one really long one, the original command replicated 4 times:

lvcreate -L10G -n rawhide-root vg_caesium_domUs && lvcreate -L1G -n rawhide-swap vg_caesium_domUs && lvcreate -L10G -n rawhide-home vg_caesium_domUs && lvcreate -L512M -n rawhide-boot vg_caesium_domUs

The virt-install command is somewhat different too – It now specifies the 4 block drives:

virt-install -p -r 512 -n rawhide -f /dev/vg_caesium_domUs/rawhide-boot -f /dev/vg_caesium_domUs/rawhide-root -f /dev/vg_caesium_domUs/rawhide-home -f /dev/vg_caesium_domUs/rawhide-swap -l -x ks="http://hydrogen/~kyl191/ks/ks.php?hostname=rawhide"



No Comments

Win2K8R2 as HVM VM on Xen 4.1

Config file:

Also, it’s working fine without the Xen extensions, though there’s (at least) one unknown device in the device manager)

No Comments