LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud
User Name
Password
Linux - Virtualization and Cloud This forum is for the discussion of all topics relating to Linux Virtualization and Linux Cloud platforms. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. OpenStack, CloudStack, ownCloud, Cloud Foundry, Eucalyptus, Nimbus, OpenNebula and all other Linux Cloud platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.

Notices

Reply
 
Search this Thread
Old 09-10-2012, 03:36 PM   #1
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Rep: Reputation: Disabled
Creating a custom CentOS VM image for KVM


I'm using the CentOS kickstart facility to create a base image for additional VMs I intend to create. I'm not completely satisfied with the approach I am taking and I was wondering if anyone might have some suggestions.

I create my base-vm.img using virt-install in conjunction with kickstart. I use a single 4GB partition and after the install is done, I make a copy of the partition using dd:

dd if=/dev/sdc1 of=base-vm.img bs=16k

I can then use this image to pre-install on drives (again using dd) and create VMs very quickly without having to go through an explicit OS install phase. I just tell virt-install that it can boot directly off a drive where I've pre-installed my base VM image.

The problem is that I want to tweak this base image a bit for the VMs I am creating. The base image is a small 4GB img file providing space for the entire file system. I want my VMs to in fact have /var mounted on a separate drive with a ten GB partition. Plus I want to define a 4GB swap partition.

The trick I am using is to do some prep work when a VM boots for the first time. The VM is configured to know it is booting the first time, and it goes out and configures a swap partition on /dev/vdb1 and a /var partition on /dev/vdc1. It then copies the existing /var contents to the /etc/vdc1 partition (after mounting it) and updates /etc/fstab to add entries for the new swap and var mounts.

Once this is done, the VM reboots itself and on reboot it will have a new /var partition mounted on /dev/vdc1 and swap space mounted on /dev/vdb1. This all works well enough but I can't help but think there is a better way. Can anyone suggest an alternative approach to accomplish what I need to do? Ultimately what I want is a base VM image as small as I can get, while at the same time providing a means to expand the size of the VMs that are based on this image.
 
Old 09-11-2012, 08:22 AM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,378

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
I'm not a fan of imaging if it can be avoided. I built a system somewhat similar to this a few years aog for a major client, and stuck with kickstarts throughout.

With a little love and affection you can craft a nice small kickstart to build a consistent base install which is always easy to update and modify. I then used puppet to perform personalisation of each host as part of the %post section of the kickstart.

The build phase of the system was mainly rooted in cobbler, and I only needed to use a specific name format to identify everything onward for what kvm config was required, what subnets the vm was in, and what build type it should have. Nice. Well, i thought so.

These sorts of solutions are always a little bit homebrew, as everyone wants something different but I think I had a happy middle ground.
 
Old 09-11-2012, 09:10 AM   #3
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
I considered going with kickstart for the whole thing. My main issue is that I have to automatically create a lot of identical VMs and I want to do it as quickly as possible. I can do this with img files, since the majority of the time in bringing up a VM is the OS installation phase and with img files I can better control how long the OS install takes. I can blast the identical img file out to multiple partitions very quickly, and can have a new VM up and running in under two minutes.

In the end I guess we all have our requirements.
 
Old 09-11-2012, 09:15 AM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,378

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
I was getting times of around 3.5 minutes to fully deploy an ESX based VM from a few wrapper scripts, which I'd say was pretty jolly good.

Possibly interestingly, one thing we did do with images was a windows server, but still via kickstart. By putting a wget in the %pre section and bailing out as soon as that was done, we used exactly the same deployment method for linux and windows. hacky I know, but very simple and robust.
 
Old 09-11-2012, 11:37 AM   #5
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
Where did you wget the file from? The host OS?
 
Old 09-11-2012, 12:57 PM   #6
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
Actually, this idea intrigues me. Can you elaborate any? A kickstart would normally of course be used for installing CentOS/Redhat, but I assume that instead of installing CentOS, you grabbed a pre-created Windows image file and copied that image directly onto a partition that you create in the %pre script?

The virt-install command requires that you specify the --location option and point it to your installation iso, if you want to use a kickstart file. I assume you just specified a dummy iso for this since in fact you were intending to install a Windows OS, not CentOS?
 
Old 09-11-2012, 03:37 PM   #7
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,378

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
ahh well you need an iso to boot the stage one in the first place, but you abort abort abort before you start the install real. This is the templated kickstart I wrote...

Code:
# Kickstart file for DRM server. Clearly NOT actually installing Linux here...
$SNIPPET('set_revision')

url --url=$tree
lang en_GB.UTF-8
keyboard uk
text

%pre
$SNIPPET('pre_anamon')
echo `date +%H:%M:%S` "CUSTOM  : Installing DRM image to /dev/sda" | tee -a /tmp/anaconda.log

wget http://build/infrastructure/$revision/drm/D1-disk0.gz -O - | gzip -dc > /dev/sda

echo `date +%H:%M:%S` "CUSTOM  : Imaging completed, reboot!" | tee -a /tmp/anaconda.log

# Last chance for anamon to forward log
sleep 10 
poweroff
as I said already, I had an overall wrapper script watching these builds and polling for the vm power status, doing reboots etc. (which was one of the uses of the anamon (anaconda monitor) tool that's out there with Cobbler. It actually built a 4 vm replicated environment simultaneously. if it wasn't me that wrote it, i'd have been impressed!

virt-install was abstracted using func, which has a direct module for kvm. So once the physical machine was built, we could happily call python libraries for func on the build server and know the magic was happening on one of a 100 target boxes.

Last edited by acid_kewpie; 09-11-2012 at 03:42 PM.
 
Old 09-11-2012, 04:21 PM   #8
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
Thanks for the post. This general approach might work for me. I've got some experimenting to do...

Peter
 
Old 09-18-2012, 05:09 AM   #9
Lirbank
LQ Newbie
 
Registered: Jul 2012
Posts: 4

Rep: Reputation: Disabled
Hi Peter,

We're working on a product where one of our goals is to simplify templating and VM management and CentOS is among the pre-configured machine templates. It's quite a different approach to virtualization deployment and management (goal is to reduce complexity). We're still in an early phase feature-wise but it might be worth checking it out, see http://witsbits.com

Love to hear back from you if you find it interesting.

/Mikael
 
Old 09-18-2012, 08:40 AM   #10
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
Thanks for the post. Our use of virtualization is somewhat different than the norm and from what I gather from the info on your home page this isn't something that would suit our needs. That said, I'm going to check your site out more thoroughly to see if there is something there for us.

Peter
 
Old 09-18-2012, 09:40 AM   #11
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux
Posts: 2,851
Blog Entries: 1

Rep: Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901
Quote:
Originally Posted by PeterSteele View Post
I considered going with kickstart for the whole thing. My main issue is that I have to automatically create a lot of identical VMs and I want to do it as quickly as possible. I can do this with img files, since the majority of the time in bringing up a VM is the OS installation phase and with img files I can better control how long the OS install takes. I can blast the identical img file out to multiple partitions very quickly, and can have a new VM up and running in under two minutes.

In the end I guess we all have our requirements.
In the windows world you would use Windows Deployment Services and PXE boot. In the Linux world, you would set up a PXE Boot Server, along with Kickstart for a 100% unattended deployment -- aside from making sure that PXE is the first boot choice. There are plenty of guides out there to do this -- have fun.
 
Old 09-18-2012, 09:50 AM   #12
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
A PXE boot isn't an option for us. We want to give our customers a bootable CD-ROM or USB stick that provides a fully self-contained boot environment.
 
Old 09-18-2012, 11:37 AM   #13
Lirbank
LQ Newbie
 
Registered: Jul 2012
Posts: 4

Rep: Reputation: Disabled
Quote:
Originally Posted by PeterSteele View Post
a bootable CD-ROM or USB stick that provides a fully self-contained boot environment.
Peter, this sounds awfully similar to what Witsbits does out of the box. You can download the ISO and burn a USB sticks or CD-ROM, and then send it to your customers. You can then restrict your customers to only be able to access certain servers from the management console. Let me know what your requirements are and we might be able to work out a solution for it.
 
Old 09-18-2012, 12:34 PM   #14
PeterSteele
Member
 
Registered: Jun 2012
Posts: 205

Original Poster
Rep: Reputation: Disabled
Send me a personal email and we can talk.

Peter
 
Old 09-18-2012, 01:07 PM   #15
Lirbank
LQ Newbie
 
Registered: Jul 2012
Posts: 4

Rep: Reputation: Disabled
Sure, please contact me on mikael@witsbits.com or http://witsbits.com/contact.php
/Mikael
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] BASH scripting, creating a custom service (CentOS) r3sistance Programming 9 05-08-2011 10:28 AM
Creating a continuous 2cd custom installer using Centos? infernal211283_ Linux - General 5 04-01-2011 07:18 AM
Creating a custom ISO image. xav.vijay Linux - Desktop 1 07-02-2007 11:02 AM
Creating custom initrd image in suse 9.1 Anmol Suse/Novell 1 01-08-2005 02:31 AM
need help with creating a custom boot image linuCs Linux - Distributions 0 07-23-2004 09:23 AM


All times are GMT -5. The time now is 04:52 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration