11:04:00 <gregdek> #startmeeting Images, 6 April 2012
11:04:00 <eucabot> Meeting started Fri Apr  6 18:04:21 2012 UTC.  The chair is gregdek. Information about MeetBot at http://wiki.debian.org/MeetBot.
11:04:00 <eucabot> Useful Commands: #action #agreed #help #info #idea #link #topic.
11:04:00 <gregdek> #meetingname images
11:04:00 <eucabot> The meeting name has been set to 'images'
11:04:00 <gregdek> #topic quick roll call
11:04:00 <gregdek> dak419, mull: raise your hands pls
11:04:00 <dak419> here
11:04:00 * mull here
11:05:00 <gregdek> Moving on!
11:05:00 <gregdek> #topic General Discussion
11:06:00 <gregdek> #info mull made a 64-bit BFEBS centos 6 image
11:06:00 <mull> It's not clear to me how to publish that, though
11:06:00 <gregdek> Oh, hspencer?
11:06:00 <dak419> it would be great to have this on the new EPC, btw
11:06:00 <dak419> (give to bteachenor for EPC)
11:07:00 <dak419> for eustore, we really don't deal with bfebs, so not sure on that one
11:07:00 <mull> or if there's even a reason to, since it's only been tested on CentOS 6 kvm so far.  should be easy to make it work other places with some minor ramdisk fiddling
11:07:00 <mull> I don't think I even have an EPC account
11:07:00 <dak419> ah, not sure what EPC is even running.
11:07:00 <dak419> new EPC not open for business yet.
11:08:00 <gregdek> So.
11:08:00 <gregdek> We have this matrix.
11:09:00 <gregdek> https://projects.eucalyptus.com/redmine/projects/images/wiki/Matrix
11:09:00 <gregdek> Is it, uh, useful? :)
11:09:00 <gregdek> Can we now put an "x" into this matrix?
11:09:00 <gregdek> And if not, what steps would that entail?
11:09:00 <dak419> gotcha. this needs to be honed for 3.1 eustore
11:09:00 <obino> shoudl we put an X or a link to the image?
11:10:00 <mull> obino!
11:10:00 <obino> yes I mispelled again should lol
11:10:00 <gregdek> A link would be even better, yes/
11:10:00 <dak419> is this for testing or denoting which we will be doing?
11:11:00 <obino> we have some images in emis.eucalyptus.com
11:11:00 <mull> I'm not sure the matrix is entirely correct
11:11:00 <obino> another point is the different version of the distros
11:11:00 <dak419> obino: I think the list (of 21) on emis is too large
11:11:00 <dak419> redundancy afaik
11:11:00 <mull> each guest image should get tested on a set of distro / HV combinations ...
11:12:00 <obino> well the matrix is a 16 slots ... not too far from 21 if you will
11:12:00 <gregdek> mull: it's a first step.  Problem is there's more than 2 dimensions.
11:12:00 <mull> gregdek, not necessarily
11:12:00 <dak419> well, the matrix has 1 distro that emis doesn't even have. :-)
11:12:00 <gregdek> Which I tried (and failed prolly) to compress properly in the proffered matrix.
11:12:00 <gregdek> So I guess what I'm saying is:
11:12:00 <gregdek> Patches welcome.
11:12:00 <gregdek> :)
11:12:00 <mull> ok
11:13:00 <mull> I think that we should be able to have images (with links to them) with a row of checkboxes
11:13:00 <mull> C5/Xen, C6/KVM, Ubuntu/KVM, etc.
11:13:00 * dak419 likes that idea
11:14:00 <gregdek> wfm.
11:14:00 <mull> the columns will contain more than one piece of info, but doesn't require multiple dimensions (because that would make it quite sparse)
11:14:00 <gregdek> Good point.
11:14:00 <gregdek> DAMN YOU SPARSE MATRICES
11:14:00 <mull> :)
11:15:00 <mull> I guess we also have to make distinctions for virtio ... fun
11:15:00 <gregdek> Although I didn't realize they would be so.  I thought having the matrix filled out eventually was part of the requirement.
11:15:00 <dak419> btw, the 3.1 eustore plan is "best effort" testing on the images, so for that goal, we don't need to go nuts here
11:15:00 <dak419> not that better testing isn't a good idea, but 3.1 has deadlines
11:16:00 <mull> gregdek: let's take a concrete example:  centos 6.2 64-bit emi
11:16:00 * gregdek nods
11:16:00 <mull> it needs to run on Lucid w/ KVM, Precise w/ KVM, C6 w/ KVM, C5 w/ Xen, ...
11:17:00 <mull> but it doesn't need to run on anything 32-bit
11:17:00 * gregdek nods.
11:17:00 <mull> nor Lucid + Xen (that's invalid)
11:17:00 <mull> etc.
11:17:00 <gregdek> OK.
11:17:00 <gregdek> And since we can only produce + QA one image at a time, better to make the list additive.
11:17:00 <gregdek> Got a sense of the columns required?
11:18:00 <obino> mull: why is lucid and xen is invalid?
11:18:00 <mull> I think so.  I don't exactly know about the virtio settings or the vmware versions (are we tracking vmware here?)
11:18:00 <mull> obino, because eucalyptus doesn't support running the xen hypervisor on lucid
11:18:00 <dmitrii> all: I am totally swamped today. I am afraid :-(
11:19:00 <gregdek> dmitrii: no worries, thx for lurking.
11:19:00 <dmitrii> gregdek: thanks, I'll do my best!
11:19:00 <obino> mull: oh... I thought we were talking about images sorry
11:19:00 <mull> obino, the lucid _guest_ needs to run under xen, but that's CentOS 5
11:19:00 <mull> distro needs to be in both the rows and the column
11:19:00 <mull> one for host, one for guest
11:20:00 <obino> got it
11:20:00 <mull> right now it's only in the rows, which is confusing
11:20:00 <obino> for host is distro or hypervisor/version?
11:20:00 <gregdek> Can we go thru and bang out the proper columns quickly?
11:20:00 <obino> I mean, if I have xen on debian ssqueeze or centos 5, is that different from image perspective?
11:20:00 <gregdek> Maybe we just start with the columns we know our current version satisfies.
11:21:00 <gregdek> Or mull can just Make It So. :)
11:21:00 <obino> it's a wiki after all :D
11:23:00 <gregdek> #action mull will rework the matrix so it better suits goals
11:23:00 <gregdek> And then the question is:
11:23:00 <obino> I still think that having xen/3 or xen/4 and kvm/version may be more appropriate for hosts, just saying :)
11:23:00 <gregdek> What images are next?  Do we know?
11:23:00 <gregdek> Do we have a list?  Priorities?
11:24:00 <obino> I'd say the common one we hear are centos, ubuntu, debian
11:24:00 <obino> perhaps ubuntu, centos debian
11:24:00 <obino> lwade will add opensuse there :)
11:25:00 <dak419> can we stick with 3 for the 3.1 timeframe, knowing we can add more later?
11:25:00 <gregdek> Sure.
11:25:00 <dak419> let me add some more product release info on images.
11:25:00 <gregdek> Really I'm just looking for a list of TODOs that will keep us moving forward every week.
11:26:00 <dak419> we've talked about lifecycle on these and feel that if we have a set of images we test for 3.1, then another set for 3.2...
11:26:00 <dak419> when the 3.1 release is EOL, so are the images for that release, so we can have a plan to remove old images from emis.euca.com as well
11:26:00 <dak419> sound good to those here?
11:28:00 <dak419> oh, and I think it's a given that some images may be supported by more than one Euca release, so they'd only be removed once no more supported Euca releases "depend" on them.
11:28:00 <mull> obino, you may be right about mentioning hypervisor versions, just in case some distro doesn't understand what LTS means
11:28:00 <mull> but for now, I've done a quick update of the matrix ... see if it makes sense
11:28:00 <obino> yep saw it
11:28:00 * dak419 afk for a few minutes
11:28:00 <obino> good start
11:28:00 * gregdek looks.
11:29:00 <mull> with this version, each row should be able to have a link to the image, and checks for which boxes are tested
11:29:00 <obino> we should mention hosts vs images
11:29:00 <obino> I mean row/column
11:29:00 <gregdek> Oh, so THAT's what it's supposed to look like.
11:29:00 <mull> obino, yes, we should label it properly.  I was trying to be quick.  :)
11:29:00 <gregdek> So. When does the first link get updated? :)
11:31:00 <mull> heh, silence
11:31:00 <obino> lol
11:31:00 <mull> well, if we're setting the bar low to start with, i.e., I can publish an EMI known to work only for a subset of them, I can hammer some of these out quickly
11:32:00 <obino> is there a hoodie price for the first link? We may get lwade to do it :D
11:32:00 <mull> of course, we also have to define the level of functionality
11:32:00 <gregdek> LOL
11:32:00 <mull> does it mean cloud-init is there?  or that key injection works?  or just that the thing is bootable?  :)
11:32:00 <obino> mull: as in the package list installed?
11:33:00 <gregdek> Yeah.
11:33:00 <obino> should we just have a puppet agent and go that route?
11:33:00 <gregdek> So the bash-injection stuff must work, I'm thinking.
11:33:00 <obino> or a very simple rc.local to pull a puppet agent?
11:33:00 <mull> bash-injection?
11:33:00 <gregdek> The current metadata injection mechanism.
11:34:00 <gregdek> To which we've been referring in the recipes meetings.
11:34:00 * obino injection sounds painful
11:34:00 <obino> mull: rc.local we have vs cloud-init
11:34:00 <gregdek> The assumption is "these images plus those recipes = lots of different running configurations."
11:34:00 <gregdek> Right, rc.local-based, sorry.  Thanks obino.
11:35:00 <mull> ok.  does that mean we're specifically precluding the use of cloud-init ?
11:35:00 <mull> that would seem strange for Ubuntu & Fedora
11:35:00 <gregdek> Nope.
11:35:00 <mull> ok
11:35:00 <obino> mull: nope, we just don't have it for all distro
11:35:00 * mull needs to read recipes minutes
11:35:00 <obino> which create issues for concistency
11:35:00 <obino> however is spelled
11:35:00 <gregdek> The rc.local stuff is under the assumption that we can throw the switch at any time to change to cloud-init when the time comes.
11:36:00 <mull> obino, can you point me to the rc.local script ?
11:36:00 * obino thanks that viglesias is not here otherwise I would be crucified on a pile of dictionaries
11:36:00 * mull hopes it's better than the one he wrote
11:36:00 <obino> https://github.com/eucalyptus/Eucalyptus-Scripts/blob/master/rc.local
11:36:00 <obino> that's the one we are using now
11:37:00 <obino> there was also the one that amazon was using, but I cannot find it online
11:37:00 <obino> since they switched to cloud-init that is
11:38:00 <mull> ok, cool.  looks like if user data is not a shell script, it doesn't barf, so you could still use cloud-init on the same image with no trouble
11:38:00 <gregdek> Oh, nice.
11:39:00 <obino> again this is a fall back for when cloud-init is not there
11:39:00 <mull> right, I understand now.  I was just afraid that it was *always* expecting user data to be a shell script
11:40:00 <obino> nope
11:41:00 <gregdek> OK, sounds like we have directin.
11:41:00 <gregdek> Direction
11:41:00 <gregdek> Any actions needed, or do we just check back and see what magic mull hath wrought each week?
11:41:00 <gregdek> Oh, figuring out how to upload...
11:42:00 <mull> well, we should look at the images we already have on emis.eucalyptus.com and match them up here, right?
11:42:00 <obino> should we setup a walrus bucket for community uploads?
11:42:00 <mull> and then figure out how to improve them
11:42:00 <obino> so we can get to test those images?
11:42:00 <mull> I'm not sure that we're ready for community uploads yet.  are we?  if there's someone itching to contribute, we should certainly arrange it
11:43:00 <mull> but if not, we should stay focused on filling out the matrix ourselves
11:43:00 <gregdek> Noooo.
11:43:00 <gregdek> Well...
11:43:00 <gregdek> we shouldn't wait on the community, is my point.
11:44:00 <obino> well I consider mull community :)
11:44:00 <gregdek> And I'm not sure it's worth recruiting.  My guess is that mull can have a dozen images before most could have one.
11:44:00 <gregdek> And this is one of those things that enables adoption.
11:44:00 <mull> you assume I have free time.  :)
11:45:00 <mull> maybe I just need to have a goal of one image per week until they're done
11:46:00 <mull> of course, for the ubuntu ones, I'm not the best guy for that.  I'm guess that's hspencer or obino or... ?
11:46:00 <gregdek> maybe shaon could help with the ubuntu ones also.
11:47:00 <obino> ubuntu we could start with what they offer: it should work with eucalyptus
11:47:00 <hspencer> honestly, why don't we pull the UEC images they offer
11:47:00 <hspencer> and test those
11:47:00 * gregdek nods.
11:47:00 <obino> but it may be too big ...
11:47:00 <gregdek> Nothing wrong with that idea.
11:47:00 <hspencer> obino, ?
11:47:00 <gregdek> Do we even need to host them?
11:47:00 <hspencer> just take the current ones
11:47:00 <gregdek> Would we be altering them at all?
11:47:00 * shaon reads
11:47:00 <hspencer> gregdek, no
11:47:00 <gregdek> Just link to 'em.
11:48:00 <hspencer> gregdek, all we have to do is tell folks how to covert our bash scripts to use cloud-init
11:48:00 <hspencer> thats all
11:48:00 <hspencer> there shouldn't be a change
11:48:00 <hspencer> actually
11:48:00 <hspencer> i take that back
11:48:00 <hspencer> we will probably have to put a sleep in them
11:48:00 <hspencer> sleep in cloud-init
11:48:00 <hspencer> for like 60 seconds
11:49:00 <hspencer> becauset hats how long it takes for the metadata to become present for the instance
11:49:00 <hspencer> gregdek, but the images are available here
11:49:00 <hspencer> http://uec-images.ubuntu.com/
11:49:00 <mull> hspencer, what about vmware support ?
11:50:00 <hspencer> mull, probably have to look at what would need to change for that
11:50:00 <mull> I'm fine with the UEC ones as a first pass (better than nothing, for sure), but will they Just Work on VMware ?
11:50:00 <mull> ok
11:50:00 <hspencer> not sure
11:50:00 <hspencer> for KVM and Xen they should be money
11:50:00 <mull> probably just initrd changes would allow them to boot
11:50:00 <hspencer> we should test for VMWare
11:50:00 <hspencer> yea
11:50:00 <mull> then it's just a matter of whether we're going to try anything with vmware tools
11:51:00 <hspencer> yea
11:51:00 <hspencer> can you do unattended installs of vmware tools on LInux?
11:51:00 <mull> hspencer, you can install open-vm-tools for sure
11:52:00 <mull> not clear what the difference between VMware-tools and open-vm-tools is these days
11:52:00 <obino> do they need to be present before booting?
11:52:00 <mull> dmitrii was going to look into that side of things, I think
11:52:00 <obino> or can we instruct cloud-init to isntall them afterward?
11:53:00 <mull> obino, possibly -- may be dependent on your exact vmware config
11:53:00 <mull> there are kernel modules provided by vmware, but I don't know if there's a case where they'd be required for the disk or network to be available
11:55:00 <mull> one of the concerns is that we don't want to require that the tools be downloaded from the internet (I'm assuming most customers who care about having these packages installed have an internal mirror
11:55:00 <obino> I propose then to skip vmware for now till we understand what is needed
11:56:00 <obino> and what are the consequences of not having the tools
11:56:00 * gregdek is in favor of not blocking.
11:56:00 <gregdek> I'd rather see more images that work for fewer configs out of the bat.
11:56:00 <obino> and dmitrii is looking into it, right mull? so we can wait till he comes back to us
11:56:00 <gregdek> At least until we have a handful.
11:56:00 <dmitrii> obino: yes, I am
11:56:00 <hspencer> this is in another language, but it shows how to do a silent install of VMware tools for Linux
11:56:00 <hspencer> http://kenntwas.de/2011/linux/vmware-tools-silent-install-unter-linux/
11:57:00 <hspencer> you can translate the page
11:57:00 <dmitrii> obino: sorry, I am in a technical meeting and can't concentrate on both :-(
11:57:00 <obino> dmitrii: no problem, I was just throwing you under the bus ROFL
11:57:00 <dmitrii> obino: you are good at that :-)
11:57:00 <hspencer> heres a better one http://ict-freak.nl/2009/12/21/bash-script-auto-configure-vmware-tools-at-boot-time/
11:58:00 <hspencer> my bad, just gives information on how to change startup
11:58:00 <obino> I'm going to bail out now guys: good conversation today!
11:59:00 <mull> the repos for both deb and rpm are here:  http://packages.vmware.com/tools/esx/index.html
11:59:00 <hspencer> cool
11:59:00 <hspencer> we can probably pull notes together and test them out on automating the install
11:59:00 <mull> I'm assuming that since they have apt and yum repos, they work.  so maybe we just write up instructions for how to script the install in user data
12:00:00 <mull> but gregdek is right, this should happen after we get images verified for the easier cases
12:01:00 <gregdek> So short term:
12:01:00 <gregdek> Goal is to bang out a few images... how do we test?
12:01:00 * gregdek is interested in a basic process that says "these don't suck".
12:01:00 <gregdek> Do we have such a process, at least in mind?
12:01:00 <gregdek> win 15
12:01:00 <noirin> /
12:01:00 <gregdek> :)
12:03:00 <mull> what cloud setups do we already have internally ?  I have CentOS 6 / KVM resources for sure, so I can test there
12:03:00 <gregdek> Perhaps we'll chat about that next week.  Since we're at the hour here.
12:03:00 <mull> I'm sure we have CentOS 5 / Xen
12:03:00 <gregdek> But that's the next thing we should be thinking about, it seems.
12:04:00 <mull> ok, maybe I can coordinate w/ obino & hspencer about that during the week next week
12:04:00 <gregdek> Simple criteria:
12:04:00 <gregdek> * Does it boot?
12:04:00 <gregdek> * Can you log in / did your key get injected?
12:05:00 <gregdek> * Does a basic metadata passthru test work?
12:05:00 <gregdek> * Do we care about networking modes? (this could be tricky)
12:05:00 <gregdek> #idea QA criteria: Does it boot?
12:06:00 <gregdek> (pardon while I capture)
12:06:00 <gregdek> #idea QA criteria: Can you log in / did your key get injected?
12:06:00 <gregdek> #idea QA criteria: Does a basic metadata passthru test work?
12:06:00 <gregdek> #idea QA criteria: Do we care about networking modes? (this could be tricky)
12:06:00 <gregdek> ok.
12:06:00 <gregdek> More we can discuss later, since obino is gone.
12:07:00 <gregdek> And mull is wandering afk so:
12:07:00 <gregdek> #endmeeting