Eucalyptus Cloud Controller VMware Image

I wanted my own ec2 cloud, to expermiment with and use for testing poolparty. I searched for a vmware instance already installed and configured that I could just download and play around with. Unfortunatley, I couldn’t find one. So, I went ahead and created my own. You can download it and take it for a spin. I tried to keep the settings as general as possible so it will require as few changes as possible

The front end is a VMware Ubuntu 9.04 server image with a single ethernet device in bridging mode. The image has a user “administrator” with password “s3cr3t” and hostname “jauntyserver”. The front end is setup to use SYSTEM networking. Basicly this means that your eucalyptus cloud will be on the same network, and use the same dhcp servers as your real network. This doesn’t give you all the features of ec2, but it seemed the most foolproof to start out with.

I set up a bridged network device on my front end with a static address:

 auto br0
iface br0 inet static
address 192.168.4.20
netmask 255.255.252.0
network 192.168.4.0
broadcast 192.168.4.255
gateway 192.168.4.1
dns-nameservers 192.168.4.1
bridge_ports eth0

Depending on your network, you may want to change this to use dhcp or different network values.

Once the image is running, you can log into the admin interface at https://192.168.4.20:8443

The eucalyptus admin username is admin and the password has been set to “s3cr3t”
The admin email is eucalyptus@mailinator.com (*Note: this means any system emails sent to the admin will be publicly viewable by anyone at http://www.mailinator.com/maildir.jsp?email=eucalyptus You can go there to check for mail, and delete it.

The Walrus url (S3) is set to
http://eucalyptus.stimble.net:8773/services/Walrus

This is walrus URL is specific to my setup. You can either reset this by changing the walrus URL at https://192.168.4.20:8443/#conf, or add eucalyptus.stimble.net to your hosts file. Whether you change the Walrus URL or use the images default, you need to be sure that the URL is resolvable by your compute nodes. It is also important to note that if you change the walrus URL you need to redownload the certificates and update your local ec2 configuration.
(On mac, I use the ruby gem ghost to manage my hosts settings)

To setup to use the new eucalyptus cloud, download the admin credentials from https://192.168.4.20:8443/#credentials. Clicking on the “Download Certificate” button downloads a zip containing your credentials and a small script, ecuarc, that will setup your shell to work with the eucalyptus server. I unziped this download to $HOME/.euca and then sourced the eucarc file:

. $HOME/.euca/eucarc

Make sure that your cloud controller is working by issuing an ec2-command. Try:

ec2-describe-images  

You should see:

IMAGE emi-39F01613    ubuntu-image-bucket/ubuntu.9-04.x86-64.img.manifest.xml admin           available   public      x86_64  machine eki-AE9D17D8    eri-17561931
IMAGE eki-AE9D17D8    ubuntu-kernel-bucket/vmlinuz-2.6.28-11-generic.manifest.xml admin       available   public      x86_64  kernel
IMAGE eri-17561931    ubuntu-ramdisk-bucket/initrd.img-2.6.28-11-generic.manifest.xml admin   available   public      x86_64  ramdisk

These are the ubuntu images I downloaded from euca-ubuntu-9.04-x86_64.tar.gz, uploaded and registered with the frontend already.

Now, you have a functioning front end cloud controller, but that’s not much use without some compute nodes. You need a computer that supports hardware virtualization. I setup a Jaunty server on my computers and installed just the node-controller as per the instructions. I also had to update the VNET_INTERFACE in /etc/eucalyptus/eucalyptus.conf. It was set to “peth0”. Probably the right option if you are using xen. I am using kvm and changed it to

VNET_INTERFACE='br0'

Change it to whatever interface your packets will be using to get to your frontend.
and then updated the configuration on the front end with the name of this node, “saint” in my case.

/usr/sbin/euca_conf -addnode compute_node_name /etc/eucalyptus/eucalyptus.conf

If your compute node is named saint, you can skip this step. Also, once again, be sure the names all resolve, or use ip addresses.

Now, you need to add a keypair

ec2-add-keypair keypair_name |tee $HOME/.euca/keypair_name
chmod 600 $HOME/.euca/keypair_name

Then edit the $HOME/.euca/keypair_name and remove first line of the file; the line with the fingerprint.

Now, in theory at least, you should be able to
ec2-run-instance emi-39F01613 -k keypair_name

After a few minutes (note: the first time it takes longer, as the instance image has to be copied over. Subsequent boots use a cached copy and boot much quicker.), we should be able to see the instance running after a few ec2-describe-instances. Once it is running:

ssh -i $HOME/euca/keypair_name -l root ip-or-name-of-instance

Now dance a little jig and enjoy your micro cloud!

Chosing a Video hosting service

This weekend was filled with another horse show. Zelly did excellent in her 4 dressage tests. Our house now has a few more blue ribbons along with a special plaque for highpoint.

Now, I have ridden horses, but I think I might understand a tenth of the equestrian details I watch at these events. So, I fallback to what I can do. Geeking out with technology. What better excuse do I need to purchase a small harddrive based HD camcorder?

But now that I have video, I am confronted with the need to actually edit it, and share it. I ended up spending far too much time comparing the various services available. There are quite a few players in the internet video space these days. Many of them are focused on promoting your video, and/or monetizing. I wasn't looking for that. I wanted a simple, clean service to store my videos and allow me to send them to family and friends. Basicly I wanted a flickr for video, without size restrictions. For the imaptient, I'll just give away my final choice, motionbox.com. Motions box feels sorta like flickr.

Motionbox provides:

• streaming in 1280x720 HD
• unlimited file size and no upload limits
• stores original video file, available for download.
• Supports the AVCHD .mts native format of my Canon HF10 camcorder
• basic video editing (ability to trim and combine videos)
• A clean, family oriented design (this is not a youtube, constantly trying to display the craziest video)
• A nice simple player
• You (or others if allowed) can order a DVD or flipbook of videos
• $30/year

Other sites I sampled:

• Blip.tv 1GB file size limit
• youtube
• vimeo
• veho
• viddyou
• reevo
• truveo
• photobucket
• smugmug

Of all the above, the only three that ended up in the final category were, vimeo, blip.tv. Vimeo actually provides my favorite interface, but the lack of avchd support and filesize limits knoked it below both blip.tv and motionbox.

Here is a raw upload from the camera.



The one thing I don't like so far, that big blue play button. I prefer the less obtrusive style, a small play button in the corner. Flickr does this, making the videos enjoyable even as stills.

I thought about rolling my own soloution, hosting the videos myself. All of the tools are available. The folks at New Bambo released a pretty complete solution, built up with ffmpeg and merb, designed to run in Amazon's cloud. The even have it all bundled up into a preconfigured Ec2 AMI. I played around with this option a little, and might come back to it.

I decided to take a look for a system that wouldn't take any work on my part, and I found motionbox. I can upload raw video directly from my camera (avoiding the hours+ of transcoding), get basic in browser editing, hosting and streaming, and still be able to download the files and take originals if I ever do want to do something different, for $30. Done.

Simulation of Evolution

One of my favorite videos. Influential.

Crysis - Mass Physics

An example computation potentially well solved as a cloud computation.

A new chart table for my boat

Every now and then I run across a new idea or technology that blows be away with how cool and useful it is. It often amazes me again when I decide to see what else this clever person has come up with and it turns out they have have done tons of awesome things. Johnny Lee just joined Jeff Hahn. This video makes me want to build a new chart table for my boat.