Cloud Computing

Just discussing some abstraction layers on top of MapReduce - Brandon Werner

Looks a bit like Dryad - Nathan Howell

Repetition of their search efforts, huh!! - Krishnan Hussein Subraman via Bookmarklet

When I advocate cloud computing to people, many ask the same question too. There is quite a bit of confusion going on in this area with different terms thrown all over the place. - Krishnan Hussein Subraman via Bookmarklet

Using Xen to develop a wide area distributed computing - Krishnan Hussein Subraman via Bookmarklet

If you are trying to push cloud computing into your organization, this might help. - Krishnan Hussein Subraman via Bookmarklet

How Smugmug uses EC2 and S3 in their business - Krishnan Hussein Subraman via Bookmarklet

Hadoop On Demand (HOD) is a system for provisioning virtual Hadoop clusters over a large physical cluster. It uses the Torque resource manager to do node allocation. On the allocated nodes, it can start Hadoop Map/Reduce and HDFS daemons. It automatically generates the appropriate configuration files (hadoop-site.xml) for the Hadoop daemons and client. HOD also has the capability to distribute Hadoop to the nodes in the virtual cluster that it allocates. In short, HOD makes it easy for administrators and users to quickly setup and use Hadoop. It is also a very useful tool for Hadoop developers and testers who need to share a physical cluster for testing their own Hadoop versions. - Brandon Werner via Bookmarklet

Dryad is an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs. A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming. - Brandon Werner via Bookmarklet

"After the previous set of tests wtih parallel Kmeans clusting using CGL-MapReduce, Hadoop and MPI I shift the direction of testing to another set of tests. This time, the test is to process large number (and volume of) High Energy Physics data files and produce a histogram of interesting events. The amount of data that needs to be processed is 1 terabyte." - Brandon Werner via Bookmarklet

3 Tera CEO compares the migration of enterprises to cloud computing with evolution and offers an interesting insight into how this migration might happen eventually. - Krishnan Hussein Subraman via Bookmarklet

Anyone else checking this out? I've been playing with it for a week. Can't get a Rocks Frontend to install anywhere, so I can't get Eucalyptus running yet. I tried Rocks on a P4 with 1.2GB RAM, and 360GB disk, 160 on IDE, 200 on SATA. I've also tried to install under VMware Server with an 18GB IDE virtual disk and 1.2GB RAM. Dies every single time at the same place. Right after downloading the base roll. - Steve J

I'm good now. I wasn't putting it together that the first two OS rolls are required yet are not part of the "base". I thought they were used for deploying the OSes in child or compute nodes. When I really dug into the exception I was getting, I found hundreds or thousands of packages being unavailable, and that was the tip I needed to add the two OS rolls not labeled as "not required". - Steve J

It was a pretty lame attempt :-) - Krishnan Hussein Subraman

Appears a bit narrow to me - Krishnan Hussein Subraman

You're on a quite a tear today. Flagging all the cloud goodness for later consumption. Need to drill my speed-reading techniques. - Benson Miller