Results 1 to 10 of 489

Thread: Easy Ubuntu Clustering

Hybrid View

  1. #1
    Join Date
    Jan 2007
    Beans
    99

    Easy Ubuntu Clustering

    Discussions about Ubuntu Clustering seem to be scattered over several forums, and discussion about the Easy Ubuntu Clustering blueprint have been dormant so long that the threads are archived and marked read-only!

    I want to re-start discussion about Ubuntu clustering here, where Ubuntu users in Education and Science can contribute: This is related to my proposed blueprint for a "Bioinformatics workstation/server for Ubuntu" https://blueprints.launchpad.net/ubuntu/+spec/biobuntu, which I've marked as obsolete now that NEBC have released Bio-Linux5 based on Ubuntu 8.04 LTS http://nebc.nox.ac.uk/biolinux.html

    The Beowulf 'clustering' and grid aspects of my blueprint overlap with those of the "Easy Ubuntu Clustering" blueprint. In particular, creation of Kerrighed packages for Ubuntu: Some (very) old Debian packages have been removed from the Kerrighed website because they are badly out of date with the upstream sources. I've compiled the latest Kerrighed sources under Ubuntu 8.04, and booted it successfully. I'll post details on the "Easy Ubuntu Clustering" wiki https://wiki.ubuntu.com/EasyUbuntuClustering

    Tony.

  2. #2
    Join Date
    Sep 2008
    Location
    Knoxville, TN
    Beans
    1,075
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Easy Ubuntu Clustering

    Hi ajt

    Glad you brought this back to the front again!

    I was partly involved in the SETI project, quite a number of years ago now though, not in the computer end, just the CPU sharing end is all. UC of Berkely would use my shared computers as a part of their Beowulf Cluster.

    More recently I have done a little on-line research into connecting a few of my computers here together into a cluster, hopefully to speed up some of the intense graphics work we do.

    I've already retrofitted 3 of my computers with gigabyte LAN cards and played around with trying to figure out how to set them up, but I either kept hitting a brick wall or when I did find some instructions, they were so far over my head it was pitiful.

    What led to this in the first place was, I wanted to learn more about how servers work, so I downloaded and installed Edubuntu.
    Edubuntu works out of the box so to speak, but all the computers become dumb terminals and use the computing power of the computer the workstations are connected to, as well as burn up all the memory available. Although I liked the idea of a server, I didn't like the fact IT had to provide everything, including the CPU resources.

    With so many computers sitting around here, I thought, why not tie them all together. After all, we have dual and quad core computers now, so how hard could it be anyhow......?

    Turns out, it's WAY over my head right now, hi hi..........
    Nonetheless, I've learned to do different things on different computers using a KVM then taking all the output and stuff it on a file server. That way I can keep working on other things while the crunching is being done on other machines. NOT a Cluster of course, just more efficient use of the machines I do have here.

    Since then I've dumbed down to a single Data File Server, which is nothing more than File Sharing, and keep all the working programs on different machines, whichever is best for the project at hand.

    But wouldn't it be nice to be able to put all that computing power to use by tying it all together like it was one large computer?

    I see it done all over the place. 4, 6, 8, up to 36, 150 and 250 computers all working together. But they are for specific projects designed to use such a system.

    Any any case, I'm going to keep monitoring this thread and hope it goes somewhere positive and progressive!

    TTUL
    Gary

  3. #3
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by Kellemora View Post
    Hi ajt

    Glad you brought this back to the front again!
    Hello, Gary.

    Thanks for joining this thread about Ubuntu Clustering

    I've been running openMosix for quite a while under Ubuntu 6.06 on our Beowulf cluster:

    http://bioinformatics.rri.sari.ac.uk

    I've posted a few times about openMosix on the Ubuntu Forums, but the threads are very dispersed so I decided to try and focus a discussion here in Education and Science where, I believe, potential Beowulf users might pick up on the thread!

    Quote Originally Posted by Kellemora View Post
    I was partly involved in the SETI project, quite a number of years ago now though, not in the computer end, just the CPU sharing end is all. UC of Berkely would use my shared computers as a part of their Beowulf Cluster.

    More recently I have done a little on-line research into connecting a few of my computers here together into a cluster, hopefully to speed up some of the intense graphics work we do.
    We ran SETI as a test load when I first built our Beowulf, but I found it increasingly difficult to justify using all that electricity and switched to running Folding@home until we got into the top 1,000

    Quote Originally Posted by Kellemora View Post
    I've already retrofitted 3 of my computers with gigabyte LAN cards and played around with trying to figure out how to set them up, but I either kept hitting a brick wall or when I did find some instructions, they were so far over my head it was pitiful.
    There are good instructions about how to setup DHCP and PXE boot Kerrighed at:

    http://www.kerrighed.org/wiki/index....hed_on_NFSROOT

    http://www.kerrighed.org/wiki/index....ROOT_(contrib)

    Quote Originally Posted by Kellemora View Post
    [...]
    But wouldn't it be nice to be able to put all that computing power to use by tying it all together like it was one large computer?

    I see it done all over the place. 4, 6, 8, up to 36, 150 and 250 computers all working together. But they are for specific projects designed to use such a system.
    It's not that complicated if you are motivated enough to experiment. My summer student Kenny Strouts wrote a brief tutorial about his project installing Kerrighed under Debian Etch:

    http://bioinformatics.rri.sari.ac.uk...rial_kerrighed

    Quote Originally Posted by Kellemora View Post
    Any any case, I'm going to keep monitoring this thread and hope it goes somewhere positive and progressive!
    Please do, and see if you can interest any of your friends and colleagues in the thread too.

    Bye,

    Tony.

  4. #4
    Join Date
    May 2008
    Beans
    16

    Re: Easy Ubuntu Clustering

    I am excited about computer clustering as well, but because of no ubuntu availability I have been trying live CD's, and one of the computers I want to use it on doesn't have support for my LAN card. and it doesn't know how to do WiFi either, it thinks it is just an ethernet connection

  5. #5
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    The kerrighed project have recently announced a Mandriva live CD running the Kerrighed Kernel:

    http://www.kerrighed.org/forum/viewtopic.php?p=607#607

    You can use this to see if your computers would run Kerrighed, but beware that it runs a DHCP server, so don't do it on someone else's LAN

    Bye,

    Tony.
    Last edited by ajt; January 14th, 2009 at 12:49 PM. Reason: typo

  6. #6
    Join Date
    Jan 2009
    Beans
    1

    Kerrighed

    Good paper (few years old though) -

    "OpenMosix, OpenSSI and Kerrighed: A Comparative Study"
    http://hal.inria.fr/docs/00/07/06/04/PDF/RR-5399.pdf

    OpenMosix is no longer maintained. Supported distributions for openSSI are Fedora Core 3 and Debian Sarge (with work in progress to port to Etch & Lenny) - so I've no idea as to the status of OpenSSI with regards to Ubuntu. (Maybe someone could shed some light on this?)

    Kerrighed is exciting, but still a research project - not yet production ready. At the moment, node failure under Kerrighed causes the whole cluster to freeze. Work is being done to solve this problem; see the following;
    http://article.gmane.org/gmane.linux...righed.user/69

    At present the best place to seek information concerning Kerrighed is the mailing list, see http://news.gmane.org/gmane.linux.cl...kerrighed.user

  7. #7
    Join Date
    Sep 2012
    Beans
    1

    Re: Easy Ubuntu Clustering

    Well this is really great to know about Clustering, i am using Ubuntu from past 6 months and well i am enjoying lot and very curious to learn more about Ubuntu.

  8. #8
    Join Date
    Oct 2011
    Location
    Oregon
    Beans
    32
    Distro
    Ubuntu Development Release

    Smile Re: Easy Ubuntu Clustering

    I'm also following this thread, I hope to see more updates soon, this is exciting software that can reduce development cost and time especially when clusters are needed to compile large projects; it would be so great to not have to create application specific software again and again.

  9. #9
    Join Date
    Dec 2006
    Beans
    36

    Re: Easy Ubuntu Clustering

    unfortunately Linux clustering is not that active as we might want it.

    OpenSSI -- is almost inactive
    Kerrighed is "rather a bit alive".

    The interest to the subject is not growing:

    http://www.google.com/trends/explore...6%2073m&cmpt=q
    (you may add "linux clustering" or "ubuntu clustering" there)

    Patching the kernel all these years were not that easy as life shows it. It is a pity that Linus Torvalds doesn't look towards clustering.

    Things like global Process IDs, global file descriptors, global socket references, etc are not even discussed at kernel.org

  10. #10
    Join Date
    Nov 2010
    Location
    UK
    Beans
    5
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Re: Easy Ubuntu Clustering

    http://www.stevekelly.eu/cluster.shtml
    in my search today I found the baove site that looks like it is easy scaling after the initial setup
    Last edited by NickJohn; November 26th, 2013 at 03:51 PM.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •