Page 4 of 49 FirstFirst ... 2345614 ... LastLast
Results 31 to 40 of 489

Thread: Easy Ubuntu Clustering

  1. #31
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by djamu View Post
    [...]
    Tony are you really implying an X server / client in the SSI nodes ? Aside from the unnecessary cost this would cause major problems as all GPU's should be of the same brand. Considering you'd only install the client X libraries on the nodes and use a remote X > XDMCP ( the machine which renders the image on screen / has display is the X server ) then still it's nearly impossible since kerrighed doesn't support LVS ( common cluster IP address ) that the X client could use to communicate with a standalone X server > could be handy for a multiseat system .
    Hello, Jan.

    The Xserver runs on the client PC, where it displays the output of the Xclient running on the SSI. My present openMosix system has two servers and 88 compute nodes. One server is the 'head' node of the cluster, providing PXE + DHCP + NFSROOT and a web server:

    http://bioinformatics.rri.sari.ac.uk

    The other server is for interactive logins via SSH/NX/VNC and is where the user's home directories are stored. They are exported via NFS to the nodes. Only the servers are allowed to migrate processes away to other nodes. The Beowulf becomes unstable if all nodes are allowed to migrate processes. The compute nodes accept migrated processes from the servers, but do NOT migrate their own local processes. This allows us to run MPI and SGE as local processes without them being migrated off nodes...

    Quote Originally Posted by djamu View Post
    I have some experience with OpenGL clusters, and that is a completely different topic in all aspects ( would be nice though ).....
    In any case, the easiest way to implement a GUI is a web-based one tied to a job scheduler.
    I've been looking at eyeOS:

    http://eyeos.org/

    I've installed it, but not done much with it yet.

    Most of the time, we just login using NX. If we want to run 'batch' jobs, we use SGE "qmon", or command-line tools to submit jobs. The webapps we run provide a web-based interface via the Beowulf web server using the Apache JK connector. Some, but not all, of the work accepted by these webapps is migrated automatically to compute nodes by openMosix. Any jobs using shared memory, in particular, are not migrated. This is due to the limitations of openMosix, as you know. Interrestingly, I did experiment with the openMosix 'MIGSHM' patch: My processes migrated away fine, but the didn't come back!

    Bye,

    Tony.

  2. #32
    Join Date
    Jan 2009
    Location
    Hungary
    Beans
    2
    Distro
    Xubuntu 8.04 Hardy Heron

    Re: Easy Ubuntu Clustering

    Hi,

    How about porting Perceus to ubuntu/debian?

    http://www.linux-mag.com/id/6386

    To be frank I am a cluster newbie, I have just built my own cluster to learn the technology, and after lot of reading and digging I have started to use Perceus, more precisely Caoslinux, for the same reason you started this wiki, as it makes cluster management easy, and it realy does the job. On the other hand, I am an Ubuntu fun, I am familiar with it, so I would love to do the same with Ubuntu. Unfortunately I am not a linux/ubuntu admin guru, otherwise I would have started the porting already.

    Probably you do not know, that Perceus is the provisioning part of Warewulf, Caoslinux is the distribution to manage them, and the whole lot is maintained by Infiscale.

    http://www.perceus.org/portal/
    http://www.caoslinux.org/index.html
    http://www.infiscale.com/

    Cheers,

    Zsolt
    Last edited by zkovax; January 23rd, 2009 at 03:26 PM.

  3. #33
    Join Date
    Sep 2006
    Beans
    1

    Smile Re: Easy Ubuntu Clustering

    Hi all!!

    Can anyone post a simple and exaustive guide on how to install a cluster of pc using Kerrighed 2.3.0 and Ubuntu 8.10 or 8.04 ?

    Thanks

  4. #34
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by masulli View Post
    Hi all!!

    Can anyone post a simple and exaustive guide on how to install a cluster of pc using Kerrighed 2.3.0 and Ubuntu 8.10 or 8.04 ?

    Thanks
    Hello, masulli.

    Not yet, but I hope we can create a guide:

    https://blueprints.launchpad.net/ubu...ntu-clustering

    Bye,

    Tony.

  5. #35
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by zkovax View Post
    Hi,

    How about porting Perceus to ubuntu/debian?

    http://www.linux-mag.com/id/6386

    [...]
    Probably you do not know, that Perceus is the provisioning part of Warewulf, Caoslinux is the distribution to manage them, and the whole lot is maintained by Infiscale.

    http://www.perceus.org/portal/
    http://www.caoslinux.org/index.html
    http://www.infiscale.com/
    Hello, Zsolt.

    Actually, I do know about Warewulf, but 'cAos' is an rpm-based distro and I want to use a deb-based distro - preferably Ubuntu

    There are similar systems like DRBL:

    http://drbl.sourceforge.net/

    These systems are very useful for 'jumpstarting' or administering Beowulf clusters - DRBL has been used to deploy Kerrighed:

    http://trac.nchc.org.tw/grid/wiki/krg_DRBL

    However, systems like this do not address the issue of how to actually use a cluster for HPC (High performance Computing). A Beowulf is not just a COW (Collection Of Workstations), it's more than that...

    What is interesting about openMosix, openSSI and Kerrighed is that they do kernel-level load balancing in a similar way that an SMP (Symmetric Multi-Processing) computer does, to make use of multiple CPU's. There is a HUGE difference in cost between e.g. building a Beowulf cluster from 16 ordinary COTS (Commodity Off The Shelf) PC's and buying in a commercial 16p SMP computer system, which is why this is a popular approach.

    The BIG difference is that, without non-COTS cluster node interconnects, node latency can be extremely large in comparison to local SMP processor interconnects. Kerrighed uses an optimised 'kernel' TCP/IP communication protocol to minimise the latency in communication between Linux kernels running on processors in different Kerrighed nodes. That's one reason why I think Kerrighed is well worth looking at.

    Bye,

    Tony.

  6. #36
    Join Date
    Sep 2008
    Location
    Las Vegas, NV U.S.A.
    Beans
    243
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Red face Re: Easy Ubuntu Clustering

    hi,

    i have tried to cover all the posts here (at least one time) so far. there has been allot of lofty discussion of architecture and so forth that drifts from (and imo confuses) the issue.

    i have only read joel adams paper on how he configures his microwulf using ubuntu one time. but it appears that he has got what the original thread post requested.

    except for the fact that he gives the instructions where the assumption is that the slave nodes will be diskless the description of how to configure a cluster using ubuntu where the nodes are connected by a cots switch is as easy as it will get without some kind of cluster installer like that for the .iso distro of ubuntu.


    once again that link is...
    http://www.calvin.edu/~adams/research/microwulf/
    ...that's it folks!

    that is the answer to this thread.


    i intend to use it as a guide to try and establish my first cluster of two identical ibm t60 laptops run by ubuntu by simply connecting them with a cat-5 patch chord (no switch) and configuring them using his configuration instructions as a guide. i don't think there will be any problem doing this (without the switch) to get started.

    perhaps the best way for those interested in actually building a cluster is to follow my lead with whatever hardware they have available. that is to say just use two boxes that run ubuntu, connect them with a cat-5 patch chord and use joel's configuration instructions as a guide.

    once you have two machines that are an actual cluster just scale up.


    peace!

    -zander

  7. #37
    Join Date
    May 2008
    Beans
    71

    Question Re: Easy Ubuntu Clustering

    What I am curious about is how the computers are connected... Certainly it won't work by just connecting two computers with a LAN cable, so it seems that a router or sth more advanced is needed...

    All I have is a modem which is provided by my ISP. I guess it is something that you connect it with a LAN cable and a telephone line so that you can access the web.

    This is what I have done a month before:
    ==================================================
    Hardware:
    1. a win95 box with 133MHz CPU & 32Mb RAM
    2. an xp box with 2GHz CPU,256Mb RAM
    3. an ADSL modem with 4 slots for LANs
    4. two LAN cables

    Procedures:
    I connected the two boxes to the modem with the LAN cables. Then, I tried to boot both boxes using liveCDs like CHAOS and ClusterKnoppix. Afterthat, I tried some floppies like GoMF and openMosixLOAF. Finally, I tried to start the following shell-script in background:
    s=0
    for (( i=1; i>0; i=i+1 ))
    do
    s=$(($s+$i))
    done

    Results.
    liveCDs: cannot boot on the old box. success to boot on the new box but cannot configure itself as a node - sth called "DHCP server" was misssing.

    floppies:
    a. GoMF can boot on both boxes, but the same problem occured - "DHCP server" was absent.
    b. openMosixLOAF can boot on both boxes, success to configure as nodes by manually entering private IPs and Gateway. can view cpu loads, ram utilization in real time from any boxes. cannot migrate the shell script because the floppy is not shipped with commands like "mosrun".
    ================================================== =======

    To sum up, any attempt to construct openmosix-cluster using a modem failed (without compiling, of course). Could anyone tell me what is a "DHCP server"?

    It seems that the so called "DHCP" is related to auto-detection & auto-configuration, and cannot be turned off. Is it possible to solve this without having to buy new hardware(s)?
    Last edited by antony_css; January 27th, 2009 at 01:07 PM.

  8. #38
    Join Date
    May 2008
    Beans
    71

    Re: Easy Ubuntu Clustering

    Well, after surfing Wikipedia, I found that my ADSL modem is actually a router...
    But that changes nothing... my ISP provided me this "modem" as a part of the internet service, but they didn't provide any further support on "advanced" usage, even a not a manual...

  9. #39
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by zander1013 View Post
    hi,

    i have tried to cover all the posts here (at least one time) so far. there has been allot of lofty discussion of architecture and so forth that drifts from (and imo confuses) the issue.
    Hello, zander.

    I think it's important to place what we are discussing in context: In fact, SIMD instructions are now used in the FPU's of modern processors and I think it's quite useful to have a little background information

    Quote Originally Posted by zander1013 View Post
    i have only read joel adams paper on how he configures his microwulf using ubuntu one time. but it appears that he has got what the original thread post requested.

    except for the fact that he gives the instructions where the assumption is that the slave nodes will be diskless the description of how to configure a cluster using ubuntu where the nodes are connected by a cots switch is as easy as it will get without some kind of cluster installer like that for the .iso distro of ubuntu.
    I don't think that building a Microwulf could be described as 'easy', and what I've read about Microwulf focusses more on the hardware aspects of how to construct a DIY Beowulf. Segregating 'system' and 'application' network traffic is a good idea, but it's not an original one:

    http://bioinformatics.rri.sari.ac.uk/bobcat/

    Quote Originally Posted by zander1013 View Post
    once again that link is...
    http://www.calvin.edu/~adams/research/microwulf/
    ...that's it folks!

    that is the answer to this thread.
    I think the Microwulf project is great, in terms of its hardware, but it doesn't bring anything new to the software requirements of building Ubuntu clusters. In relation to 'diskless' booting, for example, there has been an Ubuntu 'diskless' wiki for a long time:

    https://help.ubuntu.com/community/DisklessUbuntuHowto

    The type of Beowulf cluster installation/management packages that are discussed by the Microwulf project are well known. What I suggested is that we try to package Kerrighed for Ubuntu, because Kerrighed is a state of the art SSI (Single System Image) Linux kernel load-balancing system that has already been packaged for Mandriva Linux. I think it would be useful to package it for Ubuntu too.

    Quote Originally Posted by zander1013 View Post
    i intend to use it as a guide to try and establish my first cluster of two identical ibm t60 laptops run by ubuntu by simply connecting them with a cat-5 patch chord (no switch) and configuring them using his configuration instructions as a guide. i don't think there will be any problem doing this (without the switch) to get started.
    Make sure you use a crossover cable if you do (or check that your laptop has autosensing NIC's). It's quite common for ISP's to provide a combined ADSL modem + router + WiFi these days, but a passive ethernet hub would be another solution for a low-cost experimental setup.

    Quote Originally Posted by zander1013 View Post
    perhaps the best way for those interested in actually building a cluster is to follow my lead with whatever hardware they have available. that is to say just use two boxes that run ubuntu, connect them with a cat-5 patch chord and use joel's configuration instructions as a guide.

    once you have two machines that are an actual cluster just scale up.
    As Bart says: "Don't have a COW, man...", but that is exactly what you will create if you just connect together a Collection Of Workstations. There's a lot more to building a cluster than simply connecting computers together. However, connecting them up *is* quite important

    Quote Originally Posted by zander1013 View Post
    peace!
    A friend of mine saw graffiti in Florence recently that read:

    Make tea, not war

    I like that

    Bye,

    Tony.

  10. #40
    Join Date
    Jan 2007
    Beans
    99

    Re: Easy Ubuntu Clustering

    Quote Originally Posted by antony_css View Post
    [...]
    To sum up, any attempt to construct openmosix-cluster using a modem failed (without compiling, of course). Could anyone tell me what is a "DHCP server"?

    It seems that the so called "DHCP" is related to auto-detection & auto-configuration, and cannot be turned off. Is it possible to solve this without having to buy new hardware(s)?
    Hello, antony_css.

    A DHCP (Dynamic Host Configuration Protocol) server listens for computers on the network broadcasting DHCPDISCOVER requests and supplies them with a dynamic network configuration. This normally consists of an IP address, netmask and default router addess. Often, the DHCP server also supplies DNS name server addresses and DNS domains to search. In addition, modern DHCP servers support network booting and supply information about TFTP (Trivial File Transfer Protocol) where a boot image can be downloaded.

    Although this might sound complicated, many routers run DHCP servers so all you have to do is connect your devices to the router, and find out what their IP addresses are by running:

    ipconfig -a

    This is necessary if you want to connect your computers together, because they need to know each other's IP address. This is a common private IP network:

    192.168.0.254 router
    192.168.0.1 node1
    192.168.0.2 node2
    ...

    For a small network, you can put this information in the /etc/hosts file.

    Bye,

    Tony.
    Last edited by ajt; January 28th, 2009 at 01:02 AM.

Page 4 of 49 FirstFirst ... 2345614 ... LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •