Re: Easy Ubuntu Clustering
Quote:
Originally Posted by
ajt
Hello, djamu.
Not sure why you think it's a bad idea to use Ubuntu?
The Kerrighed kernel is based on the 'vanilla' kernel sources, so it doesn't make much difference if you compile it under Debian or Ubuntu.
True, aside from the upstream implementations on ubuntu that might not work on a downgraded kernel... can't think of any right now but I had my share of those before switching my servers back to debian ( hopeless outdated IDS packages for example, ).
Also a base debian install installs much less "bloat" then ubuntu server does.
For example > there's a new service in intrepid (forgot the name) , that let's you sign up for server monitoring...
Now I understand the need for Canonical to generate income.. but not letting me choose whether I want to install it is of a complete different magnitude... ( I try avoid using windows for the very same reason )....
In a server environment this is of less importance as the performance penalty is negligible...
But for tuned HPC cluster kernels ( to reduce interrupt jitter > whether NUMA / SMP ) every unnecessary service interrupt is 1 to many ...
So if I have the option to install a really minimal system (netinstall) or I have to search thru a stock Ubuntu server install to manually delete all unnecessary "helpful" services ... the choice should be obvious, at least common sense dictates the obvious
Quote:
....
but Sarge and Etch were crude on the Desktop compared to Ubuntu 6.06...
....
That's a long time ago, they still don't install sudo in the base install :smile:
Another point in me using debian is that a lot more packages (not desktop related) are available and rigorously tested > the very reason of it's slow cycle...
( A good example is the fail2ban package, stable on debian, typo bug in the 8.04-LTS, preventing it from restart on boot ... submitted to launchpad with fix...., intrepid has proper version ) > BTW as a matter of fact, this kind of package regression is by LTS policy forbidden. The tight release cycle prevents thorough testing...
But let's not get into any kind of flaming, IMO I really think ubuntu is a great desktop distro..., just my personal opinion..
Quote:
openSSI looks very interesting but, despite several attempts, I've now abandoned any thought of using it. We must be careful not to accuse anyone of using the 'MOSIX' source, because that's proprietary! I guess you are talking about the GPL version when openMosix was forked ;-)
Oops sorry my bad.. yes I did mean OpenMosix...
As for OpenSSI, I do think it's dead for 2 reasons.
1. Kerrighed uses a a substantial part of it's source > making OpenSSI practically obsolete
2. The XtreemOS project, which has (about) the same goal as OpenSSI ( where kerrighed is going to be part of )
http://www.xtreemos.eu/
Quote:
...
It's now unlikely the openMosix will continue (but I'm watching development of PMI:
isn't PMI a rebranded OpenMOSIX ?
Quote:
I agree that the openSSI project seems inactive (but don't dismiss it, because it has a lot of good points too). That leaves Kerrighed, which is still an active EU FP6 (Framework 6) funded project until 2010.
True, but since OpenSSI lacks the funding of the XtreemOS consortium ... chances are little it will emerge again..
http://www.xtreemos.eu/overview/plon...43452/partners
System Architecture Overview:
http://www.xtreemos.eu/science-and-r...s-architecture
Quote:
Why does venture capital mean something has to be proprietary?
It doesn't, it's just easier then providing just a service...
And I wouldn't if a couple of kernel devs chip in..
Canonical have invested a lot in Ubuntu... & Mark Shuttleworth flew in space
:lolflag:
On a more serious note.
Tony (AJT) you seem like a knowledgeable guy, I'd like to discuss in depth a couple of things ( compare my trunk compile kernel configs with yours / discuss different types of distributed FS's / tuning & jitter etc ... ) ..
But I'm a little afraid the title of the thread is not well chosen > I'll elaborate .. quite a lot of people reading this have will have a hard time grasping the type of cluster were discussing. To me it's obvious were talking (headless) HPC clustering...
In other words this thread is prone to pollution as soon as people start suggesting for example RHEL or SAN failover mechanisms as an alternative...
It's getting really late now, let's see where this is going..
Jan
Re: Easy Ubuntu Clustering
Here's an amazing article on how to turn Xboxs into a Beofwulf cluster. Looks inherently more simple than I thought.
http://www.anandtech.com/linux/showdoc.aspx?i=2271&p=8
Re: Easy Ubuntu Clustering
I also had these resources in another thread...
Re: Easy Ubuntu Clustering
I would simply be interested in using 3 old computers running as a cluster to slightly speed things up and add some elbow room. I have tried some live CD's but my old laptop's LAN card isn't supported(odd), and I would need to run a fan control program on one of the computers because it's fan control is not automatic from the bios(I find that stupid)
the last one is a pentium MMX machine.
Re: Easy Ubuntu Clustering
altonbr,
You could add this to you list o' links:
http://www.calvin.edu/~adams/research/microwulf/
Re: Easy Ubuntu Clustering
Quote:
Originally Posted by
machoo02
It's already in there ;)
Re: Easy Ubuntu Clustering
Quote:
Originally Posted by
kloplop321
I would simply be interested in using 3 old computers running as a cluster to slightly speed things up and add some elbow room. I have tried some live CD's but my old laptop's LAN card isn't supported(odd), and I would need to run a fan control program on one of the computers because it's fan control is not automatic from the bios(I find that stupid)
the last one is a pentium MMX machine.
Don't want to discourage you but.
If it's a desktop OS which load you want to automatically share between computers..... come back in a couple of years... this is not possible (yet)... Kerrighed is closest for what you want to do & can migrate processes of standard applications to other computer(s) BUT can't split an application using a single process over 2 computers and doesn't distribute threads...
Brief recap of types:
The term supercomputer might be misleading, as in most cases it's not a single OS that runs over a group of CPU's but rather independent computers that run a common application(s) over a shared network(s) > in other words it would be more correct to talk about an HPC cluster then a supercomputer... ( not 1 computer )
Depending on the type of application / dataset(s) more or less speed is required for the interconnects.
An example:
Meet the most powerfull supercomputer on the planet... in everyones :) home.
Folding@home / Seti@home ( BOINC ) > http://boinc.berkeley.edu/
This is an extreme case of an Embarrassingly Parallel ( the correct term for this kind of HPC clustering ) very little data is send over the network by a scheduler to each compute node..
This kind of number crunching doesn't require intermediate data from other processes and doesn't need distributed RAM.
A classical renderfarm is another example ( with a bit more network traffic )
http://en.wikipedia.org/wiki/Embarrassingly_parallel
On the other end you have those applications / dedicated clusters that require massive amounts of intermediate data / huge databases / shared memory > The big boys from the pictures...
These monsters usually have 3D torus shaped high speed networks ( infiniband / myrinet / 10Gb ) with multiple connections / node and are usually build for a single purpose ( you can't run any app on them because they would run inefficient )
These type of clusters usually run a kind of MPI flavor and can't run stock applications
http://en.wikipedia.org/wiki/Message_Passing_Interface
http://en.wikipedia.org/wiki/OpenMP
Closest to a real supercomputer ( in the correct sense of the word ) and what you most likely want to achieve, unrelated to the previous mentioned MPI systems ( which are individual machines connected thru a network ) are the SSI systems. > Kerrighed / (Open)Mosix ( now PMI ) / OpenSSI
http://en.wikipedia.org/wiki/Single-system_image
One more thing worth noting:
The reason why more and more clusters are built from COTS ( commodity of the shelf > normal desktop computers ) parts instead of custom hardware is that before you had 2 distinct types of CPU's
scalar > fast for single instruction single out
http://en.wikipedia.org/wiki/Scalar_processor
http://en.wikipedia.org/wiki/SISD
vector > fast for single instruction multiple out
http://en.wikipedia.org/wiki/Vector_processor
http://en.wikipedia.org/wiki/SIMD
more: http://en.wikipedia.org/wiki/MIMD
While the x86 family originally was a pure scalar CPU it was unsuited ( very slow ) to do any vector processing.. The addition of the MMX and later 3Dnow / SSE etc.. extensions ( which are vector capabilities ) made the x86 hybrid.
So every modern x86 currently has 1 scalar + 1 vector + floating point + .... / core ( the P4 with hyperthreading had 2 scalar cores with 1 vector and 1 floating point core )
The CELL CPU has 1 scalar + 8 vector cores
Roadrunner ( currently the fastest supercomp ) uses hybrid nodes ( 1 AMD Opteron coupled to 1 CELL )
wanted to keep it brief ... pfff look at all that text again :lolflag:
Jan
Re: Easy Ubuntu Clustering
The scope of EasyUbuntuClustering is very wide, which might be why the discussions on this thread have been so broad.
Is it realistic for an 'easy' clustering solution to cater to inexperienced developers, offer portable environments via VM's, provide automated load-balancing, and still maintain high QoS?
These are great goals to shoot for, but IMO you will likely have more success if you just pick one or two of the major features.
I can't comment much on the process migration (automated load balancing), since it's been more than 4 years since I've used OpenMosix. These features typically require savvy system admins, and configuration problems can be difficult to diagnose.
Automated deployment of VM's on a cluster or grid (not going to get into a definition war here. I'm referring to a grid as a federated assembly of heterogeneous clusters) is a hot topic that industry giants such as VMware (EMC), IBM, Microsoft, and Amazon are investigating. There are numerous technical subtleties relating this topic, and a logical solution might be to create a specific flavor of debian or ubuntu server that is adapted to hosting the popular VM's. This is especially relevant considering the CISC vendors have product roadmap plans that address common VM bottlenecks.
I admire your efforts to approach this complicated technical issue, and my only suggestion is to consider narrowing your scope a bit.
Re: Easy Ubuntu Clustering
Quote:
Originally Posted by
ajt
Hello, djamu.
Not sure why you think it's a bad idea to use Ubuntu?
The Kerrighed kernel is based on the 'vanilla' kernel sources, so it doesn't make much difference if you compile it under Debian or Ubuntu.
Its not just a question of building, its how the kernel is built (i.e. using the Debian / Ubuntu patching and build system). The current Kerrighed 2.6.20 kernel (the way its patched and built) would make this problematic.
However, the jump to 2.6.29 would make it much easier.
Quote:
I agree that the openSSI project seems inactive (but don't dismiss it, because it has a lot of good points too). That leaves Kerrighed, which is still an active EU FP6 (Framework 6) funded project until 2010.
I remain subscribed to the OpenSSI lists. Once in a while you'll see a flurry of activity, then radio silence for a few months. I'm still a big fan of the project, I think it will continue to advance for years to come. Its one of those projects where you need a solid block of time to actually do anything useful, so I'd imagine the devs devote what they can to it.
Quote:
Why does venture capital mean something has to be proprietary?
Canonical have invested a lot in Ubuntu...
For a long time many people used the word 'commercial' to refer to non-free software which now causes a lot of confusion and mistrust.
Kerrighed has a very good, very honest and very transparent business model. Use / share / modify the software any way you like .. if you get stuck, paid support and development is available.
You will always have those who are a little over-sensitive to this kind of arrangement. For instance, someone might ask for help to implement thread migration and get a reply that they'll be better off hiring Kerlabs (which they would be). Then, the person will start screaming 'baitware' or worse.
I don't know if you'd be able to find a MOTU that would be interested in keeping up with Kerrighed, but surely nothing is preventing someone from making it available via PPA.
Re: Easy Ubuntu Clustering
In the next month I'll try to setup a small cluster (using kerrighed or mosix) to research purpose (http://afni.nimh.nih.gov/afni). I have two dell poweredge 2900 monster (2cpu x 4 core) and several "old" dual CPU Xeon 3Ghz. The first idea is to use ubuntu hardy with a custom kernel (2.6.20 for kerrighed or 2.6.28 for mosix (it's free for researchers)). I think that it'will be a tricky job patching the kernel (with 64 bit support and all the modules to load onto pe2900 and others pcs)...but I'll try to do it, because ubuntu/debian is the most easy and powerful distro to use, configure and administrate.