Does anyone have any good resources on running two computers in parallel? I've heard it's pretty easy, but I don't really know even where to begin.
Oh, and do they have to be the same speed? Cause that would burn a hole in my plan...
You mean a Beowulf cluster?
Totally!! I really don't know much about them, but Google has helped me find my way into the world of parallel systems and programming.
However, I have found some cool stuff on PVM, MPI, Java Remote Threads, et cetera, but I don't really know which one I want to do. I also don't understand what the OS actually does or how to get an OS for a parallel system. Some Beowulf clusters run Windows AND Linux AND Unix, and some just run PVM stuff. Do you know what OSes will work on parallel systems, or how they work together?
Any info on anything I just spewed would be awesome! ;D
I used to have a lot of resources for this with the first version of beowulf but I learned the hard way to back up my files... cest la vie!
I read an article on (I think it was about using GCC) in parallel, and I think it was on a site owned by IBM.. could be wrong though as it's been a while.
But then linux and unix in general are able to do parallel processing even with one processor, more or less.
I don't know your primary goal, but for me it'd be compiling large programs (since that can take a while). GCC for example, and from their website:
Quote
0.3 Building in parallel
You can use `make bootstrap MAKE="make -j 2" -j 2', or just `make -j 2 bootstrap' for GNU Make 3.79 and above, instead of `make bootstrap' to build GCC in parallel. You can also specify a bigger number, and in most cases using a value greater than the number of processors in your machine will result in fewer and shorter I/O latency hits, thus improving overall throughput; this is especially true for slow drives and network filesystems.
So it kind of depends on platform. Then again, I'm guessing you are referring to actually using two computers ? That I wouldn't know about (though I am thinking the article I mentioned in the beginning of this post did discuss it). I think though, there is a howto on it somewhere on the linux doc project.
Well, just some thoughts.
Wow, that's actually extremely helpful, thanks guys! :D
My plan is to get a bunch of computers and hook them up via ethernet on a switch to make a Beowulf cluster. I'll probably install either FC3 or BSD of some sort on each machine, and I have to set one of them as the master server. On this one, I set the root password to be global on all the nodes, and I'll probably just make sure manually that all the /etc/passwd and /etc/group authentication stuff is all the same.
Then I have to mount a drive on the master server in fstab or with an automount or something and set up /etc/passwd so that all the users home directories are on it, and so that they all have access to it.
Then (this is where my knowledge starts to get a bit foggy) I want to use bash because I'm most familiar with it, but "they claim" that ssh makes the best remote login software...can i use them in conjunction?
Anyway, I just get all the computers to "do their thang" and it should work pretty well.
ssh and bash will work fine together.
The name might be misleading (since it's secure shell and bash is also a shell) but they're not the same thing. SSH is what you log in through, where as bash is the command interpreter. So ssh is not really a shell in that sense.
I actually have ssh into my linux boxes through windows (since I only have one monitor, I figure I should use it for the one graphical interface I have).
So I log in to my server with an ssh client, and then use bash (or whatever shell) to do my work.
Check out openssh.org
Hope that helps
O man, thanks alot metgod. I was so confused for a minute there, I understood that ssh was a login client, but because it is a "secure shell" I thought that that must mean bash can also do that. Whew....
Anyway, I'm going ahead with the "Beowulf project", probably using some distro of Linux like fedora or redhat (simply cause I already own redhat) because they're so similar. I'll let you know how it goes and what hardware I end up using. Toodles! ;D
Glad to be of help.
I actually don't know why they call it 'secure shell' (though I can take a guess) but I can see where it might be confusing.
But definitely install it.. it's much better than telnet the telnetd (daemon). Telnet is nice for some things, but when you're talking about logging into a system that you don't want intruders in, use ssh over telnet any day.
Try this sometime... open telnet (but block it at the gateway or all remote logins/however you wish), and then start a tcpdump session (or whatever your favorite sniffer is). Telnet in, and you'll see in the tcpdump session your login info.
If you get too much data (probably will), then you can always pipe it through grep to only grab certain text.
Regular POP3 mail does this too.. you can see mail passwords if you're running a sniffer.
So definitely install ssh and keep telnet daemon shutdown (if its there at all). You can use the telnet client for various things, but don't let others telnet into your system (or even yourself -- ssh will do that but do a much safer job).
Since you're looking to get an Apple anyhow, get 2 of these ;)
http://www.apple.com/xserve/cluster/
Solves both
;D
Some links I found scouring:
http://beowulf.cheme.cmu.edu/
http://blizzard.rwic.und.edu/~nordlie/zeus/
http://blizzard.rwic.und.edu/~nordlie/deuce/
http://blizzard.rwic.und.edu/~nordlie/miniwulf/
Yeah, and while I'm at it I'll have them chrome plated... ;)
Another issue I have to settle is powering multiple motherboards at once. Are there power supplies that can do that? or do I just need like 10 small ones? In either case, a normal MB starting up peaks at about 150 Watts, then goes down below 100W. That means that I'll have (150W/115V)=1.3A*10 = 13 Amps!! That's crazy!! It'll blow something!! Is there anyway to do it with less PSes?
I found out that I probably won't have any PS issues. Each PC will only have a small hd, a nic, and a small mb. The power drain will be minimal (unless I used 15000rpm scsi's or something), so each node should only use, max 50 watts on startup. 50W/115V = .43A*10 = 4.3A. Problem solved, though I guess it didn't actually start. ;D
Don't know if anyone cares anymore, but my Beowulf cluster is actually taking shape as of now (I'm a man of my word). So I went to an auction and bought 17 PIIIs and 3 G3s each with an hd and 128M RAM. Anyway, now that I have all these computers I have no choice but to build a cluster, all I need now is a fast ethernet switch (20 port, obviously). I'm hoping that the old Baystack we use as a door stop will work, I'm only going 100, not 1000 mbit/sec.
Don't think I'll be able to get it done soon, but I'll be putting up all my recent developments up here on the forum, and tutorials and whatnot on how to do it yourself. Should be fun!
First tutorial: "How to chose a parallel OS" (Coming soon)
Sounds cool. Looking forward to it.
If you are having a hard time with this project, you should consider trying Cluster Knoppix - it is a live CD distibution, obviously based on Knoppix, that includes some OpenMosix software.
check out http://clusterknoppix.sw.be/
I've looked into that, but there's no point in doing it if I just have to slip a cd in the drive and start it. I want to go through all the hastle of programming all the nitty gritty stuff. I've not done much with it recently--work and whatnot.
yeah, sure u got a point there 8)
i just dropped the idea in case u only wanted something fast, and get it working fast ;)
keep us up to date with your project, seems interesting ;)
Oh, totally--I just haven't had much time to mess with it lately. I'm taking some pretty serious math classes and working quite a bit.
oh, man - i just hate math! and the sad thing is that we have a lot of math in our university.. :/
Well, I'm a math major, so I (obviously) take quite a bit. Mmmm....math...
I actually envy you ben. Discovering that you actually like math while you're still in the position to actually take classes. I only discovered that I actually liked math after I was done with my undergrad so I didn't have much time to take anything in grad school. Now I really wished I could have. There is so much cool analysis you can do with it. Ahhh Science (including math!)!
Not fond of mathmatics myself ;P
Ive still not found many uses in my life for needing to know the diameter of circle or anything in Trig or Algebra for that matter.
i will agree with Taz, there is no point in that..
but life 's unfair .. 8)
Ha, it becomes even more useless once you finish calc 3. But that's not the point--I love it cause it's there for my mind to indulge in.
you can train your mind with more interesting and practical things..
no need to worry about maths :P