parallel

Started by benthehutt, April 12, 2005, 08:40:05 PM

Previous topic - Next topic
Print
Go Down Pages1 2
User actions
April 12, 2005, 08:40:05 PM
Does anyone have any good resources on running two computers in parallel?  I've heard it's pretty easy, but I don't really know even where to begin.

Oh, and do they have to be the same speed?  Cause that would burn a hole in my plan...
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
April 14, 2005, 02:50:05 PM #1
You mean a Beowulf cluster?
"A well known hacker is a good hacker, an unknown hacker is a great hacker..."

I don't care what your parents told you, you aren't special.
  • https://github.com/tazinator
April 14, 2005, 05:33:49 PM #2
Totally!! I really don't know much about them, but Google has helped me find my way into the world of parallel systems and programming.

However, I have found some cool stuff on PVM, MPI, Java Remote Threads, et cetera, but I don't really know which one I want to do.  I also don't understand what the OS actually does or how to get an OS for a parallel system.  Some Beowulf clusters run Windows AND Linux AND Unix, and some just run PVM stuff.  Do you know what OSes will work on parallel systems, or how they work together?

Any info on anything I just spewed would be awesome! ;D
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
April 14, 2005, 11:48:12 PM #3
I used to have a lot of resources for this with the first version of beowulf but I learned the hard way to back up my files... cest la vie!
Godaigo
All's fair in Love and Brewing.
April 15, 2005, 01:16:37 PM #4
I read an article on (I think it was about using GCC) in parallel, and I think it was on a site owned by IBM.. could be wrong though as it's been a while.

But then linux and unix in general are able to do parallel processing even with one processor, more or less.

I don't know your primary goal, but for me it'd be compiling large programs (since that can take a while). GCC for example, and from their website:

Quote
0.3 Building in parallel
You can use `make bootstrap MAKE="make -j 2" -j 2', or just `make -j 2 bootstrap' for GNU Make 3.79 and above, instead of `make bootstrap' to build GCC in parallel. You can also specify a bigger number, and in most cases using a value greater than the number of processors in your machine will result in fewer and shorter I/O latency hits, thus improving overall throughput; this is especially true for slow drives and network filesystems.

So it kind of depends on platform. Then again, I'm guessing you are referring to actually using two computers ? That I wouldn't know about (though I am thinking the article I mentioned in the beginning of this post did discuss it). I think though, there is a howto on it somewhere on the linux doc project.


Well, just some thoughts.

"My Terminal is my Soul"
April 16, 2005, 02:52:34 PM #5
Wow, that's actually extremely helpful, thanks guys! :D

My plan is to get a bunch of computers and hook them up via ethernet on a switch to make a Beowulf cluster.  I'll probably install either FC3 or BSD of some sort on each machine, and I have to set one of them as the master server.  On this one, I set the root password to be global on all the nodes, and I'll probably just make sure manually that all the /etc/passwd and /etc/group authentication stuff is all the same.

Then I have to mount a drive on the master server in fstab or with an automount or something and set up /etc/passwd so that all the users home directories are on it, and so that they all have access to it.

Then (this is where my knowledge starts to get a bit foggy) I want to use bash because I'm most familiar with it, but "they claim" that ssh makes the best remote login software...can i use them in conjunction?

Anyway, I just get all the computers to "do their thang" and it should work pretty well.
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
April 16, 2005, 03:23:01 PM #6
ssh and bash will work fine together.

The name might be misleading (since it's secure shell and bash is also a shell) but they're not the same thing. SSH is what you log in through, where as bash is the command interpreter. So ssh is not really a shell in that sense.

I actually have ssh into my linux boxes through windows (since I only have one monitor, I figure I should use it for the one graphical interface I have).

So I log in to my server with an ssh client, and then use bash (or whatever shell) to do my work.

Check out openssh.org

Hope that helps

"My Terminal is my Soul"
April 18, 2005, 11:57:53 AM #7
O man, thanks alot metgod.  I was so confused for a minute there, I understood that ssh was a login client, but because it is a "secure shell" I thought that that must mean bash can also do that.  Whew....

Anyway, I'm going ahead with the "Beowulf project", probably using some distro of Linux like fedora or redhat (simply cause I already own redhat) because they're so similar.  I'll let you know how it goes and what hardware I end up using.  Toodles! ;D
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
April 18, 2005, 01:03:24 PM #8
Glad to be of help.

I actually don't know why they call it 'secure shell' (though I can take a guess) but I can see where it might be confusing.

But definitely install it.. it's much better than telnet the telnetd (daemon). Telnet is nice for some things, but when you're talking about logging into a system that you don't want intruders in, use ssh over telnet any day.

Try this sometime... open telnet (but block it at the gateway or all remote logins/however you wish), and then start a tcpdump session (or whatever your favorite sniffer is). Telnet in, and you'll see in the tcpdump session your login info.

If you get too much data (probably will), then you can always pipe it through grep to only grab certain text.

Regular POP3 mail does this too.. you can see mail passwords if you're running a sniffer.

So definitely install ssh and keep telnet daemon shutdown (if its there at all). You can use the telnet client for various things, but don't let others telnet into your system (or even yourself -- ssh will do that but do a much safer job).


"My Terminal is my Soul"
April 18, 2005, 10:52:56 PM #9 Last Edit: April 18, 2005, 11:01:49 PM by Tazinator
"A well known hacker is a good hacker, an unknown hacker is a great hacker..."

I don't care what your parents told you, you aren't special.
  • https://github.com/tazinator
April 18, 2005, 11:34:02 PM #10
Yeah, and while I'm at it I'll have them chrome plated... ;)
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
April 20, 2005, 08:41:01 PM #11
Another issue I have to settle is powering multiple motherboards at once.  Are there power supplies that can do that?  or do I just need like 10 small ones?  In either case, a normal MB starting up peaks at about 150 Watts, then goes down below 100W.  That means that I'll have (150W/115V)=1.3A*10 = 13 Amps!!  That's crazy!! It'll blow something!!  Is there anyway to do it with less PSes?
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
August 13, 2005, 01:59:14 AM #12
I found out that I probably won't have any PS issues.  Each PC will only have a small hd, a nic, and a small mb.  The power drain will be minimal (unless I used 15000rpm scsi's or something), so each node should only use, max 50 watts on startup.  50W/115V = .43A*10 = 4.3A.  Problem solved, though I guess it didn't actually start. ;D
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
September 11, 2005, 01:14:43 AM #13
Don't know if anyone cares anymore, but my Beowulf cluster is actually taking shape as of now (I'm a man of my word).  So I went to an auction and bought 17 PIIIs and 3 G3s each with an hd and 128M RAM.  Anyway, now that I have all these computers I have no choice but to build a cluster, all I need now is a fast ethernet switch (20 port, obviously).  I'm hoping that the old Baystack we use as a door stop will work, I'm only going 100, not 1000 mbit/sec.

Don't think I'll be able to get it done soon, but I'll be putting up all my recent developments up here on the forum, and tutorials and whatnot on how to do it yourself.  Should be fun!

First tutorial: "How to chose a parallel OS" (Coming soon)
Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
September 11, 2005, 10:29:15 AM #14
Sounds cool. Looking forward to it.
"My Terminal is my Soul"
Print
Go Up Pages1 2
User actions
SMF spam blocked by CleanTalk