A PelicanHPC Setup That Works (for me!): Hardware and Software Configurations

[genechng]

Prerequisites:

I use Ubuntu 9.10 Karmic Koala (32bit) [From Michael Creel: you can build a Pelican image on any Linux distro, you just need to install live-helper using the source package, rather than the .deb]. You'll also need a powerful PC with lots of disk space to make a custom pelican CD. I use a AMD Phenom 9550 Quad Core 2.20GHz, 2GB RAM, 500GB HDD (80GB is sufficient) Desktop PC. When I use my dual-core 1GB RAM laptop to make the custom iso PelicanHPC, it sometimes crashes.

Once you're in Ubuntu, take these steps to login as root:

1. Open terminal
2. set the root password with "sudo passwd root" (if you can't, restart Ubuntu and login as Safe-Mode, you're automatically at root, change the password there)
3. Login as root with "su" (use CTRL-D to get out of root)

Make Pelican Prerequisites

1. Download the two files below from this link, other sources won't work (the files needs to be copied to /home/user)
   http://download.mi.hs-heilbronn.de/pelicanhpc/archive/
   
1. live-helper_2.0~a10-1_all.deb
2. make_pelican-v2.1 (or make_pelican-v1.9.1)
2. At the terminal, use pico text editor "pico make_pelican-v2.1" to change/add packages in the package list. These packages are optional and are for my research:
• SDL libraries (libsdl1.2-dev, libsdl-ttf2.0-dev)
• SCITE text editor (scite)
• XChat IRC chat (xchat)
• Any other required items for your research like chrome browser, etc.
  
3. You can also change the default password "PleaseChangeMe!" in the file.
   
4. You'll need to make sure a few more things are in your Ubuntu system. Type these into the terminal to install them:
   
• sudo apt-get install deboostrap
• sudo apt-get install wget
• sudo apt-get install rsync
5. Now you're ready to make the pelican script.
   

Make Pelican Custom CD
Follow these steps to make the Pelican custom CD

1. type "dpkg -i live-helper<press tab>" to install the live helper
2. Now compile the custom CD with "sh make_pelican-v2.1"
3. Sit back and watch for errors.
4. At the end of the compilation, you will get a binary.iso image in the "i386/frontend/" folder
5. Burn the binary.iso into a CD/DVD
   

Hardware Configuration
This is the hardware configuration that I use to set up the computing nodes with internet connection on the front end node (See image).




Front End Node:

• A Desktop PC: AMD Phenom 9550 Quad Core 2.20GHz, 2GB Kingston DDR2 400MHz RAM, Western Digital HDD 500GB Sata-300.
• Two network cards for the front end node:
• One on-board (for internet - file transfers)
  
• The other is a cheap ebay £2.90 Pluscom PCI Realtek RTL8169/8110 Family Gigabit Ethernet NIC)
• A cable connecting the on-board NIC to my D-Link DI-514 router
• A cable connecting the Pluscom NIC to the Netgear FS608 uplink port (any, from 2 to 8) of the switch. The downlink (port 1) could be used too but may clash with the DHCP of the router.
  

Compute Node:

• Dual-core, 1GB laptop or desktop that allows netboot/LAN boot (set from the BIOS)
  
• One build-in network card for the dual-core compute node (laptop) or a £2.90 Pluscom PCI Realtek RTL8169/8110 Family Gigabit Ethernet NIC for the desktop
• A cable to connect the NIC to one of the port (any, from 2 to 8) of the switch
• You may add more compute nodes to the other port of the switch with the same or different configurations
  

Router and Switch

• D-Link DI-514 Wireless Router + 4 port switch to connect the front end node to the internet (Optional). DHCP is set to enabled for auto IP assignment
  
• A PCWorld.co.uk web price £21.99 Netgear FS608 8 port 10/100 Switch to connect the front end node to the other nodes

Booting Up Front End and Computing Nodes

1. Boot up the PelicanHPC Custom CD on the front end node and follow instructions (all should be default until you reach the "PleaseChangeMe!" password, change the password unless you've already changed it during the editing of the make_pelican-v2.1 script.
   
2. Once you've reached the prompt, login as user with the password you changed earlier
   
3. Type "startx" to start X Windows (GUI) (CTRL-ALT-BACKSPACE kills X Windows)
   
4. Open a terminal.
   
5. At the prompt, type "pelican_setup" to configure the front end node, follow instructions and boot up your other nodes when prompted.
   
6. Now to check if your nodes work. Open bhosts (where node ip is automatically recorded) with "pico ¬/tmp/bhosts", the front end node is "10.11.12.1", all other computing nodes are "10.11.12.x"
7. Open another terminal and type "ssh 10.11.12.x" (x is the number of one of the node), once you're connected and at the prompt, type "htop" to monitor that node's activities (You can open more terminals to monitor other nodes)
   
8. To add more nodes later, use "pelican_restart_hpc"
9. You are now ready to test the MPI
   

Note:

• To get superuser access in PelicanHPC, you do "su -s"

Testing the MPI

• Files must be compiled and mpirun in /home/user for all nodes to be accessible.
• There are some code examples at: http://users.abo.fi/mats/PP2008/programs/MPI/index.html
• Make sure you have a "htop" running and monitoring the frontend node and the compute node so that you can see the processors used.
  

Follow these steps to run the MPI across all nodes:

1. Use pico or any other text editor to type in the code below. Name it greeting.c
   
2. Compile the code with: "mpicc greeting.c -o greeting"
3. Run the program with: "mpirun -np 6 --hostfile /home/user/tmp/bhosts greeting"
4. If all goes well, you shall see from "htop" at the front end and the compute node showing a few "greeting" processes. The CPU % should also increase in both nodes.

greeting.c

#include <stdio.h>
#include "mpi.h"
int main(int argc, char *argv[ ])
{
    int rank, size;
    char name[MPI_MAX_PROCESSOR_NAME];
    
    int resultLength;
    
    MPI_Init(&argc, &argv);

    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    MPI_Comm_size(MPI_COMM_WORLD, &size);

    MPI_Get_processor_name(name, &resultLength);

    printf("%s: I am %d of %d\n", name, rank, size);
    
    MPI_Finalize();

    return 0;
}

• Note: Running a more intensive program (Mandelbrot, Monte Carlo simulation, etc) will use more % of the CPU.
  

Virtual Box
Virtual box configurations and boot up are the same. Make sure you set up the front end node with two Network adapters: PCnet-FAST III NAT for connection to internet and PCnet-FAST III (Internal network, 'intnet') for connection to the compute nodes. All compute nodes should have only a single NIC: PCnet-FASTIII (Internal network, 'intnet'). Boot up the front end node first before booting up each of the node during the prompt for compute node at "pelican_setup".

Many thanks to Michael Creel for his quick replies, I got the whole thing up in 2 days!

[Michael Creel]

Thanks a lot for taking the time to write this. I'm going to put a link to this thread in the documentation section of the homepage.

In principle, you can build a Pelican image on any Linux distro, you just need to install live-helper using the source package, rather than the .deb. I have only used Debian and Ubuntu myself, but there's no reason why you can't use other distros. The additional requirements are wget, rsync, and debootstrap. I am pretty sure that subversion is not required.

[genechng]

Your welcome, this is for my future reference too! Thank you for your quick replies!! I will change the post to reflect your reply. I'm on my way to building great parallel applications!

Many thanks
Eugene

[Michael Creel]

I had a look at the opennature.org site, and I see that you are doing some very interesting scientific work. The Shift-Life app is currently running in another tab of my browser, evolving in a pretty interesting way. I can see how MPI might be useful to you. I how you find PelicanHPC to be useful. Let me know if you have any problems.
M.

[genechng]

Hi Michael,

Thanks for your comments, I'll be using PelicanHPC to do mainly Ecological Modelling with Vegetation. We're mapping the Palaeo-environment of Doggerland (North Sea basin) and there might be a big project next year. A few clusters definitely isn't enough!!

In the future, I'm moving on to modelling vagile life-forms (issues with climate change and invasive species). I shall certainly contact you if I come across any problems.

Many thanks
Eugene

[youscoul]

Hello,

I am using Pelican to Build a cluster for medical pictures. for that, i use Virtualbox or CDRom Bootable. My question is: when my application turn on on my cluster, what i must to do to store my data in the local hard disks ?

I need your help

Thanks

Best regards to you

[genechng]

Hi there,

For virtualbox, you could mount a shared folder using the command:

sudo mount -t vboxsf vbubuntu shared/

vbubuntu is the name of your virtualbox shared folder, and 'shared' is the folder name you create in your linux file systems.

For a physical setup, you'll have the internet working, I generally just save files from the internet or copy & paste from a code from google docs. You could also enable a USB drive, look around the forum on how to do this.

Regards
Eugene

[youscoul]

Thanks for your reply,

I don't working on ubuntu OS. I am using virtualbox on Windows XP and i run my frontend node there. I have access to network. My sources code are on windows partition so i want to compile & run it on cluster(virtual frontend and 4 physical nodes). For that, first i don't how access my sources folders from local disk and how to store my datas after run my programs.


Regards
Youscoul

[genechng]

Hi Youscoul,

Since you have the internet connection, would sending your files to an email and then downloading it be easier?

Eugene

[youscoul]

Hi,

I had discute with Michael, he explains me how to store data. Your idea is neat too but i do a very important work, and my data doesn't has be sending by internet. It's for medical pictures. I will try to create a partition ext2 on my frontend node that already has windows xp, and i will give that new partition to pelican for mounted in /home directory. As the virtual has limited, i decided to use pelican from bootable CDRom and i won't use virtualbox in that case. After, i will post all my configurations for you.

Best regards,

Youscoul

[genechng]

Hey Youscoul,

Excellent! It is much more convenient that way too. I generally work on virtualbox to get everything sorted prior to running the algorithms on a physical machine.

Good luck

Eugene

[youscoul]

Hi Eugene,

If i want to make my own pelican live CD adding Drivers packages for my NIC, Is that's possible ? then where i have to add those packages ?
My compute node has four cores, and Ethernet controller: Intel Corporation 82567LM-3 Gigabit Network Connection, i need to install its driver, so could you help me please ?

Best regards

Youscoul

[genechng]

Hi Youscoul

You'll have to find out the package name for your NIC. And add it to the package list in "make_pelican-v2.1". There are instructions in the tutorial above under the heading "Make Pelican Prerequisites". Instructions are also available in Michael's tutorials.

All the best
Eugene

[Michael Creel]

Unfortunately, this NIC isn't supported by the kernel used in Debian Lenny, so it's not just a matter of adding a package. This NIC is supported in newer kernels (e.g., in Debian squeeze and sid) but unfortunately, make_pelican is not currently working with squeeze or sid. So, the easiest solution is to switch the NIC.

[youscoul]

Hi,
I try to build my own pelican liveCD on ubuntu 9.10.
I had installed live-helper. And my ubuntu already has latest versions of debootstrap, rsync and wget.
 shell: dpkg -i live-helper XXXX is doing and ok.
but when I type "sh make_pelican xx" to build, after  7 minutes i have messages errors such as :
P: Begin unmounting filesystems...
umount: proc-live: n'a pas été trouvé
umount: sysfs-live: n'a pas été trouvé
umount: devpts-live: n'a pas été trouvé

So what means that ?
Best regards to you,

Youscoul

[Michael Creel]

Those aren't errors, those are the normal messages to see when a build is finished. In the directory where you did this, you should find a directory amd64 or i386, depending on which architecture you used. In that directory, there will be a directory "frontend", which should have a file "binary.iso". That is your new CD image. If it is not there, then a problem occurred.

[youscoul]

Yes,

I have a directory amd64 and fronted in it. But no binary file is available, i have just  this: amd64/frontend/auto, cache, chroot, and config directory.

Cheers,
Youscoul

[Michael Creel]

OK, then you should scroll up in the terminal to find an error message, to see what is the problem. Which version of live-helper are you using?

[youscoul]

I am using, live-helper_2.0~a16-1_all.deb.

[youscoul]

This is another message error:
rsync: change_dir "/home/ycoulibaly/pelicanhome" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1057) [sender=3.0.6]
rsync: change_dir "/home/ycoulibaly/packages" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 2

Did  anyone has the same error building its own pelicanlive.

Thank