NOTE: BEWARE OF SEMAPHORE EXHAUSTION! >>>
(ERROR: p4_error semget failed for setnum: 0)

Migration to the new AMD Opteron cluster

Extensive upgrade of the software environment of NNIN/C has recently been completed and applied to the second AMD Opteron cluster which comprise BladeCenter chassis’s 3 and 4. For the past month NNIN/C jobs have been restricted to chassis’s 1 and 2 while the upgrade has proceeded. Beginning on Wednesday, October 24, 2007, access to the first cluster will be curtailed and all users are requested to begin running jobs on the new and improved cluster #2.

The new cluster provides the following functionality:

• Condor for parallel jobs. The parallel universe for the condor scheduler requires a wrapper script, called condor-mpirun, to handle the arguments passed in a normal parallel environment to mpirun but, with a scheduler, passed to condor_submit. This allows condor to allocate the required number of processors and to initialize the starting directory, among other functions. The condor-mpirun script has been developed for the electronic structure code VASP, but should be generally applicable to all parallel codes. A complete description of the location of this script and the way to use it will be documented in the days that follow.
• Intel compilers including intel 10.0 as well as the previous version, 9.1. Version 9.1 will remain the default until tests of the 10.0 compilation are completed.
• Python 2.5 compiler.
• AMD core math library (acml). This package, documented in http://developer.amd.com/acml3.jsp#docs, contains LAPACK, BLAS level 1,2,3 and some FFT routines optimized for the 64 bit AMD Opteron. This library should provide performance enhancements over the standard Intel math kernel library (mkl).
• MPI implementations: mpich, mpich2 and openmpi. Open-MPI is the preferred implementation, but mpich may be required for some applications.