This week at the SC07 conference, we won the HPC Challenge. Judged by a committee led by Jack Dongarra (University of Tennessee, and Oak Ridge National Lab) and Jeremy Kepner (MIT Lincoln Lab), the goal of the competition is to focus the HPC community’s attention on developing a broad set of HPC hardware and HPC software capabilities that are necessary to productively use HPC systems.
Our entry this year was done in the Python language, extended to a parallel environment with Star-P. We won in the category of "Most Productivity" - based 50% on performance, and 50% on code elegance, clarity, and size.
Python is a high-level, dynamically typed, multi-paradigm (procedural, objectoriented and functional) interpreted language created by Guido Van Russom. Natively, Python does not have data types and containers such as matrices and lacks linear algebra. and signal processing functions. Instead, these are added to the language through Python extension modules. Currently, the de facto Python module for numerical computing is NumPy authored by Travis Oliphant and others.
The Star-P package in Python is an extension module that can be imported just like any of the modules in the Python standard library. The syntax and semantics in Star-P Python closely model those in NumPy.
The basic premise of the model is to maintain compatibility in syntax with serial codes written using the NumPy module. In most cases, the user must not be burdened with having to think “in parallel”, keep track of distributions or worry about which portions of the code runs in serial and which in parallel. This allows users with a large existing serial application to port it to run in parallel with the least amount of effort.
The tests were run at a high performance cluster at the San Diego Supercomputing Center. The cluster consists of 32 nodes, where each node contains one quad-core Intel Xeon 5140 processor and 8 GB of memory, for a total of 128 cores and 256 GB of memory. The interconnect is IP over Infiniband.
Here are the scaling results for 3 of benchmarks. The full submission and presentation can be found here.
