ESPRESO is an ExaScale PaRallel FETI SOlver developed at IT4Innovations. Main focus of the development team is to create a highly efficient parallel solver which contains several FETI based algorithms including new Hybrid Total FETI method suitable for parallel machines with tens or hundreds of thousands of CPU cores. The solver is based on highly efficient communication layer on top of pure MPI. The layer was developed specifically for FETI solvers and uses several state-of-the-art communication hiding and avoiding techniques to achieve better scalability.
ESPRESO is implemented using C++ and it is compiler and MPI implementation independent, even though the Intel compiler is the preferred choice. The only dependency is the Intel MKL library (Sparse and Dense BLAS routines and MKL version of PARDISO sparse solver). Solver uses three levels of parallelization: 1.) message passing on node level, 2) threading inside a node using Cilk++ and 3.) vectorization using Intel MKL library or Cilk++. Intel Xeon Phi accelerators are also natively supported.
The current version has been successfully tested together with PermonCube problem generator on the HLRS Hermit Cray XE6 machine running on up to 9261 CPU cores and successfully solving problem of size over 1 billion of unknowns. Another large scale benchmarks were executed on SurfSara Cartesius supercomputer, where 1.6 billion unknown problem was decomposed into 8000 sub-domains and solved using the same number of CPU cores. All these large scale tests were performed using FETI method and it is expected, that using Hybrid Total FETI algorithm the solver will scale to tens or hundreds of thousands of cores using MPI + OpenMP hybrid parallelization technique.