PARCEL 1.01(May 20, 2020)

PARCEL 1.01 is available.

The following points have been corrected in this update.

  • 1. Error handling is added for zero diagonal elements in the D-ILU preconditioner.
  • 2. Memory allocation in the D-ILU preconditioner is changed to from static allocation to dynamic allocation.
  • 3. The additive Schwarz preconditioner with zero overlap width is unified with the block Jacobi preconditioner.
  • 4. A configuration file for the PGI compiler is added.
  • 5. Sample programs are modified.
    • ・A sorting method in the data conversion routine from CRS to DIA is improved.
    • ・Template files for main algorithms are stored in each example.
    • ・The names of input parameters in each example are changed to common names.
    • ・The r.h.s. vector vecB in each example is changed to the same data.
    • ・ExampleC, 2_C are modified for DIA in addition to CSR.
    • ・All nonsymmetric solvers become available in example2.


Source code



Japanese version


English version



The source code is licensed under the GNU Lesser General Public License(LGPL)

Old Version

Click Here

Purpose and Overview of the Program Development

Matrix solvers for simultaneous linear equation systems are classified into direct and iterative solvers. In most of extreme scale problems, iterative solvers based on Krylov subspace methods are essential from the viewpoints of the computational cost and the memory usage.
The PARallel Computing ELements (PARCEL) library provides highly efficient parallel Krylov subspace solvers for modern massively parallel supercomputers, which are characterized by accelerated computation and less performance improvement in inter-node communication.
The PARCEL is based on a hybrid parallel programming model with MPI+OpenMP, and supports the latest communication avoiding (CA) Krylov methods (the Chebyshev basis CACG method and the CA-GMRES method) in addition to the conventional Krylov subspace methods (the CG method, the BiCGstab method, and the GMRES method).
In the CA-Krylov methods, the number of collective communication is reduced to avoid a communication bottleneck.

The PARCEL supports two matrix formats (Compressed Row Storage (CRS) format and Diagonal (DIA) format), two data types (Double precision and Quad precision), and can be called from programs written in C and Fortran.

Main Features

≪Performance comparison of CG solvers with Block-Jacobi preconditioner using 2,304 cores on the JAEA-ICEX (Intel Xeon E5-2680 v3, 4xFDR Infiniband)≫
Three dimensional Poisson equation with the problem size of 1152 x 1152 x 1152.



Elapse time[s]



Memory usage[GB]




[1]Barret, R. et al.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM(1994)
[2]Mayumi, A., Idomura, Y., Ina, T., Yamada, S., Imamura, T.,: Left-preconditioned communication-avoiding conjugate gradient methods for multiphase CFD simulations on the K computer, In: Proceedings of the 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ScalA 2016, Piscataway, NJ, USA, pp.?17?24. IEEE Press(2016)
[3]Idomura, Y., Ina, T., Mayumi, A., Yamada, S., Matsumoto, K., Asahi, Y., Imamura, T.: Application of a communication-avoiding generalized minimal residual method to a gyrokinetic five dimensional Eulerian code on many core platforms, In:Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ScalA 2017, New York, NY, USA, pp.?7:1?7:8. ACM(2017)
[4]Idomura, Y., Ina, T., Mayumi, A., Yamada, S., Imamura, T.: Application of a preconditioned Chebyshev basis communication-avoiding conjugate gradient method to a multiphase thermal-hydraulic CFD code, Lecture Notes in Computer Science, vol.?10776, 257?273(2018)
[5]Hoemmen, M.: Communication-avoiding Krylov subspace methods. Ph.D.thesis, University of California, Berkeley(2010)
[6]Suda, R., Cong, L., Watanabe, D., Kumagai, Y., Fujii, A., Tanaka, T.: Communication-avoiding CG method: new direction of Krylov subspace methods towards exa-scale computing. RIMS Kokyuroku 1995, 102?111(2016)


Computer Science Research and Development Office, Center for Computational Science & e-Systems, Japan Atomic Energy Agency


※ Substitute @ for (at).