PARCEL SYCL v1.2 / API Reference User Guide API Reference Search Download 日本語

18 C API

From C, include capi/parcel_sycl_c.h and call the functions in this chapter. They are implemented in capi/wrapper_parcel_sycl.cpp (the wrapper), which runs the same solvers of the library. The meaning and defaults of the settings, the results, and the convergence states are given in Common Items and Convergence by Solver; the workflow and examples in “C” in the “Usage” chapter of the User Guide.

18.1 Mapping to the Common Items

The common settings correspond to the fields of parcel_options, and the results to the fields of parcel_result, as follows. The defaults and meanings are those of the common items. Differences in how a setting is expressed are shown in the right column.

Common setting Field of parcel_options Difference in expression
Convergence tolerance tolerance —
Maximum number of iterations max_iteration —
Preconditioner type precon_algo —
Number of Jacobi iterations jacobi_iter —
Number of block Jacobi blocks blockjacobi 0 means automatic
Scaling scaling_matrix, scaling_vector -1: automatic, 0: off, 1: on
Restart length (pgmres) gmres_restart —
Orthogonalization method (pgmres) gmres_qr_algo —
Basis generation size ss, tt cagmres_s, cagmres_t —
QR factorization method (pcagmres) cagmres_qr_algo The field default is PARCEL_QR_MGS. With IMPLICIT, Cholesky QR is used if the field is left at this default
Q mode cagmres_q_mode —
Spectrum of the preconditioned matrix precon_spectrum 0: false, 1: true
Type of the inner linear solver linsolve_algo Combination with precon_spectrum = 1 returns PARCEL_ERR_ARG
Tolerance of the inner linear solver linsolve_tolerance 0 or less means the solver’s default (field default 0)
Maximum iterations of the inner linear solver linsolve_max_iteration Same as above
Restart length of the inner linear solver linsolve_restart Same as above
Preconditioner of the inner linear solver linsolve_precon_algo, linsolve_jacobi_iter, linsolve_blockjacobi 0 blocks means automatic. The scaling cannot be specified (always automatic)
Shift-invert shift_enable, shift_sigma Enabled with shift_enable = 1. The wrapper assembles A−σIA - \sigma I
Size of the Krylov subspace ks_m, ks_k_keep, ks_nev —
Maximum number of restarts ks_max_restart —
Orthogonalization method (ks_arnoldi) ks_qr_algo —
Initial vector eigvec_seed and the first n_local elements of eigvec With eigvec_seed = 1, eigvec is used as the initial vector
Common result Field of parcel_result
Convergence state converged_reason
Iteration count iterations
Residual residual
True residual true_residual
Number of converged eigenpairs ks_nconv
Number of inner solver failures linsolve_failures

The common constants have names prefixed with PARCEL_ (PARCEL_PRECON_ for the preconditioners, PARCEL_QR_ for the QR factorizations, PARCEL_LINSOLVE_ for the inner linear solvers, PARCEL_CAGMRES_Q_ for the Q modes, and PARCEL_CONVERGED_RTOL etc. for the convergence states; constants).

18.2 Constants

The solver is specified with the following constants.

Constant Value Solver Function to call
PARCEL_SOLVER_PCG 0 pcg parcel_solve_linear
PARCEL_SOLVER_PBICGSTAB 1 pbicgstab same
PARCEL_SOLVER_PGMRES 2 pgmres same
PARCEL_SOLVER_PCAGMRES 3 pcagmres same
PARCEL_SOLVER_POWER 4 power (one eigenvalue of largest magnitude) parcel_solve_eigen
PARCEL_SOLVER_INVERSE_POWER 5 inverse_power (one eigenvalue of smallest magnitude) same
PARCEL_SOLVER_KS_ARNOLDI 6 ks_arnoldi (largest-magnitude side; near σ in shift-invert mode) same

The values of the settings are specified with the following constants (meanings in the common constants).

Constants Values Fields
PARCEL_PRECON_NONE, PARCEL_PRECON_POINTJACOBI, PARCEL_PRECON_BLOCKJACOBI, PARCEL_PRECON_JACOBI 0, 1, 2, 4 precon_algo, linsolve_precon_algo
PARCEL_QR_CGS, PARCEL_QR_MGS, PARCEL_QR_CHOLESKYQR, PARCEL_QR_CAGMRES_CHOLESKYQR 0, 1, 2, 202 gmres_qr_algo and ks_qr_algo (CGS and MGS only), cagmres_qr_algo
PARCEL_LINSOLVE_PGMRES, PARCEL_LINSOLVE_PCG 0, 1 linsolve_algo
PARCEL_CAGMRES_Q_AUTO, PARCEL_CAGMRES_Q_EXPLICIT, PARCEL_CAGMRES_Q_IMPLICIT 0, 1, 2 cagmres_q_mode

The return values of the functions are as follows. The results of the argument checks are made consistent on all ranks with MPI_Allreduce before returning, so even if only one rank is invalid, all ranks return the same error.

Return value Value Meaning
PARCEL_OK 0 Completed normally. Whether the iteration converged is not included (check converged_reason of parcel_result)
PARCEL_ERR_ARG -1 Invalid argument or setting
PARCEL_ERR_DISTRIBUTION -2 The row blocks of the ranks do not cover the whole matrix without overlaps or gaps (requirements of parcel_solve_linear())
PARCEL_ERR_NOT_BUILT -3 The specified solver was excluded from the build (see “Selecting the Solvers to Build”)
PARCEL_ERR_RUNTIME -4 A failure inside the library. In a build with MPI it is not returned to the caller (below)

When a failure occurs inside the library (insufficient device memory, a matrix whose size exceeds the range of the library’s internal integers, etc.), the wrapper writes a message ([parcel_c] exception: ...) to standard error and then calls MPI_Abort(comm, 4) to terminate all ranks. Such a failure can occur on only one rank; if PARCEL_ERR_RUNTIME were returned and left to the caller, the other ranks would keep waiting in the collective communication inside the solver and the job would hang. In a build without MPI there is nobody to wait, so the wrapper writes the message and returns PARCEL_ERR_RUNTIME.

The convergence state (converged_reason of parcel_result) is expressed with the following constants. Their meanings and the cases in which they occur for each solver are given in Convergence Status in Common Items.

Constant Value Meaning
PARCEL_CONVERGED_RTOL 1 The convergence tolerance (tolerance) was satisfied
PARCEL_ITERATING 0 Not run (e.g., not solved because of an argument error)
PARCEL_DIVERGED_ITS -1 The maximum number of iterations (for ks_arnoldi, the maximum number of restarts) was reached
PARCEL_DIVERGED_NANORINF -2 NaN / Inf appeared in the checked value (residual or eigenvalue estimate)
PARCEL_DIVERGED_BREAKDOWN -3 Breakdown of the algorithm

18.3 parcel_options

Description Structure that holds the settings of the solver, preconditioner, convergence check, etc.

Synopsis

typedef struct parcel_options { double tolerance; ... } parcel_options;

Fields The table gives the defaults set by parcel_options_init(). The meanings are those of the common settings. For the inner linear solver, linsolve_tolerance, linsolve_max_iteration and linsolve_restart are set to 0 (0 or less means “use the solver’s default”), because the default maximum number of iterations differs between inverse_power (5000) and ks_arnoldi (2000) and cannot be expressed by one field. Fields used only by solvers excluded from the build remain 0.

Field Default Description
double tolerance 1.0e-8 Convergence tolerance (relative residual for the linear solvers, relative change of the eigenvalue for power / inverse_power, Ritz residual for ks_arnoldi)
double linsolve_tolerance 0 Tolerance of the inner linear solver of the eigenvalue solvers. 0 or less means the solver’s default (1.0e-10 for both inverse_power and ks_arnoldi)
double shift_sigma 0.0 Shift σ of shift-invert (when shift_enable=1)
int max_iteration 10000 Maximum number of iterations (linear solvers, outer iterations of power / inverse_power; not used by ks_arnoldi)
int precon_algo PARCEL_PRECON_NONE Preconditioner type KK: PARCEL_PRECON_NONE (0), _POINTJACOBI (1), _BLOCKJACOBI (2), _JACOBI (4)
int jacobi_iter 1 Number of Jacobi iterations
int blockjacobi 0 Number of block Jacobi blocks. 0 means automatic
int scaling_matrix -1 Scaling of the preconditioner matrix. -1: automatic, 0: off, 1: on
int scaling_vector -1 Scaling of the preconditioner vectors (same values)
int index_base 0 Index base of nstart and col (0 or 1)
int verbose 0 With 1, rank 0 prints the device name and the settings
int gmres_restart 30 Restart length mm of pgmres
int gmres_qr_algo PARCEL_QR_MGS Orthogonalization method of pgmres: PARCEL_QR_CGS (0) or _MGS (1). Others (including _CHOLESKYQR (2)) return PARCEL_ERR_ARG
int cagmres_s 8 ss of pcagmres
int cagmres_t 4 tt of pcagmres (the restart length is s×ts \times t)
int cagmres_qr_algo PARCEL_QR_MGS QR factorization method of pcagmres: PARCEL_QR_CGS (0), _MGS (1), _CHOLESKYQR (2), _CAGMRES_CHOLESKYQR (202; only with cagmres_q_mode = PARCEL_CAGMRES_Q_IMPLICIT)
int precon_spectrum 0 Spectrum of the preconditioned matrix, common to the eigenvalue solvers. 0: eigenvalues of AA, 1: eigenvalues of AK−1AK^{-1} (KK is built from precon_algo etc.)
int linsolve_algo PARCEL_LINSOLVE_PGMRES Type of the inner linear solver of inverse_power: PARCEL_LINSOLVE_PGMRES (0) or PARCEL_LINSOLVE_PCG (1)
int linsolve_max_iteration 0 Maximum iterations of the inner linear solver. 0 or less means the solver’s default (5000 for inverse_power, 2000 for shift-invert of ks_arnoldi)
int linsolve_restart 0 Restart length of the inner pgmres. 0 or less means the solver’s default (30 for both)
int linsolve_precon_algo PARCEL_PRECON_JACOBI Type of the preconditioner of the inner linear solver KinK_{in}. Not used with precon_spectrum=1
int linsolve_jacobi_iter 1 Number of Jacobi iterations of KinK_{in}
int linsolve_blockjacobi 0 Number of block Jacobi blocks of KinK_{in} (0 means automatic)
int ks_m 30 Maximum dimension of the Krylov subspace mm of ks_arnoldi
int ks_k_keep 10 Number of Ritz pairs kkeepk_{keep} kept at a restart of ks_arnoldi
int ks_nev 4 Number of eigenvalues nevnev computed by ks_arnoldi
int ks_max_restart 200 Maximum number of restarts of ks_arnoldi
int shift_enable 0 With 1, shift-invert (eigenvalues near σ). ks_arnoldi only
int eigvec_seed 0 With 1, the first n_local elements of eigvec are used as the initial vector of ks_arnoldi. With 0, the wrapper builds it from the global row numbers
int cagmres_q_mode PARCEL_CAGMRES_Q_AUTO Q mode of pcagmres: PARCEL_CAGMRES_Q_AUTO (0), _EXPLICIT (1), _IMPLICIT (2). The combination rules are those of the common items; a combination violating them and values other than 0 to 2 return PARCEL_ERR_ARG. With _IMPLICIT, if cagmres_qr_algo is left at the default PARCEL_QR_MGS, PARCEL_QR_CHOLESKYQR is used, and PARCEL_QR_CGS returns PARCEL_ERR_ARG
int ks_qr_algo PARCEL_QR_CGS (the build-time default of the library) Orthogonalization method of ks_arnoldi: PARCEL_QR_CGS (0, CGS2) or PARCEL_QR_MGS (1, MGS2). Other values give PARCEL_ERR_ARG. The default is read from the library by parcel_options_init()

Requirements Always fill in the defaults with parcel_options_init() and then change only the necessary fields. Fields not used by the called solver are ignored. However, the range of tolerance (finite positive), max_iteration (1 or greater) and the preconditioner fields (precon_algo (one of the constants), jacobi_iter (1 or greater), blockjacobi (0 or greater), scaling_matrix and scaling_vector (-1, 0, 1)) is always checked, and that of linsolve_algo, linsolve_precon_algo, linsolve_jacobi_iter, linsolve_blockjacobi and linsolve_tolerance (not NaN) is always checked for the eigenvalue solvers; PARCEL_ERR_ARG is returned if a value is out of range. A value of 0 or less for linsolve_tolerance, linsolve_max_iteration and linsolve_restart means “the solver’s default”. The size of the structure may change between versions, so build the wrapper and the Fortran module with parcel_sycl_c.h of the same version.

MPI Give the same contents on all ranks.

See Also parcel_options_init(), Common settings

18.4 parcel_result

Description Structure that receives the convergence status of a solve.

Synopsis

typedef struct parcel_result {
  int    converged_reason;
  int    iterations;
  double residual;
  double true_residual;
  int    ks_nconv;
  int    linsolve_failures;
} parcel_result;

Fields The meanings are those of Results in Common Items, and the definitions differ between solvers (Meaning of the Iteration Count and Residuals). The values are the same on all ranks. If the solve was not done because of an argument error, converged_reason = PARCEL_ITERATING (0), iterations = 0, and the residuals are NaN.

Field Description
int converged_reason Convergence state. Positive means converged, negative means failure (constants)
int iterations Iteration count. Total inner iterations for pgmres, total basis vectors for pcagmres, outer iterations for power / inverse_power, restarts for ks_arnoldi
double residual Residual: the relative value compared with tolerance. ∥r∥/∥b∥\|r\|/\|b\| for the linear solvers (recurrence for pcg / pbicgstab, true residual for pgmres / pcagmres), relative change of the eigenvalue for power / inverse_power, maximum of the Ritz residual estimates of the ks_nev target pairs for ks_arnoldi. NaN if not computed
double true_residual True residual: ∥b−Ax∥/∥b∥\|b - Ax\|/\|b\| at the end of a linear solver. NaN for the eigenvalue solvers
int ks_nconv Number of converged eigenpairs (out of ks_nev). 0 for other solvers
int linsolve_failures Number of inner solver failures (inverse_power and shift-invert of ks_arnoldi). 0 otherwise

See Also parcel_solve_linear(), parcel_solve_eigen(), convergence state constants

18.5 parcel_options_init()

Description Fills parcel_options with the defaults.

Synopsis

void parcel_options_init(parcel_options *opt);

Arguments

Name In/Out Type Description Default
opt Out parcel_options * Structure to fill with the defaults —

Requirements opt is not NULL.

Return Value None. The defaults are read from solver objects of the library, so they agree with the common defaults (table of parcel_options).

MPI Local operation.

Example

parcel_options opt;
parcel_options_init(&opt);
opt.tolerance = 1.0e-9;

18.6 parcel_solver_from_name()

Description Returns the value of PARCEL_SOLVER_* for a solver name.

Synopsis

int parcel_solver_from_name(const char *name);

Arguments

Name In/Out Type Description Default
name In const char * Solver name ("pcg", "pbicgstab", "pgmres", "pcagmres", "power", "inverse_power", "ks_arnoldi") —

Return Value The corresponding value of PARCEL_SOLVER_*; -1 for an unknown name.

MPI Local operation.

Notes Not included in the Fortran module.

18.7 parcel_eigen_count()

Description Returns the number k of eigenpairs returned by an eigenvalue solver. Used to size the arrays passed to parcel_solve_eigen().

Synopsis

int parcel_eigen_count(int solver, const parcel_options *opt);

Arguments

Name In/Out Type Description Default
solver In int PARCEL_SOLVER_* —
opt In const parcel_options * Settings (ks_arnoldi uses ks_k_keep) —

Return Value 1 for power / inverse_power, opt->ks_k_keep for ks_arnoldi (of which only the first ks_nev are guaranteed to converge), 0 for the linear solvers and invalid values.

MPI Local operation.

See Also parcel_solve_eigen()

18.8 parcel_solve_linear()

Description Solves the linear system Ax=bAx = b (pcg, pbicgstab, pgmres, pcagmres).

Synopsis

int parcel_solve_linear(int solver, const parcel_options *opt, PARCEL_C_COMM comm,
                        parcel_int n_local, parcel_int nstart, parcel_int n_global, parcel_int nnz_local,
                        const parcel_int *rowptr, const parcel_int *col, const double *val,
                        const double *b, double *x, parcel_result *res);

Arguments

Name In/Out Type Description Default
solver In int One of PARCEL_SOLVER_PCG, _PBICGSTAB, _PGMRES, _PCAGMRES. Others return PARCEL_ERR_ARG —
opt In const parcel_options * Settings. Commonly used fields are tolerance, max_iteration and the preconditioner fields. pgmres uses gmres_restart and gmres_qr_algo; pcagmres uses cagmres_s, cagmres_t, cagmres_qr_algo and cagmres_q_mode —
comm In PARCEL_C_COMM Communicator (MPI_Comm with MPI; communicators other than MPI_COMM_WORLD are allowed. int 0 without MPI) —
b In const double * Owned part of the right-hand side (length n_local) —
x In/Out double * Initial guess on input, owned part of the solution on output (length n_local) —
res Out parcel_result * Results. Not returned if NULL —

The matrix arguments are as follows (common with parcel_solve_eigen()).

Argument Description
parcel_int n_local Number of owned rows (1 or greater)
parcel_int nstart Global row number of the first owned row (in the index_base convention)
parcel_int n_global Global number of rows (sum of n_local over all ranks)
parcel_int nnz_local Number of nonzeros in the owned rows
const parcel_int *rowptr Row pointer (length n_local + 1). Only differences from rowptr[0] are used, so it may be 0-based or 1-based
const parcel_int *col Column indices (length nnz_local), global (in the index_base convention)
const double *val Matrix elements (length nnz_local)

Requirements The matrix satisfies the conditions of Input Matrix in Common Items.

Return Value PARCEL_OK or a negative error code (constants). Whether the iteration converged is not included in the return value; check converged_reason of res.

MPI Collective operation. Call on all ranks of the communicator with the same solver and the same contents of opt. The progress and computing times are printed to the standard output of rank 0 (Standard Output in Common Items).

See Also parcel_options, parcel_result, Convergence by Solver

Example

parcel_options opt;
parcel_options_init(&opt);
opt.tolerance   = 1.0e-9;
opt.precon_algo = PARCEL_PRECON_BLOCKJACOBI;
parcel_result res;
int rc = parcel_solve_linear(PARCEL_SOLVER_PGMRES, &opt, MPI_COMM_WORLD,
                             n_local, nstart, n_global, nnz_local, rowptr, col, val, b, x, &res);
if(rc != PARCEL_OK){ /* argument error etc. */ }
else if(res.converged_reason <= 0){ /* not converged (PARCEL_DIVERGED_*) */ }

18.9 parcel_solve_eigen()

Description Solves the eigenvalue problem Av=λvAv = \lambda v (AK−1AK^{-1} with precon_spectrum=1) (power, inverse_power, ks_arnoldi).

Synopsis

int parcel_solve_eigen(int solver, const parcel_options *opt, PARCEL_C_COMM comm,
                       parcel_int n_local, parcel_int nstart, parcel_int n_global, parcel_int nnz_local,
                       const parcel_int *rowptr, const parcel_int *col, const double *val,
                       double *eigval, double *eigvec, parcel_result *res);

Arguments The matrix arguments, opt, comm and res are the same as in parcel_solve_linear(). With k = parcel_eigen_count():

Name In/Out Type Description Default
solver In int One of PARCEL_SOLVER_POWER, _INVERSE_POWER, _KS_ARNOLDI —
eigval Out double * 2*k elements. Real and imaginary parts alternate (eigval[2*j] is the real part of the j-th eigenvalue, eigval[2*j+1] the imaginary part). For power / inverse_power, eigval[0] holds the eigenvalue and eigval[1] is 0 —
eigvec In/Out double * n_local*k elements, column-major: the n_local elements from eigvec + j*n_local are the owned part of the j-th eigenvector. A complex conjugate pair is stored as two vectors, real part and imaginary part (Output of Eigenvalues and Eigenvectors). For ks_arnoldi with eigvec_seed=1, the first n_local elements are the initial vector (input) —

The eigenvalues computed and the main fields used are as follows.

Solver Computes Main fields
power One eigenvalue of largest magnitude tolerance, max_iteration, precon_spectrum
inverse_power One eigenvalue of smallest magnitude The above + linsolve_algo, linsolve_tolerance, linsolve_max_iteration, linsolve_restart, linsolve_precon_*
ks_arnoldi ks_nev eigenvalues on the largest-magnitude side (near σ in shift-invert mode) tolerance, precon_spectrum, ks_m, ks_k_keep, ks_nev, ks_max_restart, ks_qr_algo, shift_enable, shift_sigma, eigvec_seed, and linsolve_* in shift-invert mode

Requirements

Return Value Same as parcel_solve_linear(). For inverse_power and shift-invert, a nonzero res->linsolve_failures means that the inner linear solver did not converge in some iterations. res->ks_nconv of ks_arnoldi is the number of eigenpairs that satisfied the tolerance.

MPI Collective operation (same as parcel_solve_linear()).

See Also parcel_eigen_count(), Notes on the Eigenvalue Solvers, Convergence by Solver

Example

const int k = parcel_eigen_count(PARCEL_SOLVER_KS_ARNOLDI, &opt);   /* = ks_k_keep */
double *ev = malloc(sizeof(double) * 2 * k);                         /* Re, Im alternating */
double *v  = malloc(sizeof(double) * (size_t)n_local * k);           /* column-major */
int rc = parcel_solve_eigen(PARCEL_SOLVER_KS_ARNOLDI, &opt, MPI_COMM_WORLD,
                            n_local, nstart, n_global, nnz_local, rowptr, col, val, ev, v, &res);

18.10 parcel_options_size() / parcel_result_size()

Description Returns sizeof(parcel_options) and sizeof(parcel_result). Used to check the Fortran derived types.

Synopsis

int parcel_options_size(void);
int parcel_result_size(void);

Return Value The sizes of the structures [bytes] when the wrapper was built.

Notes The sizes of the structures may change between versions, so build the wrapper, the Fortran module and user programs with parcel_sycl_c.h of the same version.

18.11 parcel_get_version() / parcel_version_string()

Description Returns the version of the library. The values are those of Include/parcel_version.hpp when the wrapper was built.

Synopsis

int parcel_get_version(int *major, int *minor, int *patch);
const char *parcel_version_string(void);

Arguments major, minor, patch: Out. Version numbers. Arguments given as NULL are not written.

Return Value parcel_get_version() returns PARCEL_OK. parcel_version_string() returns a string of the form "1.2.0" (static storage; do not free it).

MPI Local operation. Can be called before MPI is initialized and in builds without MPI.

18.12 run_parcel_sycl_c() (deprecated)

Description The C API of v1.0 (legacy API). Solves Ax=bAx = b with pcg. Kept for compatibility. For new programs, use parcel_solve_linear().

Synopsis

void run_parcel_sycl_c(PARCEL_C_COMM mpi_comm_c, parcel_int n, parcel_int nnz,
                       double *crsA, parcel_int *crsCol, parcel_int *crsRow_ptr,
                       double *vecx, double *vecb);

Arguments

Name In/Out Type Description Default
mpi_comm_c In PARCEL_C_COMM Communicator —
n In parcel_int Number of owned rows —
nnz In parcel_int Number of nonzeros in the owned rows —
crsA In double * Matrix elements —
crsCol In parcel_int * Global column indices (0-based) —
crsRow_ptr In parcel_int * Row pointer (0-based) —
vecx In/Out double * Initial guess on input, solution on output —
vecb In double * Right-hand side —

Requirements The first row of each rank’s range is obtained as the prefix sum (MPI_Exscan) of n over the ranks. If n is equal on all ranks this gives n*myrank, and uneven partitions can also be handled. The input arrays are not modified.

Return Value None. On failure, rank 0 prints the error code to standard error.

MPI Collective operation.

Notes The convergence tolerance is fixed at 1.0e-9, the maximum number of iterations is the wrapper’s compile-time macro MAX_ITERATION (default 10000), and the preconditioner is determined by -Dparcel_solver_precon_algo=<0|1|2|4>. For block Jacobi (2) and Jacobi (4), matrix and vector scaling are enabled, and BLOCKJACOBI_BLOCK (default 1) and JACOBI_ITER (default 1) are used for the number of blocks and the number of iterations. The integer arguments changed from int in v1.0 to parcel_int (64 bits) (“Versions and Compatibility” in the User Guide).