PARCEL SYCL v1.2 / API Reference User Guide API Reference Search Download 日本語

19 Fortran API

From Fortran, use the module parcel_sycl_mod (capi/parcel_sycl_mod.f90) and call the procedures in this chapter. They are implemented by the same wrapper as the C API (capi/wrapper_parcel_sycl.cpp), which runs the same solvers of the library. The meaning and defaults of the settings, the results, and the convergence states are given in Common Items and Convergence by Solver; the workflow and examples in “Fortran” in the “Usage” chapter of the User Guide.

19.1 Mapping to the Common Items

The common settings correspond to the components of parcel_options, and the results to the components of parcel_result, as follows. The defaults and meanings are those of the common items. Differences in how a setting is expressed are shown in the right column.

Common setting Component of parcel_options Difference in expression
Convergence tolerance tolerance —
Maximum number of iterations max_iteration —
Preconditioner type precon_algo —
Number of Jacobi iterations jacobi_iter —
Number of block Jacobi blocks blockjacobi 0 means automatic
Scaling scaling_matrix, scaling_vector -1: automatic, 0: off, 1: on
Restart length (pgmres) gmres_restart —
Orthogonalization method (pgmres) gmres_qr_algo —
Basis generation size ss, tt cagmres_s, cagmres_t —
QR factorization method (pcagmres) cagmres_qr_algo The component default is PARCEL_QR_MGS. With IMPLICIT, Cholesky QR is used if the component is left at this default
Q mode cagmres_q_mode —
Spectrum of the preconditioned matrix precon_spectrum 0: false, 1: true
Type of the inner linear solver linsolve_algo Combination with precon_spectrum = 1 returns PARCEL_ERR_ARG
Tolerance of the inner linear solver linsolve_tolerance 0 or less means the solver’s default (component default 0)
Maximum iterations of the inner linear solver linsolve_max_iteration Same as above
Restart length of the inner linear solver linsolve_restart Same as above
Preconditioner of the inner linear solver linsolve_precon_algo, linsolve_jacobi_iter, linsolve_blockjacobi 0 blocks means automatic. The scaling cannot be specified (always automatic)
Shift-invert shift_enable, shift_sigma Enabled with shift_enable = 1. The wrapper assembles A−σIA - \sigma I
Size of the Krylov subspace ks_m, ks_k_keep, ks_nev —
Maximum number of restarts ks_max_restart —
Orthogonalization method (ks_arnoldi) ks_qr_algo —
Initial vector eigvec_seed and the first n_local elements of eigvec With eigvec_seed = 1, eigvec is used as the initial vector
Common result Component of parcel_result
Convergence state converged_reason
Iteration count iterations
Residual residual
True residual true_residual
Number of converged eigenpairs ks_nconv
Number of inner solver failures linsolve_failures

19.2 Constants

The module has the following named constants, all of type integer(c_int). The meanings of the setting values are given in the common constants, and those of the convergence states in Convergence Status in Common Items.

Constants Values Use
PARCEL_SOLVER_PCG, PARCEL_SOLVER_PBICGSTAB, PARCEL_SOLVER_PGMRES, PARCEL_SOLVER_PCAGMRES 0, 1, 2, 3 Linear solvers (solver of parcel_solve_linear)
PARCEL_SOLVER_POWER, PARCEL_SOLVER_INVERSE_POWER, PARCEL_SOLVER_KS_ARNOLDI 4, 5, 6 Eigenvalue solvers (solver of parcel_solve_eigen)
PARCEL_PRECON_NONE, PARCEL_PRECON_POINTJACOBI, PARCEL_PRECON_BLOCKJACOBI, PARCEL_PRECON_JACOBI 0, 1, 2, 4 precon_algo, linsolve_precon_algo
PARCEL_QR_CGS, PARCEL_QR_MGS, PARCEL_QR_CHOLESKYQR, PARCEL_QR_CAGMRES_CHOLESKYQR 0, 1, 2, 202 gmres_qr_algo and ks_qr_algo (CGS and MGS only), cagmres_qr_algo
PARCEL_LINSOLVE_PGMRES, PARCEL_LINSOLVE_PCG 0, 1 linsolve_algo
PARCEL_CAGMRES_Q_AUTO, PARCEL_CAGMRES_Q_EXPLICIT, PARCEL_CAGMRES_Q_IMPLICIT 0, 1, 2 cagmres_q_mode
PARCEL_CONVERGED_RTOL, PARCEL_ITERATING, PARCEL_DIVERGED_ITS, PARCEL_DIVERGED_NANORINF, PARCEL_DIVERGED_BREAKDOWN 1, 0, -1, -2, -3 Convergence state (res%converged_reason)

The return values of the functions are as follows. The results of the argument checks are made consistent on all ranks before returning, so even if only one rank is invalid, all ranks return the same error.

Return value Value Meaning
PARCEL_OK 0 Completed normally. Whether the iteration converged is not included (check res%converged_reason)
PARCEL_ERR_ARG -1 Invalid argument or setting
PARCEL_ERR_DISTRIBUTION -2 The row blocks of the ranks do not cover the whole matrix without overlaps or gaps
PARCEL_ERR_NOT_BUILT -3 The specified solver was excluded from the build (see “Selecting the Solvers to Build”)
PARCEL_ERR_RUNTIME -4 A failure inside the library. In a build with MPI it is not returned to the caller (below)

When a failure occurs inside the library (insufficient device memory, a matrix whose size exceeds the range of the library’s internal integers, etc.), the wrapper writes a message ([parcel_c] exception: ...) to standard error and then calls MPI_Abort to terminate all ranks. Such a failure can occur on only one rank, and if it were left to the caller, the other ranks would keep waiting in the collective communication. In a build without MPI, PARCEL_ERR_RUNTIME is returned.

19.3 parcel_options / parcel_result

Description The derived type parcel_options that holds the settings and the derived type parcel_result that receives the results. Both are bind(C).

Synopsis

type(parcel_options) :: opt
type(parcel_result)  :: res

Fields Components are referred to as opt%tolerance, res%converged_reason, etc. The components of parcel_options and the defaults set by parcel_options_init are as follows. The meanings are those of the common settings. For the inner linear solver, linsolve_tolerance, linsolve_max_iteration and linsolve_restart are set to 0 (0 or less means “use the solver’s default”), because the default maximum number of iterations differs between inverse_power (5000) and ks_arnoldi (2000) and cannot be expressed by one component.

Component Type Default Description
tolerance real(c_double) 1.0d-8 Convergence tolerance
linsolve_tolerance real(c_double) 0 Tolerance of the inner linear solver. 0 or less means the solver’s default (1.0d-10 for both inverse_power and ks_arnoldi)
shift_sigma real(c_double) 0.0 Shift σ of shift-invert (when shift_enable=1)
max_iteration integer(c_int) 10000 Maximum number of iterations (not used by ks_arnoldi)
precon_algo integer(c_int) PARCEL_PRECON_NONE Preconditioner type
jacobi_iter integer(c_int) 1 Number of Jacobi iterations
blockjacobi integer(c_int) 0 Number of block Jacobi blocks. 0 means automatic
scaling_matrix integer(c_int) -1 Scaling of the preconditioner matrix. -1: automatic, 0: off, 1: on
scaling_vector integer(c_int) -1 Scaling of the preconditioner vectors (same values)
index_base integer(c_int) 0 Index base of nstart and col (0 or 1)
verbose integer(c_int) 0 With 1, rank 0 prints the device name and the settings
gmres_restart integer(c_int) 30 Restart length mm of pgmres
gmres_qr_algo integer(c_int) PARCEL_QR_MGS Orthogonalization method of pgmres: PARCEL_QR_CGS or PARCEL_QR_MGS. Others return PARCEL_ERR_ARG
cagmres_s integer(c_int) 8 ss of pcagmres
cagmres_t integer(c_int) 4 tt of pcagmres
cagmres_qr_algo integer(c_int) PARCEL_QR_MGS QR factorization method of pcagmres
precon_spectrum integer(c_int) 0 Spectrum of the preconditioned matrix. 0: eigenvalues of AA, 1: eigenvalues of AK−1AK^{-1}
linsolve_algo integer(c_int) PARCEL_LINSOLVE_PGMRES Type of the inner linear solver of inverse_power
linsolve_max_iteration integer(c_int) 0 Maximum iterations of the inner linear solver. 0 or less means the solver’s default (5000 for inverse_power, 2000 for shift-invert of ks_arnoldi)
linsolve_restart integer(c_int) 0 Restart length of the inner pgmres. 0 or less means the solver’s default (30 for both)
linsolve_precon_algo integer(c_int) PARCEL_PRECON_JACOBI Type of the preconditioner of the inner linear solver KinK_{in}. Not used with precon_spectrum=1
linsolve_jacobi_iter integer(c_int) 1 Number of Jacobi iterations of KinK_{in}
linsolve_blockjacobi integer(c_int) 0 Number of block Jacobi blocks of KinK_{in} (0 means automatic)
ks_m integer(c_int) 30 Maximum dimension of the Krylov subspace mm of ks_arnoldi
ks_k_keep integer(c_int) 10 Number of Ritz pairs kkeepk_{keep} kept at a restart of ks_arnoldi
ks_nev integer(c_int) 4 Number of eigenvalues nevnev computed by ks_arnoldi
ks_max_restart integer(c_int) 200 Maximum number of restarts of ks_arnoldi
shift_enable integer(c_int) 0 With 1, shift-invert. ks_arnoldi only
eigvec_seed integer(c_int) 0 With 1, the first n_local elements of eigvec are used as the initial vector of ks_arnoldi. With 0, the wrapper builds it from the global row numbers
cagmres_q_mode integer(c_int) PARCEL_CAGMRES_Q_AUTO Q mode of pcagmres. The combination rules are those of the common items; combinations violating them return PARCEL_ERR_ARG
ks_qr_algo integer(c_int) PARCEL_QR_CGS (the build-time default of the library) Orthogonalization method of ks_arnoldi: PARCEL_QR_CGS (CGS2) or PARCEL_QR_MGS (MGS2). Other values give PARCEL_ERR_ARG

The components of parcel_result are as follows. The values are the same on all ranks. If the solve was not done because of an argument error, converged_reason = PARCEL_ITERATING (0), iterations = 0, and the residuals are NaN.

Component Type Description
converged_reason integer(c_int) Convergence state. Positive means converged, negative means failure
iterations integer(c_int) Iteration count (Meaning of the Iteration Count and Residuals)
residual real(c_double) Residual. NaN if not computed
true_residual real(c_double) True residual. NaN for the eigenvalue solvers
ks_nconv integer(c_int) Number of converged eigenpairs. 0 except for ks_arnoldi
linsolve_failures integer(c_int) Number of inner solver failures

Requirements Always fill in the defaults with parcel_options_init and then change only the necessary components. Components not used by the called solver are ignored. However, the range of tolerance (finite positive), max_iteration (1 or greater) and the preconditioner components (precon_algo (one of the constants), jacobi_iter (1 or greater), blockjacobi (0 or greater), scaling_matrix and scaling_vector (-1, 0, 1)) is always checked, and that of linsolve_algo, linsolve_precon_algo, linsolve_jacobi_iter, linsolve_blockjacobi and linsolve_tolerance (not NaN) is always checked for the eigenvalue solvers; PARCEL_ERR_ARG is returned if a value is out of range. It is advisable to confirm with parcel_options_check() / parcel_result_check() that the sizes of the derived types match the structures of the wrapper.

MPI Give the same contents on all ranks.

19.4 parcel_options_init

Description Fills parcel_options with the defaults.

Synopsis

call parcel_options_init(opt)

Arguments

Name In/Out Type Description Default
opt Out type(parcel_options) Derived type to fill with the defaults —

Return Value None (subroutine). The defaults agree with the common defaults (table of parcel_options).

MPI Local operation.

Example

type(parcel_options) :: opt
call parcel_options_init(opt)
opt%tolerance = 1.0d-9

19.5 parcel_solve_linear

Description Solves the linear system Ax=bAx = b (pcg, pbicgstab, pgmres, pcagmres).

Synopsis

ierr = parcel_solve_linear(solver, opt, comm, n_local, nstart, n_global, nnz_local, &
                           rowptr, col, val, b, x [, res])

Arguments

Name In/Out Type Description Default
solver In integer(c_int) One of PARCEL_SOLVER_PCG, PARCEL_SOLVER_PBICGSTAB, PARCEL_SOLVER_PGMRES, PARCEL_SOLVER_PCAGMRES. Others return PARCEL_ERR_ARG —
opt In type(parcel_options) Settings. Commonly used components are tolerance, max_iteration and the preconditioner components. pgmres uses gmres_restart and gmres_qr_algo; pcagmres uses cagmres_s, cagmres_t, cagmres_qr_algo and cagmres_q_mode —
comm In integer(c_int) Communicator (Fortran integer handle; communicators other than MPI_COMM_WORLD are allowed) —
n_local In integer(parcel_int_kind) Number of owned rows (1 or greater) —
nstart In integer(parcel_int_kind) Global row number of the first owned row (in the index_base convention) —
n_global In integer(parcel_int_kind) Global number of rows (sum of n_local over all ranks) —
nnz_local In integer(parcel_int_kind) Number of nonzeros in the owned rows —
rowptr(*) In integer(parcel_int_kind) Row pointer (length n_local + 1). Only differences from the first element are used, so it may be 0-based or 1-based —
col(*) In integer(parcel_int_kind) Global column indices (length nnz_local, in the index_base convention) —
val(*) In real(c_double) Matrix elements (length nnz_local) —
b(*) In real(c_double) Owned part of the right-hand side (length n_local) —
x(*) In/Out real(c_double) Initial guess on input, owned part of the solution on output (length n_local) —
res Out (optional) type(parcel_result) Results. Not returned if omitted —

Requirements The matrix satisfies the conditions of Input Matrix in Common Items.

Return Value integer(c_int): PARCEL_OK or a negative error code (constants). Whether the iteration converged is not included in the return value; check res%converged_reason.

MPI Collective operation. Call on all ranks of the communicator with the same solver and the same contents of opt. The progress and computing times are printed to the standard output of rank 0 (Standard Output in Common Items).

See Also parcel_options / parcel_result, Convergence by Solver

Example

call parcel_options_init(opt)
opt%index_base  = 1
opt%tolerance   = 1.0d-9
opt%precon_algo = PARCEL_PRECON_BLOCKJACOBI
rc = parcel_solve_linear(PARCEL_SOLVER_PGMRES, opt, MPI_COMM_WORLD, n_local, nstart, n_global, &
     nnz_local, rowptr, col, val, b, x, res)
if (rc /= PARCEL_OK) then
   ! argument error etc.
else if (res%converged_reason <= 0) then
   ! not converged (PARCEL_DIVERGED_*)
end if

19.6 parcel_solve_eigen

Description Solves the eigenvalue problem Av=λvAv = \lambda v (AK−1AK^{-1} with precon_spectrum=1) (power, inverse_power, ks_arnoldi).

Synopsis

ierr = parcel_solve_eigen(solver, opt, comm, n_local, nstart, n_global, nnz_local, &
                          rowptr, col, val, eigval, eigvec [, res])

Arguments opt, comm, the matrix arguments (n_local to val) and res are the same as in parcel_solve_linear. With k = parcel_eigen_count:

Name In/Out Type Description Default
solver In integer(c_int) One of PARCEL_SOLVER_POWER, PARCEL_SOLVER_INVERSE_POWER, PARCEL_SOLVER_KS_ARNOLDI —
eigval(*) Out real(c_double) 2*k elements. The real part of the j-th eigenvalue (1-based) is eigval(2*j-1) and the imaginary part eigval(2*j). Two elements are needed even for power / inverse_power; eigval(1) holds the eigenvalue and eigval(2) is 0 —
eigvec(*) In/Out real(c_double) n_local*k elements. The owned part of the j-th eigenvector is eigvec((j-1)*n_local+1 : j*n_local). A complex conjugate pair is stored as two vectors, real part and imaginary part (Output of Eigenvalues and Eigenvectors). For ks_arnoldi with eigvec_seed=1, the first n_local elements are the initial vector (input) —

The eigenvalues computed and the main components used are as follows.

Solver Computes Main components
power One eigenvalue of largest magnitude tolerance, max_iteration, precon_spectrum
inverse_power One eigenvalue of smallest magnitude The above + linsolve_algo, linsolve_tolerance, linsolve_max_iteration, linsolve_restart, linsolve_precon_*
ks_arnoldi ks_nev eigenvalues on the largest-magnitude side (near σ in shift-invert mode) tolerance, precon_spectrum, ks_m, ks_k_keep, ks_nev, ks_max_restart, ks_qr_algo, shift_enable, shift_sigma, eigvec_seed, and linsolve_* in shift-invert mode

Requirements

Return Value Same as parcel_solve_linear. For inverse_power and shift-invert, a nonzero res%linsolve_failures means that the inner linear solver did not converge in some iterations. res%ks_nconv of ks_arnoldi is the number of eigenpairs that satisfied the tolerance.

MPI Collective operation (same as parcel_solve_linear).

See Also parcel_eigen_count, Notes on the Eigenvalue Solvers, Convergence by Solver

Example

integer(c_int) :: k
real(c_double), allocatable :: ev(:), v(:)
k = parcel_eigen_count(PARCEL_SOLVER_KS_ARNOLDI, opt)     ! = ks_k_keep
allocate(ev(2*k), v(n_local*k))                             ! ev: Re, Im alternating; v: one column per vector
rc = parcel_solve_eigen(PARCEL_SOLVER_KS_ARNOLDI, opt, MPI_COMM_WORLD, n_local, nstart, n_global, &
     nnz_local, rowptr, col, val, ev, v, res)

19.7 parcel_eigen_count

Description Returns the number k of eigenpairs returned by an eigenvalue solver. Used to size the arrays passed to parcel_solve_eigen.

Synopsis

k = parcel_eigen_count(solver, opt)      ! integer(c_int)

Arguments

Name In/Out Type Description Default
solver In integer(c_int) PARCEL_SOLVER_* —
opt In type(parcel_options) Settings (ks_arnoldi uses ks_k_keep) —

Return Value 1 for power / inverse_power, opt%ks_k_keep for ks_arnoldi (of which only the first ks_nev are guaranteed to converge), 0 for the linear solvers and invalid values.

MPI Local operation.

19.8 parcel_options_check() / parcel_result_check()

Description Checks whether the sizes of the Fortran derived types match the structures of the wrapper.

Synopsis

logical :: ok
ok = parcel_options_check() .and. parcel_result_check()

Return Value .true. if they match. Internally compares parcel_options_size() / parcel_result_size() with c_sizeof.

Notes Not called automatically; it is advisable to check at the start of the program. If they do not match, rebuild the wrapper and the module from the same version.

19.9 parcel_options_size() / parcel_result_size()

Description Returns the sizes [bytes] of the structures parcel_options and parcel_result of the wrapper.

Synopsis

n = parcel_options_size()      ! integer(c_int)
n = parcel_result_size()

Notes The sizes of the structures may change between versions, so build the wrapper, the module and user programs from the same version.

19.10 parcel_get_version / parcel_version_string

Description Returns the version of the library (the same values as parcel_get_version() / parcel_version_string() of C).

Synopsis

integer(c_int) :: major, minor, patch, ierr
character(len=32) :: str
ierr = parcel_get_version(major, minor, patch)   ! PARCEL_OK
call parcel_version_string(str)                  ! "1.2.0" (truncated if short; the rest is blank)

19.11 run_parcel_sycl_f() (deprecated)

Description The Fortran API of v1.0 (legacy API). Solves Ax=bAx = b with pcg. Kept for compatibility. For new programs, use parcel_solve_linear.

Synopsis It is not included in the module, so write the following interface yourself. The communicator is passed by reference and n and nnz by value; integer arguments and arrays are 64-bit (integer(c_int64_t)). Pass the CRS 0-based.

interface
   subroutine run_parcel_sycl(mpi_comm_f, n, nnz, crsA, crsCol, crsRow_ptr, vecx, vecb) &
        bind(c, name="run_parcel_sycl_f")
     use iso_c_binding
     implicit none
     integer(c_int) mpi_comm_f
     integer(c_int64_t), value :: n
     integer(c_int64_t), value :: nnz
     real(c_double) crsA(*)
     integer(c_int64_t) crsCol(*)
     integer(c_int64_t) crsRow_ptr(*)
     real(c_double) vecx(*)
     real(c_double) vecb(*)
   end subroutine run_parcel_sycl
end interface

Arguments n is the number of owned rows, nnz the number of nonzeros in the owned rows, crsA the matrix elements, crsCol the global column indices (0-based), crsRow_ptr the row pointer (0-based), vecx the initial guess on input and the solution on output, and vecb the right-hand side.

Requirements The first row of each rank’s range is obtained as the prefix sum of n over the ranks. If n is equal on all ranks this gives n*myrank, and uneven partitions can also be handled. The input arrays are not modified.

Return Value None. On failure, rank 0 prints the error code to standard error.

MPI Collective operation.

Notes The convergence tolerance is fixed at 1.0e-9, the maximum number of iterations is the wrapper’s compile-time macro MAX_ITERATION (default 10000), and the preconditioner is determined by -Dparcel_solver_precon_algo=<0|1|2|4>. For block Jacobi (2) and Jacobi (4), matrix and vector scaling are enabled, and BLOCKJACOBI_BLOCK (default 1) and JACOBI_ITER (default 1) are used for the number of blocks and the number of iterations. The integer arguments changed from 32 bits in v1.0 to 64 bits (“Versions and Compatibility” in the User Guide).

Example

rowptr0 = rowptr - 1          ! convert the 1-based CRS to 0-based
col0    = col - 1
call run_parcel_sycl(MPI_COMM_WORLD, n_local, nnz_local, val, col0, rowptr0, x, b)