6 parcel::matrix
A class that holds a sparse matrix. The matrix is registered in CRS format (set_crsMat()). A registration function for DIA format, set_diaMat(), is also public but cannot be used in this version (passing a matrix registered with it to solver() throws std::runtime_error).
6.1 Constructor
Description Creates an empty matrix object.
Synopsis
template< class parceltype = parcel_double >
class matrix;
parcel::matrix<parceltype>::matrix(sycl::queue *q);Arguments
| Name | In/Out | Type | Description | Default |
|---|---|---|---|---|
parceltype |
In (template) | PARCEL data type | Precision in which the matrix entries are stored | parcel_double |
q |
In | sycl::queue * |
Queue of the device on which the matrix is placed | none |
Requirements q must be an in-order queue. It is not checked here; set_crsMat() checks it and throws std::invalid_argument (also when q is NULL). The queue must not be destroyed until you have finished using the matrix.
MPI Local operation.
See Also set_crsMat()
Example
parcel::matrix<parcel_double> A(&q); // same as parcel::matrix<> A(&q);6.2 set_crsMat()
Description Registers the CRS matrix of the row block owned by each rank and builds the MPI halo communication table.
Synopsis
template< class Ttype >
void set_crsMat(parcel::comm *comm, PARCEL_INT nstart, PARCEL_INT nend,
size_t n, size_t nnz,
PARCEL_INT *rowptr, PARCEL_INT *crsCol, Ttype *mem);Arguments
| Name | In/Out | Type | Description | Default |
|---|---|---|---|---|
Ttype |
In (template) | floating-point type | Type of the input array mem (double, etc.). Converted to the parceltype of the matrix at registration |
none |
comm |
In | parcel::comm * |
MPI communicator for the SYCL version of PARCEL | none |
nstart |
In | PARCEL_INT |
Global row index (starting from 0) of the first locally owned row | none |
nend |
In | PARCEL_INT |
Global row index of the last locally owned row (inclusive) | none |
n |
In | size_t |
Number of locally owned rows (= nend - nstart + 1) |
none |
nnz |
In | size_t |
Number of nonzero entries in the locally owned rows | none |
rowptr |
In | PARCEL_INT * |
Row pointer of length n + 1. Local values, with the start of the locally owned rows as 0 |
none |
crsCol |
In/Out | PARCEL_INT * |
Column indices of length nnz. On input, global 0-based column indices. On output, they are overwritten with local indices |
none |
mem |
In/Out | Ttype * |
Matrix entries of length nnz. On output, they are reordered within each row |
none |
Requirements The following are checked, and an exception is thrown on violation.
- The queue of the matrix (
qof the constructor) is in-order. Otherwisestd::invalid_argument(-DPARCEL_REQUIRE_IN_ORDER=0disables the check). nandnnzfit in the range ofPARCEL_INT. Otherwisestd::overflow_error. This can happen only in a build with-DPARCEL_INT32; rebuild without-DPARCEL_INT32.- After the column indices are converted to local indices, the number of nonzeros
nnzof each rank and the sum of the local rows and the halo elements each fit in the range ofPARCEL_LINT(at most with the default 32 bits). If even one rank exceeds the range, all ranks throwstd::overflow_error. The remedy is to rebuild with-DPARCEL_LOCAL_INT64or to increase the number of ranks (in a-DPARCEL_INT32build, remove-DPARCEL_INT32and add-DPARCEL_LOCAL_INT64). - The number of bytes of halo elements sent and received between each pair of ranks fits in the range of the MPI
int. If even one pair exceeds it, all ranks throwstd::overflow_error. The remedy is to increase the number of ranks. - The row block distribution and the local CRS arrays satisfy the following conditions. Otherwise
std::invalid_argument. is the sum ofnover all ranks (the number of rows of the matrix).commis not NULL.n = nend - nstart + 1. A rank that owns no rows is not allowed.nstart.rowptris not NULL.rowptr[0] = 0,rowptris nondecreasing, androwptr[n] = nnz.- The row blocks [
nstart,nend] of all ranks cover without gaps or overlaps. They need not be ordered by rank number. - Column indices lie in .
For the last group of conditions, the results of all ranks are gathered by collective communication, and if there is a violation all ranks throw the same std::invalid_argument. The check can be disabled with -DPARCEL_NO_CRS_CHECK=1. In that case, the behavior when the conditions are not satisfied is undefined (wrong results, abnormal termination, deadlock). If allocation of device memory fails, util_sycl::device_bad_alloc is thrown (Handling of Settings and Exceptions).
MPI Collective operation (MPI_Allgather / MPI_Allreduce for the checks, MPI_Alltoall, MPI_Isend / MPI_Irecv, MPI_Barrier). All ranks of comm must call it.
Notes crsCol and mem are overwritten in place (column indices are converted to local indices, and within each row the columns owned by the local rank are placed first and the halo columns after them). rowptr is not modified. If it returns with std::invalid_argument, the arrays have not yet been rewritten, but if it returns with std::overflow_error for the PARCEL_LINT range or for the halo byte count, crsCol and mem have already been rewritten. The arrays remain of type PARCEL_INT; before being sent to the device they are copied into PARCEL_LINT arrays (by default this temporarily uses 4(n + nnz) extra bytes on the host). If a second matrix (e.g. for shift-invert) is created from the same arrays, the column indices are converted twice and halo communication breaks in MPI runs. When creating multiple matrices, duplicate the arrays before the first set_crsMat(). There is also an overload set_crsMat(comm, nstart, nend, n, nnz, rowptr, crsCol, double *hi, double *lo) that passes the high and low parts of double-double in separate arrays. Like the template version, it performs the same checks and registers the communicator and the halo communication table (crsCol, hi, and lo are reordered in place).
In a build without MPI (-DPARCEL_MPI=0) the same form can be called. Since one process owns the whole matrix, nstart = 0 and n = nend - nstart + 1 (the total number of rows) are required; otherwise std::invalid_argument is thrown (checked even with -DPARCEL_NO_CRS_CHECK=1). There is no halo and no collective communication. The form without a communicator, set_crsMat(n, nnz, rowptr, crsCol, mem), can also be used (only in a build without MPI).
See Also parcel::comm; the “Data Structures” section (CRS format); “Setting the Matrix” and “MPI Decomposition” in the “Usage” chapter
Example
parcel::matrix<parcel_double> A(&q);
A.set_crsMat<double>(&comm, nstart, nend, n, col.size(), rowptr.data(), col.data(), val.data());6.3 n_local()
Description Returns the local number of rows of the matrix (the number of rows owned by this rank).
Synopsis
size_t n_local();Return Value For the CRS format, the n passed to set_crsMat() (nend - nstart + 1).
Requirements Call after the matrix has been registered with set_crsMat(). For a matrix that has not been registered, std::runtime_error is thrown.
MPI Local operation.
Notes ks_arnoldi takes the local number of rows from this value, not from the length of EigenVec.
Example
std::vector<double> evec_h(A.n_local() * k_keep);