Skip to main content
`Ctrl`+`K`

# Changes: 3.16

General:

- Change `MPIU_Allreduce()` to always returns a MPI error code that should be checked with `CHKERRMPI(ierr)`

- Add support for A64FX and Cray compilers

- Add support for ESSL 5.2 and later; drop support for ESSL <=5.1

- Add support for NEC-SX Tsubasa Vector Engine

- Add support for NVIDIA HPC SDK

Configure/Build:

- Configure requires Python-2.7 or Python-3.4+.

- Remove `--with-kokkos-cuda-arch`. One can use `-with-cuda-gencodearch` to specify the cuda arch for Kokkos. Usually not needed since PETSc auto detects that

- For `--download-hdf5`, disable `--download-hdf5-fortran-bindings` by default

- Add OpenCascade package to PETSc and allow `--download-opencascade`

- Add support for hypre in device mode for both NVIDIA and AMD GPUs

- Extend detection of C++ dialect to C++17. Configure now also takes minimum and maximum required C++ dialect of packages into account when choosing the C++ dialect

Sys:

- Add `PetscDevice` class to manage discovered GPU devices

- Add `PetscDeviceKind`

- Add `PetscDeviceCreate()`, `PetscDeviceConfigure()`, and `PetscDeviceDestroy()`

- Add `PetscDeviceContext` class to manage asynchronous GPU compute support via a fork-join model

- Add `PetscStreamType` and `PetscDeviceContextJoinMode`

- Add `PetscDeviceContextCreate()`, `PetscDeviceContextDestroy()`, `PetscDeviceContextSetDevice()`, `PetscDeviceContextGetDevice()`, `PetscDeviceContextSetStreamType()`, `PetscDeviceContextGetStreamType()`, `PetscDeviceContextSetUp()`, `PetscDeviceContextDuplicate()`, `PetscDeviceContextQueryIdle()`, `PetscDeviceContextWaitForContext()`, `PetscDeviceContextFork()`, `PetscDeviceContextJoin()`, `PetscDeviceContextSynchronize()`, `PetscDeviceContextGetCurrentContext()`, `PetscDeviceContextSetCurrentContext()`, and `PetscDeviceContextSetFromOptions()`

- Deprecate `petsccublas.h` and `petschipblas.h` in favor of `petscdevice.h` and `petscdevicetypes.h`

- Add GPU event timers to capture kernel execution time accurately

- Remove `WaitForCUDA()` and `WaitForHIP()` before `PetscLogGpuTimeEnd()`

- Add MPIU_REAL_INT and MPIU_SCALAR_INT datatypes to be used for reduction operations

- Add MPIU_MAXLOC and MPIU_MINLOC operations

- Add `CHKERRCXX()` to catch C++ exceptions and return a PETSc error code

- Remove `PetscStack` routines from public headers, this class should now be considered private

PetscViewer:

- `PetscViewerHDF5PushGroup()`: if input path begins with `/`, it is taken as absolute, otherwise relative to the current group

- Add `PetscViewerHDF5HasDataset()`

- `PetscViewerHDF5HasAttribute()`, `PetscViewerHDF5ReadAttribute()`, `PetscViewerHDF5WriteAttribute()`, `PetscViewerHDF5HasDataset()` and `PetscViewerHDF5HasGroup()` support absolute paths (starting with `/`) and paths relative to the current pushed group

- Add input argument to `PetscViewerHDF5ReadAttribute()` for default value that is used if attribute is not found in the HDF5 file

- Add `PetscViewerHDF5PushTimestepping()`, `PetscViewerHDF5PopTimestepping()` and `PetscViewerHDF5IsTimestepping()` to control timestepping mode.

- One can call `PetscViewerHDF5IncrementTimestep()`, `PetscViewerHDF5SetTimestep()` or `PetscViewerHDF5GetTimestep()` only if timestepping mode is active

- Error if timestepped dataset is read/written out of timestepping mode, or vice-versa

PetscDraw:

AO:

IS:

VecScatter / PetscSF:

PF:

Vec:

- Add `VecMean()` to calculate arithmetic mean of elements of a vector

- Add `VecBoundToCPU()` to query information set with `VecBindToCPU()`

PetscSection:

- Extend `PetscSectionView()` for section saving to HDF5

- Add `PetscSectionLoad()` for section loading from HDF5

PetscPartitioner:

Mat:

- `MATMPIKAIJ` now tracks the object state of the AIJ matrix describing the blockwise action of the KAIJ matrix and automatically rebuilds internal data structures before executing operations with the KAIJ matrix if the state has changed

- Factorization types now provide their preferred ordering (which may be `MATORDERINGEXTERNAL`) to prevent PETSc PCFactor from, by default, picking an ordering when it is not ideal

- Deprecate `MatFactorGetUseOrdering()`; Use `MatFactorGetCanUseOrdering()` instead

- Add `--download-htool` to use hierarchical matrices with the new type `MATHTOOL`

- Add `MATCENTERING` special matrix type that implements action of the centering matrix

- Remove -mat_mumps_icntl_7 1 option, use -pc_factor_mat_ordering_type <type> to have PETSc perform the ordering (sequential only)

- Add `MATSOLVERSPQR` - interface to SuiteSparse QR factorization

- Add `MatSeqAIJKron()` - Kronecker product of two `MatSeqAIJ`

- Add `MatNormalGetMat()` to retrieve the underlying `Mat` of a `MATNORMAL`

- Add `MatNormalHermitianGetMat()` to retrieve the underlying `Mat` of a `MATNORMALHERMITIAN`

- Add `VecCreateMPICUDA()` and `VecCreateMPIHIP()` to create MPI device vectors

- Add accessor routines for device index data of `MATSEQAIJCUSPARSE` matrices: `MatSeqAIJCUSPARSEGetIJ()` and `MatSeqAIJCUSPARSERestoreIJ()`

- Add accessor routines for device data of `MATSEQAIJCUSPARSE` matrices: `MatSeqAIJCUSPARSEGetArray()`, `MatSeqAIJCUSPARSERestoreArray()`, `MatSeqAIJCUSPARSEGetArrayRead()`, `MatSeqAIJCUSPARSERestoreArrayRead()`. `MatSeqAIJCUSPARSEGetArrayWrite()`, `MatSeqAIJCUSPARSERestoreArrayWrite()`

- Add support for `MATHYPRE` matrices on NVIDIA and AMD GPUs

- `MatPreallocatorPreallocate` performance significantly improved

- Add `MatGetColumnReductions()` developer routine to calculate reductions over columns of a matrix

- Add `MatGetColumnSums()`, `MatGetColumnSumsRealPart()`, `MatGetColumnSumsImaginaryPart()` to compute sums over matrix columns

- Add `MatGetColumnMeans()`, `MatGetColumnMeansRealPart()`, `MatGetColumnMeansImaginaryPart()` to compute arithmetic means over matrix columns

- Add `MatBoundToCPU()` to query information set with `MatBindToCPU()`

- Rename `MATHARA` in `MATH2OPUS`, supporting distributed memory operations with hierarchical matrices

PC:

- Add `PCSetPreSolve()`

- Add `PCQR` - interface to SuiteSparse QR factorization for `MatSeqAIJ`, `MATNORMAL`, and `MATNORMALHERMITIAN`

- Add support for `MATNORMAL` in `PCASM` and `PCHPDDM`

- Add support for BoomerAMG from `PCHYPRE` to run on NVIDIA and AMD GPUs

- `PCShellGetContext()` now takes `void*` as return argument

- Rename `PCHARA` in `PCH2OPUS`, supporting distributed memory operations with hierarchical matrices

KSP:

- `KSPGetMonitorContext()` now takes `void*` as return argument

- `KSPGetConvergenceContext()` now takes `void*` as return argument

SNES:

- Add `SNESSetComputeMFFunction()`

- Add support for `-snes_mf_operator` for use with `SNESSetPicard()`

- `SNESShellGetContext()` now takes `void*` as return argument

SNESLineSearch:

TS:

- Add `-ts_type irk` - fully implicit Runge-Kutta solvers

- Add `TSTrajectory` interface to the CAMS library for optimal offline checkpointing for multistage time stepping schemes

- Add option `-ts_trajectory_memory_type <revolve | cams | petsc>` to switch checkpointing schedule software

- Add option `-ts_trajectory_max_units_ram` to specify the maximum number of allowed checkpointing units

TAO:

- `TaoShellGetContext()` now takes `void*` as return argument

DM/DA:

- Change management of auxiliary data in DM from object composition to `DMGetAuxiliaryVec()`/`DMSetAuxiliaryVec()`, `DMCopyAuxiliaryVec()`

- Remove `DMGetNumBoundary()` and `DMGetBoundary()` in favor of DS counterparts

- Remove `DMCopyBoundary()`

- Change interface for `DMAddBoundary()`, `PetscDSAddBoundary()`, `PetscDSGetBoundary()`, `PetscDSUpdateBoundary()`

- Add `DMDAVecGetArrayDOFWrite()` and `DMDAVecRestoreArrayDOFWrite()`

- `DMShellGetContext()` now takes `void*` as return argument

DMSwarm:

- Add `DMSwarmGetCellSwarm()` and `DMSwarmRestoreCellSwarm()`

DMPlex:

- Add a `PETSCVIEWEREXODUSII` viewer type for `DMView()`/`DMLoad()` and `VecView()`/`VecLoad()`. Note that not all DMPlex can be saved in ExodusII format since this file format requires that the numbering of cell sets be compact

- Add `PetscViewerExodusIIOpen()` convenience function

- Add `PetscViewerExodusIISetOrder()` to generate “2nd order” elements (i.e. tri6, tet10, hex27) when using `DMView` with a `PETSCVIEWEREXODUSII` viewer

- Change `DMPlexComputeBdResidualSingle()` and `DMPlexComputeBdJacobianSingle()` to take a form key

- Add `DMPlexTopologyLoad()`, `DMPlexCoordinatesLoad()`, and `DMPlexLabelsLoad()` for incremental loading of a `DMPlex` object from an HDF5 file

- Add `DMPlexTopologyView()`, `DMPlexCoordinatesView()`, and `DMPlexLabelsView()` for incremental saving of a `DMPlex` object to an HDF5 file

- Add `DMPlexSectionView()` saving a `PetscSection` in association with a `DMPlex` mesh

- Add `DMPlexSectionLoad()` loading a `PetscSection` in association with a `DMPlex` mesh

- Add `DMPlexGlobalVectorView()` and `DMPlexLocalVectorView()` saving global and local vectors in association with a data layout on a `DMPlex` mesh

- Add `DMPlexGlobalVectorLoad()` and `DMPlexLocalVectorLoad()` loading global and local vectors in association with a data layout on a `DMPlex` mesh

- Add `DMPlexIsSimplex()` to check the shape of the first cell

- Add `DMPlexShape` to describe prebuilt mesh domains

- Add `DMPlexCreateCoordinateSpace()` to make an FE space for the coordinates

- Add the automatic creation of a Plex from options, see `DMSetFromOptions()`

- The old options for `DMPlexCreateBoxMesh()` NO LONGER WORK. They have been changed to make the interface more uniform

- Replace `DMPlexCreateSquareBoundary()` and `DMPlexCreateCubeBoundary()` with `DMPlexCreateBoxSurfaceMesh()`

- Remove `DMPlexCreateReferenceCellByType()`

- The number of refinements is no longer an argument to `DMPlexCreateHexCylinderMesh()`

- Add `DMSetLabel()`

- Replace `DMPlexComputeJacobianAction()` with `DMSNESComputeJacobianAction()`

- Add `DMSNESCreateJacobianMF()`

- Change `DMPlexComputeBdResidualSingle()` to take `PetscFormKey` instead of explicit label/value/field arguments

- Add `DMPlexInflateToGeomModel()` which pushes refined points out to a geometric boundary

- Separate EGADS and EGADSLite functionality, add `DMPlexCreateEGADSLiteFromFile()`

- Remove `DMPlexReverseCell()` and `DMPlexOrientCell()` in favor of `DMPlexOrientPoint()`

- Remove `DMPlexCompareOrientations()` in favor of `DMPolytopeMatchOrientation()`

- Add `DMPlexGetCompressedClosure()` and `DMPlexRestoreCompressedClosure()`

- Add `DMPlexMetricCreate` as a helper function for creating a (P1) Riemannian metric.

- Add `DMPlexMetricCreateUniform` as a helper function for creating a uniform metric.

- Add `DMPlexMetricCreateIsotropic` as a helper function for creating an isotropic metric.

- Add `DMPlexMetricEnforceSPD` for enforcing that a metric is symmetric positive-definite.

- Add `DMPlexMetricNormalize` to apply L-p metric normalization.

- Add `DMPlexMetricAverage` to average an arbitrary number of metrics.

- Add `DMPlexMetricAverage2` to average two metrics.

- Add `DMPlexMetricAverage3` to average three metrics.

- Add `DMPlexMetricIntersection` to intersect an arbitrary number of metrics.

- Add `DMPlexMetricIntersection2` to intersect two metrics.

- Add `DMPlexMetricIntersection3` to intersect three metrics.

FE/FV:

- Change `PetscFEIntegrateBdResidual()` and `PetscFEIntegrateBdJacobian()` to take both `PetscWeakForm` and form key

- Add `PetscFEGeomGetPoint()` and `PetscFEGeomGetCellPoint` to package up geometry handling

DMNetwork:

- Add `DMNetworkCreateIS()` and `DMNetworkCreateLocalIS()`

- Remove nv from `DMNetworkAddSubnetwork()`

DMStag:

- Add `DMStagStencilToIndexLocal()`

DT:

- Add `PetscWeakFormCopy()`, `PetscWeakFormClear()`, `PetscWeakFormRewriteKeys()` and `PetscWeakFormClearIndex()`

- Add `PetscDSDestroyBoundary()` and `PetscDSCopyExactSolutions()`

- `PetscDSGetContext()` now takes `void*` as return argument

- Add `PetscWeakFormReplaceLabel()` to change labels after mesh modification

Fortran:

- Add support for `PetscInitialize(filename,help,ierr)`, `PetscInitialize(ierr)` in addition to current `PetscInitialize(filename,ierr)`