1. Kernel methods
Every basis function is a translate of the same radial kernel, centred on one sampled point \(x_j\):
\(\varphi_j(x) = k(x, x_j)\)
and the approximation is the linear combination \(u_h(x) = \sum_j c_j\, \varphi_j(x)\). The linear system is dense, since every basis function is nonzero at every point. For a moderate number of points, this method is often fast and very accurate, and only the coefficients \(c_j\) have to be computed. Some of the available kernels are listed below, with \(r = \lVert x - x_j \rVert\):
\(k_{\text{Gaussian}}(x, x_j) = \exp\!\left(-\dfrac{r^2}{2\sigma^2}\right)\)
\(k_{\text{Matérn-3/2}}(x, x_j) = \left(1 + \dfrac{\sqrt{3}\,r}{\sigma}\right)\exp\!\left(-\dfrac{\sqrt{3}\,r}{\sigma}\right)\)
\(k_{\text{Matérn-5/2}}(x, x_j) = \left(1 + \dfrac{\sqrt{5}\,r}{\sigma} + \dfrac{5 r^2}{3\sigma^2}\right)\exp\!\left(-\dfrac{\sqrt{5}\,r}{\sigma}\right)\)
Matérn kernels have a finite smoothness, while the Gaussian kernel
is infinitely smooth. They are useful when the solution itself has
limited smoothness. Another option,
FundamentalSolutionKernel, uses the Green's function of
the PDE (\(\log r\) in 2D for the Laplacian), so that each basis
function solves the equation away from its center.
from scimba_jax.linear_approximation.basis.kernel_basis import KernelBasis
from scimba_jax.linear_approximation.basis.kernel_function import GaussianKernel
from scimba_jax.linear_approximation.collocation.collocation_elliptic import (
EllipticCollocationScheme,
)
from scimba_jax.linear_approximation.variables.collocation_variables import (
CollocationVariables,
)
# One basis function per sampled point: the centers are the collocation
# points, where the PDE is enforced.
basis = KernelBasis(
dim=2,
output_dim=1,
kernel_function=GaussianKernel(sigma=0.5),
centers=interior_points, # (n_points, 2), drawn by a DomainSampler
learnable_center_bool=False, # fixed centers: only the linear weights are computed
basis_type="scalar",
)
variables = CollocationVariables(basis=basis, nb_variables=1)
scheme = EllipticCollocationScheme(
pde=pde,
variables=variables,
collocation_points=interior_points,
bc_collocation_points=boundary_points,
)
scheme = EllipticCollocationScheme.solve(scheme)
# 600 interior + 80 boundary points, sigma=0.5: 3.0e-06 relative L2 error
# against sin(pi x) sin(pi y) on the unit square.
# Other kernels only require changing kernel_function. Values for two points 0.5 apart:
# GaussianKernel(sigma=0.5) -> 0.6065
# Matern32Kernel(sigma=0.5) -> 0.4834
# Matern52Kernel(sigma=0.5) -> 0.5240
# ExponentialKernel(sigma=0.5) -> 0.3679 (C0 only, sharp at r=0)
# FundamentalSolutionKernel(dim=2) -> -0.1103 (Green's function log(r)/2pi)
2. Random features
The collocation scheme is the same, but the radial basis is replaced by a nonlinear one:
\(\varphi_j(x) = \tanh\!\left(\dfrac{W_j \cdot (x - x_j)}{s} + b_j\right)\)
where \((W_j, b_j)\) are drawn once from a fixed random distribution and are not modified afterwards, as in extreme learning machines. As for kernel methods, only the coefficients \(c_j\) are computed.
from scimba_jax.linear_approximation.basis.random_network_basis import (
RandomFeaturesBasis,
)
# Same collocation scheme with a nonlinear basis: (W_j, b_j) are drawn once
# and fixed, as in extreme learning machines.
basis = RandomFeaturesBasis(
dim=2,
output_dim=1,
centers=interior_points,
scale=0.5,
activation="tanh",
learnable_center_bool=False, # default: fixed centers
learnable_features_bool=False, # default: fixed features
)
variables = CollocationVariables(basis=basis, nb_variables=1)
scheme = EllipticCollocationScheme(
pde=pde,
variables=variables,
collocation_points=interior_points,
bc_collocation_points=boundary_points,
)
scheme = EllipticCollocationScheme.solve(scheme)
# Same problem as above: 1.4e-08 relative L2 error, more accurate than the
# kernel basis here, but the PRNG key has to be kept.
3. Localized random features (RFM)
The Random Feature Method multiplies the random features of each patch by a smooth window function that vanishes outside the patch. The assembled system is then sparse, as for a mesh-based method, while still not requiring a mesh. This method thus combines a meshless approximation with the locality of finite element or finite volume schemes.
from scimba_jax.linear_approximation.basis.random_network_basis import (
RandomFeaturePOUBasis,
)
# Random Feature Method (Chen, Chi, E & Yang, 2022): features localised on patches
# by a partition of unity, which gives a sparse system without a mesh.
basis = RandomFeaturePOUBasis(
dim=2,
output_dim=1,
centers=patch_centers, # (n_patches, 2)
radii=0.6, # shared, or (n_patches, 2) for anisotropic patches
n_features_per_patch=8,
activation="tanh",
)
All the bases on this page are fixed by construction: their centers and shapes are drawn once and are not optimized. If they are optimized by gradient descent, the basis becomes a small neural network, and the method becomes a PINN.