scimba_torch.neural_nets.coordinates_based_nets.features¶

Neural networks with feature transformations such as Fourier features.

Classes

`EnhancedFeatureNet`(in_size, **kwargs)	A network that generates learnable features such as Fourier features.
`FlippedMLP`(in_size, out_size, **kwargs)	Combine flipping feature transformations with a Multi-Layer Perceptron (MLP).
`FourierMLP`(in_size, out_size, **kwargs)	Combines Fourier feature transformations with a Multi-Layer Perceptron (MLP).
`FourierResNet`(in_size, out_size, **kwargs)	Combines Fourier feature transformations with a ResNet architecture.
`GenericFeatureNet`(in_size, out_size, **kwargs)	A template for a general network with enhanced features.
`GenericFourierNet`(in_size, out_size, **kwargs)	Combines Fourier feature transformations with a specified neural network.
`GenericMultiScaleFourierNet`(in_size, ...)	Combines Fourier feature transformations with a specified neural network.
`GenericPeriodicNet`(in_size, out_size, **kwargs)	A neural network that appends a periodic embedding before the first layer.
`MultiScaleFourierMLP`(in_size, out_size, **kwargs)	A linear combination of Fourier feature transformations with MLP.
`MultiScaleFourierResNet`(in_size, out_size, ...)	Combine Fourier feature transformations with Residual Networks (ResNet).
`PeriodicMLP`(in_size, out_size, **kwargs)	A neural network that combines periodic feature transformations with an MLP.
`PeriodicResNet`(in_size, out_size, **kwargs)	Combine a periodic feature transformations with a ResNet architecture.

class EnhancedFeatureNet(in_size, **kwargs)[source]¶

Bases: Module

A network that generates learnable features such as Fourier features.

The weights are initialized using a normal distribution.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
**kwargs (Any) –
Additional keyword arguments:
- nb_features (int, default=1): The number of features generated by the network.
- type_feature (str, default=”fourier”): The type of feature transformation to apply.
- mean (float, default=0.0): The mean used for initializing the weights.
- std (float, default=1.0): The standard deviation used for initializing the weights.

re_init(mean, std)[source]¶

Reinitialize the weights of the linear layer using a normal distribution.

Parameters:

mean (float) – Mean value for normal distribution initialization.
std (float) – Standard deviation for normal distribution initialization.

forward(x)[source]¶

Apply the feature transformation to the input tensor x.

Parameters:: x (Tensor) – Input tensor.
Return type:: Tensor
Returns:: Transformed feature tensor.

class GenericFeatureNet(in_size, out_size, **kwargs)[source]¶

Bases: ScimbaModule

A template for a general network with enhanced features.

A feature can be a periodic embedding, Fourier features, etc.

parameters(flag_scope='all', flag_format='list')[source]¶

Get parameters of the neural net.

Parameters:

flag_scope (str) – Specifies which parameters to return. Options: ‘all’, ‘last_layer’, ‘except_last_layer’.
flag_format (str) – Specifies the format Options: ‘list’, ‘tensor’.

Return type:

list[Parameter] | Tensor

Returns:

A list of parameters or a single tensor containing all parameters.

Raises:

ValueError – If an unknown flag_scope or flag_format is provided.

set_parameters(new_params, flag_scope='all')[source]¶

Set parameters.

Parameters:

new_params (Tensor) – new parameters.
flag_scope (str) – ‘all’, ‘last_layer’, ‘except_last_layer’

Raises:

ValueError – If an unknown flag_scope is provided.

re_init_features(mean, std)[source]¶

Reinitialize the weights of the EnhancedFeatureNet layer.

Use a normal distribution with the specified mean and standard deviation.

Parameters:

mean (float) – Mean value for the normal distribution.
std (float) – Standard deviation for the normal distribution.

class GenericFourierNet(in_size, out_size, **kwargs)[source]¶

Bases: GenericFeatureNet

Combines Fourier feature transformations with a specified neural network.

The network first generates enhanced features (such as Fourier features), concatenates them with the original input, and then passes the result through a user-specified network architecture.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Additional keyword arguments:
- nb_features (int, default=1): Number of features generated by EnhancedFeatureNet.
- type_feature (str, default=”fourier”): Type of feature transformation
- Other keyword arguments are passed to the EnhancedFeatureNet and to whichever network class is specified by the user.

Learnable Parameters:

features (EnhancedFeatureNet): A network that generates enhanced features such as Fourier features.
net: (ScimbaModule): A neural network that processes the input and features.

forward(x, with_last_layer=True)[source]¶

Compute the forward pass.

Apply the feature transformation, concatenate the features with the original input, and pass the result through the neural network to produce the output.

Parameters:

x (Tensor) – Input tensor.
with_last_layer (bool) – Whether to include the last layer in the forward pass. (default: True)

Return type:

Tensor

Returns:

Output tensor after passing through the neural network.

class FourierMLP(in_size, out_size, **kwargs)[source]¶

Bases: GenericFourierNet

Combines Fourier feature transformations with a Multi-Layer Perceptron (MLP).

The network first generates enhanced features (such as Fourier features), concatenates them with the original input, and then passes the result through a fully connected MLP.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Additional keyword arguments:
- nb_features (int, default=1): Number of features generated by EnhancedFeatureNet.
- type_feature (str, default=”fourier”): Type of feature transformation applied.
- Other keyword arguments are passed to EnhancedFeatureNet and GenericMLP classes.

class FourierResNet(in_size, out_size, **kwargs)[source]¶

Bases: GenericFourierNet

Combines Fourier feature transformations with a ResNet architecture.

The network first generates enhanced features (such as Fourier features), concatenates them with the original input, and then passes the result through a series of residual blocks. It is a specialization of GenericFourierNet.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs – Additional keyword arguments.

class GenericPeriodicNet(in_size, out_size, **kwargs)[source]¶

Bases: GenericFeatureNet

A neural network that appends a periodic embedding before the first layer.

The network first generates periodic features, passes the input through it, and then: passes the result through the chosen architecture.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Additional keyword arguments:
- domain_bounds (required): The bounds of the domain for the periodic features.
- nb_features (optional): The number of features generated by the EnhancedFeatureNet.
- type_feature (optional): The type of feature transformation applied (e.g., “periodic”).
- Other arguments passed to the EnhancedFeatureNet class and whichever network class is specified by the user.

Raises:

KeyError – If domain_bounds is not provided in kwargs.
ValueError – If domain_bounds is not a torch.Tensor or has incorrect shape.

Learnable Parameters:

features (EnhancedFeatureNet): A network that generates enhanced features such as Fourier features.
net: (ScimbaModule): A neural network that processes the input and features.

compute_periods(domain_bounds)[source]¶

Compute the periods for the periodic embedding from the domain bounds.

Parameters:: domain_bounds (Tensor) – The bounds of the domain for periodic features.
Returns:: The computed periods.
Return type:: torch.Tensor

forward(x, with_last_layer=True)[source]¶

Compute the forward pass.

Apply the periodic transformation and pass the result through the MLP to produce the output.

Parameters:

x (Tensor) – Input tensor.
with_last_layer (bool) – Whether to include the last layer in the forward pass. (default: True)

Return type:

Tensor

Returns:

Output tensor after passing through the neural network.

class PeriodicMLP(in_size, out_size, **kwargs)[source]¶

Bases: GenericPeriodicNet

A neural network that combines periodic feature transformations with an MLP.

The network first generates periodic features, passes the input through it, and then passes the result through a fully connected MLP.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Other keyword arguments including:
- domain_bounds (required): The bounds of the domain for the periodic features.
- nb_features (optional): Number of features generated by EnhancedFeatureNet.
- type_feature (optional): Type of feature transformation applied (e.g., “periodic”).
- Other arguments passed to EnhancedFeatureNet and GenericMLP classes.

class PeriodicResNet(in_size, out_size, **kwargs)[source]¶

Bases: GenericPeriodicNet

Combine a periodic feature transformations with a ResNet architecture.

The network first generates periodic features, passes the input through it, and then passes the result through a ResNet.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Other keyword arguments including:
- domain_bounds (required): The bounds of the domain for the periodic features.
- nb_features (optional): The number of features generated by the EnhancedFeatureNet.
- type_feature (optional): The type of feature transformation applied (e.g., “periodic”).
- Other arguments passed to the EnhancedFeatureNet and GenericResNet classes

class FlippedMLP(in_size, out_size, **kwargs)[source]¶

Bases: GenericFeatureNet

Combine flipping feature transformations with a Multi-Layer Perceptron (MLP).

The network first generates flipped features, passes the input through it, and then passes the result through a fully connected MLP.

This class is only available for 2D inputs on the unit square.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Other keyword arguments including:
- nb_features (optional): Number of features generated by EnhancedFeatureNet.
- type_feature (optional): Type of feature transformation applied (e.g., “flipped”).
- Other arguments passed to EnhancedFeatureNet and GenericMLP classes.

forward(x, with_last_layer=True)[source]¶

Compute the forward pass.

Apply the periodic transformation and pass the result through the MLP to produce the output.

Parameters:

x (Tensor) – Input tensor.
with_last_layer (bool) – Whether to include the last layer in the forward pass (default: True)

Return type:

Tensor

Returns:

Output tensor after passing through the MLP.

class GenericMultiScaleFourierNet(in_size, out_size, **kwargs)[source]¶

Bases: ScimbaModule

Combines Fourier feature transformations with a specified neural network.

The network first generates enhanced features (such as Fourier features), concatenates them with the original input, and then passes the result through a user-specified network architecture. The result is obtained as a linear combination of the Fourier networks.

Parameters:

in_size (int) – int
out_size (int) – int
**kwargs –
Additional keyword arguments:
- means (list[float], default=[0.0]): Initialize the weights of the EnhancedFeatureNet layers
- stds (list[float], default=[1.0]): Initialize the weights of the EnhancedFeatureNet layers
- nb_features (int, default=1): The number of features generated by the EnhancedFeatureNet
- type_feature (str, default=”fourier”): The type of feature transformation applied
- Other keyword arguments are passed to the EnhancedFeatureNet and to whichever network class is specified by the user.

Learnable Parameters:

features (EnhancedFeatureNet): A network that generates enhanced features such as Fourier features.
net: (ScimbaModule): A neural network that processes the input and features.

re_init_features(means, stds)[source]¶

Reinitialize the weights of the EnhancedFeatureNet layer.

Use a normal distribution with the specified mean and standard deviation.

Parameters:

means (list[float]) – List of mean values for the normal distribution.
stds (list[float]) – List of standard deviations for the normal distribution.

forward(x, with_last_layer=True)[source]¶

Compute the forward pass.

Apply the feature transformation, concatenate the features with the original input and pass the result through the neural network to produce the output.

Parameters:

x (Tensor) – Input tensor.
with_last_layer (bool) – Whether to include the last layer in the forward pass. (default: True)

Return type:

Tensor

Returns:

Output tensor after passing through the neural network.

class MultiScaleFourierMLP(in_size, out_size, **kwargs)[source]¶

Bases: GenericMultiScaleFourierNet

A linear combination of Fourier feature transformations with MLP.

The networks first generate enhanced features (such as Fourier features), concatenate them with the original input, and then passes the result through a fully connected MLP.

The result is obtained as a linear combination of the Fourier networks.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs –
Additional keyword arguments:
- means (list[float], default=[0.0]): Initialize the weights of the EnhancedFeatureNet layers
- stds (list[float], default=[1.0]): Initialize the weights of the EnhancedFeatureNet layers
- nb_features (int, default=1): The number of features generated by the EnhancedFeatureNet
- type_feature (str, default=”fourier”): The type of feature transformation applied

class MultiScaleFourierResNet(in_size, out_size, **kwargs)[source]¶

Bases: GenericMultiScaleFourierNet

Combine Fourier feature transformations with Residual Networks (ResNet).

The networks first generate enhanced features (such as Fourier features), concatenate them with the original input, and then passes the result through a fully connected ResNet. The result is obtained as a linear combination of the Fourier networks. It is a specialization of the generic class GenericMultiScaleFourierNet.

Parameters:

in_size (int) – The input dimension (number of features in the input tensor).
out_size (int) – The output dimension (number of features in the output tensor).
**kwargs – Additional keyword arguments.