Collection of open machine learning papers
| Design choice | Alternatives covered below |
|---|---|
| Input domain | Sets, graphs, point clouds, surfaces, spheres |
| Symmetry and output | Permutations, rigid motions, local frame changes; invariant scalars or transforming outputs |
| Hidden features | Scalars, vectors, higher-order tensors |
| Interactions | Pooling, attention, local messages, spectral filters, diffusion |
| Scale and sampling | Hierarchical pooling, mesh resolution, discretization transfer |
| Latent geometry | Euclidean or hyperbolic representations |
Geometric blueprint (2021 review). Organizes architectures by domain geometry, symmetry and scale: grids, groups, graphs and manifolds. Invariant outputs stay fixed under a group action; equivariant outputs transform predictably with the input.
When symmetry improves generalization (2021). Elesedy and Zaidi prove a benefit for linear models when the target and distribution obey the specified compact-group symmetry. This supports matched structural priors, not the claim that every imposed symmetry improves every task.
Scalar invariant theory (2021). Uses invariant scalar coefficients and geometric inputs to parameterize equivariant vector functions. SO(d) and O(d) differ on reflections; this does not establish universality of local scalar message passing.
Permutation symmetry versus graph expressivity (2018). GIN matches the 1-WL test under stated aggregation assumptions. Permutation invariance is necessary for graph-level predictions but does not ensure that different graph structures can be distinguished.
Geometric Weisfeiler–Leman (2023). Tests discrimination of geometric graphs modulo symmetry. Equivariant propagation can distinguish arrangements lost by locally invariant aggregation; conclusions depend on retained geometry, tensor order and graph connectivity.
Invariant completeness (2024). GeoNGNN and specified DimeNet, GemNet and SphereNet variants achieve E(3)-completeness with full connectivity and sufficient depth. Equivariant hidden features are not necessary; practical cutoff implementations do not inherit the theorem.
Deep Sets (2017). Applies an elementwise map, sums latent features and maps the result to an output. The construction is permutation invariant; representation theorems require assumptions about the input domain and latent space.
Limits of sum-decomposition (2019). For continuous representations of scalar sets, universal sum-decomposition needs latent dimension at least the maximum set size. Invariance alone does not make a fixed-width pooled representation universally expressive.
Set Transformer (2018). Uses attention to model interactions among set elements and invariant pooling for set outputs. Inducing points reduce attention cost to linear in set size when their number is fixed.
PointNet (2016). Combines shared pointwise features with symmetric max pooling. Handles unordered point clouds directly; permutation invariance does not imply exact rotation invariance, and local geometric interactions are limited.
PointNet++ (2017). Applies PointNet hierarchically to metric neighborhoods and aggregates multiple scales. Adds local geometric structure and addresses varying sampling density, extending the global pooling of PointNet.
Point Transformer (2020). Uses local vector attention with relative position encoding for point-cloud classification and segmentation. Position-aware attention models local geometry; it does not by itself enforce full rotation equivariance.
Spectral graph filters (2013). Learn filters in the graph Laplacian’s eigenbasis. ChebNet (2016) uses polynomial filters with local graph support, avoiding eigendecomposition during filtering. Polynomial degree controls the receptive field.
Wavelet scattering (2011). Cascades wavelets, modulus and averaging. Provides deformation-stability bounds under stated regularity assumptions; exact translation invariance is an infinite-window limit. Stability to deformations is distinct from exact symmetry.
Group-equivariant CNNs (2016). Extends convolution and weight sharing to group actions, including discrete rotations and reflections. Equivariance is built into the layer; augmentation instead asks training to learn behavior across transformed examples.
E(2)-steerable CNNs (2019). Solves kernel constraints for planar rotations, reflections and their subgroups using group representations. Feature types specify how channels transform; scalar-only channels are one choice, not the general construction.
3D steerable CNNs (2018). Builds kernels from an analytic steerable basis to map scalar, vector and tensor fields equivariantly under rigid motions. Continuous equivariance and errors introduced by voxel sampling should be evaluated separately.
Spherical CNNs (2018). Defines rotation-equivariant correlation on spherical signals and computes it with generalized Fourier transforms. Avoids the varying distortions produced by flattening a sphere into a planar image.
Tensor field networks (2018). Uses spherical-harmonic filters to propagate scalar, vector and higher-order tensor features on 3D points. Equivariance specifies their transformation under rotations, translations and point permutations.
SE(3)-Transformer (2020). Combines invariant attention weights with equivariant value features. Models interactions on 3D graphs and point clouds while preserving roto-translation laws; attention and equivariance play different roles.
E(n)-equivariant GNNs (2021). Uses invariant distances and coordinate-difference updates without higher-order tensor features. Enforces translations, rotations and reflections; reflection-invariant scalar outputs cannot distinguish mirror-related inputs.
Geodesic CNNs (2015). Constructs local geodesic polar patches on surfaces and learns filters for shape descriptors. Intrinsic neighborhoods support shape correspondence; orientation choices must be handled when defining directional filters.
MoNet (2016). Uses learned weighting functions over local pseudo-coordinates to generalize convolution on graphs and manifolds. Coordinates encode domain geometry; several earlier geometric convolutions arise as special cases.
MeshCNN (2018). Convolves on mesh edges and pools through learned edge collapses. Uses mesh connectivity and geometric edge features; pooling changes the discretization according to the task rather than using fixed grid subsampling.
DiffusionNet (2020). Learns diffusion times for surface communication, with pointwise networks and gradient features. Laplacian-based diffusion connects it to spectral filtering. Transfer across meshes and point samples does not imply invariance to arbitrary shape changes.
Gauge-equivariant CNNs (2019). Makes feature transformations consistent under changes of local coordinate frame on a manifold. The icosahedral implementation applies this to spherical signals; gauge changes are coordinate choices, not physical deformations.
Gauge-equivariant mesh CNNs (2020). Uses anisotropic kernels and parallel transport to compare orientation-carrying features across vertices. Extends isotropic graph aggregation with intrinsic directional information while respecting local frame changes.
Frame averaging (2021). Averages a backbone over a covariantly chosen set of transformations. Can enforce exact symmetry without redesigning each layer; computational cost depends on frame size and continuity is a separate property.
Continuity of frames (2024). Dym et al. show that common unweighted frame constructions can turn smooth functions into discontinuous equivariant ones. Weighted frames preserve continuity under their conditions, qualifying symmetry guarantees based only on averaging.
Approximate equivariance (2022). Relaxes weight sharing and steerable-kernel constraints. Improves on unconstrained and strictly equivariant baselines in smoke-plume and jet-flow forecasts; ocean-current results are similar across models after tuning.
Equivariant molecular diffusion (2022). Jointly denoises atom types and 3D coordinates using an equivariant network. Coordinates and noise lie in a zero-center-of-gravity subspace; centering handles translations, while equivariant denoising preserves rotational symmetry.
Riemannian flow matching (2023). Trains generative vector fields on manifolds. Closed-form geodesics permit simulation-free training on simple geometries; general geometries use spectral distances and simulated conditional paths. Sampling still integrates the learned flow.
SchNet (2017). Uses continuous distance-based filters for atoms off a regular grid. Predicts invariant energies and derives forces from the energy surface; distance-only local representations constrain the geometric information available.
DimeNet (2020). Associates messages with directed edges and combines distances with angles using radial and spherical bases. Angular information distinguishes environments that distance-only neighborhood aggregation can confuse.
PaiNN (2021). Propagates scalar and vector features to learn equivariant atomic representations. Supports tensorial property prediction and molecular spectra; vector features retain directional information absent from purely invariant hidden states.
NequIP (2021). Uses E(3)-equivariant tensor features to learn interatomic energies and forces from quantum calculations. Its data-efficiency results concern tested systems and reference calculations; symmetry does not remove chemical-domain coverage requirements.
MACE (2022). Builds higher-body-order equivariant messages. Increasing interaction order can reduce required message-passing depth; body order, geometric tensor order and number of graph layers describe different forms of model capacity.
Allegro (2022). Learns strictly local equivariant many-body representations without message passing between atomic environments. Locality supports parallel simulation, while the cutoff limits which interactions the representation can directly model.
Controlled symmetry comparisons (2025). Rapidash compares equivariant and less constrained convolutions. Gains depend on task geometry; reference frames improve the tested models. Symmetry breaking through equivariant frames differs from breaking equivariance.
Force error versus molecular simulation (2022). Across water, molecules, a peptide and materials, low force error does not reliably predict stable trajectories or accurate observables. Evaluate rollout stability and system-specific statistics alongside held-out force errors.
Smooth potentials: eSEN (2025). Ablations find neighbor caps, missing cutoff envelopes and discretized representations can impair energy conservation despite similar test errors. Conservation-tested models show stronger error–property correlations in the studied systems.
Open Catalyst 2020 (2020). Tests energies, forces and relaxation for adsorbates on catalyst surfaces. Splits separate composition and adsorbate generalization; accuracy on configurations of familiar molecules is a different evaluation target.
Matbench Discovery (2023). Evaluates thermodynamic-stability screening from unrelaxed crystal structures. Accurate energy regressors can still produce false positives near the convex-hull boundary; discovery precision and recall measure utility that average energy error misses.
Graph neural networks · Scientific machine learning · Computer vision · Feature extraction · Generalization
Star Issue