List of Contents

1. Graph Neural Networks for Atomic Systems
2. Invariant and Equivariant Neural Networks
3. Irreducible Representaion on SO(3) and Spherical Harmonics
4. L2-norm of the Spherical Harmonics Expansion
5. Train of Thought

Graph neural networks (GNNs) are a type of neural network that are designed to process data that is represented in the form of a graph. In a graph 𝒢 = (𝒱, ℰ) , the data is represented as a set of nodes (also called vertices) i ∈ 𝒱 , and the connections between the nodes are represented as edges (i, j) ∈ ℰ. GNNs operate on graphs by passing messages between the nodes in the graph. These messages are used to update the representation of each node, aggregating the representations of its neighboring nodes. This process of message passing is repeated over multiple layers, with each layer updating the node representations based on the representations of its neighbors in the previous layer. Given feature f_i^l on node i and edge feature e_ij^l at the l-th layer, it first aggregates messages m_ij^l from neighbors and then update features on each node as follows:

m_ij^l = ϕ_m(f_i^l,f_j^l,e_ij^l)   and   f_i^l + 1 = ϕ_u(f_i^l,∑_{j ∈ 𝒩(i)}m_ij^l)

where 𝒩(i) denotes the set of neighbors of node i and ϕ_m and ϕ_u are learnable functions.

A graph permutation means re-ordering the nodes. Permutation invariance of GNNs means that the output of is not affected by order of the nodes in the input graph. This permutation invariance is ensured by aggregating functions.

They are well suited for property prediction of atomic systems because they can handle both discrete and topological structures, which are commonly found in atomic systems. There are two main ways to represent atomic systems as graphs: 2D chemical bond graphs and 3D spatial graphs. 2D Chemical bond graphs, represent atomic systems as a set of nodes, with each node representing an atom, and edges representing chemical bonds between atoms. These graphs capture the chemical structure of the system, but do not consider the spatial arrangement of the atoms. 3D spatial graphs, on the other hand, represent atomic systems as a set of nodes arranged in 3D space, with edges representing the spatial relationships between the nodes. These graphs enabled representation of the geometric relationships between atoms in a molecule or crystal, but the key challenge in moving from 2D chemical bonding graphs to 3D spatial graphs is to remain invariant or equivalent to the geometric transformations acting on atom positions, in other words, be equivariant to the Euclidean group E(3) of rigid motions (rotations, translations, and reflections) in addition to the standard permutation equivariance.

Invariant neural networks and equivariant neural networks are types of neural networks that have been proposed for use with 3D atomistic graphs.

Invariant neural networks operate on invariant information, such as distances and angles between atoms. This means that they are designed to produce the same output regardless of the orientation or position of the atomic system in space. This can be useful for predicting properties that are independent of the orientation of the atomic system, such as its energy levels or bond lengths.

Equivariant neural networks, on the other hand, operate on geometric tensors like type-L vectors. These tensors encode information about the orientation and position of the atomic system in space, and the neural network is designed to be equivariant to changes in the orientation and position of the atomic system. This can be useful for predicting properties that depend on the orientation of the atomic system, such as its vibrational frequencies or electromagnetic properties.

In previous work, invariant graph neural networks constructed for 3D atomic graphs can be divided to reply on different geometric information such as distances [SchNet,], bond angles [DimeNet] or dihedral angles [SphereNet, GemNet], and symmetry groups [Equiformer]. Our work falls into the category of exploring symmetry, more specifically irreducible representation, to reduce the dimension of representation and therefore simplify graph structure of generic GNNs.

Irreducible representations (irreps) of the special orthogonal group SO(3) describe how objects or functions can transform under rotations in 3D space.

Spherical harmonics is a generalization of form of the Fourier series, where the sines and cosines are replaced by more general functions that are defined in terms of spherical coordinates rather than Cartesian coordinates. Spherical harmonics are functions defined on the surface of a sphere that form a complete set of irreps of SO(3). This means that any function defined on the surface of a sphere can be represented as a linear combination of spherical harmonics, and these harmonics capture all of the possible rotational symmetries of the function.

In the context of quantum mechanics, the relationship between irreps and spherical harmonics can be understood as follows: each irrep of the rotation group corresponds to a unique set of spherical harmonics, which are used to describe the angular dependence of the wavefunction for a particle confined to a spherical region. Specifically, each irrep of the rotation group is characterized by a unique set of quantum numbers, the total angular momentum quantum number (l) and the z-component of the angular momentum quantum number (m), where l is the degree and m is the order of the spherical harmonics. The dimension of the irrep is equal to 2l + 1, and the spherical harmonics in the irrep are labeled by the index m, which ranges from -l to l.

Clebsch-Gordan decompositions are used to decompose data with a group symmetry, such as data on a sphere or manifold, into irreps of the symmetry group. Specifically, Clebsch-Gordan coefficients are used to decompose the tensor product of two irreps of a group into a direct sum of irreps.

The relationship between irreps of SO(3), spherical harmonics and Clebsch-Gordan decompositions can be understood as follows: spherical harmonics provide a generic basis for representing functions on the surface of a sphere that are invariant under rotations. When data on a sphere is decomposed using Clebsch-Gordan decompositions, the resulting decomposition can be expressed in terms of spherical harmonics, which capture the rotational symmetries of the data. This makes it possible to use standard neural networks to process the decomposed data, allowing for more efficient and effective analysis of the data.

The L2-norm of a spherical harmonics expansion is the sum of the squares of the coefficients in the expansion. It is a measure of the smoothness of the expansion and is often used to regularize the expansion, especially in inverse problems where the expansion is used to approximate some underlying field.

||a||² = ∑_l ∑_m |a_lm|²

where l is the degree, m is the order and a_lm is the coefficient of the spherical harmonics expansion.

The L2-norm can be interpreted as a generalization of the concept of the Euclidean norm (length), for multi-dimensional vectors. In the case of spherical harmonic expansion, the expansion coefficient vector is a multi-dimensional vector in a space where each coefficient a_lm corresponds to a dimension. The L2-norm is a measure of the length of the expansion vector in this high-dimensional space.

The L2-norm of a spherical harmonics expansion has properties that make it useful for analyzing spherical harmonics expansions:

1. It is always non-negative: ||a||² >= 0.
2. It is equal to zero if and only if all of the coefficients a_lm are zero: ||a||² = 0 if and only if a_lm = 0 for all l and m.
3. It is invariant under rotation: rotating the field represented by the expansion does not change the L2-norm.
4. It is additive: the L2-norm of the sum of two expansions is equal to the sum of the L2-norms of the individual expansions.