A Universal Approximation Property for Abelian Group and Semigroup Operations

Abelian Neural Networks

We study the problem of modeling a binary operation that satisfies some algebraic requirements.We first construct a neural network architecture for abelian group operations and derive a universal approximation property.Then, we extend it to abelian semigroup operations using the characterization of associative symmetric polynomials.For each case, by repeating the binary operations, we can represent a function for multiset input thanks to the algebraic structure.Naturally, our multiset architecture has size-generalization ability, which has not been obtained in existing methods.We train our models over fixed word embeddings and demonstrate improved performance over the original word2vec and another naive learning method.