Embedding Recurrent Neural Networks into Predator-Prey Models

Joos Vandewalle , Yves Moreau

We investigate how changes of coordinates allow us to find equivalences between different neural networks. We have shown how to embed different types of neural networks (having the hyperbolic tangent or the logistic sigmoï d as activation function) into predator-prey models (also called Lotka-Volterra systems) of the form:

$$z_i = \lambda_i z_i + z_i \sum_{j=1}^{m}M_{ij}z_j {\mbox\ with\ }i=1,\cdots,m$$

The Lotka-Volterra system will be of higher dimension than the original system but its n first variables will have a behavior equivalent or identical to that of the original system (if n) was the number of variables of the original system. Such a Lotka-Volterra representation of neural networks is of interest for several reasons. Lotka-Volterra systems are a central tool from mathematical biology used for the modeling of competition between species and a classical subject of dynamical system theory. We expect that such a representation will allow us to apply methods and results from the study Lotka-Volterra systems to neural networks. Such systems have a simple quadratic nonlinearity (not all quadratic terms are possible in each equation), which might be easier to analyze than the sigmoïdal saturations. Since we know that dynamical neural networks can approximate arbitrary dynamical systems for any finite time because of their equivalence with dynamical single-hidden-layer perceptrons, our result also serves as a simple proof that Lotka-Volterra systems enjoy the same property; and we will investigate them further from that point of view.