State Space Representations of Deep Neural Networks

This paper deals with neural networks as dynamical systems governed by finite difference equations. It shows that the introduction of k-many skip connections into network architectures, such as residual networks and additive dense networks, define kth order dynamical equations on the layer-wise transformations.