Optimization of ordinary differential equations (ODE) solutions using modified recurrent neural networks
Abstract views: 52
/ 
PDF downloads: 28
Keywords:
S-Curve Function; Modified Recurrent Neural Networks (mRNN); Ordinary Differential Equations (ODE); Alternative Activation Function (AAF); Identity Function (IdFn).Abstract
This paper introduces a hybrid approach for solving ordinary differential equations (ODE) using modified recurrent neural networks (mRNNs). The approach combines mRNNs with novel optimization techniques. Crucially, when training an mRNN, training data points should be selected from the open interval (a, b) to avoid training the network with the boundary points. This approach reduces computational errors by avoiding boundary region training. Furthermore, we propose a transformation that maps training points from a potentially broader interval [a, b] into corresponding points within the open interval (a, b), before training. This allows the network to be trained on points that are similar in the open interval, which leads to improved accuracy. The proposed model demonstrates higher
accuracy compared to existing mRNN models. A numerical example and corresponding simulations demonstrate the mathematical effectiveness of this approach.
References
M. Abadi et al., TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.
A. Bachouch et al., Deep neural networks algorithms for stochastic control problems on finite horizon: Numerical applications, arXiv:1812.05916, 2020.
C. Basdevant et al., Spectral and finite difference solutions of the Burgers equation, Comput. & Fluids 14 (1986), 23–41.
A. G. Baydin et al., Automatic differentiation in machine learning: A survey, J. Mach. Learn. Res. 18 (2018), 43 Paper No. 153.
C. Beck and A. Jentzen, Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations, J. Nonlinear Sci. 29 (2019), 1563–1619.
C. Beck et al., Solving stochastic differential equations and Kolmogorov equations by means of deep learning, arXiv:1806.00421, 2018.
C. Beck et al., Deep splitting method for parabolic PDEs, arXiv:1907.03452, 2019.
C. Beck et al., Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations, J. Numer. Math. 28 (2020), 197–222.
C. Beck et al., An overview on deep learning-based approximation methods for partial differential equations, arXiv:2012.12348, 2020.
R. Bellman, Dynamic programming, Princeton University Press, Princeton, NJ, 1957.
J.-D. Benamou, B. D. Froese, and A. M. Oberman, Two numerical methods for the elliptic Monge-Ampère equation, M2AN. Math. Modell. Numer. Anal. 44 (2010), 737–758.
C. Bender and R. Denk, A forward scheme for backward SDEs, Stochastic Process. Appl. 117 (2007), 1793–1812.
C. Bender, N. Schweizer, and J. Zhuo, A primal–dual algorithm for BSDEs, Math. Finance 27 (2017), 866–901.
P. Beneventano et al., High-dimensional approximation spaces of artificial neural networks and applications to partial differential equations, arXiv:2012.04326, 2020.
Y. Bengio, Learning deep architectures for AI, Now Publishers Inc., 2009.
J. Berg and K. Nyström, A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing 317 (2018), 28–41.
J. Berner, M. Dablander, and P. Grohs, Numerically solving parametric families of high-dimensional Kolmogorov partial differential equations via deep learning, Adv. Neural Inform. Process. Syst. 33 (2020).
