ICERM workshop on scientific machine learning
Some highlights from the January, 2019 ICERM Workshop on Scientific Machine Learning
Here is the workshop home page.
A general theme of this meeting was that neural networks give rise to a certain class of functions, and one can try to approximate general functions by this restricted class. A considerable number of talks talked about using neural networks as a tool to approximate solutions to various types of dynamical systems.
Hanin’s talk on likelihood of exploding gradients in RelU networks.
This talk uses random matrix theory to prove a result that describes how increasing the depth/width of a neural network influences the likelihood of “exploding or vanishing gradients.”
See Boris Hanin, Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients.
Petersen’s talk on function theoretic properties of functions represented by Neural Networks.
This talk shows that the class of functions computed by neural networks, viewed inside the standard Banach spaces like $L^{p}$, does not have good properties.
See Philipp Petersen, Topological properties of the set of functions generated by neural networks of fixed size.
Carlberg’s talk on reduction of order in numerical PDE’s using machine learning.
This talk explained how ideas from neural networks could be applied to improve very large scale computational solutions to PDE’s. It was hard to follow in detail but very impressive.
See the many papers of Kevin Carlberg.
Le Song’s talk on ODE methods for sequential Bayesian analysis
See Le Song.
Gregory Valiant’s talk on what can be learned.
See Gregory Valiant, Learning Populations of Parameters.
Valiant discusses how empirical probabilities can be unreliable indicators of the true distribution, and what you can do about this.
He also discussed Spectrum Estimation from Samples. This is about the eigenvalues of the covariance matrix of a distribution.
See also Karoui 2008, Ledoit/Wolf 2015 – ‘Inverting Marcenko-Pastur law via discretized optimization.’