The Fab on the Beam: From Learning Physics to Automated Experiments in Electron Microscopy: Nano Seminar series

Dr. Sergei V. Kalinin, ORNL, Center for Nanophase Material Sciences
Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. However, the constantly emerging question is how to match the correlative nature of classical ML with hypothesis-driven causal nature of physical sciences. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment (AE) in imaging.
In this presentation, I will discuss recent progress in machine learning applications in electron microscopy, ranging from feature extraction, learning generative physical models, and to physics discovery via active learning.
The applications of classical deep learning methods in streaming image analysis are strongly affected by the out of distribution drift effects, and the approaches to minimize though are discussed. I further present invariant variational autoencoders as a method to disentangle affine distortions and rotational degrees of freedom from other latent variables in imaging and spectral data. The analysis of the latent space of autoencoders further allows establishing physically relevant transformation mechanisms. Extension of encoder approach towards establishing structure-property relationships will be illustrated on the example of plasmonic structures.
I will briefly discuss the transition from correlative ML to physics discovery, incorporating prior knowledge and yielding generative physical models of observed phenomena. Finally, I illustrate transition from post-experiment data analysis to active learning process. Here, the strategies based on simple Gaussian Processes often tend to produce sub-optimal results due to the lack of prior knowledge and very simplified (via learned kernel function) representation of spatial complexity of the system. Comparatively, deep kernel learning (DKL) methods allow us to realize both the exploration of complex systems towards the discovery of structure-property relationship, and enable automated experiment targeting physics (rather than simple spatial feature) discovery. The latter is illustrated via experimental discovery of the edge plasmons in STEM/EELS and 4D STEM exploration of twisted bilayer graphene structures.
****
Sergei Kalinin did his PhD at Penn and postdoc at ORNL. Early career awards include the Blavatnik, the PECASE, IEEE and 4 R&D 100 Awards. He is on the editorial boards of several nano journals and co-authored >650 publications.