Publications
(also on Google Scholar)
Information decomposition in complex systems via machine learning
KAM & Dani Bassett
Central to the complexity of complex systems is the sea of variation of all the components that hides information about organization at larger scales. Taking inspiration from lossy compression, where only the most valuable information is preserved, we develop a method powered by machine learning that identifies the most relevant variation in systems with many components, and apply it find the information about imminent deformation in a simulated glass (right) and Boolean circuitry.
Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck
KAM & Dani Bassett
Physical Review Letters (Accepted) | arxiv | project page
A measurement of any general system gives you information about its state. A well-designed measurement will get you information you care about with minimal wasted precision. It turns out that for chaotic systems, there is a precise notion of a perfect measurement: one that captures all of the information generated by the stretching of space with minimal precision. It further turns out that these measurements can be extremely coarse, as if you took a crayon to the system’s attractor in order to assign measurement outcomes to different regions. Shown on the left are two colorings we optimized with machine learning, which are able to capture almost all of the information generated by the chaos.
Interpretability with full complexity by constraining feature information
KAM & Dani Bassett
A common route to making black box machine learning interpretable is to simplify the box, for instance by using only comprehensible components that interact in simple ways. In contrast, here we demonstrate how to track the information going into the black box, and show how much interpretability that can bring. The datasets are all tabular in this work; on the left is a classic one where the task is to regress the number of bikes rented given different aspects of weather, time, and the day. By tracing the information, we can leave the black box untouched and just look where it looks, down to specific distinctions among feature values.
The Distributed Information Bottleneck reveals the explanatory structure of complex systems
KAM & Dani Bassett
Where is the information in a relationship between an input and an output? What are the most important parts of the input? We show how to address these fundamental questions by turning a relationship into a scheme of approximations that gradually reduce the amount of information utilized while maximizing fidelity. Smearing a relationship out in this way brings interpretability to deep learning and illuminates the nature of complex systems, without requiring dataset modifications or a neural network architecture more exotic than a VAE.
Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold
KAM*, Carlos Esteves*, Varun Jampani, Srikumar Ramalingam, Ameesh Makadia
ICML 2021 | project page | code | Symmetric Solids dataset
Probability distributions over the space of 3D rotations can be tough to work with in machine learning applications, generally due to the problem of normalization. We introduce a framework which allows maximal expressivity by representing the distributions implicitly, with a fully connected network. This boosts interpretability and the power to express uncertainty and multiple hypotheses, as for symmetric shapes. We release a new pose estimation dataset where the ground truths are multi-valued, and a novel method of visualizing distributions on SO(3) without having to marginalize.
KAM, Varun Jampani, Srikumar Ramalingam, Ameesh Makadia
CVPR 2022 (Selected for Oral) | code
What information can be extracted from grouped data (and no annotations)? We show that to find correspondence across groups requires focusing on certain factors of variation and excluding others, which can be used to learn useful representations.
Importantly, this offers a new route to representations which are either completely inaccessible to current representation learning methods (e.g. SimCLR), or only accessible through much slower approaches which learn a complete description of the data before partitioning.
The gif shows the evolution of representations of MNIST digits during training, where images are grouped by digit, and style information is pulled out (primarily slant and stroke thickness).
Learning ABCs: Approximate Bijective Correspondence for isolating factors of variation
KAM, Jonathon W. Kruppe, and Heinrich M. Jaeger
Physical Review Letters (2020) (Editors’ Suggestion)
A packing of glass spheres ‘remembers’ its past compression and plays the memory out seconds or even minutes into the future through stress relaxation. This behavior, found in other disordered systems like crumpled paper, has been modeled as the response of a population of simple elements with a specific distribution of timescales. The data suggest this isn’t quite right, especially as more memories are stored.
Memory in nonmonotonic stress relaxation of a granular system
KAM, Karin A. Dahmen, Heinrich M. Jaeger
What would change about earthquakes if all rocks were tetrahedra? cubes? Here we 3D-printed hundreds of thousands of millimeter-sized particles of different shapes to test current theory about the universality of plastic deformation. Interestingly, particle shape changes the maximum size of plastic events, but does not appear to influence the power law exponent of the distribution of event sizes. The exponent is found to be more in line with a mean-field treatment of plastic deformation than with models which include a rigorous treatment of stress redistribution following a plastic event.
Transforming Mesoscale Granular Plasticity Through Particle Shape
KAM, Arthur K. MacKeith, Leah K. Roth, Heinrich M. Jaeger
Primarily experimental heft — we finally accomplished segmentation of computed tomography (CT) scans of 3D-printed particles, and measured the surface erosion at the micron scale for hundreds of particles over dozens of compression experiments.
At left is a cross section of a CT scan of lens-shaped particles, before and after segmentation.
The intertwined roles of particle shape and surface roughness in controlling the shear strength of a granular material
KAM, Leah Roth, Dan Peterman, Heinrich Jaeger
Architectural Design (2017) | pdf
Usually granular materials require some sort of cohesion or confinement to build anything more fancy than a mound. By choosing the right shape of particle (here, a twisted-Z), you get a granular material that can can be poured into a mold to form rigid structures by self-interlocking.
All 12,000 Zs were cut from planks made of recycled plastic soap bottles, cooked in a kitchen oven for 18 minutes at 275 F, and twisted into shape. All 12,000.
Aleatory Construction Based on Jamming: Stability Through Self‐Confinement
KAM, Nikolaj Reiser, Darius Choksy, Clare E. Singer, Heinrich M. Jaeger
This work combined simulations and experiments on 3D-printed Z particles to try to understand how this particle shape can form loadbearing structures without any confinement. A neat insight is the necessity of axial pre-loading before removing confinement, offering mechanical control over whether the structure will be rigid or crumble.
The gif on the left shows a LAMMPS simulation where the (invisible) confinement is removed after pouring the particles.