Research Areas

Physics of Shapes

Shapes play a critical role in physics. Microstructural morphology determines the mechano-chemical behavior of a material; the aerodynamics of a vehicle is affected by the curves of the car silhouette; certain nano-scale surface texture patterns can make the surface repel or absorb liquids; just to list a few. In the Visual Intelligence Laboratory, we are interested in discovering linkages between shapes and various physical phenomena using the tools of computational differential geometry, computer vision, and machine learning.

Publications & Presentations

Websites & Other Resources

DMREF: Physics-Informed Meta-Learning for Design of Complex Materials ($1.64M + $0.1M Supplement)
Funding SourceNational Science Foundation (NSF)
RolePrincipal Investigator
DescriptionThe goal of this project is to develop meta-learning algorithms to extend machine learning models trained on one specific material system to other broader range of material systems (different species or length/time scales).

Investigating the Mechanisms of Particle Energization in Collisionless Heliospheric Shocks ($1.2M)
Funding SourceNational Aeronautics and Space Administration (NASA)
RoleCo-Investigator (PI: Gregory Howes)
DescriptionThe goal of this project is to understand the physics of collisionless heliospheric shocks by analyzing patterns from space experiments and simulations using convolutional neural networks.

Integrating multiscale modeling and experiments to develop a meso-informed predictive capability for explosives safety and performance ($7.5M)
Funding SourceU.S. Air Force Office of Scientific Research Multidisciplinary University Research Initiative Program
RoleCo-Investigator (PI: Thomas Sewell)
DescriptionThe goal of this project is to develop machine learning capacities to predict the safety and performance characteristics of explosives.

Machine learning mesoscale structure-property-performance relationships of energetic materials for multiscale modeling of shock-induced detonation ($899,563)
Funding SourceU.S. Air Force Office of Scientific Research
RoleCo-Principal Investigator (PI: H.S. Udaykumar)
DescriptionIn this project, we are looking to learn mesoscale structure-property-performance relationships of energetic materials, such as explosives, propellants, and fuels, using deep neural networks.


Geometric Deep Learning

Convolutional neural networks (CNN) have demonstrated unprecedented success in a variety of visual cognition tasks. Such a success, however, has been concentrated in mostly computer vision and image/signal processing applications, where one can enjoy a nice, grid-like structure of the data set (signal=1D grid of amplitudes, image=2D grid of pixel values, and such). Meanwhile, there is a large number of problems that could not benefit much from such powerful CNNs due to the non-Euclidean nature of the data set. For instance, graphical models, such as computer graphics objects or computer-aided design (CAD) parts, often exist as a boundary representation (B-rep) model in which the geometry of the model is represented by a thin, arbitrary-shaped boundary surface (2D manifold), rather than a grid-like representation. Hence, there is no canonical way of representing such data in a tensor format as expected by CNNs and, thus, the standard CNNs cannot analyze such data. Point clouds from 3D scanners or LiDAR systems, graphs from social network data, molecular structures of pharmaceutical compounds, protein foldings, and many other non-image-type geometric data fall into this category. At the Visual Intelligence Laboratory, we aim to develop mathematical foundations and scalable algorithms to expand CNNs to a variety of different geometric domains. Many scientific studies may benefit from such new capabilities, including computer-aided design and manufacturing (CAD/CAM), computer graphics, 3D sensing systems on autonomous vehicles, the brain mapping problem in neuroscience, and computational mechanics, just to list a few.

Websites & Other Resources


Digital Human Modeling

People produce a variety of visual cues both consciously and unconsciously. These visual cues include facial expressions, eye movements, body gestures, motion, and such. When provided with a context, we can sense the emotional and physical states of a person from these cues, such as joy, fatigue, anger, drowsiness, excitement, etc. For example, car companies these days are keen to integrate technology to detect such visual cues into their advanced driver assistance systems (ADAS) to enhance safety and convenience. For another example, many studies report that patterns of body motion are correlated and are thus indicative of the person’s performance. Furthermore, sometimes, subtle changes in such motion patterns can be a precursor to a musculoskeletal injury or a similar kind of ergonomic hazard. Therefore, accurate sensing and detection of such visual cues can become critical in many applications. The Visual Intelligence Laboratory is pioneering the research to detect and understand visual cues exhibited by people under different contexts such as driving, manufacturing, etc., by combining state-of-the-art computer vision technologies with their geometric data analysis expertise.

A study on user experience in autonomous driving scenarios ($221,398)
Funding SourceHyundai Motor Company
RolePrincipal Investigator

Driver State Detection via Deep Learning ($250,000)
Funding SourceAisin Technical Center of America
RolePrincipal Investigator

Developing Connected Simulation to Study Interactions between Drivers, Pedestrians, and Bicyclists ($1.8M)
Funding SourceU.S. Department of Transportation Federal Highway Administration (FHWA) Exploratory Advanced Research Program
RoleCo-Principal Investigator (PI: Daniel V. McGehee)
DescriptionThe goal of this project is to develop virtual reality technologies for interactive simulation among subjects within different simulation environments to study road user behaviors for enhancing road safety.


Medical Shape Analysis

By studying the shapes and textures of medical images, it is possible to detect disease and predict clinical outcomes more accurately. To this end, we are interested in characterizing the shapes and textures of diseases in medical images (X-rays, CT, PET, MRI, etc.) and correlating them with other clinical parameters.

Water and chloride movement in neurons during seizure activity ($2,122,253)
Funding SourceNational Institutes of Health
Period09/2020-06/2025 (Phase I)
RoleCo-Investigator (PI: Joseph Glykys)
DescriptionThis project aims to study water and chloride movement in neurons during seizure and the corresponding changes of neuronal morphology using deep learning.

NSF Convergence Accelerator - Track D: ImagiQ: Asynchronous and Decentralized Federated Learning for Medical Imaging ($999,770)
Funding SourceNational Science Foundation
Period09/2020-05/2021 (Phase I)
Project Pagelink
RolePrincipal Investigator
DescriptionThe goal of this project is to innovate medical imaging by collaborative model development and validation. To eliminate the hurdle of direct patient data sharing, we advocate a novel decentralized, asynchronous federated learning (FL) scheme.