Research Areas

Geometric Deep Learning

Convolutional neural networks (CNN) have demonstrated an unprecedented success in a variety of visual cognition tasks. Such a success, however, has been concentrated in mostly computer vision and image/signal processing applications, where one can enjoy a nice, grid-like structure of the data set (signal=1D grid of amplitudes, image=2D grid of pixel values, and such). Meanwhile, there is a large number of problems that could not benefit much from such powerful CNNs due to the non-Euclidean nature of the data set. For instance, graphical models, such as computer graphics objects or computer-aided design (CAD) parts, often exist as a boundary representation (B-rep) model in which the geometry of the model is represented by a thin, arbitrary-shaped boundary surface (2D manifold), rather than a grid-like representation. Hence, there is no canonical way of representing such data in a tensor format as expected by CNNs and, thus, the standard CNNs cannot analyze such data. Point clouds from 3D scanners or LiDAR systems, graphs from social network data, molecular structures of pharmaceutical compounds, protein foldings, and many other non-image type geometric data fall into this category. At the Visual Intelligence Laboratory, we aim to develop mathematical foundations and scalable algorithms to expand CNNs to a variety of different geometric domains. Many scientific studies may benefit from such new capability, including computer-aided design and manufacturing (CAD/CAM), computer graphics, 3D sensing systems on autonomous vehicles, the brain mapping problem in neuroscience, and computational mechanics, just to list a few.

Research Projects

Shape Matters: Evidences from Machine Learning on Body Shape-Income Relationship
Funding SourcePending
Project Pagelink
CollaboratorSuyong Song, PhD (Department of Economics)

ZerNet: Zernike Convolutional Neural Networks
Funding SourceOld Gold Summer Fellowship
Project Pagelink

Allen Brain Atlas Project
Funding SourcePending
CollaboratorThomas Nickl-Jockschat, MD (Department of Psychiatry)


Physics of Shapes

Shapes play a critical role in physics. Microstructural morphology determines mechano-chemical behavior of a material; aerodynamics of a vehicle is affected by the curve of a car silhouette; certain nano-scale texture patterns on a surface can create a liquid behavior that doesn’t exist in nature; just to list a few. In the Visual Intelligence Laboratory, we are interested in discovering linkages between shapes and various physical phenomena using the tools of differential geometry and machine learning.

Research Projects

Investigating the Mechanisms of Particle Energization in Collisionless Heliospheric Shocks ($1.2M)
Funding SourceNational Aeronautics and Space Administration (NASA)
DescriptionThe goal of this project is to understand physics of collisionless heliospheric shocks by analyzing patterns from space experiments and simulations using convolutional neural networks.

Integrating multiscale modeling and experiments to develop a meso-informed predictive capability for explosivessafety and performance ($7.5M)
Funding SourceU.S. Air Force Office of Scientific Research Multidisciplinary University Research Initiative Program
DescriptionThe goal of this project is to develop machine learning capacities to predict safety and performance characteristics of explosives.

Machine learning mesoscale structure-property-performance relationships of energetic materials for multiscale modeling of shock-induced detonation ($899,563)
Funding SourceU.S. Air Force Office of Scientific Research
DescriptionIn this project, we are looking to learn mesoscale structure-property-performance relationships of energetic materials, such as explosives, propellants, and fuels, using deep neural networks.


Human Performance Analysis

People produce a variety of visual cues both consciously and unconsciously. These visual cues include facial expressions, eye movements, body gestures, motion, and such. When provided with a context, we can sense emotional and physical states of a person from these cues, such as joy, fatigue, anger, drowsiness, excitement, etc. For example, car companies these days are keen to integrate a technology to detect such visual cues into their advanced driver assistance systems (ADAS) to enhance safety and convenience. For another example, many studies report that patterns of a body motion are correlated and are thus indicative of the person’s performance. Furthermore, sometimes, subtle changes in such motion patterns can be a precursor to a musculoskeltal injury or a similar kind of ergonomic hazard. Therefore, an accurate sensing and detection of such visual cues can become critical in many applications. The Visual Intelligence Laboratory is pioneering the research to detect and understand visual cues exhibited by people under different contexts such as driving, manufacturing, etc., by combining state-of-the-art computer vision technologies with their geometric data analysis expertise.

Research Projects

A study on user experience in autonomous driving scenarios ($221,398)
Funding SourceHyundai Motor Company

Driver State Detection via Deep Learning ($250,000)
Funding SourceAisin Technical Center of America

Developing Connected Simulation to Study Interactions between Drivers, Pedestrians, and Bicyclists ($1.8M)
Funding SourceU.S. Department of Transportation Federal Highway Administration (FHWA) Exploratory Advanced Research Program
DescriptionThe goal of this project is to develop virtual reality technologies for interactive simulation among subjects within different simulation environments in order to study the road user behaviors for enhancing road safety.


Medical Image Analysis

By studying shapes and textures of medical images, it is possible to detect disease and predict clinical outcomes more accurately. To this end, we are interested in characterizing shapes and textures of diseases in medical images (X-rays, CT, PET, MRI, etc.) and correlate them with other clinical parameters.

Research Projects

NSF Convergence Accelerator - Track D: ImagiQ: Asynchronous and Decentralized Federated Learning for Medical Imaging ($999,770)
Funding SourceNational Science Foundation
Period09/2020-05/2021 (Phase I)
Project Pagelink
DescriptionThe goal of this project is to innovate medical imaging by collaborative model development and validation. To eliminate the hurdle of direct patient data sharing, we advocate a novel decentralized, asynchronous federated learning (FL) scheme.

Lung Cancer Outcome Prediction
Funding SourcePending
CollaboratorsXiaodong Wu, PhD (Department of Electrical and Computer Engineering)
Brian J. Smith, PhD (Department of Biostatistics)
Yusung Kim, PhD, Bryan G. Allen, MD, PhD, John M. Buatti, MD (Department of Radiation Oncology)

Interstitial Lung Disease Quantification
Funding SourcePending
CollaboratorChanghyun Lee, MD, PhD (Department of Radiology)