Geometric Deep Learning

Convolutional neural networks (CNN) have demonstrated an unprecedented success in a variety of visual cognition tasks. Such a success, however, has been concentrated in mostly computer vision and image/signal processing applications, where one can enjoy a nice, grid-like structure of the data set (signal=1D grid of amplitudes, image=2D grid of pixel values, and such). Meanwhile, there is a large number of problems that could not benefit much from such powerful CNNs due to the non-Euclidean nature of the data set. For instance, graphical models, such as computer graphics objects or computer-aided design (CAD) parts, often exist as a boundary representation (B-rep) model in which the geometry of the model is represented by a thin, arbitrary-shaped boundary surface (2D manifold), rather than a grid-like representation. Hence, there is no canonical way of representing such data in a tensor format as expected by CNNs and, thus, the standard CNNs cannot analyze such data. Point clouds from 3D scanners or LiDAR systems, graphs from social network data, molecular structures of pharmaceutical compounds, protein foldings, and many other non-image type geometric data fall into this category. At the Visual Intelligence Laboratory, we aim to develop mathematical foundations and scalable algorithms to expand CNNs to a variety of different geometric domains. Many scientific studies may benefit from such new capability, including computer-aided design and manufacturing (CAD/CAM), computer graphics, 3D sensing systems on autonomous vehicles, the brain mapping problem in neuroscience, and computational mechanics, just to list a few.

Research Projects

• ZerNet: Zernike Convolutional Neural Networks (Project Page)
• Funding Source: Old Gold Summer Fellowship

• Allen Brain Atlas Project
• Collaborator: Thomas Nickl-Jockschat, MD (Department of Psychiatry)

Digital Human Modeling & Analysis

Each human body can be described through a set of numerical parameters which act similarly to one’s fingerprint. These body shape parameters can be used in a multitude of different scenarios. They can be used in movies, video games, virtual reality, and more to easily create highly realistic avatars and 3D models. These sets can also be analyzed to learn more about how people’s body type is related to their socioeconomic status. We use computational geometry and machine learning techniques to extract such numerical parameters from 3D scans, photographs, etc.

Research Projects

• Developing Connected Simulation to Study Interactions between Drivers, Pedestrians, and Bicyclists (U.S. DOT 693JJ31750016)
• Funding Source: U.S. Department of Transportation (DOT), Federal Highway Administration (FHWA) Exploratory Advanced Research (EAR) Program
• Funding Period: October 2017 - September 2019
• Funding Amount: $1,862,640 • Principal Investigator: Daniel V. McGehee • Role: Co-Principal Investigator (% budget:$XXXXXX)
• Summary: The goal of this project is to develop virtual reality technologies for interactive simulation among subjects within different simulation environments in order to study the road user behaviors for enhancing road safety.

• Shape Matters (Project Page)
• Project Period: 2018 - Present
• Collaborator: Suyong Song, PhD (Department of Economics, University of Iowa)

Autonomous Human-centered Vehicle Systems

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce felis justo, posuere ut lobortis et, viverra in ex. Donec nec ultricies eros. Sed mollis vestibulum scelerisque. Aenean non mi in tellus aliquam mollis sed sed felis. Nam purus purus, hendrerit eu nisi ut, mollis vehicula tortor. Ut sed metus nunc. Maecenas id tortor at eros interdum blandit. Sed eros lacus, lacinia eget dolor eu, bibendum tristique mi. Curabitur placerat augue quis nulla bibendum, a faucibus mauris vulputate. Quisque sagittis pulvinar tellus non suscipit.

• In-cabin Driver Monitoring
• Funding Source: Aisin Technical Center of America
• User Study in Autonomous Driving Scenarios
• Funding Source: Hyundai Motor Company

Medical Image Analysis

By studying shapes and textures of medical images, it is possible to detect disease and predict clinical outcomes more accurately. To this end, we are interested in characterizing shapes and textures of diseases in medical images (X-rays, CT, PET, MRI, etc.) and correlate them with other clinical parameters.

• Lung Cancer Outcome Prediction
• Funding Source: NIH R21
• Collaborators: Xiaodong Wu, PhD (Department of Electrical and Computer Engineering)
Brian J. Smith, PhD (Department of Biostatistics)
Yusung Kim, PhD, Bryan G. Allen, MD, PhD, John M. Buatti, MD (Department of Radiation Oncology)
• Interstitial Lung Disease Quantification
• Collaborators: Changhyun Lee, MD, PhD (Department of Radiology)

Data-driven Design & Simulation

It is hardly a stretch of logic to say many problems in science and engineering begin with reasoning of shapes. We are interested in teaching a computer such “geometric reasoning,” like what human designers and engineers would do when they solve a real-world problem. With an advanced machine cognition, designers and engineers will be able to focus more of their time and power on innovation and better design solutions. Furthermore, we could also learn complex patterns around geometric data that are beyond our cognitive capacity.

• High-energy Material Design and Simulation
• Funding Sources: U.S. Air Force Office of Scientific Research (AFOSR)
Multidisciplinary University Research Initiative (MURI)
• Collaborators: H.S. Udaykumar, PhD (Department of Mechanical Engineering)