Research Activities

Blue Waters
The Blue Waters National Petascale Computing Facility is a remarkable resource, and 10% of the Blue Waters time is allocated to UIUC. The proposed Center will use the Blue Waters as its main computational resource. Blue Waters is the largest and most powerful computational resource we can currently access. It has 4,224 CPU/GPU heterogeneous computing nodes. By using this alreadyavailable huge computational capability, the proposed center will not incur further expenses to the funding agency. This way, most of the external funding will be used for the solution of important scientific problems. The center will also develop allocation proposals to NSF to compete for Blue Waters time at the national level. click for more...
 MultiPhysics Problems
The emergence of new technologies across multiple fields, such as nanotechnologies, clean energy, biotechnologies and information technologies, often requires multiphysics and multiscale solutions of complex physical systems in a multidisciplinary manner. We will focus on the stateoftheart solution techniques and parallel computing strategies for multiphysics grandchallenge problems. We will pursue cuttingedge research ideas on solvers for multiscale and multiphysics problems. Multiscale systems are frequently encountered in reallife applications, where fine details may coexist with ordersofmagnitude larger structures. Multiphysics applications and examples can be derived from various disciplines, such as cosimulation of Maxwell’s equations together with Schrodinger equation, heat equation, acoustic equation, elastic equation, transport equations, etc.

MultiScale Problems
We will target the solutions of scientifically important problems that have multiscale properties, features, and complexity. We will model such systems at various scales. Starting with the atomic level, we will model large numbers of features at any level so that the correct physics for the next higher level will be revealed. If the next higher level is molecular scale, a large number of molecules will be simulated in order to obtain the correct physics in the next higher level. Depending on the system under consideration, the next higher level may be porous membrane of a biological system, for instance. Multiphysics governing equations at each scale will be solved. Continuity of the governing equations from one scale to another will be insured, instead of jumping from one set of equations to the next one, which usually involves approximations due to the large numbers of degrees of freedom (DOFs) that is typically avoided. Instead of avoiding such complexity and resorting to approximations, correct physical behavior will be modeled at every scale with inputs from the lower scale. click for more...

MultiPhase Problems
Scientifically important and complicated reallife problems often involve coexisting and interacting multiphase materials, such as solids, liquids, gases, and soft matter (i.e., biological organisms, such as the gray matter). Modeling of heterogeneous structures is difficult, but rewarding. Usually, solutions of such problems are performed by obtaining individual solutions in each homogenous part of the problem. We aim to develop holistic approaches for modeling complicated heterogeneous structures, including their interfaces. An example is the solution of a system that contains both solidstate structures and living organisms. Current technological trends stipulate that the convergence of biology and materials is inevitable. We plan to support experimental studies with modeling, simulation, and a deeper understanding of the multiphysics, multiscale and multiphase phenomena in such systems. For some problems, multiphase properties may also appear in time. Therefore, we will develop solution methods for such timedependent phase changes in the same system.

Big Data
Multiscale modeling is creating big data. One example is multimodal imaging. Another example is realtime imaging with streaming large amounts of data. Pattern matching is an important task in order to automate the evaluation of the images and arrive at diagnostics/conclusions. More importantly, the ability to analyze the big data is critical for validating the models and discovering new insights from the multiscale, multiphase simulation runs.

Intentional Programming
The idea of intentional programming is to separate the intention of the programmer (pattern of data access and computation) from the implementation details (data layout, loop structures, access indexing). The former is expressed by the programmers through a higherorder function library interface and the latter is crafted by the library provider of the higherorder functions. These higherorder functions model the most commonly used patterns in a target application domain and must be carefully implemented for each target hardware by the library provider. The higherorder functions also provide the programmer intentions to a libraryaware compiler to safely synthesize finetuned executable for the target hardware. Triolet C++ and Triolet Python are two systems developed at UIUC to make intentional programming to C++ developers and Python developers. Preliminary results show that applications expressed in higherorder functions can achieve higher performance than lowlevel OpenCL programming on a wide range of target processor architectures. We beleieve that the intentional programming systems will drastically improve the programmability of exascale heterogeneous computing systems and will likely revolutionize the programming of heterogeneous parallel systems.
Current Target Applications

Computational Synthetic Biology
Successful design of selfassembly of synthetic celllike constructs, insertion of nanopores in synthetic cells, and exchange of information between synthetic cells require simulations involving hundreds of millions of atoms. These studies can significantly benefit from high performance computing. Cellsignaling and cellcell interactions are complex phenomena, which are hard to understand with conventional modeling tools and computational resources. We aim to utilize stateoftheart methods and facilities for a significantly improved understanding of cellcell interactions. This effort is expected to produce scientific knowledge at the fundamental level, i.e., the kind of information that will lead to the answers of an array of important questions.

Integrated biological organisms, nanophysics materials and nanoelectronic devices
The key issue with the hybrid multiscale and multiphase systems is modeling the atomistic interactions between ‘electrons in solid materials’ and ‘biomolecules in liquid phase’. While this can be done for a small number (~100) of atoms at the interface between the two phases, it requires extensive computational resources in more realistic physical situations. Our goal is to treat a larger number of atoms (>>1000) at the interface between the two material phases to incorporate hybrid biologicalnanoelectronic systems into a selfconsistent multiscale scheme with more realistic physical conditions to capture the real nature of the interactions between molecules in liquid phase and electrons in solids.

Graphene
Graphene has become a very important structure due to its many uses in different applications. It has remarkable material properties. Furthermore, graphene is also very important in energy applications; it can be used as a supercapacitor. We can model graphene starting with its atoms. That is, one scale can be constructed from a smaller scale. Next higher level can be a system that involves graphene as one of its components. For example, a porous membrane made up of graphene and interacting with cells or particles may be the next higher scale in a multiscale, multiphysics, multiphase problem.

Medical Imaging
Current ultrasound imaging systems lack resolution and clarity. Microwave imaging systems have been proposed for almost two decades, but have not made their way into reallife applications, due to the complexity of the inverse problems that need to be solved. Combined use of microwaves and ultrasound simultaneously requires even more computations that can be made possible with the use of an extreme resource, such as Blue Waters. Xrays are ionizing, ultrasound lacks resolution, MRI is not effective with some tissue types. Highresolution imaging with nonionizing microwaves has been pursued for more than two decades, however, clinical systems are still not available. Solving very complicated inverse problems and developing microwaveimaging systems available will be a groundbreaking outcome.

BioTransport
The functioning and survival of biological systems in physiological and pathological situations is extremely complex and involves interactions of various forces, chemical reactions and physical transport processes at multiple scales. At the macroscale, starting from the supply of oxygenated blood to the various organs, and return of venous fluid to the lungs for cyclical detoxification, numerous multiscale phenomena occur in every biological system, including the human body. At the cell level, the shear stress on the endothelium and platelets can cause local atherosclerosis through local plaque deposition, resulting in catastrophic consequences. Inside the cell, numerous physical and chemical processes continuously take place, impacted by the DNA and genetic structure of the cell. The cell itself can be classified as a unit biological factory, which is responsible for cell differentiation, cell replication, and cell death. In normal physiological circumstances, the cell and the organs work harmoniously for system survival and growth. However, these mechanisms are interrupted and lead to abnormal behavior through propagation of disease, cell morphing, cell death and organ failure. click for more...

Superconductivity
Simulate an unconventional superconductor by solving the Schroedinger equation with enough realism and precision. This would lead to design guidelines for new superconducting materials. A tool that will predictively compute tendency towards superconductivity may be a stretch goal. We are currently simulating the electronic structure of high temperature superconductors and finding success. However, in order to simulate the superconductivity, we need to access system sizes and energy scales that will require orders of magnitude more time.

Photovoltaics
Designing novel materials for photovoltaic devices is a strategic goal. Doping and defect control is critical to making optoelectronic devices. We still do not understand how to do good defect engineering. click for more...

Solvers
The multiphysics and multiscale problems we are proposing to solve will require the solution of classical equations of physics, such as
 Maxwell’s equations
 Laplace’s and Poison’s equations
 NavierStoke’s equations
 Schrodinger equation
 Casimir force calculations
 Newton equations for molecular dynamics
 Lattice Boltzmann equations
 Discretestate equations
 Kalman filter equations

Fast Solvers
We are not targeting a bruteforce use of the gigantic resources of the Blue Waters facility. It is well known that computational resources, even the largest ones, will rapidly become insufficient as we increase the size of a problem. Instead, we will employ fast algorithms with reduced computational complexity and memory requirement to be able to solve extremely large problems. Examples of reducedcomplexity solvers are fast multipole method (FMM), multilevel fast multipole algorithm (MLFMA), adaptive cross approximation (ACA), hierarchicalmatrices (Hmatrix and H2matrix) methods, interpolative decomposition (ID) and skeletonization, compression and recompression techniques, including compressive sensing.

Parallelization
Fast solvers reduce the computation times by factors of millions, billions, and trillions. However, it is very difficult to parallelize such algorithms over thousands of processors/nodes for the purpose of obtaining acceleration of another factor of thousand, ten thousand, or hundred thousand. Adding to the difficulty of the problem, Blue Waters has a heterogeneous CPU+GPU architecture. The most efficient (in terms of parallelization) and most scalable use of Blue Waters with the reducedcomplexity fast solvers will require novel approaches to the parallelization. We propose to use the intentional programming methodology realized in Triolet C++ and Triolet Python and provide the common higherorder functions to make the parallelized solvers accessible to domain application developers.


