research interests include structural dynamics and earthquake engineering, and in particular development and implementation of a two step modular procedure that generates effective seismic input for large urban areas, and analysis of structures accounting for soil structure interaction. In more detail we are building a parallel implementation of the DRM method based uppon the OpenSees platform which will be used for earthquake hazard mitigation of large urban areas. My target from this class is to learn how to write efficient parallel code if possible oriented towards finite elements.
1) What is the scientific or engineering problem being solved?
The objective of this problem is to develop the capability for generating realistic inversion-based models of complex basin geology and earthquake sources and use this capability to model and forecast strong ground motion during earthquakes in large basins as Los Angeles. It is a problem of great importance for earthquake hazard mitigation because the assessment of the ground motion to which structures will be exposed during their lifetimes is an essential step in designing earthquake resistant structures as well as retrofiting the existing ones.
The complexity of the problem arises form several sources: The multiple spatial scales characterizing the the earthquake source and the basin response, Time scales vary from hundredths of seconds to resolve the high frequencies, The irregular geometry of such basins, The high heterogenuity of the soil materials and also the uncertainty introduced because of geology and source related parameters.
Solutions of inverse wave propagation problems present numerous difficulties, including severe nonlinearity, ill-posedness, multiple solutions, space-time coupling, discontinuous solutions, and dense ill-conditioned operators despite the fact that the forward wave propagation problem is linear, well-posed, has a unique solution whose computation can be time-marched, and possesses a sparse well-conditioned operator. This team has previously developed a parallel scalable method that addresses these difficulties, and demonstrated it for inverse acoustic wave propagation problems.
2) How well did the application achieve its scientific / engineering objective? Are simulation results compared to physical results?
The hexahedral forward earthquake simulation code has been verified against closed-form solutions and against an earlier tetrahedral code in ground motion simulations in the Los Angeles region. Agreement between the finite element simulation and the Green s function solution is excellent. Against those results the results of the inverse analysis are compared as well and some comparison results are shown in the following figure were the forward analysis is noted as "red" and the inverse as "blue".
There hasn't been though any comparison between the results of the runs and actuall natural recordings.
The target machine was Le Mieux, the HP AlphaServer system at the Pittsburgh Supercomputing Center; it sustains nearly a teraflop/s over 12 hours in solving the 300 million wave propagation ODEs that result upon spatial discretization; and executes at 25% of the peak floating point rate on the 2 Gflops/s Alpha processors. Lemieux comprises 750 Compaq Alphaserver ES45 nodes and two separate front end nodes. Each computational node contains four 1-GHz processors and runs the Tru64 Unix operating system. A Quadrics interconnection network connects the nodes. Each node is a 4 processor SMP, with 4 Gbytes of memory.
The mesh generation library was written in C and is called the etree library, which is a new database oriented method that extends the size of the possible mesh to be generated from the memory size to the disk size available. The system of equations is discretized by Galerkin finite elements in space, and explicit central differences in time. The discretized system is solved using the multiscale Gauss-Newton-conjugate gradient method.The solver is built from components of the PETSc library.
The HP Alphaserver is currently ranked at place 34 of the TOP500 list.
The code executes at 25% of the peak floating point rate of the cluster, which is relatively good given the high irregularity and multiresolution of the meshes used.
The application as it appears from the data presented in the above figured scaled fairly well from 1 to 3000 processors.