|
|
|
|
The Leading Source for Global News and Information from the evolving Grid ecosystem,
including Grid, SOA, Virtualization, Storage, Networking and Service-Oriented IT |
|
|
July 18, 2005
|
|
Scientific Applications:
New Obstacle Scattering Numerical Model Running on ENEA Grid
By Silvio Migliori (ENEA), Giovanni Bracco (ENEA) and Francesco Zirilli (Università di Roma La Sapienza)
1. Introduction
As you are flying on an airplane approaching for landing in a busy airport, you feel secure thanks to the radar system that governs the air traffic. In an apparently very different situation, you might feel secure about your health condition after hearing an expert opinion on the output of medical tests (e.g., ecography, X-ray imaging) on your body. The common scientific ground of the technologies mentioned above, and of many others not mentioned, is wave propagation and scattering.
The waves used may be of different nature. For example acoustic waves are used in ecography, electromagnetic waves are used in radars and X-ray imaging, and elastic waves are used in geological prospection. The scattering phenomena under scrutiny are those generated by the fact that these waves propagating in a suitable medium find on their ways "obstacles" (e.g., an organ of the human body in medical ecography, an airplane in air traffic control or a discontinuity in the physical properties of the underground in geological prospection). While most people do not know or do not think about it, high performance computing (HPC) is a fundamental tool in advancing our knowledge of wave propagation and scattering phenomena.
The use of HPC is based on the existence of mathematical models that describe wave propagation and scattering phenomena. The main mathematical models of wave propagation go back to 19th century mathematical physics, for example, the wave equation for acoustic waves, Maxwell's equations for electromagnetic waves. The mathematical modeling of scattering phenomena usually corresponds to choosing coefficients or boundary conditions for the equations of the propagation models.
In the 19th century, the quantitative solution of these models in the concrete circumstances where they are used today was impossible. It is only the advent of electronic computing that has made possible the quantitative solution of these models in practical situations. HPC gives us the possibility of further extending the complexity of the models that can be solved.
2. Industrial Interest in Wave Propagation, Scattering and HPC
The number of companies both in the military and civil sectors working in businesses related to wave propagation and scattering is huge and continues to grow. These companies are both manufacturers (e.g., companies building radars or biomedical instruments) or service providers (e.g., companies doing data analysis in geological prospecting). To meet the challenges posed by the competition on their own markets, these companies rely increasingly on HPC. A particularly interesting form of HPC in this context is Grid computing.
In fact, the ubiquitous presence of computing hardware (i.e. PCs, workstations, etc.) in modern companies makes possible for each company to have its own computing Grid, investing very little extra money to create and managing the Grid from the available computing hardware. Moreover, when the internal Grid is not enough, it is easy to access to state-of-the-art Grids thank to the existence of service providers that sell computing time worldwide. Of course, supercomputing, that is MIMD (Multiple Instructions Multiple Data) machines with hundred or thousands of processors or computing on dedicated hardware, cannot be available in house in a medium or small company due to costs.
We can conclude that the companies, especially the medium and small ones working on technologies related to wave propagation and scattering, are interested in developing application software able to exploit the power of Grid computing. That is, given the mathematical models of the phenomena considered, the way in which these models are solved quantitatively through electronic computing is the factor that determines the ability of the application software of exploiting the power of Grid computing.
3. The ENEA Grid Project
In order to meet the needs of the medium and small Italian companies, ENEA (Italian National Agency for New Technologies, Energy and the Environment) has started a Grid project. The main goals of this project are:
3.1 ENEA Grid
ENEA research activity is performed in 12 sites around Italy, and the ENEA Grid infrastructure is born with the mission to provide an integrated environment covering most of ENEA's multi-platform computational resources, connected in a wide area network. The key elements in the ENEA Grid implementation are: a distributed file system (AFS/OpenAFS), a wide area resource manager (Platform LSF) and a unified user interface (based on Java and Citrix Technologies). These components form the middleware that provides the functionalities typical of a Grid, "unique authentication, authorization, resource access and resource discovery," as stated by Ian Foster and Carl Kesselman in their classical paper "The Anatomy of the Grid." The choice of mature and multi-platform software components, either proprietary as LSF and Citrix or open-source as OpenAFS, greatly improves the reliability, ease of management and update of the system. ENEA Grid implementation began in 1998 and the new resources have been progressively introduced in what is now a production-quality Grid infrastructure.
The services offered by ENEA Grid include the access to computation resources, graphic and 3-D immersive facilities, and software resources, including commercial codes (e.g., Fluent, Abacus, Catia) and elaboration environments (e.g., Matlab, IDL, SAS). ENEA Grid computational resources include about 300 processors with either MIMD architecture -- as IBM SP, SGI Altix and Onyx, and Cray XD1 -- or in cluster configuration -- with Linux 32/64 systems and Apple Mac OSX. A 128-node IBM SP4, [~1 Tflops] is the most important resource at the moment. These resources are located in six computer centers in northern, central and southern Italy and are connected over the WAN by the Italian Academic and Research Network (GARR). ENEA Grid has, at present, about 600 registered users and more that 2TB of stored data, with 12TB available.
3.2 Example of a Scattering Application Running on the Grid
A highly parallelizable numerical method to solve time-dependent acoustic obstacle scattering problems has been developed by Francesco Zirilli's team. The method proposed is a generalization of the "operator expansion method." Using a Fourier transform with respect to time, the time-dependent scattering problem for the wave equation considered is transformed in an exterior boundary value problem for the Helmholtz equation depending on a parameter. The "operator expansion method" is used to solve the exterior problems for the Helmholtz equation and reduces, via a perturbative approach, the solution of each exterior problem to the solution of a sequence of systems of first kind integral equations defined on a suitable "reference surface."
The numerical solution of these systems of integral equations is challenging when scattering problems involving realistic obstacles and wavelengths small compared to the characteristic dimensions of the obstacles are solved. Let RT be the ratio between the characteristics dimensions of the obstacle and the wavelength considered. In the numerical experiments done, the greater ratios RT considered range in the interval [13-60]. This implies the use of high- dimensional vector spaces to satisfactorily approximate the corresponding systems of integral equations. That is, each system of integral equations in the sequence mentioned above after being discretized to be solved numerically becomes a large system of linear equations. Represented on a generic base, the matrices that approximate the integral operators after the discretization will be dense matrices.
Zirilli's team has found a new way of using the wavelet transform, and has introduced new bases of wavelets and a version of the operator expansion method that constructs directly element by element the coefficient matrix of the sparse linear systems that approximate in the wavelet basis considered the systems of integral equations. The resulting numerical method, using ENEA Grid, is able to deal with realistic acoustic scattering problems solving large (sparse) linear systems (up to approximately 5·105 (real)) unknowns and equations.
These results are obtained running simultaneously clusters of 500 jobs on ENEA's Grid. The computation of the elements of the coefficient matrices of the linear systems considered and several other aspects of the numerical algorithm proposed can be carried out independently one from the other allowing the development of an efficient solver for exterior boundary value problems for the Helmholtz equation, particularly suited to be executed on a computing Grid. Several numerical experiments involving realistic obstacles and "small" wavelengths have been performed. Some animations and virtual reality applications relative to these numerical experiments can be seen at the Web site www.econ.univpm.it/recchioni/w12/.
About the Authors
Silvio Migliori received his nuclear engineer degree from Università di Roma La Sapienza. He has been with ENEA since 1983, where he has worked in the Controlled Nuclear Fusion Department in designing and implementing experimental devices. Since 1995, he has held a position in the Computer and Networking Service, where he is now adjoint director for scientific computation. His main interests are oriented to advanced systems as computational Grids and virtual environments. He can be reached at migliori@enea.it.
Giovanni Bracco received his physics degree at Pavia University, and has worked in ENEA since 1981. He has worked in the controlled nuclear fusion department with a particular interest in plasma diagnostics systems and in the development of data analysis tools. Since 2004, he has held a position in the computer and networking service, where he works in Grid computing and distributed scientific computation.
Francesco Zirilli received a mathematics degree from Università di Roma La Sapienza. He is currently a Professor of Istituzioni di Matematiche at Università di Roma La Sapienza. His scientific interests are in mathematical modeling and scientific computing in mathematical physics and engineering. He is author of over 100 scientific publications in refereed journals and books.
1. Introduction
As you are flying on an airplane approaching for landing in a busy airport, you feel secure thanks to the radar system that governs the air traffic. In an apparently very different situation, you might feel secure about your health condition after hearing an expert opinion on the output of medical tests (e.g., ecography, X-ray imaging) on your body. The common scientific ground of the technologies mentioned above, and of many others not mentioned, is wave propagation and scattering.
The waves used may be of different nature. For example acoustic waves are used in ecography, electromagnetic waves are used in radars and X-ray imaging, and elastic waves are used in geological prospection. The scattering phenomena under scrutiny are those generated by the fact that these waves propagating in a suitable medium find on their ways "obstacles" (e.g., an organ of the human body in medical ecography, an airplane in air traffic control or a discontinuity in the physical properties of the underground in geological prospection). While most people do not know or do not think about it, high performance computing (HPC) is a fundamental tool in advancing our knowledge of wave propagation and scattering phenomena.
The use of HPC is based on the existence of mathematical models that describe wave propagation and scattering phenomena. The main mathematical models of wave propagation go back to 19th century mathematical physics, for example, the wave equation for acoustic waves, Maxwell's equations for electromagnetic waves. The mathematical modeling of scattering phenomena usually corresponds to choosing coefficients or boundary conditions for the equations of the propagation models.
In the 19th century, the quantitative solution of these models in the concrete circumstances where they are used today was impossible. It is only the advent of electronic computing that has made possible the quantitative solution of these models in practical situations. HPC gives us the possibility of further extending the complexity of the models that can be solved.
2. Industrial Interest in Wave Propagation, Scattering and HPC
The number of companies both in the military and civil sectors working in businesses related to wave propagation and scattering is huge and continues to grow. These companies are both manufacturers (e.g., companies building radars or biomedical instruments) or service providers (e.g., companies doing data analysis in geological prospecting). To meet the challenges posed by the competition on their own markets, these companies rely increasingly on HPC. A particularly interesting form of HPC in this context is Grid computing.
In fact, the ubiquitous presence of computing hardware (i.e. PCs, workstations, etc.) in modern companies makes possible for each company to have its own computing Grid, investing very little extra money to create and managing the Grid from the available computing hardware. Moreover, when the internal Grid is not enough, it is easy to access to state-of-the-art Grids thank to the existence of service providers that sell computing time worldwide. Of course, supercomputing, that is MIMD (Multiple Instructions Multiple Data) machines with hundred or thousands of processors or computing on dedicated hardware, cannot be available in house in a medium or small company due to costs.
We can conclude that the companies, especially the medium and small ones working on technologies related to wave propagation and scattering, are interested in developing application software able to exploit the power of Grid computing. That is, given the mathematical models of the phenomena considered, the way in which these models are solved quantitatively through electronic computing is the factor that determines the ability of the application software of exploiting the power of Grid computing.
3. The ENEA Grid Project
In order to meet the needs of the medium and small Italian companies, ENEA (Italian National Agency for New Technologies, Energy and the Environment) has started a Grid project. The main goals of this project are:
- providing ENEA for its internal use, and the eventual use by external users, with a computing Grid.
- stimulating the demand of Grid computing by medium and small companies, providing examples of successful applications running on the Grid, and helping the users in formulating their models in a way that can exploit the power of the Grid.
3.1 ENEA Grid
ENEA research activity is performed in 12 sites around Italy, and the ENEA Grid infrastructure is born with the mission to provide an integrated environment covering most of ENEA's multi-platform computational resources, connected in a wide area network. The key elements in the ENEA Grid implementation are: a distributed file system (AFS/OpenAFS), a wide area resource manager (Platform LSF) and a unified user interface (based on Java and Citrix Technologies). These components form the middleware that provides the functionalities typical of a Grid, "unique authentication, authorization, resource access and resource discovery," as stated by Ian Foster and Carl Kesselman in their classical paper "The Anatomy of the Grid." The choice of mature and multi-platform software components, either proprietary as LSF and Citrix or open-source as OpenAFS, greatly improves the reliability, ease of management and update of the system. ENEA Grid implementation began in 1998 and the new resources have been progressively introduced in what is now a production-quality Grid infrastructure.
The services offered by ENEA Grid include the access to computation resources, graphic and 3-D immersive facilities, and software resources, including commercial codes (e.g., Fluent, Abacus, Catia) and elaboration environments (e.g., Matlab, IDL, SAS). ENEA Grid computational resources include about 300 processors with either MIMD architecture -- as IBM SP, SGI Altix and Onyx, and Cray XD1 -- or in cluster configuration -- with Linux 32/64 systems and Apple Mac OSX. A 128-node IBM SP4, [~1 Tflops] is the most important resource at the moment. These resources are located in six computer centers in northern, central and southern Italy and are connected over the WAN by the Italian Academic and Research Network (GARR). ENEA Grid has, at present, about 600 registered users and more that 2TB of stored data, with 12TB available.
3.2 Example of a Scattering Application Running on the Grid
A highly parallelizable numerical method to solve time-dependent acoustic obstacle scattering problems has been developed by Francesco Zirilli's team. The method proposed is a generalization of the "operator expansion method." Using a Fourier transform with respect to time, the time-dependent scattering problem for the wave equation considered is transformed in an exterior boundary value problem for the Helmholtz equation depending on a parameter. The "operator expansion method" is used to solve the exterior problems for the Helmholtz equation and reduces, via a perturbative approach, the solution of each exterior problem to the solution of a sequence of systems of first kind integral equations defined on a suitable "reference surface."
The numerical solution of these systems of integral equations is challenging when scattering problems involving realistic obstacles and wavelengths small compared to the characteristic dimensions of the obstacles are solved. Let RT be the ratio between the characteristics dimensions of the obstacle and the wavelength considered. In the numerical experiments done, the greater ratios RT considered range in the interval [13-60]. This implies the use of high- dimensional vector spaces to satisfactorily approximate the corresponding systems of integral equations. That is, each system of integral equations in the sequence mentioned above after being discretized to be solved numerically becomes a large system of linear equations. Represented on a generic base, the matrices that approximate the integral operators after the discretization will be dense matrices.
Zirilli's team has found a new way of using the wavelet transform, and has introduced new bases of wavelets and a version of the operator expansion method that constructs directly element by element the coefficient matrix of the sparse linear systems that approximate in the wavelet basis considered the systems of integral equations. The resulting numerical method, using ENEA Grid, is able to deal with realistic acoustic scattering problems solving large (sparse) linear systems (up to approximately 5·105 (real)) unknowns and equations.
These results are obtained running simultaneously clusters of 500 jobs on ENEA's Grid. The computation of the elements of the coefficient matrices of the linear systems considered and several other aspects of the numerical algorithm proposed can be carried out independently one from the other allowing the development of an efficient solver for exterior boundary value problems for the Helmholtz equation, particularly suited to be executed on a computing Grid. Several numerical experiments involving realistic obstacles and "small" wavelengths have been performed. Some animations and virtual reality applications relative to these numerical experiments can be seen at the Web site www.econ.univpm.it/recchioni/w12/.
About the Authors
Silvio Migliori received his nuclear engineer degree from Università di Roma La Sapienza. He has been with ENEA since 1983, where he has worked in the Controlled Nuclear Fusion Department in designing and implementing experimental devices. Since 1995, he has held a position in the Computer and Networking Service, where he is now adjoint director for scientific computation. His main interests are oriented to advanced systems as computational Grids and virtual environments. He can be reached at migliori@enea.it.
Giovanni Bracco received his physics degree at Pavia University, and has worked in ENEA since 1981. He has worked in the controlled nuclear fusion department with a particular interest in plasma diagnostics systems and in the development of data analysis tools. Since 2004, he has held a position in the computer and networking service, where he works in Grid computing and distributed scientific computation.
Francesco Zirilli received a mathematics degree from Università di Roma La Sapienza. He is currently a Professor of Istituzioni di Matematiche at Università di Roma La Sapienza. His scientific interests are in mathematical modeling and scientific computing in mathematical physics and engineering. He is author of over 100 scientific publications in refereed journals and books.