AspectGrid

Pluggable Grid Aspects for Scientific Applications

GRID/GRI/81880/2006

Overview

Grids offer new opportunities to scientists by promoting an effective way of sharing a wide range of resources. Current trend in Grid research is moving towards a unified set of Grid services, providing a single application programming interface that can transparently provide access to a diversity of resources. The next step for Grid development is to foster a seamless access to Grids services.

The complexity of Grid systems can be hidden behind Grid middleware. Examples include user authentication and authorization and resource discovery. However, to effectively leverage the benefits of a Grid, applications must be redesigned to the specific nature of Grids. For instance, to take advantage of the hierarchical nature computational Grids, made of a federation of Clusters, applications should be redesigned to include multiple levels of parallelism.

The ideal Grid programming environment, and the one with strongest probability of success, would be very close to a desktop programming environment. This research focuses on the development and deployment of a new middleware for Grid computing that pursuits this goal. The idea is to develop Grid enabled applications (i.e., applications that can take advantage of Grid resources) that resemble to traditional desktop applications, but can run on Grids by plugging modules that provide access to Grid services. For instance, an (un)pluggable module could provide remote execution (e.g., on some Grid compute node) of a certain application block of code. Unplugging such a module would provide a normal local execution.

This research focus on exploring aspect-oriented techniques to build pluggable modules (i.e., Grid aspects) that provide access to Grid services. These modules strongly rely on current Grid infra-structure (e.g., Globus toolkit) and will provide access to the following classes of Grid services:

  • concurrent execution: specification of tasks that may be executed in parallel, synchronization among concurrent tasks and specification of multi-level parallelism;
  • distributed execution: mapping and execution of tasks into distributed resources, according to task requirements (e.g., application level scheduling), task migration between resources and support to multiple communication middleware, taking advantage of the hierarchical nature of Grids (e.g., using different methods for intra‑cluster and inter-cluster communication);
  • data services: access to multiple data sources, data distribution, replication and caching;
  • utility services: application-level mechanisms to address node faults (e.g., to decide if the computation should by retried or simply discarded), checkpoint of application data, remote IO and application monitoring.

The new middleware will provide a smother (and reversible) path to enable applications to run on Grid environments.

The middleware will be validated through several case studies and from multiple application domains. Case studies are taken from existing code bases of project members, which require access to compute resources not currently available in a small cluster. These codes will be enabled to run on Grid environments by plugging Grid modules and will contribute to increase the community Grid users. Cases studies include:

  • Optimization of biotechnological processes: this case study handles the use of evolutionary computation algorithms to two tasks in the optimization of biotechnological processes: the selection of the best mutant strains capable of achieving a high productivity in recombinant fermentation processes and the optimization of the trajectory of fermentation control variables over time.
  • Thin-film nucleation and growth: these are complex phenomena of both scientific and technological interest, and proper simulation resorts to the use of a kinetic Monte Carlo algorithm, as it is appropriate for simulating large length-scales and long physical times.
  • Mesoscopic modeling of polymer light emitting diodes: this application performs numerical simulations of charge injection, transport, trapping and recombination in PLEDs using a charge hopping model, which works at intermediate scale between atomic and macroscopic length scales, to assess the consequences of molecular properties, composition and microstructure of the polymer-based layer on device performance.

Additionally a well know case study will also be addressed (molecular dynamics simulation applied to induced unfolding of proteins and variants), where several parallel implementations are available (e.g., NAMD).

This project also addresses the development of support tools for suitable access to Grid environments. Visual tools to help the usage of Grid services will be developed, including means to visualize and to plug Grid aspects into scientific codes. These tools provide a set of pre-built modules, implementing Grid aspects that can be plugged into applications in a straightforward way, allowing the transparent execution and monitoring of applications on Grids developed under the context of the project.

Project Goals

  • Design and implementation of a new grid middleware and support tools
  • Migrate a small set of scientific applications, from multiple application domains, to grid environments
  • Promote collaborations among researchers experts in grid/parallel computing and scientists with codes that can benefit from using grid computing

Funding

The AspectGrid (GRID/GRI/81880/2006) project is a 3 years project funded by Portuguese FCT (POSI) and by European funds (FEDER).

Team

  • João Luís Sobral (CCTC-UM) – Middleware / Project Leader
  • Alberto Proença (CCTC-UM) – Middleware
  • Diogo Telmo Neves (CCTC-UM) – Middleware
  • Edgar Sousa (CCTC-UM) – Middleware
  • Rui Carlos Gonçalves (CCTC-UM) – Applications
  • Miguel Rocha (CCTC-UM) – Applications
  • António Cadilhe (CF-UM) – Applications
  • Marta Ramos (CF-UM) – Applications
  • Luís Marques (CF-UM) – Applications
  • Rui Brito (CQ-UC) – Applications
  • António Pina (CCTC-UM) – Grid Support / Deployment tools
  • Pedro Alberto (CF-UC) – Grid Support / Deployment tools

Publications

  • R. C. Gonçalves and J. L. Sobral, “Pluggable parallelisation,” in HPDC ’09: proceedings of the 18th ACM international symposium on high performance distributed computing, 2009, pp. 11-20. doi:10.1145/1551609.1551614
    [BibTeX]
    @inproceedings{Goncalves:2009,
    author = {Gonçalves, Rui C. and Sobral, João L.},
    title = {Pluggable Parallelisation},
    booktitle = {{HPDC} '09: Proceedings of the 18th {ACM} international symposium on High Performance Distributed Computing},
    year = {2009},
    pages = {11--20},
    doi = {10.1145/1551609.1551614},
    }

  • B. Medeiros and J. L. Sobral, “Checkpoint and run-time adaptation with pluggable parallelisation,” in ICPP ’11: proceedings of the 2011 international conference on parallel processing, 2011, pp. 434-443. doi:10.1109/ICPP.2011.83
    [BibTeX]
    @inproceedings{Medeiros:2011,
    author = {Medeiros, Bruno and Sobral, João L.},
    title = {Checkpoint and Run-Time Adaptation with Pluggable Parallelisation},
    booktitle = {{ICPP} '11: Proceedings of the 2011 International Conference on Parallel Processing},
    year = {2011},
    pages = {434--443},
    doi = {10.1109/ICPP.2011.83},
    keywords = {application-level checkpointing, run-time adaptation, pluggable parallelisation, aspect oriented programming},
    }

  • D. T. Neves and J. L. Sobral, “Improving the separation of parallel code in skeletal systems,” in ISPDC ’09: proceedings of the 8th international symposium on parallel and distributed computing, 2009, pp. 257-260. doi:10.1109/ISPDC.2009.36
    [BibTeX]
    @inproceedings{Neves:2009,
    author = {Neves, Diogo T. and Sobral, João L.},
    title = {Improving the Separation of Parallel Code in Skeletal Systems},
    booktitle = {{ISPDC} '09: Proceedings of the 8th International Symposium on Parallel and Distributed Computing},
    year = {2009},
    pages = {257--260},
    doi = {10.1109/ISPDC.2009.36},
    keywords = {multicore, skeletons, aspect-oriented programming, dependency injection},
    }

  • J. Pinho, M. Rocha, and J. L. Sobral, “Pluggable parallelization of evolutionary algorithms applied to the optimization of biological processes,” in PDP ’10: proceedings of the 18th euromicro conference on parallel, distributed and network-based processing, 2010, pp. 395-402. doi:10.1109/PDP.2010.89
    [BibTeX]
    @inproceedings{Pinho:2010,
    author = {Pinho, Jorge and Rocha, Miguel and Sobral, João L.},
    title = {Pluggable Parallelization of Evolutionary Algorithms Applied to the Optimization of Biological Processes},
    booktitle = {{PDP} '10: Proceedings of the 18th Euromicro Conference on Parallel, Distributed and Network-based Processing},
    year = {2010},
    pages = {395--402},
    doi = {10.1109/PDP.2010.89},
    keywords = {aspectj, aspect oriented programming, evolutionary algorithms, multicore, optimization of biological systems},
    }

  • J. L. Sobral, C. A. Cunha, and M. P. Monteiro, “Aspect oriented pluggable support for parallel computing,” in High performance computing for computational science — VECPAR 2006, , 2007, vol. 4395, pp. 93-106. doi:10.1007/978-3-540-71351-7_8
    [BibTeX]
    @incollection{Sobral:2007,
    author={Sobral, João L. and Cunha, Carlos A. and Monteiro, Miguel P.},
    title={Aspect Oriented Pluggable Support for Parallel Computing},
    booktitle={High Performance Computing for Computational Science -- {VECPAR} 2006},
    series={Lecture Notes in Computer Science},
    volume={4395},
    year={2007},
    pages={93--106},
    doi={10.1007/978-3-540-71351-7_8},
    }

  • J. L. Sobral and M. P. Monteiro, “A domain-specific language for parallel and grid computing,” in DSAL ’08: 3rd workshop on domain specific languages, 2008. doi:10.1145/1404927.1404929
    [BibTeX]
    @inproceedings{Sobral:2008a,
    author = {Sobral, João L. and Monteiro, Miguel P.},
    title = {A Domain-Specific Language for Parallel and Grid Computing},
    booktitle = {{DSAL} '08: 3rd Workshop on Domain Specific Languages},
    year = {2008},
    doi = {10.1145/1404927.1404929},
    }

  • E. Sousa, R. C. Gonçalves, D. T. Neves, and J. L. Sobral, “Non-invasive gridification through an aspect-oriented approach,” in Ibergrid ’08: proceedings of the 2nd iberian grid infrastrucutre conference, 2008, pp. 323-334. doi:10.4272/978-84-9745-288-5
    [BibTeX] [Download PDF]
    @inproceedings{Sousa:2008,
    author = {Sousa, Edgar and Gonçalves, Rui C. and Neves, Diogo T. and Sobral, João L.},
    title = {Non-Invasive Gridification through an Aspect-Oriented Approach},
    booktitle = {Ibergrid '08: Proceedings of the 2nd Iberian Grid Infrastrucutre Conference},
    year = {2008},
    pages = {323--334},
    doi = {10.4272/978-84-9745-288-5},
    url = {http://alba.di.uminho.pt/research/wp-content/uploads/Sousa-2008.pdf},
    }

Tools