Skip to Content


Office of Science Financial Assistance Program Notice 01-10: Scientific Discovery Through Advanced Computing-Advanced Computational Research in Fusion Science

Document Details

Information about this document as published in the Federal Register.

Published Document

This document has been published in the Federal Register. Use the PDF linked in the document sidebar for the official electronic format.

Start Preamble


U.S. Department of Energy (DOE).


Notice inviting research grant applications.


The Office of Fusion Energy Sciences (OFES) of the Office of Science (SC), U.S. Department of Energy (DOE) hereby announces its interest in receiving grant applications for the development of scientific simulation codes needed to address complex problems in fusion energy sciences. The goal is the creation of codes that achieve high performance on a single node, scale to hundreds of nodes and thousands of processors, and have the potential to be ported to future generations of high performance computers. This announcement is focused on some of the topical areas that are important to developing integrated models of fusion systems and require the capabilities of terascale computers. Specific areas of interest include:

  • Turbulence and transport in order to predict energy and particle confinement in plasmas,
  • Macroscopic equilibrium and stability to be able to predict stability limits in magnetically confined plasmas,
  • Magnetic reconnection in order to understand the dynamo and “sawtooth” oscillations in plasmas,
  • Electromagnetic wave/particle interactions to be able to predict heating and current drive in plasmas,
  • Boundary layer effects in plasmas in order to predict the transport of heat and particles in the edge region of a fusion device, and
  • Electromagnetic fields and beam dynamics in particle accelerators to model efficient, high-current heavy ion accelerators.

The full text of Program Notice 01-10 is available via the Internet at the following web site address:​production/​grants/​grants.html.


Preapplications referencing this program notice must be received by 4:30 P.M. EST, January 31, 2001. A response encouraging or discouraging the submission of a formal application will be communicated by e-mail within 14 days.

Formal applications submitted in response to this notice must be received no later than 4:30 P.M., March 15, 2001, to be accepted for merit review and consideration for award in Fiscal Year 2001.


Preapplications referencing Program Notice 01-10 should be forwarded to: U.S. Department of Energy, Office of Science, Office of Fusion Energy Sciences, SC-55, 19901 Germantown Road, Germantown, Maryland 20874-1290, ATTN: John Sauter. Preapplications can also be submitted via E-mail at the following E-mail address: Formal applications referencing Program Notice 01-10 should be forwarded to: U.S. Department of Energy, Office of Science, Grants and Contracts Division, SC-64, 19901 Germantown Road, Germantown, Maryland 20874-1290, ATTN: Program Notice 01-10. The above address must be used when submitting applications by U.S. Postal Service Express Mail, any commercial mail delivery service, or when hand-carried by the applicant. An original and seven copies of the application must be submitted.

Start Further Info


Dr. Stephen Eckstrand or Dr. Arnold Kritz, Office of Fusion Energy Sciences, SC-55, U.S. Department of Energy, 19901 Germantown Road, Germantown, MD 20874-1290. Telephone numbers and e-mail addresses are listed below:

Stephen Eckstrand: telephone (301) 903-5546, e-mail

Arnold Kritz: telephone (301) 903-2027, e-mail

End Further Info End Preamble Start Supplemental Information


Background: Scientific Discovery Through Advanced Computing

Advanced scientific computing will be a key contributor to scientific research in the 21st Century. Within the Office of Science (SC), scientific computing programs and facilities are already essential to progress in many areas of research critical to the nation. Major scientific challenges exist in all SC research programs that can best be addressed through advances in scientific supercomputing, e.g., designing materials with selected properties, elucidating the structure and function of proteins, understanding and controlling plasma turbulence, and designing new particle accelerators. To help ensure its missions are met, SC is bringing together advanced scientific computing and scientific research in an integrated program entitled “Scientific Discovery through Advanced Computing.” Start Printed Page 358

The Opportunity and the Challenge

Extraordinary advances in computing technology in the past decade have set the stage for a major advance in scientific computing. Within the next five to ten years, computers 1,000 times faster than today's computers will become available. These advances herald a new era in scientific computing. Using such computers, it will be possible to dramatically extend our exploration of the fundamental processes of nature (e.g., the structure of matter from the most elementary particles to the building blocks of life) as well as advance our ability to predict the behavior of a broad range of complex natural and engineered systems (e.g., the earth's climate or an automobile engine).

To exploit this opportunity, these computing advances must be translated into corresponding increases in the performance of the scientific codes used to model physical, chemical, and biological systems. This is a daunting problem. Current advances in computing technology are being driven by market forces in the commercial sector, not by scientific computing. Harnessing commercial computing technology for scientific research poses problems unlike those encountered in previous supercomputers, in magnitude as well as in kind. As noted in the 1998 report [1] from the NSF/DOE “National Workshop on Advanced Scientific Computing” and the 1999 report [2] from the President's Information Technology Advisory Committee, this problem will only be solved by increased investments in computer software—in research and development of scientific simulation codes as well as on the mathematical and computing systems software that underlie these codes.

Investment Plan of the Office of Science

To meet the challenge posed by the new generation of terascale computers, SC will fund a set of coordinated investments as outlined in its long-range plan for scientific computing, Scientific Discovery through Advanced Computing,[3] submitted to Congress on March 30, 2000. First, it will create a Scientific Computing Software Infrastructure that bridges the gap between the advanced computing technologies being developed by the computer industry and the scientific research programs sponsored by the Office of Science. Specifically, the SC effort proposes to:

  • Create a new generation of Scientific Simulation Codes that take full advantage of the extraordinary computing capabilities of terascale computers.
  • Create the Mathematical and Computing Systems Software to enable the Scientific Simulation Codes to effectively and efficiently use terascale computers.
  • Create a Collaboratory Software Environment to enable geographically separated scientists to effectively work together as a team and to facilitate remote access to both facilities and data.

These activities are supported by a Scientific Computing Hardware Infrastructure that will be tailored to meet the needs of SC's research programs. The Hardware Infrastructure is robust, to provide the stable computing resources needed by the scientific applications; agile, to respond to innovative advances in computer technology that impact scientific computing; and flexible, to allow the most appropriate and economical resources to be used to solve each class of problems. Specifically, the SC proposes to support:

  • A Flagship Computing Facility, the National Energy Research Scientific Computing Center (NERSC), to provide the robust, high-end computing resources needed by a broad range of scientific research programs.
  • Topical Computing Facilities to provide computing resources tailored for specific scientific applications and to serve as the focal point for an application community as it strives to optimize its use of terascale computers.
  • Experimental Computing Facilities to assess the promise of new computing technologies being developed by the computer industry for scientific applications.

Both sets of investments will create exciting opportunities for teams of researchers from laboratories and universities to create new revolutionary computing capabilities for scientific discovery.

The Benefits

The Scientific Computing Software Infrastructure, along with the upgrades to the hardware infrastructure, will enable laboratory and university researchers to solve the most challenging scientific problems faced by the Office of Science at a level of accuracy and detail never before achieved. These developments will have significant benefits to all of the government agencies that rely on high-performance scientific computing to achieve their mission goals as well as to the U.S. high-performance computing industry.

Background: Advanced Computational Research in Fusion Science

The Office of Fusion Energy Sciences supports a directed, basic research program to understand the elementary processes in plasmas and to use this knowledge to explore innovative approaches for confining fusion plasmas. Theoretical and computational plasma physics are critical to a fundamental understanding of plasmas, and much progress has been made during the past 25 years. The solicitation is focused on accelerating progress toward developing a quantitative understanding of nonlinear, non-equilibrium plasma systems.

The scope and complexity of the proposed projects will require close collaboration among researchers from the computational and theoretical plasma physics, computer science and applied mathematics disciplines. Accordingly, this solicitation calls for the creation of topical centers as the organizational basis for a successful application. A topical center is a multi-institutional, multi-disciplinary team that will:

  • Create scientific simulation codes that take full advantage of terascale computers,
  • Work closely with other SciDAC teams to ensure that the best available mathematical algorithms and computer science methods are employed, and
  • Manage the work of the center in a way that will foster good communication and decision making (see section on Collaboration and Coordination below).

Partnerships among universities, national laboratories, and industry are encouraged.

Applications are being sought in the six topical areas listed below.

1. Turbulence and transport: An understanding of plasma turbulence is a prerequisite to the development of first-principles models of anomalous transport in magnetically confined plasmas. The development of accurate models for plasma turbulence and the availability of more powerful, massively parallel computers will enable comparison with experimental data in greater detail than has been achieved to date. In particular, comparisons for realistic experimental conditions, including profile effects, finite beta, Start Printed Page 359flow shear, and electron effects will lead to a better understanding of the relation between plasma turbulence and anomalous transport. The development of synthetic diagnostic tools and use of scientific visualization capabilities can facilitate this. Applications are solicited for the development of large-scale particle-in-cell (PIC) codes and continuum codes needed to understand turbulence and transport. The effort may include the development of a full-torus, continuum code. It is expected that the PIC codes will include the physics associated with kinetic electrons and electromagnetic fields, and that research will proceed on including neoclassical effects in continuum codes. An important element is understanding and reducing the differences between results obtained with PIC codes and continuum codes. Also there should be a focus on reducing code redundancy and on using object oriented techniques to facilitate code modernization and collaborative software development.

2. Macroscopic equilibrium and stability: Computational methods based on sets of magneto-fluid equations for magnetized plasma that includes the effects of realistic geometry and boundary conditions will improve the efficiency, realism and accessibility of 3-D magneto-fluid models of fusion plasmas. The nearly collisionless nature of high temperature plasmas can be taken into account by supplementing the fluid equations with particle-based closures of the moment equations. Development of user-friendly codes can be utilized to pioneer new applications in plasma and fusion science. For example, magneto-hydrodynamics should predict when sawtooth crashes and large-scale disruptions will occur. Applications are solicited for the development of large-scale 3-D magneto-fluid codes needed to understand large-scale phenomena in fusion plasmas. Test problems used to compare and validate computational models can also be employed to elucidate important physics. Goals include improving computational efficiency, integrating data management and visualization tools into the codes, addressing important programmatic problems in fusion science, and advancing understanding of fundamental plasma processes of wider scientific interest such as plasma relaxation and self-organization. Focus on utilizing modern computational techniques, such as object oriented programming, can facilitate code modernization and collaborative software development.

3. Magnetic reconnection: Magnetic reconnection is the process in a magnetized plasma system that converts magnetic energy into high-speed flows and thermal energy. Because it is the basis of an important plasma transport mechanism, it impacts many plasma systems ranging from laboratory experiments to the Earth's magnetosphere, the solar corona and the astrophysical environment. Exploration of diamagnetic stabilization, both in the linear and nonlinear phase of reconnection, is essential to understand the onset of reconnection in fusion experiments. Applications are solicited for a coordinated effort that will focus on the critical scientific issues required to model and understand magnetic reconnection in the high temperature plasmas of fusion interest and the plasmas of interest to the space and astrophysical communities. The project may involve the development of new techniques for treating multi-scale phenomena such as adaptive mesh refinement and the dynamic embedding of kinetic models. It is anticipated that the use of slab geometry and a comparison of a variety of different models will allow identification of the essential physics required in the description of reconnection in high temperature plasmas. The development of adaptive mesh algorithms applied to the localized regions where the components of the magnetic field reverse, and utilized in multi-fluid codes may facilitate the modeling of high temperature plasma systems with real parameters. The computational effort may yield simulation results for direct comparison with laboratory experiments. By including the full geometry of laboratory experiments in the simulations, it may be possible to explain the observation that in a hot toroidal plasma, despite the absence of complete reconnection, the plasma energy from the entire core is expelled. Focus on utilizing modern computational techniques, such as object oriented programming, can facilitate code modernization and collaborative software development.

4. Electromagnetic wave/particle interactions: Utilization of massively parallel processing will allow accurate predictive understanding of electromagnetic wave processes affecting heating, current drive, stability, and transport in fusion relevant plasmas. It is recognized that electromagnetic waves have the potential to penetrate high temperature plasmas and provide control of the various interacting processes at work in fusion plasmas. Wave-plasma interactions are described by large systems of partial differential equations of a complicated type that are neither elliptic nor hyperbolic. These systems of equations provide a challenging test bed for new iterative matrix inversion techniques. Applications are solicited for a coordinated effort to develop a mode conversion code that is self-consistently linked with antenna-wave coupling modules. This code should self-consistently include the plasma dielectric response due to wave-driven evolution of the particle distribution function on longer time scales. Massively parallel processor platforms are to be used to determine self-consistently phenomena that are important in the interaction between waves and plasma particles, for example, wave coupling, propagation, absorption, and wave-driven equilibrium evolution. There should be a focus on reducing code redundancy and on using object oriented techniques to facilitate code modernization and collaborative software development.

5. Boundary layer effects in plasmas: The performance of tokamaks, and other toroidal magnetic devices, is dependent on the dynamics of the edge region, which is the region that connects the hot core plasma through the separatrix to the material surface of the first wall. The edge region affects a whole variety of scientific issues ranging from confinement of hot fusion plasma to plasma-wall interactions and the technology of the first-wall design. Advances in understanding the non-linear edge plasma phenomena through development of appropriate modeling tools would be most beneficial. A major plasma science challenge results from the unique properties of edge plasmas. These unique properties include the widely varying space and time scales, the interplay between closed and open magnetic field lines, and physical processes that include atomic physics and both plasma-neutral and plasma-wall interactions. Applications are solicited for a coordinated research effort to utilize and develop tools that will aid in fundamental understanding of edge plasma turbulence and transport. Initial efforts may involve validation and verification of existing codes through in depth comparisons with one another, with existing edge databases, and with analytic theory. There should be a focus on reducing code redundancy and on using object oriented techniques to facilitate code modernization and collaborative software development. The resulting community based code should incorporate full geometry, macroscopic transport, kinetic effects, and plasma-neutral interactions. With the use of efficient parallel solvers and other Start Printed Page 360advanced numerical techniques, well-resolved simulations of the edge plasma should result.

6. Electromagnetic fields and beam dynamics in particle accelerators: The physics of intense ion beams needed for Inertial Fusion Energy is both rich and subtle, due to the kinetic and nonlinear nature of the system and the wide range in spatial and temporal scales involved. Effects associated with both instabilities and non-ideal processes must be understood. 3-D chamber calculations are required in order to provide a realistically complete model of the chamber environment. These calculations would allow exploration of various propagation modes. By employing multiple modes, it is possible to compare implicit electromagnetic methods, which can eliminate fast time scales not essential to the physics, and explicit electromagnetic methods. In the accelerator, the beam dynamics is nearly collisionless and Liouvillean, and as a result emittance growth primarily takes place through complicated distortions, driven by collective behaviors, imperfect applied fields, image fields from nearby conductors and inter-beam forces. With development of qualitatively improved tools it would be possible to establish much deeper understanding of these processes. Applications are solicited to develop a source-to-target simulation capability. This includes simulations of acceleration and confinement of the space-charge-dominated ion beams through the driver; electromagnetic and magneto-inductive simulations which describe the beam and fusion chamber environment, including multi-beam, neutralization, stripping, beam and plasma ionization processes, and return current effects; and simulations which can examine electron effects and collective modes in the driver and chamber. The code development may involve adoption of exiting codes to run on computers that use a hybrid of shared and distributed memory, production of new and improved numerical algorithms, e.g., averaging techniques that allow larger time-steps, and improved physics models. It is anticipated that modern scripting techniques for steering the code and advanced data visualization tools may be employed.

Collaboration and Coordination

It is expected that all applications submitted in response to this notice will be for collaborative centers involving more than one institution. Applications submitted from different institutions, which are directed at a common research activity, may include a common technical description of the overall research project but must have a qualified principal investigator, who is responsible for the part of the effort at each institution, and separate face pages and budget pages for each institution. In addition, if the distinct scope of work proposed for each institution is not specified in the common technical description, it must be clearly stated in the individual proposals. Applicants should include cost sharing whenever feasible. Synergistic collaborations with researchers in federal laboratories and Federally Funded Research and Development Centers (FFRDCs), including the DOE National Laboratories are encouraged, though no funds will be provided to these organizations under this Notice. Further information on preparation of collaborative proposals is available in the Application Guide for the Office of Science Financial Assistance Program that is available via the Internet at:​production/​grants/​Colab.html.

Since each center will be developing new computational tools and physics models that could be useful to other centers, it is important that there be good communication between the different centers. Also, it is important to have some guidance on code capabilities and development priorities from the broader fusion, scientific and computational communities. To facilitate this process the Office of Fusion Energy Sciences has established a community governed Plasma Science Advanced Scientific Computation Institute. This institute will be responsible for organizing regular coordination meetings and annual progress reviews. It will also coordinate development of priorities for future work and ensure good communication between the fusion centers and the other SciDAC activities.


Each potential applicant is strongly encouraged to submit a brief preapplication that consists of a two to three page narrative describing the proposed research, including research objectives and technical approach(s). Each preapplication should include a cover sheet with the title of the project, principal investigator, other senior personnel, institutions involved, and the name, telephone number, and e-mail address of the principal investigator. In addition, brief, one-page vitae should be submitted for the principal investigator and other senior personnel involved in the proposed center. Preapplications will be evaluated to assess their programmatic relevance, and a response will be provided to the principal investigator within 14 days of receipt. However, notification of a successful preapplication is not an indication that an award will be made in response to a formal application.

Program Funding

Approximately $1,700,000 of Fiscal Year 2001 funding will be available for grant awards in FY 2001. Additional funding for the proposed project may be available through the Office of Advanced Scientific Computing Research for closely related research in computer science and/or applied mathematics. Applications may request support for up to three years, with out-year support contingent on the availability of funds and satisfactory progress. To support multi-disciplinary, multi-institutional efforts, funding levels of $0.6 million to $1.2 million may be requested for the first year of the project, with higher funding levels possible in future years.

As required by the SC grant application guide, applicants must submit their budgets using the Budget Page (DOE Form 4620.1) with one Budget Page for each year of requested funding. The requested funding for the proposed work in computer science and applied mathematics should be included with the other project costs on the Budget Page. However, applicants are also requested to list the proposed computer science and applied mathematics costs separately in an appendix, as the Office of Advanced Scientific Computing Research may support this part of the work (up to 20-25% of the total project cost). The Office of Fusion Energy Sciences expects to fund two or three centers, depending on the size of the awards.


Applications will be subjected to scientific merit review (peer review) and will be evaluated against the following criteria listed in descending order of importance as codified in 10 CFR 605.10(d) (​production/​grants/​605index.html):

1. Scientific and/or technical merit of the project;

2. Appropriateness of the proposed method or approach;

3. Competency of the applicant's personnel and adequacy of the proposed resources;

4. Reasonableness and appropriateness of the proposed budget.

The evaluation of applications under item 1, Scientific and Technical Merit, will pay particular attention to: Start Printed Page 361

(a) The importance of the proposed project to the mission of the Office of Fusion Energy Sciences;

(b) The potential of the proposed project to advance the state-of-the-art in computational modeling and simulation of plasma behavior;

(c) The need for extraordinary computing resources to address problems of critical scientific importance to the fusion program and the demonstrated abilities of the applicants to use terascale computers; and

(d) The likelihood that the models, algorithms, and methods, that result from this effort will have impact on science disciplines outside of fusion research.

The evaluation under item 2, Appropriateness of the Proposed Method or Approach, will also consider the following elements related to Quality of Planning:

(a) Quality of the plan for effective collaboration among members of the center;

(b) Quality of plan for ensuring communication with other advanced computation efforts;

(c) Viability of plan for verifying and validating the models developed, including close coupling with experiments for ultimate validation; and

(d) Quality and clarity of proposed work schedule and deliverables.

Note that external peer reviewers are selected with regard to both their scientific expertise and the absence of conflict-of-interest issues. Non-federal reviewers may be used, and submission of an application constitutes agreement that this is acceptable to the investigator(s) and the submitting institution.

General information about development and submission of applications, eligibility, limitations, evaluations and selection processes, and other policies and procedures may be found in the Application Guide for the Office of Science (SC) Financial Assistance Program and in 10 CFR Part 605. Electronic access to SC's Financial Assistance Guide and required forms is made available via the Internet using the following Web site address:​production/​grants/​grants.html.

In addition, for this notice, project descriptions must be 25 pages or less, including tables and figures, but excluding attachments. The application must also contain an abstract or project summary, letters of intent from all non-funded collaborators, and short curriculum vitae of all senior personnel. On the SC grant Face Page (DOE Form 4650.2), in block 15, also provide the PI's phone number, FAX number, and e-mail address.

Start Signature

The Catalog of Federal Domestic Assistance Number for this program is 81.049, and the solicitation control number is ERFAP 10 CFR Part 605.

Issued in Washington, DC on December 21, 2000.

Ralph H. De Lorenzo,

Acting Associate Director of Science for Resource Management.

End Signature End Supplemental Information


1.  This workshop was sponsored by the National Science Foundation and the Department of Energy and hosted by the National Academy of Sciences on July 30-31, 1998. Copies of the report may be obtained from:​production/​octr/​mics/​index.html

Back to Citation

2.  Copies of the PITAC report may be obtained from​ac/​report/​.

Back to Citation

3.  Copies of the SC computing plan, Scientific Discovery through Advanced Computing, can be downloaded from the SC web site at:​production/​octr/​index.html.

Back to Citation

[FR Doc. 01-78 Filed 1-2-01; 8:45 am]