Distributed Computing for Urgent NWS Requirements

Distributed Computing for Urgent NWS Requirements

1. Introduction
2. Problem Statement
3. Existing design
4. Proposed design
5. Benefits
6. Manpower requirements
7. Budget (personnel, hardware, infrastructure)

High performance computing (HPC) tends to follow Moore's Law in its growth pattern. The applications that rely on HPC, however, seem to be scaling at a rate exceeding the growth of HPC resources. Finally, acquisition of NOAA HPC assets is on a fixed schedule, and it is often impossible to anticipate just how fast applications will grow and overtake available resources.

In this proposal we will examine the processes necessary to allow new and existing applications to utilized non-NOAA HPC resources on a preferential, short-fuse basis. We have dubbed this effort the Geosciences Urgent Computing Experiment (GUCE, pronounced "goose"). In GUCE, we will investigate
the processes necessary to promote low latency submission of urgent jobs using up to 2048 CPU cores, including the migration, on-demand existing data, storage and post-processing/analysis requirements.