High Performance I/O

Project Overview

Problem: Large-scale Data Movement to/from MPP

 

Current requirements (GTC - Jaguar):

   512x1 or 1024x1: compute cores -> I/O nodes, TBs/hour, 120 hours

   multiple, simultaneous data feeds: e.g., diagnostics, analysis, restart

   controlled data drain to limit perturbation

 

Future requirements:

   Consistent I/O Bandwidths of 18 GB/s

   Remote access for continuous diagnostics

   Dealing with burstiness and real-time requirements (e.g., ITER)

 

Conclusion:

    ! synch. file writes -> store -> analyze/visualize will not scale !

        ! need asynch. operation + `in transit’ data processing !