VSC Training Course: Parallel I/O, December 6, 2019


    This course will be about parallel I/O with a special focus on portable data formats. It will introduce the use of the HDF5 and NetCDF (NetCDF4 and PnetCDF) library interfaces, and hands-on exercises (in C/C++ or Fortran) will allow to immediately test and understand their usage. Performance hints, optimization potential, and best practices for I/O will be discussed in detail throughout the whole course.

    Numerical simulations conducted on current HPC systems face an ever growing need for scalability pushing the limitations on size and properties that can be accurately simulated. Therefore, ever larger data sets have to be processed, be it reading input data or writing results. Serial approaches on handling I/O in a parallel application will dominate the performance on massively parallel systems, leaving a lot of computing resources idle during those serial I/O phases.

    In addition to the need for parallel I/O, input and output data is often processed on different and maybe even heterogeneous platforms. Conversion processes can impose a high level of maintenance when different data representations are needed. Portable, self-describing data formats such as HDF5 and netCDF can help to solve these problems.

    Agenda (preliminary) & Content:

    08:45 - 09:00   ––––––– Registration –––––––
    09:00 - 09:30   Parallel I/O Strategies
    09:30 - 11:00   Parallel NetCDF (PnetCDF/NetCDF4)
    11:00 - 11:30   ––––––– Coffee Break –––––––
    11:30 - 12:30   Parallel NetCDF (PnetCDF/NetCDF4)
    12:30 - 13:00   HDF5
    13:00 - 14:00   ––––––– Lunch Break –––––––
    14:00 - 15:00   HDF5
    15:00 - 15:30   ––––––– Coffee Break –––––––
    15:30 - 16:00   HDF5
    16:00 - 17:00   Optimization and Profiling
    17:00 - 17:00   ––––––– End of Course –––––––


    Unix/Linux and either C/C++ or Fortran in particular; a basic understanding in parallel programming with MPI would be advantageous but is not mandatory.


    Sebastian Lührs (Jülich Supercomputing Centre)



    Date, Time, and Location:

    06.12.2019, 08:45 - 17:00,
    FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)


    Registration for this course is closed.

    Registration deadline is Monday, November 11, 2019, with priority rules. Acceptance will be approved on November 12, 2019 As long as seats are available there will be an extended registration period without priority rules.

    Priority for acceptance: first - active users of the VSC systems, second - students and members of Austrian universities and public research institutes, third - other applicants.


    VSC users: none
    Students and members of Austrian universities and public research institutes: none
    Students and members of other universities and public research institutes: 60 €
    Others: 200 €

    Information about payment will be provided with the confirmation email.

    Coffee breaks are included in the course fee (lunch is not included).

    Slides & Exercises:

    Opens external link in new windowSlides [pdf] - Sebastian Lührs: Parallel I/O and Portable Data Formats

    Opens external link in new windowExercises [tar.gz] - Sebastian Lührs: Parallel I/O and Portable Data Formats

    Local Organizer / Contact:

    Claudia Blaas-Schenner, vsc-seminar@list.tuwien.ac.at


    Training events of VSC:

    Opens external link in new windowvsc.ac.at/training