VSC Training Course: Parallel I/O and Portable Data Formats (HDF5 and NetCDF)

    Course Material: 

    The course material is available for registered attendees only.

    Course Description: 

    Processing large data sets on high-performance computing (HPC) systems including reading input data and writing results faces an ever growing need for scalability.

    In addition, input and output data are often processed on different, heterogeneous platforms. Conversion processes tend to be hard to maintain and can be prone to errors.

    Portable, self-describing data formats such as HDF5 and netCDF can help to solve these problems. They are widely used in scientific computing across multiple disciplines.

    Course Contents:

    This course will give an overview on the basics of HPC parallel I/O and introduces to the use of the HDF5 and netCDF library interfaces on HPC systems.

    HPC I/O

    General introduction to Parallel I/O and Portable Data Formats

    Benefits of Parallel I/O with NetCDF

    • Introduction
    • File format
    • Difference PnetCDF <-> NetCDF4
    • Basic file handling using PnetCDF
    • Partial I/O
    • Performance hints

    Benefits of Parallel I/O with HDF5

    • Introduction
    • File format
    • Creating HDF5 datasets
    • Attributes
    • Groups
    • Reading and writing array datatypes
    • Partial I/O
    • Parallel HDF5
    • Performance hints

    Date:

    19.10.2015, 9:00 - 17:00

    Course Language:

    English

    Course Format:

    Lecture with hands-on exercises.

    Practical exercises require the use of a bring-along laptop.

    Prerequisites:

    Participants should have an VSC-3 account and be familiar with accessing the systems and submitting batch jobs. If you don't have an account yet a training account will be provided for the course.

    Location (changed):

    Technische Universität Wien
    Festsaal, 1st floor,
    Karlsplatz, 1040 Vienna, Austria

    Costs:

    This course is free of charge for accepted participants.

    Instructor:

    Sebastian Lührs, Jülich Supercomputing Centre

    Application/Registration:

    Please register via email to michael.alexander@tuwien.ac.at, confirmations will be sent on a rolling basis within a week.

    Due to available space, the maximum number of participants is limited.

    Contact:

    Michael Alexander: +43158801420756, michael.alexander@tuwien.ac.at