Abstract. IASI (Infrared Atmospheric Sounding Interferometer) is the core instrument of the currently three Metop (Meteorological operational) satellites of EUMETSAT (European Organization for the Exploitation of Meteorological Satellites). The MUSICA IASI processing has been developed in the framework of the European Research Council project MUSICA (MUlti-platform remote Sensing of Isotopologues for investigating the Cycle of Atmospheric water). The processor performs an optimal estimation of the vertical distributions of water vapour (H2O), the ratio between two water vapour isotopologues (the HDO / H2O ratio), nitrous oxide (N2O), methane (CH4), and nitric acid (HNO3), and works with IASI radiances measured under cloud-free conditions in the spectral window between 1190 and 1400 cm−1. The retrieval of the trace gas profiles is performed on a logarithmic scale, which allows the constraint and the analytic treatment of ln[HDO] – ln[H2O] as proxy for the HDO / H2O ratio. Currently, the MUSICA IASI processing has been applied to all IASI measurements available between October 2014 and April 2020, so more than 1.4 billion individual retrievals have been performed. Here we describe the MUSICA IASI full retrieval product data set. The data set is made available in form of netcdf data files that are compliant with version 1.7 of the CF (Climate and Forecast) metadata convention. For each orbit an individual standard output data file is provided. These files contain for each individual retrieval information on the a priori usage and constraint, the retrieved atmospheric trace gas and temperature profiles, profiles of the leading error components, information on vertical representativeness in form of the averaging kernels as well as averaging kernel metrics, which are more handy than the full kernels. We discuss data filtering options and give examples of the high horizontal and continuous temporal coverage of the MUSICA IASI data products. The standard output data files provide comprehensive information for each individual retrieval resulting in a rather large data volume (about 25 TB for the more than five years of data with global daily coverage). This at a first glance apparent drawback of large data files and data volume is counterbalanced by multiple possibilities of data reusability, which are briefly discussed. In an extended output data file the same variables as in the standard output data files are provided in addition to Jacobians for many different uncertainty sources and Gain matrices (due to this additional variables it is called the extended output). It is limited to 74 observations over a polar, mid-latitudinal and tropical site. We use this additional Jacobian and Gain data for assessing the typical impact of different uncertainty sources – like surface emissivity or spectroscopic parameters – and different cloud types on the retrieval results. We offer two data packages with DOI for free download via the repository RADAR4KIT. The first data package has a data volume of about 17.5 GB and is linked to https://doi.org/10.35097/408 (Schneider, et al., 2021b). It contains example standard output data files for all MUSICA IASI retrievals made for a single day (more than 0.6 million). Furthermore, it includes a ReadMe.pdf file with a description of how to access the total data set (the 25 TB) or parts of it. This data package is for users interested in the typical global daily data coverage and in information about how to download the large data volumes of global daily data for longer periods. The second data package is linked to https://doi.org/10.35097/412 (Schneider et al., 2021a) and contains the extended output data file. Because it provides data for only 74 example retrievals, its data volume is only 73 MB and it is thus recommended to users for having a quick look on the data.
P2C: Understanding Output Data Files via On-the-Fly Transformation from Producer to Consumer Executions