OzFlux Data: Network integration from collection to curation
Abstract. Measurement of the exchange of energy and mass between the surface and the atmospheric boundary-layer by the eddy covariance technique has undergone great change in the last two decades. Early studies of these exchanges were confined to brief field campaigns in carefully controlled conditions followed by months of data analysis. Current practice is to run tower-based eddy covariance systems continuously over several years due to the need for continuous monitoring as part of a global effort to develop local-, regional-, continental- and global-scale budgets of carbon, water and energy. Efficient methods of processing the increased quantities of data are needed to maximise the time available for analysis and interpretation. Standardised methods are needed to remove differences in data processing as possible contributors to observed spatial variability. Furthermore, public availability of these datasets assists with undertaking global research efforts. The OzFlux data path has been developed (i) to provide a standard set of quality control and post-processing tools across the network, thereby facilitating inter-site integration and spatial comparisons; (ii) to increase the time available to researchers for analysis and interpretation by reducing the time spent collecting and processing data; (iii) to propagate both data and metadata to the final product; and (iv) to facilitate the use of the OzFlux data by adopting a standard file format and making the data available from web-based portals. The fundamentals of the OzFlux data path include the adoption of netCDF as the underlying file format to integrate data and metadata, a suite of Python scripts to provide a standard quality control, post-processing, gap filling and partitioning environment, a portal from which data can be downloaded and an OPeNDAP server offering internet access to the latest version of the OzFlux data set. Discovery of the OzFlux data set is facilitated through incorporation in FluxNet data syntheses and the publication of collection metadata via the RIF-CS format. This paper serves two purposes. The first is to describe the datasets, along with their quality control and post-processing, for the other papers of this Special Issue. The second is to provide an example of one solution to the data collection and curation challenges that are encountered by similar flux tower networks worldwide.