[netCDF #TCU-710461]: netcdf on lustre

Hi Jesse,

> I have an HPC cluster using lustre as our backend file systems. The
> cluster serves primarily weather models, such as the WRF and GFS.
> 
> One thing we observed is that netcdf writes can often be very slow on
> lustre.  Do you have any recommended tuning procedures for netcdf on lustre?

No, sorry, we don't currently test on lustre.  However, if you have configured
lustre with a large disk block size and are writing netCDF files with lots of
records and lots of record variables (variables that use an unlimited dimension)
then you could be seeing a problem with writing such data a variable at at time
instead of a record at a time:

  https://www.unidata.ucar.edu/jira/browse/NCF-142

You haven't said what version of the library you're using, but the fix above is
in the nccopy utility in version 4.2, and in some of the utilities in the most
recent release of NCO (the NetCDF Operators software from UC Irvine).  

Also, are you using parallel I/O?  Use of parallel-netcdf may be a solution 
worth
looking at if you're writing classic-format files, or the HDF5-based parallel 
I/O
in netCDF-4 otherwise.

If you have a small example that demonstrates the bad performance, we could try 
to 
reproduce it and diagnose the problem.

--Russ

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: TCU-710461
Department: Support netCDF
Priority: Normal
Status: Closed