This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
>To: address@hidden >From: Ben Foster <address@hidden> >Subject: Re: 20000207: netcdf i/o on ibm vs cray >Organization: HAO >Keywords: slow netCDF I/O on Cray Hi Ben, > I have written a netcdf file with time as the unlimited > dim, with 7482 records. The time dimension is called "day" > and there is an int coord var called day(day) (yyddd). > > When I do an ncdump -c on the file on the ibm (e.g. blackforest), > it prints the 7482 integers in a few seconds, but on the Cray > (chipeta) it takes several minutes. I don't think this is > happening because of load on the machines. Is the i/o really > that much slower on the Crays than the IBM? > > FYI, the file is on mss /TGCM/data/ncep_79001-99177.nc > (about 161 MB). Here is the ncdump with the first 20 days: First, sorry it's taken so long to reply; I've been going to a lot of all-day meetings recently. The default configuration for use on the Cray uses an FFIO specification that favors sequential access, but you can probably do much better by playing with the NETCDF_FFIOSPEC environment variable. For some possible specifications for various I/O scenarios, see: http://www.unidata.ucar.edu/packages/netcdf/guidec/guidec-14.html#HEADING14-0 or try setting NETCDF_FFIOSPEC to "cachea:8:256:0" before running ncdump to see if it makes a difference. Some users have reported huge performance differences on Crays by changing this FFIO specification. See, for example: http://www.unidata.ucar.edu/glimpse/netcdf/3261 --Russ