Re: 971008: Speed of Data Access in NetCDF

>To: address@hidden
>From: address@hidden
>Subject: Speed of Data Axxess
>Organization: Memorial Sloan Kettering Cancer Center
>Keywords: 199710081902.NAA13025

Hi Dr. Erdi,

>       The system developed at our center Memorial Sloan Kettering
> Cancer Center is using netCDF 1.3 version (dating to 1993) running on
> OpenVMS 6.2. We are reading large data files (>50Mbytes) and saving
> them on a network of 100 MB/sec. It takes about 50 seconds which is
> too slow.
> 
> In your later versions is there an improvement in speed for data
> access?
> 
> If I update the netCDF software to version 3 will I have speed
> improvements?

Yes, it will probably be faster.  The new version may be much faster,
depending on whether you are writing out arrays of floating-point
numbers or not, and whether your OpenVMS platform uses IEEE
floating-point as its native representation.

If you are writing out floating-point arrays, the new version is
probably much faster because it replaces one-function-call-per-float
with a single call for an array of floats, which eliminates much of the
function call overhead.  There are additional efficiencies in
optimizations possible when converting from native floating-point to IEEE
format, if that conversion is necessary.

It's definitely worth a try.  We would be interested in hearing about
the results.

--Russ