[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Urgent Question about NetCDF

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: Urgent Question about NetCDF
  • Date: Thu, 14 Nov 2013 11:12:27 -0700
Hello again, Dick:

From the information in your note, it is not clear what is needed to speed up your processing of those ~2000 daily netCDF files.   However, since that naar library page was published a couple years ago, Unidata has developed Windows-specific netCDF binary libraries that are available at:

http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html

We can't say for sure that these libraries will speed up your processing, but they are probably much more up to date than the ones you are using so it is a place to start.

There are other ideas for aggregating the 2000 files into a single logical dataset via a THREDDS server or using the netCDF Java library for aggregation.   Another thought is to concatenate the files into one physical file using the NetCDF Operators.   But more information is needed -- especially about where the bottlenecks are -- to gain a sense of whether that sort of approach would speed things up or not.

If you have any questions about the Windows netCDF binaries available at the URL cited above, please contact Ward Fisher.   I'll send Ward a copy of this note and I'll send a copy to support-netcdf so this interaction gets into our support database.

I hope this helps.   Good luck and we'll see you soon.

--  Ben


On Thu, Nov 14, 2013 at 1:48 AM, Dick M.A. Schaap <address@hidden> wrote:
Dear Ben,

As MARIS we are involved in the EMODNet Physics project where we every day download ca 2000 NetCDF files from the European operators of ca 900 metocean stations (EuroGOOS) with 24 hours of timeseries of waves, currents, winds, temperature, sealevel, etc

Thereafter we extract those files into our local database and use it to provide the EMODNet users with 60 days graphics of the parameters per station.

Our platform: Windows

Development environment: .Net van microsoft

Used netcdf library: Scientific data-set library and tools

Url naar library:                                 http://research.microsoft.com/en-us/downloads/ccf905f6-34c6-4845-892e-a5715a508fa3/

This extraction of reading and writing takes us several hours per day, also because some NetCDF files are ca 2 Mb each includingb multiple parameters at multiple depth levels.

Can you advise us another library under Windows for faster extraction?

Hoping to hear from you soon.

Regards

Dick M.A. Schaap