[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #BDO-300797]: catalog choking on folder of OK .ncml files

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [IDV #BDO-300797]: catalog choking on folder of OK .ncml files
  • Date: Fri, 21 Nov 2014 15:38:14 -0700
Hi Brian,

re: IDV Catalog listing of directory with lots of NCML files that reference data
through URLs

> Yes I did this and it never came back.
> Help ??

Julien and I proved that the catalog view in the IDV will _eventually_
show all of the NCML files.  We did this by:

- create a new server side view of a single NCML file

  This listing is reasonably fast.

- create a server side view of 10 NCML files

  This listing is slow, but not slow enough to look like the IDV is
  hung.

- full set

  We didn't bother to wait for this to complete, but we are convinced
  that it eventually will.

What is going on?

Evidently, netCDF Java is actually opening the dataset referenced in each
URL named in the NCML files.  It seems to take between 4 and 5 seconds to
get the information from the 7 URLs referenced in each NCML file.  If the
wait time scales linearly, then it should take 335 (5 x 67) seconds to generate
the list for all of the NCML files you have in the NCEP1 6 hourly pressure level
data server side set of files.

Can this be sped up?

We don't know.  We'll have to run more tests to determine if the slowness
is really in the netCDF Java code or simply a slowness in the ESRL web
server's response to the query being made.

Sorry we didn't have better news!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: BDO-300797
Department: Support IDV
Priority: Normal
Status: Closed