[IDV #VFL-607764]: Problem getting time dimension specified for loading data using 'Directory' option and looping over time

Hi Dave,

The NcML file can be loaded through the feeling lucky option, so that's ok.

I'm not sure why it's only loading one grid - I am able to see all three
on my machine. Where are you located? Maybe I could drop by and see
what's up.

One quick test would be to try loading the ncml file from a fresh start
of the IDV, in case there are any cache issues at play.

Cheers,

Sean

> Hi Sean,
> 
> Thanks for the quick feedback.  I used the 'feeling lucky' option as
> I'm not sure which option the .ncml file is supposed to be used with.
> My directory structure is as follows:
> 
> saudi-c1-int1.rap.ucar.edu:/var/autofs/mnt/qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/test%
> ls -la
> total 3936
> drwxrwxr-x 2 gochis rap    4096 Jun 27  2013 .
> drwxrwxr-x 3 gochis rap    4096 Jun 27  2013 ..
> -rw-rw-r-- 1 gochis rap 1338516 Jun 26 17:11
> 2008052000.LDASOUT_DOMAIN1.nc.nc
> -rw-rw-r-- 1 gochis rap 1338344 Jun 26 17:11
> 2008052100.LDASOUT_DOMAIN1.nc.nc
> -rw-rw-r-- 1 gochis rap 1338520 Jun 26 17:11
> 2008052200.LDASOUT_DOMAIN1.nc.nc
> -rw-r--r-- 1 gochis rap     322 Jun 27  2013 LDASOUT_DOMAIN1.ncml
> saudi-c1-int1.rap.ucar.edu:/var/autofs/mnt/qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/test%
> 
> 
> So, I edited the .ncml file to be the following to point to this
> directory:
> 
> saudi-c1-int1.rap.ucar.edu:/var/autofs/mnt/qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/test%
> cat LDASOUT_DOMAIN1.ncml
> <?xml version='1.0' encoding='UTF-8'?>
> <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'>
> <aggregation dimName='time' type='joinExisting'>
> <dimension name="time" orgName="Time" />
> <scan
> location='/qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/test'
> suffix='.nc' />
> </aggregation>
> </netcdf>
> saudi-c1-int1.rap.ucar.edu:/var/autofs/mnt/qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/test%
> 
> 
> 
> Lastly, if it helps I have put the 3 input files on my ftp site at:
> ftp.rap.ucar.edu/incoming/irap/gochis/
> 
> 
> Thanks again for the help!
> -Dave
> 
> 
> 
> 
> 
> On 2013-06-27 10:38, Unidata IDV Support wrote:
> > Hi Dave,
> >
> > There should't be anything else if all of the files live in the top
> > level
> > of the directory identified in the scan tag. How is your directory
> > structure
> > laid out?
> >
> > Also, when you load the ncml file, what Data Source Type are you
> > using
> > (the drop-down box on the Data Choosers Tab)?
> >
> > Sean
> >
> >> Hi Sean,
> >>
> >> OK, I did what you suggested and one file loaded. Is there an
> >> additional wildcard or something we need to add to the .ncml file to
> >> get
> >> all files to load?
> >>
> >> -Dave
> >>
> >> On 2013-06-27 10:24, Unidata IDV Support wrote:
> >> > Greetings David,
> >> >
> >> > The IDV aggregation code is a little...shall we say, picky.
> >> > Currently, the time dimension must
> >> > be named "time" for this feature to work (case sensitive).
> >> >
> >> > In order to aggregate your files, try using the attached ncml
> >> file.
> >> >
> >> > If you open the NcML file, you will see on the 5th line a tag like
> >> > this:
> >> >
> >> > scan location='LDASOUT' suffix='.nc'
> >> >
> >> > Change the location to the directory on disk where your data are
> >> > stored. If you
> >> > then load the ncml file into the idv through the Data Choosers ->
> >> > Files, then
> >> > the aggregation should work.
> >> >
> >> > Please let me know how it turns out.
> >> >
> >> > Cheers,
> >> >
> >> > Sean
> >> >
> >> >> Full Name: David Gochis
> >> >> Email Address: address@hidden
> >> >> Organization: NCAR/RAL
> >> >> Package Version: 4.0u1 build date:2013-03-29 18:10 UTC
> >> >> Operating System: Linux
> >> >> Hardware: Java: home:
> >> >> /var/autofs/mnt/linux-amd64/debian/squeeze/local/IDV_4.0u1/jre
> >> >> version: 1.6.0_41 j3d:1.5.2 fcs (build4)
> >> >> Description of problem: There was an error loading the data:
> >> >> Error creating data source:file.grid with:
> >> >>
> >> /qnap-25/NDHMS/cases/Col_Headwat/tmp_data/test/2008052100.LDASOUT_DOMAIN1.nc.nc
> >> >> Grid data source failed making data set:
> >> >>
> >> /home/gochis/.unidata/idv/DefaultIdv/tmp/multigrid_3a11d392-abc3-4ef0-ac8c-25c0394b1b910.ncml
> >> >> Dimension not found= time
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Hi there,
> >> >> I am attempting to load several files at once which are sequenced
> >> in
> >> >> time. I've been trying to add the appropriate time variables to
> >> these
> >> >> files using nco commands. I have been mimicking other file
> >> formats
> >> >> that seem to work ok but these files are not working. I get the
> >> error
> >> >> message above upon clicking the 'Add Source' button in  Data
> >> Chooser
> >> >> -> Directory option.  Can you advise on what is missing/wrong
> >> with the
> >> >> attached file?  The attached file is a single time (the first
> >> time) of
> >> >> output. The other files simply increment the 'time' variable by
> >> the
> >> >> seconds since the start of the run (e.g. 3600, 7200, etc).
> >> >>
> >> >> Thank you,
> >> >> Dave Gochis
> >> >>
> >> >> ******************
> >> >> Stack trace:
> >> >> java.lang.IllegalArgumentException: Dimension not found= time
> >> >> at
> >> >>
> >> ucar.nc2.ncml.AggregationOuterDimension$DatasetOuterDimension.getNcoords(AggregationOuterDimension.java:656)
> >> >> at
> >> >>
> >> ucar.nc2.ncml.AggregationOuterDimension$DatasetOuterDimension.setStartEnd(AggregationOuterDimension.java:675)
> >> >> at
> >> >>
> >> ucar.nc2.ncml.AggregationOuterDimension.buildCoords(AggregationOuterDimension.java:161)
> >> >> at
> >> >>
> >> ucar.nc2.ncml.AggregationExisting.buildNetcdfDataset(AggregationExisting.java:97)
> >> >> at ucar.nc2.ncml.Aggregation.finish(Aggregation.java:404)
> >> >> at ucar.nc2.ncml.NcMLReader.readNetcdf(NcMLReader.java:492)
> >> >> at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:437)
> >> >> at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:252)
> >> >> at ucar.nc2.ncml.NcMLReader.readNcML(NcMLReader.java:202)
> >> >> at
> >> >>
> >> ucar.nc2.dataset.NetcdfDataset.acquireNcml(NetcdfDataset.java:841)
> >> >> at
> >> >>
> >> ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:680)
> >> >> at
> >> >>
> >> ucar.nc2.dataset.NetcdfDataset.openDataset(NetcdfDataset.java:418)
> >> >> at
> >> >>
> >> ucar.nc2.dataset.NetcdfDataset.acquireDataset(NetcdfDataset.java:513)
> >> >> at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:104)
> >> >> at ucar.nc2.dt.grid.GridDataset.open(GridDataset.java:90)
> >> >> at
> >> >>
> >> ucar.unidata.data.grid.GeoGridDataSource.doMakeDataSet(GeoGridDataSource.java:1113)
> >> >> at
> >> >>
> >> ucar.unidata.data.grid.GeoGridDataSource.getDataset(GeoGridDataSource.java:1135)
> >> >> at
> >> >>
> >> ucar.unidata.data.grid.GeoGridDataSource.doMakeDataChoices(GeoGridDataSource.java:1162)
> >> >> at
> >> >>
> >> ucar.unidata.data.DataSourceImpl.getDataChoices(DataSourceImpl.java:1852)
> >> >> at
> >> >>
> >> ucar.unidata.data.FilesDataSource.sourcesChanged(FilesDataSource.java:418)
> >> >> at
> >> >>
> >> ucar.unidata.data.grid.GeoGridDataSource.sourcesChanged(GeoGridDataSource.java:591)
> >> >> at
> >> >>
> >> ucar.unidata.data.FilesDataSource.initWithPollingInfo(FilesDataSource.java:372)
> >> >> at
> >> >>
> >> ucar.unidata.data.FilesDataSource.initAfterCreation(FilesDataSource.java:283)
> >> >> at
> >> >>
> >> ucar.unidata.data.grid.GeoGridDataSource.initAfterCreation(GeoGridDataSource.java:496)
> >> >> at
> >> >>
> >> ucar.unidata.data.DataManager.createDataSource(DataManager.java:1597)
> >> >> at
> >> >>
> >> ucar.unidata.idv.IntegratedDataViewer.createDataSource(IntegratedDataViewer.java:1980)
> >> >> at
> >> >>
> >> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1897)
> >> >> at
> >> >>
> >> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1831)
> >> >> at
> >> >>
> >> ucar.unidata.idv.IntegratedDataViewer.makeDataSource(IntegratedDataViewer.java:1804)
> >> >> at
> >> >>
> >> ucar.unidata.idv.chooser.IdvChooser.makeDataSource(IdvChooser.java:943)
> >> >> at
> >> >>
> >> ucar.unidata.idv.chooser.PollingFileChooser.doLoadInThread(PollingFileChooser.java:192)
> >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> >> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> >> Source)
> >> >> at java.lang.reflect.Method.invoke(Unknown Source)
> >> >> at ucar.unidata.util.Misc$2.run(Misc.java:1221)
> >> >> at ucar.unidata.util.Misc$3.run(Misc.java:1249)
> >> >>
> >> >>
> >> >>
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: VFL-607764
> >> > Department: Support IDV
> >> > Priority: Normal
> >> > Status: Open
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: VFL-607764
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> 
> 


Ticket Details
===================
Ticket ID: VFL-607764
Department: Support IDV
Priority: Normal
Status: Open