[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDV #RLV-572542]: Error loading .nc files using "Aggregate Grids by Time"



Rich, Alexa,

Actually I asked John Caron about fixing this with NcML, but we could not come 
up with a good solution. CF requires degrees_east only (there is no 
degrees_west via NcML). 
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#longitude-coordinate

Again, I suggest fixing the data file. You never know who is going to use your 
data down the road and you don't want them tripped up by the same problem.

Best,

Unidata IDV Support

> Alexa,
> If you want to try that short bit of Python code to modify your files, the
> USGS has a site license for Enthought Python (which includes Netcdf4python)
> which works on 32/64bit Windows, Mac and Linux.   All you need to do is:
> 1. Register at https://www.enthought.com/accounts/register/  using your
> usgs e-mail address
> 2. Install Canopy Express https://www.enthought.com/store/
> 3. Go to Canopy Package Manager, search for "netcdf" and nstall "netcdf4"
> 
> Alternatively, you could create a free account at Wakari.io, upload your
> netcdf files, run the python code, and then download them again.
> 
> But if you want to just flip your longitude values, it seems you *could* do
> this in NcML using one of two methods:
> 1) You could just change the units of longitude from "degrees_east" to
> "degrees_west" (or vice versa).
> 2) You could actually write out the flipped longitude values and use the
> <values> tag to specify them in the NcML.
> 
> -Rich
> 
> 
> On Wed, Dec 18, 2013 at 7:28 PM, Alexa Van Eaton <address@hidden> wrote:
> 
> > Hi there - many thanks for your email. I will check with my colleague
> > Larry Mastin (Ccd) to see if there could be a way to accomplish what you've
> > described using Fortran. Seems worth a shot!
> >
> > All the best
> >
> > Alexa
> >
> > address@hidden> wrote:
> >
> > > Alexa,
> > >
> > > One more thing: We will also be offering this in November of 2014:
> > https://github.com/Unidata/tds-python-workshop
> > >
> > > Best Wishes,
> > >
> > > Unidata IDV Support
> > >
> > >> Alexa,
> > >>
> > >> Unfortunately, strictly speaking we don't really offer help in this
> > area (though I am hoping this will change in the future by offering a cloud
> > based ipython notebook server tailored with Unidata technology including
> > netcdf4-python). You can try the Python Enthought distribution though at
> > the USGS you'll have to pay for a license (I think). I am CCing your
> > colleague Dr. Rich Signell. Rich may have some ideas on getting you going
> > with netcdf4-python at the USGS.
> > >>
> > >> Best Wishes,
> > >>
> > >> Unidata IDV Support
> > >>
> > >>> Thanks again for your helpful reply. I had some trouble running the
> > python
> > >>> script - do I need to have the netCDF version 4 library implemented on
> > top
> > >>> of HDF5? I don't believe I have either of those yet, but could do if it
> > >>> means fixing this problem and not having to re-run the entire
> > simulation(!).
> > >>>
> > >>> All the best
> > >>>
> > >>> Alexa
> > >>>
> > >>>
> > >>> address@hidden> wrote:
> > >>>
> > >>>> Alexa,
> > >>>>
> > >>>> NcML can work a lot of magic, but in your case you may be out of
> > luck. I
> > >>>> noticed the data are not evenly spaced along the grid (is this true?)
> > in
> > >>>> the longitude dimension. If the data had been evenly spaced you could
> > maybe
> > >>>> have used
> > >>>>
> > http://www.unidata.ucar.edu/software/thredds/v4.4/netcdf-java/ncml/Cookbook.html#valuestofix
> >  the grid values.
> > >>>>
> > >>>> But the root issue is you need to fix the data file. After all,
> > someone
> > >>>> may use your data down the road and get tripped up over this longitude
> > >>>> problem.
> > >>>>
> > >>>> I actually think you can fix these data fairly easily with a bit of
> > >>>> Python. In fact here is that bit of Python:
> > >>>>
> > >>>> import netCDF4
> > >>>> f = netCDF4.Dataset('/tmp/netCDF_PIC_119.nc', 'r+')
> > >>>> lon = f.variables['lon']
> > >>>> lon[:] = lon[:]*-1
> > >>>> f.close()
> > >>>>
> > >>>> When I open your NcML aggregation, the data end up over Kenai, Alaska
> > (Mt.
> > >>>> Redoubt?)
> > >>>>
> > >>>> Best Wishes,
> > >>>>
> > >>>> Unidata IDV Support
> > >>>>
> > >>>>> Thank you so much for your reply. This works perfectly!
> > >>>>>
> > >>>>> I wonder - can I also write a little script in the NCML to read in
> > the
> > >>>>> longtitude as degrees west (instead of the default degrees east)? I
> > >>>> should
> > >>>>> have used a negative number (instead of positive) to specify the
> > latitude
> > >>>>> the Alaskan volcano in my simulation, so now the data are plotting
> > >>>>> somewhere in Russia.
> > >>>>>
> > >>>>> Thanks again - I really appreciate your help, this has opened up so
> > many
> > >>>>> possibilities for our work on eruption dynamics.
> > >>>>>
> > >>>>> All the best
> > >>>>>
> > >>>>> Alexa
> > >>>>>
> > >>>>>
> > >>>>> address@hidden> wrote:
> > >>>>>
> > >>>>>> Alexa,
> > >>>>>>
> > >>>>>> Sorry for the delayed response. We have been short-staffed here on
> > >>>> account
> > >>>>>> of the AGU conference.  The IDV is trying to aggregate the data but
> > >>>> can't
> > >>>>>> find an aggregation dimension of "time". This can be solved fairly
> > >>>> simply
> > >>>>>> by writing your own NcML (
> > >>>>>>
> > >>>>
> > http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ncml/v2.2/Aggregation.html
> > >>>> ).
> > >>>>>> Attached you will find an example NcML file that should work for
> > your
> > >>>> data.
> > >>>>>> Note I replaced the aggregation dimension "dimName" from "time" to
> > >>>> "date".
> > >>>>>> Please give that a try. It looks like you are working with
> > interesting
> > >>>>>> volcanic data.
> > >>>>>>
> > >>>>>> Please keep us up-to-date on your progress.
> > >>>>>>
> > >>>>>> Best Wishes,
> > >>>>>>
> > >>>>>> Unidata IDV Support
> > >>>>>>
> > >>>>>>
> > >>>>>>> Hi there - thanks for your reply.
> > >>>>>>>
> > >>>>>>> I've uploaded two of the NetCDF files I'm been working with. I'm
> > not
> > >>>>>>> actually sure where the NCML is located, however, I'm using the
> > >>>> latest
> > >>>>>>> release of the IDV (v 4.1) so I assume there is a default location
> > >>>> for
> > >>>>>> it?
> > >>>>>>>
> > >>>>>>> All the best
> > >>>>>>>
> > >>>>>>> Alexa
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> address@hidden> wrote:
> > >>>>>>>
> > >>>>>>>> Alexa,
> > >>>>>>>>
> > >>>>>>>> Sorry for the delayed response. Please upload the netCDF sample
> > >>>> data
> > >>>>>> here (
> > >>>>>>>> http://motherlode.ucar.edu/repository/alias/idvupload) along with
> > >>>> the
> > >>>>>>>> NCML. Let us know when it is uploaded by responding to this
> > >>>> message.
> > >>>>>>>>
> > >>>>>>>> Also, I don't know if this is the problem, but you may wish to
> > >>>>>> familiarize
> > >>>>>>>> yourself with the Time Coordinate of the NetCDF Climate and
> > >>>> Forecast
> > >>>>>> (CF)
> > >>>>>>>> Metadata Conventions (
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.0/cf-conventions.html#time-coordinate
> > >>>>>>>> ).
> > >>>>>>>>
> > >>>>>>>> Best Wishes,
> > >>>>>>>>
> > >>>>>>>> Unidata IDV Support
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> Greetings IDV support,
> > >>>>>>>>>
> > >>>>>>>>> I've just started using IDV to analyze netCDF files from 3d
> > >>>>>> atmospheric
> > >>>>>>>>> simulations and I am very impressed with the program. Thank you
> > >>>> for
> > >>>>>> all
> > >>>>>>>>> your hard work.
> > >>>>>>>>>
> > >>>>>>>>> I have multiple files that vary only by time, and would like to
> > >>>> try
> > >>>>>> to
> > >>>>>>>> load
> > >>>>>>>>> them in using the "Aggregate Grids by Time" option under data
> > >>>> source
> > >>>>>> type
> > >>>>>>>>> (goal = plot the same field at different time steps and generate
> > >>>> an
> > >>>>>>>>> animation). I've gotten the following error:
> > >>>>>>>>>
> > >>>>>>>>> -----------
> > >>>>>>>>> There was an error loading the data:
> > >>>>>>>>> Error creating data source:aggregated.netcdf.grid with:
> > >>>>>>>>> [/home/avaneaton/output/netCDF_PIC_106.nc,
> > >>>>>>>>> /home/avaneaton/output/netCDF_PIC_107.nc]
> > >>>>>>>>> Grid data source failed making data set:
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > /home/avaneaton/.unidata/idv/DefaultIdv/tmp/multigrid_b7c4c046-ee15-428c-9af3-09f0db845a590.ncml
> > >>>>>>>>> AggregationExisting: no coordinate variable for agg
> > >>>>>>>>> dimension= time
> > >>>>>>>>> ------------
> > >>>>>>>>>
> > >>>>>>>>> So it looks like the ncml script does not recognize the time
> > >>>> stamp
> > >>>>>> in my
> > >>>>>>>>> files. Any suggestions for troubleshooting this issue?
> > >>>>>>>>>
> > >>>>>>>>> All the best
> > >>>>>>>>>
> > >>>>>>>>> Alexa
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Ticket Details
> > >>>>>>>> ===================
> > >>>>>>>> Ticket ID: RLV-572542
> > >>>>>>>> Department: Support IDV
> > >>>>>>>> Priority: Normal
> > >>>>>>>> Status: Closed
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Ticket Details
> > >>>>>> ===================
> > >>>>>> Ticket ID: RLV-572542
> > >>>>>> Department: Support IDV
> > >>>>>> Priority: Normal
> > >>>>>> Status: Closed
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>>
> > >>>>
> > >>>>
> > >>>> Ticket Details
> > >>>> ===================
> > >>>> Ticket ID: RLV-572542
> > >>>> Department: Support IDV
> > >>>> Priority: Normal
> > >>>> Status: Closed
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>
> > >
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: RLV-572542
> > > Department: Support IDV
> > > Priority: Normal
> > > Status: Closed
> > >
> >
> >
> > --
> > NSF Postdoctoral Fellow, Volcanology
> > U.S. Geological Survey
> > David A. Johnston Cascades Volcano Observatory
> > 1300 SE Cardinal Court, Bldg. 10, Suite 100
> > Vancouver, WA 98683 USA
> > ph (360) 993 8921
> > mobile (208) 596 8265
> >
> >
> >
> >
> 
> 
> --
> Dr. Richard P. Signell
> 
> 


Ticket Details
===================
Ticket ID: RLV-572542
Department: Support IDV
Priority: Normal
Status: Closed