This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Alexa, Great! I am glad it worked. I was contemplating writing a blog entry for our Unidata Developer Blog on this experience as it makes a good case study. Would it be OK if I used your data/ncml for this purpose? It is no problem if you would prefer not to. Best, Unidata IDV Support > Dear IDV Support (and Sean), > > This works perfectly. I realized the file naming scheme was actually actually > giving year + day of the year, so I adjusted the dateFormatMark using the > syntax in this example > http://docs.opendap.org/index.php/Dynamic_Aggregation_Tutorial and now the > dates/times are being read accurately. > > As a side note, I noticed that Projections—>Enable Adaptive Resolution is on > by default in the IDV, which automatically subsamples the data. But once that > function was unchecked and the data reloaded, everything looks good. Big > thanks to the both of you. > > All the best > > Alexa > > > > On Apr 1, 2016, at 12:06 PM, Unidata IDV Support <address@hidden> wrote: > > > > Hi Alexa, > > > > What I was describing below had nothing to do with the problem. Please see > > the > > attached NCML. This file should correctly aggregate the variables over > > the time dimension, and also it will extract the times from the file name, > > so > > you don't have to manually describe them. > > > > Let us know how it goes. > > > > Best, > > > > Unidata IDV Support > > > > PS Thanks to my colleague Sean Arms for the NCML lessons :-) > > > >> Sure thing, I’ll have a play and keep you posted. All the best, Alexa > >> > >> > >>> On Mar 31, 2016, at 4:21 PM, Unidata IDV Support <address@hidden> wrote: > >>> > >>> Alexa, > >>> > >>> I think I know what the problem is, but don't have time to try before I > >>> go home. > >>> > >>> I think the problem is that we have only changed one variable when we > >>> should > >>> have changed them all, at least all those that are listed under the Field > >>> Selector. The IDV may be getting confused on this. > >>> > >>> Do you want to give that a shot and let me know what happens? > >>> > >>> Best, > >>> > >>> Unidata IDV Support > >>> > >>>> Alexa, > >>>> > >>>> Yes, thanks for the clarification. (I think the ordering of our messages > >>>> got > >>>> jumbled there.) > >>>> > >>>> I see what you are saying, and when I load each individual file > >>>> separately I can > >>>> see the data are definitely not the same so something is amiss. We'll > >>>> resume > >>>> tomorrow. > >>>> > >>>> Best, > >>>> > >>>> Unidata IDV Support > >>>> > >>>>> Hi there — thank you for for your reply. Using your syntax below, I > >>>>> tried aggregating three files (edited script attached) — different > >>>>> files this time. Again, the files are read into IDV with the correct > >>>>> time stamps, and I can toggle between the times, but the strange thing > >>>>> is the plot does not change with time. It’s like it’s only plotting one > >>>>> time. -Alexa > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>> On Mar 31, 2016, at 3:46 PM, Unidata IDV Support <address@hidden> > >>>>>> wrote: > >>>>>> > >>>>>> Alexa, > >>>>>> > >>>>>> Great! Glad we are making progress and also that you are hanging in > >>>>>> there. This > >>>>>> stuff is not so easy. > >>>>>> > >>>>>> Replace the current values element with something like this: > >>>>>> > >>>>>> <values>0 3 11</values> > >>>>>> > >>>>>> But if that is too onerous because you have many file, we will have to > >>>>>> go via > >>>>>> the time extraction route. Let me know... > >>>>>> > >>>>>> Best, > >>>>>> > >>>>>> Unidata IDV Support > >>>>>> > >>>>>>> Hello — this looks great, and I appreciate your help with this. Your > >>>>>>> ncml script worked for the two test files and the times look right. > >>>>>>> However, since we have some missing satellite images, not all the > >>>>>>> files are spaced exactly 10 mins apart. Is there a way to specify > >>>>>>> time gaps between individual files begin aggregated? Otherwise, > >>>>>>> perhaps extracting the time from the file name would be good route. > >>>>>>> -Alexa > >>>>>>> > >>>>>>> > >>>>>>>> On Mar 31, 2016, at 3:29 PM, Unidata IDV Support <address@hidden> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi again, > >>>>>>>> > >>>>>>>> The embedded NcML is a bit hard to read so attached, you will find > >>>>>>>> that same NcML file. > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> > >>>>>>>> Unidata IDV Support > >>>>>>>> > >>>>>>>>> Hi Alexa, > >>>>>>>>> > >>>>>>>>> It is a little tricky but you can fix these problems with a bit of > >>>>>>>>> NcML. To get us going, let's look at the following NcML which can be > >>>>>>>>> read into the IDV and will allow you to view the > >>>>>>>>> `channel_14_brightness_temperature' over time. This file should be > >>>>>>>>> located next to your `hdf' files. Please read the comments in the > >>>>>>>>> NcML > >>>>>>>>> for more information about what is going on here. > >>>>>>>>> > >>>>>>>>> ,---- > >>>>>>>>> | <?xml version="1.0" encoding="UTF-8"?> > >>>>>>>>> | <netcdf > >>>>>>>>> xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> > >>>>>>>>> | <!-- Create a time coordinate variable that will be used for > >>>>>>>>> the aggregation. --> > >>>>>>>>> | <variable name="time" type="int" shape="time" > > >>>>>>>>> | <!-- Let's add some units. Alexa: please make sure these > >>>>>>>>> units are correct. --> > >>>>>>>>> | <attribute name="units" value="minutes since 2014-04-04 > >>>>>>>>> 19:09"/> > >>>>>>>>> | <!-- Specify this variable as a time coordinate --> > >>>>>>>>> | <attribute name="_CoordinateAxisType" value="Time" /> > >>>>>>>>> | <!-- Increment each file listed below by 10 minutes. Alexa, > >>>>>>>>> please make sure this --> > >>>>>>>>> | <!-- assumption about the times is correct. --> > >>>>>>>>> | <values start="0" increment="10" /> > >>>>>>>>> | </variable> > >>>>>>>>> | <!-- Specify the coordinates of time lat lon for this one > >>>>>>>>> variable. Please use --> > >>>>>>>>> | <!-- this as an example for other variables you are interersted > >>>>>>>>> in. In other --> > >>>>>>>>> | <!-- words, you can copy and paste this variable NcML element > >>>>>>>>> for your other --> > >>>>>>>>> | <!-- variables. --> > >>>>>>>>> | <variable name="channel_14_brightness_temperature" shape="lines > >>>>>>>>> elements" type="short"> > >>>>>>>>> | <attribute name="coordinates" value="time pixel_latitude > >>>>>>>>> pixel_longitude" /> > >>>>>>>>> | </variable> > >>>>>>>>> | <!-- Now you are ready to do your aggregation, which is the > >>>>>>>>> easy part at this point --> > >>>>>>>>> | <aggregation dimName="time" type="joinNew"> > >>>>>>>>> | <variableAgg name="T"/> > >>>>>>>>> | <netcdf location="geocatL1.MTSAT-1R.2014044.190900.hdf"/> > >>>>>>>>> | <netcdf location="geocatL1.MTSAT-1R.2014044.191900.hdf"/> > >>>>>>>>> | </aggregation> > >>>>>>>>> | </netcdf> > >>>>>>>>> `---- > >>>>>>>>> > >>>>>>>>> In summary, this NcML should aggregate over time the two of HDF > >>>>>>>>> files > >>>>>>>>> you provided. You will see only one variable that is aggregated over > >>>>>>>>> time; `channel_14_brightness_temperature'. I'll leave it up to you > >>>>>>>>> to > >>>>>>>>> do the others though do not hesitate to ask for help. > >>>>>>>>> > >>>>>>>>> Also, if interested, there are also fancier tricks we can do here > >>>>>>>>> such > >>>>>>>>> as NcML can be smart enough to extract the times from the file > >>>>>>>>> names. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> > >>>>>>>>> Unidata IDV Support > >>>>>>>>> > >>>>>>>>>> Ok sounds promising. Here is the link to another file, which > >>>>>>>>>> should differ only in time — > >>>>>>>>>> https://drive.google.com/file/d/0B1jVlP9ZfAZCRDVyczU3N2NMbFU/view?usp=sharing > >>>>>>>>>> -Alexa > >>>>>>>>>> > >>>>>>>>>>> On Mar 30, 2016, at 4:49 PM, Unidata IDV Support <address@hidden> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Alexa, > >>>>>>>>>>> > >>>>>>>>>>> Indeed we may be able to patch things up with a bit of ncml. I > >>>>>>>>>>> only found one > >>>>>>>>>>> file at the drive.google location. Would it be possible to see > >>>>>>>>>>> more so that we > >>>>>>>>>>> have something to aggregate? > >>>>>>>>>>> > >>>>>>>>>>> Best, > >>>>>>>>>>> > >>>>>>>>>>> Unidata IDV Support > >>>>>>>>>>> > >>>>>>>>>>>> Hello IDV Support, > >>>>>>>>>>>> > >>>>>>>>>>>> I am working on reading in a number of HDF files using > >>>>>>>>>>>> ‘aggregate grids by time’ or similar. I am able to read the > >>>>>>>>>>>> files in individually without a problem, but it looks like the > >>>>>>>>>>>> time variable is not recognized. The error message is "Grid data > >>>>>>>>>>>> source failed making data set….Dimension time does not exist”. > >>>>>>>>>>>> Any ideas for how to specify the time variable? Could this be > >>>>>>>>>>>> achieved with an .ncml script? > >>>>>>>>>>>> > >>>>>>>>>>>> Example file is located here: > >>>>>>>>>>>> https://drive.google.com/file/d/0B1jVlP9ZfAZCRFhkRVM2aVVRVGc/view?usp=sharing > >>>>>>>>>>>> > >>>>>>>>>>>> All the best > >>>>>>>>>>>> > >>>>>>>>>>>> Alexa > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Ticket Details > >>>>>>>>>>> =================== > >>>>>>>>>>> Ticket ID: LPW-471935 > >>>>>>>>>>> Department: Support IDV > >>>>>>>>>>> Priority: Normal > >>>>>>>>>>> Status: Closed > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Ticket Details > >>>>>>>> =================== > >>>>>>>> Ticket ID: LPW-471935 > >>>>>>>> Department: Support IDV > >>>>>>>> Priority: Normal > >>>>>>>> Status: Closed<alexa.ncml> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> Ticket Details > >>>>>> =================== > >>>>>> Ticket ID: LPW-471935 > >>>>>> Department: Support IDV > >>>>>> Priority: Normal > >>>>>> Status: Closed > >>>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>> > >>> > >>> Ticket Details > >>> =================== > >>> Ticket ID: LPW-471935 > >>> Department: Support IDV > >>> Priority: Normal > >>> Status: Open > >>> > >> > >> > > > > > > Ticket Details > > =================== > > Ticket ID: LPW-471935 > > Department: Support IDV > > Priority: Normal > > Status: Closed<alexa.ncml> > > Ticket Details =================== Ticket ID: LPW-471935 Department: Support IDV Priority: Normal Status: Closed