This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
hope so.note that the dateFormatMark="MOD13Q1.A#yyyyDDD" is just counting chars, then applying the java SimpleDateFormat, so the date in the filename just has to start at character 9, it doesnt have to match MOD13Q1.A
On 3/26/2013 1:38 PM, Blodgett, David wrote:
Ahhh, so the "/" in the variable name will work to represent groups. I didn't want to believe that would work, glad it will. Thanks! - Dave On Tue, Mar 26, 2013 at 2:36 PM, John Caron <address@hidden <mailto:address@hidden>> wrote: update stardate 2349873.432874 ok, enhance actually already works, but i ran into another problem with joinExisting. So I retreated to joinNew using the kludge of extracting the date from the filename. this only works if the filenames are uniform: <?xml version='1.0' encoding='UTF-8'?> <netcdf xmlns='http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2'> <aggregation dimName='time' type='joinNew'> <variableAgg name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' /> <scan location='G:/work/blodgett/' suffix='.hdf' subdirs='false' dateFormatMark="MOD13Q1.A#yyyyDDD"/> </aggregation> </netcdf> you also have to list the variables you want aggregated explicitly; i only did one, you can add the others: <variableAgg name='MODIS_Grid_16DAY_250m_500m_VI/Data_Fields/250m_16_days_NDVI' /> this form should work with the 4.3.16 release. let me know if this is adequate for you for now, and any problems. john ps: use this as reference: http://www.unidata.ucar.edu/software/netcdf/ncml/v2.2/AnnotatedSchema4.html#aggregation On 3/26/2013 1:29 PM, Blodgett, David wrote: > ... enhance ... enhance ... go back ... enhance ... > > Love it! > > Thanks John, so no easy path forward on aggregations yet, but the master > plan includes it? > > I'm happy with that. > > > On Tue, Mar 26, 2013 at 1:59 PM, John Caron <address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>>> wrote: > > Hi David: > > I had a look at this again. I changed HdfEosModisConvention to use a > 1D instead of a scalar time coordinate. > > However, joinExisting isnt working because the dataset needs to be > "enhanced" (with HdfEosModisConvention) first, before the time > coordinate is available. Currently joinExisting aggregation doesnt > do that. > > I can probably hack that in. Really I want to get the GRID feature > collection working, but thats going to be a few months. Aggregation > is old and crufty. > > Anyway, Ill let you know if i can get "enhanced" working. > > > John > > > > On 3/20/2013 7:50 AM, Blodgett, David wrote: > > Morning John, > > I pulled down the latest thredds war and have been testing a few > things > out. Did you get temporal aggregation working on the structured > data at > all? Any hints on how to structure the ncml for that? > > Looks like the spatial stuff is working. Fantastic. > > Thanks! > > - Dave > > > > On Mon, Mar 4, 2013 at 9:07 AM, Blodgett, David > <address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>> wrote: > > OK, we'll keep an eye out for the release, thanks so much John! > > No worries on the aggregation, the tiles are big enough > that I think > we can deal without aggregation. > > > On Mon, Mar 4, 2013 at 8:43 AM, John Caron > <address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>>__>> wrote: > > On 3/1/2013 12:03 PM, Blodgett, David wrote: > > John, > > Really happy you added this stuff in, what's your > plan for > releasing it? > > > going to try to release 4.3.16 in next week or two. > > > > You think there's any chance that tiled aggregation > will work? > > > im reluctant to recommend using whats there currently. > But doing > it right is a challenge and wont be done soon. > > > > > - Dave > > > On Tue, Feb 26, 2013 at 8:30 AM, address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> > <address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> > > <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>> > wrote: > > Yes the tiling scheme is consistent for all the > MODIS tiles. > > Thanks, > > Jason Werpy > Enterprise Architect > Information Dynamics > USGS/EROS > Sioux Falls, SD 57198 > phone: 605-594-2723 <tel:605-594-2723> > cell: 605-690-3576 <tel:605-690-3576> > fax: 605-594-2530 <tel:605-594-2530> > address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> > > > NOTICE: This email may contain confidential, > proprietary, > or competition > sensitive bid or proposal procurement > information. Unauthorized > disclosure of this information may carry > criminal penalties as set forth > in the Procurement Integrity Act, 41 U.S.C. 423, as > amended. Further, > the unauthorized disclosure of certain > commercial information by civil > servants may result in fines or imprisonment > under the > Trade Secrets Act > (18 U.S.C. 1905). If you have received this > information in > error, please > delete it, including all copies, and notify the > sender of > the error > immediately. > > > On Feb 26, 2013, at 8:26 AM, "Blodgett, David" > <address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>> wrote: > > In this case, the data is 16 day from about > 2000 to > present. So, on the order of 275 time > steps. There are > daily products and annual products though, > in wich case, > we are looking 12 to 4000+ time steps. > > Jason, is the tile scheme you sent over > used for all the > different resolutions? Looking at a few of > them it looks > like that is the case. > > - Dave > > > > On Tue, Feb 26, 2013 at 7:48 AM, John Caron > <address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>> > <mailto:address@hidden <mailto:address@hidden> > <mailto:address@hidden <mailto:address@hidden>>__>> > > wrote: > > Heres an image of this file. > > questions about aggregations: how many > time steps > will be in each aggregation ? how many > different > aggregations (tiles?) will you serve? > > > > On 2/24/2013 6:55 PM, David Blodgett wrote: > > Sweet, thanks John. There was data > in the tile, > but was hard to find... BORING, I > know. Sorry > about that. > > Here's a tile over the great lakes > to give you a > coast to compare locations to. > > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__BROWSE.MOD13A1.A2000065.__h11v04.005.2008238024700.1.jpg > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/BROWSE.MOD13A1.A2000065.h11v04.005.2008238024700.1.jpg> > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf> > http://e4ftl01.cr.usgs.gov/__MOLT/MOD13Q1.005/2000.03.05/__MOD13Q1.A2000065.h11v04.005.__2008238031620.hdf.xml > <http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/2000.03.05/MOD13Q1.A2000065.h11v04.005.2008238031620.hdf.xml> > > - Dave > > On Feb 24, 2013, at 6:33 PM, John > Caron wrote: > > Hi: > > Im hacking away at this, but > the datafiles i > downloaded _seem_ to have > nothing but missing > values. Or im doing something > wrong. Can you > send me a file that you know > has real data in > it. thanks > > John > > On 2/24/2013 9:22 AM, David > Blodgett wrote: > > Hi John, > > Actually, I was assuming > that the > sinusoidal projection was > in there, doh! > That was based on > gdal_translate creating > a geotiff with this > projection info: > > PROJCS["unnamed", > GEOGCS["Unknown datum > based upon the > custom spheroid", > > DATUM["Not_specified_based_on___custom_spheroid", > SPHEROID["Custom > spheroid",6371007.181,0]], > PRIMEM["Greenwich",0], > > UNIT["degree",0.__0174532925199433]], > PROJECTION["Sinusoidal"], > > PARAMETER["longitude_of___center",0], > > PARAMETER["false_easting",0], > PARAMETER["false_northing",0], > UNIT["metre",1, > AUTHORITY["EPSG","9001"]]] > > There's a georeferencing > section here > > (http://www.gdal.org/frmt___hdf4.html > <http://www.gdal.org/frmt_hdf4.html>) that > has a bit more info about > gdal's > implementation. Might be > something > helpful there? Looks like > the ODL text > you point to is where its > coming from. > > 4 and 5 are questions for > Jason and maybe > Jordan. Guys? > > - Dave > > > > > > On Feb 23, 2013, at 1:22 > PM, John Caron > wrote: > > > On 2/22/2013 2:07 PM, > David Blodgett > wrote: > > Welcome back John. > Hope you had a > good trip! > > thanks! > > I actually just > spent a bunch of > time with this the > other day and have > been meaning to > summarize what > I'm finding. > > The first issue we > are running > into is your #1 > below. In this > case there > is an external xml > file that > contains a lot of > metadata, including > information about > the projection > (sinusoidal), which > is currently > unsupported by > NetCDF-Java? > > 1) where do you get > this xml file? > > 2) im looking at > > > MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf > > MOD13Q1.A2000065.h00v08.005.__2008238080422.hdf.xml > > in your union > subdirectory. The xml > file does not seem to > have projection > info in it. > > 3) the internal ODL text > (StructMetadata) has > > ... > > UpperLeftPointMtrs=(-20015109.__354000,1111950.519667) > > LowerRightMtrs=(-18903158.__834333,-0.000000) > Projection=GCTP_SNSOID > > ProjParams=(6371007.181000,0,__0,0,0,0,0,0,0,0,0,0,0) > > SphereCode=-1 > > ... > > which may be enough to > calculate the > lat/lon values. Since > this is the > best choice, lets > investigate it > before we go down any > other paths. > > 4) Can anyone track > down the code in > the hyrax handler that > does this? Or > does the hyrax handler > actually link > in the hds-eos libraries ?? > > 5) how confident are > you that all the > files that you want to > aggregate have > the same lat/lon > geolocation ? > > > John > > > > > > > >