Greetings Akshay,
So I think I have the ncml fixed to do the right thing. Since each nc file
had time already defined and this was a joinExsiting aggregation, there was
actually no need to do anything with time, like extracting the date/time
from the file name. Please try the attached ncml and see if that works for
you.
Cheers,
Sean
On Sat, Apr 9, 2016 at 9:08 AM, Sean Arms <address@hidden> wrote:
> Oops, I see you did that already. Let me take a look real quick.
>
> Thanks!
>
> Sean
>
> On Sat, Apr 9, 2016 at 9:06 AM, Sean Arms <address@hidden> wrote:
>
>> Greeting Aksha,
>>
>> One thing I see is that the time format is not quite right - what you
>> have is extracting year, minute, day. I'd change yyyymmdd to yyyyMMdd.
>> However, there is something else going on here that isn't quite right.
>> Would it be possible to get a copy of two or three of your files so that I
>> can test locally? If not, the output of `ncdump -h` for one of them would
>> be helpful.
>>
>> Thanks!
>>
>> Sean
>>
>>
>> address@hidden> wrote:
>>
>>> New Client Reply: Regarding NCML Aggregating ( montly axis at fixed
>>> date )
>>>
>>> Hi!
>>>
>>> Thanks for reply and for pointing what I am doing wrong, I just modified
>>> ncml file, and executed, I am getting netcdf file as output but though
>>> variable named time is created in netcdf but data its wrong, also timing
>>> information like, months since 1960-01-01 00:00:00 is not added please go
>>> through attachment.
>>>
>>> This is what I did
>>>
>>> akshay@las:~/Desktop$ pwd
>>> /home/akshay/Desktop
>>>
>>> akshay@las:~/Desktop$ ls air/* -1
>>> air/air_19600101.nc
>>> air/air_19600201.nc
>>> air/air_19600301.nc
>>> air/air_19600401.nc
>>> air/air_19600501.nc
>>> air/air_19600601.nc
>>> air/air_19600701.nc
>>> air/air_19600801.nc
>>> air/air_19600901.nc
>>> air/air_19601001.nc
>>> air/air_19601101.nc
>>> air/air_19601201.nc
>>>
>>> akshay@las:~/Desktop$ cat test.ncml
>>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>>> <aggregation dimName="TIME" type="joinExisting">
>>> <variableAgg name="T"/>
>>> <scan dateFormatMark="air_#yyyymmdd"
>>> location="/home/akshay/Desktop/air" suffix=".nc" subdirs="false" />
>>> </aggregation>
>>> </netcdf>
>>>
>>> akshay@las:~/Desktop$ java -Xmx1g -classpath netcdfAll-4.6.5.jar
>>> ucar.nc2.dataset.NetcdfDataset -in test.ncml -out test.nc
>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
>>> further details.
>>> NetcdfDatataset read from test.ncml write to test.nc finished=true
>>>
>>>
>>> akshay@las:~/Desktop$ ncdump -h test.nc
>>> netcdf test {
>>> dimensions:
>>> LON = 120 ;
>>> LAT = 115 ;
>>> bnds = 2 ;
>>> TIME = 12 ;
>>> TIME_strlen = 20 ;
>>> variables:
>>> float LON(LON) ;
>>> LON:long_name = "Longitude" ;
>>> LON:units = "degrees_east" ;
>>> LON:axis = "X" ;
>>> LON:modulo = 360.f ;
>>> LON:point_spacing = "even" ;
>>> LON:standard_name = "longitude" ;
>>> float LAT(LAT) ;
>>> LAT:long_name = "Latitude" ;
>>> LAT:units = "degrees_north" ;
>>> LAT:axis = "Y" ;
>>> LAT:point_spacing = "even" ;
>>> LAT:standard_name = "latitude" ;
>>> double TIME_bnds(TIME, bnds) ;
>>> float AIR(TIME, LAT, LON) ;
>>> AIR:missing_value = 32766.f ;
>>> AIR:_FillValue = 32766.f ;
>>> AIR:units = "degC" ;
>>> AIR:long_name = "Air Temperature Monthly Mean at
>>> Surface" ;
>>> AIR:add_offset = 239.65f ;
>>> AIR:scale_factor = 0.01f ;
>>> AIR:Ferret_Precision_Note = "This variable written using
>>> Ferret was converted from SHORT to FLOAT" ;
>>> AIR:history = "From
>>> http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/coads/1degree/global/enh/air.mean.nc";
>>> ;
>>> char TIME(TIME, TIME_strlen) ;
>>> TIME:_CoordinateAxisType = "Time" ;
>>> TIME:long_name = "time coordinate" ;
>>> TIME:standard_name = "time" ;
>>>
>>> // global attributes:
>>> :history = "FERRET V6.93 1-Apr-16" ;
>>> :Conventions = "CF-1.6" ;
>>> }
>>>
>>>
>>> akshay@las:~/Desktop$ ncdump test.nc | awk '/^ TIME =/{f=1}f'
>>> TIME =
>>> "1960-01-01T00:01:00Z",
>>> "1960-01-01T00:02:00Z",
>>> "1960-01-01T00:03:00Z",
>>> "1960-01-01T00:04:00Z",
>>> "1960-01-01T00:05:00Z",
>>> "1960-01-01T00:06:00Z",
>>> "1960-01-01T00:07:00Z",
>>> "1960-01-01T00:08:00Z",
>>> "1960-01-01T00:09:00Z",
>>> "1960-01-01T00:10:00Z",
>>> "1960-01-01T00:11:00Z",
>>> "1960-01-01T00:12:00Z" ;
>>> }
>>>
>>>
>>> akshay@las:~/Desktop$ ferret
>>> NOAA/PMEL TMAP
>>> FERRET v6.96
>>> Linux 2.6.32-573.7.1.el6.x86_64 64-bit - 12/02/15
>>> 9-Apr-16 14:36
>>>
>>> yes? use test.nc
>>> **TMAP ERR: Time axis info missing or incomplete for dataset
>>> unrecognizable time encoding
>>> yes? sh d
>>> currently SET data sets:
>>> 1> ./test.nc (default)
>>> name title I J K
>>> L M N
>>> TIME_BNDS
>>> 1:2 ... ...
>>> 1:12 ... ...
>>> AIR Air Temperature Monthly Mean at 1:120 1:115 ...
>>> 1:12 ... ...
>>> TIME time coordinate ... ... ...
>>> 1:12 ... ...
>>>
>>> yes? sh g air
>>> GRID GMH2
>>> name axis # pts start end
>>> LON LONGITUDE 120mr 20.5E 139.5E
>>> LAT LATITUDE 115 r 74.5S 39.5N
>>> normal Z
>>> TIME T 12 r 1 12
>>> normal E
>>> normal F
>>> yes? list time
>>> VARIABLE : time coordinate
>>> FILENAME : test.nc
>>> SUBSET : 12 points (T)
>>> 1 / 1:"1960-01-01T00:01:00Z"
>>> 2 / 2:"1960-01-01T00:02:00Z"
>>> 3 / 3:"1960-01-01T00:03:00Z"
>>> 4 / 4:"1960-01-01T00:04:00Z"
>>> 5 / 5:"1960-01-01T00:05:00Z"
>>> 6 / 6:"1960-01-01T00:06:00Z"
>>> 7 / 7:"1960-01-01T00:07:00Z"
>>> 8 / 8:"1960-01-01T00:08:00Z"
>>> 9 / 9:"1960-01-01T00:09:00Z"
>>> 10 / 10:"1960-01-01T00:10:00Z"
>>> 11 / 11:"1960-01-01T00:11:00Z"
>>> 12 / 12:"1960-01-01T00:12:00Z"
>>> yes?
>>>
>>>
>>> I expect output like this, above as you can see time index are numeric 1
>>> to 12, and also array time contains datetime stamp but its not correct
>>>
>>> yes? sh g air
>>> GRID GMH2
>>> name axis # pts start end
>>> LON LONGITUDE 120mr 20.5E 139.5E
>>> LAT LATITUDE 115 r 74.5S 39.5N
>>> normal Z
>>> TIME T 12 r 1960-01-01 1960-12-01
>>> normal E
>>> normal F
>>>
>>>
>>> Thanks in advance.
>>>
>>> Regards,
>>> Akshay
>>>
>>>
>>>
>>> ----- Original Message -----
>>> address@hidden>
>>> To: address@hidden
>>> Cc: address@hidden
>>> Sent: Friday, April 8, 2016 10:54:57 PM
>>> Subject: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly
>>> axis at fixed date )
>>>
>>> Greetings Akshay,
>>>
>>> I think the issue is that you are extracting the date from the filename
>>> (good), but are rewriting the information using the NCML ouside of the
>>> aggregation (i.e. <values start="0" increment="1" />). Can you try using
>>> the same NcML you have, but remove this part of the NcML:
>>>
>>> <variable name="TIME" type="double" shape="TIME">
>>> <attribute name="units" value="months since 1960-01-01 00:00:00"
>>> />
>>> <values start="0" increment="1" />
>>> <attribute name="axis" value="T" />
>>> </variable>
>>>
>>> Thanks!
>>>
>>> Sean
>>>
>>>
>>> address@hidden> wrote:
>>>
>>> > New Ticket: Regarding NCML Aggregating ( montly axis at fixed date )
>>> >
>>> > Hi!
>>> >
>>> > This query is regarding time variable declaration in NCML, here my aim
>>> > is to define time axis at 1th of every month that is same as that of
>>> > date stamp in filename.
>>> >
>>> >
>>> > My Input :
>>> >
>>> > I have list of files like below
>>> >
>>> > # ls -1 | head -10
>>> > air_19600101.nc
>>> > air_19600201.nc
>>> > air_19600301.nc
>>> > air_19600401.nc
>>> > air_19600501.nc
>>> > air_19600601.nc
>>> > air_19600701.nc
>>> > air_19600801.nc
>>> > air_19600901.nc
>>> > air_19601001.nc
>>> >
>>> >
>>> > This is what I tried :
>>> >
>>> > Following is my ncml
>>> >
>>> > <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2
>>> ">
>>> > <variable name="TIME" type="double" shape="TIME">
>>> > <attribute name="units" value="months since 1960-01-01
>>> 00:00:00" />
>>> > <values start="0" increment="1" />
>>> > <attribute name="axis" value="T" />
>>> > </variable>
>>> > <aggregation dimName="TIME" type="joinExisting">
>>> > <scan dateFormatMark="air_#yyyymmdd"
>>> location="/path/to/AIR"
>>> > suffix=".nc" subdirs="false" />
>>> > </aggregation>
>>> > </netcdf>
>>> >
>>> >
>>> > This is what I received after restarting thredds data server:
>>> >
>>> > I use ferret for data analysis, this is how time indexes are created
>>> >
>>> > yes? set data "http://127.0.0.1/thredds/dodsC/test/test.nc";
>>> > yes? sh g air
>>> > GRID GFP1
>>> > name axis # pts start end
>>> > LON LONGITUDE 120mr 20.5E 139.5E
>>> > LAT LATITUDE 115 r 74.5S 39.5N
>>> > normal Z
>>> > TIME TIME 674 r 01-JAN-1960 12:00 31-JAN-2016
>>> > 12:24
>>> > normal E
>>> > normal F
>>> > yes? list/l=1:10 t[gt=air]
>>> > VARIABLE : T
>>> > axis TIME
>>> > FILENAME : test.nc
>>> > FILEPATH : http://127.0.0.1/thredds/dodsC/test/
>>> > SUBSET : 10 points (TIME)
>>> > 01-JAN-1960 / 1: 0.000
>>> > 31-JAN-1960 / 2: 1.000
>>> > 02-MAR-1960 / 3: 2.000
>>> > 01-APR-1960 / 4: 3.000
>>> > 02-MAY-1960 / 5: 4.000
>>> > 01-JUN-1960 / 6: 5.000
>>> > 02-JUL-1960 / 7: 6.000
>>> > 01-AUG-1960 / 8: 7.000
>>> > 31-AUG-1960 / 9: 8.000
>>> > 01-OCT-1960 / 10: 9.000
>>> >
>>> >
>>> > This is expected output :
>>> >
>>> > Here I expect time index to be created like below, at first of every
>>> > month, or else 15th of every month
>>> >
>>> > 01-JAN-1960 / 1: 0.000
>>> > 01-FEB-1960 / 2: 1.000
>>> > 01-MAR-1960 / 3: 2.000
>>> > 01-APR-1960 / 4: 3.000
>>> > 01-MAY-1960 / 5: 4.000
>>> > 01-JUN-1960 / 6: 5.000
>>> > 01-JUL-1960 / 7: 6.000
>>> > 01-AUG-1960 / 8: 7.000
>>> > 01-SEP-1960 / 9: 8.000
>>> > 01-OCT-1960 / 10: 9.000
>>> >
>>> >
>>> > Hope someone can guide me
>>> >
>>> > Thanks in advance.
>>> >
>>> > Have a nice day !
>>> >
>>> > Regards,
>>> > Akshay Hegde
>>> >
>>> >
>>> > Ticket Details
>>> > ===================
>>> > Ticket ID: UXM-196070
>>> > Department: Support netCDF Java
>>> > Priority: Normal
>>> > Status: Open
>>> > Link:
>>> >
>>> https://andy.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=26875
>>> >
>>> >
>>>
>>>
>>>
>>> Ticket Details
>>> ===================
>>> Ticket ID: UXM-196070
>>> Department: Support netCDF Java
>>> Priority: Normal
>>> Status: Open
>>>
>>>
>>>
>>>
>>> Ticket Details
>>> ===================
>>> Ticket ID: UXM-196070
>>> Department: Support netCDF Java
>>> Priority: Normal
>>> Status: Open
>>> Link:
>>> https://andy.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=26875
>>
>>
>>
>
Ticket Details
===================
Ticket ID: UXM-196070
Department: Support netCDF Java
Priority: Normal
Status: OpenAttachment:
akshay.ncml
Description: Binary data