[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Subject: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )
Date: Sat, 09 Apr 2016 09:08:26 -0600
Oops, I see you did that already. Let me take a look real quick.

Thanks!

Sean

On Sat, Apr 9, 2016 at 9:06 AM, Sean Arms <address@hidden> wrote:

> Greeting Aksha,
>
> One thing I see is that the time format is not quite right - what you have
> is extracting year, minute, day. I'd change yyyymmdd to yyyyMMdd. However,
> there is something else going on here that isn't quite right. Would it be
> possible to get a copy of two or three of your files so that I can test
> locally? If not, the output of  `ncdump -h` for one of them would be
> helpful.
>
> Thanks!
>
> Sean
>
>
> address@hidden> wrote:
>
>> New Client Reply: Regarding NCML Aggregating  ( montly axis at fixed date
>> )
>>
>> Hi!
>>
>> Thanks for reply and for pointing what I am doing wrong, I just modified
>> ncml file, and executed, I am getting netcdf file as output but though
>> variable named time is created in netcdf but data its wrong, also timing
>> information like, months since 1960-01-01 00:00:00 is not added please go
>> through attachment.
>>
>> This is what I did
>>
>> akshay@las:~/Desktop$ pwd
>> /home/akshay/Desktop
>>
>> akshay@las:~/Desktop$ ls air/* -1
>> air/air_19600101.nc
>> air/air_19600201.nc
>> air/air_19600301.nc
>> air/air_19600401.nc
>> air/air_19600501.nc
>> air/air_19600601.nc
>> air/air_19600701.nc
>> air/air_19600801.nc
>> air/air_19600901.nc
>> air/air_19601001.nc
>> air/air_19601101.nc
>> air/air_19601201.nc
>>
>> akshay@las:~/Desktop$ cat test.ncml
>> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>> <aggregation dimName="TIME" type="joinExisting">
>>                 <variableAgg name="T"/>
>>              <scan dateFormatMark="air_#yyyymmdd"
>> location="/home/akshay/Desktop/air" suffix=".nc" subdirs="false" />
>> </aggregation>
>> </netcdf>
>>
>> akshay@las:~/Desktop$ java -Xmx1g -classpath netcdfAll-4.6.5.jar
>> ucar.nc2.dataset.NetcdfDataset -in test.ncml -out test.nc
>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> SLF4J: Defaulting to no-operation (NOP) logger implementation
>> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
>> further details.
>> NetcdfDatataset read from test.ncml write to test.nc finished=true
>>
>>
>> akshay@las:~/Desktop$ ncdump -h test.nc
>> netcdf test {
>> dimensions:
>>         LON = 120 ;
>>         LAT = 115 ;
>>         bnds = 2 ;
>>         TIME = 12 ;
>>         TIME_strlen = 20 ;
>> variables:
>>         float LON(LON) ;
>>                 LON:long_name = "Longitude" ;
>>                 LON:units = "degrees_east" ;
>>                 LON:axis = "X" ;
>>                 LON:modulo = 360.f ;
>>                 LON:point_spacing = "even" ;
>>                 LON:standard_name = "longitude" ;
>>         float LAT(LAT) ;
>>                 LAT:long_name = "Latitude" ;
>>                 LAT:units = "degrees_north" ;
>>                 LAT:axis = "Y" ;
>>                 LAT:point_spacing = "even" ;
>>                 LAT:standard_name = "latitude" ;
>>         double TIME_bnds(TIME, bnds) ;
>>         float AIR(TIME, LAT, LON) ;
>>                 AIR:missing_value = 32766.f ;
>>                 AIR:_FillValue = 32766.f ;
>>                 AIR:units = "degC" ;
>>                 AIR:long_name = "Air Temperature Monthly Mean at Surface"
>> ;
>>                 AIR:add_offset = 239.65f ;
>>                 AIR:scale_factor = 0.01f ;
>>                 AIR:Ferret_Precision_Note = "This variable written using
>> Ferret was converted from SHORT to FLOAT" ;
>>                 AIR:history = "From
>> http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/coads/1degree/global/enh/air.mean.nc";
>> ;
>>         char TIME(TIME, TIME_strlen) ;
>>                 TIME:_CoordinateAxisType = "Time" ;
>>                 TIME:long_name = "time coordinate" ;
>>                 TIME:standard_name = "time" ;
>>
>> // global attributes:
>>                 :history = "FERRET V6.93   1-Apr-16" ;
>>                 :Conventions = "CF-1.6" ;
>> }
>>
>>
>> akshay@las:~/Desktop$ ncdump  test.nc | awk '/^ TIME =/{f=1}f'
>>  TIME =
>>   "1960-01-01T00:01:00Z",
>>   "1960-01-01T00:02:00Z",
>>   "1960-01-01T00:03:00Z",
>>   "1960-01-01T00:04:00Z",
>>   "1960-01-01T00:05:00Z",
>>   "1960-01-01T00:06:00Z",
>>   "1960-01-01T00:07:00Z",
>>   "1960-01-01T00:08:00Z",
>>   "1960-01-01T00:09:00Z",
>>   "1960-01-01T00:10:00Z",
>>   "1960-01-01T00:11:00Z",
>>   "1960-01-01T00:12:00Z" ;
>> }
>>
>>
>> akshay@las:~/Desktop$ ferret
>>         NOAA/PMEL TMAP
>>         FERRET v6.96
>>         Linux 2.6.32-573.7.1.el6.x86_64 64-bit - 12/02/15
>>          9-Apr-16 14:36
>>
>> yes? use test.nc
>>  **TMAP ERR: Time axis info missing or incomplete for dataset
>>              unrecognizable time encoding
>> yes? sh d
>>      currently SET data sets:
>>     1> ./test.nc  (default)
>>  name     title                             I         J         K
>>  L         M         N
>>  TIME_BNDS
>>                                            1:2       ...       ...
>>  1:12      ...       ...
>>  AIR      Air Temperature Monthly Mean at  1:120     1:115     ...
>>  1:12      ...       ...
>>  TIME     time coordinate                  ...       ...       ...
>>  1:12      ...       ...
>>
>> yes? sh g air
>>     GRID GMH2
>>  name       axis              # pts   start                end
>>  LON       LONGITUDE          120mr   20.5E                139.5E
>>  LAT       LATITUDE           115 r   74.5S                39.5N
>>  normal    Z
>>  TIME      T                   12 r   1                    12
>>  normal    E
>>  normal    F
>> yes? list time
>>              VARIABLE : time coordinate
>>              FILENAME : test.nc
>>              SUBSET   : 12 points (T)
>>  1    /  1:"1960-01-01T00:01:00Z"
>>  2    /  2:"1960-01-01T00:02:00Z"
>>  3    /  3:"1960-01-01T00:03:00Z"
>>  4    /  4:"1960-01-01T00:04:00Z"
>>  5    /  5:"1960-01-01T00:05:00Z"
>>  6    /  6:"1960-01-01T00:06:00Z"
>>  7    /  7:"1960-01-01T00:07:00Z"
>>  8    /  8:"1960-01-01T00:08:00Z"
>>  9    /  9:"1960-01-01T00:09:00Z"
>>  10   / 10:"1960-01-01T00:10:00Z"
>>  11   / 11:"1960-01-01T00:11:00Z"
>>  12   / 12:"1960-01-01T00:12:00Z"
>> yes?
>>
>>
>> I expect output like this, above as you can see time index are numeric 1
>> to 12, and also array time contains datetime stamp but its not correct
>>
>> yes? sh g air
>>     GRID GMH2
>>  name       axis              # pts   start                end
>>  LON       LONGITUDE          120mr   20.5E                139.5E
>>  LAT       LATITUDE           115 r   74.5S                39.5N
>>  normal    Z
>>  TIME      T                   12 r   1960-01-01           1960-12-01
>>  normal    E
>>  normal    F
>>
>>
>> Thanks in advance.
>>
>> Regards,
>> Akshay
>>
>>
>>
>> ----- Original Message -----
>> From: "Unidata netCDF Java Support" <address@hidden
>> >
>> To: address@hidden
>> Cc: address@hidden
>> Sent: Friday, April 8, 2016 10:54:57 PM
>> Subject: [netCDFJava #UXM-196070]: Regarding NCML Aggregating  ( montly
>> axis at fixed date )
>>
>> Greetings Akshay,
>>
>> I think the issue is that you are extracting the date from the filename
>> (good), but are rewriting the information using the NCML ouside of the
>> aggregation (i.e. <values start="0" increment="1" />). Can you try using
>> the same NcML you have, but remove this part of the NcML:
>>
>> <variable name="TIME" type="double" shape="TIME">
>>         <attribute name="units" value="months since 1960-01-01 00:00:00"
>> />
>>         <values start="0" increment="1" />
>>          <attribute name="axis" value="T" />
>> </variable>
>>
>> Thanks!
>>
>> Sean
>>
>>
>> address@hidden> wrote:
>>
>> > New Ticket: Regarding NCML Aggregating  ( montly axis at fixed date )
>> >
>> > Hi!
>> >
>> > This query is regarding time variable declaration in NCML, here my aim
>> > is to define time axis at 1th of every month that is same as that of
>> > date stamp in filename.
>> >
>> >
>> > My Input :
>> >
>> > I have list of files like below
>> >
>> > # ls -1 | head -10
>> > air_19600101.nc
>> > air_19600201.nc
>> > air_19600301.nc
>> > air_19600401.nc
>> > air_19600501.nc
>> > air_19600601.nc
>> > air_19600701.nc
>> > air_19600801.nc
>> > air_19600901.nc
>> > air_19601001.nc
>> >
>> >
>> > This is what I tried :
>> >
>> > Following is my ncml
>> >
>> > <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";>
>> > <variable name="TIME" type="double" shape="TIME">
>> >         <attribute name="units" value="months since 1960-01-01
>> 00:00:00" />
>> >         <values start="0" increment="1" />
>> >          <attribute name="axis" value="T" />
>> > </variable>
>> > <aggregation dimName="TIME" type="joinExisting">
>> >              <scan dateFormatMark="air_#yyyymmdd"
>> location="/path/to/AIR"
>> > suffix=".nc" subdirs="false" />
>> > </aggregation>
>> > </netcdf>
>> >
>> >
>> > This is what I received after restarting thredds data server:
>> >
>> > I use ferret for data analysis, this is how time indexes are created
>> >
>> > yes? set data "http://127.0.0.1/thredds/dodsC/test/test.nc";
>> > yes? sh g air
>> >     GRID GFP1
>> >  name       axis              # pts   start                end
>> >  LON       LONGITUDE          120mr   20.5E                139.5E
>> >  LAT       LATITUDE           115 r   74.5S                39.5N
>> >  normal    Z
>> >  TIME      TIME               674 r   01-JAN-1960 12:00    31-JAN-2016
>> > 12:24
>> >  normal    E
>> >  normal    F
>> > yes? list/l=1:10 t[gt=air]
>> >              VARIABLE : T
>> >                         axis TIME
>> >              FILENAME : test.nc
>> >              FILEPATH : http://127.0.0.1/thredds/dodsC/test/
>> >              SUBSET   : 10 points (TIME)
>> >  01-JAN-1960 /  1:  0.000
>> >  31-JAN-1960 /  2:  1.000
>> >  02-MAR-1960 /  3:  2.000
>> >  01-APR-1960 /  4:  3.000
>> >  02-MAY-1960 /  5:  4.000
>> >  01-JUN-1960 /  6:  5.000
>> >  02-JUL-1960 /  7:  6.000
>> >  01-AUG-1960 /  8:  7.000
>> >  31-AUG-1960 /  9:  8.000
>> >  01-OCT-1960 / 10:  9.000
>> >
>> >
>> > This is expected output :
>> >
>> > Here I expect time index to be created like below, at first of every
>> > month, or else 15th of every month
>> >
>> >  01-JAN-1960 /  1:  0.000
>> >  01-FEB-1960 /  2:  1.000
>> >  01-MAR-1960 /  3:  2.000
>> >  01-APR-1960 /  4:  3.000
>> >  01-MAY-1960 /  5:  4.000
>> >  01-JUN-1960 /  6:  5.000
>> >  01-JUL-1960 /  7:  6.000
>> >  01-AUG-1960 /  8:  7.000
>> >  01-SEP-1960 /  9:  8.000
>> >  01-OCT-1960 / 10:  9.000
>> >
>> >
>> > Hope someone can guide me
>> >
>> > Thanks in advance.
>> >
>> > Have a nice day !
>> >
>> > Regards,
>> > Akshay Hegde
>> >
>> >
>> > Ticket Details
>> > ===================
>> > Ticket ID: UXM-196070
>> > Department: Support netCDF Java
>> > Priority: Normal
>> > Status: Open
>> > Link:
>> >
>> https://andy.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=26875
>> >
>> >
>>
>>
>>
>> Ticket Details
>> ===================
>> Ticket ID: UXM-196070
>> Department: Support netCDF Java
>> Priority: Normal
>> Status: Open
>>
>>
>>
>>
>> Ticket Details
>> ===================
>> Ticket ID: UXM-196070
>> Department: Support netCDF Java
>> Priority: Normal
>> Status: Open
>> Link:
>> https://andy.unidata.ucar.edu/esupport/staff/index.php?_m=tickets&_a=viewticket&ticketid=26875
>
>
>



Ticket Details
===================
Ticket ID: UXM-196070
Department: Support netCDF Java
Priority: Normal
Status: Open
Prev by Date: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )
Next by Date: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )
Previous by thread: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )
Next by thread: [netCDFJava #UXM-196070]: Regarding NCML Aggregating ( montly axis at fixed date )
Index(es):
- Date
- Thread