[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDV #RZP-160177]: unable to read netcdf file.
- Subject: [IDV #RZP-160177]: unable to read netcdf file.
- Date: Fri, 11 Dec 2015 09:53:52 -0700
Hi Mark,
Here, you will find a [test2.nc] file that can be read into the
IDV. (All reference links can be found at the bottom.) Also, here is the
[DAP] link that the IDV can read.
When I need guidance from CF in the form of examples, I usually turn to
the [NODC] website, and specifically, in this case, the section on
[grids]. Using that resource as a guide, we can use a bit of [Python] to
create a data file according to these standards. I know you may be more
familiar with IDL, but Python is fairly straightforward to understand.
First create a netCDF file to store your data.
,----
| import netCDF4
| import numpy as np
|
| try:
| ncfile.close()
| except:
| pass
| ncfile = netCDF4.Dataset('/home/python/work/test2.nc',
| mode='w',
| format='NETCDF4_CLASSIC')
| ncfile
`----
Set some global attributes pertaining to standards and whatnot.
,----
| ncfile.Conventions = 'CF-1.6'
| ncfile.Metadata_Conventions = 'Unidata Dataset Discovery v1.0'
| ncfile.cdm_data_type = 'Grid'
| ncfile.featureType = "grid"
| ncfile.nodc_template_version = 'NODC_NetCDF_Grid_Template_v1.1'
`----
Now create the `lat', `lon', `z', `time' dimensions.
,----
| lat_dim = ncfile.createDimension('lat', ds['latitude'].size)
| lon_dim = ncfile.createDimension('lon', ds['longitude'].size)
| z_dim = ncfile.createDimension('z', ds['pressure'].size)
| time_dim = ncfile.createDimension('time', None)
`----
Create `lat' variable.
,----
| lat = ncfile.createVariable('lat', np.float32, ('lat',))
| lat.units = 'degrees_north'
| lat.long_name = 'latitude'
`----
Create `lon' variable.
,----
| lon = ncfile.createVariable('lon', np.float32, ('lon',))
| lon.units = 'degrees_east'
| lon.long_name = 'longitude'
`----
Create `z' variable
,----
| z = ncfile.createVariable('z', np.float32, ('z',))
| z.units = 'hPa'
| z.long_name = 'pressure'
`----
Create `time' variable.
,----
| time = ncfile.createVariable('time', np.float64, ('time',))
| time.units = 'hours since 2000-01-01'
| time.long_name = 'time'
`----
Create `temp' variable.
,----
| temp = ncfile.createVariable('temp',np.float64,('time','z', 'lat','lon'))
| temp.units = 'K' # degrees Kelvin
| temp.standard_name = 'air_temperature' # this is a CF standard name
`----
Now set your data. The `ds' variable contains your original data. The
code for this is not shown.
,----
| lat[:] = ds['latitude'].data
| lon[:] = ds['longitude'].data
| z[:] = ds['pressure'].data
| time[:] = np.array([0])
| temp[0,:] = ds['temperature'].data
`----
You are done. Now close the file.
,----
| ncfile.close();
`----
Now let's look at the result of what we have done with `ncdump'.
,----
| ncdump -h ../../../test2.nc
`----
,----
| netcdf test2 {
| dimensions:
| lat = 181 ;
| lon = 361 ;
| z = 100 ;
| time = UNLIMITED ; // (1 currently)
| variables:
| float lat(lat) ;
| lat:units = "degrees_north" ;
| lat:long_name = "latitude" ;
| float lon(lon) ;
| lon:units = "degrees_east" ;
| lon:long_name = "longitude" ;
| float z(z) ;
| z:units = "hPa" ;
| z:long_name = "pressure" ;
| double time(time) ;
| time:units = "hours since 2000-01-01" ;
| time:long_name = "time" ;
| double temp(time, z, lat, lon) ;
| temp:units = "K" ;
| temp:standard_name = "air_temperature" ;
|
| // global attributes:
| :Conventions = "CF-1.6" ;
| :Metadata_Conventions = "Unidata Dataset Discovery v1.0" ;
| :cdm_data_type = "Grid" ;
| :featureType = "grid" ;
| :nodc_template_version = "NODC_NetCDF_Grid_Template_v1.1" ;
| }
`----
Compare the results of this `ncdump' with the file you originally
uploaded to us. Do you see the differences?
,----
| ncdump -h ../../../test.nc
`----
,----
| netcdf test {
| dimensions:
| longitude = 361 ;
| latitude = 181 ;
| pressure = 100 ;
| variables:
| float longitude(longitude) ;
| longitude:longitude = "degrees" ;
| float latitude(latitude) ;
| latitude:latitude = "degrees" ;
| float pressure(pressure) ;
| pressure:pressure = "hPa" ;
| float temperature(pressure, latitude, longitude) ;
| temperature:temperature = "K" ;
| temperature:Projection = "cylindrical" ;
| float probability(pressure, latitude, longitude) ;
| probability:probability = "1" ;
|
| // global attributes:
| :Title = "test" ;
| }
`----
At this point, you should have enough information to tidy up your netCDF
file with IDL. Let us know if you have any additional questions.
Best,
Unidata IDV Support
[test2.nc]
https://motherlode.ucar.edu/repository/entry/show/RAMADDA/Unidata/Staff/Julien+Chastang/mark/test2.nc?entryid=3cb9100e-6a62-4c5c-af37-8cd9966fcce6
[DAP]
https://motherlode.ucar.edu/repository/opendap/3cb9100e-6a62-4c5c-af37-8cd9966fcce6/entry.das
[NODC] https://www.nodc.noaa.gov/data/formats/netcdf/v1.1/
[grids] https://www.nodc.noaa.gov/data/formats/netcdf/v1.1/grid.cdl
[Python] https://github.com/Unidata/unidata-python-workshop
> The data is from the AIRS instrument on AQUA. The data is basically
> retrievals.
> There are 100 pressure levels and I am averaging 10 years of data for each
> day of the
> year. I sent you the data for one day. There are two 3d fields:
> probability and air temperature.
> Now the probability field is the probability
> that the air will decrease below some temperature. This is a data product
> that
> airlines would like to have if they want to avoid flying in those regions. The
> goal is to visualize some of these regions as “bubbles” of cold air in the
> upper troposphere. The other field is just the averaged temperature.
>
>
> I am creating the files using IDL netcdf commands
> [first var is probability]
> [second var is air temperature]
>
> s=size(first_var)
> dim1=ncdf_dimdef(id,'longitude',s(1))
> dim2=ncdf_dimdef(id,'latitude',s(2))
> dim3=ncdf_dimdef(id,'pressure',s(3))
>
> varlon=ncdf_vardef(id,'longitude',dim1,/float)
> varlat=ncdf_vardef(id,'latitude',dim2,/float)
> varpres=ncdf_vardef(id,'pressure',dim3,/float)
>
>
> ncdf_attput,id,varlat,'latitude','degrees',/char
> ncdf_attput,id,varlon,'longitude','degrees',/char
> ncdf_attput,id,varpres,'pressure','hPa',/char
>
> var1_data=ncdf_vardef(id,v1name,[dim1,dim2,dim3],/float)
> var2_data=ncdf_vardef(id,v2name,[dim1,dim2,dim3],/float)
>
> ; create the attributes
>
> ncdf_attput,id,var1_data,v1name,'K',/char
> ncdf_attput,id,var2_data,v2name,'1',/char
> ncdf_attput,id, /GLOBAL, 'Title', filename
> ncdf_attput,id,var1_data,'Projection','cylindrical'
>
> ncdf_control,id,/endef
>
> ; now write out data
>
> ncdf_varput,id,varlat,latgrid
> ncdf_varput,id,varlon,longrid
> ncdf_varput,id,varpres,pgrid
>
> ncdf_varput,id,var1_data,first_var
> ncdf_varput,id,var2_data,second_var
>
>
>
> address@hidden
> Chief Scientist
> Science and Technology Corporation
> 10015 Old Columbia Road
> Suite E-250
> Columbia, MD 21046
> phone: 410-309-0818 x 106
> cell: 240-481-7390
> http://markschoeberl.com
>
>
>
>
> > On Dec 10, 2015, at 3:44 PM, Unidata IDV Support <address@hidden> wrote:
> >
> > Hi Mark,
> >
> > Yes, it is not always as easy as we would like for data providers to adhere
> > to
> > standards.
> >
> > Let's step back for a moment. What is the provenance of these data? Where
> > are
> > they from? How are you converting them into netCDF format?
> > These data are straightforward so we should hopefully be able to arrive at
> > the
> > goal here quickly.
> >
> > Best,
> >
> > Unidata IDV Support
> >
> >> okay, actually the web site you pointed me at was helpful. In your
> >> documentation, I suggest that
> >> you point to this site.
> >>
> >> Just so I understand what you are saying. My variable ‘latitude’ should
> >> be named ‘grid_latitude’
> >> and ‘temperature’ should be ‘air_temperature’
> >>
> >> However, there is no variable named ‘probability’ … how do I deal with
> >> that or other variables that
> >> I might invent that have no standard name.
> >>
> >> Thank you,
> >>
> >> Mark
> >>
> >>
> >> address@hidden
> >> Chief Scientist
> >> Science and Technology Corporation
> >> 10015 Old Columbia Road
> >> Suite E-250
> >> Columbia, MD 21046
> >> phone: 410-309-0818 x 106
> >> cell: 240-481-7390
> >> http://markschoeberl.com
> >>
> >>
> >>
> >>
> >>> On Dec 10, 2015, at 1:24 PM, Unidata IDV Support <address@hidden> wrote:
> >>>
> >>> Mark,
> >>>
> >>> The problem with these data is that they do not follow CF (or any)
> >>> conventions
> >>> so the IDV cannot open them.
> >>>
> >>> http://www.unidata.ucar.edu/software/netcdf/conventions.html
> >>>
> >>> I would also maybe suggest specifying the central longitude for the
> >>> cylindrical
> >>> projection. (Panoply must be assuming a central longitude of 0, I guess.)
> >>>
> >>>
> >>> Best,
> >>>
> >>> Unidata IDV Support
> >>>
> >>>
> >>>> File has been uploaded.
> >>>> Please let me know what I need to do to the nc file to get this IDV to
> >>>> work.
> >>>> Thanks
> >>>>
> >>>> - mark
> >>>>
> >>>> On Dec 10, 2015, at 10:40 AM, Unidata IDV Support <address@hidden> wrote:
> >>>>
> >>>> Mark,
> >>>>
> >>>> Please upload your data to
> >>>>
> >>>> http://motherlode.ucar.edu/repository/alias/idvupload
> >>>>
> >>>> and let us know when it has been uploaded.
> >>>>
> >>>> It is also possible that you simply need to select a "Data Source Type"
> >>>> of "Grid
> >>>> files (netCDF/GRIB/OPeNDap/GEMPAK)" in the IDV Dashboard, "Data
> >>>> Choosers" tab.
> >>>>
> >>>> Best,
> >>>>
> >>>> Unidata IDV Support
> >>>>
> >>>>
> >>>> I want to use the idv package to visualize my data.
> >>>>
> >>>> I created a netcdf file. It can be read and and visualized by PANOPLY.
> >>>> But Idv does not recognize it as a netcdf file - obviously it is looking
> >>>> for something I don't have in my file.
> >>>>
> >>>> I studied your documentation and even looked at an example file. But it
> >>>> beats me what I am doing wrong.
> >>>>
> >>>> Can you help?
> >>>>
> >>>> The file is too big to attach.
> >>>>
> >>>> Here is the PANOPLY screenshot. Can you help. Thank you. - Mark
> >>>>
> >>>> address@hidden
> >>>> Chief Scientist
> >>>> Science and Technology Corporation
> >>>> 10015 Old Columbia Road
> >>>> Suite E-250
> >>>> Columbia, MD 21046
> >>>> phone: 410-309-0818 x 106
> >>>> cell: 240-481-7390
> >>>> http://markschoeberl.com
> >>>>
> >>>>
> >>>>
> >>>> I want to use the idv package to visualize my data.
> >>>>
> >>>> I created a netcdf file. It can be read and and visualized by PANOPLY.
> >>>> But Idv does not recognize it as a netcdf file - obviously it is looking
> >>>> for something I don't have in my file.
> >>>>
> >>>> I studied your documentation and even looked at an example file. But it
> >>>> beats me what I am doing wrong.
> >>>>
> >>>> Can you help?
> >>>>
> >>>> The file is too big to attach.
> >>>>
> >>>> Here is the PANOPLY screenshot. Can you help. Thank you. - Mark
> >>>>
> >>>> address@hidden
> >>>> Chief Scientist
> >>>> Science and Technology Corporation
> >>>> 10015 Old Columbia Road
> >>>> Suite E-250
> >>>> Columbia, MD 21046
> >>>> phone: 410-309-0818 x 106
> >>>> cell: 240-481-7390
> >>>> http://markschoeberl.com
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Ticket Details
> >>>> ===================
> >>>> Ticket ID: RZP-160177
> >>>> Department: Support IDV
> >>>> Priority: Normal
> >>>> Status: Closed
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: RZP-160177
> >>> Department: Support IDV
> >>> Priority: Normal
> >>> Status: Closed
> >>>
> >>
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: RZP-160177
> > Department: Support IDV
> > Priority: Normal
> > Status: Closed
> >
>
>
>
Ticket Details
===================
Ticket ID: RZP-160177
Department: Support IDV
Priority: Normal
Status: Closed