[netCDF #XFB-271301]: netCDF question, char/string attributes, global or variable atts.

Greg,

> You're absolutely right about netcdf4 - I am using this version on purpose
> to take advantage of the compression.  When I made the older v3 type, it
> was quite large so I experimented with v4.  Guess I need to find all that
> I can about the newer API for v4 so I can keep moving forward in the new
> week because I need to get a particular piece of code written prior to
> AMS.  So, I'll try.

Is there any way you can use netCDF-4 classic model files, instead of netCDF-4
files?  The former support compression with the netCDF-3 data model and data
types, and appear to be completely implemented in the current netCDF-4 Fortran
library.  However (and I'm somewhat alarmed to discover this), the netCDF-4
Fortran library is not just missing documentation and tests for the netCDF-4
string type, but also missing implementation of support for the netCDF-4
NF90_STRING data type.  That explains the dearth of examples and test for
that type in the source code.

I was mistaken in thinking that a work-around was to use the NF90_VLEN type 
instead of the NF90_STRING type.  As you've discovered by digging around,
the NF90_VLEN functions are also missing documentation, and I suspect their
implementation is also incomplete.

This all works in the C and Java libraries, but apparently you're the first
to try to make use of the netCDF-4 Fortran string type in the netCDF-4
enhanced data model, rather than just using the netCDF-4 classic-model format.
I'm also embarrassed that we didn't notice this issue earlier.

I'd like to discuss workarounds in more detail for your AMS deadline, 
including use of a little C program to make the change to append to the 
history attribute, a C program to convert that attribute to the NC_CHAR
type that the F90 library can handle, or other alternatives, but need to
know more about the problem first.  I'll try calling later ...

--Russ

> On Jan 11, 2014, at 2:58 PM, Unidata netCDF Support wrote:
> 
> > Hi Greg,
> >
> >> I am really struggling with something in netCDF and getting a little
> >> desperate for help - I've done google search much of the last 2 days,
> >> checked support/mail archives, stack-overflow, etc.  I'd greatly
> >> appreciate if you could give me additional guidance on this topic.
> >>
> >> I have a 64-bit offset NC file I created using NCL reading, processing,
> >> and writing data from an original GRIBv2 file to make the NC file.  Now,
> >> for further post-processing and deriving new data, I'm reading in that
> >> NC file (and others), doing some math on the data, and writing a new
> >> output NC file, using F90 (ifort compiler on Yellowstone).
> >>
> >> I am having a bear of a time with character variables.  And I do
> >> understand that strings are not stored, but, rather, array of
> >> characters.  So, when I created my first NC file, I made a global
> >> attribute called "history" which I can clearly see when I do ncdump -h
> >> on the file and it looks like this:
> >>  string :history = "Created Tue Dec 31 11:08:39 MST 2013" ;
> >
> > It's apparently not a 64-bit offset netCDF file, but a netCDF-4 file
> > instead.  You can verify this using ncdump -k, assuming you have an
> > ncdump linked against netCDF version 4.0 or later.
> >
> > If the file was written using NCL, maybe it's configured for the default
> > to be writing netCDF-4 files.  NetCDF-4 files use an enhanced data model
> > and support additional data types, such as "string", that are not
> > accessible to older netCDF-3 libraries or programs linked against them.
> >
> > For more information on the differences between the netCDF classic data
> > model and the netCDF-4 enhanced data model, see this FAQ section on
> > "Formats, Data Models, and Software Releases":
> >
> >  http://www.unidata.ucar.edu/netcdf/docs/faq.html#formats-and-versions
> >
> > Evidently, NCL creates netCDF-4 attributes using the netCDF-4 primitive
> > type "string" instead of an array of chars.
> >
> >> And, since I'm doing additional computations on data, I want to append
> >> to that history variable.  So I'm trying to read in this global
> >> attribute to capture that string, append new text to the end and then
> >> write to my new NC output file.
> >
> > To read an attribute of type "string" in netCDF-4, you need to know
> > about netCDF-4 variable-length types, as the netCDF-4 primitive type
> > NF90_STRING is handled like the NF90_VLEN user-defined type, because
> > a read returns a pointer to allocated space that must later be
> > explicitly freed when you're done with it.
> >
> > I see there are no good examples of using NF90_STRING type data in our
> > on-line examples, and the documentation for use of that type is poor.
> > The netCDF Fortran-90 users guide was written before the netCDF-4 APIs
> > were added, and it looks like the needed rewrite of the docs was not
> > completed before the developer working on that project left.
> >
> > I'll try to create a new example that reads and writes NF90_STRING
> > data using the F90 API, but it may have to wait until after the AMS
> > meeting.  In the mean time, I recommend seeing if you can make NCL
> > write either netCDF classic format files or netCDF-4 classic model
> > files, neither of which use strings for attributes.
> >
> >> I've read 10s of web pages trying to find anyone who has applied
> >> NF_GET_ATT function and there's just something I'm missing.
> >>
> >> When I use this:
> >>               nf_status = nf90_inquire_attribute(ncid, NF90_GLOBAL,
> >> attname, xtype, leng, j)
> >> where attname='history', it returns xtype=12 and leng=1.
> >> The tutorial page (for netcdf) says xtype should be one of NF_BYTE,
> >> NF_CHAR, ..., which are just consecutively numbered 1 to 6, so I am at a
> >> loss what 12 indicates.  And, of course, a length of this attribute of 1
> >> is not helpful.  The string reported by ncdump is clearly longer.
> >
> > The netCDF-4 documentation for nf90_def_var mentions using NF90_STRING
> > as a netCDF variable type, but it's not mentioned in the docs as a valid
> > return type for any of the other netCDF functions!
> >
> > The source file fortran/netcdf_constants.f90 has all the netCDF-4 type
> > codes, which should have been documented in the Users Guide:
> >
> >  ! extra data types:
> >  integer, parameter, public :: &
> >       nf90_ubyte = 7, &
> >       nf90_ushort = 8, &
> >       nf90_uint = 9, &
> >       nf90_int64 = 10, &
> >       nf90_uint64 = 11, &
> >       nf90_string = 12, &
> >       nf90_vlen = 13, &
> >       nf90_opaque = 14, &
> >       nf90_enum = 15, &
> >       nf90_compound = 16
> >
> >> And, then when I attempt just to read in the attribute anyway, of course
> >> I encounter the error: "NetCDF: Attempt to convert between text &
> >> numbers" when using:
> >>   nf_status = nf90_get_att(ncid, NF90_GLOBAL, attname, str_hist)
> >> where str_hist is a f90 char(len=128).
> >>
> >> I just cannot comprehend how to retrieve textual global attributes of
> >> netcdf files, nor text attributes of a variable, such as "units" which I
> >> have clearly set in my variables.  The above inquire_attribute and
> >> get_att functions do exactly the same thing as my history example.
> >>
> >> Any help would really be appreciated!
> >
> > I sympathize with your frustration at the incomplete documentation.
> >
> > I've added a bugtracking issue about this, so it doesn't continue to get
> > ignored:
> >
> >  https://bugtracking.unidata.ucar.edu/browse/NCFORTRAN-28
> >
> > Thanks for reporting the problem!
> >
> > --Russ
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: XFB-271301
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: XFB-271301
Department: Support netCDF
Priority: Normal
Status: Closed