[netCDF #MKO-964114]: some questions

Hi Doug,

> I have a couple of questions about merging or combining multiple netCDF 
> files. Do they have to have the same dimensions? For example, I have one 
> netCDF file with time dimension, but another one with different time 
> dimension. For example, the first one covers a span of four years from 1995 
> to 1998, but the second one from 1970 to 2000. Do I have to adjust them 
> before merging? Another question is how should I do if there are several 
> variables from different netCDF files have the same name? They may have 
> different meanings.

The basic netCDF library software doesn't include utilities for merging netCDF 
files.
For that, I recommend you use the NCO (netCDF Operators) package, with its 
ncrcat and
ncecat utilities specifically designed to support various common patterns of 
merging
netCDF files.  Alternative packages that might also support the functionality 
you need
include NCL and CDO.  You can find more information about these packages here:

  http://www.unidata.ucar.edu/netcdf/software.html#NCO
  http://www.unidata.ucar.edu/netcdf/software.html#NCL
  http://www.unidata.ucar.edu/netcdf/software.html#CDO

I know a little about these packages, but am not an expert user, so I can't 
answer
the specific questions you're asking about their support for merging data.

--Russ


> From: Unidata netCDF Support <address@hidden>
> To: address@hidden
> Cc: address@hidden
> Sent: Friday, May 4, 2012 10:17 AM
> Subject: [netCDF #MKO-964114]: some questions
> 
> > Russ:
> >
> > Thank you very much for your answers. They are very helpful. But some new 
> > questions raise up. Regarding the first question, a variable has a fixed 
> > size or shape, which is defined by its dimensions.
> 
> That's right except that the shape can grow along a dimension declared as 
> "unlimited".  So a variable could have a shape such as (time, lon, lat)
> where lon and lat are fixed size to represent a latitude-longitude grid,
> but time could be declared "unlimited", so that more data can be added along
> the time dimension for later time steps.
> 
> A netCDF-3 file can have only one unlimited dimension, but a netCDF-4 file
> may have multiple unlimited dimensions.
> 
> > No matter how many variables stored in a netCDF file, they have to have the 
> > same size or shape or dimensions if they are stored in the same netCDF 
> > file. Is my understanding right?
> 
> No, that's wrong.  Dimensions of different sizes may be used in declaring
> variables of different shapes in the same netCDF file.  Variables that share
> dimensions can be defined on the same grid, but variables on different grids
> using completely different dimensions can also be defined in the same file.
> 
> > Then, How many dimensions can a variable have? Any limitation?
> 
> A variable may have up to 1024 dimensions, although typically variables
> have 0, 1, 2, 3, or 4 dimensions.  1024 different dimensions may be defined
> in a file, so you could have lots of variables defined on different grids.
> 
> > As for the second question, can I consider the structure of netCDF as the 
> > file system under Windows? Hierarchical, root, branch, multiple folders or 
> > directories containing a variety of files?
> 
> No, you can't mount a netCDF file as a Windows file system.  However a 
> netCDF-4
> file is similar conceptually, in that groups are organized hierarchically and
> each group has its own name space or scope for naming dimensions, variables,
> and attributes.  The top-level unnamed group is analogous to the root 
> directory
> in a file system.  Each group can have any number of subgroups.  However, you
> can't store a file in a group, only netCDF metadata and data objects.
> 
> > Can I install NcML on Windows platform? NcML creates binary files different 
> > from netCDF?
> 
> Yes, NcML is part of netCDF-Java.  It is pure Java, so it works fine on 
> Windows:
> 
>   http://www.unidata.ucar.edu/software/netcdf-java/
> 
> NcML doesn't "create" netCDF files, it "wraps" them to provide additional
> metadata or to aggregate multiple netCDF files and make them appear as if
> they are one netCDF file.  NcML can wrap files in other formats as well that
> are supported by the Unidata Common Data Model (CDM).  This provides a way to
> leave files in their original format, but make them appear to the netCDF-Java 
> library as if they are netCDF files or conform to netCDF metadata conventions.
> 
> NetCDF-Java can read or write netCDF-3 files and read netCDF-4 files, but it
> cannot yet write netCDF-4 files.
> 
> --Russ
> 
> > ________________________________
> > From: Unidata netCDF Support <address@hidden>
> > To: address@hidden
> > Cc: address@hidden
> > Sent: Thursday, May 3, 2012 6:00 PM
> > Subject: [netCDF #MKO-964114]: some questions
> >
> > Hi Doug,
> >
> > > I am a new user of netCDF and wish you could help me with my questions.
> > > 1. Within one netCDF file, can variables have different dimensions? In my 
> > > imagination, netCDF stores one variable (array) in one layer. Each layer 
> > > may not have relation with others. In this way, variables in one netCDF 
> > > can have different dimensions.
> >
> > No, a single variable is defined to be a specific shape, determined by the 
> > list of dimensions
> > it uses.
> >
> > However, netCDF-4 files have named "groups", and each group may have 
> > dimensions and variables
> > with the same names but different sizes or shapes as dimensions and 
> > variables in other groups.
> > It is analogous to the situation with files in directories, where the same 
> > name can refer to
> > different files in different directories of a hierarchical file system.
> >
> > So maybe what you are calling "layers" are like netCDF-4 "groups".
> >
> > > 2. When we create a netCDF file, do we have to put all data in the CDL 
> > > file? It may not be complex but time consuming. Can we link a 
> > > well-defined external file to the CDL file? For example, indicate the 
> > > external fiel location on the hard drive.
> >
> > No, you don't need to use CDL files (which are human-readable text files).  
> > NetCDF files
> > (which are binary files) may be created directly with custom programs or 
> > utilities.
> > Typically, you either use a CDL file to create metadata, then the ncgen 
> > utility to turn
> > that into a binary netCDF file.  Or you use a program to create e netCDF 
> > file, then use
> > ncdump to see that as a CDL file.  But you don't need to manually modify a 
> > CDL file to
> > keep it consistent with a netCDF file (which I think you are asking about, 
> > if I understand
> > your question).
> >
> > There is no standard or convention for including data by reference or 
> > linking in a binary
> > netCDF file, although you could store file names as values of string type 
> > attributes or
> > variables.  However, netCDF Java supports NcML files, which can store links 
> > to external
> > data in one or more other files, which can be netCDF files.  For more 
> > information about
> > NcML, and especially the use of NcML for aggregation, here:
> >
> >   http://www.unidata.ucar.edu/software/netcdf/ncml/
> >
> > --Russ
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: MKO-964114
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> Russ Rew                                         UCAR Unidata Program
> address@hidden                      http://www.unidata.ucar.edu
> 
> 
> 
> Ticket Details
> ===================
> Ticket ID: MKO-964114
> Department: Support netCDF
> Priority: Normal
> Status: Closed
> 
Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: MKO-964114
Department: Support netCDF
Priority: High
Status: Closed