[netCDFJava #MAO-189374]: Reading HDF5 files containing 'hdf5 external links'

Hi Didier:

We use NcML Aggregation for this kind of thing. The main limitation is that its
only used in Netcdf-Java, though it _may_ eventually get added to the C
library. But if you are using java, this is a viable option now:

http://www.unidata.ucar.edu/software/netcdf/ncml/


> Hi,
>
>
>
> Thanks for your answer and information.
>
>
>
> To be more precise, what I'm trying to do with external links is to bring
together some data with different criteria.
>
>
>
> I send you a more explicit example to illustrate what I'm trying to do.
>
>
>
> I have some data netcdf4 files (it could be also hdf5 files) that contains
some data :
>
>
>
> J1.nc
>
> J2.nc
>
> J3.nc
>
> J4.nc
>
> J5.nc
>
>
>
> Now I want to organise them with different criteria, for instance by 'Cycle'
and by 'Pass'.
>
>
>
> So, I create two hdf5 files that contain links to the source data files
J1.nc, J2.nc .... :
>
>
>
> The first hdf5 file dataByCyle.h5 contains 3 groups : Cycle001, Cycle002,
Cycle003. An each group contains references to source data files:
>
> Cycle001
>
> External link --> J1.nc
>
> External link --> J2.nc
>
> Cycle002
>
> External link --> J3.nc
>
> External link --> J4.nc
>
> Cycle003
>
> External link --> J5.nc
>
>
>
> The second hdf5 file dataByPass.h5 contains 2 groups : Pass001, Pass002. An
each group contains references to source data files:
>
> Pass001
>
> External link --> J1.nc
>
> External link --> J5.nc
>
> Pass002
>
> External link --> J2.nc
>
> External link --> J3.nc
>
> External link --> J4.nc
>
>
>
> This allow to classified data (to have a different view), because I need to
access data either by 'cycle' or by 'pass' (in this example).
>
>
>
> That's a way that it seems to me rather efficient.
>
>
>
> I first thought to use only Netcdf4 file format, but 'external links' doesn't
exists in Netcdf format. And the 'external link' offers the possibility to have
different view of the data without to duplicate them (because of the amount of
data).
>
>
>
> I also plan to test the possibility to use these files into a TDS. (TDS
4.0).
>
>
>
> I hope my explaination will clarify the purpose of using 'external links'.
I'm also always open to other suggestions.
>
>
>
> HDF5ExtLinkExample.zip as attached file contains files to illustrate this
example.
>
>
>
> Best regards,
>
>
>
> Didier.
>
>
>
>
>
> -----Message d'origine-----
> De : Unidata netCDF Java Support
[mailto:address@hidden]
> Envoy : lundi 23 fvrier 2009 16:48
>  : didier earith
> Cc : address@hidden
> Objet : [netCDFJava #MAO-189374]: Reading HDF5 files containing 'hdf5
external links '
>
>
>
> Hi Didier:
>
>
>
> Thanks, we are evaluating the feasibility of supporting this feature. Im
wondering how you intend to use this? I see that the hdf5 file contains nothing
but the link to the netcdf4 file.
>
>
>
> Also, in terms of CF conformance, I would recommend that you put the
attribute alt:positive = "up" on the alt variable, per section 4.3:
>
>
>
> "For this reason the attribute positive as defined in the COARDS standard is
required if the vertical axis units are not a valid unit of pressure"
>
>
>
>
>
> > Hi,
>
> >
>
> > I send you some files as example :
>
> >
>
> > - JA2_IPR_2PcP021_001_20090126_085038_20090126_094651.nc is the file that
contains data. It's a netcdf 4 file format (it could also be a HDF5 file
format),
>
> >
>
> > - extlink_source.h5 is the hdf5 file that contains a reference (external
link) to JA2_IPR_2PcP021_001_20090126_085038_20090126_094651.nc file.
>
> >
>
> > - dump.log is the result of a h5dump (-H) of the extlink_source.h5 file.
>
> >
>
> > extlink_source.h5 file has been created with C API of the hdf5 1.8.2
library.
>
> >
>
> > Thanks,
>
> >
>
> > Didier.
>
> >
>
> > -----Message d'origine-----
>
> > De : Unidata netCDF Java Support
[mailto:address@hidden]
>
> > Envoy : vendredi 20 fvrier 2009 17:43
>
> >  : didier earith
>
> > Objet : [netCDFJava #MAO-189374]: Reading HDF5 files containing 'hdf5
external links '
>
> >
>
> > Im sorry Didier, we dont support "external links". If you send me the
files, I will evaluate if we can add that feature in the future.
>
> >
>
> > > Full Name: Didier Earith
>
> > > Email Address: address@hidden
>
> > > Organization: CLS
>
> > > Package Version: 4.0.34
>
> > > Operating System:
>
> > > Hardware:
>
> > > Description of problem: Hi,
>
> > >
>
> > > I'm try to use Netcdf-java to read HDF5 files with 'HDF5 external
>
> > > links'.
>
> > >
>
> > > I'm using Netcdf-java 4.0.34.
>
> > >
>
> > > It seems that Netcdf-Java can't read data which are declared as
>
> > > 'external link' in the HDF5 files.
>
> > >
>
> > > When reading the HDF5 file, I get no errors, but there no variable are
>
> > > found.
>
> > >
>
> > > If I dump the HDF5 file with h5dump all is fine.
>
> > >
>
> > > Is it possible to use Netcdf-java library to read HDF5 files containing
>
> > > 'HDF5 external links' ?
>
> > >
>
> > > Thanks.
>
> > >
>
> > > Didier.
>
> > >
>
> > >
>
> >
>
> >
>
> > Ticket Details
>
> > ===================
>
> > Ticket ID: MAO-189374
>
> > Department: Support netCDF Java
>
> > Priority: Normal
>
> > Status: Open
>
> >
>
> >
>
> >
>
> > Cliquez sur l'url suivante
>
> >
https://www.mailcontrol.com/sr/ZLTfjrpxYGXTndxI!oX7UoWpENiwadNGkUjoCoATthjAy59twyYb95UxruDQLd2!kpERGUhe2VHC8dmstCo9uQ==
>
> > si ce message est indsirable (pourriel).
>
> >
>
>
>
>
>
> Ticket Details
>
> ===================
>
> Ticket ID: MAO-189374
>
> Department: Support netCDF Java
>
> Priority: Normal
>
> Status: Open
>
>
>
>


Ticket Details
===================
Ticket ID: MAO-189374
Department: Support netCDF Java
Priority: Critical
Status: Closed