[netCDFJava #VKG-526884]: Issue in opening IASI BUFR file

Greetings Ghansham,

Would you mind sending the message below to the cf-sattelite list? This will 
help us keep the two issues separate and help others find (hopefully) a 
solution.

In the mean-time, I would suggest checking out page 5-3 in the following 
document:

http://newsroom.gsfc.nasa.gov/sdptoolkit/docs/HDF-EOS5_UG.pdf

I do not have much experience with HDF5 or satellite data, but this seems to 
describe your issue.

Cheers!

Sean

> Thanks a lot Sean Sir. Meanwhile can you help me to sort out this one.
> Sometime back I had  a mail conversation with John sir. Below is given the
> conversation.
> He referred to a way in which netcdf-4 can write HDF-5 files with a kludge
> to handle the shared dimension. Can you just help me to get on such sample
> hdf-5 file and a sample program that write such file.
> It will be a great help. Just go through the mail conversation first.
> 
> regards
> Ghanshm
> 
> 
> R/Sir
> 
> With the reference to this link:
> 
> http://www.unidata.ucar.edu/mailing_lists/archives/cf-satellite/2011/msg00069.html
> 
> I have pasted the contents of the link here
> 
> Re: [cf-satellite] Query related to CF conventions
> 
> - *To*: cf-satellite@xxxxxxxxxxxxxxxx <cf-satellite@DOMAIN.HIDDEN>
> - *Subject*: Re: [cf-satellite] Query related to CF conventions
> - *From*: John Caron <caron@xxxxxxxxxxxxxxxx <caron@DOMAIN.HIDDEN>>
> - *Date*: Fri, 29 Jul 2011 06:40:45 -0600
> 
> On 7/28/2011 10:32 PM, ghansham sangar wrote:
> 
> R/Sir
> 
> 
> I have an issue related to CF conventions 2D coordinate variables (I mean
> lat/lon arrays corresponding to satellite images) I have been able to
> handle the case where lat/lon arrays have one to one correspondence with
> satellite imagery pixels.
> 
> But if we have some scaling like:
> 1. Image Data ( 5600* 5600)
> Lat/Lon Arrays (1400*1400) each
> 
> Or I have the other ways
> 2. Image Data(1400x1400)
> Lat/Lon Arrays (2800*2800).
> 
> 
> How to indicate that in the image pixel should refer to which Lat/Lon array
> pixel. Like in first case every four pixels of image correspond to 1
> lat/lon array pixel.
> 
> I am using HDF 4 for writing products.
> 
> I have tried to search on internet. It says something about dimscale. But I
> am not aware what exactly to do.
> 
> regards
> Ghansham
> 
> 
> _______________________________________________
> cf-satellite mailing list
> cf-satellite@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> 
> HI Ghansham:
> 
> 
> 1. I understand that HDF-EOS has a convention for your situation. Are you
> using that? Could you explain how it works?2. CF uses netCDF as its file
> format. HDF4 lacksshared dimensions, so is not an complete fit to the CF
> model. Why are you writing in HDF4?
> 
> John
> 
> 
> 
> 
> 
> I  am not using HDF-EOS.
> I am sorry in the last mail, I mentioned it was HDF-4. But as in the coming
> time we plan to generate most of the data products in HDF-5.
> Is it now possible in HDF-5 context?
> HDF-5 has dimension scale feature.
> I also read that HDF-5 dimension scale is not exactly what is netcdf shared
> dimension.
> But now I want to know is there a way in which we can accommodate such
> datasets in HDF-5.
> I mean to say can we have a 2-D coordinate variable (as stated in the
> previous mail) in HDF-5:
> 
> Image Data (5600*5600)
> Lat Array (1400*1400), Lon Array (1400*1400)
> 
> OR
> 
> Image Data (1400*1400)
> Lat Array (2800*2800), Lon Array (2800*2800)
> regards
> Ghansham
> 
> 
> 
> 
> HDF5 has dimension scales, but they are not the same as shared
> dimensions (close,
> no cigar).
> 
> You want to use netCDF4 for this. It writes HDF5 files, with a kludge, I
> mean workaround, for shared dimensions.
> 
> 
> 
> 
> 
> 
> 
> address@hidden> wrote:
> 
> > Greetings Ghansham,
> >
> > I don't think the fix is out yet (John is out this week, but he identified
> > the bug some time ago), but when it gets released it will be in netCDF-Java
> > 4.3.  That means we will have to wait until the IDV incorporates
> > netCDF-Java 4.3 before you will be able to load this file into the IDV. We
> > hope to have this done in the next month or two. Unfortunately, there are
> > some very big changes to the way netCDF-Java 4.3 will handle grib files, so
> > we are treading very carefully about how to best move to the new version of
> > netCDF-Java.
> >
> > Sorry for the delay,
> >
> > Sean
> >
> > > R/Sir
> > >
> > > Any updates on this one?
> > >
> > > regards
> > > Ghansham
> > >
> > > address@hidden> wrote:
> > >
> > > > Greetings Ghansham,
> > > >
> > > > I do apologize for the delay. Thank you for uploading the sample file.
> > > > Your example revealed a bug and it's currently being fixed. I'll let
> > you
> > > > know when the fix gets released to github.
> > > >
> > > > Cheers,
> > > >
> > > > Sean
> > > >
> > > > > I have uploaded the required BUFR file quite some time ago.
> > > > > Is there any update on this ticket.
> > > > > Kindly help
> > > > >
> > > > > regards
> > > > > Ghansham
> > > > >
> > > > > address@hidden> wrote:
> > > > >
> > > > > > Greetings Ghansham,
> > > > > >
> > > > > > You may upload the data on our RAMADDA server in the following
> > > > location:
> > > > > >
> > > > > >
> > > > > >
> > > >
> > http://motherlode.ucar.edu/repository/entry/show/RAMADDA/Unidata/Software/CDM/Support+Uploads
> > > > > >
> > > > > > Cheers!
> > > > > >
> > > > > > Sean
> > > > > >
> > > > > > > Can I upload it on one of Unidata RAMADDA servers?
> > > > > > > I have uploaded some data sometime back.
> > > > > > >
> > > > > > > regards
> > > > > > > Ghansham
> > > > > > >
> > > > > > > address@hidden> wrote:
> > > > > > >
> > > > > > > > can you use a service like
> > > > > > > >
> > > > > > > > http://www.dropbox.com/
> > > > > > > >
> > > > > > > > ??
> > > > > > > >
> > > > > > > > > Its larger than 25 MB.
> > > > > > > > > Do you have some server where I can upload that.
> > > > > > > > >
> > > > > > > > > regards
> > > > > > > > > Ghansham
> > > > > > > > >
> > > > > > > > > On Sat, Jan 21, 2012 at 6:56 AM, Unidata netCDF Java Support
> > > > > > > > > address@hidden> wrote:
> > > > > > > > >
> > > > > > > > > > Hi Ghansham:
> > > > > > > > > >
> > > > > > > > > > Can you sent the problem bufr file?
> > > > > > > > > >
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > > > R/Sir
> > > > > > > > > > >
> > > > > > > > > > > I have one IASI BUFR file. I am not able to send it
> > through
> > > > email
> > > > > > > > because
> > > > > > > > > > > size exceeds 25 MB.
> > > > > > > > > > > But I am seeing one of the variable name coming
> > NULL(selected
> > > > > > entry
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > > snapshot attached).
> > > > > > > > > > > If you need I can upload it somewhere.
> > > > > > > > > > > I think some of the BUFR table might be missing. Can you
> > just
> > > > > > point
> > > > > > > > out
> > > > > > > > > > > what should be done to get it right.
> > > > > > > > > > > The reason is due the variable name coming NULL, I am not
> > > > able to
> > > > > > > > open it
> > > > > > > > > > > in IDV.
> > > > > > > > > > > Due to that reason I am not able save it as NcML.
> > > > > > > > > > >
> > > > > > > > > > > regards
> > > > > > > > > > > Ghansham
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Ticket Details
> > > > > > > > > > ===================
> > > > > > > > > > Ticket ID: VKG-526884
> > > > > > > > > > Department: Support netCDF Java
> > > > > > > > > > Priority: Urgent
> > > > > > > > > > Status: Open
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Ticket Details
> > > > > > > > ===================
> > > > > > > > Ticket ID: VKG-526884
> > > > > > > > Department: Support netCDF Java
> > > > > > > > Priority: Urgent
> > > > > > > > Status: Open
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Ticket Details
> > > > > > ===================
> > > > > > Ticket ID: VKG-526884
> > > > > > Department: Support netCDF Java
> > > > > > Priority: Critical
> > > > > > Status: Open
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: VKG-526884
> > > > Department: Support netCDF Java
> > > > Priority: Critical
> > > > Status: Open
> > > >
> > > >
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: VKG-526884
> > Department: Support netCDF Java
> > Priority: Critical
> > Status: Open
> >
> >
> 
> 


Ticket Details
===================
Ticket ID: VKG-526884
Department: Support netCDF Java
Priority: Critical
Status: Open