[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: THREDDS repo and HDF5 datasets with huge attributes

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: THREDDS repo and HDF5 datasets with huge attributes
  • Date: Thu, 30 Apr 2015 18:45:43 +0000
John,

See pull request #131.

Basically, the code for getting the Fractal Heap ID for a huge object was bad.

Note that the code was also only trying to figure out the heap ID for a huge 
obect of subtype 1, but my reading of the file format specification 
(https://www.hdfgroup.org/HDF5/doc/H5.format.html) indicates that the same 
method should be applied to subtype 2. The sample SMAP files I was workingwith 
used subtype 1.

rbs



On Apr 30, 2015, at 11:26, John Caron <address@hidden> wrote:

> Hi Robert:
> 
> Just sent a note about the branch changes. Sorry for the abrupt switch.
> 
> As for the damn H5 bug, I just got a note from THG with an updated 
> explanation of that, but I havent had a chance to look at it yet. I will 
> forward it to you. Would be truly awesome if you could find the problem and 
> give us a pull request against master.
> 
> John
> 
> On Wed, Apr 29, 2015 at 8:01 PM, Schmunk, Robert B. (GISS-611.0)[TRINNOVIM, 
> LLC] <address@hidden> wrote:
> 
> John,
> 
> Having not seen any pertinent activity on the THREDDS repository on Github 
> regarding the bug in NJ’s ability to read HDF5 datasets with huge attributes 
> (using dense attribute storage), I have been trying to track down the bug 
> myself. It’s been a maze, comparing the NJ source code to the H5 format 
> documentation, and I’d rather not think about how long I’ve doing so. But 
> nevertheless, I think I found where the problem is and how to fix it! 
> Certainly I have managed to open some sample SMAP datasets with attributes of 
> length > 100 kB.
> 
> But on checking back on the THREDDS repo today, I found the 4.6.0 and 4.6.1 
> branches have both disappeared, which is very odd because they were there 
> late last night. In fact, it seems like all the numbered branches have 
> disappeared except for 4.5.6 and 5.0.0 (!?). Is something flaky going on with 
> GitHub or is there something else happening behind the curtain with managing 
> the NJ/THREDDS source code?
> 
> rbs
> 
> 
> --
> Robert B. Schmunk
> Webmaster / Senior Systems Programmer
> NASA Goddard Institute for Space Studies
> 2880 Broadway, New York, NY 10025
> 
> 
> 

--
Robert B. Schmunk
Webmaster / Senior Systems Programmer
NASA Goddard Institute for Space Studies
2880 Broadway, New York, NY 10025