[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[TIGGE #BXZ-922182]: Re: .missing files & retransmissions

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [TIGGE #BXZ-922182]: Re: .missing files & retransmissions
  • Date: Thu, 20 Apr 2006 14:14:18 -0600
Baudouin,

We've imagined three ways to address your data-verification issue:

1.  Add an option to the LDM to compute the MD5 signature and compare it.

This is the one I originally imagined.  It should be relatively easy to 
implement.  It has a disadvantage of computing the MD5 signature for *all* data 
feeds -- regardless of how important they are.

2.  Modify the grammar of the LDM configuration-file (etc/ldmd.conf) to allow 
for the computation and checking of MD5 signatures on an individual REQUEST and 
ACCEPT basis.

If the LDM is to do the verification, then this is a better solution than #1.  
It would take longer to implement.  It also doesn't address the problem of 
verifying data-products after they've been written to disk.  Which brings us to 

3.  Have the ingester add the MD5 signature to a manifest-file, which is 
transmitted separately, and have a pqact(1)-based mechanism on the receiving 
end use the manifest-file to verify correct reception.

Since you're already using (or going to use) a CONDUIT-like manifest-file 
mechanism to verify reception, this mechanism would seem to be a natural 
extension of that solution.  Besides providing end-to-end verification of 
transmission, it also allows continued confidence in the data-products after 
they've been stored in the archive.

We recommend solution #3 but would like to know what you think.
Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: BXZ-922182
Department: Support IDD TIGGE
Priority: Normal
Status: On Hold