[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #KJU-295092]: LDM pipe, without -close

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #KJU-295092]: LDM pipe, without -close
  • Date: Tue, 07 Jun 2016 12:55:41 -0600
Andy,

> We have been using the -close option.  But, when many products arrive in a
> short period, we get hundreds of copies of our decoder attempting to run,
> and simply run out of resources.  So, I was thinking we could write a
> decoder to handle PIPE contents serially, and spawn a (limited) number of
> threads to process products.

One of the trade-off costs of Python is that it's slower than, for example, C.

> I also considered using the -metadata option, and looking for a particular
> sequence in the metadata, but I'm not able to understand/decode the binary
> parts of the metadata.

I wouldn't bother with the "-metadata" option. As far as I know, every 
data-product contains a means to distinguish how long it is. For example, WMO 
textual bulletins always start with SOH, CR, CR, LF and GRIB messages contain 
the length of the product in their header. If you write your decoder so that it 
understands the format of the data-product, then the decoder will be able to 
determine product boundaries.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: KJU-295092
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.