[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20050504: LDM: PIPE-ing to scripts

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: 20050504: LDM: PIPE-ing to scripts
  • Date: Wed, 04 May 2005 15:26:25 -0600
Scott,

>Date: Wed, 04 May 2005 13:04:37 -0800
>From: "Scott Swank" <address@hidden>
>Organization: NOAA/NWS
>To: Steve Emmerson <address@hidden>
>Subject: Re: 20050504: LDM: PIPE-ing to scripts
>Keywords: 200504291922.j3TJMbKx001883

The above message contained the following:

> Depending on the datatype and the decoder, I will know whether it's 
> binary data or text data.  Won't cat > $1 fail on binary data?  If not, 
> great, but if so, how do you recommend I redirect stdin to output the file?

The cat(1) utility works just fine on non-textual data.  You can verify
this yourself via the command

    cat $HOME/bin/rpc.ldmd | wc

Any decoder you write, however, shouldn't just use the cat(1) utility
because that's just a less efficient way of performing the FILE action.
Typically, decoders invoked via PIPE actions transform the data-product
into something that is more easily accessible by a subsequent analysis
and display package (e.g., GEMPAK, McIDAS).

> > There are LDM decoders that have handled terabytes of binary data just 
> > fine.
> How are they getting the data out of stdin for manipulation?

The compiled decoders that handle non-textual data-products typically
read the data from the standard-input stream using either the fread() or
read() functions.

One can also write a perl(1) script that does something similar.

I take it you're not familiar with UNIX programs that read non-textual
data from standard-input.  You might want to consult with a UNIX person
at your site.

> Thanks,
> Scott

Regards,
Steve Emmerson