20050513: LDM: PIPE-ing to scripts

Scott,

>Date: Fri, 13 May 2005 07:36:06 -0800
>From: "Scott Swank" <address@hidden>
>Organization: NOAA/NWS
>To: Steve Emmerson <address@hidden>
>Subject: Re: 20050512: LDM: PIPE-ing to scripts
>Keywords: 200504291922.j3TJMbKx001883

The above message contained the following:

> The pqinsert command shows the full byte count of the file that it 
> inserts, and the 'ldmadmin watch' that is running on the receiving 
> machine sees the product with the correct byte count.  Interestingly 
> enough, of the 6 products that I'm currently testing (of 6000), 4 of 
> them work fine, and 2 fail to be extracted from the queue.  Any thoughts 
> as to what might be wrong with my code?  There is nothing in the ldm 
> logfile to indicate a problem.  In fact, the logfile contains almost 
> nothing (except heartbeat notes).

Then it is likely that everything is working correctly, from the LDM
perspective, and the problem lies with the decoder.

Unfortunately, I didn't see anything obviously wrong with the code.

If I were to have written the doecoder, however, I wouldn't read an
"int" at a time via fread(), but would read a large number of bytes 
into a large buffer via read() and use the return value of read() to
tell me how many bytes were read.  For example:

    char        buf[8192];
    ssize_t     nbytes;

    while ((nbytes = read(0, buf, sizeof(buf)) != -1)
        write(outFd, buf, nbytes);

This assumes that the "-close" option was specified to the PIPE action.
If that's not the case, then the decoder won't terminate.  Perhaps
that's your problem.

> Thanks,
> Scott

Regards,
Steve Emmerson