[LDM #FHI-907414]: Can't do a pqinsert with scripts

Hi Gilbert,

> I am having an issue where I apparently can't get the LDM to insert
> a product into the LDM queue, using the latest version of the LDM
> (6.10.1). This worked fine on previous versions of the LDM, and
> I have not changed anything with it. Here is a script I use to
> download EMWIN data as a backup in case NOAAport goes down,
> illustrating the issue. Someone else I know is able to run
> this and get it to work using an older version of the LDM
> (6.9). If you wish to log into my system to test it, it
> is in /home/ldm/wxpscripts. See below.
> 
> *******************************************************************************
> Gilbert Sebenste                                                     ********
> (My opinions only!)                                                  ******
> Staff Meteorologist, Northern Illinois University                      ****
> E-mail: address@hidden                                  ***
> web: http://weather.admin.niu.edu                                      **
> Twitter: http://www.twitter.com/NIU_Weather                            **
> Facebook: http://www.facebook.com/niu.weather                           *
> *******************************************************************************
> 
> #! /bin/bash
> # Copyright under the GPL, 2007, by Stonie R. Cooper of
> # Planetary Data, Incorporated.
> # This script and any derivative work must contain this header
> # credits/license information.
> # stonie<dot>cooper<at>planetarydata<dot>com
> # There is no warranty to the fitness of this script, it's
> # usefullness, freedom of bugs, etc.  This is as-is - use
> # at your own risk.
> 
> TEMPDIR="/tmp/emwin"
> QUEUELOC="/dev/shm/ldm.pq"
> ELOGIN="emwin:emwin@"
> EHOST="140.90.6.240"
> 
> TIMESLOTS=( \
> "2" \
> "5" \
> "15" \
> "60" \
> "180" \
> )
> 
> FILESLOTS=( \
> "newtwomins.zip" \
> "newfivemins.zip" \
> "newfifteen.zip" \
> "newonetext.zip" \
> "newthreetxt.zip" \
> )
> 
> DATASLOTS="TXT:IDS|DDPLUS:Y:HEADER GIF:EXP:N:FILE"
> #DATASLOTS="TXT:EXP:Y:HEADER GIF:EXP:N:FILE"
> 
> if [ -r /etc/profile ]; then
> source /etc/profile
> fi
> if [ -r /home/ldm/.bash_profile ]; then
> source /home/ldm/.bash_profile
> fi
> if [ -r /home/ldm/.bashrc ]; then
> source /home/ldm/.bashrc
> fi
> #if [ -z "${GEMPAK}" ]; then
> #  if [ -r /usr/local/nawips/Gemenviron.profile ]; then
> #    source /usr/local/nawips/Gemenviron.profile
> #  else
> #    exit
> #  fi
> #fi
> 
> THISHOUR=`date -u "+%H"`
> THISMINUTE=`date -u "+%M"`
> THISCOUNT="0"
> THISLIST=""
> 
> REMAINDER=`echo "${THISMINUTE} % 2" | bc`
> if [ "${REMAINDER}" -eq 0 ]; then
> THISCOUNT="1"
> THISLIST="0"
> fi
> 
> REMAINDER=`echo "${THISMINUTE} % 5" | bc`
> if [ "${REMAINDER}" -eq 0 ]; then
> THISCOUNT=`expr "${THISCOUNT}" + 1`
> THISLIST="${THISLIST} 1"
> fi
> 
> REMAINDER=`echo "${THISMINUTE} % 15" | bc`
> if [ "${REMAINDER}" -eq 0 ]; then
> THISCOUNT=`expr "${THISCOUNT}" + 1`
> THISLIST="${THISLIST} 2"
> fi
> 
> if [ "${THISMINUTE}" -eq 1 ]; then
> THISCOUNT=`expr "${THISCOUNT}" + 1`
> THISLIST="${THISLIST} 3"
> fi
> 
> REMAINDER=`echo "${THISHOUR} % 3" | bc`
> if [ "${REMAINDER}" -eq 0 -a "${THISMINUTE}" -eq 1 ]; then
> THISCOUNT=`expr "${THISCOUNT}" + 1`
> THISLIST="${THISLIST} 4"
> fi
> 
> if [ "${THISCOUNT}" -eq 0 ]; then
> exit
> fi
> 
> THISLIST=( ${THISLIST} )
> 
> INTERNALCOUNT="0"
> sleep 15
> while [ "${INTERNALCOUNT}" -lt "${THISCOUNT}" ]
> do
> if [ ! -d ${TEMPDIR}/${TIMESLOTS[${THISLIST[${INTERNALCOUNT}]}]} ]; then
> mkdir -p ${TEMPDIR}/${TIMESLOTS[${THISLIST[${INTERNALCOUNT}]}]}
> fi
> if [ ! -d ${TEMPDIR}/${TIMESLOTS[${THISLIST[${INTERNALCOUNT}]}]} ]; then
> exit
> fi
> 
> cd ${TEMPDIR}/${TIMESLOTS[${THISLIST[${INTERNALCOUNT}]}]}
> LOCATION=`pwd`
> if [ "${TEMPDIR}/${TIMESLOTS[${THISLIST[${INTERNALCOUNT}]}]}" != 
> "${LOCATION}" ]; then
> exit
> fi
> for EACHZIP in `echo "${FILESLOTS[${THISLIST[${INTERNALCOUNT}]}]}" | tr ':' ' 
> '`
> do
> wget --passive-ftp -q ftp://${ELOGIN}${EHOST}/${EACHZIP}
> unzip -qq ${EACHZIP} > /dev/null 2>&1
> rm -f ${EACHZIP}
> FILECOUNT="0"
> for eachFile in `find . -type f ! -empty -print`
> do
> FILECOUNT=`expr "${FILECOUNT}" + 1`
> if [ "${FILECOUNT}" -gt 999 ]; then
> FILECOUNT="1"
> fi
> SEQUENCE=`printf "%03d" "${FILECOUNT}"`
> SEQUENCE="999"
> HEADER=`head -c 27 ${eachFile} | tr '\r' '_' | sed "s/__//g"`
> WMO=`echo ${HEADER} | cut -c1-18`
> PIL=`echo ${HEADER} | cut -c20- | tr '_' ' '`
> PIL=`echo ${PIL} | grep ^[A-Z]`
> if [ -n "${PIL}" ]; then
> PIL=`echo ${PIL} | awk -F" " '{print $1}'`
> PILSIZE=`echo ${PIL} | wc -c`
> if [ ${PILSIZE} -gt 4 ]; then
> PRODUCT=`printf "%s /p%s" "${WMO}" "${PIL}"`
> else
> PRODUCT="${WMO}"
> fi
> else
> PRODUCT="${WMO}"
> fi
> THISFILE=`echo ${eachFile} | awk -F"/" '{print $NF}'`
> FILETYPE=`echo "${THISFILE}" | awk -F"." '{print $2}'`
> THISFEEDTYPE="EXP"
> THISPREAMBLE="Y"
> THISPRODUCT="HEADER"
> for EACHTYPEINLIST in `echo "${DATASLOTS}"`
> do
> SUFFIX=`echo "${EACHTYPEINLIST}" | awk -F":" '{print $1}'`
> FEEDTYPE=`echo "${EACHTYPEINLIST}" | awk -F":" '{print $2}'`
> PREAMBLE=`echo "${EACHTYPEINLIST}" | awk -F":" '{print $3}'`
> HPRODUCT=`echo "${EACHTYPEINLIST}" | awk -F":" '{print $4}'`
> if [ "${FILETYPE}" = "${SUFFIX}" ]; then
> THISFEEDTYPE="${FEEDTYPE}"
> THISPREAMBLE="${PREAMBLE}"
> THISPRODUCT="${HPRODUCT}"
> fi
> done
> if [ "${THISPREAMBLE}" = "Y" ]; then
> printf "\001\r\r\n%s\r\r\n" "${SEQUENCE}" > temp
> fi
> cat ${eachFile} >> temp
> if [ "${THISPREAMBLE}" = "Y" ]; then
> BOOL=`cat temp | tr '\003' '@'  | grep "@" | wc -l | awk -F" " '{print $1}'`
> if [ ${BOOL} -eq 0 ]; then
> printf "\r\r\n\003" >> temp
> fi
> fi
> if [ "${THISPRODUCT}" = "HEADER" ]; then
> pqinsert -q ${QUEUELOC} -s ${SEQUENCE} -f "${THISFEEDTYPE}" -p "${PRODUCT}" 
> temp > /dev/null 2>&1
> elif [ "${THISPRODUCT}" = "FILE" ]; then
> pqinsert -q ${QUEUELOC} -s ${SEQUENCE} -f "${THISFEEDTYPE}" -p "EMWIN 
> F/${THISFILE}" temp > /dev/null 2>&1
> fi
> rm -f ${eachFile} temp
> done
> rm -rf *
> done
> INTERNALCOUNT=`expr "${INTERNALCOUNT}" + 1`
> done

That's quite a script!

Would you please do me a favor and try running it manually. Edit out the "> 
dev/null 2>&1" strings at the end of the pqinsert(1) commands and change 
"pqinsert" to "pqinsert -vl-" so that we can see any error messages. Then 
execute the script using the command "sh -x" so that the shell prints every 
command that it executes.

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: FHI-907414
Department: Support LDM
Priority: Normal
Status: Closed