[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[LDM #VLF-828955]: ldm-6.11.5 pqinsert problems
- Subject: [LDM #VLF-828955]: ldm-6.11.5 pqinsert problems
- Date: Tue, 11 Jun 2013 16:14:46 -0600
Julie,
It's not obvious what the problem is.
It would help greatly if I could log onto the system in question as the LDM
user. Is that possible?
The log files that you sent indicate the the build when OK. Is there anything
untoward in the log file of the running LDM system?
> I need to be able to send data via the ldm to NOAA. I currently have a
> test environment set up to send data from one machine to another at
> NCAR. I am having problems inserting things into the queue. I don't
> know if this is related or not, but I also get a segmentation fault when
> I run "pqinsert -h". I am using ldm-6.11.5 on a Debian (6.0.7) machine.
>
>
> I have been trying to insert multiple different types of netcdf files
> via a Python script that gets called when new data is available. From
> those log files, it appeared that the data was being inserted properly:
>
> pqinsert INFO: 52444 20130610225933.581 EXP 000
> pireps.20130610.22.1.combined.nc
> pqinsert INFO: 145936 20130610225934.529 EXP 000
> pireps.20130610.21.1.combined.nc
>
> However, I used "notifyme", "ldmadmin watch -f EXP", and "pqmon" and all
> indicated that nothing was being inserted in the queue.
>
>
> I tried to insert a file manually but received a message saying that the
> file was already in the queue (it was inserted by my script when that
> new data arrived)
>
> puffin%/home/ldm/bin/pqinsert -v -p pireps.20130610.22.1.combined.nc -q
> /home/ldm/data/ldm.pq -f EXP pireps.20130610.22.1.combined.nc
>
> pqinsert ERROR: Product already in queue:
> 171647304352d7c99992282d5a937e8d 52444 20130610230427.551 EXP 000
> pireps.20130610.22.1.combined.nc
>
>
> I turned off my InputWatcher and manually added another file of the same
> type:
>
> puffin% /home/ldm/bin/pqinsert -v -p pireps.20130610.23.1.combined.nc -q
> /home/ldm/data/ldm.pq -f EXP pireps.20130610.23.1.combined.nc
> pqinsert INFO: 17100 20130610232347.397 EXP 000
> pireps.20130610.23.1.combined.nc
>
> However, I used "notifyme", "ldmadmin watch -f EXP", and "pqmon" and all
> indicated that nothing had been inserted in the queue.
>
>
> I tried to insert another file manually:
>
> puffin% pqinsert -vx -f EXP cron.out
> Jun 10 22:18:09 DEBUG: Mapping 514121728
> pqinsert INFO: 26beef895707015ac77ba6bfd8364d97 1186
> 20130610221809.829 EXP 000 cron.out
>
> And according to "notifyme -v -l -", "ldmadmin watch -f EXP", and
> "pqmon" this worked.
>
> notifyme -v -l -:
> Jun 10 22:16:22 notifyme[3594] NOTE: Starting Up: localhost:
> 20130610221622.822 TS_ENDT {{ANY, ".*"}}
> Jun 10 22:16:22 notifyme[3594] NOTE: LDM-5 desired product-class:
> 20130610221622.822 TS_ENDT {{ANY, ".*"}}
> Jun 10 22:16:22 notifyme[3594] INFO: Resolving localhost to 127.0.0.1
> took 0.000673 seconds
> Jun 10 22:16:22 notifyme[3594] NOTE: NOTIFYME(localhost): OK
> Jun 10 22:18:23 notifyme[3594] INFO: 1186 20130610221809.829 EXP
> 000 cron.out
> Jun 10 22:23:48 notifyme[3594] NOTE: LDM-5 desired product-class:
> 20130610221809.829 TS_ENDT {{ANY, ".*"}}
>
> ldmadmin watch -f EXP:
> Jun 10 22:18:10 pqutil INFO: 1186 20130610221809.829 EXP 000
> cron.out
>
> pqmon:
> Jun 10 23:24:54 pqmon NOTE: Starting Up (7461)
> Jun 10 23:24:54 pqmon NOTE: nprods nfree nempty nbytes maxprods
> maxfree minempty maxext age
> Jun 10 23:24:54 pqmon NOTE: 1 1 122068 1264 1
> 1 122068 500001552 4004
> Jun 10 23:24:54 pqmon NOTE: Exiting
>
>
> I ran "pqinsert -h" and got:
> pqinsert: invalid option -- 'h'
> Segmentation fault
>
> I ran "pqinsert -h" on a different machine with ldm-6.6.5 and got the
> usage statement.
>
> This morning, I reinstalled ldm-6.11.5 to see if that would help with
> the segmentation fault, but it did not. I still received a segmentation
> fault.
>
> I just tried inserting three files manually:
>
> puffin% /home/ldm/bin/pqinsert -v -p pireps.20130611.00.1.combined.nc -q
> /home/ldm/data/ldm.pq -f EXP pireps.20130611.00.1.combined.nc
> pqinsert INFO: 157216 20130611173634.568 EXP 000
> pireps.20130611.00.1.combined.nc
>
> puffin% pqinsert -vx -f EXP cron.out
> Jun 11 17:36:50 DEBUG: Mapping 514121728
> pqinsert INFO: 2338ca72d7b5b04fb4b978a89e6033ec 1190
> 20130611173650.234 EXP 000 cron.out
>
> puffin% /home/ldm/bin/pqinsert -v -p edr_hourly.20130611.00.nc -q
> /home/ldm/data/ldm.pq -f EXP edr_hourly.20130611.00.nc
> pqinsert INFO: 970632 20130611173743.242 EXP 000
> edr_hourly.20130611.00.nc
>
>
> The size of the files are 154K, 1.2K, and 948K, respectively.
>
>
> The only one that shows up is cron.out:
>
> notifyme -v -l -:
> Jun 11 17:35:45 notifyme[9897] NOTE: Starting Up: localhost:
> 20130611173545.454 TS_ENDT {{ANY, ".*"}}
> Jun 11 17:35:45 notifyme[9897] NOTE: LDM-5 desired product-class:
> 20130611173545.454 TS_ENDT {{ANY, ".*"}}
> Jun 11 17:35:45 notifyme[9897] INFO: Resolving localhost to 127.0.0.1
> took 0.000734 seconds
> Jun 11 17:35:45 notifyme[9897] NOTE: NOTIFYME(localhost): OK
> Jun 11 17:37:16 notifyme[9897] INFO: 1190 20130611173650.234 EXP
> 000 cron.out
>
> ldmadmin watch -f EXP:
> Jun 11 17:36:51 pqutil INFO: 1190 20130611173650.234 EXP 000
> cron.out
>
> pqmon:
> Jun 11 17:41:48 pqmon NOTE: Starting Up (10056)
> Jun 11 17:41:48 pqmon NOTE: nprods nfree nempty nbytes maxprods
> maxfree minempty maxext age
> Jun 11 17:41:48 pqmon NOTE: 1 1 122068 1264 1
> 1 122068 500001552 298
> Jun 11 17:41:48 pqmon NOTE: Exiting
>
>
> Can you please help me figure out why these other files are not getting
> inserted into the queue? I'm not sure how to proceed. Just FYI, I set
> this up last year to transfer these same file types with ldm-6.5.1 on a
> Debian (6.0.4) machine and it worked fine.
>
> Thanks,
> Julie
> --
> Julie Prestopnik
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: address@hidden
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: VLF-828955
Department: Support LDM
Priority: Normal
Status: Closed