Re: 20050615: problems creating the product queue for the LDM

Ryan,

The CONDUIT feed averages around 1GB per hour, and can exceed
2GB an hour. 

Did you send your ldmadmin entry for your pqsize entry to Steve E.?
What size queue are you trying to make? On solaris,
if you try to use 2000000000 as the queue size, you might have hit the
limit that could be mmapped due to the overhead of the number of slots
etc. On our solaris box, prior to 64 bit large file queues, I used
1950000000 as the queue size due to the above limitation. Of course, 
if you were a "0" off in typing that number, mmap would certainly fail 
as well. Older versions of perl had limitations with using abbreviations
such as 2G or 2000M.

It could take a minute or two to make a 2GB queue if you didn't use the 
"-f" flag for "ldmadmin mkqueue".

Steve Chiswell
Unidata User Support


On Wed, 2005-06-15 at 19:05, Ryan Lyman wrote:
> Steve,
> 
> thanks for your response...i've tried most of your suggestions either 
> previously or after your email, with little results.  the 'ipcs -a' 
> command showed no processes using mapped regions.  the machine is 
> running a solaris OS and the swap could handle the 
> product-queue...though there will be little room for anything else. 
> still, i am not sure the machine itself can handle the load as i've just 
> recently added the CONDUIT feed to this LDM.  i think i'll make a 
> recommendation for computer upgrade first, then reinstall the LDM 
> there...the timing is good.  thanks for your help.
> 
> aloha,
> ryan lyman
> 
> 
> Steve Emmerson wrote:
> 
> > Ryan,
> > 
> > 
> >>To: address@hidden
> >>From: Ryan Lyman <address@hidden>
> >>Subject: creating the product queue for the LDM
> >>Organization: University of Hawaii
> >>Keywords: 200506142135.j5ELZPZu013322 LDM LDM-6.3.0 ldm.pq
> > 
> > 
> > The above message contained the following:
> > 
> > 
> >>I've sort of inherited ldm administration job for the university of 
> >>hawaii meteorology department (vapor.soest.hawaii.edu) and i was having 
> >>problems creating the pqact.  i have a feeling the problem is related to 
> >>the hardware, as the computer is probably close to a decade old, but i 
> >>wanted to run it through you guys (they are running an older version of 
> >>the LDM (6.0.13), and i know i should upgrade it, but i am pushing a 
> >>computer upgrade first, which should take place in a couple of months.)
> >>
> >>at any rate, it appears the pqact was contaminated so i deleted using 
> >>the 'ldmadmin delqueue' command and tried to rebuild it using 'ldmadmin 
> >>mkqueue'.  it takes an absurdly long time to do the latter 
> > 
> > 
> > If you KNOW, a priori, that sufficient space exists for the
> > product-queue, then you can speed things up considerably by using the
> > "-f" (fast) option, i.e.,
> > 
> >     ldmadmin mkqueue -f
> > 
> > 
> >>then i get the following error message:
> >>
> >>Jun 14 20:59:12 pqcreate[5414]: mmap: 0 0 508051456: Resource 
> >>temporarily unavailable
> >>pqcreate: create "/home/vapor/ldm/idd/data/ldm.pq" failed: Resource 
> >>temporarily unavailable
> >>Jun 14 20:59:13 UTC vapor.soest.hawaii.edu : make_pq: mkqueue failed
> > 
> > 
> > The above error is likely due to one of the following:
> > 
> >  *      EMFILE  The number of mapped regions would exceed an
> >  *              implementation-dependent limit (per process or per system).
> > 
> >  *      ENOMEM  MAP_FIXED was specified, and the range [*ptrp, *ptrp + 
> > extent)
> >  *              exceeds that allowed for the address space of a process; or 
> > if
> >  *              MAP_FIXED was not specified and there is insufficient room 
> > in
> >  *              the address space to effect the mapping.
> > 
> > 
> >>i've done a few google searches and dug through the unidata site, with 
> >>little results.  any clue to what the problem could be?
> > 
> > 
> > It appears that your operating-system doesn't have the resources
> > necessary to run the LDM -- as currently configured and in the current
> > environment.
> > 
> > Are there any other programs running that use mapped regions?  If so,
> > can they be terminated?  The ipcs(1) utility might prove useful for 
> > discovering this.
> > 
> > Is the operating-system an older version of Linux?  If so, is the
> > swap-file large enough to hold the entire product-queue (on top of
> > everything else that is must contain)?  The top(1) utility might prove
> > useful for discovering this.
> > 
> > 
> >>thanks for your help.
> >>
> >>aloha,
> >>ryan lyman
> >>
> >>-- 
> >>
> >>-------------------------------------------------------
> >>  Ryan Lyman                   Mauna Kea Weather Center
> >>  Institute for Astronomy      Forecast Meteorologist
> >>  640 North A'ohoku Place      Tel: (808) 932-2323
> >>  Hilo, Hawaii 96720-2700      Fax: (808) 933-0737
> >>
> >>              http://mkwc.ifa.hawaii.edu/
> >>-------------------------------------------------------
> > 
> > 
> > Regards,
> > Steve Emmerson
> > 
> > 
> >>NOTE: All email exchanges with Unidata User Support are recorded in the
> >>Unidata inquiry tracking system and then made publicly available
> >>through the web.  If you do not want to have your interactions made
> >>available in this way, you must let us know in each email you send to us.