This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Daryl Herzmann wrote: > > Hi Anne, > I hope you do not mind the direct email, but I have some ideas to > bounce off somebody. And since nobody here at Iowa State seems to know > much about LDM, I will pick your mind. > > I have been working on the Workstation ETA project for the Weather > Service here in DSM. As a part of the project, I have been running the > model on linux boxes here at Iowa State. So far so good, but I have > bigger plans in mind. > > One of my ideas is to distribute the WSETA to many machine here on > campus and at the weather service and have each machine run a varying > version of the model (physics, resolution, dt, etc...). The problem is > then passing all of the correct initialization data out to the workers and > then collecting the results. I looked at using NFS, SSH(scp) and rsync, > but I thought they where too hard to configure and not robust enough. So > then I started playing with LDM and have been very impressed so far. > > Currently, I have a worker machine pqinsert the gempak output file > and send it back to my analysis server. One of the questions I have is > how to use a logical naming convention. For instance, this morning's 00z > ETA run comes back from each worker as 00081700_wseta.gem, so I have an > overwritting problem. Then I figured that I would just have each machine > name their files different, so the server would store each on > seperate. This was ok, but I would rather do something different. I quess > my question is, when I use pqinsert the data comes accross as > > "Aug 17 17:21:51 pqutil: 2663936 20000817171922.950 EXP 000 > 00081200_abe_wseta.gem" > > I would like to be able to do something like the text products > come by as. > "Aug 17 17:11:26 pqutil: 907 20000817171047.997 IDS|DDPLUS 230 > WUCN10 CWUL 171700" > > meaning my ideal file would be > "Aug 17 17:21:51 pqutil: 2663936 20000817171922.950 EXP 000 00081200 > KFR abe > > So then I know that the file was created on the machine called abe > and using the Kain-Fritcsh Cumulus scheme. Then I could use the pqact file > to organize the returned data files a little more logically. Can this be > done with pqinsert without naming the file differently? > > hope this all makes a little bit of sense. Am I going about my > project correctly? Am I utilizing LDM capabilities? Any and all > suggestions will be greatly appreciated. > > I am still a novice with LDM, so if a "READ THE MAN PAGES!" is > required as an answer, please let me know :) > > Thanks much, > Daryl > > __ ,,,, > / \_|$$$$|__, > | ==========} > ,,,,,, Daryl Herzmann [ JR - MTEOR ] ,,,,,,,, +--+ JOHN |_ > !!!!!! address@hidden !!!!!!!! ___ | DEERE ,_/ > $$$$$$ www.public.iastate.edu/~akrherz $$$$$$$$/__|==-__-----_| > |||||| ICQ: 25560887 AOL: "Daryl Herz" ||||||||_______{}_____o_____ Hi Daryl, My apologies for the slow response. Somehow I overlooked this one. I think I understand the gist of your problem (although maybe not the details). However, the bad news is that pqinsert will only insert the product into the queue with the file name as the product identifier. If you want to use pqinsert you will need to ensure the file name is in the form you want it before you pqinsert it. You might be stuck with having each machine name their files differently. I'm going to speak with my coworker about your question. Perhaps there is an alternative, but I would expect that it wouldn't be simple. I will let you know what I find out. Otherwise, it seems like a fine use of the LDM. It would serve well as a way to gather all the files from a variety of hosts. pqact could certainly be useful in organizing the files. If I understand your problem correctly, if you simply appended the host name to each file (as it looked like you were doing) then you could group all the files with the same prefix together into a common directory. If I don't seem to be understanding this correctly please let me know. I'm happy to discuss this further. Anne -- *************************************************** Anne Wilson UCAR Unidata Program address@hidden P.O. Box 3000 Boulder, CO 80307 ---------------------------------------------------- Unidata WWW server http://www.unidata.ucar.edu/ ****************************************************