This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Hi Daryl, > To: address@hidden > From: Daryl Herzmann <address@hidden> > Subject: python + LDM > Organization: UCAR/Unidata > Keywords: 200506231613.j5NGDeZu022545 The above message contained the following: > I mucked up a small peice of python code that bridges pqact and python via > Twisted Python. Basically it allows for non-blocking processing of > products while the STDIN pipe remains open. Currently, I am invoking > python for each product received, which is not good (Bad daryl!) It's my understanding that a Python decoder will run about three orders of magnitude slower (or, eqivalently, use about three orders of magnitude more CPU) than the equivalent C decoder. > I wanted to run this by you folks, before I sent a more polished version > to the ldm-users list. Perhaps somebody else has already done this and I > suspect better! Or there are some python hacks there that can comment on > this code... It appears that your code would only be good for textual products because it looks for an STX control-character -- and binary products would have plenty of those. > There are many reasons why this is interesting. The main one being the > non-blocking nature and the ability for a long running python process that > is not reinvoked constantly.... I'll run it past our Python expert. > later, > daryl > > #=cut==== cat bridgefe.py ===== > > #!/usr/bin/env python > > import ldmbridge > from twisted.internet import reactor > > class myProductIngestor(ldmbridge.LDMProductReceiver): > > # Here is where you would write your processing code! > def processData(self, data): > print "I would like to do something here" > > fact = ldmbridge.LDMProductFactory( myProductIngestor() ) > > reactor.run() > > #=cut==== cat ldmbridge.py ===== > > import sys, re > > from twisted.internet import stdio > from twisted.protocols import basic > from twisted.internet import reactor > > class LDMProductReceiver(basic.LineReceiver): > delimiter = '\n' > productDelimiter = '\003' > > def __init__(self): > self.productBuffer = "" > self.setRawMode() > > def rawDataReceived(self, data): > pos = data.find(self.productDelimiter) > if (pos == -1): > self.productBuffer += data > else: > self.productBuffer += data[:pos] > reactor.callLater(0, self.processData, self.productBuffer) > self.productBuffer = data[pos:] > > def connectionLost(self, reason): > reactor.stop() > > def processData(self, data): > raise NotImplementedError > > class LDMProductFactory( stdio.StandardIO ): > > def __init__(self, protocol): > stdio.StandardIO.__init__(self, protocol) > > def connectionLost(self, reason): > reactor.stop() > > Regards, Steve Emmerson > NOTE: All email exchanges with Unidata User Support are recorded in the > Unidata inquiry tracking system and then made publicly available > through the web. If you do not want to have your interactions made > available in this way, you must let us know in each email you send to us.