This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
Kelvin, > Quick question: is all of this stuff done in serial mode, i.e., > does the LDM system allow for overlapping in things like > compression, transmission, or is everything done in sequence? I > realize that the data are collected in sequence, but my point is > that, if the processing takes longer than the transmission, then a > bottleneck exists within the software that might be eliminated by > overlapping various functions. I'm speaking with zero knowledge of > how LDM works in this regard, so forgive me if I'm totally out to > lunch.... You can certainly be compressing one or more products while transmitting others. The LDM uses a separate process for sending data to each downstream LDM, and that process shares the CPU or CPUs with all the other processes running on the machine, including a process compressing a data product. So a compression process can be running concurrently with a transmission processes. However, the LDM cannot start sending a partial product before it has been completely stored in the in-memory product queue. So you cannot overlap compression and sending the same data product. The LDM protocol requires the size of the product right up front, so you cannot send a product before its complete size is known, which would be the case if it was still being compressed. Similarly, an LDM cannot begin relaying a product from an upstream site to a downstream site until the product is completely received from the upstream site and stored in the product queue. In this case it's not a matter of not knowing its size, but preventing deadlocks and other similar problems ... --Russ