What is Filespooler? Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication. Filespooler is a tool in the Unix tradition of “do one thing and do it well.” It is designed to integrate nicely with decoders (to handl...| www.complete.org
Asynchronous communication is communication between two endpoints that doesn’t have to happen in real time or near-real-time. Compared to synchronous communication, asynchronous communication lets you trade latency for reliability. Asynchronous communication is closely related to the concept of store-and-forward networking and delay/disruption-tolerant networking. In fact, most asynchronous implementations are also store-and-forward and delay-tolerant, so these terms are often used intercha...| www.complete.org
Since Filespooler is an ordered queue processor by default, it normally insists on a tight mapping between the sequence numbers in job files and execution order in a queue. This poses challenges when you want to send data from multiple sources to a single destination. There are two main strategies for doing this: On the receiving side, maintain a separate queue for each origin. This is quite easy to do, since Filespooler queues are very lightweight.| www.complete.org