[om-list] Re: MTShell and pipes

[Mark Butler]

Tom,

  OS level pipes are FIFO.  If you want to fork in multiple branches you need to write code for that purpose, which whould be very easy with a simple pipe abstraction layer.  If you want some pipe destinations to read far beyond where others current are, you will have to implement buffering in the pipe branching layer.

Pipes are not random access devices, so if you really need a single client to have random access to the same stream of data, you will need to use temporary files or something equivalent.

Assuming you have lots of memory, modern operating systems (particularly Linux) can exchange large messages via temporary files very rapidly.  Files are the usual choice for multi-process random access to data.  The only other alternative generally amounts to writing your own filesystem in shared memory, which is not trivial.

The same thing goes for writing a compiler - I would always write an interpreter first.  Depending on the semantics of your language, compiling might not help very much.  For example, many languages (Java, Perl, Smalltalk, LISP) make extremely heavy use of dynamic memory allocation.  Compiling  helps, but doesn't make nearly as big a difference as with a language like C that forces / allows programmers to worry about very low level optimization details, in this case stack vs. dynamic allocation.

Case in point:  A Java Vector of size N requires at least N+1 dynamic memory allocations, which by their very nature are *slow*. The low level pointer orientation of C++ allows you to allocate arrays of objects eith one allocation, about two orders of magnitude faster.  Java is *much* friendlier to the programmer, but compiling it doesn't help that much either, unless the critical code only deals with simple arrays of floats and integers. 

- Mark