[om-list] Re: MTShell and pipes

[Tom and other Packers]

Mark

    Assuming I didn't care if one process (at one end of a branched pipe)
was a lot slower than another, I could still use pipes, right, (without
buffering)?

    Is there any way to speed up one process while it's being waited on by
the faster process, while the faster process is not really doing anything?

tomp

----- Original Message -----
From: "Mark Butler" <butlerm at middle.net>
To: "Tom and other Packers" <TomP at burgoyne.com>
Cc: "OM List" <om-list at onemodel.org>
Sent: Monday, November 12, 2001 8:51 AM
Subject: [om-list] Re: MTShell and pipes


Tom,

  OS level pipes are FIFO.  If you want to fork in multiple branches you
need to write code for that purpose, which whould be very easy with a simple
pipe abstraction layer.  If you want some pipe destinations to read far
beyond where others current are, you will have to implement buffering in the
pipe branching layer.

Pipes are not random access devices, so if you really need a single client
to have random access to the same stream of data, you will need to use
temporary files or something equivalent.

Assuming you have lots of memory, modern operating systems (particularly
Linux) can exchange large messages via temporary files very rapidly.  Files
are the usual choice for multi-process random access to data.  The only
other alternative generally amounts to writing your own filesystem in shared
memory, which is not trivial.

The same thing goes for writing a compiler - I would always write an
interpreter first.  Depending on the semantics of your language, compiling
might not help very much.  For example, many languages (Java, Perl,
Smalltalk, LISP) make extremely heavy use of dynamic memory allocation.
Compiling  helps, but doesn't make nearly as big a difference as with a
language like C that forces / allows programmers to worry about very low
level optimization details, in this case stack vs. dynamic allocation.

Case in point:  A Java Vector of size N requires at least N+1 dynamic memory
allocations, which by their very nature are *slow*. The low level pointer
orientation of C++ allows you to allocate arrays of objects eith one
allocation, about two orders of magnitude faster.  Java is *much* friendlier
to the programmer, but compiling it doesn't help that much either, unless
the critical code only deals with simple arrays of floats and integers.

- Mark

_______________________________________________
om-list mailing list
om-list at onemodel.org
http://www.pairlist.net/mailman/listinfo/om-list