points by DonHopkins 6 years ago

Owen Densmore recounted John Warnock's idea that PostScript was actually a "linguistic motherboard".

(This was part of a discussion with Owen about NeFS, which was a proposal for the next version of NFS to run a PostScript interpreter in the kernel. More about that here:)

https://news.ycombinator.com/item?id=17077721

Owen Densmore's discussion of John Warnock's "Linguistic Motherboard" idea for PostScript:

https://donhopkins.com/home/archive/NeWS/linguistic-motherbo...

    Date: Tue, 20 Feb 90 15:20:52 PST
    From: owen@Sun.COM (Owen Densmore)
    To: don@cs.UMD.EDU
    Subject: Re:  NeFS

    > They changed the meaning of some of the standard PostScript operators,
    > like read and write, which I don't think was a good idea, for several
    > reasons... They should have used different names, or at least made ..

    Agreed.  And I DO see reasons for the old operators.  They could
    be optimized as a local cache for NeFS to use in its own calcs.

    > Basically, NeFS is a *particular* application of an abstraction of
    > NeWS. The abstract idea is that of having a server with a dynamically
    > extensible interpreter as an interface to whatever library or resource
    > you want to make available over the network (let's call it a generic
    > Ne* server).

    Very true.  This has been particularly difficult for me to get across
    to others here at Sun.  I recently wrote it up for Steve MacKay and
    include it at the end of the message.

    > It's not clear to me if NeFS supports multiple light weight PostScript
    > processes like NeWS.

    I asked Brent about this, and he agreed that it's an issue.  Brent
    has been talking to a guy here who's interested in re-writing the
    NeWS interpreter to be much easier to program and debug.  I'd love
    to see them come up with a NeWS Core that could be used as a generic
    NetWare core.

    I think you should send  your comments off to nfs3 & see what happens!
    I agree with most of your points.

    Owen

    Here's the memo I consed up for MacKay:

======================================================================

Window System? ..NeWS ain' no stinkin' Window System!

-or-

Swiss Army NeWS: A Programmable Network Facility

======================================================================

Introduction

NeWS is difficult to understand simply because it is not just a window system. It is a "Swiss Army Knife" containing several components, some of which contribute to its use as a window system, others which provide the networking facilities for implementing the client-server model, all embedded in a programmable substrate allowing extremely flexible and creative combination of these elements.

During the initial implementation phase of the Macintosh LaserWriter software, I temporarily transfered from Apple to Adobe working closely with John Warnock and other Adobe engineers. At lunch one day, I asked: "John, what do you plan to do after LaserWriter?" His answer was interesting:

        PostScript is a linguistic "mother board", which has "slots"
        for several "cards".  The first card we (Adobe) built was a
        graphics card.  We're considering other cards.  In particular,
        we've thought about other network services, such as a file
        server card.

He went on to say how a programmable network was really his goal, and that the printing work was just the first component. His mentioning using PostScript for a file server is particularly interesting: Sun's next version of NFS is going to use PostScript with file extentions as the client-server protocol!

This paper explores NeWS in this light: as a Programmable Network Facility, a major part of Sun's future networking strategy.

NeWS Networking Components

NeWS has realized John's notion of a programmable network facility. It has populated the initial PostScript mother board with several cards vital to netwoking:

    -Device Independent Graphics Primitives: The key importance of the
     PostScript graphics model for networking is that it is device
     independent, able to represent with one file the same drawing on
     *all* the network hosts.  The Folio fonts contribute to this: they
     can be rendered at a given point size on all screens, regardless
     of resolution.

    -Host Independent Interpreter: PostScript is an interpreted language
     that runs on all Sun workstations without recompilation.  The
     significance of this for networking is that small code fragments
     can run on any host without any knowledge of the host's
     architecture.  Thus one host sending a code fragment to another
     host need not be concerned whether that host is a Sun-3, Sun-4, or
     even Macintosh!  This extends to client programming languages as
     well: all client languages have the same interface to the host.
     As an illustration of this point, both C and Lisp clients use
     the TNT toolkit with no special interfaces for each language.

    -Network Socket Primitives: NeWS has added primitives for listening
     for connection requests, for socket files, for host name and
     internet number management, and for reading and writing on the
     network connection.  In fact, the NeWS "server" code is not
     written in C, it is written in PostScript.  The initial version,
     and the one I run on my own machine today, is 10 lines of code!

    -Light Weight Processes (LWP): A network host has many independent
     activities taking place at one time.  Each of these uses a
     separate LWP.  These activities need not be window oriented,
     and in fact most are not.  Consider WUE alarms, for example.
     A LWP could be created on each WUE NetStation whose only task
     was to listen for activities on other WUE NetStations and to
     cause a WUE Alarm to occur when an interesting event occurs.

    -Garbage Collection: When data is allocated within NeWS, it is
     returned to the free memory pool when no process references it.
     This is needed in a multiprocessing network host because separate
     LWPs can reference the same shared data.  (No process can safely
     free data itself, because other processes might reference it.)
     Garbage collection manages this problem for the processes, and
     thus is an important Network facility.

    -Programmable Events: NeWS has a flexible, very easy to use
     event management scheme.  Processes fill out templates of events
     they are interested in.  When these are matched by events occuring
     within the system, they are delivered to the interested process.
     Events are *far* more useful than simple keyboard and mouse events.
     They can be artificially manufactured for any sort of inter-LWP
     communication.  For the previous WUE Alarm, for example, the WUE
     "Listening" process would communicate to WUE-Friendly applications
     by filling out a WUEAlarmEvent and sending it.  The interested
     applications would have filled out a template which would "catch"
     the event.

    -Programmable Error Handling: NeWS uses the standard PostScript
     exception handling, extended to work within the context of LWPs.
     Thus an error in one LWP is isolated to that process only.
     Network programs can "wrap" incomming code inside of the special
     "stopped" error handler which will catch errors occuring inside
     the code fragment.  This is vital to safe network programming.

What IS NeWS Anyway!

"Well, I'm glad NeWS has all these useful parts, but just what IS NeWS anyway; how do I get at and use all this stuff."

Well you may ask! It does seem confusing.

Formally, NeWS is simply a Unix command, just like "ls" and "cd". Its syntax is:

    xnews [PostScript-code]

i.e. it is a command that executes PostScript code fragment. If the xnews command is given without an explicit argument, it defaults to executing "(NeWS/init.ps) run", which simply looks for the file "init.ps" in the "$OPENWINHOME/etc/NeWS" directory. It is this file that uses the NeWS components to setup the X11/NeWS window system.

As a silly example of handing in your own commands to NeWS, we can do this:

        bigmac% xnews "1 1 add ="
        2

This simply tells xnews to add 1 & 1 and print the results. Oddly enough, I have written a somewhat more complex script to fill out my Traval Advance forms, adding up all the rows and columns and printing out a version of the form!

Now that we've gotten though the fundamentals, the rest of the paper will look at individual examples of using NeWS in novel ways. Although we present programs to illustrate how things are done, the reader may skip over them, the accompaning text will explain what the program does.

The Ten-Line Server

As mentioned above, "xnews" is simply a Unix command that executes a PostScript program, defaulting to "(NeWS/init.ps) run". Init.ps performs several initialization tasks and executes the commands in various other files. When its done with all of that, it defines and executes a small procedure called "server". This program uses the NeWS networking facilities to build a LWP which listens for connection requests from "client" programs. Whenever a request is heard, the LWP awakens and creates and initializes a new LWP just for the client's usage.

Here's the ten line version of the server. This actually works (I've run it for quite some time!), and is explained in Chapter 10, Networking NeWS, of the Unix Networking book published last year.

    /server { % - => -; create the NeWS client-server listner process
        {   clear newprocessgroup           % -;    Clean stack, init process group
            (%socketl2000) (r) file         % soc;  Open socket for listening
            {   dup acceptconnection        % soc f;Loop forever creating clients
                {   exch pop 200 dict begin % f;    Create userdict
                    initmatrix newprocessgroup % f; Init graphics & process
                    cvx exec                % -;    Execute the connection!
                    currentprocess killprocessgroup%Cleanup client & children
                } fork pop pop              % soc;  Clear process & file
            } loop                          % soc;  Loop forever creating clients
        } fork pop                          % -;    Fork server & toss process
    } def

The significance of this is that we can just as easily build other network services with NeWS, using code much the same as the above. NeWSPrint, for example, uses a similar PostScript program to provide network printing facilities. NeWSPrint does not require a special, modified version of xnews; it simply uses the standard OpenWindows product.

ps2bits: A PostScript to Rasterfile Program

A second use of running xnews as a Unix filter is illustrated by the ps2bits probram. This Bourne shell script reads a PostScript file on its "standard in", and emits the resulting Sun raster file on its "standard out". The shell script does this by having xnews:

-Read in a few standard PostScript initialization files,

-Initialize the PostScript environment (& fix a Folio bug!)

-Build an 8.5 by 11, color, 72 dpi resolution "page" canvas

-Execute the PostScript stdin file to draw that page

-Write the resulting image to stdout as a raster file

Here is the program:

    #! /bin/sh
    xnews "
        (NeWS/pack.ps)      (r) file cvx exec
        (NeWS/basics.ps)    (r) file cvx exec
        (NeWS/redbook.ps)   (r) file cvx exec
        500 dict begin false setautobind () stringwidth pop pop
        612 792 8 [1 0 0 -1 0 792] null buildimage setcanvas
        (%stdin) run clippath (%stdout) (w) file writecanvas
    "

Note: The line with the "buildimage" command may be modified to convert the program to black & white, or to change the resolution to 300 dpi, or to change the page size. See Appendix A for a more general script.

Once the image is made, it can easily be looked at in SunView, XView, or NeWS using any of their raster file viewers. Here's a trivial viewer, "bitwin", using the TNT toolkit:

    #! /bin/sh
    psh <<BITS
        /win [ClassCanvas] [] framebuffer /new OpenLookBaseFrame send def
        {   clippath pathbbox scale pop pop
            (${1}) readcanvas imagecanvas
        } /setpaintproc /client win send send
        (${1}) /setlabel win send
        100 100 425 550 /reshape win send
        /activate win send
        /map win send
    BITS

This Bourne shell script takes a single command line argument naming the raster file. It uses the NeWS "PShell" to sent a short program to the running xnews server. The program will put the file name in its header, and will fill itself with the raster file image. Stretching the window will stretch the image.

To use these on the demo file "tiger.ps" which was initially created on the Mac using Adobe Illustrator, these two steps are taken:

    bigmac% ps2bits < $XNEWSHOME/demo/PostScript/tiger.ps > /tmp/bits
    bigmac% bitwin /tmp/bits &

The first command converts the tiger drawing into a raster file. The second creates a window for viewing the raster file.

Although the complexity has been fairly minor in the previous examples, we've illustrated some powerful capabilities:

    -We can easily use NeWS to provide a simple Unix facility for
     converting PostScript drawings into Sun raster files.  The
     PostScript may come from any of the many widely available sources,
     even the Macintosh.

    -These raster files can be used by any toolkit, including SunView.
     They can even be used by other raster devices such as printers.

    -Using the networking primitives used in the 10-line server, we
     could easily build a similar server for providing raster files
     to anyone on the network.

Using these NeWS facilities in similar creative ways is exactly what NeWSPrint does. The documentation folks use a similar "filter" to strip out all the general header information in Mac PostScript files to gain an average 8-1 compression, vastly reducing their disk storage requirements. A homework assignment in our Usenix NeWS tutorial is to create a generalized Image Server for the network using these facilities.

WUE Alarms with Programmable Events

WUE has the notion of "alarms" which can be used to alert applications of changes in status of network objects. Applications can use this for anything from "hot links" to file status alarms. One of the WUE examples is on file status: I want to know when the monthy report reaches 90% completion. Hot links are similar: I want this graph to change whenever the associated spreadsheet changes.

Here is one way to implement WUE file alarms using NeWS programmable events:

    -WUE provides a Property Sheet which includes a file completion
     field holding a percentage quantity.

    -When the Property Sheet is completed, it creates and sends a
     WUEStatusEvent for the given file.  The network name of the object
     is stored in the event's Action field.  The event includes
     auxillary information about the file in the ClientData of the
     event.

    -Applications following WUE object status changes use the NeWS
     "expressinterest" primitive to catch all WUEStatusEvent's for the
     desired file.  It then looks at the ClientData to determine
     whether or not further action should be taken.

The WUE Property Sheet would build and send the event this way:

    createevent dup begin
        /Name   /WUEStatusEvent def
        /Action (/home/project/status/january) def
        /ClientData 1 dict dup begin
            /Completion .70 def
        end def
    end sendevent

The application builds an interest for its TNT event manager to handle. It does so by building a matching event template for WUEStatusEvent and the /home/project/status/january filename. It also includes a "callback" procedure which decides what to do with the completion information. In this case, we want the alarm to occur when the file is more than 90% complete:

    createevent dup begin
        /Name 1 dict dup begin
            /WUEStatusEvent { % event => -; check if complete yet
                /ClientData get /Completion get .90 ge {
                    AlertApplication
                } if
            } def
        end def
        /Action (/home/project/status/january) def
    end MyEventMgr expressinterest

This results in the AlertApplication procedure being called when the /home/project/status/january report is within 10% of completion.

Puny WOF (Workgroup Object Facility)

The above alerting works only for applications running on the same workstation. This is because the event that is sent is only visible to interests expressed on the same workstation. We can easily extend our model to include sending the event to many network workstations by using the same techniques as the 10-line server:

    -WUE friendly workstations start a WUE event listening service when
     they start up NeWS.  This uses exactly the same techniques as the
     10-line server, but using a different socket file.  They also
     regester themselves with a centralized list of WUE friendly
     workstations.

    -The WUE event listening service starts a second process which
     expresses interest in all local WUE events.  When any WUE
     event is caught, it is "forwarded" to each of the workstations
     listed in the central list.  It does this by opening a socket
     file to each workstation, and sending a small program down
     the socket which sends the WUE event on the remote machine.

    -The Property Sheet sends the WUEStatusEvent exactly as before.
     Local applications running on the same machine operate exactly as
     before.  In addition, however, the local WUE event forwarder
     causes the same event to be sent on each of the regestered
     workstations.  Applications on those machines would now catch the
     forwarded event exactly as if it were local.  The application need
     not be modifed!

This basically implements a Puny WOF (Workgroup Object Facility). And it can be done very easily indeed! Admittedly that this is quite puny and many details about robustness etc. have to be managed. On the other hand, using xnews as a WOF prototyper is extremely appealing. (I plan to prototype a versions of Puny WOF as soon as I get a second host set up and running in my office.)

Multi-Media: NeWS Device Control

Multi-media devices can be managed by NeWS, using programmable events to interface to applications. This provides a logical division between the device "driver", which may be implemented in any of several ways, and the application API to the device, which is via NeWS events.

As an example, consider a graphics tablet input device with an RS232 interface. This device emits x,y data in a well-known format to the serial device when the tablet's stylus moves. As an initial implementation, the driver half of the device was written in PostScript as a process which opens and reads the file /dev/tty. When there is no data on the device, the process blocks, pausing until data arives for the device. When the data arives, the read returns with the device data. This is packaged into an event as in the WUE Alarm example.

Typically the device data needs processing that is better suited to C programming. The solution is to rewrite the device manipulation as a NeWS client using either CPS or the TNT Wire Service. The application interface, however, continues to be the same event interface. The improved performance incurrs no change in the application interface. The notion of this variety of client "daemon" has been used in TNT for providing shared text service among applicaions. A voice interface to EMacs was also made in AD using the same technique.

Another significant advantage to this division of labor into a device layer using an event interface to applications is that it gracefully evolves into sharing the single device among multiple applications. The device is temporarily "owned" by one application, and is given to another application using an agreed upon protocol. Two common styles are used:

    -The keyboard uses a "focus" model where certain UI events not
     associated with the keyboard determine which application owns it.

    -The mouse is typically owned by the application in which it was
     last "clicked".

In the case of the graphics tablet, applications wanting to use the tablet would express interest in the tablet's button being clicked within it. From then on, that application will receive the motion change events for the tablet.

Although device ownership may seem like a trivial problem, fairly complex systems are typically needed to solve it for a given device. The VOX server from Olivetti for voice is used for just this kind of thing: negociation between applications for ownership of the voice device. Because VOX is not integrated into a more general event system, however, it cannot manage the user interface for which application currently owns it. It can only provide the regestry. When OpenLook defines the "focus" model for ownership of the voice device, a NeWS oriented voice service will much more easily integrate into the rest of the system.

Just as WUE Alarms had a network component, certain multi-media devices will be expensive enough to warrent sharing on the network. Scanners and fax machines will be attached to individual machnes, but we may want them to be temporarily owned by other workstations. The same "server" technology used by the 10-line server and the Puny WOF can also be used here.

Summary and Vision

The interesting software architectures of the next decade will leverage network technologies. We will see WUE become the Net Finder, replacing the simplistic Mac Finder in the network intensive workstation world. WUE will manage network resources among WUE-Friendly NetStations, workstations with advanced, easily configured network software platforms. NeWS is such a network tool: a programmable network facility. Another is the newly proposed NFS which uses file server extended PostScript as its client-server language.

We hear far too many NeWS vs. X arguments based on window and UI toolkit arguments alone. These, in my opinion, miss the main strengths of NeWS. It is a programmable network facility which can be used for many purposes. It can prototype new network services. It can be a universal device controler. It can be used to integrate multiple hosts. All these elements also need to be evaluated when considering NeWS.

References

[1] PostScript Language Reference Manual, Adobe Systems, Addison Wesley, 1985

[2] The NeWS Book, Gosling & Rosenthal, Springer-Verlag, 1989

[3] Unix Networking-ch 10: Networking NeWS, Owen Densmore, Hayden Books, 1989

[4] NFS-3 Design Document, Brent Calaghan, Sun internal document, 1990

Appendix A: A more general ps2bits shell script.

    #! /bin/sh
    # Parse cmd line args: ps2bits [-size W H] [-dpi X Y] [-color]
    X=72; Y=72; W=8.5; H=11; Z=1
    while [ ${#} -gt 0 ]; do
        case "${1}" in
            -dpi)   shift; X=${1}; shift; Y=${1} ;;
            -size)  shift; W=${1}; shift; H=${1} ;;
            -color) Z=8 ;;
        esac; shift
    done
    # Run NeWS as Unix filter. Note use of Bourne shell variables!
    xnews "
        (NeWS/pack.ps)      (r) file cvx exec
        (NeWS/basics.ps)    (r) file cvx exec
        (NeWS/redbook.ps)   (r) file cvx exec
        500 dict begin false setautobind () stringwidth pop pop
        $X $W mul $Y $H mul $Z [$X 72 div 0 0 $Y -72 div 0 $H] null buildimage
        setcanvas
        (%stdin) run clippath (%stdout) (w) file writecanvas
    "