PROJECT SUMMARY

Title of Effort: Application-Specific Operating Systems for High Performance Computing

The SPIN operating system uses three new technologies to achieve its goals:

1. (1) extensibility mechanisms appropriate for complex systems software,

2. (2) safe programming language constructs for operating systems, and

3. (3) dynamic compilation for operating systems and applications.

The first of these technologies allows applications to install their extensions into a running operating system. The second ensures that the extensions do not violate the integrity of the base operating system or other applications running at the same time. The third makes it feasible, from the standpoint of code quality, to defer the binding of system modules and extensions until runtime. While these technologies are being developed in the context of the SPIN operating system, they will be transferable to other domains, where an architecture for safely and efficiently extending a base set of services is required.

Extensibility in SPIN is achieved through the use of an event-based invocation mechanism. Low-level kernel primitives and user-installed extensions raise events to signal important system transitions; kernel routines and user extensions register interest in and are subsequently invoked in response to these events. In this manner, the kernel primitives and the user extensions are decoupled from one another, enabling extensions to be added and removed as the kernel runs. The SPIN operating system includes a set of core operating systems services, such as virtual memory, networking, scheduling, and storage management, that have interfaces amenable to to an event-based extension model.

System extensions (and the operating system itself) are written in Modula-3, a modular programming language with a well-defined, type-safe subset. Extensions written in Modula-3 are guaranteed to conform to the interfaces against which they have been compiled, and are thus isolated from one another and from the kernel. With this guarantee, it becomes possible to link and run safely application extensions in the kernel. Part of this project involves extending Modula-3 so that the compiler may support a broader set of safety mechanisms.

SPIN applies dynamic compilation techniques to provide efficient dispatching of the run-time installed extensions. By compiling at run-time, when the exact set of extensions registering interest in a particular event is known, fast versions of event dispatchers can be produced; when the set of extensions interested in an event changes, the event dispatcher is recompiled. In addition, the procedure boundaries between kernel and extension code can be broken down through run-time inline-expansion. By making event dispatching cheap, events and spindles can be used aggressively to support highly flexible systems.

• Extensible threads, scheduling, virtual memory, and networking have been implemented in the SPIN kernel. These services enable the kernel to support client extensions and user-level programs.

• A performance-scalable video server has been implemented as an extension to the SPIN kernel. This server demonstrates that service extensions are feasible and can result in good performance.

• A stand-alone, native version of the SPIN kernel now exists. The first version of the system was implemented as an appendage to the MACH operating system kernel, which resulted in unnecessary kernel complexity and size. The current kernel, which relies on a large number of vendor-specific device drivers, is less than 1/3 the size of the initial version, and builds almost 10 times faster. The system runs on the entire Digital ALPHA line of computers.

• The Cove memory-safe dialect of C was designed, implemented, and evaluated. Based on the evaluation, the technology was transferred to Modula-3, and compiler and runtime extensions to Modula-3 were implemented. These extensions allow the kernel and extensions to use Modula-3 with increased safety assurances and/or performance benefits.

• A COFF-compatible dynamic linker was implemented in the SPIN kernel. The linker enables application-specific event handlers to be incorporated into the kernel at run-time.

• An initial interface between the static and dynamic compilers was designed to support fast, effective dynamic compilation. A prototype of the dynamic compiler that supports the interface was used to demonstrate that applying dynamic compilation to event dispatching in the SPIN kernel brings performance benefits.

• GCC was evaluated as a potential infrastructure for the static compiler; the evaluation proved negative. We are pursuing two alternative replacements: the Vortex and Multiflow optimizing compilers. For the former we implemented a Modula-3 front-end on top of the Vortex optimizing backend; for the latter we redesigned and extended the static/dynamic compiler interface to incorporate full loop unrolling and are implementing the new static and dynamic compilers.

• A version of the DEC OSF/1 UNIX Operating System will be implemented as an extension to the SPIN extensible kernel. This server will support an industry-standard API through a set of decomposed system services.

• A database server and a WORLD-WIDE-WEB HTTP Server, which rely on SPIN's extensible virtual memory, storage and network primitives, will be implemented. These servers will demonstrate the effectiveness of tight integration between system services and application services. We will provide INTERNET access to the SPIN web page through our SPIN HTTP Server.

• A distributed shared memory system that uses virtual memory protection faults to determine data inconsistencies will be implemented using the SPIN extensible system. This system will show the benefits of integrating memory management and networking in an application-specific fashion.

• The implementation of our memory-safe, systems-programming-friendly dialect of Modula-3 will be completed.

• Automatic dynamic compilation will be incorporated into the SPIN kernel. Its effectiveness at optimizing event dispatching will be evaluated.

• Microsoft Research. Our network kernel debugging interface and debugger are being used by the Microsoft Research Lab in an experimental kernel they are building. Contact: Richard Draves (rpd@microsoft.com).

• Carnegie Mellon University. An alpha-version of the SPIN kernel has been transferred to the FOX Project. They are currently implementing our network protocol architecture in terms of FOXNET. Contact: Prof. Robert Harper (rwh@cs.cmu.edu).