kernelthread.com

© Amit Singh. All Rights Reserved. Written in February 2004


What NeXT? The Mach Factor

Along with NeXT's operating system came the kernel: a version of Mach, which became the kernel foundation of Apple's future systems. It would be instructive to briefly discuss the origins and evolution of Mach.

Rochester's Intelligent Gateway

A group of researchers at the University of Rochester, New York, began development of an "intelligent" gateway system named RIG (Rochester's Intelligent Gateway) in 1975. RIG was meant to provide uniform access (via terminals, for example) to a variety of computing facilities, both local and remote. Local facilities could be locally connected disks, magnetic tapes, printers, plotters, batch processing or time-sharing computers, and so on. Remote facilities could be available through a network like the ARPANET. RIG's operating system, called "Aleph", ran on a Data General Eclipse minicomputer.

The Aleph kernel was structured around an interprocess communication (IPC) facility. RIG processes could send messages to each other, where the destination was specified by a process number and a port number. A port was an in-kernel message queue identified by the same { process #, port # } tuple. A process could have several ports defined within itself, each of which could be used to wait for a message to arrive on. A process X could "shadow" another process Y (X receives a copy of every message sent to Y), or X could "interpose" Y (X intercepts all messages sent to, or originating from Y).

RIG was killed a few years later due to several fundamental shortcomings in its design: a 2KB limit on the size of a message (because of the limited address space provided by the underlying hardware) and the resulting IPC inefficiency, no protection for the ports (port numbers were global, and any process could create and use them - thus, any process could send a message to any other process), no way to notify the failure of a process to another process that depended on it (without explicit registration of such dependencies), etc.

Accent

One of the people who worked on RIG was Richard Rashid. In 1979, he moved to Carnegie Mellon University, where he worked on Accent, a network operating system kernel. Like RIG, Accent also used IPC as the basic system structuring tool. However, Accent addressed RIG's shortcomings: ports now had "capabilities", and copy-on-write memory mapping was used to facilitate large message transfers. Messages could be sent to processes on another machine through an intermediary process. Accent had flexible and powerful virtual memory management, which was integrated with IPC and file storage.

Accent was developed to support two distributed computing projects, Spice (distributed personal computing) and DSN (a fault-tolerant distributed sensor network). Accent was the name of a product (a spice) sold by Accent Inc. The only ingredient of this product was monosodium glutamate (MSG). In computing, one often abbreviates "message" as "msg".

Within a few years, the future of Accent did not look promising as well. It needed a new hardware base, including support for multiprocessors and portability to various hardware. Accent also had difficulty supporting Unix software.

Mach

The sequel to Accent was called Mach.

When Mach was developed, UNIX had been around for over fifteen years. While the designers of Mach subscribed to the importance and usefulness of UNIX, they also noted that UNIX was no longer as simple or as easy to modify as it once was. Richard Rashid called the UNIX kernel a "dumping ground for virtually every new feature or facility."

Mach's design goals were largely a response to the inexorably increasing complexity of UNIX. These goals included:

Mach's implementation used 4.3BSD as the starting code base. Mach's designers had RIG and Accent to refer to when it came to creating message-passing kernels. DEC's Tenex Operating System provided some ideas for Mach's virtual memory subsystem. As Mach evolved, portions of the BSD kernel were replaced by their Mach equivalents, and various new components were added.

When it was published in 1986, the original Mach paper hailed it as "A New Kernel Foundation For UNIX Development". While not everybody saw or sees it that way, Mach went on to become a rather successful system. As far as Apple is concerned, the paper's title might as well have been "A NuKernel Foundation ..."

Richard Rashid recounts that after a series of unsuccessful naming attempts, he came up with the name MUCK (Multiprocessor Universal Communication Kernel). One of his colleagues, Dario Giuse, who was Italian, inadvertently pronounced MUCK as "Mach", and the name stuck.

Initially the Mach designers presented four basic abstractions in the kernel:

Another basic Mach abstraction is a memory object. A memory object can be thought of as a container for data (including file data) that is mapped into the address space of a task. Mach required a paged memory management unit (PMMU), and provided an excellent interface to the machine dependent MMU facilities via its pmap interface. The VM subsystem was designed to support large, sparse virtual address spaces, and was integrated with IPC. Unlike traditional UNIX, which implied a contiguous virtual memory space (with the heap and the stack growing towards each other), Mach allowed for sparse address spaces, where regions of memory could be allocated from anywhere in the address space. Memory could be shared read/write, in a structured manner, and copy-on-write (COW) was used both for sharing memory between tasks as well as to optimize copy operations. The generalized memory object abstraction allowed for "external" (out-of-kernel, user-space) memory pagers (to handle page faults and page-out data requests), that could reside even on another machine.

An important decision of CMU was to provide all Mach software free of distribution fees or royalties, with unrestrictive licensing.

Mach did not provide (nor was meant to provide) any I/O, networking, or file system capabilities by itself. It was meant as an operating system to create operating systems from. It was hoped that this would keep things simple, and promote portability of operating systems. The real operating system was supposed to run as a user-level Mach task. Release 2.0 of Mach, as well as the rather successful Release 2.5, had monolithic implementations — BSD and Mach resided in the same address space. Release 2.5 was used by the Open Software Foundation (OSF) as a kernel basis for their OSF/1 operating system. 2.x was also the basis of Mt. Xinu, NEXTSTEP, OPENSTEP, Omron LUNA/88k, Multimax (Encore) and so on.

The Mach 3 effort was started at CMU and continued by OSF. This was the first "true microkernel" version in the sense that BSD ran as a user space Mach task, with only fundamental features provided by the Mach kernel. Many operating systems were ported to the conceptual "virtual-machine" provided by the Mach API, and it is possible for several user-mode operating system interfaces to execute on top of Mach.

The Mach-US symmetric multi-server operating system has a set of servers providing generic system services and a library that is loaded into each user process. Examples of generic services include file management, terminal management, local interprocess communication, process management, networking, and device management. The emulation library, or multiple libraries, use the generic services to emulate different operating systems by intercepting system calls for each process and redirecting them to the appropriate emulation services. Mach emulators exist for various BSD versions, DOS, OSF/1, SVR4, VMS, OS/2, Macintosh operating system, HPUX, and so on.

Other changes and improvements in Mach 3.0 included:

Richard Rashid went on to become the head of Research at Microsoft, while another co-inventor, Avie Tevanian, went on to become the Chief Software Technology Officer at Apple.

MkLinux

Apple and OSF began a project to port Linux to a variety of Power Macintosh platforms running on top of OSF's implementation of Mach. This led to a core called "osfmk", and the overall system was MkLinux. The first release of this "Linux hosted on Mach" system, based on Mach and Linux 1.3.x, came out as MkLinux DR1 in early 1996. Later releases moved to Linux 2.0.x and beyond. One of the releases was incorporated into Apple's Reference Release. Note that MkLinux uses a single-server approach: the monolithic Linux kernel runs as a single Mach task.

Mac OS X uses osfmk as base (with BSD running as part of the kernel itself), and includes many MkLinux enhancements.

OSF first became Open Group, and then Silicomp.

Musical Names

Apple's OS strategy after acquiring NeXT was two-pronged: they would keep improving Mac OS for the consumer desktop market, and would create a high-end operating system called Rhapsody based on NeXT technology. Rhapsody would mainly be targeted towards the server and enterprise markets.

In contrast to the chromatic aberrations such as Pink and Red, Apple also had a string of musically inspired code-names for their operating system projects. Copland and Gerswhin were named after Aaron Copland and George Gerswhin, both American composers. Rhapsody in Blue is a famous work of Gershwin.

<<< The NeXT Chapter main Strategies >>>