Applelust.com: Terminal Mac - Darwin/OS X The Fifth BSD

Darwin/Mac OS X: The Fifth BSD

"UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." - Doug Gwyn

In my previous article, we discussed the history of UNIX, and this week we will take a more in depth look at this history as it applies to Mac OS X and the "five BSDs:" BSDI, FreeBSD, NetBSD, OpenBSD, and now Darwin/Mac OS X. There are many UNIX-based operating systems today, and many users and even administrators would probably have difficulty distinguishing between some of them. A hot topic in the world of computers is Linux, but the BSD's have been around for longer, and some administrators would suggest that the BSD's are the only way to go for "high-hit" and high stability environments. More importantly to the readers of Applelust, the BSD lineage has particular importance to the development of Mac OS X as it is from this lineage that first NeXT and now Apple take their cue. In this article we will explore these operating system relationships a little further.

"Pre-modern" BSD:

As noted in my previous article, much of the early work on the Berkeley Software Distribution (BSD) in the 1970's was based on code originally from AT&T. In fact, all BSD source distributions up through 4.3BSD-Tahoe required the user to also purchase a source license from AT&T because a significant portion of the BSD source came directly from AT&T UNIX. This additional cost became prohibitively expensive for vendors wanting to use the BSD-derived, TCP/IP networking code for the PC market. These vendors requested that Berkeley break out this "BSD-only" code and provide it as a separate product. This "wholly-BSD" product was released as Network Release 1 in 1989, and it became an instant success. Work began soon thereafter to rewrite the AT&T portions of the rest of the 4.3BSD code to produce a feature-complete, BSD operating system without the costly licensing constraints imposed by AT&T.

Network Release 2, the fruit of this labor, was released in June 1991. The source code in this release was entirely based on Berkeley code and could be downloaded without an expensive AT&T license. Unfortunately, the Network Release 2 was not completely functional since it lacked six files that did need the AT&T license. It was the intent of the software engineers at Berkeley to rewrite these six files, but these files proved too complicated to rewrite in a timely fashion. It would be six more months until William Jolitz had finished replacement files for these six AT&T files. Very soon thereafter he posted a fully compilable and bootable system for personal computers built with the Intel 386 CPU. This release was the first of the 386BSD distribution. It was available for free download, and within a short time a large number of users around the world were using it. With such a large number of installed users, the daunting task of maintaining and enhancing 386BSD became increasingly difficult to do for a person with a full time, "real" job. Others soon took up the mantel and over the next several years, five interrelated BSD-based distributions were spawned each generally focusing different goals: BSDI, NetBSD, FreeBSD, OpenBSD, and finally Darwin/Mac OS X.

BSDI:

Berkeley Software Design, Incorporated (BSDI) was a company and a distribution dedicated to the more commercial aspects of BSD. This BSD/OS distribution was based heavily on 386BSD, but it was not free. In the case of BSDI BSD/OS, the licensee paid for the privilege of seeing the source code and gaining technical support. For many companies, this technical support was worth the added cost when compared to the free 386BSD source available for download. Further, at the time, it was believed by many that the BSDI distribution would probably be of a higher quality since professional developers were employed to maintain the code. Today, BSDI is now Wind River and the most current distribution is BSD/OS 4.3. Again, support and certain value-added products have allowed Wind River to be successful in offering a non-free UNIX in the free-UNIX marketplace of today. This version of BSD is geared towards high performance, mission critical server environments where commercial support is required.

In the early 1990's, BSDI became embroiled in a lawsuit with Unix System Laboratories (USL) owned, at the time, by AT&T and later by Novell. It seems that BSDI was attempting to market their product as a real UNIX, and they included code that AT&T felt contained trade secrets and was covered by copyright laws. A lawsuit ensued and both parties were at each other's throats. Unfortunately, the BSDI distribution was not the only distribution affected by these legal proceedings. NetBSD, FreeBSD, and OpenBSD were also based on potentially "tainted" code. After a long and convoluted battle, a settlement was reached in which three files out of the 18,000 files that made up the Networking Release 2 distribution would be dropped and some other files would need to be edited. This "blessed" or unencumbered distribution become known as 4.4BSD- lite, and it was released in 1994. As part of their settlement, USL and Novel (who purchased the USL from AT&T a few months before) agreed to let other BSD distributions alone as long as they adopted the changes made in 4.4BSD-lite. The dark cloud of legal uncertainty that grew from this lawsuit is, to many, the reason why Linux quickly surpassed BSD in popularity. Today this difference in popularity is evidenced by the fact that you can order a "Linux Box" from any one of a large number of mainstream vendors, but you would be hard pressed to find a mainstream vendor that will market a "Free/Net/OpenBSD Box."

NetBSD:

According to the original NetBSD release notes posted to the comp.os.386bsd.misc newsgroup on April 20, 1993, "NetBSD, as its name implies, is a creation of the members of the network community and without the 'net, it's likely that this release wouldn't have come about." This first NetBSD release was based heavily 386BSD and the various 386BSD patchkits that were released. This release also began the tradition of leveraging a very large network community to produce a complete distribution that included not only the applications found in 386BSD, but also additions made by this community. Also, as stated in the original NetBSD 0.8 release notes, the goals of NetBSD include "an escape from the political wars surrounding what we consider a wonderful operating system and the rapid development of a stable release which we would consider of 'production quality.'" The comment on the political climate of the day underscores the thoughts many in the BSD community shared. It was at this time that Linux also began to gain an ever-strengthening foothold on the "free UNIX" market as Linux carried much less "political baggage." As with any other BSD-derived OS, when the settlement was reached between Novell and BSDI, the maintainers of NetBSD were required to synchronize their BSD with 4.4BSD-lite.

Initially, many thought that NetBSD would just serve as an interim 386BSD release in the growing time span between "official" 386BSD releases. As time went on, and it became increasingly obvious that there would be very little if any development from Jolitz and the 386BSD group, NetBSD gained notoriety as an operating system in its own right. Other developers began working on porting NetBSD additional platforms such as the Macintosh, Atari ST, Amiga, and PC platforms. The focus of NetBSD became portability and University/Academic research. Today, the NetBSD ports page reports some level of support for over 50 different platforms including, among others: Intel, Macintosh, Playstation2, and even Sega Dreamcast. This dedication to portability has been a "God-send" to many in the computer science research community. Not only do students and researchers get their hands on a full-fledged, opensource operating system, they also gain the ability to develop exciting new technologies on low cost or free equipment long since thrown aside as useless. These new technologies then often find their way into other opensource operating systems such as FreeBSD, OpenBSD, Linux, and now Darwin/Mac OS X.

FreeBSD:

FreeBSD continued the tradition of supporting the i386 platform set down by 386BSD. Though FreeBSD has been ported to the Alpha processor, it is primarily focused on developing a highly optimized version of BSD for the Intel/AMD platform. In 1992, Jordan Hubbard (who is now employed by Apple) and his colleagues began working with 386BSD and they soon released their changes through a patchkit referred to as the "Unofficial 386BSD Patchkit." Based on these patches and the underlying 386BSD operating system, David Greenman, then at Walnut Creek, suggested that this group produce their own BSD. FreeBSD was born. The first 1.0 distribution of this new operating system was released in December 1993. Walnut Creek prepared the CDROM distribution channel for the OS, and even offered their own high bandwidth servers as FTP sites. Just as NetBSD was required to upgrade to 4.4BSD-lite upon settlement between Novell and BSDI, so was FreeBSD. Hence, FreeBSD 2.0 was released in November of 1994.

From the beginning, another focus of FreeBSD was the less computer literate user. While NetBSD appealed to the "techie," FreeBSD attempted to make the install process and system maintenance as simple as possible through the use of scripts, and CDROM distributions (similar to what Linux attempts to do today). Further, the FreeBSD Ports Collection, now modeled by the other BSDs, makes porting third party, opensource applications relatively simple and system administration that much easier. Also as Linux gained in popularity, FreeBSD has tried to remain competitive through the inclusion of a Linux emulation mode. Today, the FreeBSD user can run just about any compiled Linux program on FreeBSD without worrying about recompiling the code for FreeBSD.

Another point of interest not just for FreeBSD, but for NetBSD and for OpenBSD as well, is their development model. In today's world of opensource software, the variability in development models is astounding. Many projects function as the Linux kernel project does: developers answer to one person who decides whether something stays or goes. In the case of the Linux kernel, when a developer wants to add a line of code, he or she must pass that code to Linus Torvalds who then decides whether he likes the code the developer wants to add. Although in principle this sounds like a good way to keep the kernel nice and clean, in practice it is becoming increasingly difficult for one person to handle such an arduous task. The modern BSDs use a very different development model from that of the Linux kernel model. FreeBSD has a committee of 200 developers called the "Committers" whose members are the only ones allowed to make changes to the FreeBSD source tree at any time. In NetBSD, this committee is referred to as the "Port Maintainers." Further, a Core Team selects the Committers and arbitrates disputes governs these BSD distributions. Thusly, this Core Team acts as a board of directors. Beginning in October 2000, the FreeBSD Core Team became an elected body whose members are elected from and by the population of the Committers.

OpenBSD:

OpenBSD is considered by many to be one of the most secure operating systems in existence. From its inception, OpenBSD has focused on making BSD as secure as possible. As stated on the OpenBSD web site, the developers' "efforts emphasize portability, standardization, correctness, proactive security, and integrated cryptography" and OpenBSD can also boast "four years without a remote hole in the default install." How can Theo de Raadt, the lead developer of OpenBSD, claim such a record? The OpenBSD story began in 1995 when OpenBSD first diverged from NetBSD due to a disagreement between Theo de Raadt and the NetBSD core team concerning the future development of NetBSD. OpenBSD 2.0, the first release of this new operating system, was released in October 1996. Since Theo is from Canada, he was able to take advantage of his Canadian residency and include, throughout the operating system, advanced cryptography forbidden by United States export laws. For example, the Blowfish algorithm is used to automatically encrypt user passwords. In addition to the strong cryptography included in OpenBSD, the entire source base of the distribution has been audited line-by-line in order to find any potential holes. This audit not only produced a more secure operating system, it also removed many of the programming hassles and bugs plaguing BSD-derived operating systems since the 1970's and 1980's.

Since OpenBSD was originally derived from NetBSD, it shares much of the portability for which that system is so well known. Though it does not support the 50+ platforms supported by NetBSD, it does support such platforms as i386, PowerPC, m68k (pre-ppc Macs), Alpha and several more. Even more importantly though, many of the changes made to OpenBSD to improve the robustness and the security of the operating system have "trickled down" to other BSD-derived systems like NetBSD, FreeBSD, and even Darwin/Mac OS X. As work on this operating system continues, new advances in secure systems design are being made.

Darwin/Mac OS X:

To many, Darwin/Mac OS X is the "fifth BSD", and should therefore be included in any discussion concerning BSD. Further, with the Dawin FAQ Apple states "that apart from a few architectural difference (such as our use of the Mach kernel), we try to keep Darwin as compatible as possible with FreeBSD (our BSD reference platform)." With such a statement, Apple has begun to gain a sizable following among both the BSD community and the Linux community by proving their ability to put a UNIX-based operating system on the desks of even the most computer illiterate. Without repeating much of what I have written in my previous article, it is an exciting time to be a Mac enthusiast as now we can draw from this very rich history by joining the "BSD club."

The future looks bright as people in all BSD and Linux areas consider what is next for Darwin. After all, now that Apple has opensourced their operating system, others have a real say in how the operating system evolves. People are working on advanced journaled file systems, distributed file systems, and clustering tools. Of particular interest is a recent announcement by the Secure Trusted Operating System Consortium (STOS) in which they say they want to "give the BSD-based operating system, Darwin, the features and capabilities of a trusted operating system." You can bet that as Darwin/Mac OS X matures, Apple will look at and potentially integrate these advancements whether they come from a dedicated project like the STOS or from something added to Net/Free/OpenBSD.

In addtion to the web sites referenced throughout the article, the following sites were also used as sources. Please feel free to email me or visit these sites for more information. Note: Image of Hexley, the Darwin-OS Mascot, is Copyright 2000 by Jon Hooper and all other images are owned by their respective copyright holders.