Scientia et Macintosh

Mathematica: The Mathematical Revolution

"As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality."
- Albert Einstein

On January 8, 2002 in San Francisco, the IDG Macworld Expo kicked off in traditional fashion with Apple CEO Steve Jobs speaking to the Macintosh faithful. This year, the focus was overwhelmingly Mac OS X-centric and only those firms peddling native wares were even mentioned. During his keynote speech, Steve was joined by the likes of George Lucas (via video) of Lucas Film, Todd Bradley of Palm, Shantanu Narayen of Adobe, Theodore Gray of Wolfram Research, and several other notables. You may not recognize all of the names on that list, but you most likely recognize all of the companies. That is, perhaps, save one: Wolfram Research. Since 1987 Gray, along with Stephen Wolfram, and their brilliant developers at Wolfram Research have been working to produce perhaps the most ubiquitous program in modern mathematics: Mathematica. Mathematica has always been available for the Mac (along with about 15+ other platforms), but only recently has it become available for Mac OS X. To those of us in scientific fields, this is an exciting development and one well worth an article or two.

Why should the readers of Applelust care about this software you may ask? After all, how many of these readers are mathematicians? Well today, not just mathematicians in hallowed, academic halls use this powerful software, but it turns out that there are many facets in our world that can be described through mathematical constructs. For example, former pro skateboarder Dan Gesmer set out to design new trucks for his skateboard and he has successfully revolutionized the skateboard industry. Similarly, it has long been understood that music is very mathematical in nature and recently Mathematica was used to completely compose an original, groundbreaking piece: "6 Integers." Even a particular Japanese fashion designer bases much of her work on the computational ability of Mathematica. And let's not forget Economics where the entire field is based on past trends, present data, and predictive models. According to a section of Wolfram's web site, the number of different fields that employ Mathematica is on the rise as the purveyors of art, science, and finance all strive to find new and interesting technology to perform their trades.

In this article, I have asked my good friend and colleague, Edward Brothers, to aide us in the discussion of this important product with an addition of an "Application" section. Ed has been using Mathematica on a daily bases for much of his graduate career as a way of improving his theoretical models. Although this Application section is highly technical in nature, it will show the interested reader the thought processes that go into putting together a mathematical model. He will focus on much of the work that we do in the Merz Computational Biochemistry Research Group in the Pennsylvania State University Department of Chemistry. Here we perform cutting edge, biomolecular, quantum mechanics development, and Mathematica is just the tool we need to propose and then test a model before going to the trouble of actually developing, debugging, and maintaining the Fortran or C++ source required to implement the model. Before we delve into this application, I should probably provide the reader with an understanding of what Mathematica is and why people use it.

Mathematica Explained

Years ago when theoreticians wanted to model a mathematically defined system or perhaps derive a new mathematical construct, they would need to sit down to work with a box of pencils and many reams of paper. They would then need to perform calculation after calculation on paper until they were satisfied with their results. Some of the most fundamental axioms of mathematics were discovered and proven in just such a fashion. With the advent of the computer in the 20th century, theoreticians were finally able to have a machine to do the "heavy lifting" associated with the pencil and paper approach. However, as the computer evolved and became faster and faster, two problems continued to plague theoreticians of all research areas. First, computers were (and are) never fast enough to do everything that the scientist wanted (or wants) to do. Today we still have this problem and we are always looking forward to the next processor or the next improved parallel algorithm. Second, there existed no singular mathematical tool that was generalized enough to model, in a dynamic nature, complex mathematical systems. Therefore, for each new model that the theoretician attempted to study, he or she would need to write a whole new program and/or use a multitude of disparate and incompatible programs. An obvious problem with this approach is that the scientist could spend an inordinate amount of time working to precisely define the problem at hand, and the only way to discover whether the scientist was on the "right track" to designing their mathematical model was to finish it. This approach is acceptable assuming that your model and any underlying assumptions that you made are correct. As one can imagine, a first time test of a model will often result in a failed attempt, and the scientist ends up wasting valuable time writing unusable code.

Save waiting for computers to become infinitely fast, the first problem is probably unsolvable. Mathematica, though, attempts to address the second problem by providing a platform on which the theoretician has, at his or her fingertips, (1) a rich set of integrated and dynamic mathematical tools that do not require the use of different and incompatible programs, and (2) access to a dynamic environment where the user can easily "try-out" new models without the need for Fortran or C++ programming, debugging, and testing. For example, if the Mathematica user needs to use a Hermite polynomial to model a particular quantum mechanical system, he or she does not need to write and debug the code for the polynomial itself. The scientist merely needs to put a function call to HermiteP in their Mathematica Notebook (ie: the document system of Mathematica). Mathematica provides an extremely large, well developed (and well debugged) toolbox of common, and not so common, mathematical constructs used in mathematical models. The user is able to put these constructs together in any way he or she likes and then verify, through trial and error, that they truly understand the problem they are attempting to solve. Further, in areas that do not generally employ programming of any kind (such as the arts), the user can sit down and work with a GUI to allow their ideas flow without paying the learning curve penalty of going to one of the traditional languages such as C++ or FORTRAN

Mathematica's Success

In the lab, we have been working with Mathematica 4.1 on local Mac OS X, Windows98, and Mac OS 9 machines, and also on several IBM RS/6000 supercomputers we have at the University. As another point of comparison, we also have access to Mathematica 4.0 on the National Center for Supercomputing Applications (NCSA) SGI supercomputer. According to Wolfram's supported platform site, Mathematica is actually developed and maintained for over 15 different platforms including, in addition to those already mentioned, Linux, NextStep, HP-UX, True64, and many more. One point that should be underscored is our appreciation of Wolfram for successfully building a product that is truly capable of running on multiple platforms with little or no loss in functionality when moving from one platform to the next. Whether the user is running Mathematica on a Mac OS X workstation or an IBM or SGI supercomputer, the interface looks and feels very similar. In a research environment such as ours, where the user may be on one or more different platforms in a single day, this feature parity is very important. One would be hard pressed to find another professional software developer whose product is supported anywhere close to this well on as many platforms.

For the visual interface itself, Wolfram decided to take the more minimalist approach with all of their supported platforms. When the user starts up the GUI, a simple pallet (see figure) along with a blank Notebook are presented (additional pallets are also available). Again, whether the user is working in Mac OS X, Mac OS 9, Windows, AIX, Linux or IRIX (SGI) the function pallet looks almost identical. Further, if one uses the non-GUI, command line version on any system, the interface is identical between the different platforms. As you can see on the pallet, the user has direct access to familiar mathematical symbols corresponding to integrals, sums, matrices, and specialized characters. Any calculations then are executed at the backend or "MathKernel" process level. This MathKernel process is completely separate from the front-end interface. On UNIX-based systems such as Mac OS X, you are able to have multiple, separate MathKernel processes going at once. Further, if one has a favorite desktop machine that may not have the computational muscle to perform some of the complex calculations Mathematica may require, the user is welcome to off-load the potentially expensive calculation to a dedicated server. This remote, compute-server configuration is handled through the GUI, and it is relatively simple to complete.

Simple Mathematica Examples
(see attached Notebook #1)

As you will see in the Application section, Mathematica is designed to handle mathematical functions at every level. In the first example below, the simple, algebraic equation is solved with a single line of Mathematica code. In the next example, integral calculus becomes child's play.

Example #1: The equation solver in action.

Example #2: Calculus is also no match to Mathematica.

As seen below, one of the special strengths of Mathematica is the direct incorporation of graphics (such as the 2D and 3D graphs below) and even sound. Through this graphics integration, the user can see the results of his or her calculations and models very quickly instead of days, weeks, or even months down the road. For those who have interest in studying pictures and art, Mathematica is also able to bring in arbitrary graphics files and formats with ease. These files can be integrated with other files, or if the bitmap (or whatever) is a picture from which data needs to be derived, Mathematica can do that as well.

Example #3: 2D Plot: In[#]:=Plot[Sin[x] + Sin[1.6x], {x, 0, 40}]

Example 4: 3D Plot: In[#]:=Plot3D[Sin[x y], {x, 0, 4}, {y, 0, 4}, PlotPoints -> 300]

Just like any tool though, Mathematica is not without its faults. Unfortunately, the pictures above are completely static in nature such that the user is unable to manipulate them through rotations and translations in real time. Further, the user is unable to use their mouse to pinpoint interesting places on the graphs in an interactive manner. Also, in order to resize the structure you need to drag a corner of the window then wait a significant amount of time for the graphic to redraw at the larger size. If the user would like to have some level of real time manipulation, he or she is welcome to use the "RealTime3D" module. The problem then seems to be that; at least in the Mac OS X implementation that I tried, the RealTime3D module is very buggy. At times, the notebook will completely blank and need to be redrawn, and the quality of the pictures just seems much less when compared to the Default3D mode. It appears that either (1) OpenGL is not able to render at the required quality or (2) the RealTime3D implementation is too new and a little buggy. So if I were to have one complaint during my time with Mathematica, it is what I found to be a lack of interactivity between the user, the graphics, and the text. It is entirely possible that this problem may actually be an oversight on my part because, as one quickly learns, Mathematica is a massive and complex application and often something you think is missing today is found tomorrow.

Platform Comparison

Lastly, as part of the review process, several platforms were tested and compared. A simple Mathematica load script was developed that benchmarked several everyday operations. The script was executed from the command line and the time it took to run was counted in CPU time. A more thorough benchmark is being developed, but the one included here should give the interested reader an idea of the performance they can expect. It should be noted that the PowerMac in these tests was a simple dual processor desktop machine, but that multiple processors were NOT being utilized by Mathematica on the PowerMac or any other of the platforms tested. It is also interesting to note that the $2000 desktop PowerMac is almost as fast as the much more expensive IBM RS/6000, SP node and is, in fact, faster then the other platforms tested.

Platform	Processor(s)	Memory	Operating System	Time (sec)
Apple PowerMac	2 X 450 Mhz G4	512 MB	Mac OS X 10.1.4	25
SGI Octane	2 X 300 Mhz R12000	384 MB	IRIX 6.5.15f	57
IBM RS/6000	4 X 332 Mhz PPC 604	2 GB	AIX 4.2.1	37
IBM RS/6000	8 X 375 Mhz POWER3	4 GB	AIX 4.3.3	22
*SGI Origin2000	64 X 195 Mhz R10000	12 GB	IRIX 6.5.14f	78
* The version of Mathematica available on the SGI Origin 2000 tested for this review was 4.0.

Now that you have an idea of what Mathematica can do, a fellow graduate student in my laboratory has prepared a relatively simple example of a "real world" use of Mathematica. If you have Mathematica or MathReader, please see the attached Notebook #2.

Mathematica's Application to Computational/Theoretical Chemistry

It is quite obvious how Mathematica's data manipulation and statistical abilities would be useful to an experimental scientist, and it is equally as obvious how its graphical abilities would be pedagogically beneficial in many fields. It is less apparent how Mathematical would be useful in computational chemistry, or at least it was to this author. My argument prior to using this package was that in a field where mathematical competency is the norm, tedious problems are solved with customized codes, and most of the mathematical difficulties routinely encountered have been analytically solved for at least a decade, a program with a learning curve as steep as Mathematica's is of limited utility. Then I undertook writing my own quantum chemical code (also known as ab initio code, meaning from first principles, i.e. without approximation) and discovered exactly how wrong I was.

Before discussing the package itself it is necessary to cover a bit of modern quantum chemical theory. The most basic form of ab initio theory still in common use is Roothan formulation of Hartree-Fock theory, the details of which exceed the scope of this software package review. Basically, HF theory provides the total energy of a molecule at a specific geometry with improved accuracy at larger basis sets, and neglects only electronic correlation's contribution to the total energy, i.e. electronic kinetic energy in a molecule is the same as a gas of non-interacting electrons. This total energy is made up of the atomic nuclei repulsion interactions, the non-interactive kinetic energy of the electrons, the attraction of the electrons to the nuclei, the repulsion of the electrons on each other, and the exchange energy, which arises from the nature of the electron and is why quantum chemistry is necessary in the first place. All of the terms in the energy with the exception of the inter-nuclear repulsions require the electronic distribution of the molecule to be known. This distribution is determined by the electronic occupancy of orbitals, which are defined mathematically below. To determine the occupancy of the orbitals (more commonly called basis functions) it is necessary to know the energy of each of the electronic interactions listed above. Therefore energy depends on distribution, and distribution depends on energy, and thus we have an iterative problem. More mathematically stated:

Where F is the Fock operator matrix, C is the molecular orbital coefficients, S is the orbital overlap matrix, _ is the molecular orbital energies, and F is a function of C which is a function of F. The Fock matrix is made up of the electronic interactions listed above, and it is these terms (primarily the electronic repulsion) that dominate the computational time of any ab initio program. Before we talk about calculating any of these terms, we need to discuss the orbitals commonly used today.

The most mathematically correct shape or type of orbital is called a Slater-type orbital, and has the form:

Where N is a normalization constant, x, y and z are Cartesian coordinates centered on atom A which possesses the basis function in question, a is an orbital exponent that has been empirically derived such that the basis function is of the correct size for that atom, and i, j and k are the angular momentum exponents that determine the shape of the orbital, e.g. if i+j+k=0, it is an s orbital, if i=1 and j=k=0, it is a px orbital, if j=2 and I=k=0 it is a dy^2 orbital. (Note that i+j+k=3 are the f orbitals, and i+j+k=4 are the g orbitals, which is about as far as most basis sets go at the present time.) The STO was very popular in the early days of quantum chemistry, and is still used in some areas of quantum chemistry such as semiempirical theory. This type of basis function suffered from one major difficulty, however: it was very difficult to integrate the product of several STOs over all Cartesian space, something necessary to the calculation of the terms listed above.

The STO was therefore replaced by the GTO, or Gaussian type orbital:

The shape of this orbital is incorrect, due to the squared term in the exponential, but is much easier to integrate over due to the Gaussian product rule that states that the product of any two Cartesian Gaussians is a third Gaussian. The improper shape was compensated for by adding several Gaussians together (called contracted GTOs) to approximate one STO:

These CGTOs are the basis functions used in almost all modern quantum chemistry. Integrations over these functions are the actual nuts and bolts of computational chemistry, and it was during the writing of code to integrate over these functions that I found myself using Mathematica.

For the purposes of this review, I am only going to talk about the simple two-orbital terms, and then a quick bit about matrices, to illustrate the efficacy of using Mathematica during the development of ab initio code. The two orbital terms are:

If the orbitals in the overlap term are both uncontracted s functions, i.e. the angular momentum exponents are all zero and there is only one Gaussian per basis function which is the simplest possible scenario, the first equation becomes:

Which simply becomes another s Gaussian, and is trivial to solve using published integration tables. Even given a lot of these, it may not justify the intellectual effort of learning to use Mathematica, as they are simple enough that doing even a few dozen by hand may be faster than learning this package. But please examine what the simplest kinetic energy term becomes:

This is basically three overlap integrals between s and d type functions, which may or may not be equivalent depending on whether the molecule has been rotated for computational efficiency, and the above function is not a trivial integral to do by hand. This begins to justify the amount of effort that goes into climbing the Mathematica learning curve. Compound this with the fact that any new quantum chemical code must be rigorously tested up to g functions (total angular momentum=4, 17 permutations) as this is the highest angular momentum commonly found, and that most of these functions will be highly contracted (i.e. consist of many Gaussians per basis function) and the amount of labor necessary to test by hand become impossible. Compare this with attached notebook. Given any completely defined pair of basis functions, it is quite easy to test any new code. This is especially important when it is considered that most commercial codes only give total values for the kinetic energy, etc., and does provide values on individual pairs. This is also extremely important as you move to more complicated integrals, like the three center electronic nuclear attraction integrals and the four center electronic repulsion integrals, which are untenable by hand.

We've seen at this point that Mathematica is useful, indeed possibly necessary, for development of quantum chemical code by ensuring accuracy in the integrals used to construct the Fock operator matrix and the molecular energy. However, I would make the argument that it is even more important for its matrix manipulation abilities. Let's reexamine the HF equation and reduce it to an eigenvalue problem using the orthogonalizing transformation matrix X:

Note that s is obtained by calculating the eigenvalues of the overlap matrix and then s^(- 1/2) is obtained taking the inverse of the square root of each element, and U is of course the eigenvectors of the overlap matrix. This allows:

This reduces the entire problem to an eigenvector-eigenvalue problem, and is at the heart of the iterative process necessary to solve the HF equation. Unfortunately, it is also extremely tedious to do by hand.

Consider water (H2O) done at the smallest published basis set, STO-2G, where each function is made of 2 Gaussians. Each hydrogen would have an s function, and the oxygen would have 2 s functions and three p functions, for a total of seven basis functions and fourteen primitive gaussians. The above steps would then have to be performed on a 7x7 matrix, which is untenable by hand, even assuming that you have all the integrals and matrix elements readily available. With Mathematica, you can do this in minutes (counting set-up and typos). The added bonus is that you can also see the intermediate matrices, which are never revealed in commercial codes. Thus rather than doing a large proof that the X matrix is symmetric, which you should do any way as it builds character, you can simply do a number of mathematical experiments and determine that it is. You can also store the matrices in arrays that are easy to examine without the concerns about efficient storage that come with writing quantum chemical code.

Thus Mathematica is useful both for validation and as a "doodle pad" for the numerate chemist who wishes to test an idea rapidly without sacrificing accuracy. This package has greatly enhanced my ability to produce items of scientific value. The learning curve was frustrating, but I cannot imagine working as a computational chemist without it, and I think most people in mathematically rigorous fields would feel the same way.

Conclusions

As illustrated in this review, Mathematica is a complex and powerful program. So much so, that it is impossible to discuss every feature available to the Mathematica user. In addition to several getting started manuals, the professional version of Mathematica actually includes a 1500 page, hard cover book ("The Mathematica Book" by Stephan Wolfram) that is designed to show as many aspects of Mathematica as possible. Further, the depth of the on-line help is very impressive (though sometimes difficult to search). Complete examples are available throughout the included Mathematica literature both from the hard cover versions and from the online versions. The online literature is especially useful as it is possible to cut and paste from the included examples to a users own notebook for testing purposes. All-in-all, the developers of Mathematica should be proud of their work.

For more information on the topics discussed in this article, please see the Wolfram Mathematica web site, and the following references:

Levine, I. N. "Quantum Chemistry: 5th Ed." Prentice Hall, Upper Saddle River, NJ, 2000. This is an excellent text for beginners and covers STOs and GTOs in depth.

Szabo, A. and Ostlund, N.S. "Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory: Revised Edition" Dover Publications, Inc., Mineola, NY, 1989. This is the bible of quantum chemistry, although it is very information dense.

Wolfram, S. "The Mathematica Book 4th Edition" Cambridge University Press, 1999. This book is included in the professional package and provides a very detailed explanation of the use of Mathematica.

- Lance M. Westerhoff

Lance generally writes for the Terminal Mac column of Applelust where he discusses the UNIX world as it applies to Mac OS X. He is also a graduate student and a UNIX systems administrator in the Merz Research Group at the Pennsylvania State University.

- Edward N. Brothers

Ed Brothers is a guest writer this week, and he is currently a graduate student in the Merz Group at Penn State. His research centers around the developement of new density functionals and more approximate density dunctional theory.