Macintosh Programmer’s Workshop

Apple’s Macintosh Programmer’s Workshop (MPW) is the most powerful development environment available on the Macintosh. It is more than just an environment for writing programs: it also offers powerful file-manipulation and task-automation functions.

Newcomers to MPW often describe it as being “UNIX-like”. While many features of MPW are indeed modelled on UNIX, its approach is fundamentally different from the conventional UNIX command line—and quite different from any other Macintosh development environment, for that matter.

People familiar with other Mac-based development environments tend to consider MPW to be complex and unfriendly. But then, MPW can do many things that those other environments cannot. MPW is admittedly “un-Mac-like” in many ways, but then, regardless of what else you may be familiar with, MPW is like nothing else you’ve ever seen!

MPW is available for downloading from Apple. Or you can read more info...

Comparison: MPW Versus UNIX

MPW has some UNIX-like features:

• Standard “command-plus-arguments” model, with the usual convention that options begin with a hyphen. Compare doing a detailed wildcard directory listing in UNIX

  

  with MPW

  

  Note the use of less cryptic command names: “Files” for “ls”, “StreamEdit” for “sed” and so on, as well as the use of Macintosh-specific characters for wildcards and other special purposes. Also commands are case-insensitive, like the MacOS filesystem itself.

• Commands may be combined into shell scripts, to automate complex tasks, complete with variables, conditional tests, control constructs etc.
• Input/output redirection (“<” and “>”), piping (“|”), and command output substitution(`cmd ...`).
• You can write your own programs in the form of tools that run under the MPW environment. These tools can be invoked from scripts, and they can read data from standard input and write to standard output, just like UNIX programs. Besides this, MPW tools have access to pretty much all of the features of the MacOS, including QuickDraw graphics and windows.
• The use of file name suffixes to identify different kinds of text files: .p for Pascal source files, .c for C source, .a for 68K assembler source, and so on, though, for the most part, there is no real requirement for this. Since all the files are still text files, standard MacOS file typing still applies.

It also has features like nothing available in UNIX:

• Seamless shell/editor integration: you can use editor commands in shell scripts, and bind shell commands to editor keys. Every window is just a text file, and can be edited as one.
• Not a command line, but a command window. Pressing Return does nothing more than insert a Return character into the text: to actually execute any part of the window contents as a command, you must select it and press Enter (or -Return). There is one shortcut: if you press Enter without selecting anything, the entire line containing the insertion point is executed as a command.
• You can attach named markers to any parts of the contents of any text file, and quickly reselect those parts of the contents by selecting the marker name from MPW’s “Mark” menu.

  These markers get saved (along with your font and tab settings, current selection, window position etc) in “MPSR” resources in the file’s resource fork, thus they are completely transparent to compilers and any other tools that might read the text, while at the same time they are persistent—they reappear for use the next time you open that file in MPW.

• The shell/editor integration allows for input/output redirection to be applied from or to a selected part of a window’s contents.
• Related to the above is the ability to redirect output to the same file, or part of a file, that was used as input in an earlier part of the command chain.

  For example, imagine I want to expand tabs to spaces in a selected part of the contents of a window. A command like the following will not work:

  

  because the output will be opened by the Shell before the Entab command executes, which means its input will be truncated to zero-length. Instead, you must do this:

  

  The interesting thing is, while the second form works under MPW, the equivalent construct doesn’t seem to work under most UNIX shells. It works under bash, but not under csh, tcsh or sh.

• Another useful application of input/output redirection on partial files is the ability to build up a result from a series of operations in incremental steps.

  For example, you might generate a listing of files in the current directory with a Files command. Then you might generate a count of the number of lines in certain of those files (not all of them) by inserting the command “Count -l” in front of each one you’re interested in and pressing Enter. Finally, you might copy and paste some of those counts (not all of them) into an “Evaluate” command to total them up. No need to try to figure out, in advance, a single command sequence to do it all! (Though you can do it that way if you want to...)

• A convention whereby most commands that return current settings for something put them in the form of a command for applying that setting.

  For example, the Set command lets you set new values for shell variables, as well as display the current values:

  Set TheVar 'The Value'

  sets the variable TheVar to the value 'The Value'. You can then query the value of this variable with the command:

  Set TheVar

  which produces this output:

  Set TheVar 'The Value'

  which is exactly what you would type to set the variable to that value!

• Support for multiple programming languages, even in the same project. Object file names conventionally end in a .o or .x suffix; however, the source file suffix is not stripped. For example, compiling a source file called Fred.p with a 68K Pascal compiler produces an object file with name Fred.p.o, and compiling an assembler source file with name Bert.s with the PowerPC assembler produces an object file with name Bert.s.x.

  The point with this is that the Make utility can use default rules to automatically decide which compiler to invoke to generate which object file.

• Make targets can consist of multiple components. This is handy for assembling MacOS resource files (which have no analog on any other platform), for example

  

  where the Link command generates the executable code for MyProg (in “CODE” resources), while the Rez command includes other resources, such as message strings, menu definitions, icons and so on.

  But there is a pitfall.

• The Make command adds a couple of interesting options: “Make -e” forces the generation of all build commands, regardless of whether the targets are out of date or not, while “Make -r” generates, not a sequence of build commands, but a list of all the root targets in the Makefile—that is, the targets that are not themselves the subjects of further dependencies.

  Thus, the command

  Make `Make -r`

  will generate any needed build commands for out-of-date parts of all root targets. It is easier to use this command than to require every Makefile to begin with a dummy “Everything” default target that depends on every other target, as is commonly found in UNIX Makefiles.

• Consistency of wildcard specifications: “?” always represents an arbitrary single character, whether in a filename specification, or in a search expression, or whatever—compare UNIX, where this is only true in shell command lines, whereas in regular expressions “?” means that the preceding character is optional, and “.” represents an arbitrary single character.

  Indeed, in MPW, any valid regular expression may be used as a wildcard filename specification.

• A single Search command that is equivalent to both UNIX fgrep and grep, and is smart enough to figure out which algorithm to use. In UNIX, grep does regular-expression searches, while fgrep only looks for simple strings. The latter can use fast algorithms such as Boyer-Moore matching, even though lots of seasoned UNIX hands are unaware of this and use grep for all their file searches. The difference is not insignificant: I did a timing test once on a DEC Alpha and found that fgrep was 5 times faster than grep doing a simple string search. So why haven’t those smart UNIX folks managed to figure out how to roll both programs into one?
• Support not just for regular expressions, but for selection expressions, which are an additional layer built on top of conventional regular-expression search patterns. In MPW, selection expressions are mainly used with the Find and Replace commands: Find selects something in an editor window, while Replace selects something and replaces it with something else. These commands can be used in various ways, for example

  Find /regexp/

  searches forward from the current position for a match for regexp, while

  Find \regexp\

  searches backward from the current position. So far, perhaps, not much more than you can do with normal UNIX regular expressions. Until you come to constructs like

  Find selection1:selection2

  which selects everything from a match for selection1 to a match for selection2, inclusive (either or both may involve forwards or backwards searches). Another valid form is

  Find markername

  where markername is the name of one of the markers previously defined for this file.

• Pathname drag-and-drop. UNIX users are accustomed to terse file and directory names, but MacOS users are not. But who wants to type long-winded pathnames? With MPW, you don’t have to: just drag the file or directory into an MPW window, and its full pathname will be inserted at the desired point, with any appropriate quoting of special characters, ready for use in a command, or for further editing, or whatever. If only UNIX systems had this level of integration between their GUI and command-line windows...
• Consistency of option syntax. UNIX commands use a bewildering variety of different ways of specifying command-line options: prefixed by single hyphens, double hyphens, no hyphens at all, single letters which may or may not be grouped following a single hyphen, and so on. Values may follow the option after whitespace, or an equals sign or other delimiter character, or immediately with no separator.

  In MPW, there is (thankfully) a single universal convention: an option begins with a single hyphen. Multiple options must be specified separately. If an option takes a value, it is separated from the option by whitespace.

• Building for other system configurations: One odd thing with traditional UNIX development tools is that it is less than easy to use them to build programs that will run on differently-configured systems, such as older versions of the OS. In particular, the UNIX linker, by default, will always look at the shared libraries currently installed on your system for inclusion in the link. This makes things tricky if you want the resulting program to run on another system that might not have exactly the same versions of the same shared libraries installed.

  The MPW linkers, on the other hand, do not have the concept of “default” or “system” libraries: everything that goes into the link must be specified explicitly. Not only that, but MPW introduces the concept of “stub” libraries: these are cut-down versions of shared libraries, which contain nothing but the exported names from the full libraries, plus version information. The linker uses these in exactly the same way it would use the full library—that is, to record, in the executable, the name of the library and its version against the corresponding symbols imported from that library. When the program actually runs, the system loader will still expect to find the full library with the correct version in the usual way.

  This allows you, for example, to build your program with a stub library that specifies an older version than the actual shared library on your system. Or you can include a stub for a shared library that doesn’t actually exist on your system (if you’re cross-compiling for another system), or conversely, omit a stub for a shared library that is installed, in case you don’t want to rely on some feature that is only available on newer systems. And you can do all this in the confidence that it will work—that the linker isn’t “accidentally” doing something extra behind your back that you weren’t expecting.

• Building for multiple system configurations: Related to the above is the ability to build a program that will dynamically query its environment for optional features, and use them only if they’re present. You can “weak-link” against particular shared library symbols, or against the entire contents of a library. Then, when the program runs on a system without those symbols present, no error will be reported, but the missing symbols will resolve to the nil address. It is then up to the program to check that the symbol values are non-nil before trying to use them.

  An alternative approach, used for example under Microsoft Windows, is to write the program code to manually load the library and search for the appropriate symbols within it. But this is a lot of extra work, when weak-linking makes things so much easier.

There are some UNIX features lacking in MPW:

• No multitasking within the MPW environment: Scripts can invoke tools and other scripts, but tools cannot invoke scripts or other tools.

  One consequence of this is that the Make tool cannot actually invokes the commands it generates: it merely writes them to standard output, where it is easy enough to capture them in a window, and select and execute them. Alternatively, you can write a simple wrapper script to capture the commands to a temporary file and execute them there (in fact, Apple provides one). But I prefer the manual approach, because

  1. I like to be sure what commands my Makefiles are generating, and
  2. If a command fails because of a compilation syntax error or other reason, I can delete the successfully-completed commands and reexecute the rest after the fixing the problem, instead of rerunning Make.
• No equivalent to UNIX egrep-style search patterns with alternatives.
• Because of the close shell/editor integration, you don’t have a choice of alternative plug-in shells. But then, given the typical confusion in the UNIX world caused by differences between shells, this is probably just as well...

Comparison: MPW Versus Project-Oriented Environments

Project-oriented development environments like Metrowerks CodeWarrior are very popular. They are built around a pretty simple model: take a bunch of source files, compile them all, and link them into a single executable. They probably deal nicely with 90% of programming needs (though your choice of problems to solve is often subtly influenced by the tools you have available to solve them!). But they do tend to fall down in a number of areas:

• Dealing with more than one link operation. Consider the simplest case: building a “fat” application, with both 68K and PowerPC executable code. In your CodeWarrior project file, you must first set up two targets, one for a 68K-only executable and one for a PowerPC-only executable. Then, once you have successfully built each of these for the first time, you must go back and set up a third target, using the “MacOS Merge” processor, to combine the two into a fat executable.

  In MPW, by contrast, you can set up the build rules to generate the fat executable directly, without producing any intermediate executable targets.

• The clumsiness of multiple links also applies to the setting of compiler options. For instance, the 68K and PowerPC C compilers have entirely separate preference panels in CodeWarrior. This makes sense for settings to do with target-specific code generation (generating 68020/68881 code only makes sense for the 68K compiler, while Altivec generation only makes sense for PowerPC), but it leads to duplication of common language-specific settings (such as “Require Prototypes”). Here, it doesn’t make sense to have different settings for different targets, since the source files are the same, after all.

  The way MPW deals with this is to have separate Make variables, COptions and PPCCOptions which specify options for the 68K and PowerPC C compilers respectively. However, it is easy enough to define these variables partly in terms of another, custom variable, which provides common language settings, e g

  CommonCOptions = -proto strict
  COptions = {CommonCOptions} -mc68020 -mc68881
  PPCCOptions = {CommonCOptions} -vector on

  so changing the common options is done in one place, just by updating the definition of CommonCOptions.

• Project data compatibility: Another problem with project-based development environments is that the format of the project file keeps changing between versions. This means in particular that, if you want to make your complete project source code available on the Net, it’s going to be difficult to use by somebody running an older version of the environment, since they’re going to have to recreate the project file from scratch.

  In MPW, on the other hand, the Makefile format and build commands have remained largely stable for a long time. If you do happen to use newer features that might not be available in the version of MPW that someone else is using, the odds are that they will be able to get around it by making small changes to the build files, rather than having to rewrite them from scratch.

  CodeWarrior Pro 5 and later tried to get around this by introducing the feature of exporting a project file to XML format, as well as being able to reconstruct a project file by reimporting the XML form. Of course, this feature is of no use to those still stuck with earlier versions of CodeWarrior.

• Project data compatibility: Yet another problem is that the project data can sometimes get corrupted in mysterious ways. I once had a build where the compiler kept barfing on the line “#include <QDOffscreen.h>”, insisting it could not find that include file, even though there was indeed a copy, with the right name, in exactly the same place along with all the other include files that it was finding! Finally I tried dumping the “<project> Data” folder, forcing CodeWarrior to recreate it, whereupon the problem magically went away.

  It’s hard to conceive of MPW Makefiles, which are just text files, becoming corrupted in any comparable fashion.

• Liveness of data. CodeWarrior has a couple of nice features in its Errors & Warnings window and its Compare Files display. All error and warning messages reported by compilers during a build are collected in the Errors & Warnings window, where you can double-click them to go to the relevant section of the appropriate source file. Similarly, the Compare Files function puts up a very nice side-by-side display of the differences between two files or two folders, and lets you step through the different sections for closer examination. MPW has nothing that looks this good.

  Trouble is, the CodeWarrior displays cannot be saved to disk in any fashion that keeps their “liveness”. Imagine I want to save the error list to disk so I can come back to it and continue fixing the errors later: I can save the list to disk all right, but when I try to reopen it, it’s just a text file, and I can no longer double-click entries to go to them! And the Compare Files display cannot be saved to disk at all: If I want to come back to the comparison later, I have to rerun it.

  In MPW, compilers and other tools report all their messages in the form of ho-hum plain text. But there is an interesting twist. All the messages that refer to parts of files take a form that looks like this:

  File "CheeseList"; Line 234 # Expected Gorgonzola, but got Cheddar instead

  Remember what I said above about MPW being, not a command line, but a command window? It just so happens that, if you click within the above line and press Enter, MPW will open the specified file and take you right to the line mentioned in the message! (This works because “File” and “Line” are defined as commands, and the human-readable message is put after the “#” comment delimiter, where MPW pays no attention to it.)

  It may only be plain text, but it’s “live” plain text, and it stays live even after saving it to a file and reopening it, or copying it to another application and back again, or whatever.

• File size limitations. An interesting limitation of the CodeWarrior editor is that any source file it opens must be loaded entirely into memory. Indeed, this is a fundamental feature of the entire CodeWarrior architecture: even the compilers and other plug-ins require that the files they operate on reside entirely in memory. The MPW editor, on the other hand, only loads as much of a source file into memory as it needs, and can keep unsaved changes in a temporary file on disk if there isn’t enough room in memory. And of course the MPW compilers and other tools often operate on files as streams, reading only as much as they need at a time and writing out results as they are generated.

  Now, I admit this is not a big issue with any realistic sizes for program sources on present-day machines. But there are other kinds of files you might want to operate on: for example, Web server logs can easily reach many hundreds of megabytes in size. I guess this difference reflects the fact that MPW is so much more than just a program-development environment.

• Speed. If a single CodeWarrior project builds more than one “target”, then CodeWarrior will recompile all the source files related to a target for that target, even a source file which is shared with another target. That is to say, if a source file A is common to targets S and T, then it will be compiled once for S, and once again for T. A discussion on comp.sys.mac.programmer.codewarrior revealed that the rationale behind this is that the different targets might have different preference settings for that language, which is why the compiled code can’t be shared. But surely the environment should be smart enough to check the settings, and share the code if they’re the same?

  With an MPW makefile, sharing of compiled code is the default behaviour: source file A will normally be compiled only once, and the result linked into both targets S and T. But if you really want a separate copy of the compiled code for each target, perhaps with different build settings, it is easy enough to set this up, by specifying different names for the two object files, with different rules for generating them from the same source file. As with most things in MPW, the choice is yours.

• Mix-and-match project construction. Since an MPW makefile is just a text file, it is easy to construct the makefile for a new project by copying and pasting bits from one or more existing makefiles—you have the full power of the editor available, for example, for using search-and-replace to change an old target name to a new one.

  The closest CodeWarrior comes to this facility is that of project “stationery”, where you can use a single existing project file as the starting point for constructing a new one. Project files cannot be edited with the CodeWarrior editor in the same way that ordinary source files can.

• Project documentation. I recently had to build in support for a piece of third-party USB hardware into an application for a client. The hardware vendor provided a driver library file in CodeWarrior format, which I was able to link into the project, add calls to from the application, and test that it actually worked with the hardware. So far so good.

  Then the client reported that the application would no longer run on older Macs without USB, even if you weren’t trying to use the functions provided by this hardware. I discovered that the driver library included an implicit import of USBServicesLib, which is a MacOS shared library that only loads if the machine has a USB interface, which my machine did. I had not included this library in my project, yet it was automatically being brought in by CodeWarrior during the build, as a “strong” import requirement. The result was that the application could not be launched on a system where this library was not present.

  The solution, of course, was to explicitly include USBServicesLib in the project, such that it came before the hardware driver library in the link order. Then I could specify that USBServicesLib was to be weak-imported, which meant that the application was to be allowed to run even if this library wasn’t present. Of course, my code had to explicitly check whether the library was present before attempting to use the functions of the USB hardware.

  This kind of trickiness is all very well, but where in the project do you document it? Otherwise, some future programmer who inherits my code may not appreciate the significance of this aspect of the project settings, and introduce a bug by inadvertently changing it. Having a separate “Read Me” file introduces a separate project component that I have to remember to keep up to date. But the CodeWarrior project file doesn’t provide any place for me to insert my own comments!

  Contrast this with an MPW Makefile. There is no such thing as implicit import of libraries, so I would have had to explicitly include USBServicesLib in the build from the beginning—the link order is not important. Also, it would be easy enough to stick a comment next to the -weaklib directive that weakly imports this library, explaining that this is to allow the program to run on machines without USB hardware.

• Speed. On my beige G3/266 Mac, it can take CodeWarrior Pro 5.3 about 6 seconds just to open a modest-sized project file. Starting up the environment to begin with takes almost as long. By comparison, MPW 3.5 takes about a quarter of a second to start up, including the time it takes to set up my custom key bindings, menu and path definitions and so on. And of course any file, whether Makefile or source file, opens pretty much instantly. Even the MPW editor feels a bit more responsive than the CodeWarrior one.

  It wasn’t always this way. Back in the early days of project-oriented environments, it was MPW that was big and slow, and the project-based systems that were small and fast. What’s happened is that the project environments have become larger, slower and more complicated, as they try to fill in all the major gaps that keep becoming apparent in their functionality, that MPW was able to do all along. Trouble is, I don’t think their original architectures were designed to cope with all of this. Thus, whether you like it or not, the MPW architecture turns out to be the one that is better at standing the test of time.

  For a long while, one of the ways I used to customize my MPW installation was to remove the -Q keyboard shortcut for the “Quit” menu item. This is because it was so close to the -W shortcut for closing windows that I would occasionally hit it by accident and quit the environment when I just wanted to close a window, and it used to be a pain when it took several seconds to start MPW up again.

  These days, I do this same customization with CodeWarrior, but I don’t bother doing it with MPW any more—MPW is so fast to start up now that it no longer matters.

MPW Tips & Tricks

MPW-Related Links

Back to LDO’s Home Page