Thankful to be a Cocoa Programmer

The company I work for is interested in Microsoft's .NET framework; I am not. But, I was scrounging for free content on iTunes the other day and came across the .NET Rocks podcast, and out of idle curiosity downloaded a few episodes. They are well done, and put together by friendly people with an unflagging enthusiasm for Microsoft technologies. And they make me very happy to spend much—unfortunately not all—of my time working on Macs and iPhones.


Apparently, the target audience is comprised of developers putting together custom business applications; the kind of vertical apps corporate America (and apparently Dubai) consume by the megalines of code. Not the general purpose, high degree of finish, large user base applications I've spent much of my career writing, but database frontends and the like. It's the Bizarro world out there, where right justifying labels is considered a major advance in GUI development.


Any iPhone developers would be well served listening to the recent cast on Windows Mobile. As someone who was scrounging for work when the Mac had 3% market share, I can sympathize with the pathos of a product manager for Windows Mobile trying to put a brave face on disaster, but come on. This exchange pretty much sums up the level of wishfulness and straw grasping:
Host: So let me ask the question a different way: the next version, whatever it's called; should Apple be scared?
Rob Tiffany: Very scared.
Hosts: (laughs) YES!
Rob Tiffany: Very scared.
Hosts: I knew it.
Rob Tiffany: Yeah, yeah, we're working on some secret sauce out there.
Host: Not too secret anymore!

I will congratulate the hosts for not being blind to WinCE's current flaws: they gave Rob a hard time about how the phone app on their phone was glacially slow; I'm just amazed they think it will get better. That old saying about a second marriage being the triumph of hope over experience..


And the episode on complying with the corporate governance rules of the Sarbanes Oxley law... If I had to do that sort of thing, I'd seriously consider going to work at Home Depot. How does one show up at work every day doing that sort of thing?


And that's the thing. I'm nearly always happy either going into work—assuming I won't be spending the whole day fixing OLE bugs on the PC—or pulling out my MacBook and add a refinement to an iPhone app. Life is sweet. I get to work in an application framework which was designed right from the start; light weight and powerfully elegant. I'm not one of a hundred cogs living in a condo in Dubai; I'm a sole practitioner, or an unblended part of a small team. I write software people don't have to be paid to use. I don't have to wait for the secret sauce which never comes. I am a Cocoa programmer, and for that I am thankful.

Revamping a Dated Cocoa Application: InCDius 2.5

iPhone development has re-energized my love of programming. My day job requires me to do too much Win32 coding, and that is soul deadening for a Cocoa programmer. The combination of corporate priorities, and the fact I can add features much more quickly and with fewer bugs in Cocoa, leads me to spend a huge fraction of my time in Visual Studio, when I'd rather be in XCode. It's been draining.


I released InCDius 2 back in 2002, and it never got out of beta, although beta 16 was quite stable, and I apparently have a number of loyal (yet put upon) users. It is just an Audio CD database, not a personal media database like the well regarded Delicious Library. It was fast, and pretty darn stable, although a few users have had database corruption issues—please back up to XML people. I knew for years that I should release a new version, but could never quite muster the energy. It embarrassed me having this old, non-Intel application, gradually getting less stable with each OS release application; and I was considering my options for killing it entirely.


But my Remote Remote GH for iPhone application makes heavy use of the SQLite database, so I felt confident I could transition InCDius away from the Berkeley DB database. This transition had been the most daunting of the reasons keeping me from releasing an update.

My first impulse was to refactor the whole application using Core Data, but that would not have added anything but development time; the GUI was wired up quite well; the application was already scriptable. All I needed was a new database backend, and Apple ships OS X 10.5 with SQLite 3 right in /usr/lib. By choosing SQLite, I was no longer responsible for compiling and upgrading my database library; I can rely on it being there and just working. I tried using the QuickLite SQLite wrapper, but it was not capable for the task, as it creates a large number of objects maintaining its internal cache, to the point of locking up my Mac. No, I had to call the SQLite C API directly. As long as I minimize random access into the database, I get good results.

I also wanted this to be a release that could live on its own for a very long time; who knows how long I'll go without finding the time to update it. I wanted code which would last a very long time. This meant not only transitioning to Intel, but compiling for Intel 64-bit, and removing every deprecated API or framework message I could root out. I decided to make the minimum OS X version 10.5 (Leopard), and to transition the codebase to Objective C 2.0, although it isn't quite all there yet. I'll have to work on the garbage collection, but I am using both Objective C 2.0 fast enumeration loops and properties where appropriate.

Forward thinking means removing the past, so out went a few features tied to old APIs. I had been allowing users to import information from Classic Mac OS's CD Player preference file, but that involved using old school Resource Manager calls. Anybody ever going to use this feature again? No. Then out it goes, along with the Resource Manager.

I had 2 separate ways to play Audio CD tracks. One used QuickTime; one used Core Audio. Kept the Core Audio.

NSTableView has a new (to me) way to deal with selections with NSIndexSets, and deprecated the messages I had had been using. Goodbye old messages.

My own Objective C objects were filled with ints, unsigned ints, longs, and even the occasional unsigned. Hello NSInteger, NSUInteger and 64 bit computing.

I had been using a custom version of NSTableView to draw blue stripes. Goodbye custom class, hello check box in Interface Builder and grey stripes.

Obviously, I had been using .nib files for building my GUI. Time to upgrade everything to .xib files.

A user sent me his database (as a .zip archive of XML files) of 15,000 individual disks. Wow, and I had been thinking InCDius was zippy fast and ready for release. Massive performance tuning and the decision to load the database at runtime instead of upon the first search. Much simpler and reliable, and lets the user do something else while everything gets ready.

Take time to remove all the warning messages. You will save yourself a huge amount of hassle in programming, but especially in Cocoa programming if you eliminate all the compile warnings. You want to know right away if the compiler doesn't know if NSView responds to "releese". Do not let warnings build up. Fix them. Fix them all.

A Google Tech Talk by Linus Torvalds inspired me to change the unique identifier key I had been using to lookup disks in the database; he uses SHA1 hashes to avoid corruption in Git, and it seemed to me that a SHA1 hash of the Audio CD's Table of Contents would be as close as I could get to being unique; although there will be rare instances when 2 CDs cannot both be in the database.

So after all this, what do I have? A fast, reliable(?), clean, focused little application, which I can send out into the world; spend the next few months fixing the occasional bug and adding the occasional feature, and which people can use for a good long while.

Re-imagining a Desktop Utility into Cocoa Touch

If anybody's starting up a business based around iPhone development, and need a chief engineer, I'm sure you can find my e-mail address.


Well the momentous day came, and the iPhone SDK is in our hands. I wish I were not so incredibly busy at my day job, or I would take a couple weeks off and learn the ins and outs. But, I've decided I should start small and port a little application I wrote, Remote Remote GH for MythTV, which I'm afraid I have not been properly maintaining, but has the advantage that it would be much more useful on an iPhone than on a laptop, and it only uses Cocoa and various Core Foundation routines. I'm going to go through the exercise here of paring the interface down to size and making it more in touch with the Touch's way of doing things. Then sometime in the next couple months, I'll take the time to actually do it.


Let's look at the original and see how we can get it down to 320x460 and 480x300 (which are about the available sizes once you discount the "status bar" region of the display). Not only do you have a smallish screen, there is inconvenient text entry, and any widget on screen should be at least 44x44 pixels to deal with the stubby fingers of adult humans. On the plus side, there is multi-touch and 3D movement via the accelerometer.

• Disconnect/Connect Button This should probably be done away with entirely, and the app should try to keep a connection up when visible.
• Show Keys Button This brings up a list of keyboard keys which are appropriate for the current state of the MythTV. The idea was for the user to control the MythTV with the same key presses used when sitting in front of the computer running the MythTV frontend. Keyboard input on the iPhone being discouraged, replacement with a series of contextually appropriate button panels is appropriate. (A panel for watching recordings, a panel for navigating, a panel for DVDs, etc.)
• Scrolling Status Console This was a area devoted to telling the user what has happened. This can be condensed into a 2 line status widget at the bottom of each screen.
• Navigation Buttons Live TV, DVD, etc., can be put into a standard navigation bar, to allow the user to jump to a common task, which I will pare to Live TV, Recordings and Video
• Recorded Programs Table This can be moved to its own panel, and condensed, with perhaps a sub-panel which the user can navigate to with more complete information for each recorded program.
• Big Scrub Bar I like the idea of making very long scrub bars to allow for fine control over where in a program the user can jump. This is a good candidate for a control which is always along the long axis regardless of the orientation of the device, and will only be visible in play back mode panels.

As I said, I have very little time on hand, but I was able to make a first step of compiling my network protocol classes, which compiled after a mere 5 code changes—removing #include "Cocoa/Cocoa.h", and replacing call to Carbon's NewPtr(...)/DisposePtr(...) with malloc(...)/free(...).

Then it was just a copy/paste job to make a stripped down version of my original application delegate, which just stuffed replies from the server into a simple UILabel in my main window. Amazingly, this worked on the second try, with a connection forming with the server, followed by periodic requests for status. If anything, the networking code is more solid than in the desktop application.

I'll update this later, as I start to build the GUI, and figure out how best to minimize my power usage (keeping the Wi-Fi unit off for extended periods).

Port Model Train Software to OS X - Under the Hood

This is the second of two posts on my experience helping to port Train Player to Mac OS X. This is about my theories and designs for using OS X application technologies in a project which shares code with a Windows C++ application. This is not a code level look at such things, but a top level view. Also, while the framework was designed from the start to be cross-platform sharing code between Mac and PC, TrainPlayer for the PC is still using its old codebase, and may never make use of the techniques described below.

Why Is Cross-Platform Development Hard?

It may be a myth that the Russians chose a different railroad gauge to prevent invasion, but it is not a myth that commercial OS companies avoid open standard development APIs to keep their developers their developers. Neither Microsoft nor Apple cares much for Java, with Microsoft going so far as to invent a language, C^# which looks like someone took a thesaurus to Java. The careful software engineer can still write the bulk of an application in vanilla C++ with wrappers to native GUI and application frameworks. But you have to be very careful.

Rejected Paths

You can write C++ applications in the Qt framework and have it run on Windows, Mac OS X, and Linux. I've written three such applications and every day use such Qt applications as MythFrontend and a Perforce client. My problem with Qt is that I don't believe it possible to write a first class OS X application in Qt, you can make a serviceable Windows application in Qt, but then again Windows users are not known for their esthetic sense. Also, there are licensing fees for commercial development, and Qt does a lot of weird things with C++ to allow the visual design of interfaces, and as a side effect locking you into Qt.

We could have written the whole application in Cocoa with no re-use of the Windows code, using just the document schema. But, that's an awful lot of wisdom to throw away.

The Path We Took

What is more valuable, the code you write, or the code you get for free with a framework? The framework code, because you won't have to throw money and time at maintenance. Do not replicate what are given for free on a platform for the sake of sharing cross-platform code, even if it means replicating some bit of functionality on the other platforms, or horrors having the Mac version have a different behavior than the PC version. I can't tell you how many times I've heard the same excuse about how the manual will be confusing if they aren't exactly the same. Get used to it, Mac applications are similar to PC applications, but they are different and Mac users demand their differences. And, few read the manual.

For example, most applications have simple menu needs. You could setup all the menu items in TrainPlayer in Apple's Interface Builder in an hour. You could add a menu item in 4 minutes. It is not worth anybody's time putting together some elaborate framework of XML descriptions files so that you don't have to edit the menus in both Interface Builder and Visual Studio. Yes, occasionally you will forget to add a menu item to the other platforms; better that then maintaining 500+ lines of code for cross-platform menus. Such code would be especially problematic in Cocoa where many menu items are loosely wired to the first NSResponder instead of a document or an application classes. On the other hand, if your menu needs are complex--perhaps you have a dozen different versions of your application, all with different menu subsets--then its time to write yourself an XML schema and go to town. Thankfully, such was not the case here.

Of course, Cocoa makes it so easy to do just about anything in the GUI that every time this comes up you end up just whipping up a little objective C class to provide data to: the car collections dialog, the pre-defined layouts browser. If any of these did anything truly complicated, the proscribed action would be to create a cross-platform model class, but anytime the Objective-C to C++ glue code is nearly the same size as the actual C++ code, it's time to just keep that platform specific.

Which brings me to what, if anything, is cross platform here? Anything that involves manipulation of the document class and its data structures is cross platform. We created a subclass of NSDocument and had it host an instance of the C++ object which contains the actual document data structures. The NSDocument subclass does things like pass in menu commands and connecting the cross platform code to the main view in the document window.

Cross-platform GUI

Any view which is composed of non-standard content, i.e. isn't composed of buttons, text, sliders, etc, but is something special to this application is created by adding a NSView we called a LayeredView and giving it a C++ based DrawingSource. This would include the main document view, the clock, the control panel, the side view of trains in the control panel and in the toolbar, and the switch editing panel. All these views, and only one NSView class. All this rendered content, and the logic for it is entirely cross platform. On the PC, I've written a corresponding MFC class which provides the same function as a proof of concept.

The LayeredView object for each platform provides:

• Hosting for an arbitrary number of layers.
• Capture of mouse and keyboard events passed to the DrawingSource
• Drawing each layer when requested by the source.
• Popping up contextual menus
• Zoom level support

A DrawingSource object provides:

• A list of drawing primitives (more later) for each layer
• An affine transformation for each layer
• Mouse tracking
• Key press handling
• Determination of which contextual menu to show.
• Idle handling

Because the LayeredView on the Mac is an Objective-C object, the DrawingSource does not talk directly to the LayeredView but forwards requests for such services as layer invalidation, size changes, and screen coordinate conversions through a C++ DrawingDelegate object which the LayeredView installs in the DrawingSource when they are initialized together. In a mixed language setup, you will have to create these small interface objects anytime cross-platform C++ has to interface with some other language.

Drawing Lists

The DrawingSource provides a DrawingList which is just an STL vector of primitive graphic operations on primitive graphic structures. For example, adding a point to the current path is an operation, stroking the current path is an operation, drawing a bitmap at a given point with a given rotation inside a given box is an operation, etc. Lists have a number of advantages over the alternative of providing a wrapper API--providing a whole slew of drawing methods like FrameRect(), FillRect(), FrameOval(), FillOval(), FrameBezier(), FillBezier(), DrawText(), ClipPath()...--as graphic toolkits often don't map well to other toolkits. The PC TrainPlayer, for instance, makes heavy use of GDI Brush objects, to which a Cocoa programmer might say "What that?" and munge together some state object which approximates a Brush. With a DrawingList you are just creating a list of platform neutral instructions for drawing something, you don't have to worry about passing through a platform appropriate drawing context, you don't have to worry about flushing too often or not often enough. There will be a single routine implemented which knows all about drawing on the current platform and it will be optimized for rapid drawing. On Mac using the Quartz API, on the PC using GDI+. You could easily imagine alternative renderers based upon OpenGL or any other modern toolkit.

DrawingLists are conducive to layered drawing. Just keep a separate list for each layer. If nothing changes in that layer--maybe it's the layer of station switches in TrainPlayer and the current update involves the cars moving--then no need to recalculate it, just redraw it.

Another nice thing about DrawingLists, which I didn't take advantage of here because of all the legacy code, is that they can be generated on a separate thread. On multi-core machines (i.e.) all new machines, this can be a big win, especially if determining what to draw is computationally intensive. And each layer could have its own thread. Graphic toolkits tend to require drawing be done in the main thread, making any preparatory step which can be done in child threads helpful.

Data Types

I prefer using cross-platform data structures based upon STL or Boost, but sometimes this is not practical. Necessity and performance needs sometimes force the use of platform specific structures. You don't want to spend your rendering time converting raw bitmaps into platform native images. Therefore, I defined a set of macros:
#if _MAC
#include
typedef CGImageRef ImageBasis;
typedef CFStringRef StringBasis;
typedef std::string FileNameBasis; // treate as UTF-8
typedef CFURLRef FilePathBasis;
typedef CGAffineTransform AffineTransformBasis;

...

#define STRING_RETAIN(nativeString) if(nativeString != 0) ::CFRetain(nativeString);
#define STRING_RELEASE(nativeString) if(nativeString != 0) ::CFRelease(nativeString);
#define STRING_LENGTH(nativeString) (nativeString == 0)?0: ::CFStringGetLength(nativeString)
...
#else // not _MAC
#include
#include
typedef std::wstring FileNameBasis;
typedef std::wstring FilePathBasis;
typedef boost::shared_ptr ImageBasis;
typedef Gdiplus::Matrix AffineTransformBasis;
typedef std::string StringBasis;
...

In general, the cross-platform code get passed these structures, and they pass them through unchanged except for memory management issues through to the platform specific code.

And by the way, whatever string class you use, make sure any string the user sees is created and stored as Unicode; it's 2007 people!

Mac Specific Code

Rendering was done with the Quartz API with an assist from ATSUI for text rendering. Layers were handled via CGLayerRef (which TrainPlayer later simulated to get OS X 10.3 support). Utility windows, dialogs and the inspector floater were straight Cocoa using Core Bindings to interface with my NSDocument derivative. This was my first major use of Core Bindings, and I was extremely disappointed in its stability; the biggest problem I had in this development was trying to get around odd crashes deep in Core Bindings.

PC Specific Code

As I said, I only created a proof of concept drawing list renderer for the PC using the GDI+ toolkit embedded in an MFC application. GDI+ can do much of what Quartz can, although I missed the ability to cast shadows from arbitrary objects, and I had to do extra work to get layers.

Conclusions

I'm thankful for the opportunity to try out my theories on cross-platform development on a live target. I'm most pleased to come up with a methodology which has the potential to share large amounts of code between platforms, while still allowing me to create a first class OS X application. I'm especially fond of the drawing list design, and find it a good way to factor drawing calls even if cross-platform development wasn't important.

If I had had more time to devote to the project, I'd have worked at integrating technologies like the platform specific undo managers into the design, but with the birth of my daughter I have zero excess development time. Things could always be better, but I gave TrainPlayer a good start.

The Golden Age of OS X Independent Software

I've found myself buying a lot of software for my MacBook these days. It isn't because I'm swimming in money, because I most certainly am not. It's because I have software needs and wants, and my peers are out there satisfying them. Here is what I've paid my own money for in the last few months:

• Tables is a very satisfying spreadsheet which is nothing but a spreadsheet. It is very Mac-like, understated, stable and does what I expect of it. I chose it over Mesa, even though Mesa did charts because of its obvious emphasis on details. And now it does charts too, although they need a little work. I'd been using AppleWorks way past its expiration date.


• YummyFTP is exactly what I was looking for in an FTP client. Believe me, I tried many of its competitors before settling on this classy little Cocoa gem. I needed a client which could deal with an incredibly unstable Chinese FTP site, and this was up to the job.


• DVDPedia was recommended on the HT Guys podcast and I like it too. It has its share of interface issues, but a lot of craft has gone into it.


• Remote Buddy I've mentioned before. I know from experience that input devices are cranky things, but Remote Buddy makes handling them seem effortless.


• Super Duper!'s free features are so good, I haven't even paid for the premium features, but I probably will. After getting the heads up on this backup software from The Maccast I used it to transfer the contents of my hard drive before upgrading to a 200GB internal. Took a long while, but it was obviously very careful about protecting my data.

What do all this software have in common? Open their package contents in the Finder and you will see the unmistakable traces of Cocoa development. It's a lot easier today for one or two people to write an insanely polished application because we have Cocoa to handle the parts that every application does and we can concentrate on doing what makes our applications unique.

Porting Model Railroading Software to OS X From Windows

Last week, TrainPlayer International released a Mac OS X version of Train Player, their application for designing model railroads. I had something to do with this. I setup the general structure of the application at the start of the porting project, implemented a number of features, and worked into the debugging stage. But my real job has been nothing but debugging for the last two months, and I have nothing left to give to side projects, so they've had to get along without me.

This blog entry is the first of two on the project. This will cover GUI differences between a Windows application and a Mac OS X application, in terms of user expectations. The second will be on designing an application under the hood to share code between a Cocoa application and an old MFC application.

What Mac Users Want

The first thing I noticed about the interface was the surfeit of toolbars. With buttons for nearly every action.

When I worked in Illinois, we called this "Marketing Mode." Compare this with the left

and the right side of Apple's Pages word processor.

Apple Pittsburgh spent untold hours categorizing the actions required to lay out text, and distilled them into six icons. I can imagine the pressure they felt to add a special button here or there for someone's pet function. "What about a spell check button?" But it is important that toolbar icons be few and large in order that mouse acquisition times be short, requiring thoughtful selection of buttons. Notice there is plenty of room under the large icons for actual descriptive words; sometimes a word is worth a thousand pictures.

The second thing I noted was how often the user was required to open a modal dialog box, sometimes two, or even three nested modal dialogs to configure some user interface element. For example, to change the labels for a series of railway cars you would select a car, choose "Properties..." from the "Train" menu, to bring up this modal dialog:

which you would modify, hit "OK" and repeat for every car in the series. Modal dialogs are great for programmers; work flow is linear and controlled. If the user hits "Cancel" you just don't change your data structures, while if they hit "OK" nobody expects you to allow them to undo it; you've already given them a chance to back out. It's just so easy. But, Mac users these days are used to more.

The more they expect is a modeless, floating inspector window.

This allows the user to select, change and rapidly select and change again, all without losing focus on the document. It should allow for undoing changes (although apparently that functionality hasn't been added yet to Train Player, but should). The user should feel more comfortable and able to work in a more free flowing manner.

The Windows version used GDI graphics for its rendering needs, resulting in pixelated track paths like this:

I felt OS X's Quartz graphics environment would create much cleaner lines if we treated tracks as continuous paths instead of discrete segments and the Windows programmer was willing to make the changes needed to get nice anti-aliased and beautifully joined paths like these:

Similar effects could be achieved under Windows with GDI+, but I felt the typical Mac user would immediately see what was missing and demand higher quality. Quartz also allows any object to cast a shadow, making this effect: extremely easy to produce. Simple as it is, this translucent shadow makes the cars pop out and resulting in a more realistic look.

Along the same vein, Windows users see this clock:

while Mac users will see this clock with a composited reflective crystal and hands throwing shadows:


The Windows version had a modal dialog to choose individual cars from collections. It had two modes: an icon view and a table view, with each collection selectable via a tab.

I felt there was no need for two modes as the Mac table class was capable of displaying the larger image; and I am leery of using tabs to represent categories, especially when the user could install or create their own. Therefore, I replaced the tabs with a single column table as you might see in an iLife application, and I merged the two modes into a single table grid.


In other areas, I left it up to the experts at TrainPlayer, they know model railroading, whereas I only know generic applications. So when they say this is how you build up a set of tracks, this is how you build a set of tracks:

And this is how you run a simulation on a completed railroad:

I wish YouTube didn't add an ugly filter to its videos. You have to see how pretty it is.

What Was Achieved

In many ways, I'm proud of the result so far, and I expect the application to go through a rapid process of refinement now that it is available to a wider Mac user audience. Areas of the GUI where I varied it from the Windows original are definite improvements, some of which could be back ported to give the Windows user base a taste of Mac refinement.

In other places, I'm not thrilled. After I left the project, too many buttons crept into the GUI, and text fields really should support undo. Today's standards are high, it's hard to keep up, especially while reusing large chunks of C++ code, and bitmap images from a Windows application. The good thing is there is always version 1.0.1.

Porting is a balancing act. You've got to know what code and what GUI elements can be reused, and which must be re-implemented in order to avoid the dreaded "Looks like a Windows port" review. I think we've avoided that, and that's an accomplishment in itself.