DVD Main Title Size: Where the Gigabytes Go

For my personal edification, I've been surveying my DVD collection in terms of how much each movie uses on disk. This is the main feature only, none of the extra fluff, but it does include wasted space taken up by language tracks which I do not speak. This is movies only, not TV shows.


So far I've found 187 movie DVDs around the house. There are discs my son has reduced to coaster status, and I've no idea where ⅔^rds of the Indiana Jones trilogy ended up. I feel that is enough to get some feel for what the average movie uses on disk and how it correlates, if at all, with quality.


So some stats:

• Disks: 187 disks
• Average size of Main Feature: 5.10GB
• Largest Main Feature: 12.94GB over 2 Disks: The Return of the King
• Largest Main Feature on a Single Disk: 7.60GB King Kong
• Smallest Main Feature: 3.08GB Sleepless in Seattle (which had plenty of room for both Thai and Korean subtitles)
• Total GB: 948

Some surprises:

I have the Superbit version of Panic Room, the cheapest Superbit title I could find. Sony claims that Superbit is great because they fill up the whole disk with movie, making it as uncompressed as possible while allowing for both a Dolby Digital and a DTS soundtrack. Well, Panic Room clocks in at a measly 5.66 GB, leaving nearly 2 GB of room which could have been used for less compression. Having said that, the picture quality is quite good as it is. My other Superbit title, Adaptation, leaves around half a gig of empty space on the disk. Is it false advertising if you claim the highest quality DVD, and it turns out you could have made it even better by using less compression?

And how did Pirates of the Caribbean end up with only 3.61 GB? It was a special effects blockbuster with a huge marketing budget. They couldn't spend some tweak time getting a better picture?

Breakfast At Tiffanies is just the opposite, despite being quite pedestrian in its appearance, having only mono sound, and being none too long at 115 minutes manages to use up 7.18 GB. Where does it all go?



The disks that have the most movie on them tend to be major blockbusters: The Lord of the Rings, Minority Report, etc. where a lot of tweaking time can be spent getting everything to fit, or are just plain long (Patton). Disks with the lowest space usage tend to be catalog romantic comedies originally released pre-DVD. The studios didn't spend much time getting the highest quality disk, perhaps in the hope of coming back in a few years with a "Widescreen Collector's Edition", such as was the case with The Big Lebowski (3.54 GB), L.A. Story (3.57 GB), and Sleepless in Seattle (3.08 GB). By way of comparison, a short exercise DVD "Post Natal Yoga" clocks in at 3.94 GB.

[UPDATE: Here's a histogram of number of disks per size. The cut off for a single-sided, single layer disk, 4.7 GB minus a few trailers, is pretty obvious. I'm a bit surprised by the shape of the curve; if I had had to guess beforehand, I would have thought that there would have been two peaks, one just below the maximum capacity of a single layer, and another just below the maximum for a dual layer. But once you get passed the minimum for a single layer, it's almost as if it's following a random uniform distribution.]
[UPDATE: Updated for more disks surveyed.]

Logitech Tech Support Driving Me Up the Wall

I had written this entry weeks ago, meaning to do some more research before posting it, and I had forgotten it with my busy work schedule, but this blog entry: ill-Logitech seemed so similar to my experience I thought I should post this.

Before I begin this in earnest, I like Logitech products. I have owned many of their products: speakers, Harmony remotes, game pads, and cameras. They make the kind of gadgets I love to own. I am also not angry with Logitech or their tech support person who was courteous if pointedly unhelpful. This is just a story of spinning one's wheels in the pursuit of technological promise. I am also not including the text of actual exchanges with the tech support person at her request; you will have to rely on my characterizations of what happened. If anybody at Logitech is interested, this is Incident: 061224-000580

Now in earnest. I came across a cool racing simulator for the Mac, Redline by long time Mac gaming house Ambrosia. I had a good time playing it from the keyboard of my MacBook (1.83 GHZ Dual Core, 2GB RAM), and had some idea that if I added a steering wheel I could enhance my fun, and maybe give my wife, who's learning to drive, some simulated road time.

I went Googling and found that Logitech's MOMO racing wheel was well regarded and people had used it to play Redline on their Macs. I took a deep breath (maybe not literally) and ordered one from Amazon. It came, and like a kid at Christmas, I scrambled to unpack it from it's hefty box, mount it firmly on my desk, plug it all in and fire up Redline. Open up Options, Analog Controls, and what do I see but that the Force Feedback controls were disabled. Oh, it could see the wheel. I could calibrate the steering and braking, assign the horn beeping to one of the buttons on the wheel, etc, but I was denied the joys of having my steering wheel lose resistance when spinning out on ice, or stuttering in my hands when I slammed into a canyon wall at 120 MPH.

First things first, replug, restart, reboot, always greyed out.

I did notice that the motors in the wheel were working as whenever I plugged it into a USB port, the wheel would spin itself back to center.

Then I went to the Redline support forum, and found other people with the exact same problem, and others for whom the wheel worked as advertised.

And by advertised, I refer you to Logitech's Product Page which gives the following requirements:

• PC with 166 MHz or faster Intel Pentium® or compatible AMD® processor
• 64 MB RAM
• 20 MB available hard disk space
• CD-ROM drive
• USB port
• Windows® 98, 2000, Me, or XP
• Mac® OS X 10.2.3 or later*

* Requires game that supports force feedback. Check with developer

Needless to say my MacBook exceeds these specifications in every way, excepting the PC part or the Windows part. And the developer certainly says Redline supports force feedback.

I posted onto the forum and exchanged some information with the Ambrosia people, but they didn't know what was happening, and the copy of the wheel they had in house was working fine. I began to wonder if it was a bad batch of wheels or an incompatible firmware.

I sent Ambrosia the USB Prober trace for the wheel:


Low Speed device @ 3 (0x1D100000): ............................................. Composite device: "Logitech MOMO Racing "
Device Descriptor
Descriptor Version Number: 0x0110
Device Class: 0 (Composite)
Device Subclass: 0
Device Protocol: 0
Device MaxPacketSize: 8
Device VendorID/ProductID: 0x046D/0xCA03 (Logitech Inc.)
Device Version Number: 0x0019
Number of Configurations: 1
Manufacturer String: 4 "Logitech "
Product String: 24 "Logitech MOMO Racing "
Serial Number String: 0 (none)
Configuration Descriptor
Length (and contents): 41
*** omitted raw descriptor ****
Number of Interfaces: 1
Configuration Value: 1
Attributes: 0x80 (bus-powered)
MaxPower: 80 ma
Interface #0 - HID
Alternate Setting 0
Number of Endpoints 2
Interface Class: 3 (HID)
Interface Subclass; 0
Interface Protocol: 0
HID Descriptor
Descriptor Version Number: 0x0100
Country Code: 33
Descriptor Count: 1
Descriptor 1
Type: 0x22 (Report Descriptor)
Length (and contents): 87
*** omitted raw descriptor ****
Parsed Report Descriptor:
Usage Page (Generic Desktop)
Usage (Joystick)
Collection (Application)
Collection (Logical)
Report Count............ (1)
Report Size............. (10)
Logical Minimum......... (0)
Logical Maximum......... (1023)
Physical Minimum........ (0)
Physical Maximum........ (1023)
Usage (X)
Input................... (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield)
Report Count............ (10)
Report Size............. (1)
Logical Maximum......... (1)
Physical Maximum........ (1)
Usage Page (Button)
Usage Minimum........... (1)
Usage Maximum........... (10)
Input................... (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield)
Usage Page (65280)
Usage 0 (0x0)
Report Count............ (4)
Input................... (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield)
Report Count............ (1)
Report Size............. (8)
Logical Maximum......... (255)
Physical Maximum........ (255)
Usage Page (Generic Desktop)
Usage (Y)
Input................... (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield)
Report Count............ (3)
Usage Page (65280)
Usage 1 (0x1)
Input................... (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Bitfield)
End Collection
Collection (Logical)
Usage 2 (0x2)
Report Count............ (7)
Output.................. (Data, Variable, Absolute, No Wrap, Linear, Preferred State, No Null Position, Nonvolatile, Bitfield)
End Collection
End Collection
Endpoint 0x81 - Interrupt Input
Address: 0x81 (IN)
Attributes: 0x03 (Interrupt no synchronization data endpoint)
Max Packet Size: 8
Polling Interval: 10 ms
Endpoint 0x01 - Interrupt Output
Address: 0x01 (OUT)
Attributes: 0x03 (Interrupt no synchronization data endpoint)
Max Packet Size: 8
Polling Interval: 10 ms


I would think the data in this probe log would be useful to tracking down where the device was failling to be recognized. For example, the USB standard for Physical Input Devices indicates to me there should be another "PID" interface which could be used through a driver to cause a force feedback action.

Regardless, I thought it best to not bother Ambrosia too much; they are a small house and might not have the resources to track down such an obscure problem.

I e-mailed tech support at Logitech and shortly got a response asking me to perform a variety of actions, most of which were reasonable, of the "is it plugged in?" variety, but one really stuck in my craw and seemed to be an open invitation to give up bothering Logitech and go away. The tech support asked me to test the wheel with a PC game! And if I didn't, no tech support for me.

First of all, it's a near fluke that I own a PC. It will give you a hint to know my PC's network name is SCORNED. Second of all, they didn't say "Download this tester app from our website." They didn't even say, "Here's a list of games which we know work. Try a demo." No, they told me to find an unspecified game to test it on what could well have been my non-existent PC. Or no tech support for me.

Well, I mulled on that for a week or two. Meanwhile, I got an e-mail saying that if they didn't hear from me within 120 hours they would assume I was happy and close the incident.

I won an Amazon gift certificate in a drawing at work, so I looked around and found a popular game for the PC, GTR2, which was known to work with the Logitech MOMO, and I bought a $20 copy.

Of course, the MOMO wheel worked on the PC. I wasn't very impressed by how hard the force feedback was, but it was definitely moving, and the green force feedback lamp on the wheel was lit. In passing, I prefer Redline to GTR2. Realism is good and all that, but it's a lot more fun bouncing off of canyon walls than sliding around some ugly Formula 1 track.

I e-mailed Logitech support, confident that they would now be interested in "The Mystery of The Wheel That Didn't Work on a Mac." Not very much. They asked me to uninstall and reinstall Redline. I know uninstalling is second nature to PC people, but what exactly does that mean on a Mac? I trashed my preferences and overwrote the Redline application with a clean copy. Still no force feedback.

I e-mailed Logitech support with this datum, and was told it was obviously an application specific problem and I should go bother Ambrosia. I downloaded a cute little game called Jammin' Racer which has a picture of the Logitech Momo in their instructions. None of the promised rumbling. The lack of force feedback was not application specific.

I e-mailed Logitech to let them know I had disproved the application specific theory, and offered to do whatever USB Probing they would need to trace down the problem. I also asked if I could blog about this experience. The support tech said they preferred not, and (I'm paraphrasing) I should get lost. Actually, she said I should go bother Mac support. (Who is that? Apple support?) She did mention there was only 1 version of the firmware, so that shot down my working theory that some people had one firmware and force feedback, while others had a different firmware and none.

Warning, Mac programmger voodoo ahead. There is a ForceFeedback framework amongst the system frameworks, and I modified Apple's USB Notification Tool example to find the MOMO wheel and try to extract information about it. Surprisingly, the "FFIsForceFeedback" routine returned true, but the "FFCreateDevice" routine returned FFERR_NOINTERFACE, which I take it means the MOMO does not match the profile of a USB force feedback device. This is probably where Redline and Jammin' Racer fail.

I cannot say who is at fault for the wheel not working properly. The ForceFeedback framework is not well documented, and is probably not a priority at Apple. It's possible there is a hardware incompatibility between the USB ports on my MacBook and the MOMO. It's possible the MOMO is not properly following the PID specification. It's a complicated system with many failure points; I just know I paid for a force feedback wheel and didn't get one.

[UPDATE:] Reader Jonas writes to point out that perhaps the problem is with the LogitechForceFeedback.kext extension
/System/Library/Extensions/LogitechForceFeedback.kext/
which (apparently) comes pre-installed with OS X. His research indicates it is PowerPC only.
[Update 2:] Jonas gets back to me and tells me the kext included with Mac OS X 10.5 developer releases works to bring force feedback support to Intel based Macs according to the forums.

It's Time for Carbon Apps to Support RTF on the Clipboard

I don't know when the 'styl' resource was defined, but I do know it is very, very old. For those not in the know, back in the days of the Mac II, if you wanted to copy and paste bold/italic/underlined/colored/text from your application to another, you put both a 'TEXT' Handle on the clipboard, and a Handle filled with individual style records (the 'styl' flavor), with each record corresponding to a range of the raw text in the 'TEXT' resource. You could even put in foreign words and phrases by switching to lets say a Japanese font. This was what was done in 1987, it was what was done in 1997, and it is what most Carbon apps still do in 2007. I fired up iTunes Friday, copied something from a dialog, and looked at what was on the clipboard: a 'TEXT' resource and a 'styl' resource.

Well, what's wrong with it?

Here is an individual style record from TextEdit.h:
struct ScrpSTElement {
long scrpStartChar;
short scrpHeight;
short scrpAscent;
short scrpFont;
StyleField scrpFace;
short scrpSize;
RGBColor scrpColor;
};
And here are the things you can put in a StyleField:
normal = 0,
bold = 1,
italic = 2,
underline = 4,
outline = 8,
shadow = 0x10,
condense = 0x20,
extend = 0x40
What's wrong?

• No superscript or subscript, not even one level. Try pasting data from a scientific app to MS Word. H⁺₂SO₄^-2 becomes H+2SO4-2.
• It takes an old fashioned QuickDraw font ID, not an ATSUI ID, so it may be difficult to map to the fonts you're actually using.
• No paragraph alignment information. You can have any justification you want as long as it's left.
• Text encoding is by necessity implied by the QuickDraw font encoding.
• The OS has to go through and endian swap every element before pasting across Rosetta.

So what is the alternative?

There is nothing preventing you from supporting the 'RTF ' (Rich Text Format) flavor on the clipboard. It's a Microsoft invention, so you know Word will like it, and it satisfies all the problems I enumerated above. Plus, it's the favored flavor of any Cocoa application, so your text should be more accurately transferred to newer applications.

It's just a matter of implementing it. As I was already mixing in Cocoa into my Carbon application, I built up an NSAttributedString, extracted the string's content using the RTFFromRange:documentAttributes message and stored that raw data in a Handle. When I, inversely, wanted to paste, I extracted styled text information from an NSAttributedString created from the data in an RTF Handle. I did it all in a day for my application. (Sorry, can't share the code.) This has the added advantage of using the exact RTF flavor Cocoa applications get for free.

Alternatively, you could pretty easily create simple RTF documents from scratch using the description of the format in O'Reilly's RTF Pocket Guide. Parsing such documents looks to be a lot harder, but doable, just ignore the tags you aren't interested in.

I could hardly be happier with the result. My users will be getting a nice little present come the next release, and I'm one step closer to expunging the last bit of QuickDraw from my application. If you are maintaining a Carbon application which has anything but the simplest text formatting requirements, do your users a favor and look into supporting the 'RTF ' clipboard flavor.
[Entry has been edited for content since first publication]

The Freedom of EV-DO

My dad is an interesting, and smart guy. He left retail banking 20 years ago to start a software company in his basement, then transitioned into brokering the sale of small planes. And for much of that 20 years, he's sat in that basement, making sales call after sales call. I can't estimate how many thousands of times he's told a prospect to check "w w w dot global dash air dot com." As his phone, his computer and his dial-up Internet connection were in that basement, and he couldn't afford to be away from them, the basement became a prison of sorts.


And then a piece of technology came along to change this. He bought a Dell laptop (a Dell Inspiron E1405), and more importantly, he signed up for Sprint wireless broadband and bought a Sprint Mobile Broadband USB Modem. Suddenly, when combined with a healthy number of cell phone minutes, his business was located not in his basement, but wherever he happened to be. He could visit his friends in West Virginia and Florida. He could spend a week in my basement. He could even hang out with my mom while she was watching grandbabies all over the greater Twin Cities area. Technology has set him free.


And the EVDO modem is faster, even in Dad's rural basement, than his soon-to-be-cancelled dialup. He could stop waiting for the phone company to bring DSL one more mile out of town.


This is the sort of thing technology should be doing all the time. As an engineer, I am always on the lookout for ways to improve the lives of my users. The EVDO engineers at Sprint and elsewhere have markedly improved the lives of my Dad and others like him. It's why a lot of us got into engineering in the first place. Congratulations and thanks to them.

Netscape Plugin Development in XCode

A desperate Google searcher emailed me today asking about getting together an XCode project for a Netscape style plugin. Since this, apparently, is not well known, there is a NetscapeMoviePlugin project in the standard set of examples installed with your developer tools.


/Developer/Examples/WebKit/NetscapeMoviePlugIn/


One thing I would not recommend is using any of the QuickTime API calls in that plugin, they are laughably out of date. Nor was it obvious, to me, how you add things like JavaScript properties. But it is a start for an XCode project.

And I won't get started on what a sorry state of affairs that something so archaic as Netscape plugins are still used to add functionality to Mac browsers.

Building WebKit on OS X when you have Qt installed

I ran into trouble building WebKit (Safari) on my MacBook. When I entered the command to build it
% WebKit/WebKitTools/Scripts/build-webkit --debug
the build script immediately started complaining about qmake not understanding the "-r" option. This seemed really strange to me as I hadn't thought Safari was a Qt application; it certainly looks much better than any Qt application I've ever seen on the Mac, and I've written 3. It turns out that Safari is not a Qt application but the build script does look for the QTDIR environment variable and will try to build a Qt version if it finds that variable. [Thanks to Mark Rowe on the WebKit team for this information].

So the solution to my problem was to temporarily undefine QTDIR.

% echo $QTDIR
% unsetenv QTDIR
% WebKit/WebKitTools/Scripts/build-webkit --debug


Netscape Style Browser Plugin Development on OS X

If you are creating or maintaining Netscape style browser plugins on the Mac, you must get the nightly build of the open source version of Safari from WebKit.org. As someone who occasionally has been reduced to doing printfs to the system log in order to debug plugins remotely on a 10.3 box, it is luxurious having a debug build of the host application. Just follow the instructions for compiling a debug version and debugging it from XCode; if you have everything setup correctly you can debug the application and the plugin at the same time. Total setup time of less than an hour. It's fantastic.

I've tried to compile Firefox from source (using Visual Studio under Windows) and it was nearly impossible to get a debug build up and running. Actually, I don't know if it is nearly impossible or simply impossible since I gave up after a couple of days. Creating a Safari debug environment is a snap by comparison.

And the WebKit team is unbelievably responsive to bugs you find in WebKit itself. I've reported 2 problems over the last week and both were solved by the next day.

Chroma Keying BMP files in OS X

I've been contracted by Train Player International to help port their TrackLayer model railroad design software to OS X. It has been going quite well, and I'm grateful the proprietor, Jim, has allowed me to apply my theories of cross-platform development, and create a first class Cocoa application. But it is a work in progress, and there are both bugs and unimplemented features.

Regardless of its fragile state, Jim sent our development version out to select users for feedback and to prove it isn't completely vaporous. And feedback we got. Some of it positive, some negative, but the one thing everybody hated was the launch time; it was abysmal on older hardware and not particularly peppy even on my MacBook. I will not tolerate intolerable performance in any app I write, so I've made it my first priority to fix.

It didn't take long to track down the culprit, the reason TrackLayer was taking 13 seconds to launch on my MacBook. I was pre-loading 83 small image files representing train car tops and sides. These files were 24 bit BMP files with neither masks nor alpha channels; they were chroma keyed--a color was set aside to represent blank space--and my code was responsible for converting it to an a form with an alpha channel which could be composited with other images. Here's an example file of the same format:


My code replaced the pink area with transparency. The problem was with my original code to manipulate BMP images. I had even written a comment to myself to say it would be slow and inappropriate for heavy use. The major problem is that NSImages are not easily manipulated; their API is extremely limited in that regard. Basically, there is colorAtX:Y and setColor:atX:y which I had been using to change the chroma key color one pixel at a time:

// don't do this, it is slow
NSColor* clearColor = [NSColor clearColor];
int width = (int)originalSize.width;
int height = (int)originalSize.height;

NSBitmapImageRep *repWithAlpha = [[[NSBitmapImageRep alloc]
initWithBitmapDataPlanes:NULL
pixelsWide:width
pixelsHigh:height
bitsPerSample:8
samplesPerPixel:4
hasAlpha:YES
isPlanar:NO
colorSpaceName:NSCalibratedRGBColorSpace
bytesPerRow:0
bitsPerPixel:32] autorelease];

for(int x=0; x < width; x++)
{
for(int y=0; y < height; y++)
{
NSColor* originalColor = [(NSBitmapImageRep*)anImageRep colorAtX:x y:y];
if([originalColor isEqual:chromaColor])
{
[repWithAlpha setColor:clearColor atX:x y:y];
}
else
{
[repWithAlpha setColor:originalColor atX:x y:y];
}
}
}

It's slow just looking at it, what with the creation of an NSColor object for each pixel. And I needed to come up with something much faster. The problem is there is not a lot of documentation on doing what I want with either NSImages or CGImageRefs. There was this short section of the Quartz 2D Programming Guide, which recommends using the CGImageCreateWithMaskingColors function new to OS X 10.4, but isn't very clear on its use, and that's why I'm writing this blog entry to clear it up.

In order to create a masked image from my BMP, I had to:

• Create an NSImage from the BMP file
• Use colorAtX:y to get the color at the top left corner which I assume is the chroma key
• Ask Quicktime to create a CGImageRef from the file
• Use CGImageCreateWithMaskingColors to create a masked copy
• Convert the CGImageRef to the NSImage I can use

That's right, I create 4 separate versions of the image to get what I need. And it is still more than 10x as fast as the original.


NSColor* chromaColor = [(NSBitmapImageRep*)anImageRep colorAtX:0 y:0];

size_t bitsPerComponent = ::CGImageGetBitsPerComponent(originalImage);
float maxComponent = float((int)1 < < bitsPerComponent)-1.0;
float redF = rintf([chromaColor redComponent]*maxComponent);
float greenF = rintf([chromaColor greenComponent]*maxComponent);
float blueF = rintf([chromaColor blueComponent]*maxComponent);

const float maskingMinMax[] = { redF, redF, greenF, greenF, blueF, blueF };
CGImageRef maskedImage = ::CGImageCreateWithMaskingColors(originalImage, maskingMinMax);


Using this new code dropped the launch times on my MacBook from 13 seconds to under 2; pretty peppy.

I've put together a sample project: ChromaKeyTester.dmg which contains all the code for this, so I hope you Google searchers found what you sought. This only works with the variety of BMPs I needed to open, but it could be made more general.


[UPDATE]
I received an e-mail from a coder with evidently more experience than I at manipulating images in Cocoa. He made a number of suggestions, none of which I found worthwhile pursuing because I'm just not going to get my launch times much faster. I've gotten the launch time on my MacBook down to 1.5 seconds (and this is the worst case, where an image populated document is automatically re-opened, forcing the main thread to wait for the image loading thread to complete). Even if his conversion methods took zero time they could not push the launch time below 0.9 seconds. I realize 0.6 seconds on my MacBook might be 1.2 seconds on a PowerBook G4, but still, it's fast enough.

Still, other programmers might have a more dire need for squeezing performance, so here are a summary of his suggestions with my parenthetical reasons for rejecting them:

• Use a reasonable format like PNG instead of BMP (Can't do this for backwards compatibility. That was the first thing I asked.)
• Skip the reading top/left corner for chroma key color step. (Can't, different images use different colors.)
• My method leaves traces of the chroma key in the anti-aliasing (Sorry, that's just the test image. The actual production images are hand tweaked pixel by pixel with no anti-aliasing. And I could always expand the range of colors masked by CGImageCreateWithMaskingColors. )
• Do the conversion with a Quartz Composer script. (This is a cool idea, but sounds complicated to setup, and could you really load the Quartz Composer framework and execute a script in less than 0.3 seconds or so? Maybe, the Quartz Composer application launches in less than a second, and can merrily load and display my BMPs at a rate limited 60 fps and 7% rendering load.)
• Use a NSCIImageRep--which is a Cocoa wrapper around a Core Image Image. (I had not been aware of this very useful looking object, and this seems the most practical suggestion. If I have the time for extra speed tuning, I will explore this.)
• Get some C or C++ BMP reading code and read and convert the raw data myself. (This would have to be a big win performance wise for me to have to deal with the vagaries of the BMP format myself. And there would be no guarantee of a big win, as Apple's converter code is presumably optimized for Altivec or SSE3 or whatever.)

I will remind myself to read up on using NSCIImageRep and the Core Image framework. It sounds extremely useful.

[Another Update: Turns out there is code in Apple's examples for dealing with converting arbitrary bitmap formats to a reference format: either RGBA integer, or RGBA float. This is in the Tableau example for the Accelerate framework. Look for the message "constructIntegerReferenceImage". You can easily refactor this code to run through the reference image buffer and do your chroma keying before taking the step to create an NSBitmapImageRep].

Upgrading a 4GB iPod Mini to 8GB

I've been watching the DealRAM prices for 8GB Compact Flash modules. My intent has been to upgrade my wife's 4GB Pink iPod Mini which has sentimental value to me (in addition to having what I consider the perfect form for music playing). Well, the price of a solid state Transcend 8GB 120x speed compact module (model # TS8GCF120) just dipped to $100 at one of my favorite vendors: newegg.com, and it became time to pull the trigger. (It's actually become cheaper since I bought mine.)

There are plenty of instructions on the Web for surgically replacing an iPod's hard drive. I went with this instructional video. The only difference was the thinner form of the compact flash module. Also, I had to determine which way to plug the old drive's cable into the CF module. The trick is to look at the rail guides on the sides of the old MicroDrive, compare them with the rail guides on the CF module and plug the cable in maintaining orientation. Also, I reused the blue rubber bumper from the old drive to keep the new module from rattling around. I also took the opportunity to install a new battery.

As for performance, it seems to be working fine. It filled up with 6GB of data over FireWire in a reasonable amount of time (sorry, didn't time it, maybe 20 minutes). It just has a lot more space.

[post was updated for minor fact correction]

[Update: The upgraded iPod has been working fine for 10 months. Today, I installed a 2nd card, a Transcend TS8GCF133, into my own 6GB Silver iPod Mini. The original Microdrive failed and it was either upgrading or trashing. Seems to work just fine.]

Replacement Nipples (for the ThinkPad)

My wife has an X23 series model 2662 IBM ThinkPad. For years, she's avoided using the TrackPoint pointing device embedded in the keyboard as the standard rounded one was extremely abrasive and worn. She'd always attach an external mouse or (after meeting me) a trackball, or when in need using a piece of tissue to protect her finger. Well, dutiful husband that I am, I finally bothered to do a Google search if there was some replacement nipple or eraser tip for her model.

I was happy to find that Lenovo sells an inexpensive ThinkPlus TrackPoint Cap Collection. So for $10 shipped, I bought a variety of replacement caps, and she likes the wide flat type with the rim. Much more comfortable.

MythFrontend and HDHomeRun Wireless on a MacBook

[Update: check out Signal GH, an iPhone utility for monitoring the signal quality of an HDHomerun].

I've been running the MythTV Frontend for Intel OS X several hours a week for the last month. This is on my original MacBook (1.83GHz Core Duo, 2GB RAM) connected wirelessly via a 802.11g router to the basement MythTV server. I'm typically on the couch in the TV room upstairs, or on the bed in the basement; 30 feet from the wireless hub at most.

Performance has been good, and I've been happy with it, but it does lock up if you quit the application directly from Live TV mode. It's an odd sort of lock up in that the video keeps playing, but the beach ball of death cursor indicates the GUI itself is locked. So I'll command-tab out and force quit the application. Of course, the audio is nothing compared to the 5.1 speakers on the actual MythTV, but I have been known to watch even such sound intense shows as 24 on the MacBook, too lazy to watch it properly. Occasional flickers could be wireless instability, noise on the antenna, other processes grabbing the MacBook CPU or packets lost in Ethernet. It's hard to say.

I had worried whether my Ethernet network was up to the task of carrying 2 HD streams down from the HDHomeRun in the network closet to the MythTV (both the PC and the HDHomeRun are 100Base-T, the routers are 1000Base-T and the wiring is Cat-6) while simultaneously streaming a 3rd stream back up the same Ethernet cable to the wireless hub in the network closet. But, this works surprisingly well. I do like to give Live TV a couple seconds of buffer to give it a chance for error.

The GUI actually works better than the Linux version on my desktop, where I long ago lost the overlay GUI in TV mode. On the Mac, I actually see the Paused/Play time HUD window when I pause playback. Interlacing is not as good though.

In preparation for writing this blog entry, I put the MythTV frontend into a small window mode, and I'm just noticing it mirrors the video content in the OS X Dock. This is probably a bad idea, and a waste of cycles. I know it's cool and all, in a 2001 sort of way, but not that cool. Also, the UI for changing the window size is completely lame: holding down the right arrow key while it counts up to 480 horizontal pixels when I have a perfect good 4, 8 and 0 keys is unforgivable.

In summary, MythFrontend for OS X is nice to have, and I'm glad network and computer hardware is finally up to the task of this kind of high bandwidth application.

Be careful with your CFBundleVersion

This post is along the way of a reminder to myself should I ever have to figure out how my application's plist has caused the Launch Services manager to act strangely.

If you accidently setup your CFBundleVersion element badly, by for example, putting it in quotes, you will end up with the Launch Services database treating it as if it had a version of zero. You can see this clearly by running the lsregister utility:
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -dump

[Update: lsregister has moved in Leopard:
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -dump
]

Static Libraries dependent upon static libraries in XCode

Just a helpful hint to try and prevent your wasting time like we did at my day job. We have a large C++ application which has several different levels (Standard, Pro, Ultra....). So to speed compilation and linking, we put large chunks of code common to all the levels into separately compiled static libraries. Thus, we only have to compile that code once every time we build all the applications, resulting in a massive time savings.

Unfortunately, this was not working under XCode as when we linked in our libraries, there was always some symbol or other which was not available, and we ended up putting large amounts of code into the main project and compiling it for each application level (making our build process at least twice as long as it should have been).

My boss finally got so sick of this he spent a couple days tracking down the problem, and it was that our static libraries were dependent upon other static libraries, but those libraries were not linked in until the application as a whole was linked. Somehow, this confused the XCode linker (ld) and it ended up missing symbols. The trick was to link the other sub-dependent libraries into the libraries static libraries and not directly from the main project.

I know the above is confusing. Here's a more graphical description.
(Before)

• Main Project


• Static Library 1 Project (needs Static Library 2 & 3)


• Static Library 2 Project


• Static Library 3 Project

(Now)

• Main Project


• Static Library 1 Project (needs Static Library 2 & 3)


• Static Library 2 Project


• Static Library 3 Project

HDHomeRun - Networked HDTV Tuner

[Update: check out Signal GH, an iPhone utility for monitoring the signal quality of an HDHomerun].

I have a PCHDTV 3000 PCI card in the Dell Dimension which serves as my MythTV. Since DVB drivers became part of the standard Fedora install, it has been problem-free. But, it doesn't meet all of my needs.

First, there are occasions when the broadcast networks show 2 or even 3 good shows at once. I'd like to be able to catch these rare alignments, but my PCHDTV can only record one show at once. I could buy another, but that would be the last open PCI slot in the Dimension, and what if I wanted to expand internal storage later with an SATA card? Second, the MythTV was banished to the basement for excessive noise, which means it is about 80 feet of RG6 cable from the main antenna splitter in the second bedroom, meaning it is getting less signal, and noisier signal than an upstairs device. Third, to be useful, the MythTV has to be on all the time, which means draining a minimum of $9/month of electricity. Fourth, I've never been able to figure how to access a PCHDTV card with non-root level privileges; I'm sure it's possible, I just can't figure it out, which means I run the mythtv backend as root. This is frowned upon.

Then, I came across the HDHomeRun networked digital TV tuner (both ATSC and QAM). This simultaneously overcomes all 4 problems with the PCHDTV.

• It has 2 tuners.
• It doesn't use up a PCI slot.
• It can be located as close to the antenna as I like.
• It can be accessed via any user account with network access.
• I can access the data stream directly from my MacBook, albeit kludgily, even when my MythTV is off. [Update: It is no longer cludgy on the Mac. You can buy an HDHomerun bundled with El Gato's slick EyeTV application.]

It does cost more than a PCHDTV, but less than two.

I held out for a while, but I finally ordered one. It came Wednesday. I've been happy.

MythTV 0.20 has built in support for this device. However, be warned that you have to open your firewall to its traffic. If you try to run mythtv-setup and you can't get it to scan for channels. Maybe it's your firewall.

I was amazed by how well my MythTV box pumped massive amounts of data. At one time, I simultaneously recorded 3 HD programs over the air--2 over Ethernet from the HDHomeRun, and 1 via the PCHDTV--while watching a recording of Day Break from the night before, and it was smooth as silk. There have been a lot of improvements in MythTV over the last year; I don't think it could have taken all this back then on my aging hardware.

Lately, whenever I buy new hardware, I measure it's electrical usage. There is a hidden cost to every gadget we buy. According to my Kill-A-Watt, my HDHomeRun draws 6 Watts at all times, whether capturing or not [Update: I've heard that subsequent revisions (those you would buy new today) use less power] . A low power mode while awaiting network commands would be nice, but the usage isn't bad compared to the 90 Watts the MythTV box draws. So it's less than a dollar a month to power it. However low the power, it does get a little warm, and I worry about it's life span, but time will tell.

In order to control the device from my Mac, I downloaded the hdhomerun_config source code from the Silicon Dust Forum. It was a simple matter to create an XCode project and compile the command line tool for Mac OS X, and I could then use the instructions for using VLC to watch TV directly from the device on my MacBook. Entering a series of obscure command line commands is much too klunky, and I hope VLC will absorb these functions, but it did work and looked great on the 13.3 inch screen.

Dell 2005FPW with the LG LST-3510A HDTV receiver

I have a two year old Dell 20.1" widescreen LCD display (2005FPW). I also have three LG LST-3510A HDTV receiver/DVD players. Just so any Googlers might be curious, yes you can watch HDTV on your DVI capable LCD monitor and this receiver. No, you can't watch upconverted DVDs, at least not with the firmware in this particular 3510A (1.93), it gives me a warning about needing a HDCP connection and refuses to play. HDTV is a little distorted, as the 2005FPW is not a true 16:9 monitor (closer to 16:10), but it looks fine driven at 720p.

I like the 3510A as an HDTV receiver. I bought this particular unit for $49 shipped via an eBay auction, The low price due to the description saying it couldn't play DVDs well. Well, guess what? I don't care. It's a sensitive, reliable, flexible HDTV tuner and that is what matters to me. I can hook it up with DVI, VGA or component connectors to any number of display devices. Plus, it has both coax and optical digital audio connectors. I got another one cheap (not as cheap) with the preferred 1.91 firmware because the seller didn't know how to get it out of Spanish menu mode.

Here's its manual.

As is typical, neither eBay units came with remotes. Luckily, my third unit has a remote. Also, you can get most (not all, the DVD arrow keys are wrong) of the functionality from a cheap One-For-All remote. The One-For-All even has the codes for a proper fast forward/reverse on the DVD player, which the original remote does not have. Even the Harmony remote database has a few holes, so I'm glad to have at least one master remote.

Actually, I have little clue how people choose the proper eBay price for one of these things. I just saw an auction close for $58+shipping on a unit which was clearly marked as broken and being sold for salvage/repair. But it did come with a remote.

I don't really need 3 HDTV receivers. This third one is a family gift.

XCode debugger exited with status (1)

All of a sudden an XCode project which I was sharing with a collaborator refused to be debugged. The debugger was evidently exiting unexpectedly with "exited with status (1)" message. 

It took a long while to figure this out, but the project had been modified by my collaborator to hard code ppc generation. There were 3 such references in the project file. Had to change that to get the debugger to work again.

How not to use the boost singleton class

This is an odd and frustrating warning I was seeing while using the templated singleton class which comes with the boost C++ libraries while compiling under Visual Studio 2005. I used the following code to define a class which would be controlling a thread. It worked, but I was scared by the warning.



template< class T > class MySingleton : public boost::detail::thread::singleton < T >
{
};

class MyPayloadThread : public MySingleton< class MyPayload >
{

};



1>c:\src\hdrs\MySingleton.h(133) : warning C4624: 'MySingleton < T > ' : destructor could not be generated because a base class destructor is inaccessible
1> with
1> [
1> T=MyPayload
1> ]
1> c:\src\MySingleton.cpp(16) : see reference to class template instantiation 'MySingleTon < T > ' being compiled
1> with
1> [
1> T=MyPayload
1> ]


I did a lot of googling and couldn't find a direct answer to how I was supposed to give access to the singleton's private destructor. Then I had an epiphany about how I was supposed to use this class. The proper usage is:


typedef boost::detail::thread::singleton < class MyPayload > MyPayloadThread;

In retrospect, it's obvious that I couldn't sub-class singleton, I had to make use of it as a pure template.

New version of Remote Remote GH for MythTV

Just posted some fixes I did over the last several hours in a new release.

Blogger Dashboard Widget

Google has a bunch of Dashboard Widgets out today. Just testing to see how that works.

Setting up a Cheap Version Control Server for XCode

If you program, you need a version control system. We've all heard it, we all know it. But it's a trouble to setup and a pain in the pocketbook to dedicate a server, and even if you have a used computer free for the asking, it's still going to take up space and $10/month for electricity.

Because of a collaborative project, I felt an acute need for a version control system. My collaborator and I were spending much too much time stepping on each other's code trying to maintain a collection of source files via diffs and hand editing. So Sunday morning, I woke up with the resolve to set up a system.

My first thought was just to run a version control system on the Linux box I use for MythTV, but I've been rebooting into Windows XP every couple days to play Madden 06, and during the summer, when nothing much is on TV, I often turn it off to save energy. So it wouldn't be an ideal situation.

I thought of posts I've read about hacking NAS boxen to run Linux and a quick bit of googling came up with the consensus winner, the Linksys Storage Link for USB 2.0 Disk Drives NSLU2. This is a really cheap ($80), amazingly small, embedded Linux computer with 100 Base-T ethernet and 2 USB 2.0 ports. Plus it has an active development community of hackers extending what it can do.


I also decided to use a popular open source version control system Subversion (svn), if for no other reason that it isn't CVS, which I hate, or Perforce, which hates me.

Disclaimer: I am not a Linux expert. I am a decent Mac coder and software designer. You could very easily end up with a $80 paper weight, and not a good paper weight at that. Be careful.

Setting up the NSLU2

I am not going to go deep into setting up an NSLU2. I used the Unslung firmware, the instructions which came with the firmware, and the NSLU2-Linux HowTo.

Some important warnings about the process: test out the NSLU2 before modifying it. Bring it up in its unhacked form. Have it format your drive as EXT3. Make sure it works. Mount the shared volume on your Mac, copy some files, etc. The ethernet on the first unit I bought failed moments before I was going to flash its firmware, saving me from returning it modified. Think about your needs when it comes to hardware, I had a 60GB notebook drive in a compact USB 2.0 case lying on my desk; the capacity is fine for running a version control system, it's low power (it's actually powered by the two USB ports in the NSLU2), and I already had it. I used a cable tie to attach it to the NSLU2, and it became one, very portable solution. My Kill-A-Watt says the NSLU2 and the hard drive together draw 4W idling (9VA).



[Update:
One thing I didn't take into account while setting this up is spinning the drive down to save noise and wear. According to this page, this is problematic and should be looked into before settling on a drive. I'm sitting at my desk 3 feet from the NSLU2 and I can definitely hear a bit of sandy whirl coming from the drive. Of course, the beauty of a server is that I can store it anywhere with an ethernet port and a electrical socket, like next to the upstairs router.]

[Update: I replaced the 2.5 inch drive with a standard 3.5 inch external USB drive, and that has been going strong for 2 years now. Having said that, I should look into backing it up.]

Use the NSLU2 web interface to give yourself an account with the same name as your account on your Mac. This will make ssh'ing into the unit just a bit easier. In the text below, I will use "jim" as a user name.

Again using the web interface, Give the NSLU2 a static IP address in the range used by your router, for example 192.168.0.21 if your router uses 192.168.0.x IP addresses which is common.

Change the password for the admin account to something more secure.

Also make a group for all the collaborators of your project. Add yourself and the collaborator accounts to this group. This can all be done from the web interface. Make sure to give all these new accounts their own private folder.

At this point, you should have installed the new firmware, "unslung" Linux from running embedded to running off the USB 2.0 hard drive and been able to telnet into the box (Enable telnet from the web interface), and know your root password. Your NSLU2 is not going back to its old self, it's married to its hard drive.

I'm going to take over with the instructions.

Installing OpenSSH.

• Telnet as root into the NSLU2 and install the OpenSSH package.

#ipgk update
#ipkg install unslung-feeds
#ipkg update
#ipkg install openssh

• Reboot (unplug the NSLU2, plug it back in, push the on button, wait)
• Turn Telnet back on via the Web Interface.
• Telnet into the NSLU2
• Edit the ssh server configuration file at /opt/etc/openssh/sshd_config (I used #sudo vi /opt/etc/openssh/sshd_config)


...
#LoginGraceTime 2m
PermitRootLogin no
#StrictModes no
MaxAuthTries 6
RSAAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys
...

• If you like the bash shell, then
  #ipkg install bash
• Make sure your user account is setup to take you to your home directory with your favorite shell by editting /etc/passwd

jim:sc0MOzL8dVGa6:2003:501::sc3B6pVbVCgxc:/dev/null
to
jim:sc0MOzL8dVGa6:2000:501::/jim/:/opt/bin/bash

• Edit /opt/etc/init.d/S40sshd by changing

/opt/sbin/sshd
to
/opt/sbin/sshd -f /opt/etc/openssh/sshd_config

• Restart the NSLU2

So at this time, you should be able to ssh into the NSLU2 via a command like: ssh jim@192.168.0.21 Confirm this is so, and if not remember that you can probably get back in via telnet. A point about what we have done. We disabled ssh as root because hackers might get lucky guessing your root password. My /var/log/messages file is filled with miscreants trying to get in as root.

Setting up public/private keys

Y
[Update: I had had outdated instructions for setting up certificate based login here. OS X Leopard makes it much easier. Instructions Here.]

At this time, you should be able to ssh into your server without being asked for a password or passphrase as long as you are running SSH Agent. Now it's time to install svn.

Setting up the svn server

Subversion is a version control system. Version control systems are complex and have a high learning curve. Subversion is complex and has a high learning curve. There is a free O'Reilly book which covers its usage pretty well. At least read the Getting Started Chapter.

• Install the svn package


#ipkg install svn

• Copy the directory tree of the project for which you are starting source control onto the NSLU2 (let's say in /tmp/myproject)[Update: don't copy the build directory]
• If you haven't already, create a directory for your project.
• Make sure that your project group has access to this new directory.
• Import the project under the control of svn


#mkdir /MyProject
#chown jim:myprojectgroup /MyProject
#cd /MyProject
#svnadmin create /MyProject
#svn import /tmp/myproject file:///MyProject/myproject -m "initial import"

At this time you should have a project under version control. You just need to be able to access it from your Mac.

Setting up the svn client on the Mac

[Update: Try to install the same version of subversion on your Mac as you do on the NSLU2. I had trouble with version 1.4 on the Mac and 1.3.2 on the NSLU2.
svn --version
]
Here are two ways to install Subversion:

1) Go this page on apple.com.

2) Install Fink and use Fink Commander to keep installing packages until svn-client and svn-shlibs are listed as current. I found that fink balked at compiling all the pre-requisite libraries, but I could install those from binary and finally get svn itself to compile. It will take a while to compile.

At this point, you should have the svn client installed in your /sw/bin/ directory and be ready to checkout your project.

Checking out your project

• Create a clean directory into which you want to check out your project.
• Open up a Terminal window and cd into the newly created directory.

$svn checkout svn+ssh://192.168.0.21/MyProject This is where it was important that you use the same user name for both the NSLU2 and your Mac. Notice I didn't have to specify "jim@". It was all implied.

At this point, you should have your project checked out onto your Mac.

Setting up XCode

• Open up the XCode project file in your checked out project directory.
• In the Groups And Files panel, choose the topmost blue project icon.
• Right click on the project; select Get Info
• In the General tab, choose Subversion as the SCM System
• In the General tab, click the Edit... button. Enter /sw/bin/svn
• In the General tab, click Enable SCM.

At this point, you should be able to commit changes, revert changes, and other basic source control functions from XCode's SCM menu.

Allowing outside collaborators access to the svn server

Obviously, a big part of source control is allowing outside collaborators access to your code. That's one of the big reasons we've gone to the trouble of using the ssh tunneling method for accessing the svn server. It should allow safer access to your network. But people outside your local area network will need to locate the NSLU2 from the Internet.

To do this, you will have to have a router which supports DynDNS. I have a previous blog entry about setting up a DynDNS account. In addition to the setup there, you will have to have your router redirect port 22 (ssh) traffic to the NSLU2. Do not do this until you are satisfied you are secure; it's a jungle out there.

Also, your collaborators will have to setup their own DSA public/private keys for their own accounts on the NSLU2. An easy way to download their private key is via CyberDuck. I had initially recommended the old standby Fugu, but I didn't have a Universal binary version of Fugu and when I looked for one, found a lot of positive feedback for CyberDuck. Just be sure to choose the SFTP option for the new connection.


At this point, you should have a network setup where a collaborator could checkout your project using a command like:
$svn checkout svn+ssh://mydomain.homeip.net/MyProject

Good luck, and sorry if I missed anything.