Refitting with Core Data

I have a project InCDius 2 with which I demonstrate my coding skills, and update my Cocoa skills. I know I have some users out there, and I'm thankful for them, but it's mainly about me, about either getting me jobs or educating myself.

One of the big new technologies in Mac OS X 10.4 Tiger is Core Data. Core Data is gives you two things: it makes it very easy to create a document or a database for your your Cocoa applications, and it allows you to add functionality without writing code. I'm mostly interested in moving to an Apple maintained database framework; I've used Sleepy Cat's Berkeley DB libraries for the back store of InCDius and I have never been satisfied with the result. I do not have the time to learn the libraries complicated and obscure API well enough to give a rock solid and efficient user experience. I cobbled a usage together once, and it works OK and is quite fast, but people report occasional corruptions. I'd rather use a database with a pre-existing Cocoa API which Apple is actively developing, and I don't have to worry about getting a static library to link properly with a x86/PPC universal binary.


Anyway, I want to refit the database in my pre-existing application. This would be a lot easier if I were writing this from scratch or there wasn't an installed base of people with their carefully filled databases, or if some users weren't still on Panther. So I need to support both databases for a while.

So my needs are:

• Code to translate from an old database to a new one.


• Abstracted records where most of the application does not need to know how they were created or how they are stored


• GUI to let the user know I'm switching the database over.


• If I'm running under Panther or Jaguar, I have to be using the old database.


• If I'm running under Tiger, I could be using either database.


• If I'm running on x86, I need to be using only the new database, because I don't want to deal with endian issues or keeping two versions of the Berkeley static library compiled and linked in. It was painful enough to get it compiled in the first place on one platform.

As in most such things, the first and most important step is increasing the level of abstraction. The first rule of programming says that all problems can be solved with a high enough level of abstraction. We'll see.


BTW, if you are an InCDius user who's come across this entry. You can often fix database problems by opening up a terminal window and pasting in the following two commands (assumes that InCDius is installed in the Applications directory):


cd /Library/Application\ Support/Generally\ Helpful\ Software/InCDius


/Applications/InCDius\ /InCDius\ GH\ 2.app/Contents/Resources/ db_recover -c

WWDC - Some Impressions

I'm in the Detroit airport on a 3.5 hour layover awaiting the flight to Manchester after a week at Apple's World Wide Developer Conference. As a "rank and file" developer, I've a few observations of the conference as a whole.



Firstly, the big news was the announcement of the Intel transition. When it comes down to it, I just don't care very much. I've never done any Altivec programming, and I've never written a line of PowerPC assembly. If Apple thinks they can make future Powerbooks which are both cooler and faster than staying with PPC, well that would be very nice. I was already going to have to put in a huge amount of modernization/refactoring on my next project and this will just force the issue. I know there are people out there who've warred for years with PC bigots about which is better PPC or X86, and to them this is a premature surrender; luckily I don't have that kind of baggage. I also don't think it will affect my near term hardware buying decisions, I always knew that if I bought a new Powerbook, there would be one 30% faster within 6 months, and it's going to be a long while before Apple ships a 64-bit Intel PowerMac.



Secondly, there was no second biggest announcement. I was hoping for the release of dual core Powerbooks or some other improvement which would have some immediate affect on my computing experience. The keynote was a bit too short for my taste. No, "one more thing."



Thirdly, I am always impressed by the joyful professionalism of the Apple engineering staff. They love their jobs and are always friendly and helpful even when I'm struggling to even know what words to use in a question. I spent some time talking to the maintainer of the AppleUSBAudio driver, and he is a great young guy, who actually seemed enthusiastic about my little Turtle Beach project.



Fourthly, I think the developers who need to write in C++, and by extension have to use Carbon are getting tired of their second class status. I would love to use Cocoa and Objective C for everything too, thus the name of this blog, but sometimes on cross-platform projects, or where there is a huge amount of legacy code, that is just not possible. It would be helpful if Apple devoted more engineering time to keeping Carbon and Core Foundation on par with Cocoa. For example, the Core Data classes in Cocoa are fantastic, but they won't do a cross-platform developer a lick of good. On the other hand, people who don't use Cocoa just because the Objective-C message calling syntax makes them queazy should suck it up and just learn it.



Fifthly, the new generation of developers are so amazingly young. I talked to the Delicious Library guys and I don't know if they can legally drink yet. I feel so old.

The Thinkpad X23 and the Dell 2005FPW LCD Monitor

I own a Dell 2005FPW LCD monitor, which has a native resolution of 1680x1050. My wife owns a Thinkpad X23 model 2662. The Thinkpad uses an integrated ATI Mobility Radeon 7000 as it's graphics engine. I thought, "Hey it would be nice if I could plug my nice widescreen monitor into my wife's little Thinkpad." So I tried, and it didn't work because there was no 1680x1050 available in the Windows XP display panel.


Then I thought, that perhaps there was a more modern driver available, so I went to ATI's web site and it turns out that they do not provide drivers for integrated notebook video cards. So I went to IBM's web site which a lot of searching ended me up at Lenovo's driver page for the X23. I had some trouble actually downloading from the site from the Thinkpad, due to some Java problem, and ended up using my 12" PowerBook 867 Mhz to download the driver, and move it over to the Thinkpad with a thumb drive.


Installation was quick and easy, and successful. My wife can now surf the web in beautiful widescreen if she so chooses.


I also have a USB hub available which I can plug into the desktop PC, my PowerBook or the ThinkPad which with a single plug gives printing, external keyboard, trackball, and audio out. It's a pretty sweet setup.

Darwin Station

Well as luck would have it, Apple has open sourced much of their core operating system, including the driver for USB audio, as part of their Darwin project. So I downloaded the AppleUSBAudio-199.2.10 archive.


Getting it to compile was a bit of a chore, as they are transitioning towards using real 64 bit integers for things like time stamps, so I had to cast this and that, and change a few variable types. I haven't actually figured out how one installs a driver in place of its original.


But I did notice something strange, a probable bug that might lead to the driver not recognizing if a device could support an S/PDIF connection. In the AppleUSBAudioEngine::AddAvailableFormatsFromDevice method there is a for loop


for (altSettingIndx = 1; altSettingIndx < numAltInterfaces; altSettingIndx++) {
...
}


and then after the for loop, the altSettingIndx variable is used again to test for an S/PDIF interface, but I don't think it's a valid index and it looks like the test was supposed to be in the for loop.


I'll try to find someone at WWDC to confirm this is a bug.

What Does the Spec Say?

If I want to get S/P DIF bypass mode, I should first try to do it in a non-hackish, standard compliant way. So, what does USB.ORG have to say about the matter? Well, it does provide some info in the form of version 1.0 of the Universal Serial Bus Device Class Definition for Audio Devices, and more importantly the supplementary Universal Serial Bus Device Class Definition for Audio Data Formats.

What do I learn from this? Well, I find out that what I want is the device to support a Type Ⅲ format. A Type Ⅰ format would be normal PCM. A Type Ⅱ format would be the case where my receiver had a USB connector and took in AC-3 or MP3 files directly. In a Type Ⅲ format, the compressed format is sent along as pseudo-PCM which is what I need. Unfortunately, the specification says that if my device supports such a thing, it should have an alternate streaming interface exactly the same as the PCM interface, except the format type should be FORMAT_TYPE_III, and of course my AAM device has no such interface.

The other potentially useful bit of information that a Type Ⅲ stream “The non-PCM encoded audio bitstreams that are transferred within the basic 16-bit data area of the IEC1937 subframes (time-slots 12 [LSB] to 27 [MSB]) are placed unaltered in the two available 16-bit audio subframes per audio frame of the Type Ⅲ formatted USB stream. The additional information in the IEC1937 subframes (channel status, user bits etc.) is discarded. Refer to the IEC1937 standard for a detailed description of the exact contents of the subframes.” Well that is interesting, and if true, how is the receiver supposed to figure out that it's seeing a compressed stream? Or will a supporting USB device add this information back in for you?

Oh, I watched the director's cut of Daredevil on my PC using the AAM, and I got beautiful DTS 5.1. Given the importance of sound in that film, surround sound is the way to go. Also, the director's cut is the way to go; the story has much better flow, and makes more sense.

The Proof is in the Bi-Phase Coding

Well, I figured it might be helpful to figure out what my speakers expected to see if they were ever to play AC-3 files over S/PDIF. So back to Google. Ah, some company called Audio Precision puts out a series of technotes on audio electronics. Technote 26 is on "The Digital Interface," and it lays out what my speakers expect to see. (You need to register to get the full technote.)

Of non-practical interest to me, the signal sent over the TOS-Link uses Bi-Phase Coding which is a clever encoding scheme which insures that regardless of the data (all binary 1's, all binary 0's or any mixture) the receiver gets a constant stream of regularly spaced signal transitions with which to sync itself.

Of practical import is a little history. The S/P DIF standard did indeed only expect to transport PCM encoded, uncompressed, two channel data, and then at some point, someone asked whether it would be possible to transport multi-channel compressed audio over the same connection, using the same protocol. Would the wires, encoders circuitry, decoder circuitry, etc. care if instead of PCM style data, there were the bytes from a compressed audio stream? Well no, especially if a bit was set in every data frame saying that it was "invalid" audio, and if periodically a channel status block was sent out saying that the channel was sending out "non-audio". Bytes is bytes. A receiver which didn't know about compressed formats would see such a stream and go into mute mode awaiting valid audio. A receiver designed to play raw AC-3 files, would know to see if the stream being sent appeared to be an AC-3 stream. Newer receivers might even look for Microsoft audio streams, and future receivers might know to look for some futuristic format. The only requirement being that the compressed audio fit in a bandwidth adequate for uncompressed stereo, and that your receiver know about it. This extension (hack) to S/P DIF is called IEC61937.

So the question still remains, how do you set the invalid audio bit of the PCM frame, and the channel status packet?

It's in the Chips

As I mentioned in my previous post, the Turtle Beach Audio Advantage Micro uses a USB interface by some company called C-Media Electronics, Inc. A quick Google search finds them in Taiwan. This is the present and future of electronics manufacturing, Taiwanese companies both designing and manufacturing items branded by American companies. It's why PowerBooks ship direct to you from factories on Taipei.

C-Media is quite happy, I'm sure, to sell anyone with enough cash the exact same chip set they sold Turtle Beach, the CM102 USB Audio Controlller. And the provided technical documentation is quite interesting. It looks like almost all the needed functionality you would need to implement a AAM like device is on this single integrated circuit. All that would be needed would be a USB connector, circuit board, a variety of capacitors and resistors and a TOS-Link emitter package; it'd be fun to slap these things together if I did that sort of thing.

But not a lot of help towards knowing what the driver should send the device to get my goal of AC-3 playthrough. It does mention something called "Xear 3DTM Sound Technology," but that is probably something proprietary and useless to me, so I will ignore it for now.

My answers are elsewhere.

Investigating the Audio Advantage Micro

If you've installed Developer tools on your Mac you will find a useful little application called USB Prober.
/Developer/Applications/Utilities/USB Prober.app

If you have a AAM plugged in, and you run USB Prober, you find out what the Apple driver has to work with in configuring and sending it data.
Full Speed device @ 3 (0x19110000): ............................................. Composite device: "USB Audio"
Device Descriptor
Descriptor Version Number: 0x0110
Device Class: 0 (Composite)
Device Subclass: 0
Device Protocol: 0
Device MaxPacketSize: 8
Device VendorID/ProductID: 0x0D8C/0x0001 (C-MEDIA ELECTRONICS INC.)
Device Version Number: 0x0010
Number of Configurations: 1
Manufacturer String: 1 "C-Media INC."
Product String: 2 "USB Audio"
Serial Number String: 0 (none)

It's been a while since I did driver development, but let's break this down: it's a composite device which means it can be treated as two or more types of devices. For instance, it might be a outlet for audio data, and also be a volume control. The actual manufacturer of the USB interface chip was not Turtle Beach, but instead is somebody called C-Media Electronics, who've been issued a VendorID of 0x0D8C by the maintainer of a list of unique vendor IDs. and this is their first product using this ID.

Below this we have 2 interfaces. These are alternate pathways towards controlling the device. A driver will follow a very standardized series of calls over the USB bus to set up a communication pathway, these public interfaces lets the driver what format the device expects on the other side.

The first interface is given as Interface #0 - Audio/Audio Control
and a perusal indicates it is a standard USB protocol for controlling audio playback by giving a standard way to support muting, volume, etc. (I guess)

The second interface is given by Interface #1 - Audio/Audio Streaming
and basically says that this interface will accept a stream of PCM formatted audio data in an isochronous stream. The PCM is 2 channel stereo with a sampling rate of 48Khz. Very similar to what you get off of an audio CD except an audio CD is natively 44.1kHz. It's an isochronous stream which basically means the USB bus will try hard to allocate the stream a constant bandwidth, it's better for your mouse movement to be choppy than your audio playback.

But nothing about S/P-DIF. How does the PC driver send AC-3 to the receiver if it only supports PCM? Well there must be a way.

Necessity is the Mother of Drivers

I've been doing some online research on digital audio.

It all started when I purchased the device pictured below, a Turtle Beach Audio Advantage Micro, at a CompUSA store for $30.

What interests me about this product is I have a set of Logitech Z-5500 digital, 5.1, speakers, which have an unused optical input. The Audio Advantage has an optical output. So I can plug the AAM into my Powerbook 12" and get beautifully clean sound avoiding the auditory ugliness of a ⅛" mini-jack. And it works! iTunes sounds clearer than ever.

But there is something else I would like.

As of Mac OS X 10.3, the DVD Player application has an interesting option. If you have a S/P DIF output (which for all intents and purposes, the TOS-Link on the AAM is), and the driver for that output supports what is called bypass mode, the DVD Player can be set to not decode the AC-3 soundtrack on your DVD, it can be lazy and not convert the beautiful 5.1 soundtrack into old school stereo. Instead, the raw AC-3 file will get spit out the S/P DIF and into your stereo receiver, and if your receiver can handle it (which the Z-5500 most certainly can), you will get surround sound of the highest quality instead of lame stereo with a Pro-Logic II effect to make it sound sort of like surround sound.

So, the DVD Player supports it, the AAM supports it, my speaker system supports it. What doesn't support it? The Apple provided driver does not know about the AAM, and therefore cannot tell the DVD Player to send AC-3 streams through. The AAM supports this bypass mode on the PC, my desktop Dell sounds great, because Turtle Beach has written a driver for Windows XP. They rely on Apple's generic audio driver for the basic stereo support we Mac users get. If someone in the know were to write a Mac driver for the AAM, you could get surround sound out of a Powerbook, Mac Mini, eMac, iMac or iBook for $30+. PowerMac G5's already have an optical output.

As it happens, I have written a handful of USB drivers in my professional life, but never for OS X, and never involving audio. So I am investigating how I would go about writing a driver, and I will test out using the blog form to publish my finds here, if I abandon the effort someone might find my entries useful.

Of course, I'll get my copy of Mac OS 10.4 (Tiger) in a few days, and it's USB audio driver might support the AAM, and I'll say forget it.

On 5.1 sound

Well, it's quite late, and I spent the last 4 days not getting much work done. We are late in the development cycle at my day job and I am in desperate need of two weeks off. My problem with contract work is I have to pay for my own vacations. So I haven't gotten proper R&R in over a year. All those weeks of problem solving and design, of never turning my brain off, have caught up with me. Anyway, I have a new toy to distract me.

Dell had a low price on Logitech Z-5500 desktop speakers. My primary interest was the number of inputs: six. Four analog and two digital which means no more unplugging my PC to give an input for my Powerbook, etc. And it's my first foray into the wonderful world of Dolby Digital 5.1 sound. My Sony DVD player is more than happy to output 96/24 (whatever that means) over the coaxial digital line, so I can watch movies on my widescreen 20.1" LCD monitor while being immersed in sound waves. I watched the Two Towers tonight and I am just amazed how much proper sound adds to the effect. Sonically, it is more real than real. Amazing.

And this is with the subwoofer set to 1 on an 11 point loudness scale, and the satellites at 3. I wouldn't want to keep the downstairs neighbors up.

I also bought a cheap Turtle Beach Audio Advantage Micro USB to optical adaptor, which does not give my Powerbook 5.1 output unfortunately, but does give it beautiful, hum-free, stereo output. iTunes sounds better than I've ever heard it. What a bargain.

Welcome to My Soon to be Abandoned Blog

I wanted to post a comment on Anne Althouse's blog, which required me to setup this blog account. I doubt I will ever make use of this, but I might use it to run a diary of a Cocoa based multimedia application I have been considering writing. If I do the project, I'll be back. You can see my homepage at http://genhelp.com