Project the video of your iPhone app to thousands of adoring fans! This is an iOS 4.0 rewrite of the “iPhone App Video Mirroring” code described here: http://www.touchcentric.com/blog/archives/3. As I said last time, “Why can’t we be like Steve, using our apps on our phone, with the display projected for all to see?” Now you can.

I wrote the original code so members of our local iPhone Developer’s Meetup could show our apps to each other. Since then it’s been used by hundreds of developers. It’s proven invaluable for group demos, trade shows, client meetings, investor pitches, and so on. Sadly, iOS 4.0 completely broke the old code, so a rewrite was necessary. Happily, iOS 4 makes previously difficult things fairly easy.

TVOutManager running on an iPad

A bonus: this code is now entirely safe for the App Store. Zero private APIs are used. (You do have the option to use one, as you’ll see below, but it’s turned off by default.)

This code has been tested on iOS 4.0.1 on an iPad and an iPhone 4. If you find problems with other devices, please let me know in the comments.

What’s New

Like before, this is a fire-and-forget solution to mirroring the display of your iOS device (iPhone, iPod Touch, iPad) on an external display. The new version uses public APIs, so it can be used in apps published on the App Store. It supports detecting cabling (plugging and unplugging the display), orientation changes, and even offers a “tv safe mode” for displaying your app on an older analog video device. (You probably don’t even know what that is, because you’re a young whippersnapper who grew up with LCD flatscreen TVs, whereas I am testing this code on my old beloved Apple ][+ color display, and thus find the TV safe mode handy.)

Under the hood, new in this release is the TVOutManager singleton class. In the previous release, I’d made the TVOut manager code as a category on UIApplication, thinking that would make it easier to use and simpler to understand. Based on the emails I got, I think it was definitely simple to use, although not really simpler to understand.

So now TVOutManager is its own class. It’s a singleton, so you simply invoke commands on the shared instance. For example, here’s how you start showing video:

[[TVOutManager sharedInstance] startTVOut];

Stopping is just as easy:

[[TVOutManager sharedInstance] stopTVOut];

While the cable is connected, changes to the device orientation are tracked and the mirrored output is rotated (with a little animation effect).

How To Use It

It's crazy easy. Just three steps:

1. Get the files TVOutManager.m and TVOutManager.h from github and add them to your project.
2. In your app controller, add this to the other imports near the top: #import "TVOutManager.h"
3. In your application's -applicationDidFinishLaunching: method, add this line: [[TVOutManager sharedInstance] startTVOut];

Get the Code!

A sample project that includes the TVOutManager singleton can be downloaded from github:

http://github.com/robterrell/TVOutManager

Connections and re-connections

Last year our cable options were limited to composite and component cables used for playing videos on a TV. Since the launch of the iPad, Apple also offers a VGA connector cable, ostensibly iPad-only but which is what I used with an iPhone 4 while developing this code. I also tested with the composite out cable. Both cable seem to work fine. I’m not sure about iPhone 3GS and older devices, though — those devices may not support the VGA cable, although they certainly do support the composite and component cables.

While the TV output is running, if the video output cable is disconnected, the class will catch a notification, and stop sending video to the external window. Likewise, if a video output cable is connected, the class will catch that notification, and automatically start showing the video.

You don't even need to call -startTVOut -- just initialize the singleton sharedInstance, and the class will watch for cable connection notifications.

App-Store Safe!

Unlike the previous version, this class uses app-store safe methods. You can ship apps that use this code. We are using an Apple-approvied safe-and-friendly method of collecting bitmaps of all visible windows, and then drawing that bitmap on the external screen, which is also displayed using Apple-approved non-private APIs.

The previous version used the MPTVOutWindow class, which seems have to disappeared from iOS 4.0, and UIGetScreenImage() to copy the image of the screen. MPTVOutWindow has always been forbidden, but for a brief time, Apple allowed apps using UIGetScreenImage() onto the store (from December of 2009 until July 2010). However, with the release of iOS 4.0, Apple stuck the “private” designation on it again, and apps are once again being rejected for it. (Apple’s terrible communciation with developers generally means that developers learn this by submitting an app and getting rejected.)

So, here I’ve made use of UIGetScreenImage() an option. Why would you ever use it? For one thing, in my testing it’s quicker than the non-private methods. It also captures the entire screen display, including the status bar, which is handy for our particular use. UIGetScreenImage() also captures the output of EAGLViews, which the Apple-approved method does not (although Apple does have yet another method for getting that).

OpenGL Support

There was widespread belief that the pervious version wouldn’t work with OpenGL. That mistaken belief came from me — I didn’t realize that UIGetScreenImage() would capture the contents of an OpenGL view. In fact, it does, and it works great.

I dropped this new TVOutManager version into a simple test project made with the Unity 3 beta (http://unity3d.com), added one line to the AppController.mm file to call -startTVOut, and changed the USE_UIGETSCREENIMAGE constant to YES. The Unity game played on the TV screen. The framerate dropped from 30 fps to about 26 fps. Definitely watchable and playable. (I’ve gotten into a bad habit of playtesting Unity games on a big screen TV.)

Using UIGetScreenImage means you can’t ship this code; it’s just for demos. If you need to ship TV out code, you’ll have to change the class to follow Apple’s example of converting an EAGLView to a UIImage at http://developer.apple.com/iphone/library/qa/qa2010/qa1704.html. If there’s demand, I can add this to the TVOutManager in a future release.

Debugging with the Video Out Cable Attached

Obviously you can’t debug an app in Xcode without having the device connected via USB. Likewise, you can’t view the video output without having the VGA cable attached. So debugging was basically impossible.

Thankfully the latest Simulator includes an option to show an external display, but the simulator can’t start and stop this external display while the app is running, so it can’t be used to debug notifications like UIScreenDidConnectNotification.

To work around the problem, I figured I’d have to buy or make a Y-adapter cable: strip down the VGA cable to see what pins it used, and either wire in a USB connector, or make a full Y adapter that merged the pins required for both cable. Intensive Googling led to parts I could buy from Sparkfun. Reality set in (these days I don’t have much free time for tinkering and soldering) and I started to look for something a bit more prefab.

Reading some ancient forum posts about iPod dock issues, I found a reference to the Kensington K33368 “4-in-1 Car Charger,” which is a USB cable that includes a full pass-through dock connector, intended for an FM tuner. I guessed that they’d have wired all of the pins, and it would allow me to plug a second dock cable into the first. These are no longer manufactured, but a few online stores still had stock. I ended up buying one from a seller on eBay. It turned out to work perfectly. I plugged the Kensington cable into my Mac, and the RCA video out or VGA cable into the Kensington cable, and was able to debug away. I highly recommend this setup if you need to debug while watching the display on the big screen.

Improvements Needed

Currently, device rotation is correctly tracked for the Portrait and Landscape Left initial starting positions. You can turn the device 90 degrees to the left or right and the view on the external monitor will rotate to match. (There’s even a nice animated rotation of the screen contents on the external video window.) However, portraitUpsideDown is not handled.

It’s possible to confuse it with other starting orientations. This would be easily fixable if we knew what the previous orientation was, but since the UIDeviceOrientationDidChangeNotification doesn’t pass in the previous orientation (or even the new orientation — you have to get it from UIDevice) the code is essentially making a guess about how it should turn the mirrored view. This guess happens to work fine for all starting positions and turns I usually care about (i.e. portrait and landscape left, and 90° turns from these).

If you need to fix this, one possible solution is to track the device orientation at -init and across the lifetime of the object, and figure out the proper rotation of the view at each device orientation change notification. It seemed like a big hassle and I didn’t need it so I didn’t do it, but if someone can contribute it, that would be great.

Mirroring Other Apps

Adam Curry asked via email if it would be possible to run this in the background (in iOS 4.0) and magically mirror any app running on the device. I tried it, but it didn’t work. Seems like the external monitor support switches to whatever app is in the foreground. This is consistent with other apps that play video (i.e. the iPod app, StreamToMe, etc.) so I’m guessing this is just not possible without digging deeper into private APIs.

Compatibility Note

This class relies on certain APIs (i.e. [UIScreen screens]) that only exist in iOS SDK 3.2 and above. If you need to support older devices, you’ll have to use the older version.

The older version uses a class called MPTVOutWindow, which is either missing or renamed in iOS 4.0. This is what causes the older version to crash. The new version walks through the [UIScreen screens] array to find an external screen, and creates a window on that screen (using the UIWindow screen property). Being documented and all, this method appears to be fully future-proof.

TVOutManager Class Reference

-sharedInstance
Creates a singleton instance (if it doesn’t already exist) and returns it. Starts listening to cable connection/disconnection and device orientation notifications.

-startTVOut
Creates a window on the second screen at the highest resolution it supports, and starts a timer (at the frames per second rate defined in the class file) to copy the screen contents to the window. If no screen is attached, -startTVOut will simply report a failure to the console.

-stopTVOut
Stops the periodic video-mirror timer (or thread) and releases the offscreen window.

tvSafeMode
When tvSafeMode is YES, the class will scale down the output size by 20%, so that the entire picture can fit within the visible scan area of an analog TV. If you don’t know what an analog TV is, don’t worry, you’ll probably never see one.

kUseBackgroundThread
By default, the class will use an NSTimer to periodically copy the screen display. If you’re doing something that blocks the runloop, a background thread may work better for you. Simply change the define to YES and the class will spawn a thread to periodically copy the screen.

define USE_UIGETSCREENIMAGE

Set to NO to use the App-Store- and Simulator-safe method of capturing the screen. Set to YES to get the highest framerate and capture the status bar, OpenGL views and UIKit transitions.



If you listen to WUNC at all, you’ll be hearing about this on-air once or twice, I bet.

Hi I am a complete newbie to Xcode and programing in Cocoa and I was wondering if I cout get some help or a brief tutorial on implementing the UIApplication+TVOut.m in a small project and I am trying to build please any and all help would greatly be appreciated

Sure, it’s pretty simple:

1. Add the file “UIApplication+TVOut.m” (download link) to your XCode project. (i.e. Drag the file from the desktop into the project window’s list of files; check the “copy files” option if you want.)
2. In your project’s file list, right click on the “Frameworks” group item, and pick “Add existing framework”. Pick the “MediaPlayer.framework” framework.
3. In your project’s file list, there should be an app delegate file. It’ll be named whatever you called the project, i.e. “MyTestProjectAppDelegate.m”. In that file, find the method called “applicationDidFinishLaunching:” and add “[application startTVOut];” to it, like this:

That’s all there is to it.

That will generate a warning upon build, since we didn’t define “startTVOut” in a header anywhere, but it’s safe to ignore it. (If you’re the kind of person who always turns on “treat warnings as errors,” then there’s one more step — also add (and #import) the header file, UIApplication+TVOut.h.)

The goal was to have a single, flexible script. And specifically, I wanted something I could call from an AppleScript, which would place the results into a spreadsheet without lots of text parsing in AppleScript (because text parsing is tedious in AppleScript, although to be fair, everything is tedious in AppleScript.)

The script I came up with is mostly Ben Chatelain’s excellent perl script (again based on the work of Erica Sadun). To better fit my workflow, I modified Ben’s script in the following ways:

• Accepts the app ID, category, “free” or “paid”, and country name as arguments the command line.
• Returns just a number (the app’s rank in the given category), or “n/a” if it’s not ranked in the top 100 for that category/country combination.
• Arguments (except app ID) are optional — the script will assume “Top Overall”, “United States”, and “free”.)
• The script will cache the most recent data (and curl is invoked with -z). This way, if you call the script a dozen times in a row, it only pulls the page once, and the rest of the invocations use the cached data (stored in /tmp).
• To check the “Top Apps” overall rankings, pass in “Top Overall” for the category name.
• Likewise, to see the ranking for every country in the world, pass in “world” for country. The script will check the top apps for every country. In this case, the returned results are a string of “countryName: rank\n”, one per country.

You call it from the command line like this:

app_rank.pl appID categoryName free|paid countryName

rob$ app_rank.pl 300872165 News free
37

Or to see the results for the whole world (results edited for blog brevity):

rob$ app_rank.pl 300872165 News free world
Lebanon: 47
Saudi Arabia: 73
Pakistan: 74
Mexico: 71
Colombia: 61
Hungary: 89
Croatia: 19
...et cetera...

(How is my english-language not-localized North-Carolina-specific app #19 in Croatia?!?)

Here’s the AppleScript I use to populate a spreadsheet:

AppleScript to get the app rank and put it into Numbers

Here’s the spreadsheet it fills out. The AppleScript pulls the appID to look up from the first cell in each column, and then adds the timestamp and the app’s rank, so I can add new app IDs as I get them.

App rank data in Numbers

I scheduled the script to run every few hours (using Ligon) and so I’m slowly accumulating data. Of course, I can never sleep my laptop or I’ll miss some data. This concept would probably be best realized as something that runs on a server via cron. Since it’s a perl script that uses curl instead of some CPAN package it’s probably easily portable to just about anywhere. It would be simple to move this script to a server and collect the data in some other fashion.

Note that you can also use this same technique to track your app’s success against competing apps.

The script is here (3k zipped perl script).

I’m the organizer of the Raleigh iPhone Developer’s Meetup and it’s basically impossible for our group’s members to demo their apps. The “crowd people around a phone” method breaks at around four people. And our last meetup had 20 people!

Why can’t we be like Steve, using our apps on our phone, with the display projected for all to see?

I have a solution, if you’re willing to make a special build of your app that’s not meant for the app store. No jailbreaking required! Although a willingness to use private methods is.

Update: There’s a new version for iOS 4.0 here: http://www.touchcentric.com/blog/archives/123

How It Works

Inside the iPhone SDK exists a private API for video output. A class called MPTVOutWindow will display whatever it contains via the video output dock connector. (Thanks to Erica Sadun who discovered this class and figured out how to use it!)

I created a UIApplication category (that is to say, an extension to the existing UIApplication class) that uses MPTVOutWindow to mirror the device display onto the TV display. The goal was to make it seamless — drop the file into a project, recompile, and go. And it’s really just that easy, with a couple of caveats.

Caveat #1: apps that use this won’t be allowed into the App Store. That’s because it uses a couple of private APIs (the MPTVOutWindow class and UIGetScreenImage()). But since you’re already an iPhone application developer, creating a private build for your own use is normal.

Caveat #2: it doesn’t copy parts of the screen that belong to OpenGL. The class could be extended to do that, if you needed to.

Update: actually, it works fine with OpenGL. I demoed an OpenGL game using this at about 20 fps.

Getting Started

First, get a video out cable. It’s the same cable you’d use to watch videos from your iPhone on a TV using the dock connector (note: headphone-jack video cables from older iPods do not work). Apple’s official cable, Apple Component AV Cable, is $50 for composite video. A similar cable is only $14.18 at MonoPrice.com. Component cables are also available. `

How To Use

Add the file UIApplication+TVOut.m to your project. Also, you’ll need the “MediaPlayer.framework” framework, which contains references to the private API we’re using.

Somewhere in your code you need to call start. Anytime after your app’s primary window has been created, call startTVOut:

     [[UIApplication] sharedApplication] startTVOut];

That’s about all you need to do. The application category handles everything necessary to mirror the device’s display. The startTVOut method creates the MPTVoutWindow, adds the necessary subviews, rotates them to match your app’s orientation, and centers them to appear properly on the TV. A background thread copies the bits from your main display into a view in the MPTVOutWindow. (You can configure the frames per second in the #define at the top of the file. 12 fps works well for me.)

iPhone app displaying its output on a TV

In my test app, I call it from applicationDidFinishLaunching:. One thing to remember about applicationDidFinishLaunching: is that it gets called before the runloop starts, so your primary window isn’t on screen yet. I call it with a brief delay to give the runloop a chance to get going.

- (void)applicationDidFinishLaunching:(UIApplication*)application {	
// give the runloop a chance to start
    [[UIApplication sharedApplication] performSelector: @selector(startTVOut)
    withObject: nil afterDelay: .1];
}

And actually, you don’t even need to call startTVOut. The category can start itself. The code includes an override for a private method called reportAppLaunchFinished:. This undocumented method seems to get called after the app has finished loading. (Being undocumented, I’m not really sure what it does, or what the negative consequences of overriding it has. As far as I can tell, it’s only used to clean up the Default.png display.) I’ve commented out that method, since it’s a bit mysterious. But if you uncomment the method, it will automatically start the TV out display for you. No changes to your code needed!

Mirroring Other Apps?

Since this code runs as part of your app, it stops running when your app stops running. So it’s not a general purpose full-time video mirroring solution.

However, using other private methods, it’s possible to launch other apps while your app remains running, thus mirroring whatever app is in front. I’ve managed to mirror Safari (launched using the private openURL:asPanel: method). I’m sure you can figure ways to mirror other apps using some of the private app launching methods.

Not for Public Consumption!

Since this uses private methods, it would almost certainly not survive Apple’s app review process. To avoid getting your app rejected, you should remove this code from your release builds. This is pretty easy — set the current target to Release and uncheck the checkbox next to the file’s name. Also, if you call startTVOut manually, you’ll need to put some #ifdefs around that call, so it won’t get compiled into release builds.

Great, Where’s the Code?

Grab the latest code from here:

http://groups.google.com/group/iphone-developers-nc/web/UIApplication_TVOut.m

If you have any comments or questions, feel free to email me.

– Rob