Year: 2008

Beta testing on iPhone 2.0

NOTE: It’s not clear if the information in this essay is covered by the NDA or not. The instructions presented here are either referenced on Apple’s own site or referenced in the publicly available version of Xcode. If requested, I will remove this post.

During Steve Jobs’ WWDC Keynote the announcement of Ad Hoc distribution meant one thing to most of us iPhone developers: beta testing. Unfortunately, the instructions for setting up this beta testing environment are not well documented. I used the following steps to setup a beta test for Twitterrific on the iPhone.

Updating Xcode for beta builds

This is the part of the whole process that eluded me the most. There is a new requirement to add entitlements to your Xcode project. Luckily, someone pointed me to this discussion on Apple’s website. It provides the following steps:

  1. Open up your ad hoc provisioning profile in Text Edit and verify that it contains a “<key>get-task-allow</key>”. If it does not, generate a new provisioning profile using the Program Portal.
  2. In Xcode, select File > New File…
  3. In the dialog source list, under iPhone, select “Code Signing”. Select “Entitlements” then click the Next button.
  4. Name the file “dist.plist” and put it in the root of your Xcode project. Click Finish.
  5. Open dist.plist and uncheck the “get-task-allowed” checkbox. Save the file.
Now that you have setup the entitlements, you’ll want to create a new configuration for your beta builds. I found it easiest to make a copy of my existing App Store distribution configuration and change the code signing setup:
  1. Make sure that you new beta configuration is the Active Configuration. Xcode has some problems if you try to modify Code Signing parameters on a configuration that is not active.
  2. Use Project > Project Settings to open the build settings. Change the “Code Signing Provisioning Profile” to be “Any iPhone OS Device” and then select the name of the provisioning profile you chose in the Program Portal. If you don’t see the name listed, make sure that the .mobileprovision file is located in ~/Library/MobileDevice/Provisioning Profiles and that the “Code Signing Identity” is set to “iPhone Distribution”.
  3. Use Project > Edit Active Target to open the target settings. Change the “Code Signing Entitlements” to be “dist.plist”.
  4. Clean the build and you should then be able to build a beta release.
Xcode can get confused when changing provisioning and other code signing settings, so don’t be afraid to quit and restart to get things synced up.

Adding beta testers

Of course the next step is to sign people up for your beta test. The most important thing to get from them is their device ID. Each iPhone or iPod touch has a unique identifier. You’ll need this to add them to the Ad Hoc distribution.

There are several ways to get this information. iTunes will display the Identifier if you click on the Serial Number after selecting the device in the source list. Pressing Cmd-C (Mac) or Ctrl-C (Windows) will copy the 40 character hex string to the clipboard.

If your beta tester is on a Mac, another solution is to use the iPhone Configuration Utility, which can be downloaded from Apple’s site. After selecting the device, you’ll see the same Identifier field which can be copy and pasted as text.

Finally, you can also use Erica Sadun’s Ad Hoc Helper. Your beta tester can use this application to send you an email with the device’s information directly from the device.

Creating a beta build

After collecting device identifiers from your beta testers, you’ll need to go into the Program Portal and add the devices. I’ve found that the easiest way to manage this is by using the tester’s name as the device name.

Once you’ve finished entering up to 100 device identifiers, you’ll need to add these devices to the your Ad Hoc provisioning profile (Edit > Modify.)

After you’ve finished updating the profile, Download the .mobileprovision file and move it into ~/Library/MobileDevice/Provisioning Profiles.

Now quit and restart Xcode so that it recognizes the new provisioning and perform the following steps:

  1. Select the beta configuration you setup earlier.
  2. Open your project settings with Project > Edit Project Settings.
  3. Update the “Code Signing Provisioning Profile” to use the name of the provisioning you just installed in MobileDevice.
You’ll notice that it’s quite a bit of work to update the provisioning, and since you’re modifying the contents of the project, you’re going to need to checkin .xcodeproj changes to your version control system. My recommendation is to get all your beta testers lined up and do it all at once.

Distributing and installing the beta

At this point, you can do a beta build and it can be run by your beta testers. The only thing left to do is getting them the software.

To be honest, I don’t know how this part works for users that are on Windows. I’m sure it’s possible, but you’re on your own as far as the steps involved.

Updated August 9th, 2008: Several readers have informed me that you can drag both the provisioning profile and the .app folder to the Applications section in iTunes on WIndows. Once there, they will get moved over to the device with the next sync.

For beta testers on the Mac, you’ll need to send them two pieces of data: a ZIP file that contains the .app bundle created by Xcode and a copy of the Ad Hoc .mobileprovision file in the MobileDevice folder.

To install the .mobileprovision file, the beta tester can just drag it onto the iTunes icon in the Dock. After unzipping the .app bundle, your tester can drag the application into Application in the iTunes Library. The next time they sync, the beta version of your application should appear on the device.

If your testers are managing a lot of different applications, they may find that the iPhone Configuration Utility is easier to use than iTunes. It has facilities to manage multiple devices, provisioning profiles and applications.

Hopefully you’ll find this information useful and allows you to increase the overall quality and reliability of your iPhone application. Happy testing!

Written on August 6th, 2008 for Development

Listeners found this review helpful

A major feature of the App Store are the user reviews about the software being offered. There’s just one problem: software is not music. I’ve never had an MP3 crash or lack features. Applications also evolve and improve; I’m pretty sure the Jimi Hendrix track I’m listening to right now is the same one he recorded in 1969.

The App Store in iTunes fails to address these fundamental differences between their latest offering and what has been offered previously (media.) There is so much potential here: iTunes could be a great way for developers to collect feedback and find problems. Instead, we get gems like these:

The icon to this App scares me so much… That I’m too afraid to install the App. That bird looks angry like it wants to peck my eyes out for even concidering [sic] to install the application.

If you are gullible enough to watch FOX “News,” then you are gullible enough to download this app and work for them for FOX for free– you already are in a way, just by watching. This would be a great app for those of you that like to monitor “ethnic” types when the nation goes to “Code Orange,” or, God forbid, “Code Red!” Make sure you have this app when you’re digging your bomb shelter or spying on your neighbors’ subversive activities.

What makes this worse is that flagging reviews as inappropriate content seems to have no effect. I have flagged reviews of my own products, and those of other developers, and nothing has changed. If Apple wants developers and users alike to take this system seriously, they must address this problem immediately. Yes, it’s tedious and costly to do this review, but with continued neglect this system will end up being like YouTube for software.

If you have doubts that this will happen, take a look at the most helpful review for Band. Users are already learning how to game the system.

Some have suggested that buying the app should be a requirement before leaving a review. I agree, but this will not completely mitigate the need to vet content. A large percentage of applications are free: the trolls will just download before going on their merry way.

If all of this wasn’t depressing enough for developers, I’ll leave you with my biggest disappointment: reviews are a one way street. I’m not one to feed the trolls, but many of the reviews I’m seeing would benefit from a “Just try this…” or “We’re working on that…” type of response. There’s not even a link to our support on the reviews page.

I remain hopeful that someone at Apple will see what’s going on and have the power to fix it. My only advice would be to act quickly: the longer you wait, the harder it will be to clean things up.

Written on July 16th, 2008 for Development, Observation, Opinion

Bugging

It’s pretty clear that the App Store is a huge hit. We’re all loving the ability to customize our iPhones and iPod touches with cool new software!

But with any big new release, there are problems that didn’t pop up while beta testing. As iPhone developers, we’re finding ourselves in a position where we can’t help customers who are encountering these issues.

Let’s take a couple of examples with our app Twitterrific. My friend Jeffrey Zeldman reports a problem where the application would crash just after launching. Another developer friend, Alex King, is having a problem with an authentication alert appearing when it shouldn’t. As a conscientious developer, I want to help these people and fix these bugs. The problem is that I have no tools to do so.

Jeffrey’s problem appears to be something with data that is stored on the phone. Alex’s problem is likely to be an issue with how data is being loaded from the network (from Twitter.) Note that I say “appears” and “likely”; I don’t know for sure, and that’s what is bugging me.

The first problem is knowing where the crash is occurring. The iPhone generates a crash report that automatically gets synced with iTunes. The crash report is stored in ~/Library/Logs/CrashReporter/MobileDevice. Unfortunately, this crash report is “raw” and developers don’t have tools to make it easy to understand (e.g. “symbolicating” crash reports only happens when they are loaded through Xcode’s device organizer.)

And even if I could interpret these crash reports, I’d be faced with another problem. There’s no way to gather additional information about what’s happening on Jeffrey’s and Alex’s device. With a desktop application that’s acting up, developers will often add logging and other kinds of output that track what is occurring around the time of a crash or other misbehavior.

In Jeffrey’s case, I would want to dump out the internal database that’s in use at the time of the crash. For Alex’s bug, I’d want to track the network requests to Twitter and the corresponding response. It’s easy to add this logging to Twitterrific, but the only way to retrieve the results is if you are a registered developer. There is no way I’m going to ask Jeffrey and Alex to spend $99 and install Xcode just so I can collect some debug output.

Assuming that I could get some debugging output, the next step in resolving these problems would be to create a special build with a proposed fix. After having Jeffrey and Alex verify the fix, I’d distribute that same build to a larger group of testers so that we could test for regressions (e.g. we don’t break some functionality in the process of fixing something else.)

The big problem here is that the only way to install software on an iPhone or iPod touch is with the App Store. There are also no provisions for beta testing. Without the ability to sign code, there is no way for a user to get code onto a device: most users fall into this category.

The only way to “test” a fix is to release the changes to tens of thousands of users. It’s the developer equivalent of playing Russian roulette.

(Note: there may be workarounds to some or all of these problems, but with the NDA in place, it’s difficult for developers to share their experiences and solutions.)

Apple has done an fantastic job with the tools that let us develop iPhone software. That’s clearly evident from the fantastic work we’ve seen displayed in the App Store. Unfortunately, the tools that developers use to analyze and debug problems are sorely missing at this point in time.

It’s our hope that this essay will do two things:

  • As a user, please be extra patient when developers tell you that they are working on a problem—it’s hard work at the moment and the time it takes to resolve an issue will be longer than with a desktop application.
  • We hope that bringing these shortcomings to Apple’s attention will help them address the issues and improve the iPhone SDK.

Until then, these problems will be bugging us all.

Updated July 15th, 2008: If you’re a developer, please feel free to submit a duplicate (“me too”) bug on the following Radars:

Updated July 16th, 2008: After fixing bugs for customers, Brent Simmons notes that there are frustrations with the final part of the development cycle: making public releases.

Updated July 23rd, 2008: Brandon Sneed has discovered some techniques for doing remote debugging of iPhone applications.

Updated August 6th, 2008: We don’t have to play Russian roulette anymore.

Updated August 8th, 2008: I discovered how to “symbolicate” crash logs.

Updated November 10th, 2008: Getting preferences and data from customers who are having problems just got a lot easier.

Written on July 14th, 2008 for Development, Observation

Known issues…

First off, thanks to everyone for the kind words about the new version of Twitterrific. It’s really great to get all this awesome feedback! Don’t forget to post a review on iTunes: we want as many stars as possible :-)

As with any first software release, there are a few known issues:

  • After you upload a picture to TwitPic, the keyboard disappears. This is an Apple bug which we can’t fix (they are working on the problem.) In the meantime, you can try to tap on the text area and when the loupe appears, so will the keyboard. If that doesn’t work, try tapping the “Close” button and then tap the Reply or Post button again to refresh the interface (your text will be preserved.)
  • Some images uploaded to TwitPic are not rotated correctly. Again, this is a bug with the EXIF data that Apple is putting into the JPEG image. We are working on a workaround now, but didn’t have it ready in time for the launch. We’ve also heard from TwitPic that they are working on some AJAX code that will rotate the images directly in the browser, so that may help in the short term.
  • Scrolling is jerky. Because we have links and variable height rows in the list, we are pushing the limits of the processor. We have some ideas of how to improve this in the future, by only displaying the text of the tweet after the scroll finishes (similar to how the avatar images work now.) This is really tricky code to write and we had a very tight deadline—trust me when I say it bugs us, too :-)
  • The Sigur Rós ad from The Deck displays incorrectly. The root of the problem is that Twitterrific can’t handle HTML entities. It could, but that would compound the problems with jerky scrolling mentioned above. We’re working with the folks at The Deck to change this over to UTF-8 so it will work OK on the phone.
  • Entering a “+” sign in your tweet ends up as a space after being posted. It’s surprising that no one typed this character in several months of testing. We’ll fix this ASAP.
  • Launching the application can cause a restart of the iPhone. All indications at this point are that this is a problem with the iPhone OS and not Twitterrific (it’s happening with other applications.) We are working with Apple to resolve the issue. Please make sure to send any crash reports if prompted by iTunes. No matter how much you beta test, there’s always one problem that slips by—even if you’re Apple.
  • Sorry, but we can’t offer discounts or free upgrades for the iPhone version to current Twitterrific users. There are no provisions on the App Store for developers to do this. Please contact Apple if you think there should be a way to do this—we already have :-)
  • The push notification service announced by Apple during the WWDC keynote will not be available until later this year. We don’t know if Twitterrific will be able to support it or not: it requires additions/changes at the source of the data (Twitter) and the volume of information may be too great for Apple to handle. We are definitely keeping an eye on things and talking to the right people.

If you find something that’s not on this list, please let us know by using the contact form at the Iconfactory. Select “Support” in the drop-down menu. Thanks!

Written on July 11th, 2008 for Miscellaneous

Plug-ins the Cocoa way

For many graphics applications, plug-ins are a way of life. This essay presents a method that minimizes the amount of work required to support different host applications, or even different versions of the same host. I used these techniques in our IconBuilder plug-in for Photoshop. These instructions have already helped one fellow developer; with the help of Google, hopefully there will be more.

My goal was to use Cocoa and NIB files produced by Interface Builder for driving the plug-in user interface. As I began to work on this project, I also realized that it was possible make the user interface independent of the underlying binary file formats: a CFM shim can load a Mach-O bundle.

Here are the steps to accomplish the task:

  1. Photoshop loads the plug-in which is built with Carbon.
  2. The Carbon plug-in searches for a Cocoa bundle in the plug-in’s resources with CFBundleGetBundleWithIdentifier and then loads it with CFBundleCreate.
  3. Once the bundle is loaded, I get a function pointer to the Cocoa code to execute using CFBundleGetFunctionPointerForName.
  4. After I’m in the Cocoa code, I create my own NSAutoreleasePool and then call NSApplicationLoad(). At this point, you can work with controller objects to get things setup and call [NSApp run] to get a run loop going.
  5. From this point on, I can use all the Cocoa and Foundation frameworks. Interface Builder NIB files work fine, too.

I use the Carbon code base with both Codewarrior and Xcode to produce both the CFM and Mach-O versions of the shim.

The biggest advantage to this approach is that your user interface and processing code is independent of the glue code that passes data back and forth to the host application. You could, for example, easily adapt the “plug-in” part to work with Aperture’s new plug-in architecture and not have to worry about 99% of the rest of the code. We take this approach with our IconBuilder product: PS7 and CS3 have slightly different plug-in specifications, so we offer two plug-ins that use the same UI/processing bundle.

Another advantage to this approach is that you can do testing without having to run under the host application. I use a small test harness that loads the UI/processing bundle and feeds it data. This makes it much simpler to work in a debugger without the host application getting in the way. You could also use a similar technique to build a standalone application.

Finally, there’s much better documentation than when I figured all this stuff out. Here’s something at ADC that will help get you started.

Written on July 8th, 2008 for Advice, Development
Newer
Older