Setting up and using Code Coverage

My background includes many years of Verification work on complex computer and graphics chips, so I was happy to learn that Xcode had some support for testing and code coverage.   Code coverage is very useful as a guide to developing tests to let you see what portions of your code have been exercised.

The following example code coverage results are displayed using the tool CoverStory.   They show what percentage of various classes have been exercised in the left pane, along with details about which lines in a particular file have not been covered in the right pane (highlighted in red).

Code Coverage Display 2

Setting up to gather coverage results

I found that it was recommended that one create a new configuration and a new scheme for gathering coverage results.   I’m using Xcode 4.6, so the following assumes that version of the tool.

Configuration

A configuration allows one to set all kinds of settings (build options, build locations, compiler settings, etc) while a scheme allows one to control different phases (Build, Run, Test, Profile, Analyze, Archive) and select the configuration to use for each phase.

Adding a new configuration wasn’t all that intuitive.  The answer in the post How to add configurations to Xcode 4? explained it pretty well.    Basically select your project and the Info tab.   There will be a little + sign below the Configurations section.  I chose to duplicate my Debug configuration and call it “Debug Coverage”.

Scheme

Select Product->Scheme->Manage Schemes
Choose drop down from little wheel icon and duplicate your apps scheme. I chose to call my new scheme ‘Coverage”.

For this new scheme, change all occurrences of Debug to Debug Coverage.

Customize your configuration

I got some background information about how to customize the configuration for coverage from the articles Covering it all up and How to Measure Code Coverage in Xcode.

What it boils down to is selecting the Project Configuration (or specific target if you wish) and locating the section Apple LLVM compiler 4.2 – Code Generation.

Change the following flags:
Generate Test Coverage Files: Select Debug Coverage configuration. Set to YES.
Instrument Program Flow: Select Debug Coverage configuration. Set to YES.

Note:  As of Xcode 4.5 you apparently don’t need to add -lgcov to linker options.

Application does not run in background

You need to add the key “Application does not run in background” to your .plist file and set it to YES.

This is discussed in the post code coverage with xcode 4.5 – no gcda generated as well as the CoverStory wiki page.

Generating coverage

Run the simulator and play with the app. After you are done, click the Home button on the simulator. Do NOT just stop the simulator.    Alternatively you can run a set of unit level tests.

Using CoverStory to see coverage results

Download CoverStory from the project home.

CoverStory Icon

Your coverage data should appear in a directory such as:

/Users/<user>/Library/Developer/Xcode/DerivedData/<app_name>…/Build/Intermediates/<app_name>.build/Debug Coverage-iphonesimulator/<app_name>.build/Objects-normal/i386

Getting there is a little tricky since the Library directory under /Users/<user> is normally hidden.    One way to do this is to go to Organizer->Projects. Click on the little right arrow (next to the directory path) for a project that has a Derived Data path. From there, navigate to the project/configuration you are interested in and follow the path provided above.  There should be .gcno (GNU Compiler Notes file) and .gcda (coverage) files.   Right click a .gcda file and open with CoverStory.

From there, open future files from CoverStory. It will remember the current directory to make it easy.

Next, use the CoverStory Open menu and find Objects-normal directory. Select i386 folder. All the coverage files will be loaded and presented.

Note: I found that sometimes I needed to exit and restart CoverStory to get it to open the folder.

Advertisements

5 responses to “Setting up and using Code Coverage

  1. Pingback: Setting up for Unit Testing | Finalize.com: My journey with iOS and other code adventures

  2. Pingback: Functional Coverage and Coverpoints | Finalize.com: My journey with iOS and other code adventures

  3. Pingback: The development of Contacts2Web | Finalize.com: My journey with iOS and other code adventures

  4. Hi there, There’s no doubt that your web site may be having internet browser compatibility problems.
    When I look at your blog in Safari, it looks fine but when opening in Internet Explorer, it’s
    got some overlapping issues. I simply wanted to give you a quick heads up!
    Apart from that, excellent site!

    Like

  5. Hi Timberland,

    I just checked the site with Windows Vista/IE 9, Windows 7/IE11 and Windows 8/IE10 and didn’t see any issues.

    Which OS and IE version are you using?

    Scott

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s