Thinking outside the box to debug: Charles Proxy, push and a forced crash

An issue on just one device

A couple of weeks ago, one of my colleagues reported that she found an issue with the iOS app for AAFES EXTRA, an app that I had developed for BlueSoHo for their client The Army & Air Force Exchange Service.

The iOS app was not displaying the expected updated content.  The Android app was working fine.  To make things more difficult, the problem was only occurring on one specific device and could not be replicated on any other devices, even of the same model.

Along comes a push

The AAFES app was using Urban Airship for push notifications.   This is a great, reliable service and the associated SDK makes for an easy integration (well documented with lots of nice features).  While satisfied with the service, we were exploring using Amazon SNS as possibly a more cost-effective solution for BlueSoHo’s needs.

In the course of testing Amazon SNS with the AAFES app (at the same time as the content bug appeared), I discovered a crash scenario with sending a push to AAFES.  It turns out that the iOS push code was relying on specific push related fields from Urban Airship (specifically an id field) and would crash if absent (I’m sure most of us have experienced a crash from inserting nil into a dictionary).

The crash bug should of course be fixed (though not a problem while using Urban Airship), but in the meantime I decide to make use of this crash to further debug the content issue!

Crashlytics logging

It happens that the AAFES app was well instrumented with crash logging.  An examination of the code showed that I had specifically added logging code around the fetch of content from the server and subsequent caching.

I suspected that either the fetch was consistently failing, or the cache code had a bug (cached content used if a fetch fails due to network connection issue, etc.).  Both seemed unlikely since other devices were not impacted.  An examination of a crash log would help to shed some light on this.

My goal now was to send a production push to this one device and purposefully cause a crash with a push message that omitted the Urban Airship id field (taking advantage of the crash bug scenario that was uncovered).

Sending a push from the command line

I found a nice article on about sending push messages.  It included a starter project and instructions on how to send a push from the command line.

The original article appeared at  but has since been replaced by an updated article (no longer using command line).   Using the Wayback Machine I was able to retrieve the original article.

I created a gist of the PHP command line script for easy reference.

Preparing the script to send a push just requires a few steps.

Create the PEM file

Open Keychain Access and export your push certificate to your Desktop, saving as a .p12 file.  Next convert the file to PEM using Terminal:

$ cd ~/Desktop
$ openssl pkcs12 -in myCert.p12 -out mycert.pem -nodes -clcerts

Edit the push script

Add your device token (see section below “Getting the device token”) and certificate’s passphrase (chosen during previous export) to the push script.

// Put your device token here (without spaces):
$deviceToken = 'ad648e26c765d78bad945eaf7eed7b192ffe6ed47fbfcfe973a11e7ed96931f2';
// Put your private key's passphrase here:
$passphrase = ''; Certificate Authority

You’ll need to make one additional edit to the script as mentioned in the answer to the post Push Notification in PHP using PEM file.  The referenced Entrust certificate can be found here.

Run the script

From Terminal execute the following:

$ php newspush.php 'Hello World'

Getting the device token

How do you get the device token?   Urban Airship does provide a list of device tokens and registration times on their portal, but unfortunately they are not sorted in any particular fashion, and with thousands of users this was not a feasible option.  If we were using a Debug build, we could check the Console log, but this is a Production build.

Luckily Charles Proxy can be used to accomplish this.

Charles Proxy

Charles enables a developer to view all of the traffic between their machine and the Internet. This includes requests, responses and the HTTP headers (which contain the cookies and caching information).  When running Charles on your Mac, you can point your device to this proxy to snoop on the device traffic.   This will allow us to capture the device token which in our case is registered whenever the AAFES app launches.

The following articles will help you to get started:

Please note that there was one extra step needed for setup as detailed in the article TRUSTING CUSTOM ROOT CERTIFICATES ON IOS 10.3.

Navigate to Settings > General > About > Certificate Trust Testings and turn the switch on for your custom certificate.

Once Charles was setup, I was able to get the device token by launching the AAFES app, filtering in Charles on “urban” and checking the API calls.   One of them included a request such as:

{{ “channel”: { “background”: true, “push_address”: “7d90f1b4da830ecaa1455ffab3b8e240f817a9aae5c1deb1927fb681221d51d8“, “opt_in”: true, “set_tags”: true, “tags”: [], “device_type”: “ios” }, “identity_hints”: { “user_id”: “xxF3fUpbSf2-W1276Fh41g”, “device_id”: “B47F3053-950D-44ED-ACBC-DA401F272786” }}

The device token is shown in red above.

Completing the Debug

I was able to force a crash and get a crash log.  The crash log showed me that the fetch of content had succeeded and that the cache was not used.

This left me with only one possibility that I could think of – the server was sending custom content to that one device.

Another session with Charles showed that this was indeed what was happening.  The content being provided to the device was different from all other installations of the app.

Time Travel

Knowing that the content was incorrect, I next searched for any difference in headers being sent to the server.   I found a cookie that included “time travel” in the name.   While I wasn’t directly involved in the effort, I then recalled a feature developed by our server team that allowed one to present content to a device from a different time period.   This feature allowed one to preview content that may not have been released to the general public.  The cookie could only be enabled in a specific fashion.

The issue here was that the cookie had not been removed after the time travel feature was used, causing the AAFES app content to be stuck at a certain date.







Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s