It's update time! As we rapidly approach release, this month has been mostly about our beta test, learning from it and prepping Brili (the product) as well as Brili (the company) for launch. Here's how it went down.
In anticipation of assisting our customers during the beta period, we set up UserVoice-powered support site at http://support.brili.co. It can also be reached from the main navigation on this site as well. There, we posted:
- Feature Request Voting
- User Support Forums
- Getting Started Guide
- iOS Beta Testing Guide
- Known Issues List
It's also configured to track as tickets all support requests sent to firstname.lastname@example.org. So far, it's been a good system, though in hindsight I would have done a better job of publicizing it to our users. Anyway, check it out. If the answer you need isn't there already, it'll be there soon.
Setting Up Analytics
While a beta period results in some really useful customer feedback, not everyone takes the time to tell you how the app is working for them. In order for us to have an actionable view into how customers are generally succeeding (or not) with the app so we can make any necessary tweaks to the user experience, we integrated MixPanel analytics. It's a pretty powerful system that summarizes customer usage patterns and lets us automate communications to them based on their actions in the system.
The first automated message we set up was a simple one: a day after signing in for the first time, we ask how it's going and if there's anything we can do to help.
Kicking Off The Beta
After a couple of false starts due to not really understanding how to correctly version our app builds in Apple's new system, we figured THAT out and then successfully received approval to start the iOS beta on March 6th. Because we could, we also opened up access to the web-based version of Brili so families with non-iOS devices could try it too.
Kyle and I had previously vowed that we would not get a haircut (in my case, shave my beard, 'coz I'm bald) until we launched the beta, so this occasion was doubly momentous.
We promptly sent a communication to all pre-registered customers, inviting the early adopters among them to give Brili try, providing them with a web form on our site to confirm participation and the device they wanted to test with.
The old digital marketing skills from my previous life got a bit of rust knocked out of them when I realized our open and click-through rates on the beta announcement email were, though well above marketing industry averages, far from 100%. So not everyone got the message. Be that as it may, we ended up with a couple of hundred intrepid families, educators and behavior professionals signing up, which is fine for our immediate purposes.
Once a day as the registrations came in, I updated our customer database, sent new beta users a special "What to expect next" email (which also got a less-than-100% open rate) and then imported iOS testers into the iTunes Connect system.
Fun with iOS8 TestFlight
On the iOS front, we quickly realized that while it's good to have a system like TestFlight for pre-app store testing, it's a bit awkward for the average user to figure out. It involves upgrading to iOS 8 for some, then downloading the TestFlight app from the App Store, then waiting for the invite email from the system.
Since the email is sent by the iTunes Store and not Brili, our testers needed to be prompted to look for it. As a test, I sent invites to 16 friends of ours, but I didn't tell them to look for the email. Only two of them (the only people over sixty years old - go figure) noticed it and installed the app. Everyone else had to search through their trash after I contacted them to check. Anyway, this led to my including a very explicit "Look for an email from iTunes" message in my communications.
My dad found a weird bug that caused the app to seem like it hadn't saved the user's changes when it actually had. We promptly fixed that. The next day, a user complained of blank screens and a generally nonfunctional app. GASP!
I got on the phone with her as soon as possible, and she kindly walked me through the steps to reproduce the issue. It turned out that when she was grabbing a slider we use to set point values on tasks, the mobile front-end framework we were using was interpreting that as the user's intent to swipe the whole screen to the right. Since we hadn't actually implemented any such feature, the screen was swiping to reveal blankness, and then everything broke.
Kyle figured that one out over the weekend and we tried to push a fix the day that Apple had its big iTunes Store and iTunes Connect outage. Not only that, but our web hosting provider was simultaneously upgrading their system, preventing us from deploying the new code to the web version for over a day. When it rains, it pours!
We also ran into challenges with our MixPanel setup. It was collecting most of the data we were asking it to, but not in a useful way. We couldn't easily see, for instance, which users were getting started okay with routines versus those who were getting stuck.
It was all fixable but meant deploying new builds to iTunes and hoping customers installed the updates. Unfortunately, not everyone does this promptly, if at all: imposing new action items on customers is generally problematic.
Slowly, with the help of customer emails and despite our analytics snafu, a picture of our onboarding results began to form. It wasn't as pretty as we'd hoped.
(Onboarding is the process of getting users completely up and running with an application so that they're starting to realize the value that it offers.)
Though users could easily figure out how to set up routines, our first attempt at the user flow had almost everyone getting confused about how to actually use them with their kids.
In our defence, we saw this coming. When we developed our user experience (UX) design initially, we had planned on keeping the parent and kids apps as completely separate. When we started developing, though, we realized it made more sense to have parents and kids use the same app, with a "Parent Mode" and a "Kid Mode".
So while we had built in a way of switching from one mode to the other, we hadn't had time to make it easy to find and intuitive. We didn't want to delay kicking off the beta test, so we went with what we had in the hope that we could explain it in the introductory emails and docs.
Well, it turns out people generally don't read stuff. (In fact, I'm really thankful if you've made it this far into this post.)
Another way we were messing up our users was a result of not having a version of "Kid Mode" that worked on a handheld device yet. This meant that parents could set up the routine from an iPad and run it from the same device, but if they set it up from their phone, they had to log their child in from a different device (something that being in TestFlight beta didn't simplify at all, because of having to install again.)
Anyway, all this meant I always had to provide a long-winded answer to the question "how do I use this with my kid?" Suboptimal.
Rather than look for ways to explain these issues, provide instructions and hope for the best, we decided on Monday to devote a few days to address root causes. We had always intended to do this, just not in this order. The reprioritization caused the postponement of payment processing integration because, quite simply, there's no point in trying to bill people for stuff they're not deriving value from.
- We promptly got our freelance designer and animator, Juan Hernandez, to rework the "Manage Kids" screen in Parent Mode so that parents could easily enter Kid Mode for a specific child from there with a single tap.
- Kyle got to work on setting up a demo routine that gets created automatically when a user creates a profile, so parents can try Brili in Kid Mode right away to see it working.
- Jill started making Kid Mode completely responsive so it can work on any sized screen, including handhelds. No small task, but she's done it!
- I mocked up an onboarding flow that includes instructional screen overlays and pop-up callouts pointing to the buttons that parents need to tap to get going. Juan made it prettier, then handed it to Kyle.
We're most of the way done on these changes now, and hope to push a much improved version to TestFlight and the web app tonight or tomorrow. It's been a gargantuan effort for Team Brili, and I'm proud of how everyone has pulled through to make these much needed improvements.
Our public launch is coming very, very soon. We appreciate everyone's patience!