Discovery

We surveyed pros who participated in the initial, manually-run tests. Our goal was to learn about their experience and pain points when they participated in a tiered bonus test. 

Confusion over what bookings qualify 

Pros found it difficult to determine what jobs actually qualified for the bonus because there was no indication in the Handy for Pros app and not all services qualified.

“Yes every job had to be emailed to handy. They had issues tracking the jobs. It was NOT a good experience and not worth the issues to participate.”

BUILDING AN EXPERIENCE

Lack of visibility hinders trust and success  

The lack of any pro experience when it came to a tiered bonus program led to confusion and anger on behalf of qualified pros. 

Questions over what jobs qualify along with limited awareness and information around the bonus caused confusion. Justifying completion to our Support team for bonus payment led to frustration and anger. 

This posed the question: how might we get pros excited to successfully complete Quests? 

Introducing Pro Quests

Consistent Communication

Know what Quest is happening, as it happens. Pros receive an email promoting the Quest 3 days before start, and get text messages upon start, progress upon job completion, and at Quest end.

Visibility in App 

Pros can see the Quest details on a dedicated page and know what jobs qualify for a Quest before claim.

Show me the money!

Timely and accurate payouts - no more fighting for payment.

HOW WE GOT THERE

To get started, we mapped out various user journeys and matched key features and content to each stage, focusing on the problems pros faced: 

1. How might we elevate pro confidence in Quests?

2. How might we make it easy to find Quests- and qualified jobs?

3. How might we always keep pros in the loop about the Quest they’re participating in?

Working with a Product Manager and Business lead, we honed in on a simple user experience that packs a big punch. 

We leaned into text messaging as the main method of communication. SMS is a direct line to the Pro, and is harder to miss than an email or push notification. Pros would still be notified of Quests through email, and eventually Push notifications, to ensure widespread awareness.

Pros need easy access to their own Quest information. With each job completed, a pro receives a text updating them on their progress, and can track it in their Handy for Pros app. Each job eligible for the Quest was clearly marked in the app as well to mitigate any confusion.

I explored various UI treatments for the new app pages for Quests. The addition of quests in the app was a simple flow. Most of the design explorations focus on UI for the project.

We made the conscious decision to introduce a new tab for Quests in the More page navigation. Not only are Quests front and center, but we planned to build a “Quests hub”: an all-in-one location to see current, future, and past quests and progress. With its own location, we would build Quests into a central component in the app.

Who is this? 

Communication through text messages is an essential feature to the success of this Quest experience. Through testing, we learned the messages lacked in two areas. First, there was no introduction of the sender. This lead to some users to distrust the message, and think it was spam. Second, the messages were dry and not personalized. As a result, users didn’t feel welcomed or excited to complete the Quest. .

Can I make more?

Overall, users thought the experience was clear and liked the ability to make more money (who wouldn’t?!). The idea of a Quest resonated as fun and adventurous to users, which is something we wanted to build on through in-app marketing and gamification through concurrent Quests (the ability to keep completing jobs to earn more).

PHASED LAUNCH 

To get learnings quickly and allow Ops to keep testing tiered bonusing, we took a phased approach to launch. 

Release 1 included automated communications (SMS & email) to pros to make them aware of Quests and their progress, and automated Quest payouts.

Previously, all communications to pros and payouts for Quest completion were manually completed by an Ops team member. Not only were these processes time consuming, but it opened up room for error, especially around payments. This launch increased visibility into Quests for pros and decreased reliance on manual Ops processes to run Quests.

Release 2 was the launch of the mobile experience. 

FEATURE SUCCESS 

The mobile experience launched in Oct. 2020. Post release surveys show that pro satisfaction with Quests increased by 1.7 points to 7.8 upon release of the mobile component (compared to the SMS-only experience). 

• The in-app Quest page meaningfully contributed to a higher Quest completion percentage (33% vs .9%).

• Pros who are opted into a Quest complete 3x more jobs than those in the control group. 

• The number of pros who complete more than 1 job a week increased by 3%. 

• Recent Quest completion rates have increased to ~30%.

While we haven’t hit our goal for jobs per pro, overall Quests are still deemed successful. We don’t want to encourage a change in pro behavior to only complete jobs when Quests are available and begin to decrease claim rates across the board. Multiple variants of Quests are still run each week in different regions, and has become a go-to tool for our Ops team to boost the capacity of our current pro base and encourage engagement. 

For confidentiality reasons I have omitted the actual values for these metrics.