Self-service onboarding: the critical part of the product success

How did we define an activation moment?‍

A moment when a new user realizes the solution’s value for the first time sounds pretty clear. As product builders, we know exactly what kind of value proposition we communicate. 

Well, this is true and false at the same time: 

• You may miss the moment when value is extracted as perceived by the users; or
• You may optimize for one group of new users while rejecting another.

We defined our activation moment as the one when a new user sends their first document. By the end of 2021, a team consisting of a product manager, a product analyst, and a user researcher was working on new activation metric criteria for our evolving product. 

There were three key steps to define an activation metric:

1. Find actions (or events) that have predictive power on conversion to paid;
2. Run interviews to understand when users truly understand the value of PandaDoc;
3. Summarize the findings to pick a metric related to a predictive event supported by the user interviews.

Our product analyst Artsiom Kazubouski used the Critical Success Index (Venn diagram approach) and Information Value / Weight of Evidence approach to assess one variable's predictive power over another. The data analysis helped define a list of actions, their frequency, and predictive power. 

The user research let us understand better what makes people buy PandaDoc products. We talked to recent customers (purchased within the last few weeks) to learn:

1. What in-product experience defined the moment they realized a desire to pay?
2. What was their process leading to that moment?
3. What have they tried in the app?

The third step — summarizing data proved to be much easier than we thought — without complicated Customer Journey Maps (CJMs), connecting different insights, and looking for patterns.

We learned from quantitative and qualitative research that the defining moment is related to creating or using a template to create a document. Quite often, new users went through additional valuable actions when interacting with templates, such as using variables to save time on document creation. 

Then, hindsight kicked in. It made much more sense to measure activation as it relates to the template experience. After all, PandaDoc helps you save time on document management, while many other solutions only offer eSignatures.

‍Key thought #2: You can define an activation metric based on qualitative and quantitative research. It is simple but not easy.

How do we approach experimentation?‍

We have a core metric — the rate at which new users activate our solution. In addition, there is a set of metrics describing the rate at which they go through additional valuable actions. We focus on the key activation moment but keep adding additional experiments to see whether we can move additional metrics along. 

At PandaDoc, we have a product manager responsible for activation and onboarding. The role requires expertise in user experience (because onboarding can significantly influence UX), empathy, and strong analytical and management skills.

Either way, we have an experimental backlog of:

1. User experience fixes and changes;
2. Short and simple experiments;
3. Bigger bets that significantly change the experience.

We are trying to balance between tiny changes and bigger bets. New users evaluate templates by selecting the right one and learn how to use them using our prompts. Soon, we will experiment with guiding the user through a complete use case (such as building a proposal) through a predefined flow and a template. That takes longer but shows the whole experience from start to finish and lets us not be stuck in local maxima.

‍Key thought #4: Bigger changes are required to solve the most pressing issues.