When your institution is shopping around for a new platform for its students and alumni, the first question on your mind is, "Will this work?"
That question is harder to answer than you might think. It can be tricky to define what the success of your new platform should look like, and to decide which data are relevant when measuring that success.
It's not uncommon for shops to pick the wrong data to evaluate the success of their new platform. Thinking that the platform is performing better than it is, they lock themselves into ongoing contracts that don't actually deliver the results they expect.
In this post we highlight four common mistakes that shops make when evaluating the performance of their platforms and how to avoid making them.
1. Don't use adoption as your primary measure of success.
Instead, measure engagement. It's encouraging to watch your constituents sign up for your platform, and it's easy to tout the size of your user base as a sign of success. But just signing up for something doesn't mean anything—people have to actually use your platform for it to be a success.
Have you ever signed up for an online platform, clicked around, become bored or confused, and then left forever? It happens more often than you might think, and it means that user count is often little more than a vanity metric.
In addition to tracking the number of users you have, keep track of monthly active users (MAU, in the biz) and how those active users are using your platform. What actions are they performing, and how often? Are they getting what they want or need? Are they coming back regularly, or are they jumping ship? The answers to these questions will give you a holistic understanding of the health of your platform.
2. Don't count checked boxes on user profiles as success.
Instead, track activity. When they sign up for your platform, your constituents may have the opportunity to check a box to indicate that they're willing to be a mentor, give a mock interview, offer advice—or any number of other things. We often see shops referencing the number of users who have "signed up as mentors" or "registered as members of the career advice network" as a sign of success, when the truth is more complicated.
When a user checks a box on their profile, they haven't actually accomplished anything yet. They haven't helped or been helped by anyone. They may not have even finished the sign up process or logged back in to see what's new.
Rather than counting the number of users who have ticked off checkboxes, track how many users form connections and find what they're looking for on your platform. Look for positive outcomes and ongoing user activity.
3. Don't use absolute numbers.
Instead, use percentages. It's exciting when 1,000 people sign up for your new platform, but that number is only a drop in the bucket for an institution with an alumni community of 300,000. Three-tenths of one percent of a drop in the bucket, in fact.
When evaluating the health of your platform, be sure to set goals and weigh your results relative to the size of your community and the amount of time and effort your team invests. You signed up 500 users, but did you reach your goal of 5% community-wide adoption? How much staff time did it take to recruit those users—were your techniques efficient? Is your platform reaching enough of your community to be cost-effective?
Framing your numbers as percentages will keep you—and the vendor you work with—accountable to the goals you choose at the outset.
4. Don't use static figures.
Instead, measure ongoing growth and activity.
"500 people signed up in the first six months." That's great! How many people signed up in the following six months? How many people signed up last week? How many sign up on a daily basis?
Every platform needs to continue to grow in order to stay healthy. In the same way students cycle through your institution every four years (give or take), users will cycle through your platform, signing up as students and continuing to use it as alumni.
Citing the same figure over and over again as a measure of your success can breed complacency. Keep your data up to date and monitor user signups and activity over time to make sure your platform continues to grow. If it turns out to be just a flash in the pan, why continue to pay for it?