Customer development interviews, focus groups, beta testing … they are all great ways to learn what your customers want before you actually build it. They are often based on small groups of people simply because it’s hard to interview hundreds of people in a short time.
What extra benefits could you get if you could interview hundreds of people at once, even in the middle of a brainstorming session about your next big idea? What if you could tap into hundreds of your actual users, right now?
GuideStar figured out a way to do just that, and it’s a very interesting case study in setting up a “Lean Advisory Panel” of your users.
Applying Lean Startup in the Enterprise
Andrew Crookston was Senior Program Manager at GuideStar, which is the leading provider of nonprofit information about charities and nonprofits to the public in order to help them better evaluate their charitable giving decisions. GuideStar is an important organization in the nonprofit space, and has a passionate membership base of people who support their mission to make this information easily accessible.
As GuideStar began to contemplate new products and major new features and designs in their longstanding site, they decided it would be good to engage their members for ideas. They are a 19 year old company, with all the legacy processes and technology that implies, but they wanted to take a very nimble approach to their work.
Andrew and others at GuideStar became excited to apply Lean Startup techniques to their enterprise. GuideStar wanted to apply Lean Startup validation techniques to any MVP for new product development or new features. How could they get that validated learning?
Setting up a customer advisory panel
Initially the team discussed setting up a small customer advisory panel. Perhaps a technical board of a dozen or so people.
Andrew was inspired by a 2009 post by Eric Ries on Customer Advisory Boards, and suggested that instead what GuideStar needed was an advisory panel of at least a hundred members from their site.
Incentivizing participation through reputation
GuideStar is a nonprofit itself, and as such was not anxious to have to bait hundreds of people into joining their advisory panel with Amazon gift cards or other financial incentives. And begging wouldn’t do either.
Instead, they proposed the panel as the insider’s track to GuideStar 2.0, and they asked members to apply for a one year appointment to the panel. They emphasized that it was a selective process, and that those selected would have their name and organization posted on the GuideStar website. They listed out a specific set of responsibilities for the panel members, including attendance at regular webinars on new feature development and participation in regular surveys on new products and features to be developed next.
The implications were clear. This is a prestigious group to be in, and membership in this advisory panel would be another way to show others that you have a valued reputation in the nonprofit community.
Ask and you shall receive – maybe more than you expect
GuideStar has an email list of hundreds of thousands of people, and their marketing department culled the list down to those most active in the last few weeks, and then further cut the list to about 60,000 emails. GuideStar then sent an email to that list inviting them to apply online to become a part of this panel. The email contained the participation requirements, and a link to an application form that encouraged you to explain a little about your background and how you would be able to contribute a unique voice to the development of GuideStar 2.0.
Andrew and his team were delighted with the response, and had a “high five” moment. In the first twelve minutes, they received 133 applications. The goal had only been to get 100 people on this panel, and so clearly there was substantial interest from their membership base.
After two hours, over 400 applications had been received. In the application form was an optional field asking users to explain why they should be selected, and over 86% of people were filling it out.
GuideStar members were actively selling themselves to be part of this panel! Applications included statements from people touting decades of experience in the nonproft industry, listing their achievements and awards, the budgets they manage, how they use GuideStar, and how their perspective would bring innovative ideas to GuideStar 2.0.
Getting the board involved
GuideStar’s board of directors were delighted to see this outreach effort to members, and also got in on it themselves. Andrew gave a presentation to the board on their evolving “Lean IdeaLab” efforts, and talked about the success of the Lean Advisory Panel.
After the presentation, the thought hit Andrew, Why not also involve the board in the Lean Startup concepts of Build, Measure, and Learn? They have a vast amount of experience and may be excited to participate in evaluating new product ideas. So Andrew discussed it with his CEO who supported the idea, and Andrew invited the board to a 30 minute online presentation in which several board members joined. The board members gave feedback on MVP’s that were demoed as well as on other new product ideas and features that were presented. One of the board members emailed Andrew and the board members after the presentation, “This was great. Well thought through, well organized and well executed. Hope our feedback was helpful. Many thanks!”
Early participation from the advisory panel
With so many people applying to be part of this panel, GuideStar faced a tough decision. How would they narrow down the large number of applications to only 100 people?
After much debate, they went ahead and accepted most of those who had applied, 850 people in total. While they had concerns that this would take away some of the prestige of the application process, they decided it was better to get as much engagement as possible and to capitalize on the interest and enthusiasm of their membership.
With the application process complete, it was time to seize the momentum and engage the panel quickly.
GuideStar organized four kickoff web meetings, and demonstrated 4 different MVP’s and 10 feature ideas. To gather feedback, they conducted surveys as part of the meetings. The surveys were completed “live” in the meetings by attendees in order to maximize the response rate and feedback gathered.
Based on those survey responses they identified the top MVP’s to build, and every few weeks after that panel members were invited to online demo’s of the MVP’s and given the opportunity to provide additional feedback.
In addition to the efforts of Andrew’s group, other Product Owners at GuideStar also began to use the advisory panel to send out mockups and feature ideas to the panel for feedback.
Offsite brainstorming with the Advisory Panel
A particularly interesting use of the advisory panel was at an offsite that GuideStar held in May of 2013. Offsite brainstorming sessions are a great way to get teams into a more creative space, away from their email, and thinking long term.
The problem with offsites is often that only those who attend benefit, and as soon as they go back onsite to the office, the excitement wanes and new initiatives flounder.
GuideStar had a group of 12 team members at a full day offsite, but they also brought along several hundred panel members virtually. After a half day of brainstorming, the GuideStar employees had brainstormed a number of ideas. But where to go next with those ideas? And which ones would members value most?
In the early afternoon, they sent out a survey to the Advisory Panel, and presented their ideas from the morning. This allowed them to instantly validate or discard ideas with hundreds of their users. With the panel’s feedback, they now knew what to focus on the rest of the afternoon.
Several members of the “Lean IdeaJam” offsite said that engaging the advisory panel was the best part of the day, to “be able to see in real-time what the panel thought were the most promising ideas.”
Keep the momentum and learning going
GuideStar now regularly engages their advisory panel through demos, surveys, and feedback sessions. What started with a hopeful vision of having 100 panel members now regularly receives 350 responses from members.
What could you learn from your customers? What more could you learn from hundreds of your customers? GuideStar’s example shows how to gather large amounts of feedback in a very efficient way. Keeping their advisory panel involved throughout the process doesn’t slow them down from frequently developing MVP’s and iterating on ideas. All they had to do was ask their members to participate. What have you asked of your customers lately?
Want more details?
If you are interested in any more of the details on how GuideStar was able to assemble such a large and engaged advisory panel, just email me at Arin@AgilityFeat.com and I’ll get you in touch with Andrew Crookston.
