Every Team in the World Should Do Retros

Recently, over breakfast of chilaquiles with my colleague Andrei, I once again found myself extolling the virtues of the only meeting type in the world that I wholeheartedly love: The Retro.

Andrei and I were having a working breakfast about our company Remote Year. Andrei came on board a few months ago as a Programs Team Manager and works with teams that lead 3–4 of our programs. The talk eventually drifted to feedback and Andrei asked if I had any thoughts on how to best structure feedback sessions and then how to capture actionable insights.

As the question was coming out of his mouth, I started to jitter with excitement, as I always do when I get to tell someone new about the beauty of a Retro, but let’s back up.

Retros are a 1-hour meeting that are held regularly and allow teams to review what went well, what went meh, and what went badly about the work that has been delivered in the past week, two weeks, or month.

I first experienced a Retro when I was with a company called Case Commons, then a client of Pivotal Labs NYC. I learned then that Retros are best done on a Friday afternoon, usually with an open beer, in a relaxed atmosphere. The session starts with a facilitator drawing a happy face, a meh face and a sad face on a white board or on a shared google doc. For roughly 10 minutes the team members independently write down what they feel was great, shrug and bad about the work that has happened leading into the Retro. Teammates can +1 stuff that they agree with.

Template for Retros

Everyone then sits down and the facilitator touches on each thing from the column, mixing it up between the faces, until all are discussed. They ask for teammates to sound off on what is implied by each piece of feedback. Anything that becomes a larger discussion should be parked and have an action item applied to it. The sad column should produce a number of activities in the Action Items column that are intended to improve or remedy the things that did not go well. Individuals should volunteer or be assigned responsibility for Action Items. At the next Retro, the facilitator should review the Action Items from the last one, and see what’s been done or not done.

Despite the eye-rolling nature of this phrase, Retros should be safe spaces for the team to discuss mistakes or things that went poorly. Feedback should be given and be meant kindly. Retros should embody a spirit of “No Blame” culture so that issues can be discussed freely and honestly. Make no mistake, getting a team to sit down in person and talk freely about mistakes isn’t easy.Most people have never done it and it can be uncomfortable especially when things aren’t totally hunky-dory. It’s a muscle that teams need to exercise. That’s why the correct cadence for Retros is usually weekly. It gets people comfortable with giving feedback to each other, and also holds people accountable for the action items and helps improve things in a tangible way. This helps them believe that continual improvement is both possible and expected. Retros should not be allowed to slide off the schedule. At its heart, a Retro is a planning meeting, one specifically meant to address issues and weaknesses. What could be more important than that?

Retros are great for other reasons:

• The structure provides an opportunity to discuss positives and negatives, so that folks can remember that even when mistakes are made, good things happen as well.

• Retros work really well with 100% of team sizes. They are as effective for 1:1s as they are for large teams. I do a 1:1 retro monthly with my boss, and I’ve been in retros with teams of 30 devs.

• Retros are domain agnostic. I have a friend who has a bakery and she leads retros with her team. A friend of mine who is a lawyer has retros with her manager. The main point is to be able to address where things get have gotten sticky, and there isn’t a team or a company in the world that couldn’t use that.

• Retros are always there, people get used to that and start noting and storing feedback for them. Teams and individuals start to get more thoughtful about process and more creative about improvement possibilities. Retros help people get better at getting better.

Get better. Do Retros.

10 things PMs should do when they start at a new company

10 things PMs should do when they start at a new company

Three months ago I joined a start-up called Remote Year as Head of Digital Product. That fancy title aside, I was coming in as the first PM on the ground and the company’s first hire to a new digital product department. Remote Year is a service based company that brings together groups of people who want to work and travel the world in the company of an awesome community. I was so excited to be joining a team that was on the crest of a wave that symbolized a new way of thinking about work and life.

Remote Year is the first company I joined where software was not the driver, but an augmenter and an enhancer. I was nervous about this, but mostly excited as I am a UX-Design-heavy PM and working with user needs is my wheelhouse. I was also excited about the opportunity to bring UX research and design thinking practices to an IRL service.

Remote Year dogfoods like no company out there. It is 100% remote with a large majority of its 100 person staff traveling with or between our programs as they criss-cross the planet. I was coming from a company that espoused a philosophy of co-location as a cornerstone of good practices, so this was unchartered territory for me. I was also coming from a big company where I had designers, engineers, data scientists on my team from day one. Then it was just me, for a while at least.

Three continents, four countries and countless Zoom calls later, I’m taking stock of what has gone down and creating a list of things I did that worked well, things that I wish I had done better, and things I really wish I had done at all, though I won’t say explicitly which is which. As a team of one, (about to become a team of two) in an entirely new environment, trip-ups were inevitable, but I hope this list will be a help to PMs who are entering into any new company, as these practices can be broadly applied to lots of different orgs.

1. Explain the broad vision of your process to all your stakeholders

Whether your company is brand new to certain methodologies or old hat at agile and UX, have a conversation with the key folks you need to work with about what it IS that you do, and what the outcomes are is incredibly helpful, as many things can be different from org to org. Present an overview of agile product management, what is comprised in discovery and framing, what activities you will do at a high level, and what the expected outcomes are.

Agile, despite being nearly 25 years old, still isn’t something that is clearly understood, and how user centered practices fit into agile is even less understood. Have an hour long conversation supported with a few slides and visuals, and get folks asking questions.

2. Make it crystal clear what you need and when

During that agile overview/education session, make it clear how much you need stakeholders to participate, either in attending Discovery activities, or in attending group debriefs to discuss the insights gathered during rounds of research. Set placeholders for them on the calendar and try to make them stick. Push for this and be explicit about its importance. I’m not saying it will happen, but be clear about why you need them there.

One of the most valuable things about the Discovery process is that it will steward a team that participates to the same destination. If you’ve done all the work alone, and not brought the team along with you in incremental steps, it’s a lot longer of a journey to get them to your page as Discovery draws to a close.

3. Make sure you do constructive stakeholder interviews

When I first joined Remote Year, I travelled to Asia, so most of the folks I needed to speak to were at least 3 time zones away, and many were 13 hours behind. This, to put it lightly, threw me for a loop and I can cop to letting this one slide, but understanding what your colleagues who are guiding the company are thinking about is really important in terms of understanding implications and opportunities for digital. I’m doing them now, and it’s actually alright that it’s a bit later. I think you can grab value from this exercise at any point.

If you need a template for a good stakeholder interview, check out the book “UX Team of One,” by Leah Buley. In fact, that book is a god-send for anyone who needs process support as they figure out a new company environment.

4. Set consistent touch-points and come hell or high water, stick to them

This one is pretty similar to #2 just on the one-on-one level. There are probably 3–5 people that you should be checking in with on a weekly or bi-weekly basis. These intimate conversations are updates that help you stay abreast of what work is going on at a granular level and vice versa. This helps create transparency on both sides and assists collaboration. This is easier when you’re in the same office, but can still be challenging regardless if schedules are busy, (which they always are.)

5. Repeat back the deliverables timeline early and often

This conversation should be easier if you’ve done #1 and laid out the big picture from the beginning, but misalignments can still happen. If you’re used to doing things with certain team members, in a certain order, with certain resources, you make assumptions about when things are going to happen.

In terms of having this conversation with your boss, start with the question, “When do you expect us to start committing code?” and work backwards from there in terms of what you need in order to make that happen. Be explicit, and do not mince words. Over promising and under delivering will feel worse than saying the hard thing.

6. Immerse yourself in the product and service

The best way to kick this off is to get a group of roles in a room and do an in-depth, 2–3 hour user journey mapping. Use this exercise to get into the real nitty gritty of each role. I don’t just map the customer journey, but Remote Year staffers as well. Map as many roles as you can, and deep dive into particulars if possible. I had a two hour session and then four additional interviews to fill in the gaps. The User Journey Map should give you as good a sense of the inner machinations of the org and service as possible.

Unfortunately, I had to wait quite a while to get the right group in the same room (and user journey mappings don’t work over Zoom,) so I started immersing myself by joining programs and participating in the service itself. Watch how things work, listen to conversations that are happening, constantly take notes, be a fly on the wall, keep your eyes open as you watch how people interact with colleagues and customers. This helps you conduct better…

7. Do user interviews

Obviously as you’re doing Discovery, you’ll be doing a lot of things, but NOTHING is EVER as important as doing user interviews for a new PM coming into a company. Once again, I point you to UX experience Team of One if you need a reminder about a script template. Create a script that allows you understand the motivations, the incentives, the challenges, frustrations, pain points and current tools of your customers. The ultimate question you’re asking yourself is “How are things happening for my customer now, and what should change first?”

Trickiest thing about an interview: it has to feel like a conversation but still collect consistent data. It’s your job to guide towards this outcome. At Remote Year our customers are whip smart people who have a lot of questions for me. I view it as a sign of respect to them and the product to get the most out of the interview by saying “we can grab a coffee after and chat through those questions, but I really want to make sure I get your thoughts on a few things and these sessions fly by.” They’ll understand that this isn’t a casual chat, but actually that I’m taking the exercise and their time seriously.

8. Synthesize, and then find ways for others to use the insights gathered

I had to make the switch from post-it synthesis to Trello board synthesis so I can share it out more easily. I have also realized that synthesis is so much better if it’s done with someone else. So if you’re hanging on your own, do your darndest to pull in some of your colleagues into the process.

If you can, pull in colleagues who will derive a lot of value from seeing organized insights from your customers. I’ve really enjoyed pulling in folks from various corners of Remote Year and bringing them into the insights conversation. I hope to do this a lot more often in the coming months.

9. Don’t beat yourself up

This happened to me a lot. Things felt like they were moving very slowly, not surprising given I was doing the job of a three person team alone. I knew that things like stakeholder interviews were slipping, I wasn’t sure when I would get people together for a user journey map. I was having trouble finding time to do the work and then communicate what the work was indicating. At times, I didn’t prep enough in advance to get stakeholders involved. I was away from home. It was tough.

But… I am also traveling the world for my job. Meeting incredible groups of new people each month, and getting to know colleagues who are some of the most wonderful people in the world. I get to go to the beach in Mexico. I say things like “I think March is going to be in Lima.” I’m also working on creating products for a company that is the first mover in a new way of working, and is blazing a trail into a way of life that is bringing joy and inspiration to its customers. When you get down, take stock, remember the good stuff, make a list, and get started again.

10. Call your PM buddies (and data scientists and devs)…a lot

When I would get stuck this is the thing that would get me unstuck the fastest. I am so lucky that I have worked with so many amazing people in the past who I can call friends, who are willing to give of their time and ears to offer perspective and advice. Reaching out for help is not admitting you don’t know, it’s admitting that a conversation will assist you in getting there. Thank you to my squad for all of the good stuff you’ve given me over the past few months and looking forward to more learnings in 2017.

Reframing the User Research Conversation

Who has struggled with implementing user research? Do you have hard sell clients/stakeholders on the value of user research? Are the necessary costs of these activities debated?  Although many speak of current UX practices as the era of user-centered products, we still find ourselves explaining the what and the why of user research and how it is the beating heart of creating a great user experience.

When projects start, the costs and time required to implement user research often takes our stakeholders by surprise. Most often, clients who haven't had experience with user centered thinking and hence, don’t understand what user research entails. They think they want to experience user centered design processes, but often get cold feet at the changes that these processes bring.  At worst, the client doesn’t agree to research, which leaves us with our assumptions unvalidated and our designs untested. At best, we find a way forward, but often our start times are delayed, and expectations still aren’t clear. The London Pivotal Labs Product and Design teams decided to take steps to fix this broken chain and smooth our client on-boarding process to user research.

At Pivotal London, we were a small office for many months. We didn’t have extensive networks to reach into for recruiting participants and our capacity was low.  On a project that was particularly strapped for time, we engaged Criteria, in order to get five interviews scheduled with only two days to prep. Using a recruitment agency meant: 

• We cut the time needed to secure interviews by half.

• We could specify a specific profile of user.

• Our interview subjects were not tainted by personal connection, incorrect knowledge or leading set-ups.

• The attrition rate for interviews went down to nearly zero.

• The client received the invoice for the cost directly.

The office grew and we took on more projects that required a research plan. Many teams came up against a user research roadblock with clients. While using recruitment agencies helped cut down on admin time, sometimes it didn't make sense to use one, which left us double over an barrel when it came to getting user research off the ground.  

We found that different clients had different reactions to the costs of user research. Small start-ups worried about overhead costs. Large corporations worried about deadlines and IP. In most scenarios, getting ongoing buy-in through the course of the engagement was tricky at best.

Fast forward a few months. London Product and Design had tripled in size and had gathered enough information about what our user research challenges were to tackle the problem head on. We now had the capacity to create a strategy for how to bake this process into our engagements, which included using Criteria for all interview recruitment.

We came up with a concept of “Packages,” and workshopped how our experiences fit into what became small, medium and large. Our goal with these packages was to outline our research expectations and process, given the scope of the product, and get client buy-in. The outcome; the costs for discovery + framing and ongoing research were now baked into our engagements.

Clients and businesses always need to understand the costs, constraints and time structures around any work done.  The lack of transparency around the nuts and bolts of how user research happens put it at risk of being accomplished.  This puts our clients at a disadvantage when it comes to creating great products.  We believe that these templates can provide a foundation that can just as easily be used by start-ups and product companies as they can by consultancies.

The London Product + Design Team identified a gap in the sales process, and a risk when it came to delivering great lean practices to our clients. Our hope is that we can continue to work with these concepts and refine them as we get new clients and new products through the door. We would love to know if other offices are using similar techniques, and how these have helped or hindered your product development.

Pivotal Labs London Launches Product Office Hours

Remember Office Hours in University?  If you don’t, it was a time that professors just left open in their calendars and students could book some time during those blocks to come and have a chat.   As a student, you usually went to the office hours for the classes you cared about most, or maybe the class where you were really struggling but wanted to do well. It was part brainstorm, part catch-up, part therapy.  It was a chance to clear your head and perhaps it kept you out of the weeds for a few weeks after.  Sometimes it was the only time you got to talk about a paper or a project that you’d worked on for weeks or months.

Building products, as an entrepreneur or a product manager, can sometimes feel a lot like being a perpetual student.  You are always faced with new information coming at you, you have to synthesize it, make decisions and make the idea your own.  A lot of the time you’re going at it alone or with a very small team, and you often feel pretty isolated.  What do you do when challenges come up, like when sign-ups drop, or your growth levels out?  Who do you talk to?

Pivots have met a ton of CEOs and Product Managers at meet-ups and conferences who are working on great ideas and have really promising products.  We shoot the breeze, take a look at their products, offer ideas, and help them focus on the possible solutions.  The appreciation we receive often seems disproportionate to what we think we offer until it came clear how valuable collaboration and community is when you feel as though you’re doing it alone.  So we formalized the process and rolled out Pivotal Labs Product Office Hours.  This program of lunches, where Pivotal Product Managers and Designers meet with people who are building things and chat about the challenges they face, went live in NYC and San Francisco last year.  We are really psyched that we are now bringing it to our London office in Autumn 2014.

So what’s the deal, really (I hear you ask)?  The deal is that you send us an email to office-hours@pivotal.io, tell us a bit about yourself and your product (Make sure you mention you’re in London.)  Someone will be in touch to schedule our lunch and maybe ask some clarification questions.  Then you come down to our Old Street office on a Friday from 1230-130pm, eat some free lunch and a few Pivotal Product Managers and Designers offer you no-holds-barred advice on how to tackle your most pressing product issues.  Can you come in again in a few weeks to let us know how things are going?  YES, please do.  The lunch is free, the advice is free, the bus fare over costs £1.45.  There’s really no reason not to sign-up and come in.

So what’s in it for Pivotal Labs?  Well, we get to meet awesome people who are doing cool things.  We get to exercise our critique muscles and we get to have cool conversations.  If you want some more specifics, you can check out these blogs about WeeSpring and Notable and how to frame your problem, to get an idea of how these sessions go.  There is no general template, each session evolves in response to the product discussion at hand.

We hope to see you soon, our doors are open.

Great Things Come in Small Sizes: Writing Stories that Work for your Team

When I tell people a big part of my job as a Product Manager is to write stories, I tend to be met with blank stares or quizzical looks. But if you’ve been anywhere near a project at Pivotal Labs, you’d know that stories are the life’s blood of the development process.

Simply put, stories are features that are broken up into small pieces of work. For example, “As a user, I want to log-in to the website, so that I can see my homepage.” These are written by PMs and logged in a Tracker tool. Dev’s then “pick up” these stories and use them to direct their work. A list of stories is called the Backlog and reading a Backlog should give someone a general idea of the product that is being built. There are a number of benefits to clients that stories provide.

• Stories provide transparency. Stakeholders can read the stories in laymen’s terms, and ask questions about what is there and what is not.

• Stories provide control. Clients can look at how stories are prioritized in the backlog and have meaningful conversations about why certain features are more important than others.

• Stories help lower risk. Clients often have limited resources at their disposal and delivering smaller pieces of work at a rapid pace helps clients get their products in front of their audience faster.

When we work with clients that are new to working in an Agile environment, we explain user stories by presenting INVEST.

• Independent

• Negotiable

• Valuable

• Estimable

• Small

• Testable

Out of these, the word “small” is a bit quixotic because the appropriate story size is a range with both an upper and lower bound. So what is a good story size and what are the tradeoffs? At Pivotal, we prefer to have stories that are estimated to take less than 2 pair days. This suits our goals, but because the size range can vary between methodologies and practitioners, we get a lot of questions around what is correct versus not correct. Here is a quick rundown of what we see at Pivotal when we use small stories.

• Flexibility in planning, prioritizing, and cutting scope. One of the important skills we push our clients to develop is to see the complexity and cost of their vision.

• Greater accuracy of the velocity. Velocity tends to be more accurate when the stories are finer grained, because there is less likely to be hidden complexity and issues.

• Forces earlier and better understanding of the overall system.

As mentioned this process of turning stories into as small bits as possible has its trade-offs. Obviously the more moving pieces that you introduce into a process the more complex things can become. Each story becomes part of a larger picture and it is easy to get lost in the leaves. It’s also quite easy to have stories get duplicated, as well as have certain stories be blocked by work that is not completed yet.  But PMs should rigorously audit their backlogs to make sure that all stories are relevant and usable.  Most stories should be written within a few weeks of when a Dev would pick them up.

So what does this look like in practice? After an inception, the project team has a list of Epic stories. These are the higher-level stories out of which smaller stories are created. Here’s an illustration of this breakout process.

“Shopper should be able to recommend a product to a friend.”

After discussing the “how” of this story it can be broken up into smaller stories that capture facets of how a user will accomplish this task.

“Shopper can provide an email address of their friend.”
“Shopper can provide a customized message when recommending something to a friend.”
“A link is sent to the friend of shopper that links back to the recommended product.”

…and so on and so forth.

1 and 2 points story are generally the most common. 3 points story also occur regularly, but much less frequently. Higher points like 5 and 8 are rare and generally are because there’s technical unknowns and risks. At the same time, Pivotal Tracker assumes a starting velocity of 10 points, which also tends to be median for a pair. This translates to stories generally being about 1 pair day or less. Only rarely are stories at 2 pair days or more. Of course this is the estimate, and not the actual amount of time spent and this can change, shift or vary, but the team will learn to adjust estimates as the project moves forward.

Ultimately breaking up stories into the smallest possible increments does is makes it possible to predict with some accuracy when certain features will be complete. It’s never an exact science, and unforeseen things arise that can set predictions off track BUT Product Managers and Devs are able to control the size and breadth of stories. As development is often an uncertain practice, teams should grab onto the practices that help them gather more insight, more transparency and more control wherever and whenever they can. So…keep your stories small.

*Note: This post was started by Brendan Kao and finished/edited by Rosemary King an indeterminate amount of time later. 

Getting to Zero: Strategizing a Validation Process with Mobilise

How do you get started? This was a question that was being asked on both sides of the table at Pivotal Labs London’s inaugural office hours. Our Associate Director Robbie Clutton, Associate Director of Design Martina Schell, and I sat down with an organization called Mobilise Public Ltd to help them hash out their product hypothesis and potential strategy for an MVP launch. Mobilise asked us how these sessions usually went, we replied that the session really shapes itself and asked them to start describing the problem they are trying to solve.

Mobilise is a public sector company that is in the process of transitioning from service delivery to product delivery. Mobilise has extensive experience working with public and community sector in underserved areas and a deep understanding of their various users’ needs and desires. This wealth of experience means that Mobilise has many ideas of what products could be useful. This shift in business model was instigated when Mobilise spotted an opportunity to capitalize on underutilized public spaces that could help raise engagement in local communities. So their question for us was how they should best get started.

Their hunch on how to best evolve their business is building a platform that breaks down the barriers to using community spaces, while simultaneously giving small entrepreneurs an opportunity to engage directly with their own communities. Our aim during the session was to understand what Mobilise’s product hypothesis is for this product. Once you have a clear hypothesis the path to validate that hypothesis emerges. All product hypotheses should ruthlessly eliminate as much complexity as possible and drive to the core idea.

Mobilise’s hypothesis boils down to a simple concept: If the community spaces in question were easy to access and easy to use, then people in the surrounding community would be eager to use them.

After discussing the hypothesis it was clear that Mobilise would be well served by leveraging their existing relationships with community organizations and groups in order to help define the MVP. We identified several opportunities for research and discovery that should help them understand how their core user groups were completing these activities now. These included shadowing and interviews with organizations that have successfully built active communities with minimal help from digital products. It is also crucial that Mobilise get an understanding of what other tools to do similar work, and where gaps exist and what folks would like to see automated vs. what they would want to continue to do themselves.   Lots of companies view this type of intensive discovery work as overkill but with complex problems, unclear user needs, and premium development costs, making sure you validate your hypothesis with specific answers carries its weight in gold.

Sometimes the first step is discovering the first step.

There is No Manual: Changing Your Product’s Rules to Appeal to New Customer Segments

Every company wants to see their customer base grow. This is one of the baseline metrics of success. But if a product is built for one core group of users, how do you know which way to move your product as you try to appeal to a new segment? In our most recent product office hours, we spoke to a company called Effektif about this issue and how they could plot their path forward.

Effektif has spent the last few months building a BPM tool that could be billed to Enterprise level clients as the “simpler” solution. After a brief perusal of the product, it was clear that compared to market competitors like Salesforce and SAP, Effektif was a more straight-forward tool. In addition to continuing to improve the product for their Enterprise customers, Effektif wants to expand their market to small and medium sized companies. As Effektif’s rep, Peter Hilton says, “we see simpler tools as both the future for big companies and a way to introduce BPM to smaller companies,” which is a great hypothesis for them to validate.

It was clear that Effektif understands the Enterprise market really well. Their tool didn’t require massive tutorials, but there was an expectation that their customers would receive training and would check out a manual. The usability of Effektif’s BPM reflected this. There were several features and flows that I didn’t even notice until Peter gave us a demo and navigation was not intuitive.

I talked to Peter about my experience as a Product Manager at a smaller company and how I would have used this tool. A lot of the product’s value wasn’t initially apparent, but after I saw the demo I thought of several instances where it would have been incredibly useful.

During our session, we encouraged that Effektif stop thinking about how to market the value of their Enterprise BPM to small businesses, and start thinking about how to evolve a version of the tool into a Small-Business BPM. Applying old rules to new customers doesn’t work, you’ve got to create a new foundation of understanding. Effektif needs throw out their assumptions about small businesses and carve out time to discover what’s valuable and what’s not so important. This means getting out of the building and having conversations with people who work at small companies, which will drive out a new rules like, “I don’t have time to do a training,” “I need to be able to easily delegate these tasks without extensive discussion,” and “I really need a BPM that has X template.”

Without a solid understanding of what a new customer segment needs to accomplish, the chances of creating an indispensable product are slim. Kathy Sierra, one of my favorite product people once said, “People don’t want to be great at your product, they want to be great at the context.” In Effektif’s case, people want to be great at their jobs, and they want a tool that doesn’t get in the way of that. Attempting to capture new customers means throwing out the old rule book and finding the stuff that makes your product’s value crystal clear.

Open Signal: A Case Study of Rapid Fire User Research Overhaul

As a tech consultant, I help companies optimize how they build software. I talk to a lot of people about implementing new practices and processes that will help them put the user value at the center of the development practice. But with many of the companies I work with full assimilation of these concepts is a distant goal. Until yesterday, I’d never seen an organization learn about a process and start implementing it full-force the next day. That, my friends, is catching sight of a unicorn in the wild.

This unicorn is called OpenSignal and they came to visit me in a Pivotal Labs Office Hours session to chat about beta testing their new app. Ellie, Teresa, and Jaleh, told me about WifiMapper, a brilliant (IMO) native app tool that would help users locate public and private sources of wifi and provide some info about the location.

They had done a lot of work towards their research plan. They had recruited 60 people for a group of super-user participants. They had two focus groups scheduled in their office. They were preparing to conduct some pretty huge global surveys. They wanted the feedback that they got to drive their next stage of development and they came to me to figure out some ways to help them do that. How could they best decide which usability issues to work on and where they should focus enhancements?

I took a deep breath and broke the news that if I would recommend anything, it would be to cancel the focus groups and redo their whole research strategy.

Instead of getting mad, OpenSignal got curious. Here are some of the problems we talked about.

Problem 1: Focus Groups are ineffective

Focus groups are highly risky way of gathering information. If people are shy they won’t speak up. If they feel like others know more, they won’t say that they don’t understand. One aggressive person can take over the session and run it off the rails.

Essentially, people influence people.

Additionally, a focus group format would not allow OpenSignal to watch their users play with the app, only hear about how people felt about it after the fact. Getting late feedback means that they were going to miss out on 99% of the valuable information that they would have caught if they had seen the app used for the first time in front of their eyeballs.

Solution: 1-on-1’s

Scrap the focus group and reschedule each participant as an individual user interview. This will allow OpenSignal to go in-depth with each person, digging into their personal experiences, challenges and hearing how they are reacting to the tool as they using it.

The transcripts of these interviews will contain nuggets of pure gold and uncut diamonds. The added bonus is that user interviews have a relatively small overhead. Generally, five 1-hour interviews are enough to help surface patterns in user behavior and usability issues strongly enough to help set priorities. This leads us to the second major problem that OpenSignal had…

Problem 2: No synthesis means you’re losing stuff

No plan for how to use the feedback and incorporate it into their development pipeline. Currently, OpenSignal fed bugs directly into their development team, but no one was really prioritizing them. Suggestions, usability complaints, and opportunities were tackled on an ad hoc, scattershot basis, and the work wasn’t really well connected to overall feature goals or metrics.

When you don’t take the time to synthesis and plan, all of the value of user research is lost.

Solution: Download and find the themes

OpenSignal needed a few baseline synthesis activities that would help them shake out the nuggets of wisdom from their interviews, as well as their general feedback pipeline. A really simple one is the whole team going through the raw feedback and write one take-away per sticky, talking them through and then bucketing them into similarities. Then you pull out themes, challenges and opportunities out of the buckets. These can then be mapped to specific business goals.

The main point of having the synthesis session is to clearly identify the patterns that emerged from your conversation and map them to your overall business goals. This helps the team understand where user value maps to goals and also sets priorities. BUT in order to achieve this you need your whole team to participate, which leads us to their final problem.

Problem 3: Not Enough Org Participation

The three OpenSignal ladies who joined me for Product Office Hours were collaborating on the user research stuff, but were finding it difficult to get other parts of the organization consistently involved. They hadn’t found a way to involve the developers, and so in turn devs didn’t see user feedback as part of their own workflow. This has made it doubly difficult for them to incorporate the feedback organically into the development backlog.

It can be difficult for small start-ups when there is so much going on, but I tend to say:

Not making the time to participate in user research is like ignoring a chainsaw as you try to cut down a tree with a butter-knife.

Solution: Make it really available

I encouraged these ladies to start having conversations with OpenSignal leadership about user interviews and ask them to try to listen to at least 1–2 of the user interview sessions. With OpenSignal, an invite is probably all that is needed, but if there is resistance, sometimes a knuckle cracking is in order to get higher-ups to tune in. At a small organization like OpenSignal, making sure that leadership is as close to direct user feedback as possible is critical to keeping priorities straight.

With developers, I take a really soft approach. I book space for them to listen to the interviews. I invite them all to both the interviews and the synthesis, and then announce to the whole team whenever the interviews are happening, day before, morning of and five minutes before the interview starts. This helps it get it in front of their faces. The same goes for synthesis. Leave the door wide open, and bang a drum that it’s going on. More often than not, curiosity will get the better of them.

The Full Monty

The next afternoon I went down to the OpenSignal office and was floored by how quickly they had implemented some of my suggestions.

• They had cancelled the focus groups and scheduled several user interviews.

• They had taken all of the existing customer feedback that was in their ZenDesk and put it through a synthesis process on a wall in the middle of the office.

• They spoke about the new user research process with their CEO and invited him and other leadership folks to participate in the upcoming user interviews.

• Finally, they sat me down and conducted their first user interview with me as their guinea pig, with an amazing script that was copied from a script template. They nailed the exercise and delivered it like true pros. It was this process that offered the ah-ha moment for them as to why this stuff is the absolute Poo.

“So much of what you said is what I’ve been saying, but can’t get anyone to take action on.”

“We had so many conversations around whether or not people knew what to do there, but couldn’t decide how to proceed.”

This is what this type of direct user research helps organizations do, understand what’s important vs. what can be put off.

There is very little ambiguity when you’re sitting next to a user and they do not notice, ever, that a button is clickable, or that there is another page of information that they can access. A few days later I got an update from Ellie saying that they had several interviews, videoed them, and shared out the live-feed with the whole organization and a ton of people listened. They also did a synthesis exercise and presented the synthesis to the whole organization. In the words of Ellie herself: “It was fascinating seeing the users go through our app, and at points frustrating because we were thinking ‘why can’t you see the button? it’s right there!” I have never been so impressed by a company’s commitment to putting their customer at the center of their process. OpenSignal, kudos for just doing it. You guys rock.

Getting Back to Play: How Play and Playfulness Can Lead to Better Products

When I was about 10 I found a picture of a Jackabasselope, also known as a Jackalope. For those that don’t know, a Jackabasselope is a rabbit with horns. I think I found the image in a Far Side comic. I had a pet rabbit at the time, and I came up with elaborate stories about Jackabasselopes. They had horns made of pure gold, and would put curses on anyone that hunted them. I played pretend with this idea for hours. At one point I even made paper maché horns for poor Mr. Bun. Eventually I lost interest in these games, and forgot about the Jackabasselope.

Fast forward 20 years. I was gifted some deer horns by a friend’s father. I had them for several months, before an afternoon of crafting with gold leaf brought my Jackabassalope back from the dead. I spent some time applying layers of gold leaf to my horns, while making up stories that smaller Jackabasselopes have greater gold density in their horns and are therefore more desirable. Blah blah blah.

I realized that afternoon that I’d been playing pretend with this idea for nearly 20+ years and it resulted in a beautiful new display for my walls. his got me thinking about play and the effect that the act of play has on us as adults, specifically on adults who are in creative industries. How have we developed into creative people? How can we think about making leaps in innovation and creativity more consistently?

My mum is an early childhood development specialist and I grew up hearing her talk about how kids develop. From birth to the age of 6, children move through three different buckets of play. These are fluid states and they overlap and come up again depending on what a child is doing. 

States of Play

 Autosphere

Play as it connects to ourselves and what is directly around us. It’s here that we recognize what is “self” vs. what is “other.” It’s here that we explore the initial human geography and develop our sense of taste, touch, sight, sound and smell.

 Microsphere

This connects us to our immediate environment. We’ve expanded to include the people and objects around us, and we include them in our games. It’s a creation of configurations, patterns and rhythms. It’s where children can start to explore make believe worlds and use representation and pretend to start mastering skills. Rules are flexible and often discarded to make way for extraordinary circumstances.

Macrosphere

Play as it connects us to a larger societal context. In the macrosphere, children can take tasks or activities that are “adult” and bring new life to them simply because they don’t know what’s allowed vs. not allowed. When kids play in these different spheres, they develop all kinds of cognitive and emotional skills. The way we played as children deeply affects how we think as adults. Moreover, continuing to force ourselves into these playful states helps us as adults to think about things in innovative ways. Albert Einstein, Pablo Picasso, and Nobel prize winning scientist Andre Geim all credit continuing to view their work as play as key to producing their best efforts. 

Products That Make Use of Play

Some of the best products out there make use of these concepts of play and how it causes us to connect with ourselves, our direct communities and the greater world. Here’s two examples of products that make use of play:

 Google Doodles

With their Doodles Google took at concept as simple at a Logo and turned it into an opportunity to connect their users to the macrosphere, by using cartoons and gifs to educate us about events, scientific concepts, and moments in history. It’s a simple but effective way to get our eyeballs on things that we normally wouldn’t pay attention to, or put us in touch with a larger community.

Single Page Narrative Apps

Single Page narrative apps like EveryLastDrop have seen a huge explosion in popularity because it keeps users bounded in a microsphere type story. One perspective, one journey, and it let’s them be a part of it by lfetting them control the pace, experience movement, and patterns.

Play as a Mindset

Sometimes play can simply be a mindset or a process, like Why’s philosophy of Irresponsible Coding, in which he posits that behaving as if there are no rules, consequences or any way that things are “supposed to be done” helps drive him towards bigger technical breakthroughs.

So ask yourself:

• How does your product connect your customer to their world?

• Does it invite them to find out more?

• Does it invite them to participate in the story?

• Can you examine your product for opportunities to make it more playful?

• To hear more about play and creativity, come check out my talk at UserConf London this May!