Researching New Feature Opportunities

The Challenge

At Vinted, an online market place for selling and buying second hand clothes, I was working in a team which was responsible for developing additional paid features that create new revenue streams for the company. We call these features value-added-services (short VAS), as they are supposed to enhance the experience of heavy users and provide shortcuts to make them more successful sellers.

vinted_devices_small
Vinted is available through the web and natives apps and aims to bring private sellers and second hand lovers together.

In product teams you constantly ask yourself 'What's next?'. Working on a complex, matured product where a lot of lessons learned have already gone into, this question is not easy to answer. You have to see things holistically, both with regard to the user experience and the system. You should assume that someone else had your idea before and dismissed it for good reason.

Asking 'What's next?' in my team usually meant 'What is the next paid feature we should build?'. As we're talking about taking users' money here, finding the answer becomes even more tricky.

So what do you do to find the answer? You'll start with research.


The Approach

My plan for approaching the challenge of identifying opportunities for new VAS features comprised 3 steps:

  1. Workshop to gather internal knowledge and declare assumptions
  2. User Interviews to identity problems and opportunities along the seller success journey
  3. Survey to validate and prioritise most important problems and opportunities

WORKSHOP

When you join a company you can be assured that there is a ton of things you don't know yet. Reading through pages over pages of documentation and reports is boring and time consuming. And in the end, you might not even get the full picture as not every discussion and decision was documented.

What's way more fun and time-saving is meeting with people for a workshop and get their knowledge first hand.

remote-workshop-via-realtimeboard_after-workshop
As my team and I worked from different locations, our workshops were conducted completely online using RealtimeBoard.

However, it's easier said than done. I had to make sure to meet the right people. Right people are people who can give you the answers you're looking for or are experts in the field you're exploring. So I invited researchers who were having a lot of exposure time with users as well as the designer who worked in my team before I joined. It's also crucial to involve the people who will be affected by the decisions you make (most likely the people in your product team), so that everyone gets on the same page from the very beginning.

Don't get hung up on ideas too early

Facing the exciting challenge of building a new feature or service lets you and your co-workers automatically come up with ideas you would love to test out. The problem is: Testing ideas is time consuming, even in a low-fi way. Therefore getting hung up on ideas too early in the process comes at a risk. You might spend time working on ideas without having ticked the 3 ingredients of successful innovation:

design-thinking-framework
For innovation to be successful, it needs to fulfil all 3 criteria: It needs to be desired by humans, technically feasible, and a viable business model.

Using ideation to declare your assumptions

On the other hand, ideas are a valuable source of assumptions: Talking about ideas is good because it concretises things. And concretising things lets you realise what you already know and what you don't know yet. And this eventually gets you to the assumptions you want to test in your research.

brainstorming_small-notes_compressed
We brainstormed ideas not to ideate solutions, but to start the discussion about our assumptions.

So although my co-workers and I talked a lot about ideas and features that got explored in the past, I didn't use this input to inspire ideas for the new feature, but rather to prepare the next step on my plan: interviews with users.

USER INTERVIEWS

Keeping my assumptions outside of the interview

The most important aspect of early stage need finding research is to be as open as possible and not biased by your own ideas. So already throwing assumptions in can be risky. Instead you want to leave your assumptions aside and not take them into the interview. This is incredibly hard, but I manage it by being mindful about the formulation of questions and by applying a mindset I learned from Erika Hall: Try to disprove your assumptions rather than proving them.

Having said that, I still found that declaring assumptions with the team early on creates a common ground that becomes very powerful later when discussing insights and prioritising opportunities.

assumptions-matrix_%403x
After we declared our assumptions, I prioritised assumptions based on risks and unknowns. This method is taken from Lean UX.

Recruiting the right users with the help of analytics

For our value-added-services, we focused on so-called 'heavy sellers'. These are users who mainly use our product to sell items, have many listings, upload new ones regularly and sell several items per month. Why? Because for those sellers Vinted is like a small business they take very seriously, hence their willingness to spend money on extra services that make their 'business' more efficient is higher than for regular sellers.

To find those users in the real world, I reached out to our analysts who came back to me with a list of users fulfilling the criteria I gave them. However, the user data we can pull from analytics are limited, as we do not require a lot of information during signup and profile completion. Although we know a lot about user success metrics like listings and sales, we know not so much about user demographics.

Using a screener to get detailed user information

I therefore asked users to complete a screener survey in which I verified existing information and gathered additional details. The result was a list of users with detailed information from which I could build the sample for the interviews. At the same time the survey speeded up scheduling as participants had to select their preferred time slots for the interview and leave their contact details.

screener
Providing users a set of available time slots made scheduling the interviews much easier for both sides.

Having said that, I don't always use screeners, because even if it may speed up scheduling, it still needs to be prepared, maybe translated, sent to users and then you have to wait for 2-3 days until results come in. Overall, it takes you more time than if you had reached out to users directly. So the motivation for a screener should never be speed, but rather the quality and quantity of data you want to collect about your audience prior to the interview.

Eliminating language barriers to improve interview quality

In our case we decided to use a screener for one main reason:
We decided to do remote interviews with users from Germany and France. Our French colleagues told us that conducing interviews in English could be a major barrier for French users.

At the same time, we didn't have enough French speaking people in the company who could have jumped in and conduct the interview. And outsourcing the interviews comes at high costs: Not only in terms of money you pay for external testers, but most importantly the cost of information loss when giving the control to someone outside the company with a lack of knowledge about your product and users.

Therefore we needed participants who were able to communicate fluently in English. One approach would have been phone calls, but as we were sending a screener anyway, we used it to filter people by their English proficiency level. This was indeed the part where most participants became unsuitable for our research, but we were happy that we were strict in the beginning, because it saved us a lot of hassle during the interviews.

screenflow_edited_mini
I conducted the interview via Skype or Google Hangout and recorded them using ScreenFlow. Although we managed to prevent language issues, there are always some connection problems to deal with.

Making analysis and synthesis a team sport

After 2 days and interviewing 9 participants, it was time to sit down and make sense of all the data collected.

Having learned from an experience in the past, I decided to do it differently this time. Instead of locking myself in a room and re-listening to all recordings, I decided to do a workshop with my team to talk about our insights and make sense of them together. As some team members joined the interviews as quiet observers, they had heard the users' statements with their own ears, hence they were now able to gather and discuss observations and patterns.

Making the analysis and synthesis of qualitative research findings a group effort comes with 3 benefits:

  1. It's fast
    Collecting notes, moving them back and forth and drawing connections shouldn't be done in a spreadsheet. That's what stickies are for.

  2. It's fun
    You most likely won't reach momentum when re-listening to hours and hours of video material. But you will when you and your colleagues share your findings and remind and inspire each other.

  3. It's inclusive
    By inviting other people to your process, you gain a broader perspective on your research findings, as everyone finds something different interesting.
affinity-diagram_new-3_compressed
After gathering insights, I mapped them on an affinity diagram. By writing down important questions, I guided the discussion and helped my teammates remember interesting bits of information.

Crafting a report to be shared with the wider company

Although I would always favour a collaborative analysis session over independently immersing yourself into data, a nice presentation of the main findings can go a long way when it comes to spreading insights beyond your team.

These are some extracts of the research report I created and shared with stakeholders outside of my direct team:

report-slides_small_compressed

Heavy Seller Personas as a side product

Something that turned out of the research, but had't been planned at first, were personas that represent the people I interviewed. Seeing some clear patterns about motivations and selling strategies emerge, it appeared natural to form groups that represent these different patterns.

The personas also ought to help us make better user-centred decisions when it comes to working on solutions. Even though Vinted has already created personas, these were made for general, more regular users, not necessarily heavy sellers and were therefore less applicable to our use cases.

heavy-seller-personas_compressed
Heavy seller personas derived from the interviews

Still the main outcome of the interviews wasn't supposed to be personas, but a list of problems and needs we could take to the last step of this research project.

SURVEY

Validating problems and needs in all existing markets

The goal of this step was to validate the identified problems and needs at a larger scale, so that we'll get an understanding how big each opportunity is and which ones are worth to investigate further.

In order to do this, we sent a survey to a large number of users. This time, we didn't only invite users from France and Germany, but users from all markets where Vinted is currently active to be able to detect market-related differences.

sample_compressed
Overview of the sample. Over 16,000 users responded.

Measuring frequency and severity

The survey was supposed to identify the size of each problem, but size can't just be measured by frequency alone (how often a problem occurs). We also had to take the severity of each problem into account. Furthermore we wanted to get more qualitative information on how these problems are currently solved. Therefore the survey comprised 3 main questions:

  1. Are you facing this problem?
  2. How frustrating is this problem for you personally?
  3. How do you solve it?
survey_compressed
We only asked question 2 for the problems users selected in question 1.

This is a very powerful approach because the first 2 questions let you end up with a matrix that makes it very clear on which parts you should focus your efforts. For instance, there might be a problem that occurs very frequently, but isn't very frustrating. This problem probably should be prioritised lower than a problem that has a slightly lower frequency, but is causing huge frustration and is therefore a major risk for user satisfaction and retention.

Mixing in open-ended questions

While the first 2 questions were multiple choice questions, we kept the third one open-ended. Why including the third question in the first place? Because it gives important insights on two further aspects:

First, you learn where the 'frustration' from question 2 comes from. Is it because of wasting time, data loss, or because of a very complicated workaround?

Second, it provides you a kick start into ideation because studying current behaviour will help you design a better solution for this behaviour.

IDEATION

Picking one opportunity and getting started

Of course you're job isn't done yet, when you hold the results in your hand. It's likely that several opportunities seem promising, so you still need to pick one to get started with.
Together with the PO, I aligned on the problems we wanted to look at first, presented them to the team to get commitment and then conducted an ideation workshop to think about possible solutions together.

how-might-we-s
To inspire ideation, I rephrased the chosen problems into 'How might we' challenges.

Same as with the initial assumptions workshop, high level ideation is best done in collaboration with people from various backgrounds and disciplines, like product, engineering, design and research, so I invited all of them. Many ideas popped up that were already discussed in the initial workshop. And this was great because now we knew if there is some real human need that justifies working on them.


Solution

Fixing the basics before moving further

After voting for most promising ideas and solutions during the workshop, we now had a clear idea of what's next. Based on our research and prioritisation, the PO created the strategy and roadmap for the team and got sign off from top management.

The research results made us realise, that there are a lot of hygiene factors we have to fix before developing new paid features, so the team will now work on free features for the first time.

And as soon as the basics are fixed, we have a hand full of prioritised ideas and opportunities ready to work on.

LESSONS LEARNED

Don't start with ideas. Start with interviews.

Although the approach described above looks pretty smooth, I went back and forth between ideas and problems several times. Working with people with lots of ideas made it seem quite clear what to build next. In fact, at Vinted new features often derived from one person's vision rather than user research. While having visionary people in the company has many advantages, following their personal ideas can become a risky approach if they are not the users of your product. So always challenge your co-workers' ideas against real user needs.

Focus on behaviour and feelings to reveal underlying problems

Dealing with heavy users is a tricky case, because they're so used to your product, they stop noticing and started accepting the little flaws in your product. Keep asking them about their problems won't get you far. Instead, looking into their workarounds as well as positive and negative feelings will reveal behaviours and needs they wouldn't call 'problems'. Still they might be a huge opportunity for enhancing the user experience.

Don't try to convince people of collaboration, let them experience it

Something I didn't expect was that my colleagues were not really used to cross-function collaboration and workshop settings. I had to bring this into the team and also into the company. While people outside of my product team didn't seem convinced when I told them about my approach, my teammates started loving it and became active contributors as soon as they experienced it themselves.