Hypotheticals in Product Management Meetup: Abridged Transcript

Jen Marshall

In April 2018, Brainmates hosted a Product Talks Sydney Meetup based on the “hypotheticals” format popularised by Geoffrey Robertson.

A bright and brave cross functional team of panellists were on hand to tackle an imagined (but realistic) Product Management scenario. The panellists were:

Mark Cohen, CTO at Domain

Elizabeth Pek, Director of Experience Design at BT Financial Group

Marie Kellett, former CPO eftpos

Richard Marr, APAC Regional Director at Auth0 

Jamie Finnegan, Head of Talent at finder

Clive Lam, Consultant Product Manager at Brainmates

This is an abridged transcript of the Meetup.

Jen Marshall:

An executive in a large Telco has come up with a new product idea. This new idea is going to leverage artificial intelligence to better serve B2B customers with targeted advertising solutions to end customers, or real people out on the street. There’s some really good support for this idea within the executive team, and they think this product is going to help the Telco bring a lot of new clients in as new customers. So Marie, you’re the chief product officer and you’ve been at the offsite with the exec team recently, where all of this was discussed. How are you gonna start to evaluate this idea?

Marie Kellett (CPO):

I think it always starts with the same conversation, what problem are we trying to solve? And what’s that strategy, and how does that fit into what problem we’re trying to solve?

Jen Marshall:

And practically, what steps would you take to start to decide whether we’ve got a problem to solve, and who’s it going to serve?

Marie Kellett (CPO):

I think it’s fundamentally about having a conversation with the end users. So if it’s a B2B environment, it’s a discussion with them. What problems are you having, and what problems are you trying to solve, and what’s our role in that process? And if it’s a B2B and it’s B2C, then that obviously complicates the conversation. But essentially, the B2B environment, that’s who’s buying, and that’s who’s going to be packaging that up and forwarding it onto their customers.

Jen Marshall:

Would you like to send Clive out to actually do some work to help understand this better?

Marie Kellett (CPO):

I think Clive would be a capable guy to go out and do that.

Jen Marshall:

And Mark, you’re leading the developers in this organisation, you’ve known what’s going on, you’ve heard by the water cooler and such what’s going on. So what are your next steps?

Mark Cohen (CTO):

The first thing is to understand who else is actively doing something in the space, and try and get your head around how they’re doing it. So, the benchmark of what the competition… or not necessarily competition, but other players doing similar things you’re doing. That would be my starting point.

Jen Marshall:

And once you start looking around, you’ve kind of found that some of this technology that the people you’re thinking are going to be part of the solution, hasn’t really been proven yet. So what are you going to do now?

Mark Cohen (CTO):

I’m gonna start crossing over into product land.

Jen Marshall:

Tell me about that?

Mark Cohen (CTO):

I’m going to start looking into the real MVP, and when you start working, what that MVP is, I’m gonna look at which vendors we can work with as opposed to building everything from the ground up. Especially if we’re looking at artificial intelligence, it’s very difficult to compete with people who’ve got the masses of data, already got the capability and the training models and the products in the market. So I’d look at, without mentioning big brands, but I’d look at some of the big brand offerings as a starting point.

Jen Marshall:

Okay. In the meantime, there’s a little bit of pressure building to find out how big this thing could possibly be. So I think Marie, you should send Clive out to maybe run some numbers. And Clive, if you’ve got to assess how big this market is gonna be, what’s your approach?

Clive Lam (Product Manager):

I’d be having a look at our customer base. And actually just going in and interviewing, and seeing the amount of messaging and advertising that’s going out, and what it’s costing them and what it’s worth to the business there. So I might pick the top five customers, go out, have a chat with them, understand their business a little bit better, understand the messages that they’re sending out and what that advertising business is worth to them. How much it’s costing them, their targeting mechanisms, and then working out that reach and the effectiveness of that reach. If it’s 60 percent, you’ve got an effectiveness of 60 percent, we have 40 percent decline, and then we can size it based on that. So if we can get you an additional 20 percent, how much is it gonna cost you, what’s that worth to you?

Jen Marshall:

Okay, so let’s imagine that the numbers are starting to stack up, you’ve got some tech solutions that might work, and you’re starting to feel like the customer and the problem might all match up. So you’re thinking you probably need some more people on your team, and you’re going to go and get some advice from Jamie. Jamie, when you’re starting to set up a new team that’s gonna build a new product, what’s the kind of talent that you’d be looking for and what’s the advice that you’d be potentially giving to Marie and Mark here about how to start building that team?

Jamie Finnegan (Head of Talent):

So I guess the first thing would be to sit down with Mark and Marie and work out, really, as you mentioned, what’s the business problem we’re trying to solve here, and then work out what’s the actual core talent we’re trying to build within the team to try to solve that problem? You might look at things like the competitors in the space. Are we’re gonna target direct competitors, are we not? Are we looking at people who’ve kind of done this in the past, or are we looking to build up new talent, and what level are we trying to build from there as well, is it C-level kind of higher, or is it more of an operational play? Is it a bit of both? Which comes before which? I guess from that briefing perspective, you’d also probably break down exactly as you mentioned, business problem, you look at the key pillars of the role, and two or three key things that you need to be absolutely fantastic at to be heard in this business. We’re also looking at this business as a thing that’s just been founded, and it’s a new, so we’re probably looking at our positioning in market and how we’re going to pitch this to candidates, the actual message that we’re trying to send there, and the purpose and vision of the company.

Jen Marshall:

So Elizabeth, you’ve arrived, it’s your first day and you’ve been given a PowerPoint deck with some customer numbers and market opportunity, and a bit more background about the technology and the customer and the problem. You are genuinely excited about the opportunity. You can see it. But also of course, you’ve got a lot of questions. How are you going to start off?

Elizabeth Pek (Head of UX):

Yeah, if I was in that position, I’d always start off with the whys. So back to what problem are we trying to solve, then the next I would be asking is the who. So we’ve got the numbers, we’ve been looking into those market segments, but then as a designer, I would wanna go a bit deeper and actually try to understand, it’s not just about the market segments, but do we have any other information around behavioural aspects of that customer base as well? As Marie mentioned, if it is a B2B, B2C, then that gets very complicated. I think it’s really important, when you’re starting off in a role and if you’re also looking to build a team, you have to be on the same page in terms of the vision of what it is that you’re trying to achieve. And just through experience, I think that can be weird. Early businesses can unravel very quickly when they don’t have that vision clearly articulated, or everyone’s not on the same page on that vision.

Jen Marshall:

So Richard, you’re busy putting together your team. You’re about to have to sell this thing. You have some steep sales targets, that’s never happened before. From a marketing and sales perspective, what’s your main focus at this point?

Richard Marr (Head of Sales and Marketing):

So probably, the biggest focus that I look to is really understanding the product itself. The pains, the solves, the reasons those customers have that pain, the impacts of the business, and importantly, the unique value proposition that the solution has, right? So there’s something you’re gonna take to market, because B2B advertising is done and dusted. It’s not unique. So there’s obviously something that you’re going to market with that’s quite unique, which is the key word that I heard, there was AI. So I’d really be double-clicking into that, and sort of form a target team, a sales team that would help us provide a feedback loop into the product team as we’re going out trying to sell at early adopters, and hoping that continuously improves another product right. Because to be honest, your customers are going to drive the product lifecycle. And that’s gonna be, probably a tension that you feel, that the customers are really going to try and push the product in certain ways maybe we don’t wanna take it.

Jen Marshall:

I’m going to come back to Jamie, because I think now we’re getting to that point now where we’re starting to get past the paper prototyping and the research, and we’re really starting to build stuff, and you’re trying to bring developers on and it can be hard to find some good developer talent. What’re the trade-offs going to be in terms of getting those people on board? How do you find that balance of getting somebody on board, but finding the right person?

Jamie Finnegan (Head of Talent):

Good question. I think one thing that’s non-negotiable, and that is quality. So increase speed doesn’t mean you decrease quality. You either build, partner, or acquire, so you’re either looking to things matching around your business, so probably not going to be acquiring other businesses, dependent on what funding we’ve been given at the time.

Jen Marshall:

No funding for us.

Jamie Finnegan (Head of Talent):

So you’re probably looking at, can we potentially fill in these outer stage gaps with some contract or a temporary staff in the early days, but also alongside some really key permanent hires. I guess those are some of the key things you could do to fill in for some of that. I think it’s an early stage business, it’s a growth company, you’re probably gonna want people who come from those types of backgrounds who are gonna understand they might have to wear more hats than normal, and things outside of their normal safety net. I think hiring for that volume and attitude in people, and maybe that experience is probably gonna be more beneficial in the early days certainly for that growth and scale, then, and potentially hiring people you haven’t experienced at that stage of growth in the business before.

Jen Marshall:

Mark, so you’ve got a full team. You’ve got one scrum team. Are you happy with that?

Mark Cohen (CTO):

I’m happy with one scrum team to start working on the first MVP. As long as we’ve got one scrum master team lead person who can actually run the scrum and help with the, planning the backlog.

Jen Marshall:

So you’re starting to validate your prototypes, and you’re starting to come across some truths. You’re still testing with some of the small business owners, and you thought that they might’ve been actually your real market. But what you’re realising as you keep going forward with your testing, is that they can’t really afford to pay. They’re happy with the solutions that they’ve got. The glimmer of light that you found, is that the larger-sized businesses will pay. But once you start to plug those numbers into your model, financially, it doesn’t really stack up as it had done previously in all the ways that you’d hoped. So Mark and Marie, you’re about to have to go back to your board and explain what’s happening, and give them an update in a week. How are you going to approach this conversation?

Marie Kellett (CPO):

I think there’s a conversation with the board that needs to be had, to say look, “here’s what we thought” But there is also, “here is what we’re exploring.” Whether or not we’ve worked out that me and Mark, I don’t quite know, but let’s assume that we have. I think it’s about painting a picture of, “stay with us”. It still fits with the vision, still fits with the demand and direction that the company wants to go. I think it’s still about trying to keep them engaged in that regard.

Mark Cohen (CTO):

Also, Marie, myself, Jamie, and Clive, we’re all pretty smart. And we have one scrum team, and we’ve run a real MVP, and we’ve done some real good experimentations. So it’s an experiment. We haven’t wasted the board’s money, we’ve proven or disproven, or figured out what’s going to work. So we’re giving them the results of an experiment as opposed to bad news.

Jen Marshall:

Well let’s just imagine with the flick of a magic wand that we’ve convinced the board, and we’ve cleared up our issues around funding, and we can continue on with development. Things are starting to go more smoothly, and in the meantime, one of your brightest developers have started working on some of the artificial intelligence aspects of the product. Along with the location technology you have as a telco, the team’s beginning to see that you’ll be able to predict where people are gonna be on the weekends so you can target them with direct advertising. But there are a couple of people in the product team and the team more generally who have a few ethical concerns about this. Elizabeth, how could you start to unpick this problem, do you think? Would you have any ways to explore this?

Elizabeth Pek (Head of UX):

I think you would have to have a look at sort of existing services that are out there, and seeing what the reaction to that is, or the sentiments are around that. I’m a designer and also a researcher, so I would want to use my tools to be able to unpick that specifically for, and predict that for our customers. So conducting some kind of research, maybe, in an anonymous way, so that people aren’t going to be biased. Being able to understand what might be the potential ethical issues around it.

Mark Cohen (CTO):

So I’m actually the tech person that has the issue with it. The reason why, is because the world’s changed a whole lot. When we started working on the product, we might have had terms and conditions that said, “when you use our product, we will be tracking where you are and what you’re doing, and by using the product you’re accepting that.” Even though legally we’re doing nothing wrong, the whole Cambridge Analytica thing has proven there’s a difference between ethics and legal. And you have to do the front page test. So, take a screenshot of the Sydney Morning Herald, put in your action on the headline, and say, “our new startup is tracking where you are so we can target advertising because we think we know where you’re gonna be.” Does it pass the test? You know, show it to a few people and see how they react. And the front page test is ultimately more important than the legal test at the moment.

Richard Marr (Head of Sales and Marketing):

I think if you look at what’s happening with Facebook and Tumblr, et cetera, there’s some questionable things there. If that’s what it is and we’re predicting where people will be, I think it’s a good conversation to have, but I think it’s sort of a… disrupting technology is gonna encounter some of those problems, right? So we’re gonna come up with some of those answers to those questions.

Jen Marshall:

And in the meantime, you’ve got a bit of an issue, because when you secured the last round of funding, it was on the condition that the product would start demonstrating some ROI in about June. And of course, it’s now April, and so Richard’s starting to sweat. In the meantime, one of the executives from the broader company has been out at a conference saying that they’ve got a new AI product launching soon, and some journalists start asking questions. What will you say?

Mark Cohen (CTO):

I’d seek council. If we’ve got PR, I’d be looking for corporate communications as well. It’d sorta be difficult, so I’d try and get them excited without giving too much away. To be quite honest. Sort of like Elon Musk.

Jen Marshall:

What does that look like?

Mark Cohen (CTO):

I’d sort of say, “well, we’ve seen exciting results, we’ve so many customers that’ve seen return of investment of 554 percent, and paid that through three, three and a half months.” You know, for B2B advertising, we’ve really leaped from the competitors, and get them excited, but I can’t talk too much about that. Sort of get them thinking, “well, hang on, there’s something really there that’s exciting.” You know, which would be my job. That would be my job, I’d be doing my job by doing that. So I wouldn’t feel guilty.

Jen Marshall:

Clive, they’ve also got your number ’cause you’re mates with one of the journalists at TechCrunch or something. So if they come to you behind the scenes and ask for a bit of off-the-record, what are you gonna do?

Jen Marshall:

I’ve got some good news for the people on the panel, because you have got the project to launch, the product’s launched, and you’re going to go out for a nice dinner and celebrate as a team and have a few beers. The acquisition numbers are not quite what you hoped for, but what’s really exciting is that the engagement and the way the ad serving’s working, you’re really happy with that and it’s cutting through to the end user. The team comes together shortly after this with a meeting with some of the tech folks, and some of the developers have started raising some issues about the back end of the product. What they’re saying is, is that they’ve got some concerns that if the total number of users was to rise very quickly, that that might cause the system to crash. In the meantime, marketing is planning some pretty big launch campaigns which could mean that numbers could skyrocket.

Jen Marshall:

Clive, you’re in the meeting with the team. What do you do now?

Clive Lam (Product Manager):

So one of the first things I’d probably do is actually have a look at what our projections are. Current trajectory of growth… also then, work out where that turning point is where we would need to start looking at scaling up, and what that would take. So I’d wanna have a little bit of information, at least in the back of my head, back of the napkin to look at sizing. And then I would be walking over, and pulling Richard aside saying, “hey, how big is this gonna be?” And Mark is like, “hey, just to let you know…” so I’d get both of you guys together, and also Marie, just so that I’m making sure that you guys are aware of what’s going on and say, “hey, this is what I’ve been hearing. If we scale up beyond point X number of customers, we’re gonna have some problems. So what’s our projection look like, and what’s a conservative estimate, what’s the best case scenario if we go absolutely gangbusters on this, what’s that sizing look like and I’d go back to Mark, “look, how do we solve this problem?”

Mark Cohen (CTO):

Because it’s a relatively new product, well, it’s a very new product, we’re gonna have built it using cloud infrastructure and data centres, so it’s very quick to scale the hardware side. And straight away, we’re gonna peel a team off and have them focus on performance optimization before anything changes, and make sure that the finance guys know to expect a big invoice for the cloud hosting.

Richard Marr (Head of Sales and Marketing):

Mark, he doesn’t look like a man under pressure at the moment, but if he was a man under pressure, I’d be really having a serious conversation about, “Mark, should we be concerned? Do we need to drip-feed these customers onto the platform?” And sort of be having another conversation with them trying not to prepare the customers, to say, “look, yeah, we’ve got sort of a soft launch, you’ll be fed into the pipeline as the platform starts off.” And so that there was no expectations that we’re gonna be a big bang with a gazillion customers in one hit, right. Because from a sales perspective, the worst thing that could happen in my perspective, is that we have a catastrophe like a Melbourne Cup disaster where you can’t place bets, and things like that, or Black Friday. That’s probably what I would do, if Mark, but he doesn’t look like a man under pressure.

Jen Marshall:

So since we’ve resolved all of our technical issues and the marketing plan, the marketing launch went to plan, you guys are now in fast growth, you’re beating all of your targets, and since now telcos all around the world want to buy this same technology so that they can unleash it into their markets, you’re making huge bonuses. That is a fairytale ending, please give this panel a big round of applause, thank you.

Jen Marshall

Jen Marshall | Author

Jen is CEO at Brainmates. She's interested in building the profile of Product Management and contributing to a rich discussion on the state of Product Management in Australia.