Mike Gozzo, CPO at Ada: Conversational AI and Leadership

yeah okay

record oh now we are okay there you go

all right uh so we can jump right into

it so hi Mike um so hi everyone my name

is Finn I am head of product at Zite

previously called Cloud app and we

started this little podcast called

inside the workflow where for a few

minutes you know 10 20 minutes uh we’re

going to talk to Executives about the

way uh you know they like to work and

today um I’m meeting an old friend Mike

who is a product leader in montre

uh so Mike welcome nice to see you again

Finn it’s been a while yeah yeah yeah

for for people that don’t know you yet

you know can you tell us a little you

know one minute about who you are what

do you do who do you focus on sure um so

career perspective my name is Mike godo

I work in and live in Montreal up in

Canada um I’ve been in the startup scene

and building companies for over a decade

now I had one small company I built and

sold uh back in 2012 to 2014 then I

started a company called um smooch that

was um a company that I started while

working as a consultant for um another

startup and with the founders of that

startup was able to buy the IP assets

spin it out as its own thing that was a

messaging communication platform as a

service um did business and competed

with like the likes of twilio and did

business with all of the major players

in uh customer experience expence one of

those players uh bought us in 2019 was a

great exit for us I went on to work at

zendesk I became an SVP of product over

at zes responsible for the road map and

products globally um and then at a

certain point I just missed working with

a Canadian company working with a

smaller team I wanted to get a lot

closer to llms that I had seen doing

early work um in gpt3 land with opening

eye back at zenas and I want get heck of

a lot closer to that and I joined Ada

which is the

um a lot of fun so Ada is a Canadian uh

company that builds autonomous agents

for customer support automation so our

AI agents you onboard hire manage and

promote them just like you would a human

agent and they’re able to automatically

reason through complex customer queries

by individually looking at every

question that comes in and figuring out

how they should go about solving it

using the knowledge that your company

has using the tools that your support

agents currently have

access to and bringing that all together

to really act as a virtual agent we’re

about 300 people or so 350 people spread

out across North America Europe and

Israel awesome and yeah I think the way

I actually reconnected with you is that

we use Ada at Zite um we love the tool

we started like a little slack Channel

like project adaa and we just really

wanted to get it out uh out the door to

help us you know handle the volume of

support request that we have uh the

support team so far you know loves the

the different handoffs that that they

got so yeah I think I started um I met

you around the time where you started

smooch smooch doio love the name and

then I saw that it got you know acquired

by Zenex so you know yeah great great

path there and it looks like there’s a

trend where you’re focused really on

like conversation and and support teams

correct yeah I would say that my

intention going into this space and

building products in this area was not

to do customer support I had this dream

ever since I was a kid

I was a huge I I was I am a huge Star

Trek fan and I wanted to build a

conversational computer I thought that

was the most amazing thing I had ever

seen um and I wanted to be able to have

these kinds of interactions with um not

only computers but also virtual people

huge fan of the holc and all of those

story lines about that so um as I got

into this thinking around conversational

Tech that took me on a journey of

understanding like what was

state-of-the-art what was possible and I

learned very quickly that before you

could even get to automation at least in

the earlier companies that I built that

there was just frankly platform problems

to resolve and how data and messages and

conversations would be stored and

organized and transmitted from point A

to point B so I found that my whole

career now has been about well how do I

start at the platform level where we

built that we understand how that works

then going up to the human level and how

they interact through digital platforms

and now thinking well how did the

computer itself interact so finally that

top level of what these systems look

like and support is really the entry

point here right like at the end of the

day we’re going to create the concept of

a virtual customer facing human being

that is able or artificial being that is

able to interact with and work with um

so many different systems and different

ways that you might hire it and put it

to work um and that’s a pretty exciting

thing to work on yeah 100% And I I think

I connect with you there where I um you

know we grew up in a time where we had

all of these great ideas through movies

of like here is how communication should

work here is how Tech should work and 20

years later it happens or we have all

the tools that we can uh we can like put

together it’s starting to take shape

we’re starting to live in the future

which is pretty cool exactly no Fine

Cars but we got some great little like

tools that can help everybody work

faster um in terms of AI and Ada like I

think you mentioned um around the time

of CH gpt3 is that when the company got

started or did it do something before or

no uh interestingly a and smooch got

started um at roughly the same time and

that’s when I first met Adas Founders we

were off building smoochi Montreal Adas

Founders were building Ada in Toronto

and they came at it from the perspective

of being customer service agents

themselves and trying to automate away

their job and the technology they were

using at the time was state-of-the-art

was basically using natural language

understanding to map a given utterance

or something that a customer would say

to a set of pre-programmed intents and

train intents in the system that worked

actually and is the foundation of many

of the chat Bots that you see U and you

might encounter on the web today but

that that uh capability what it really

struggled with is because it was built

on pre-programmed intents and workflows

really struggled to adapt to the nuances

and flexibility of customer service

because those conversations can go

anywhere as you know there people are in

crisis or they’re upset about something

or they’re curious about something and

their dialogue might go places you would

never predict and often chat Bots would

frustrate people because they would get

in the way and try and force you down a

pth

um it’s why when I started at Ada the

first thing I did was I authored a

product strategy and in that product

strategy I said well like let’s go back

through first principles and say what is

this business trying to do and the first

goal of this product was to

automatically resolve customer service

issues and for you know the entire

company’s history call it since 2015 or

so it had been measuring just like

everybody else in the industry a metric

called containment rate containment rate

refers to what percentage of contacts

that you’re getting from your cust

customers are you able to prevent from

going to a human customer support agent

Contin rate cost savings but I said this

seems suspect because we were getting

like 80% containment 90% containment in

certain cases but you look at the

experience that those systems provide

and they were pretty poor just as a

casual person experiencing this you

realize that this wasn’t perfect

customer support so we said okay let’s

actually now go and do manual work to

read all these conversations and figure

out of the conversations that we’re

having

how many did the bot actually resolve

how many did it actually perform as well

as a human would be and what we found

was that that number was actually way

closer to 30% and we did this by

manually annotating conversations across

like literally like tens of thousands of

conversations across or hundreds of

millions of processed

conversations and that got us to work

we’re like okay we need to figure out

how do we actually evaluate true

resolution how do we drive that number

up and give our company our customers

tools to do that and the minute we

shifted our metric and our focus on

being this high quality High bar for

resolution the easier became to

prioritize what we were doing in product

to prioritize how we were going to

market with the technology because now

we have a way of really claiming and

showing our value in a way that’s much

more um much more connected to what the

customer actually

experiences awesome yeah I feel like the

the more senior I get the more I I trust

the data but I need to verify the data

you know like

okay great we got this great metrix if

it’s too good to be true there is

probably a problem somewhere with with

with the data as well um related to AI

you know like where do you think it’s

going uh maybe I don’t know if it’s on a

it’s a very broad question super broad

question but I think specifically for

you like in in the support world you

know I mean look what I mentioned

earlier that when I started at Ada two

years ago we were automatically

resolving 30% of the conversations we

were getting despite containing

80% right now if we look at the adoptees

of our platform that is built entirely

on large language model that cohort of

customers is automatically resolving

north of 60% and the best ones are

resolving not containing 80% these are

conversations where from a human

perspective it’s really hard to

distinguish that it’s you’re not talking

to another person and that you’re

talking to an AI agent um with that

growth rate I think the first thing

we’re going to see is that on this

metric of automated resolution where the

system is providing accurate safe and

relevant responses to the questions it’s

asked we’re probably going to see the

first business emerge that has a 100%

automated resolution rate in the next 12

to 14 months like that if we just look

at the the straight line on the graph

that’s where it’s tending towards um so

I think that’s the first place but it

doesn’t stop there you know my view is

that as we start to um as we get deeper

into this into the

system the hard part is not going to be

the intelligence and how smart these

systems are everyone’s going to be able

to buy really cheap really good

intelligence every model generation that

whatever company comes up with is going

to be better and better and better the

challenge for businesses is how do you

manage an align it right so you might

purchase you know a great system of

intelligence but what are the levers

that you have there going be two things

what are the levers that you have to

control it so it gets the job you need

to get done done you know support is a

very widespread and different companies

want to do it in different ways people

throw in Revenue generation into support

there’s other like customer facing paths

that all get muddled together so the

importance is going to be how do you

control the system and align it to what

you want to do and the other important

is going to be how do you verifiably

trust the system so how transparent is

it how do you work with that so if I

think about like our own road map and

then I think more broadly to the

industry I think what you’re going to

see is AI where the cost and quality of

inference quality goes way up cost goes

way down while at the same

time um you know the companies that are

going to win are the ones that provide

tools that businesses can use that are

self-served that are easy for them to

achieve that alignment and Achieve faith

in the system working properly those two

things Happ and I think we’re in a good

place I to totally agree with you I

think even when we started looking at a

tool like Ada and and adding it in our

into our system our goal was like to

speed up a lot of the interaction

interactions that previously would just

be on a queue for 24 hours

um so we want to have a good hybrid

model where our support agents are

helping on the tough problems that are

probably harder to solve to to you know

set a better process we still in our

first days with Ada we looked at uh all

the different chats and you know try to

make sure okay it’s actually helping or

like maybe let’s add some tagging here

so we can look at it later on um so that

that’s really interesting um in working

with support teams you know zit also

focuses is quite a bit on on support we

do like a screen recorder so it’s a lot

of support agent recording like here’s

how you need to solve a problem and then

showing it to the customers and it’s

interesting how we started with a simple

tool that is like record your screen but

we see more and more needs for like

automation like uh right now a support

agent is recording a 10-minute video

walk through the the person at the end

of the line just wants a transcript just

wants the summary they want to be able

to uh to quickly you know access on on

the the video and not wait 24 hours wait

to get on a call so like really trying

to to speed things up um since you have

a ton of experience like working with

support teams or support companies that

have large support uh agent teams what’s

the best thing about working with

them I mean I think the some of the

things that was surprising to me the

more I started to work with companies

and their support teams is that we have

often as consumers this perspective that

they are these you know you call into a

customer support angle and the people

working there are um in a cost center

miserable not able to innovate don’t

have vision and whatever else it

actually couldn’t be further from the

truth like one thing that I found in

common with almost every support leader

that I’ve spoken to is tremendous amount

of empathy and a huge desire to drive

great experiences for their customer I

think they have Pik resources in like

economics that make it hard for them to

deliver that as well as it can be but

you know even the most evil people

people that the people should call them

evil but that you know the the Reddit

will often you know make fun of the comp

people or whoever else at a big Telo or

or whatnot um and it’s fun to knock on

them and you can tell lots of hilarious

stories but when you actually meet the

people behind those customer service

experiences they have a vision and

they’re trying to work towards it but

the scale and complexity of it makes it

really hard it’s why I think technolog

is essential right it’s what abl to

connect people that have a very strong

opinion of CX and what experience should

be with the reality of delivering it and

that’s how we’re going to get there

awesome yeah I started my career working

in a call center I was working for a uh

uh the Orest Sy the mreal so it was like

selling music tickets and everybody I

was working with they were all super

passionate about music and here’s what

you need to do and here’s how I can help

you um and and uh then I I worked for

the Apple store which like it’s funny

how it’s it’s all customer support all

helping customers and apple the one

criteria that they had to for hires was

empathy like if you’re a nice person

they will hire you and that they will

try to like it doesn’t matter if you’re

not uh uh intact the person you’re

talking to wants help wants someone that

is nice to interact with and a quick

answer so whatever you can do to to help

them up skill yeah um one thing that

that surprised me uh recently is the

this title of cpto so Chief product and

Technology officer I totally relate to

that to that title you know as I started

as a developer then went to product uh

can you tell us a little bit you know

about the kind of role that this entails

versus just a traditional CPU or CTO I

mean it’s definitely something that I

would say in the last in the last few

years we’ve seen emergence of quite

strongly I’ve been meeting many other

folks that are in exactly the same

perspective um folks the title cpto

might be a little bit new and less

common but actually this concept of

bringing together product and

Engineering under one leader um is being

seen in many larger companies as well um

interestingly at zenes their scale where

was like 5,000 employees globally um

when we were acquired U both product and

Engineering svps reported into a

president of product development um

whose role was to oversee both and think

through like how do we bring through the

market strategy with the execution of

this um now that person has moved on to

a different role but they still have it

together within essentially a cpto as

well so like it works at a larger scale

but I think the reason why I think it

works is that you get to have like one

one like unified vision and strategy and

most importantly one accountable person

for the technical product offering that

is coming off the line right it means

that you have one person ultimately who

is assessing the balance be technical

improvements quality of architecture and

all of the stuff that’s really really

important in building a product and

customer value and is able to answer to

the CEO and the board to explain why the

balance is what it is and navigate that

it avoids a lot of the struggles that

you can often have between a head of

product and a head of engineering where

you know there’s there could be like

there’s often debates around timelines

or debates around like um equality

perspective and what this forces to

happen is you start thinking about

architectural goodness level of quality

level of security other concerns as prod

features and as things that are

marketable and are part of the whole

package so I actually think that like

while we’ve seen this in the past in

some circumstances I think we’re going

to see it way more uh particularly you

know since um coming out of out of 2022

and the blood bath that we’ve seen in

SAS there’s a lot B call for efficiency

and a big call for like strong Vision

strong leadership and less time in

debate and more time executing um so I

think it it just drives up efficiency

over overall awesome yeah I I I do

connect with that and even talking to

myn of engineering Sachin um you know

even at the board meetings we we now

take a relay on like I’ll do the update

this time you’ll do the up dat the other

time because at the end of the day

technology and product are the same

thing so he has a specific skill set

related to security that I don’t have

I’m more on the you know customer

interviews customer focus so we

complement each other but I see at like

at some level or someone that can do

both amazing you know that probably uh

and I think you’re gonna see it the

other way you’ll see it the you’ll see

it the other way as well right there was

a great um tweet by Brandon Chu um

formerly at Shopify a couple of days ago

that you know really reflects something

that I do a lot in my role even though

like I’ve got you know huge team and

lots of lots of resources when I need to

demonstrate an idea I’ll whip up a code

editor and I’ll build a PC of it at

night and be able to show my team the

next day at the working PC to drive a

lineman and I do this the time and I

actually think my best PMS are the ones

that do this as well that don’t just

talk inigma and mockups or like prds and

documents but that will actually be able

to demonstrate 90% of the capability of

the interaction to drive alignment so I

think what you’re going to see is

that especially as we get more AI

assisted coding tools particularly as

like it becomes easier to open up cursor

or whatever and like generate your ideas

and work through it it’s going to be so

important for product managers to think

like Engineers like going he he put a

very you know controversial statement

out there Brandon he said something like

maybe within five years we’re going to

see that product and design teams are

the ones who build features and

Engineering teams built infrastructure

and security and hardening and

reliability it might actually this way

right that like we product and design

teams build user facing features end to

end I could see that happening if they

squint not sure we’re ready quite yet um

but I think there is there is a

potential there yeah I I saw that tweet

and 100% I I always saw the need of

being somewhat technical as a product

manager not just customers focused or

like you need to be well-rounded um and

we’re in a great age where I can do

proof of concepts with Chad dpte U or

even alone but it speeds it speeds up my

workflow and in instead of just

delivering mockups sometimes it’s like e

either a straight up small code change

that I want to do and I don’t want to

add it on a ticket put in a backlog

absolutely and I think the tool that I

would support most there is like a

developer just like you have a developer

turned startup found there and turn PM

right is that um this tool called cursor

cursor. I’ll def check it out it’s like

a it’s a version of Visual Studio code

that has been tuned to have some AI

features built into it deeply it is the

best experience I found so far for like

pair programming with chat GPT where if

you know what you’re doing if you’re not

a developer it’s not going to help you

become a developer I don’t think but if

you understand how to write code and how

these systems should work you’re able to

be so effective when you’re working with

this thing I’ve actually even said like

maybe I should like go on Twitch and

live stream some session of building an

app with it um because I’ll watch that

I’ll watch that 100% I probably should

do that uh but I think like there it’s

just it’s pretty magical to use and you

can you can the rate that it accelerates

you when you’re a product person who is

messing around with code is actually

pretty incredible yeah I I do a lot of

building public stuff on Twitter um and

where I help my wife build her business

and like I learned react I was like all

right let’s just open a chat gbt window

uh vs code window and let’s get started

and it’s crazy H I even just copy paste

the whole code help me you know clean

this up and tell me why like so I can

learn the language at the same time so

it’s so try try kther I think you’re G

to be pretty impressed with what it’s

able to do we’ll do 100% um just to to

uh I think spend a few minutes uh to to

end this podcast on you know you and how

you work with your teams so I I’m

guessing with a company of 300 plus

people you know you get email slack

phone calls meetings what’s your overall

you know policy how do you manage to

scale yourself up to to the level of

your team this is really hard and I

don’t think that if anyone says in the

industry they’re doing this well I think

they’re lying you know first thing first

line of defense is a privilege but my my

assistant Sarah is extremely good at

just making sure that my time is

balanced and managed and that the right

things areed me so like without doubt

this is great I’m actually looking

forward to a time when there’s tools

that allow more people to experience

this because the more you give it up the

better and you give that agency up to

someone the better it is beyond that

things that we hold really dear one is

making sure that there’s like a

dedicated day in the week without any

meetings or interruptions allowed so

that’s Tuesdays for us I think that has

been a real Boon um and you know booking

a meeting on a Tuesday is a serious

thing and we try and avoid it beyond

that you know my my directs will either

be on weekly or bi-weekly one-on ones

depending on their seniority what they

where they need to be at any time I’ve

got a weekly team meeting for my

leadership team um and then as execs we

tend to meet once a week generally

sometimes twice a week tend to do as

many offsites as we can because we’re a

remote company we did bring in a

flexible travel policy which has been a

boon for us where basically we allow our

teams to travel as much as they want

whenever they want again within reason

to be able to accomplish business goals

we see this a lot with engineering teams

that are distributed coming together to

a city at the beginning of a quarter or

mid quarter work out anything related to

the planning and architecture and then

slit up after a few days and go back and

do something the big unlock for us was

removing the red tape for that and

making it easy for people to just go in

the system book we obviously track it

but we haven’t seen any abuse in the

year and a half that we’ve been doing it

it’s just been a really powerful way for

line particularly in product

development amazing yeah we we are also

a 100% remote company adite so like we

have headquarters in uh and not

headquarters but we have a lot of people

in Utah and some in Poland so next month

we’re all traveling to uh a few of us

are traveling to Poland meet the team

and you know eat the delicious food from

there and you know a good time to B um

the meting day Tuesdays yeah I think

that’s like uninterrupted time for a

long period of time without a quick

stand up even a quick stand up I find

you know overtime can just be

a I should the the other thing that’s

really helpful I don’t do this with my

own team but I do with my executive team

is every Monday and we don’t all do this

but some of us it’s a weird thing but

every Monday I’ll write out my

priorities for the week it’s something

I’ve been doing since the smooch days

where I basically like this is what my

what is occupying my head space this

week just know about it and I found when

my colleagues do that too I find super

useful and I’d like to think that that

practice also helps folks because

there’s a lot of things that you can’t

see in the meeting you can’t see in the

regular discussion so really important

to say like these this is what’s worried

me this is what I’m inspecting this is

where I think there’s a problem

particularly like in a big org with a

complicated product like often I have to

go into the details to like look at

something that I think is not right um

so I’ll share those uh those very

transparently and I think it’s a habit

that if more people got into being

vulnerable and open with like what’s

working well what’s not working well on

a regular basis we collaborate a bit

better awesome and do you use any like

framework like EOS or like for for these

weekly weekly exact meetings or is it

mostly like priorities Round Table I we

have we have like some framework that

works for us I don’t know if it’s comes

from any system but we we basically have

like top of Mind issues for everybody

and we’ve got more long running issues

and obviously we care about our core

metrics and everything starts with the

metric view where we’re at what’s in the

way how are we

trending awesome yeah we we do something

similar but uh it’s like we follow the

EOS which is like yeah updates from the

team scorecard company issues uh people

issues like we just go through yeah I

say we’re not we’re not that structured

we should be but we’re not it doesn’t

look it it sounds structured but it’s

like as a template we c um awesome

awesome I want to be really you know

mindful of your time uh so thank you so

much for the conversation any you know

where can people find you um where can

they find

Ada uh Ada is over at ada. CX check us

out there’s a free trial available if

you want to touch and play with the

system on your own as well that’s

recently developed um otherwise you can

find me on LinkedIn or on XS well

Twitter amazing thank you Mike thanks