How do you measure chatbot success?

According to John Straw, senior advisor to McKinsey and Co and the opening speaker at Econsultancy’s Supercharged event, it is simply when the user does not realise they are being served by a robot.

Sounds pretty simple when you put it like that, right? 

Of course, actually getting to this point isn’t quite so easy. Neither is convincing businesses that artificial intelligence is actually worth investing in, especially considering it is nearing the dreaded “trough of disillusionment” on the infamous Gartner Hype Cycle.

Reflecting the various examples of brand chatbots we’ve seen throughout the past year or so, the conversation at Supercharged ranged from the inspiring to the silly. Here’s a summary of the day’s biggest talking points, along with insight into how brands of all kinds are implementing artificial intelligence.

Rapid rate of change

While many people can get carried away with what artificial intelligence might look like far into the future, John Straw kicked off Supercharged with an inspiring talk about how the technology will evolve in the next couple of years.

Right now, of course, it has its limitations, with most marketers creating augmented decision trees and calling it a chatbot. Then again, John reminded us of the prediction that bots will be in everyday use by 2020, also suggesting that the rapid rate at which the technology is evolving means the bots will look (and sound) far different to how they do now. In fact, he said that by mid-2018, the technology will have advanced so much that users won’t even realising they’re talking to a bot. 

As someone who has reviewed quite a few (mediocre) examples in the past year or so (not counting our own, of course), I feel that John's prediction sounds rather optimistic. 

Then again, as John explained, just because we’re not seeing the technology in practice right now, does not mean it is not in existence. Take the healthcare sector, for instance, where new companies such as HealthTap and Babylon Health are looking to revolutionise the early stages of patient diagnosis. 

Instead of endlessly waiting on hold to speak to a human or Googling their aches and pains, patients can liaise with AI-powered doctors to speed up and streamline the process.

As John said, the net benefit of this kind of technology is greater satisfaction, not just in the context of a doctor-patient scenario but in relation to all kinds of customer service. Instead of being passed from pillar to post and ending up “talking to a 19-year-old in a call centre”, people will be able to talk to a single entity to get the answer they want much faster. 

The benefits of NLP

A lot of brand chatbots involve scripts and decision trees to force users down a specific path. And while some can be frustratingly limited, others can work surprisingly well.

Alex Miller from Byte London cited Adidas as a prime example, with the sports brand using a scripted chatbot to enable Facebook Messenger users to book a free session in an East London fitness studio. Users could interact with the bot to book times, get reminders, and find out location details. The results showed a 76% retention rate after 23 weeks, 1.1m interactions, and 46,000 fitness sessions organised in all. 

So, scripted bots can work well for events, but what about scenarios where users are more inclined to ask questions?

JustEat is one brand that has successfully combined scripted technology with NLP (natural language processing), going on to create a chatbot that is both functional and entertaining.

To do so, it put together a large collection of possible user queries, alongside a list of how the bot would answer in response.

Of course, this still has its limitations. There’s only a certain amount of language it is programmed to recognise, however it's still a good example of a bot that goes beyond basic commands to inject personality and humour into the mix.  

For JustEat, it meant that 40% of people who interacted with the bot went on to actually place an order online, as well as the brand seeing an average dwell time of 2mins 14secs.

Programming personality into AI

Speaking of personality... according to Nick Asbury, writer for Creative Review and one-half of agency Asbury & Asbury, character remains a largely untapped area of artificial intelligence. 

This seems strange, he suggests, especially considering humans are instinctively drawn to any kind of inanimate object that appears to have a personality. Meanwhile, with most humans naturally inclined to choose text or email – even in the context of social relationships – why would we want to spend time having a conversation with Amazon's Alexa when we could skim-read textual information? 

Putting these negatives aside, the positive is that most people are also open to the idea of artificial intelligence taking on more human characteristics. As Nick explained, we’ve traditionally seen this in popular culture, with robots taking on all kinds of human traits in films ranging from Knight Rider to 2001: A Space Odyssey.

Ultimately, this means that there is a huge amount of unexplored territory in terms of chatbot tone and personality. If ‘neutral’ or an Alexa-type chatbot is the middle of the spectrum, a large percentage of all brand communication does not tend to stray very far from this. 

So, instead of concentrating on just one aspect (either functionality or personality) Nick suggests that brands should explore different areas of the tonal map – even embrace sounding like a robot. 

Nick specifically mentioned Zhuck – an app that Asbury & Asbury worked on in partnership with a Russian bank. Described as an ‘endearingly grumpy smart ass’, it was deliberately designed to be more interesting and engaging to use, with a character that set out to entertain as much as serve a functional purpose. 

Fusing AI with human roles

Unsurprisingly, a lot of discussion at Supercharged revolved around the automation of jobs, and the natural backlash that has occurred because of it.

So, from a marketer’s perspective, will we see AI disrupt specific areas such as content creation? And what about from a wider branding perspective – could we even see artificial intelligence informing brand straplines or mission statements?

While companies such as Phrasee (which uses software to generate email subject lines) shows that artificial intelligence can beat humans in terms of scale and immediacy, it still feels like we’re a long way from bots replacing human creativity.

Jukedeck is a company that uses artificial intelligence to compose music that’s suited to individual needs and contexts. Patrick Stobbs, the company’s co-founder, gave some interesting insight into this idea. When asked whether or not this kind of technology creates a filter bubble, he argued that – in contrast – it actually gives creative people the tools to improve and enhance their craft.

Other brands at Supercharged spoke about how they are using artificial intelligence to streamline services, as well as to upskill and aid traditional roles rather than automate them out. 

Nicola Millard from BT suggested that most jobs are made up of an intricate series of tasks, regardless of seniority level or industry. As a result, instead of the ‘automation will take our jobs’ scenario coming true – the reality might be more like 60% of jobs having about 30% of their roles automated in the next 10 years.

In relation to companies like BT that currently rely on people for customer service, Nicola emphasised that it will not be a battle between bots and agents, but rather a partnership that combines the (very different strengths) of the two. 

IntelligentX Brewing Company is another brand that cited this belief, insisting that its own product – a beer brewed by AI – requires human involvement throughout the entire manufacturing process. Instead of automating out the human elements, people work in conjunction with the AI (in terms of testing, assessing and providing feedback on AI-produced recipes) to create the very best result.

Dealing with data issues

The final panel talk of the day centred around how data and artificial intelligence can fuel personalisation and brand loyalty. But when does AI cross the line from cool to creepy? Moreover, with the GDPR deadline rapidly approaching, will greater regulation impact automated processes such as customer profiling and segmentation?

While this is not as relevant in cases whereby automation doesn’t have a significant or legal impact, it still reflects the dangers of using customer data to such an extent that it feels like a violation of privacy.  

For brands like ASOS, artificial intelligence certainly underpins targeting strategies, with AI processes impacting what products to show which customers and when. However, even ASOS realises that data should be used with caution, agreeing that Target’s recent fail proves some lines should not be crossed. The retail brand sent coupons for baby items to a teenager (and her unsuspecting father), having determined from data tracking that she was pregnant.

While other brands like ShopDirect show that using artificial intelligence in this way can generate results – i.e. to identify and retarget a customer who might have run out of lipstick – it’s clear that there’s a long way to go before basic human judgement becomes redundant. 

Related reading:

Nikki Gilliland

Published 6 July, 2017 by Nikki Gilliland @ Econsultancy

Nikki is a Writer at Econsultancy. You can follow her on Twitter or connect via LinkedIn.

706 more posts from this author

You might be interested in

Comments (1)

Parry Malm

Parry Malm, CEO at Phrasee Ltd.

I would suggest it's not a zero-sum game. Some advanced technologies, including the various types of A.I., can augment what humans do - both in terms of immediacy, scale... AND creativity. But it's not about replacing jobs, even though that's the headline that sells papers. It's actually about using the appropriate technology to make humans more effective.

IMO marketers should spend 95% of their time defining their problem, and 5% on implementing a solution, not the other way round. Gain extreme clarity what your challenges are, and then determine whether or not an A.I. approach is appropriate.

about 1 year ago

Comment
No-profile-pic
Save or Cancel
Daily_pulse_signup_wide

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Digital Pulse newsletter. You will receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.