What did Nursing Times do?
In 2024, Nursing Times set out to prove AI could be used to vastly improve customer experience.
Nursing Times was an AI pioneer. We believe we were the first B2B brand in the UK to offer a large language model (LLM) search service, launching Ask Nursing Times in February 2024.
We harnessed the power of AI to give instant answers to users’ questions, exclusively from the Nursing Times’s trusted expert content. We inserted AI generated related questions in articles to encourage users to explore topics further. We allowed customers to ask their own questions.
We took inspiration from the wise words of Steve Jobs from as far back as 1997, “Got to start with the customer experience and work backwards to the technology. What incredible benefits can we give to the customer, where can we take the customer. Not sitting down with the engineers and figuring out what awesome technology we have.”
Our users were able to discover content so much better. Large language model search works incredibly well. Prior to AI, most onsite search has been poor.
Ask Nursing Times does the hard work of finding exactly what the customer wants, from the vast archive of complex Nursing Times clinical and news content.
The content is presented in a far more engaging and useful way than traditional search. The user gets a summary answer to their question, referencing and linking to each article that was used to generate the answer. They are offered related questions to explore and go beyond and learn more about the topic articles.
Where was Ask Nursing Times able to take the customer? To a vast archive of everything they would ever need to know about whatever they are interested in, past and present. AI has allowed us to offer deep and rich information beyond previous surface information, which was presented on home and section pages, in newsletters or traditional Google single article search links.
What did we hope to achieve?
There are many AI battlegrounds emerging. We are concentrating on:
- How much of the value generated by good content will go to the publishers vs the AI developers?
- Where will audiences spend more time, on AI platforms or publishers’?
- As AI mediates content discovery, how can publishers ensure their brand remains visible and trusted?
To tackle those fundamental questions, we launched Ask Nursing Times to enhance our customer experience, considering how to apply cutting-edge AI technology to improve it.
We wanted to explore how AI could be used to make our websites destination sites.
If better customer experience can help us build a more direct and deeper relationship with our audience, we can attract them to spend more time on our sites than the AI platforms, our brands can remain visible and trusted and we can derive the content value from growing recurring subscriptions revenues.
As this had not been done on a UK B2B site before, we had no way to set metrics or goals but we did have ambitions and hopes.
We hoped we would see sustained use of the Ask Nursing Times search service, rather than an early curiosity peak that quickly faded away.
Given the excellent user experience, we hoped we would finally crack how to get users to read more than the single article they arrive at the site on. Would user stickiness and engagement improve with readers viewing not just the summary answer but the articles the summary answer was based on and articles in the go beyond, and learn more about this topic section?
Would the service quickly gain the trust of users by providing helpful, accurate answers with no hallucinations?
Would the service improve conversion rates? Could the tiered access model that we developed convert anonymous users to registered users and registered users to subscribers? Anonymous users need to register to ask their own questions. Registered users need to subscribe when they have asked five questions.
Would the service improve renewal rates? Would being able to find content more easily and content being displayed in a better way deliver a meaningful increase in subscriber engagement? More engaged subscribers have a higher renewal rate.
What have we achieved?
The results have exceeded all expectations, resulting in the ‘Ask’ service being added to six other emap subscription sites.
Ask Nursing Times has answered over 200,000 customer questions since launch in February 2024.
There was no curiosity curve. The initial interest has been maintained. Monthly usage is remarkably consistent. The range for the last six months has been 10 to 12,000 answers.
The articles the summary answer reply was based on have an exceptional 6.44% click thru rate. No other technology we have ever used has been able to achieve anywhere near that click thru rate.
The service is accurate, trusted and helpful. A tiny 0.12% of users have marked answers as not helpful. The editorial team has not seen any hallucinations. We have had no hallucinations reported to us by users. Providing answers only from the trusted and expert Nursing Times content has provided the reassurance users needed to rapidly adopt AI search on the site.
The service does directly convert registrants and subscribers. Links on the Ask Nursing Times promotional units and answers have delivered 175 trial registrations and 94 subscriptions. Given how hard subscriptions are to acquire and their lifetime value, the service has proven its return on investment quickly.
What have we learned?
The ‘Ask’ service works best on sites with a deep archive of long form evergreen content. Where users want a single short news update, it has less value.
We were right to do as much testing as we did to ensure the highest level of answer accuracy and user satisfaction. That said, editorial teams are much more worried about the accuracy of AI answers than customers. We have had no complaints from over 200,000 answers and seen no hallucinations. The opportunity to improve how customers can discover content is much greater than the risk.
Habits last a long time; 95% of answers come from users clicking auto-generated question links inserted in articles. Users are yet to take full advantage of AI to do their own content discovery. With hindsight, I wish we had made it clearer to users the greatest value of the service is in asking your own questions. We will add an anchored search chatbot to all pages.
The same is true for audience analytics, habits die hard. We are still relying more than I would like to on standard article tracking to understand our audience behaviour. We should make more use of the powerful ‘Ask’ analytics. Standard analytics tells us how our audience responded to content we assumed they wanted to know. ‘Ask’ analytics tells us what our customers want to know. That is far more valuable for shaping content strategy.
Marketing campaigns drive some use, but the key is to present the service where the user is. Customers get ever better at avoiding marketing emails and social campaigns. It is essential to fully integrate ‘Ask’ in articles, with all experience being in the Nursing Times brand. Be confident in the service, make it visible. Auto-generated explore questions are embedded after the sixth paragraph of every article.
The most important marketing message has been, this is an AI service you can trust as it only generates answers from a single source of content you have always trusted — Nursing Times. Showing the usage of the service has encouraged the majority of users to follow the early adopters. Every time we have crossed a milestone — 100,000, 150,000 and 200,000 answers — we have launched a big marketing campaign.
It is much quicker to buy than build. Using an expert US AI provider for the ‘Ask’ service, we were able to deploy AI powered search on seven emap websites in less than a year.
The AI technology might be new but old things hold true — better content, better content experiences and new products deliver revenue growth and higher customer engagement.
What comes next?
The biggest potential for a positive AI impact could be offering customers a hyper-personalised content experience.
We can offer each customer exactly the content they want, rather than presenting content that we think has the broadest appeal, assuming our audience have much in common. The reality is they are highly diverse.
We are working on AI powered on page personalisation — recommended for you and trending. AI powered customer conversions — presenting at article level related other things you can do; eg. if you are reading an article on digitalisation, sign up to the webinar on digitalisation. And using AI to insert a personalised article in every user’s newsletter every day. We will build a personally customisable myNursing Times page, allowing users to select the topics they want to see articles on and what has been updated since they last visited.
AI can be used to personalise a content experience without an ‘Ask / Answers’ service, but it is much more powerful with the individual customer insights the ‘Ask’ technology captures.
And then, where can we take the ‘Ask’ service? Can we make it a full research AI end-to-end service? By that, I mean chat history with updates and email updates, every time there is a new answer to a user’s question. Your own Nursing Times agent keeping you constantly informed about the topics and issues that matter to you.
If we can, the future is bright for business-to-business publishing. We will have a more valuable direct relationship with our customers.
This article was first published in InPublishing magazine. If you would like to be added to the free mailing list to receive the magazine, please register here.
