“After a couple of well-attended internal meetings, it became pretty clear to me that we were in danger of making decisions, which on the face of it seemed logical, but were not based on any understanding of what our users wanted. The problem was that with so many different elements to our site and so many creative ideas coming forward, there was a real risk that we’d confuse or disappoint our users…” This was the point at which Gordon Wilson, the head of marketing at BMJ Publishing Group, felt the need to turn to some informed user research that would give the website redesign a clear focus.
BMJ had last redesigned their website in 2006 and since then, functionality and technical capabilities had improved exponentially. The editorial team had a range of ideas – they had started continuous publication in 2008 – and this was naturally leading towards a host of real-time opportunities.
The project team decided that a user segmentation study was required. It is a key tool of the site re-designer’s trade; often it takes the form of a series of pen portraits or personae, derived from customer interviews. For BMJ, with such a huge audience, a quantitative study was required so that the relative size of each segment could be assessed.
What’s driving the segmentation?
The first task on BMJ.com was to understand what was going to drive the segmentation. The target variable, as it is known, is the single user behaviour that the new website will encourage. It provides focus not only for the segmentation process and the questionnaire construction, but also the strategic rationale underpinning the redesign.
This discussion raised profound issues: what was the commercial imperative that lay behind this redesign? This was not a straightforward nettle to grasp for the business end of a not-for-profit organisation. Yet without answering it, we would be in the dark as to what the key objective was:
* Was it to increase the number of users coming to the site – so BMJ.com could become an increasingly competitive advertising opportunity?
* Or was it to increase the number of people who were willing to pay for a subscription?
* Or was it to increase the frequency of visit?
* Or increase the amount of dwell-time amongst key audiences?
Each one of these target variables would result in very different segmentation outcomes.
Wilson explains: “The discussions this prompted were really fundamental. Inevitably, meetings kicked off with a small cadre of people involved in the redesign project. It quickly became clear that a lot more engagement with other parts of the business was a priority. We found that having an objective third-party to moderate these meetings was really helpful. Without Stingray’s involvement, there was a risk that discussions would perhaps have polarised into different camps. Once we had properly engaged with this question, we came to a foundational decision that it was all about frequency of visit from medically-qualified doctors.”
Making the questionnaire work hard
A common mistake in this kind of study is to assume too much from the analytics. Segmentation involving regression analysis and other techniques can often come across to the occasional user as a big ‘black box of tricks’ – you simply push the data in at one end and out of the other end you see perfectly formed personae with highly defined and distinct needs. But segmentation studies, like any other form of research, need clarity of thought and a certain independent discipline in terms of what questions to keep and what to leave out. An independent researcher is also free of any internal Group Think when it comes to challenging unspoken assumptions.
One of the biggest issues of any segmentation exercise is the length of the questionnaire. It needs a disciplined approach. If the questionnaire is too long, few will have the stamina to complete it; too short and there is little data to work with. This was particularly important because we knew that only a fraction of users had registered with BMJ.com. That meant that the survey would pop up on the website.
Some dos and don’ts of segmentation questionnaire design guided the survey development:
* DO include questions that allow other answers to be placed into context. Think carefully about question combinations (2+2=5).
* DO keep the respondent base the same by limiting the number of routed questions used. Any questions that involve routing can’t get included in the segmentation dataset.
* DON’T use ‘yes’ and ‘no’ questions for key usage or behavioural questions; they’re not nuanced enough for decent segmentation.
* DO set a completion time limit; scaled questions take longer to answer, so ensure high completion rates by watching the clock.
* DON’T forget to ensure that all your identified areas of data need are included; such as use of competitor sites (not just the traditional competitive set, but including those from non-publishing background), site usage and attitudes towards the site and its functionality.
Our answer to the issue of length was to split the survey in two.
1. A short three minute survey which attracted over 28,000 respondents.
2. An invitation to continue in order to complete the longer user survey (which nearly 7,000 users did).
These 28,000 respondents answered key usage and demographic questions only. And they provided the base for weighting the 7,000 segmentation survey respondents. That way we knew we had a robust assessment of every type of user and in the right proportions.
Meet the Specialist Dipper
The Specialist Dipper was one of four medically-qualified segments. The fictitious persona for this group was Dr Vernon Hampton Jnr. He is a 46 year old Cardiothoracic Surgeon and has a practice in a mid-west US hospital and an affiliation with an academic institution.
He mainly refers to journals relevant to his specialty – either in print or online. Trying to pin him down to how often he visits BMJ.com is difficult. “I must visit dozens of websites every day. Most of the time, I’ll be on PubMed or Medscape. I don’t type www.whatever.com – if I visit BMJ.com, it’s because I’ve clicked on a link from a search result or a hyperlink in a paper I’m reading.”
The last time he visited BMJ.com was because he’d clicked on a link to the recent research paper on Myocardial infarction. He hadn’t bothered looking at anything else because that wasn’t the reason for his visit. He guesses that he’ll visit BMJ.com five or six times a year – much in line with the other more general medical journals. The heavy usage is with specialist journals and websites.
His attitude is straightforward enough. BMJ is kosher and clearly authoritative. But for him, that’s a basic essential. He wouldn’t consider visiting any site that wasn’t. The problem is he hasn’t got the time (or the inclination) to browse around what he considers to be general medicine sites: “Nothing personal folks, but time is money!”
Dr Hampton proved to be a pivotal part of the website development. Being a consultant physician, BMJ.com should be an important part of his information diet. But it wasn’t. We knew Specialist Dippers accounted for over 10% of the medically-qualified user base and they were key influencers.
I was blind, but now I see…
Gordon Wilson takes up the story: “We knew the Specialist Dipper was an important audience for us, but an infuriatingly difficult person to please. It was only when we brainstormed the options that we realised the answer was right in front of our eyes. Right through this website design, we had been in the mindset of thinking in terms of the breadth of BMJ.com. What we needed to do was allow the site to expose the relevant content to different groups of users, either from BMJ.com or our other content sets. This research challenged our product mentality and opened our eyes up in terms of how we could leverage our huge audience reach far more effectively.”
And that was just the start of the journey:
* It prompted a comprehensive re-thinking of BMJ’s total online real estate – particularly crucial given the importance of e-learning to a number of other segments.
* Timeliness of content was not the key factor driving frequency of visit. This re-focused editorial thinking in terms of the way it saw research and education as part of its whole content remit.
* It offered a number of opportunities, often international in scope. Different regions suggested variance in the characteristics of users, eg. some more likely to engage for learning purposes whilst in other areas the need was more academically related.
* BMJ on iPad was launched with All-Round Browsers in mind. This large segment wanted regular weekly updates in an attractive format that allowed them to quickly scan all the latest updates they were interested in.
* A large programme of customising email alerts to encourage registration was embarked upon (registration for email alerts has been a real focus and has grown significantly since the re-launch).
* Given that a huge number of visitors came to BMJ.com via Google, a BMJ widget was introduced on Google.net.
But for Gordon Wilson, the main seismic shift was cultural: “It switched us from a product push mentality (this is what we do) to a needs-based mentality. This project woke us all up to the real potential we have to respond to the needs of the wide variety of users.”
