Data teams: ensuring they deliver value

Data-literacy is embedded across The Independent. As the UK’s first digital-only newspaper, the provision and usage of data has been instrumental in informing our development and success, with high adoption rates of editorial tools, reports and dashboards key to this.

The Independent is one of the leading global news websites with journalists based on three continents. The Independent has an average total monthly audience of 61.7m unique visitors with 59% of these outside of the UK. A strategic focus on US audience growth has been very successful and this region now represents 33% of traffic.

The Independent operates with a central data team that supports all aspects of the business from the senior leadership team downwards. The data team has grown considerably in terms of numbers as well as functionality over the last six years. Initially, the team concentrated on supporting the newsroom with analysis of audience, article and topic performance. This remains a key function of the team. However, the remit of the team has expanded over the years to incorporate analysis for The Independent’s reader revenues, licensing and syndication and eCommerce divisions, as well as building out a data engineering and modelling capability.

The Independent’s data function is built around a set of guiding principles or strategic framework which underpins how we operate and deliver to stakeholders day-to-day:

• Value to the business. With the vast volumes of data available and a lean team, it is important to align projects firmly to business objectives. Whatever we do, it must deliver value, and we are constantly looking for efficiencies in our processes and operations. Depending on the type of analysis or project delivered by the team, we look to calculate what financial return this has delivered for the business.
• Rationalisation and improvement of the technology stack. We regularly review the platforms we use to ensure they are fit for purpose, cost-efficient and continue to offer the appropriate functionality for the business. The team typically undertakes one major migration project per annum to ensure the software utilised remains best-in-class and outputs enable our business strategy.
• Timing & Automation. The aim is to get data to users at the right time and in the right format. In an always-on news organisation, there is a tendency for stakeholders to want immediate access to information in real-time. However, this is a balancing act around how the data is going to be used, the cost of providing this service and whether it is feasible. For the newsroom, in many scenarios, for example around the performance of a live blog or breaking news story, then speed is of the essence. The process of continued automation is key particularly around operations and reporting. The benefits are around freeing up resource as well as data delivery efficiencies and being focused on more value-added activity for the analysts.
• Aligning with business goals. Over the past eight years, the business has successfully diversified its revenue streams, and we have almost reached our goal of non-advertising sources making up more than half of revenue. This influences the way we structure our data warehouse and data lake as well as the associated processes. Key initiatives include the ‘Anonymous to Known’ strategy which focuses on encouraging users to register or subscribe. Therefore, the datasets around a reader are very different to that supporting the advertising technology stack. The eco-system, therefore, must be adaptable to cope with diverse datasets formats and use cases.
• Intellectual Property (IP). Where feasible, The Independent develops its own capacity in-house thus creating its own IP. Where we have the skill and capability, then this is the preferred option. The strategic framework has been underpinned by a move to the cloud. The Independent started transitioning across to the Google Cloud Platform (GCP) over five years ago. This allows us to leverage the Google technology stack for data storage, processing as well as analysis. The main benefit is that this is a flexible, scalable solution. We can spin up services as and when we need them and only pay for what we consume. This means our usage and costs are in step with the growth of the business. The adoption of GCP enables us to future proof our provision of data as much as possible. The role of the data architects and the engineers is crucial in this and how they design and construct our systems to be as flexible as possible.

Challenges

• Data Volumes. The volume of data available is constantly increasing with new sources being added regularly. Managing all of this as well as multiple frequencies and formats will continue to be a major challenge. The key to this is how to distil this vast amount of information into useful sets of data. The increasing use of AI in data management may well transform this area.
• Talent. Ultimately, the success of any data analytics team is the people in it. Given the current market conditions, there is a huge battle to obtain and retain data analysts, scientists and engineers. A huge amount of focus is placed on the amount of data, the software or the tools a data analytics team use and even more so with the noise around AI. But it still comes down to having a talented group who can interpret the data, glean insights and communicate clearly with the business. Without them, the rest counts for little.
• Legislation – data privacy and compliance. GDPR now seems a long time ago, but it was a landmark in terms of how it changed the data privacy landscape. The regulatory framework is now far more complicated, and the privacy commissioners have more power to investigate and fine any breaches of the rules. As an international business, we must ensure we are complying not only with UK legislation but also the EU. The US is quickly catching up in this area but with the added complication that it’s doing so on a state-by-state basis. Individuals are also more aware of their rights in respect of what data is held against them and how it is being used. This area will increasingly add to the complexity of what and how data is processed and what permissions are stored against a user. Therefore, being in constant contact and working with your data privacy team is vital to navigate your way through all the legislation a data team must contend with.
• Identity management. The ability to stitch disparate datasets together to create a comprehensive view of either content or a user is vital and underpins much of the work of the team. The aim is to get towards a single view of the customer and to be in a similar position for any content produced. Given the plethora of sources and the variety of identifiers within them, this is an on-going process to ensure the right keys are utilised giving us the highest degree of accurate matches. This is becoming increasingly challenging as readers have the right to block tracking as well as using multiple devices to access content. This has been mitigated by the value exchange between the publisher and the user and what data they are prepared to volunteer which allows us to create and maintain a profile of them.
• Technology evolution. Given the speed of change in machine learning and AI, these technologies and their application will have a profound impact on the processes and output for data teams. Where do these applications add the most value? How best to integrate them within our technology eco-system and what’s the best way to deploy them?

Demands continue to grow

The demands for the provision of reporting, analysis and insight continues to grow. Typically, editorial teams are time-poor. So even if the data team has built out an amazing set of dashboards, they may not be able to “self-serve”. Some of the members of the newsroom are incredibly data fluent but the majority will need assistance from the data analysts on how best to interpret the information available to them. We find that what works best is to provide the newsroom with a select number of key takeaways that they can action.

Another requirement is the speed of the response. Publishing, and news in particular, are fast paced. The ability to have data available for analysts to respond to a query quickly is paramount. There is always a balance to be struck between the quick turnaround requests of a more tactical nature vs the deep dives into a topic for in-depth analysis.

Data and analytics are integral, having helped The Independent be the only national title to successfully move from print to digital and have been crucial to the revenue diversification of the business. The Independent has invested heavily in the size of the team and the associated software. This is backed by a clear data strategy enabling the data team to grow and flex with the business. Tools, software and processes are all important but without a talented team they will count for little. The ability to build connections and trust between the data analysts and the rest of the business is crucial. There is no quick fix but get it right and data and analytics will be at the heart of the business. By ensuring the analysis provided is of high quality, timely and insightful you will build up that trust. Ultimately this is what delivers a successful data strategy.