From big data to customer insight: how do we get thick?

4 min readApr 6, 2024

We can use big data to carry out behavioural insights at scale, but ethnographic research can help understand why people opted for one option (fairness and reciprocity on organ donation message after paying tax disc).

Big data helps identify what people do. Ethnographic research helps understand why they do what they do.

1. Combine big data and thick data to understand behaviours

Why big data needs thick data, Tricia Wang

You could influence behaviours but not in the way you intended. Let’s say you created an algorithm that connected subject areas of books in your library catalogue with services or activities you provide. Someone borrowing a book on social work might well be interested in social work vacancies you want to fill. But would someone borrowing a book on how to cope with depression be comfortable with the library sending them details on activities their council or community groups provide on how to cope with depression?

Real time data helps organisations find out what’s on the ground, to identify patterns about demand in order to prepare to meet it — be it their own data, such as sensors or customer relationship systems, or others, such as carrying out sentiment analysis.

Tricia Wang coined the term “thick data” to distinguish the data gathered through ethnographic research with that of “big data” gathered from large scale databases or sensors.

It’s a question I’ve been thinking about ever since I’ve worked with customer segmentation gathered from aggregating quantitative data.

If you use customer segments as a way of selecting what type of people you carry out ethnographic research with — “deep diving” — how do you then feed in the insights from the latter to enrich the former?

If we’re taking advantage of the progress in technology to be able to harvest and carry out trials on large scale datasets, shouldn’t we also be using technology to carry out ethnographic research? I don’t mean observing how people use technology while you’re sat next to them, but creating spaces for people to debate issues and develop policies in a way that you can harvest the collective intelligence?

2. How can we support citizens to become scientists and sense makers?

If we layer this thinking onto how we co-design with people, what would a cooperative approach look like to working with and supporting user or community-driven intelligence, and co-design intelligence?

That’s why I’m intrigued about citizen science. Amidst all the concerns about big data being used for spy on people, citizens could take the tools — literally — into their own hands to uncover what’s going on in the environment around them, hold the powers that be to account as well as develop data-driven solutions.

On my very own doorstep, Loop Labs is working with children to get them excited about citizen science, while Mapping Futures are working with residents to map the invisible data on public services and spaces to help plan their neighbourhoods. Like with any collaboration between authorities and citizens, its important to define together how much autonomy we each want, in some cases people may be happy to co-design with a council, while in others they want the necessary independence to be able to use the data to hold them to account. We don’t want to be recruiting informers or super grasses!

Local authorities and health services have worked with peer researchers to develop their research work, I’d be interested to hear from public services who have supported citizen scientists and to explore with citizen scientists what would motivate you to work with local services to uncover issues and needs?

And let’s not forget people who are work on the front line of services. If you think about it, a highways engineer is creating and responding to big data, a sheltered housing warden is carrying out ethnographic research and a contact centre advisor often does both. We need to understand the journey and supply chain of the data we create and use, to identify the most effective intervention points.

Of course, we have universities and other organisations who use these different methods, some of which I highlighted above. They can dedicate more time, people and investment to applying those methods, but what research and public service organisations have in common is the need to demonstrate impact. So how about university-council collaboration on applying these methods to predict and manage demand and influence behaviours?

Beyond using these new methods to better commission and deliver services, we should be exploring how they can help us move from providing services to supporting infrastructure. What I mean by supporting infrastructure is where people can use and repurpose the different components we provide or invest in that make up a collective infrastructure.




Head of Policy Design, Scrutiny & Partnerships @newhamlondon #localgov Co-founder of #systemschange & #servicedesign progs. inspired by @cescaalbanese