“The impact agenda can feel like an attack at the level of someone’s very identity. So my plea to researchers is to remember what you love about your job, what you love about being a researcher and perhaps engaging with policy makers, school children and industry might actually enable you to get more curious” Professor Mark Reed on evaluating research impact.
Professor Mark Reed is a Professor and Research Centre Director at Scotland’s Rural College (SRUC) as well as the founder and Chief Executive of Fast Track Impact, an organisation that provides researchers with evidence-based tools and training to empower them to use their work to change the world for good.
In this episode of Research Comms, Mark helps us define ‘impact’ in the context of research, explains the strategies, methods and tools he encourages researchers to employ when assessing the impact of their work, and highlights the importance of empathy when it comes to designing research and maximising its benefits for the good of wider society.
SUBSCRIBE ON
or wherever you listen to your podcasts…
The following excerpt from the interview has been edited for brevity and clarity.
How do you define ‘impact’ in terms of research impact?
I think the biggest misunderstanding out there is the difference between engagement, communication and dissemination on one side, and impact. At what point does engagement become impact?
For me, that point is the point at which there is a benefit for someone. Maybe this is a non-human species, maybe it's future generations. But the point at which there is a benefit is the point at which you have achieved impact. And so it's really important that we look at this on a regular basis, that we're evaluating what we're doing as we're going, because ultimately the work that we're doing could have lots of winners and losers.
Whilst many comms platforms build in metrics around reach, I would argue that reach without any significant benefit is meaningless. At a minimum, I need to know that some of the people who listened to the communications around my research actually understood what I wanted to convey, and ideally as a result of that understanding they can maybe do something that they couldn't before. Maybe there is ‘good’ that comes from this.
And so my book, The Research Impact Handbook describes impact as the ‘good’ that we as researchers do out there in the world. I've used that word ‘good’ to emphasise that this is a value judgement. My ‘good’ might be your ‘bad’ and vice versa.
Have you listened to these other episodes of the Research Comms podcast?
What about assessing impact? Is there a framework or series of questions that you encourage researchers to ask in order to gauge the impact of their work?
In 2021, I published a paper with some colleagues in the journal Research Policy, which sets out a methodological framework for evaluating research impact. Essentially, the task is simple. You are tracing cause and effect relationships, where the cause is your research and the effect is your impact.
And in between, there may be a pathway to impact, a bunch of engagements, communications and the like, but ultimately what that serves to do in an evaluation is to create a golden thread that outlines the attribution between my research and at least some of that impact. Essentially what we have to do then is to create an evidence-based argument that our research either was responsible for that impact and it may be sole direct attribution — unlikely! — or that our research had some significant role to play.
That then becomes quite subjective. It's about us marshalling the evidence and typically one will do that through a social science concept of triangulation: “I'm going to get lots of different bits of evidence, piece them together and everything is pointing towards the same reality”.
This then becomes robust evidence. In terms of the methods you might use, they are wondrous and glorious. What I always encourage my research colleagues to do is to look in their own disciplinary toolkit. Whatever you do, ask yourself, “could you repurpose some of those methods to evaluate impact?”
Once you get those creative juices flowing, people start seeing the win-wins and say, “hey, there's actually interesting research questions in here. I actually want to do this evaluation.”
And that want is really important, because without that you're not going to do the evaluation and as a result, you wait until the end of your research, and at that point it turns out you've just wasted a whole load of money and time, or maybe just created a whole load of negative unintended consequences. We need to do that evaluation as we go and where we can get some win-wins back to our research, then all the better.
How do you tackle the issue of time? The problem is that generating and assessing impact can be very time-consuming and researchers tend not to have a lot of spare time outside of doing their actual research?
It is a challenge and I think it's important for many of us as communications professionals. We're in this because we want to make a difference, and so we assume that researchers are in this because they want to make a difference too. I start all of my training by asking people, “what ultimately inspires and motivates you intrinsically?”
Across the world and across disciplines, “making a difference” is an answer, but it’s not the primary driver of the average researcher. Instead it’s “curiosity, creativity and challenge”. If I can make a difference, that's the cherry on the cake. But that's not the primary driver for many of us.
Even in many applied disciplines, that's not the thing that's going on. And so when people like us come along and say “you should be achieving impact”, there’s the sense that actually you're telling researchers they can't be who they are. Many are curiosity driven by nature, and yet feel like they have to apologise for that. The whole impact agenda can feel like an attack at the level of someone's very identity.
No wonder people can become defensive. My plea to researchers is to remember what you love about your job. What do you love about being a researcher? Ask yourself whether engaging with policymakers, schoolchildren and industry might actually enable you to get more curious, to ask more interesting questions, to be more creative, to give you new challenges and come at it from that perspective. So we're intrinsically motivated and it's not just a tick box exercise.
So great, number one: we're motivated. Researchers think “I'd like to do this, but I don't have the skills, I don't have the confidence”. That's where training can come in and what I do with Fast Track Impact. But researchers will still feel that they want to do this and know how to in theory, but don’t have the time. This is the number one bar across the board.
What I do is I train people in some really effective time saving tools. Essentially this is about thinking way more strategically about what you want to achieve and with whom. So instead of this scattergun approach of “let's just put out a press release and hope for the best”, I'm asking who it is that ultimately might benefit from this research?
Once you have a laser sharp focus on how to add value to someone like that and how you can meet their needs, this is how you can make what little time you have for impact really count.
Often when researchers come to us for help to create a video or animation about their research and we ask them who they want to reach, they list a number of very different audiences, e.g. schoolchildren, older people, patients and politicians. How do you help researchers to identify and hone in on a particular audience?
The key thing here is to start in the right place. I think the temptation is to think “well, we've got a budget so let's make a great video, let's open a Twitter channel, a YouTube channel…we're going to do all of this really great comms”, but it becomes comms without a purpose or audience. You're potentially wasting money creating these incredible things that nobody will engage with.
So the next place that people go to, which is not the starting point, is thinking, ”I need an impact plan. If I know what I want to actually achieve, what is the benefit?” Now I can work out what the pathway is to the best, most efficient and most effective way of achieving those benefits for those people.
You're absolutely going to need that. But I would argue the first place you need to start is to ask who might be interested in this research? Then who of those interested people might have the influence to facilitate your impact, or to block it?
Because let's face it, either way, that's going to have a major effect on whether or not you ultimately achieve impact. And then finally, who is going to be directly impacted by your work?
Instantly, I can see that I've got high interest, highly influential and high-impact organisations, individuals, and teams within those organisations. I need to make sure that I reach out to them, that I get them onside. When I discover that there are people out there who want to see the same change as me, impact then becomes a team sport.
And what little time I've got is now providing the evidence to empower others to achieve impact. And I can achieve way more than I would ever otherwise have managed Importantly, this is what I describe as the three I’s approach: Interest, Influence and Impact. I can also use this to identify groups who are not interested in what I'm doing, at least not in the way that I'm framing it.
And also they are not influential by definition. In fact, they are the poor, the marginalised and the oppressed. And yet they may be more impacted either positively or negatively than anyone else. And I have a moral obligation to engage with them, and they may well be the hard-to-reach. And so now I've got an analysis of this, I can start to prioritise.
For me, this is two, three, maybe four groups that I'm going to reach out to, and I'll get to the others if I have time. But as we've discussed, time is a problem. So let's go with my top three, for whatever reason, and each of those contacts now is empathic contact because I understand something about their needs, their challenges and the constraints.
It’s not just “Here I am, here is my research. Can we talk?” It's “here you are. Here are the challenges I can see you're facing. Here are your strategic objectives. And here's how our research might be able to help. Can we talk?” You get a yes to that. And now I can take a step deeper into their issues and start seeing my research the world through their eyes.
With that perspective, I can co-produce an impact plan that's going to resonate. I may have more impact objectives than I have time or resources to achieve, so I have to prioritise what the most important one is. If I could just do one thing that would really make a difference, what would that be?
Now I can put what little time and energy I have into achieving that; now I can do incredible things with very limited time and resources.
You mentioned empathy there. How important is empathy in the communication of research?
Empathy threads through everything. You'll see this in much of my work, my Research Impact Handbook, and my papers. The evidence comes back again and again in different languages and in different ways to this point.
Empathy is what works if you want to achieve impact. So imagine I've reached out to some key people that I think might benefit from this work. Maybe with their help, I've got an impact plan and only at this point can I now start to think about what are the most appropriate activities.
It may be a 1-to-1 conversation with a policy colleague, it may be a workshop, it may be an infographic accompanying a blog or a video. It may be some work with the mass media. And in every single case, I'm able to ask myself, well, if I want to get this group that benefit, what is the most effective mechanism that is going to work?
If the most effective way of doing that is through the mass media or through social media, for example, then I am putting myself in the shoes of those in the individuals or those groups.
I can now create communications that will really resonate. I co-produced a guide with The Conversation which I called the Media Impact Toolkit, in which we've got a whole load of suggestions for how you can take this empathic approach to doing an analysis of who might ultimately benefit, turning that into an impact plan and then using media to achieve some of that.
Crucially, how do you go about evaluating not just the reach, but the significance of those benefits that have occurred through them?
Communications can be powerful, but I don't just instantly go there. I ask myself what is going to be the most effective and efficient mechanism to achieve that impact for that group. And that's the key thing.
I think that so much of the communications work that we do is not targeted well enough. It’s quite a broad brush scattergun and there's very limited follow-up to find out what happens and how can you build on it.
One of the key barriers to effective communications is a lack of engagement from academic experts and an increasing disconnect because there are professional services teams working in the press office, in other comms roles, and then there are teams and they are very rarely integrated.
It's certainly not in the same department. There's very little communication between them and sometimes they are working at cross-purposes and it's quite common for impact teams to actually dissuades researchers from investing too heavily in media communication in particular because it has a reputation for taking up lots of time and not actually delivering any tangible evidence of impact.
I think that the more that our communications professionals can build relationships with their colleagues and professional services, working on impact with key academics, the more likely we are to get a sense of the pipeline that's coming in terms of the impact plans so we can actually plan ahead of time.
In this way research communications professionals can add value to their research and professional services colleagues because they become part of a strategic plan. This also gives the satisfaction that research comms are genuinely helping academics to have genuine societal impact.
Have you got a tally of how many researchers you've worked with directly ever since starting Fast Track Impact?
It’s something like over 10,000 researchers from more than 200 institutions in at least 55 countries since we spun out in 2015. We’ve got some reach.
What a privilege to get to work with people from every discipline who do incredible things. Recently, I was working with people who are literally risking their lives engaging with policymakers in countries on really important issues where the governments don't like what they're doing.
So it's an incredible privilege and endlessly fascinating meeting all of these people who are doing crazy research on really important issues.
Before I let you go, I just want to ask for a recommendation of one book that you've read that has perhaps changed your perspective about what you do in the research impact field.
Dare to Lead by Brené Brown. For those who don’t know Brené Brown, she is an American Professor and Researcher, whose work deals with ideas around shame, vulnerability and leadership. Brown’s book showed me how sharing my vulnerabilities can help me connect with and inspire people to be more authentic themselves.
Research Comms is presented by Peter Barker, director of Orinoco Communications, a digital communications and content creation agency that specialises in helping to communicate research. Find out how we’ve helped research organisations like yours by taking a look at past projects…
EXPLORE MORE FROM THE ORINOCO COMMS BLOG
