I worked for fours years at TVE programme Redes. It was an excellent programme and Punset was a great science communicator who managed to do what very few could: get a wide audience excited about science, convince them that science was fantastic. For those who criticised it, it was confusing and the audience learned nothing. Who was right? What was the educational value of Redes? Can we debate over that with data, instead of opinions? No, because after fifteen years on the air, nobody had considered it interesting to evaluate the impact of the programme on the viewer. It seems absurd now; a missed opportunity. We still have no data to compare the science communication value of Redes versus that of Órbita Laika. Any businessperson would laugh at us and call us amateurs.
Five days before I wrote these lines, I was having lunch with the Viceprincipal and Academic Head of the Autonomous University of Mexico State, where I was participating in a conference, when they said: «Here at the university we have several science communication initiatives, like a podcast, YouTube interviews, a magazine, talks…» I asked: «Have you tested them for impact in order to see which may be reinforced or reformed?» There was dead silence, which I imagine would happen with a lot of universities and financial backers of popularisation projects.
You might say: «Well, we have a slight idea about what works and what doesn’t.» True. But just before I flew to Mexico, I took part in a populariser meeting in Chile, where the sociologist Eric Jensen of the University of Warwick (United Kingdom), whose work is to assess the educational potential of specific popularisation activities, was delivering a lecture. His main works revolve around zoos, and Jensen has confirmed that, with some activities, children have a very good time, laugh and enjoy with the animals and the stories of the wilful guides, but the do not learn jack, they do not change their perception of the importance of the preservation of species. If the objective is not only to entertain, but to educate as well, those «successful» activities were coming short of their role.
Because, whether we like it or not, the great challenge of scientific communication is to evaluate the impact of projects in order to distinguish the useful ones from the irrelevant ones. A drag? Yes, it is a drag. Extra cost? True as well. But if we truly believe that our mission is so important that it is above us, we must sacrifice some «good vibes» and start recognising that we are not very scientific in our job, and that there are awful popularisation activities whose impact can even be negative. Sure, negative. There are such mediocre exhibitions and boring conferences that whoever invests two hours of their time to attend will be disappointed, convinced that science is, indeed, a bore and unmotivated to attend a new event.
Those who engage in popularisation efforts as a hobby are exempt of these comments, of course. But the institutions or individuals like us, who intend to be professionals of science communication, have no choice but to start thinking about the impact evaluation of our activities. And the financial backers must make sure that they can be evaluated. There are methods to do so.
I sometimes work as a consultant for the Interamerican Bank for Development (BID). Institutions such as the BID or the World Bank invest large amounts of money in development aid projects. And impact assessment has become one of their mantras in recent years. Why? Because they realised that a large part of the thousands of millions they invested were not useful in reducing poverty or developing the receiving countries. Some projects worked on paper, but did not in practice. I think there are some parallels with science communication, with many initiatives that, historically, were not able to attack scientific illiteracy in the population.
How can we know if a project is more efficient than other? How could we recommend one strategy over another for a university? Similarly to how you check if cholesterol is reduced more with one diet or another. Evaluating their impact. Last December I organised a scientific journalism hackaton with the BID in Guadalajara (Mexico). We all left it convinced of its success, but the project evaluation revealed some things had failed, from the project presentation methodology to some speakers and the meal schedule. The second edition will be much better.
Hotels assess the satisfaction of their clients; brands assess their products; schools, their student’s performance; and we, who call ourselves science communicators, are actually performing unscientific communication. In part, due to the harmful amateurism surrounding this job, the misunderstanding that the scientist must be a populariser (collaborating is enough), and maybe due to a fear of what will be.
I do not want to quote examples of doubtful impact of popularisation activities, but I have some in mind, and, probably, so do you. Maybe there is no need for more, but for better scientific communication, and promoting the professional scientific communicator more than the amateur one. Again, I don’t mean those who are excited to devote their free time to popularisation. This is not for them, please keep doing what you want, as I do. But in a different level, and leaving aside strategic provocations, the message of this text is that we must seriously consider a rigorous evaluation of the impact of communication activities. We might find out some surprising things.
Pere Estupinyà. Writer and science communicator (New York).
«Maybe there is no need for more, but for better scientific communication, and promoting the professional scientific communicator more than the amateur one»
Illustration: Moisés Mahiques
«The institutions or individuals like us, who intend to be professionals of science communication, have no choice but to start thinking about the impact evaluation of our activities»
«Whether we like it or not, the great challenge of scientific communication is to evaluate the impact to distinguish useful projects from the irrelevant ones»