Automatgenererad bild.

Photo: Anette Andersson


28 June 2013

Effects of 20 years’ research evaluated

Mistra’s research programmes have been highly influential in environmental issues and efforts to achieve sustainable development. This is shown by a new evaluation of the results of 20 years’ research. There have been positive effects in terms of such areas as combating climate change, forestry, the marine environment and eutrophication.

Since 1994, Mistra has invested in research that can contribute solutions to many different environmental problems. On Mistra’s behalf, Sweco Eurofutures has evaluated 33 programmes costing a total of more than SEK 2 billion. Altogether, these programmes involved between 800 and 900 researchers (equivalent to the number working at one of Sweden’s smaller universities) and numerous users from the business and public sectors.

‘These programmes have been unique in their comprehensive approach and the fact that they have focused on decision-makers, companies and other users. Their purpose has been not just to develop new solutions but also to maximise their practical benefits,’ says Sigrid Hedin, who headed Sweco’s evaluation with Annelie Helmersdotter Eriksson.

The purpose has been to examine not the actual research but whether the programmes and initiatives have had the expected results and how they have affected society. In early June, the evaluation was presented in the form of two reports. These show, for example, that the programmes have given rise to new solutions in climate negotiations, resource-saving industrial processes, better management of mining waste, microorganisms that protect agricultural crops, and development of fuel cells.

In the past year, Sigrid Hedin and Annelie Helmersdotter Eriksson have studied documentation including annual reports, minutes of board meetings and previous evaluations of all the programmes. They have also compiled more than 350 questionnaire responses from people associated with the programmes, and interviewed board members, programme participants and former Mistra employees.

It sounds like both a huge and a difficult task, to say the least. What was it like working on the evaluation?
‘Well, yes, we read a huge amount… But it’s been fascinating to follow how the various programmes have developed. All the documents have formed a sort of lifeline for the initiatives and pinpointed the processes that have gone on,’ says Sigrid Hedin.

‘Measuring effects of different initiatives is the most rewarding of the various tasks we do but, at the same time, the hardest. One particular challenge here was that the programmes were impossible to cluster together in groups because of the marked differences among them in, for instance, focus, size, working methods and the number of partners involved,’ says Annelie Helmersdotter Eriksson.

Hedin adds: ‘Then there’s the difficulty that many years have often passed since the programmes were implemented. Nonetheless, many of the interviewees were crystal-clear in their responses. They’ve seen themselves as pioneers in this new form of research programme, and have reflected a great deal about what they did.’

Mistra’s initiatives are intended to build strong research environments, provide user benefits, solve key environmental problems and boost Swedish competitiveness. Has all this been achieved?
‘To a large extent, the programmes have delivered what they were intended to in the form of research results and planned activities. Strong research has been established and yielded a lot of new knowledge. Some ten research centres or institutes have also been created, such as the Rossby Centre in climate modelling and the Baltic Nest Institute in decision-making tools for environmental management of the Baltic,’ says Helmersdotter Eriksson.

‘If the participants had worked closely with one another and not been too spread out, it looks as if it would have been easier for the joint projects to continue after the programme period. That closeness would also, it seems, have facilitated efforts to bring all the results together in the final programme phase,’ Hedin points out.

Helmersdotter Eriksson continues: ‘The timing proved important for the results then to be useful. Of course, managing to predict society’s development during a long-term programme and paving the way for research are difficult. But several programmes, including the climate-related ones SWECLIM and CLIPORE, have succeeded extremely well and, for example, been able to support ongoing political processes.

‘One effect that’s unrelated to the results as such,’ she adds, ‘is that many participants feel they have gained a broader view of how different stakeholders work, and a new understanding of their own role as researchers in changing society. The participants have then been able to apply what they’ve learnt to new research collaborations.’

‘In terms of competitiveness, researchers from the programmes closely involved in industry, above all, have become employed at various companies, and they’re contributing new knowledge and approaches three. Now when I’ve been out on other assignments I’ve often reflected on how many people we’ve met who, in various ways, have some connection with Mistra. It’s a sign that both results and people have spread in society,’ says Sigrid Hedin.

You emphasise good leadership as a key factor in success. Why is that?
‘Finding the right leaders for a programme has been crucial for generating good collaboration among different research fields and between academia and users. This work can be tremendously creative but it can also lead to conflicts that drain people’s energy. We’ve seen this in some programmes,’ says Helmersdotter Eriksson.

‘A good programme director has to have an awful lot of characteristics. The person has to be a good administrator and at the same time academically proficient to enjoy other researchers’ confidence. It’s also important to listen to others and gain the confidence of companies, public agencies and other users as well.

‘The system of having a programme board with a wide-ranging composition is also emphasised by many people as a success factor. Above all, the board on which planned users have been represented has been important ahead of a second programme phase, when priorities often have to be selected from various research areas and activities,’ says Hedin.

Another task was to identify success factors in the initiatives. Are there any lessons for Mistra in its future work?
‘Mistra has been extremely keen to know about our results, and obviously that kind of interest is pleasing. At the same time, we’ve evaluated completed programmes, often going back many years. So in a sense we’re preaching to the converted: a lot of the lessons learnt over the years have already been integrated into Mistra’s ways of working,’ says Helmersdotter Eriksson.

Hedin adds: ‘One such lesson is that it’s difficult to generate collaboration across subject boundaries by combining different programme proposals in a joint interdisciplinary venture. That has often led to intractable conflicts that have affected the programmes.

‘But we’ve indicated a few interesting routes that Mistra can consider exploring in the future. Greater cofunding with other stakeholders to enhance the impact of their initiatives is one way forward. Others are shared programme leadership and early action to secure patents and other intellectual property rights to minimise conflicts, especially in programmes in which industry is involved.’

Text: Andreas Nilsson, Vetenskapsjournalisterna


Automatgenererad bild, ersätt med riktig bild eller ta bort.


Mistra Webbredaktör