Section 2: Co-designing research agenda

Key Insights

 The Delphi Approach

“The Delphi method is defined as a systematic, iterative process to elicit a consensus view from a panel of experts.” (Perveen, Kamruzzaman, & Yigitcanlar, 2019, p.600)

The Delphi approach brings together large number of people with a range of expertise, experience in a series of rounds, usually the rounds will involve questionnaires, but new Delphi approaches also tend to integrate in person and online discussions (Glass et al., 2022). A helpful introduction to the Delphi approach can be found in Niederberger and Spranger (2020). The FEDORA Project is an interesting and multi-phase example of a Delphi process that the University of Oxford is involved in. This project seeks to align Science Education with societal changes, research and innovation: the project comprises seven different work packages in collaboration with students, policy makers, teachers, international media, museums and multiple university partners (Ioannidou & Erduran, 2022).

Priority Setting Partnerships

The James Lind Alliance Priority Setting Partnerships is an established approach that brings together clinicians, patients, their families, carers, and other relevant stakeholders to identify the key research priorities for any given topic (Staley et al., 2020). Other disciplines beyond health are beginning to engage in these priority-setting partnerships: an example of a priority setting partnership that the University of Oxford is involved in has been launched by three researchers from the Department of Education to identify the top 10 priorities on English as an additional language research. This project brought together educators, parents, and young people who are invested in the education of pupils who use English as an additional language (EAL). They developed videos to communicate the research process (see for example their short video explainer). They also developed short videos as instructions for each stage of the research process and explaining how research questions were identified and developed (see Figures 1 and 2 below).

(Figures 1 and 2: stills taken from EAL Priority Setting Partnership video)

Horizon scanning

Horizon scanning is an approach that involves systematically collecting a broad range of data to explore, examine and analyse possible trends, challenges, opportunities, stakeholders and expectations that may become relevant to a particular phenomenon over time (Fűzi, Géring, & Szendrei-Pál, 2022). The approach is commonly used in Conservation (e.g., Sutherland, et al., 2022) and Healthcare (Dullabh et al., 2022; World Health Organization. (2022). A helpful introduction to the approach can be found in Géring, Király, and Tamássy (2021) or in this toolkit developed by Delaney (2014).

A highly collaborative example of Horizon Scanning is the Arctic Horizon Scan 2022. In the first phase of the project, researchers reached out to people from local and Indigenous Arctic communities, policymakers, researchers of diverse disciplines, funders of research, and invited all other stakeholders, to contribute towards the Horizon Scan by participating in an online survey. The survey was translated into Norwegian, Danish, Russian and Icelandic and asked respondents to name one or more priorities for Arctic research in the next decade. In the second phase, they held a hybrid workshop to distil and organise the questionnaire responses into a categorised set of Arctic research priorities for the next decade. To read more about this project see https://www.polar.ox.ac.uk/horizon/ (Figure 3).

 Figure 3: Screenshot of the Arctic Horizon Scan 2022 website

Developing engagement in the research process

It can be helpful to develop blogs, podcasts, and short videos to extend the reach of the research, and engage as many people as possible in the process of setting the research agenda and developing research questions.

Researcher Insight:
‘We wanted something which would be easy to circulate on social media...so the short videos [30sec] were designed for Twitter and Instagram, so that if people were on public transport scrolling through, it's all there in writing, so that you don't have to be listening to anything. So that was there because we felt that using Twitter and Instagram would be a good way of recruiting. We thought that it would reach a lot of people and allow for organic snowballing, people forwarding it, and retweeting it and so on. Then the slightly longer ones, the minute-long ones, were designed with Facebook in mind, where somebody might be sitting down at a computer, and have a bit more time to engage… So they were designed with the platform in mind, and with the sort of behaviours that we assumed accompany those platforms….And then the longer ones were there as a kind of belt and braces, if you want to know more, you can go and have a look at the longer videos now. We also wanted the longer ones as an explanation about the project, and an explanation about how to take the surveys’. (Researcher 1)

Staley and colleagues (2020) analysed different Priority Setting Partnerships (PSP’s), and highlight the key strategies that were useful in finding ways to encourage funders and other stakeholders to read and engage with the co-developed research priorities:

‘Successful dissemination strategies included: a press release summary of the Top 10 embargoed until launch day; a launch event e.g. at a scientific conference; a plain English summary for patients and carers; an academic journal article; social media activity e.g. releasing one priority per day over 10 days, or a guest blog; hosting a web page; using email lists of PSP survey participants; giving talks to patient, clinical and scientific groups; a funding organisation alerting current and past grant holders; using the channels and networks of partner organisations e.g. their newsletters; and promoting findings by word of mouth.’ (p.5)

Key Literature

Literature

• Atem, A., Bajraktarevic, J., Nguyen, D., Al Kalmashi, R., Hanna, B., Higgins, M., Lenette, C., Milne, EJ, Nunn, C., & Gardner, J. (2021). Ethics and community-based participatory research with people from refugee backgrounds. UNSW Sydney, STARTTS NSW, Coventry University, Manchester Metropolitan University. https://www.unsw.edu.au/content/dam/pdfs/unsw-adobe-websites/arts-design-architecture/social-sciences/resources/Atem-et-al-2021_Ethics-and-CBPR-report.pdf

• Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P., & Oliver, S. (2014). How to increase value and reduce waste when research priorities are set. Lancet (London, England), 383(9912), 156–165. https://doi.org/10.1016/S0140-6736(13)62229-1

• Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. Lancet (London, England), 374(9683), 86–89. https://doi.org/10.1016/S0140-6736(09)60329-9

• Dullabh, P., Sandberg, S. F., Heaney-Huls, K., Hovey, L. S., Lobach, D. F., Boxwala, A., Desai, P. J., Berliner, E., Dymek, C., Harrison, M. I., Swiger, J., & Sittig, D. F. (2022). Challenges and opportunities for advancing patient-centered clinical decision support: findings from a horizon scan. Journal of the American Medical Informatics Association : JAMIA, 29(7), 1233–1243. https://doi.org/10.1093/jamia/ocac059

• Evans, J.M., Gilbert, J.E., Bacola, J. et al. (2021) What do end-users want to know about managing the performance of healthcare delivery systems? Co-designing a context-specific and practice-relevant research agenda. Health Res Policy Sys 19, 131. https://doi.org/10.1186/s12961-021-00779-x 

• Fletcher-Watson, S., Adams, J., Brook, K., Charman, T., Crane, L., Cusack, J., Leekam, S., Milton, D., Parr, J. R., & Pellicano, E. (2019). Making the future together: Shaping autism research through meaningful participation. Autism : the international journal of research and practice, 23(4), 943–953. https://doi.org/10.1177/1362361318786721

• Fűzi, B., Géring, Z., & Szendrei-Pál, E. (2022). Changing expectations related to digitalisation and socialisation in higher education. Horizon scanning of pre-and post-COVID-19 discourses. Educational Review, 1-33 https://doi.org/10.1080/00131911.2021.2023101

• Géring, Z., Király, G., & Tamássy, R. (2021). Are you a newcomer to horizon scanning? A few decision points and methodological reflections on the process. Futures & Foresight Science, 3(3-4), e77. https://doi.org/10.1002/ffo2.77

• Glass, J., McMorran, R., Currie, M., McKee, A., Pinker, A., Reed, M., ... & Markantoni, M. (2022). Translating community resilience theory into practice: A deliberative Delphi approach. Sociologia Ruralis, 62(4), 675-698. https://doi.org/10.1111/soru.12397

• Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. (2019) Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019;22 doi: https://doi.org/10.1111/hex.12888

• Ioannidou, O., & Erduran, S. (2022). Policymakers’ views of future-oriented skills in science education.
  Front. Educ.  https://doi.org/10.3389/feduc.2022.910128/

• Niederberger, M., & Spranger, J. (2020). Delphi technique in health sciences: a map. Frontiers in public health, 8, 457. https://doi.org/10.3389/fpubh.2020.00457

• Perveen, S., Kamruzzaman, M., & Yigitcanlar, T. (2019) What to assess to model the transport impacts of urban growth? A Delphi approach to examine the space–time suitability of transport indicators, International Journal of Sustainable Transportation, 13:8, 597-613, DOI: https://doi.org/10.1080/15568318.2018.1491077

• Staley, K., Crowe, S., Crocker, J.C., Madden, M. & Greenhalgh, T. (2020) What happens after James Lind Alliance Priority Setting Partnerships? A qualitative study of contexts, processes and impacts. Res Involv Engagem 6, 41 (2020). https://doi.org/10.1186/s40900-020-00210-9

• Sutherland, W. J., Atkinson, P. W., Butchart, S. H. M., Capaja, M., Dicks, L. V., Fleishman, E., Gaston, K. J., Hails, R. S., Hughes, A. C., Le Anstey, B., Le Roux, X., Lickorish, F. A., Maggs, L., Noor, N., Oldfield, T. E. E., Palardy, J. E., Peck, L. S., Pettorelli, N., Pretty, J., Spalding, M. D., … Thornton, A. (2022). A horizon scan of global biological conservation issues for 2022. Trends in ecology & evolution, 37(1), 95–104. https://doi.org/10.1016/j.tree.2021.10.014

• World Health Organization. (2022). Emerging trends and technologies: a horizon scan for global public health. https://apps.who.int/iris/bitstream/handle/10665/352385/9789240044173-eng.pdf

Key Links

Online resources

• Delaney, K.  (2014) Horizon scan tool kit
• James Lind Alliance Priority Setting Partnerships
• Introduction to the The Delphi Approach