We have been working in collaboration with Dr. Mathieu Ouimet from the Department of Political Science, within the Faculty of Social Sciences and his team at Université Laval, since September 2018, to evaluate the Diabetes Action Canada a CIHR Strategy for Patient-Oriented Research (SPOR) Network. Dr. Ouimet’s team has expertise in conducting Social Network Analysis (SNA) to gain an understanding of relationships between individuals. This ongoing work is a component of our role as the Health Technology Assessment and Network Analytics program of Diabetes Action Canada.
In October 2019, we presented and provided our first evaluation report to the Steering Council of Diabetes Action Canada. The network evaluation, using a State of Network Evaluation framework, examined three key themes: Network Connectivity, Network Health and Network Results. The Network Evaluation was well received by the Steering Council.
Multiple methods were used to study these Network themes. A cross-sectional social network analysis of members was completed to examine Network Connectivity and to assess the frequency of interactions and the topics discussed during them and how networking effectively facilitates interactions and collaboration among its members. Network Health was assessed through semi-structured qualitative interviews, a membership survey inquiring about satisfaction and experience with the Network and a review of funding and infrastructure to evaluate Network Sustainability. Finally, Network Results and Impact were examined using of the Canadian Academy of Health Sciences (CAHS) preferred framework and indicators to measure returns on investment in health research.
Our analysis found that within Diabetes Action Canada SPOR Network, which has a membership of over 150 Patient-Partners, Researchers and Clinicians across Canada, the most common exchange of information between members relates to research and patient engagement, and that new collaborations are one of the Network's key success factors and for many members it was a significant motivating factor to join.
Network Results and Impact were examined across the 5 domains of the CAHS framework which include: advancing knowledge; capacity building; informing decision making; health impact; and economic and social impact. The ability to examine the impact of the Network on advancing knowledge, and capacity building through bibliometric evaluation of the 59 identified peer review papers that had been published by the Diabetes Action Canada SPOR Network members over the past 3 years, and to describe the support of trainees within the research groups across the country.
Influence on informing decision making was also assessed through the description of engagement with government at a municipal, provincial, federal and international level by the Diabetes Action Canada Executive Director, Scientific leads and Investigators. As the Network is only 3 years old, and is still evolving, the health, economic and social impact of the research and network activities remain to be fully realized, as may be expected based on the age and the stage of development the network.
As the projects and different initiatives mature, the Network will have further opportunity to influence healthcare and research policy as well as economic and societal well-being in Canada. We are currently developing a series of manuscripts to describe the methods used, challenges faced and results with this unique comprehensive approach to network evaluation.
If anyone is interested in learning more about Network Evaluation, please speak to Val (valeria.rac@theta.utoronto.ca) or Jim (james.bowen@theta.utoronto.ca).
Acknowledgements:
Justin Lawarée, PhD, Ashley Rhéaume, MA, Mathieu Ouimet, PhD
Département de science politique, Faculté des sciences sociales, Université Laval
James Bowen, BScPhm, MSc, Joanna Bielecki, BSc, MISt, Valeria Rac, MD, PhD.
THETA Collaborative, University Health Network, IHPME/DLSPH University of Toronto
Diabetes Action Canada. Strategy for Patient-Oriented Research Network.
Network evaluation report provided to Diabetes Action Canada. Version 2.0. October 25, 2019. (pp. 62)