Evaluating the Quality of Care Network – what have we learned?

PLOS Global Public Health is delighted to introduce a collection of papers from the Quality of Care Network, with ten papers looking at the quality of care for maternal, newborn, and child health services across a diverse group of countries. Here, the authors reflect on main lessons learned from the work, along with an eye towards policy and future programming implications.

The Quality of Care Network (QCN) is an ambitious initiative to improve the quality of care for mothers, newborns and children in network countries. Following a meeting of all 11 member countries in Accra, Ghana in 2023 the network is moving into a new phase looking towards 2030. In our QCN Evaluation collection of papers for PLOS Global Public Health we evaluate the first phase from 2017 to 2022. Our collection of studies focuses on four diverse case study network countries: Bangladesh, Ethiopia, Malawi and Uganda. We investigate network emergence, legitimacy, effectiveness, configurations and interactions, theory of change, individual, organisational and system circumstances, innovation, learning and sharing , and sustainability. We also reflect on our experiences evaluating the QCN. In this blog post we highlight our key findings and insights from our work, looking at each of these areas of study in turn.

Network Emergence

In Shawar et al 2024 we show how the network emerged more strongly and quickly in some countries than others and how this depended on the policy environment and network features in each country. QCN was conceptualized by the World Health Organization (WHO) and other global partners to facilitate learning on and to improve the quality of care for maternal and newborn health within and across low and middle-income countries. However, there was significant variance in the speed and extent to which QCN formed in the involved countries. QCN emerged most quickly and robustly in Bangladesh, followed by Ethiopia, then Uganda, and slowest and with least institutionalization in Malawi. With respect to the policy environment, pre-existing resources and initiatives dedicated to maternal and newborn health and quality improvement, strong data and health system capacity, and national commitment to advancing on synergistic goals were crucial drivers to QCN’s emergence. With respect to the features of the network itself, the embedding of QCN leadership in powerful agencies with pre-existing coordination structures and trusting relationships with key stakeholders, inclusive network membership, and effective individual national and local leadership were also crucial in explaining QCN’s speed and quality of emergence across countries. The pattern, speed, and extent of a network’s initial emergence is likely to reflect and have implications on a network’s legitimacy, configurations and interactions among involved actors, effectiveness in producing stated objectives, and sustainability—each of which are discussed below.

Network Legitimacy

In Akter et al 2023  we investigate legitimacy of the QCN in each country, finding it reflected in the government’s buy-in and perceived ownership of the network activities. In this paper we developed a conceptual framework based on three models encompassing organisational features, legitimation process, and audience beliefs; the purpose, procedure, and performance of institutions and how they problem-solve; and political, normative, and cognitive interactions within and between organisations involved in the network. We find political and normative interactions were favorable within and between countries and at the global level. Collective decisions, collaborative efforts, and commitment to QCN goals were observed at all levels. Resource sharing and common principles were limited between network countries, indicating limits to network action across countries. Cognitive interactions were also more challenging than political and normative interactions, with different methods of quality improvement or ways of working common across network organisations and countries. Following from this, bi-directional transfer, synthesis, and harmonization of concepts and methods was also largely absent. Overall, we found that further efforts and time are required for governments of all four case countries to own and embed QCN work in their health systems as routine care.

Network Effectiveness

In Djellouli et al 2023 we assess the effectiveness of the network in relation to its four strategic objectives, leadership, action, learning and accountability. We found that at the global level, QCN – led by the WHO secretariat – was effective in bringing together government and global actors via providing online and in-person platforms for communication and learning. In all countries, governance structures for quality of care were set-up, some preceding QCN, and were found to be stronger and better (though often externally) resourced at national than local levels. We also investigated how various interventions were delivered in ‘learning districts’, which were often implemented separately by different partners in different locations, and were disrupted by the pandemic. QCN actors demonstrated various awareness levels of operational plans and network activities between countries, which was lower at local than national levels, but increased from 2019 to 2022. Engagement with, and value of, QCN was perceived to be higher in Uganda and Bangladesh than in Malawi or Ethiopia. At the time of the evaluation, capacity building efforts were implemented in all countries – yet often dependent on implementing partners and donors. QCN stakeholders succeeded in agreeing on 15 core quality monitoring indicators though data collection was challenging, especially for indicators requiring new or parallel systems. As a result, accountability initiatives remained nascent in 2022. Overall, global and national leadership elements of QCN have been most effective to date, with action, learning and accountability more challenging, partner- or donor-dependent, remaining to be scaled-up, and pandemic-disrupted.

Configurations and Interactions

In Mukinda et al 2023 we use Social Network Analysis (SNA) to map QCN networks in Bangladesh, Ethiopia, Malawi, Uganda, and at the global level, identifying 566 key actors and revealing diverse network structures with a mix of centralized and multiple-hub networks. Our findings highlight the unique and complex nature of each country’s network. Common features included a low level of interactions between actors in each network, centralized interactions around a few key actors, and varied actor engagement. Specific countries show different patterns: Ethiopia’s network is more centralized, while Bangladesh, Malawi, and Uganda exhibit multiple-hub networks with diverse engagement.  The SNA results affirm our broader qualitative evaluation assessing the nature of these networks, including composition and leadership. In line with our findings in the three papers summarized above, our findings in this paper highlight the need to strengthen communication, increase interactions at all levels, especially at the local level, and enhance bi-directional links to improve network cohesion and effectiveness in quality of care initiatives. 

Theory of Change

In Dube et al 2024 we show that the theory of change for the Quality of Care Network wasn’t applied consistently across different countries. While the plan emphasized leadership, action, learning, and accountability, other approaches and methods were also used. Key challenges included a lack of necessary resources, which affected care quality, and the fact that key partners were not introduced to the plan early enough. The theory of change was created at the start but didn’t seem to guide activities or monitoring, evaluation and accountability as expected. We suggest that allowing countries to develop their own tailored version of the theory of change, based on the global framework, could improve how stakeholders are involved, help address local challenges, and ensure more focused efforts to achieve the program’s goals.

Individual, Organisational and System Circumstances

In Tesfa et al 2023 we examine how individual, organizational, and systemic factors influence the functionality of the Quality of Care Network (QCN) across the four countries, identifying common themes. Key findings at the individual level highlight the necessity of training and motivation, the influence of a culture of documentation and data use, and the critical role of leadership in driving quality improvement. Organizationally, challenges included inadequate staffing, the need for continuous skill-building, and limited physical resources. Systemically, policy alignment, partner coordination, parallel reporting systems, healthcare financing, and managing disruptive events like the COVID-19 pandemic were significant. Despite improvements in some areas, persistent issues such as high staff turnover, poor documentation practices, and uneven leadership quality hindered QCN’s effectiveness. Effective leadership and adequate resources emerged as essential for sustaining QCN activities and improving maternal and newborn health outcomes.

Innovation, Learning and Sharing

In Mwandira et al 2025  we investigate the extent to which the QCN enabled innovation, learning and sharing for improvement of quality of care. We show that the Quality of Care Network was useful in fostering learning and sharing platforms in the study countries. These platforms included both in-person and virtual meetings and seminars. The network promoted coordination among stakeholders, and several sharing platforms and meetings were conducted to equip countries with implementation updates and facility-based QoC approaches. However, innovations were not as apparent across countries as most of the approaches were adopted from a broader pre-existing maternal health era preceding the QCN. To make sharing and learning more effective in global health networks, we suggest countries adopt strategies based on their country context and capabilities, in order to incorporate strategies that will be a success rather than an unutilized opportunity.

Sustainability

In Lemma et al 2023 we investigated six key areas for sustainability the four countries should take to successfully sustain the QCN in the respective countries. The six sustainability actions are quite interconnected, and sustainability can only be ensured if all actions are met: planning opportunities for reflection and adaptation from the outset, strong government ownership, motivating micro-level actors, institutionalizing the innovation within the health system, managing financial uncertainties, and fostering community ownership.  In this study, we observed that countries variably managed to create opportunities for reflection on and adaptation of the QCN activities at the initial stage of the activities. All countries showed a strong commitment mainly at national level to fully institutionalize the QCN activities and make it their own flagship program despite some setbacks. Countries struggled to ensure financial sustainability for the program and failed to leverage their strong community health programs for the purpose of QCN activities to foster community ownership.

Our Experiences Evaluating the Quality of Care Network

We reflect on our experiences evaluating the QCN in the final paper in our collection by Seruwagi and colleagues. Evaluating such a complex network in four different countries, and during the COVID-19 pandemic, was never going to be an easy task. However, the evaluation team was the right one for the job – mostly because of how the task was approached. We had a unique team composition – globally spread, diverse in seniority and discipline/expertise (e.g. medicine, maternal, newborn and child health, policy and health systems, networks, social and behavioural science). We tried to balance supportive centralized standardization with the necessary flexibility allowing for adaptation to the diverse evaluation contexts. Our research teams in each of our four case study countries, and QCN stakeholders – led by the respective Ministries of Health – provided leadership and support for the evaluation. Overall, strong leadership, partnership, communication and coordination were key to successful implementation of this evaluation; as were co-production, flattening hierarchies among study team members and the iterative nature of data collection. There was a clear division of roles and responsibilities and embedding capacity building as both an evaluation process and outcome, and optimizing technology use for team cohesion and quality outputs.

One of the things that we found interesting is that – looking back in retrospect – some of the evaluation’s methodological adaptations, which included leveraging technology that became essential during COVID-19, have now been normalized in the post-pandemic era. At the time when we were implementing the evaluation, virtual teams and working methods were still new, and we considered it an innovative adaptation for our team to be able to largely tap into that less-known resource and approach at the time. Wonderful to now look back to appreciate what a complex period it was for the conduct of this evaluation, and the impact of COVID-19 on the conduct of increasingly online research and evaluation!

Concluding thoughts

We encourage readers to delve deeper by reading all of the papers in our QCN Evaluation collection. An understanding of the key factors shaping variable network emergence, legitimacy, effectiveness and sustainability across countries is critical to WHO and global partners, as they seek to establish QCN activities in other countries, but also more broadly for policymakers seeking to initiate global health networks and partnerships across various contexts. For the latter we are pleased that our work has recently informed WHO guidance on newtorks of care. It is also crucial for national governments and partners to implement the network well across their hospitals and health facilities, learning from our work and that of others. We hope this will happen in an expansion of the network within network countries, and to additional countries, in the coming years.

About the authors: