What is this research about?
This report examines the theory, research and measurement frameworks informing evaluation strategies in the nonprofit sector and considering their impact on resilience related to immigrant settlement in Canada. The authors outline an opportunity to empower service providers, drive better performance assessments, improve newcomer outcomes and promote more equitable, inclusive, vibrant and resilient communities for all.
What you need to know:
Resilience is defined as the ability to rebound and adapt to change, disruption, adversity and/or stress. It is concerned with the return to a state of stability or even prosperity or greater functionality. For immigrants and their families, resilience relates to their ability to settle, adapt and prosper in their new country with the aid of the settlement service system.
The authors suggest that the resilience of the immigrant settlement sector and the newcomers they serve is dependent on a number of factors. Among these, in terms of resilience, is an underappreciated component centered on accountability and performance measurement by funders.
Funder accountability and performance measurement systems are not neutral tools. They have for example, been connected to excessive organizational operational structures and reporting burdens that shift nonprofits’ accountability focus away from newcomer clients and immigrant communities toward an excessive emphasis on addressing funder accountability ‘needs’.
What did the researchers do?
The authors examine the theory, research and measurement frameworks informing evaluation strategies in the nonprofit sector and consider their impact on resilience related to immigrant settlement in Canada. The complexity and heterogeneity of settlement services delivery and the current state of the IRCC’s accountability and performance measurement system are investigated. The authors detail the opportunity to empower service providers, drive better performance assessments, improve newcomer outcomes and promote more equitable, inclusive, vibrant and resilient communities for all.
What did the researchers find?
Non-profit organizations (NPOs) apply evaluation to support social innovation, identify program improvements, motivate and engage staff, assess partnerships, build capacity, facilitate strategic decision-making, measure the impact of and/or inform the evolution of their organization’s mission and most frequently, as an instrument of accountability and legitimacy.
Accountability can be described in terms of its relationships: upward in relation to funders; downward with respect to its obligations to clients and communities served; and, internal accountabilities to an NPOs mission, staff, volunteers, partners, and management boards. An important distinction exists between instrumental accountability relating to transactions between an NPO service provider and its clients, donors or suppliers and expressive accountability that concerns value driven accountabilities to the community and organizational mission (e.g., advocacy, advancing values and extending caring functions).
Nonprofits operate in resource constrained environments and funding type has been shown to have a profound impact on how NPOs prioritize accountabilities and on the measurement strategies pursued. Research illuminates the dominance of upward accountability requirements in public service contracting which privilege instrumental accountability and short-term burdensome control mechanisms at the expense of capacity building, deeper impact and the expressive mission-focused activities that drive long-term social change. Higher proportions of government funding have also been associated with an increase in using evaluations for symbolic purposes (i.e., obtaining a “seal”). Further research demonstrates how short-term program metrics, such as cost per client measures, can work in opposition to long-term client success. For example, in many human services domains, studies have demonstrated that approaches advancing client agency, self-esteem and independence may take longer and result in non-linear paths; however, these client-led models are more effective, leading to better, more sustainable client outcomes. NPOs recognize the critical importance of nuanced staff judgements and the intangibles that drive results; however, these factors are largely ignored in the dominant funder logic models and measurement frameworks.
An evaluation of Canada’s performance with respect to newcomer integration and settlement must address full and equitable participation, recognition and belonging, both economically and in all aspects of Canadian society. This represents the public interest. Measurement of this nature would appropriately describe the gaps in performance and highlight the place of government intervention, investment and scale of collaboration, innovation and partnership with NPOs required, in order to effectively support newcomer resilience. Regrettably, our analysis revealed the IRCC’s approach to be transactional, onerous and challenging NPO capacity.
In this critical sector, NPO capacity building is essential but funding is insufficient. The government’s emphasis on instrumental accountability (e.g., contract management) at the expense of expressive, mission-focused accountability appears to be odds with the data and analysis required to address urgent settlement system needs.
The authors recommend a more balanced measurement framework that:
How can you use this research?
This research builds on previous research that recognizes immigrant and refugee-serving organizations' resilience. It also acknowledges their difficulties building and resourcing the capacity they know they need in order to help newcomers achieve better settlement and inclusion outcomes. This research is useful in ongoing advocacy efforts with major funders to continue the conversation about client outcomes and the resources organizations need to optimize them.
