(Disclosure: I have a small consulting role on this project)
Wired: Evaluation Settlement Online (WESO) is a research-based program from the YMCA of the National Capital Region that aims to understand the challenges/barriers that service providers and newcomer clients face in delivering/using remote and hybrid services. The project is uncovering evidence-based practices for SPOs to enhance their remote/hybrid services and engaging clients, creating a toolkit/framework and resources for Settlement Service Provider Organizations (SPOs) to evaluate the effectiveness of their service delivery and adapt their models accordingly to improve services, and creating a network of knowledge and promising practice sharing.
In this post I'm providing some background information about how the evaluation toolkit was created and why you should connect with the WESO team soon to learn more about and implement it in your organization.
The WESO team’s experience with an in-person intervention and all of their research data reinforces the need to fully understand the Newcomers SPOs are serving. In particular, are they interested and able to access your full suite of hybrid services?
Below I’ll describe the evolution of the WESO evaluation tools for SPOs to evaluate the quality of their hybrid services and build this understanding of their Newcomer clients.
These tools can help you improve clients' willingness to use hybrid services. They start by helping understand clients’ perspectives, skills, opinions, and interest in digital services.
The toolkit takes the guesswork out of figuring out what works and doesn’t work. By taking the WESO training session SPOs will improve how they evaluate hybrid service delivery, adapt to digital needs, and enhance client satisfaction.
Do you find yourself grappling with critical questions like:
Is my hybrid service delivery truly effective?
What aspects of our service are working well, and which ones aren't?
What do our clients really think about our services?
Find out more below about how the evidence-based toolkit was created and why you should book an information session with the WESO team to learn how your staff can be trained to implement this new, free, and useful tool.
In order to fully understand the service quality and evaluation landscape within Settlement and beyond WESO staff focused initially on understanding what type of services are better delivered in what type of service format and for what type of clients.
A targeted literature review identified relevant hybrid service quality dimensions to be applied to service delivery in the settlement sector.
The team reviewed 250 studies and reports from peer-reviewed publication venues, government guides, and surveys. 120 studies were chosen which underlined two major service delivery themes: conventional (traditional) service delivery and E-service delivery. 28 articles that include both models of service delivery were chosen that highlighted dimensions of service quality utilized in each model.
A Newcomer-Centric Framework
15 focus groups with 60 SPO staff and 2 focus groups were conducted with clients. The focus groups identified themes related to hybrid service delivery, challenges in delivering hybrid services, and what services are preferred online vs. in-person. Focus groups suggest that there is no universal answer for what services are to be offered online or in person. It comes down to the newcomers’ abilities, capacities, and preferences.
The team determined that the settlement sector should have a Newcomer-centric framework that ensures access to services for all in the way that they want, need, and prefer.
Based on the literature review and focus groups, a staff survey was designed to collect staff perspectives regarding many aspects related to hybrid service delivery, hybrid work models, and clients barriers related to online services.
260 complete responses from 112 settlement organizations in 12 provinces and territories of Canada were collected. The survey revealed an encouraging future-focused sentiment among staff members. Staff are willing to provide hybrid services and support hybrid work arrangements. 82.69% want to continue to provide online services post-pandemic. 93.08% agree or strongly agree that hybrid services should continue post-pandemic.
Unveiling Newcomer Experiences
A client survey was created to explore their experiences with hybrid services and drivers that will increase their intention to use hybrid services. The survey will provided the project with the needed information to identify drivers for intention to use hybrid services in the future. But the survey was also designed to be used as an evaluation tool by service providers to use with clients to measure service quality.
240 clients, proportional to each province’s newcomer population, completed the survey.
Based on previous research, the survey focused on 5 dimensions that affect hybrid service quality and drive clients’ intention to use hybrid services in the future:
WESO's findings revealed that staff performance and acceptance of online services are the two major influencers that impact clients’ intention to use hybrid services in the future. These two factors emerged as the primary drivers in determining clients' willingness to engage with hybrid service delivery.
Staff performance and website content and accessibility are the two major drivers affecting acceptance of online services. Effective staff performance instills confidence and trust in clients, while well-designed website content and easy accessibility enhance client independence and acceptance of online services.
While no relationship was found between web security and acceptance of online services survey participants did connect web security and their evaluations of staff performance. Clients who had felt more secure in their online transactions tended to provide higher ratings for staff performance, highlighting the importance of establishing a sense of security and trust in the service delivery process.
While client barriers to accessing online services were identified as a minor influencer, they still had an impact on acceptance of online services. Overcoming these barriers and addressing client concerns can contribute to a higher level of acceptance and utilization of online service channels.
You can learn more about WESO’s data and findings in a series of research infographics in WESO’s SettleNet.org group (registration required, access is free), and on the new WESO website.
Drawing from the wealth of data and evidence collected, the WESO team has successfully developed a comprehensive toolkit to assist Settlement Service Provider Organizations (SPOs) across Canada in enhancing their service delivery models and improving overall client experiences.
The primary goal of the toolkit was to create resources and toolkits for SPOs across Canada to evaluate the effectiveness of their service delivery and adapt their models accordingly to improve services.
The toolkit offers a range of evaluation tools, including self-assessment resources for gauging organizational readiness and hybrid service quality. Additionally, SPOs can leverage a Client Questionnaire as a data collection tool to gather valuable feedback from their clients, facilitating a deeper understanding of their needs and preferences.
The toolkit includes self-assessment tools for organizational readiness and hybrid service quality. The toolkit also includes a data collection tool, in the form of a client survey that SPOs can use to collect client feedback.
WESO has created a ready-to-use Excel template that SPOs can use to generate a report from their collected client surveys. An automated report directs the attention of the SPO to areas that need improvements, areas that have waste, and areas that they are doing right.
These automated reports efficiently identify areas that require improvement, highlight potential areas of waste, and acknowledge the aspects where the organization excels.
The toolkit comes complete with a practical Action Plan that offers a step-by-step guide for SPOs to implement the evaluation approach effectively. It empowers SPOs to assess their performance, identify strengths, and pinpoint areas for growth.
SPOs gain access to a curated collection of supplementary resources assembled by the WESO team. These resources encompass helpful links for online digital literacy, device and connectivity programs, along with invaluable tips and guidance for effective hybrid work. These include:
Ultimately, the toolkit provides SPOs with a step-by-step action plan to implement an evaluation approach to measure and identify areas SPOs are doing great, and areas for improvement.
WESO is committed to supporting SPOs in their adoption and utilization of the toolkit until the end of 2023. Front-line staff, program managers, and decision-makers are encouraged to collaborate and learn together, enhancing their evaluation processes and making informed decisions that lead to improved services and better client outcomes.
The WESO team is available until the end of 2023 to support SPOs to learn more and support their implementation of the toolkit in their work. Front-line staff, program managers, and decision-makers can learn together how to improve their evaluation of their hybrid services and make informed decisions that will improve services and client outcomes.
By learning more about this toolkit and collaborating with the WESO team, SPOs can take decisive steps towards improving their service delivery, enriching client experiences, and continuously enhancing their vital work in the settlement sector.
Connect with the team soon to book a free training session or email them for more information weso@ymcaottawa.ca.
Subscribe to get the latest posts sent to your email.
Please take this short survey to help improve the KM4S web site. The survey is anonymous. Thank you for your feedback! (click on the screen anywhere (or on the x in the top right corner) to remove this pop-up)