The YMCA of the National Capital Region’s project, Wired: Evaluating Settlement Online (WESO), is a research-based program that addresses the challenges/barriers that service providers and newcomer clients face in delivering/using remote and hybrid services. The project is uncovering evidence-based practices that Settlement Service Provider Organizations (SPOs) find promising in delivering remote/hybrid services and engaging clients, creating a toolkit/framework and resources for SPOs to evaluate the effectiveness of your service delivery and adapt your models accordingly to improve services, and creating a network of knowledge and promising practice sharing.
The WESO team has created an evaluation tool for SPOs to assess the quality of your hybrid services and increase client intentions to use hybrid services. This toolkit also includes a data collection tool, in the form of a client questionnaire, that SPOs can use to collect client feedback.
Their Quality Assessment Toolkit for Hybrid Settlement Services provides a methodology and tools you can use immediately to evaluate the quality and effectiveness of your hybrid service delivery. It also highlights tips and best practices to support SPOs to adapt and improve your hybrid service delivery models.
The WESO project defines hybrid services as:
Hybrid service refers to a service delivery mode where clients have the option to interact with service providers both offline or in-person and through online platforms. It combines the convenience of online access with the personal touch of in-person interactions, empowering clients to choose the mode that suits them best based on their specific needs at any given moment.
Why does it matter?
Like many SPOs, you may be struggling with questions like:
In February 2021 I moderated a panel that asked the question: How do I know my transition to virtual service delivery is effective? There were some great insights, tools, ideas, and discussions.
But the question has hung over our sector during the pandemic and as we more or less formally shift to a hybrid service delivery model.
The WESO toolkit is helping to answer all of these questions (disclosure: I have a small consulting role on the project).
Most importantly, the analysis you get from the Toolkit will provide you with specific areas where you need to take action to strengthen your service delivery and improve client performance. It will help you to prioritize your efforts, based on what your clients are telling you is important to them and where you perform or under-perform.
Feedback from your clients is the most essential information you need to collect and analyze to ensure your services are meeting their needs and creating necessary outcomes.
The Toolkit will tell you where you need to improve. You'll know where you’re doing well. You'll also see where you may be over-focusing (i.e. you perform highly, but they’re not as important to your clients). As well, you'll identify areas that are not particularly important to your clients, where you don’t need to focus.
The WESO Toolkit and approach take the guessing out of your evaluation work. Replace anecdotal perception of what is working with evidence from Newcomer experiences to focus on what is actually working.
What do you need to know?
The WESO toolkit is based on the findings of extensive research and engagement with newcomer clients and staff at SPOs across Canada.
The survey focused on 5 dimensions that affect hybrid service quality and drive clients’ intention to use hybrid services in the future:
A client experiment was created and implemented focused on Newcomers who had not or had not wanted to access online services. The goal was to find out if a small training intervention might help Newcomers. 40 Newcomers received digital literacy training and the WESO team evaluated their experience and feedback.
Training covered areas related to everyday practical computer skills including:
It turns out even a small intervention makes a big difference.
Two-thirds of participants overall agreed that their knowledge and skills improved by taking our training. 85% of participants have had a positive experience accessing online Settlement services after the training.
A vast majority of Newcomers WESO surveyed and trained think hybrid services should continue (85.8%) and 86.3% of Newcomers surveyed appreciate a multiple service delivery mode.
We know that there is no one-size-fits-all-Newcomers approach to hybrid service delivery. We all know that our Newcomer clients are diverse. It isn’t surprising that this diversity should extend to how Newcomers want to access services.
We need to get the hybrid service mix right.
What you can do right now
Getting it right means knowing you got it right, where you didn’t, and how to learn from what you know and incorporate it into your service planning.
The WESO team has created not only the Toolkit, but training and tools to help you implement it in your work.
The toolkit and training includes:
In addition to the existing toolkit and training, the WESO team has developed three self-paced learning modules to enhance your ability to evaluate hybrid services. These modules focus on key areas of data collection, research methodology, and statistical analysis skills. By completing these modules, you will gain the necessary knowledge to assess your services effectively and make informed decisions for service planning.
The WESO team is running training sessions now. You can attend one of their upcoming sessions. If you're on SettleNet.org, the sector’s Community of Practice, join WESO’s SettleNet.org group for info and updates. You can also access their toolkit materials and more on the Y website.