From Data to Strategy: Shaping Client-Centered Evaluation in Settlement Services (webinar recording)

In this November 2025 P2P Conference presentation, panelists asked: How confident are you that your services truly meet client needs? They introduced tools to evaluate service quality, measure client satisfaction, and support evidence-based decisions. The YMCA-led, IRCC-funded IMPRESS program is developing an online platform with customizable surveys in multiple client languages and built-in analysis for easy interpretation. With automated reports and visual dashboards, SPOs will be able to identify strengths, uncover gaps, and demonstrate impact with confidence. You will leave with insights to improve service delivery and ensure programs remain responsive, strategic, and truly centered on what matters most: your clients.

The presentation also outlines the evidence-based, participatory approach used to develop these tools, which includes extensive community consultation and national collaboration to ensure the final products meet the needs of both SPOs and newcomer clients. The presenters provide details on the specific components of the toolkit, including logic models, performance measurement frameworks, and continuous improvement plans, and the platform's features, such as pre-made question banks and automated dashboards for interpreting client feedback and service outcomes. The project builds on the previous WESO project.

AI-generated transcript:

good afternoon everyone and welcome my name is Camila and I will be your chairperson for today's session titled from data to strategy shaping cliententric evaluation in settlement services thank you for joining us as we begin we respectfully acknowledge that the YMCA of the National Capital Region pay respect to the Amishaba Alongan people who are the traditional guardians of this land we acknowledge their longstanding relationship with this territory which remains unseeded we pay respect to all indigenous people in this region from all nations across Canada who call Otava home we acknowledge the traditional knowledge keepers both young and old and we honor their courageous leaders past present and future it's our role as professionals to continue unlearning and uphold key principles such as those outlined by the Truth and Reconciliation Commission of Canada and United Nation Declaration on the Rights of Indigenous Peoples allow me to introduce the impress team who will be presenting our program reinila Haridas program manager noa Zaher research officer andrew Doson research officer to start us off I will now turn it over to Reil our program manager reil the floor is yours please amazing thank you so much Camila so now that you know a little bit about our team I'd like to tell you a little bit about what we will be presenting on today and this uh basically starts with how impress was born and kind of the background of why we think this project is so important and then moving on to what inspired Impress and what it was built upon and then we actually talk about Impress in practice along with our final tools that are going to be built through national collaboration and lastly we look ahead to feature an upcoming activities and then finish off with a question and answer period of course to gather your perspectives and questions that you may have great thank you and of course Impress is under the umbrella of the YMCA and that includes the YMCA of the National Capital Region specifically in Ottawa and why YMCA in general of course wants to have and contribute to equitable access in the community through services supports and different opportunities and they do this do this through many different programs and some of which are outlined below as you can see uh through facts outlined in the 2024 report and specifically you can see that over 19,000 newcomers were welcomed through the YMCA services and impress aims to also improve service quality and outcomes for service provider organizations who are welcoming newcomers as well all across Canada and now that you know a little bit about uh our team and uh the project itself I'd like to dive in about the overview of um how Impress was born by passing it off to Andrew thank you Reanella so I'm going to talk a bit about the why of impress um the growing need that needs to be fulfilled so in the settlement sector we are increasingly expected to demonstrate that our services work we are asked more and more to track outcomes and provide evidence of impact but this is not really easy to do not all SPOS have expertise in measurement not all of them have the capacity for evaluation sometimes data is collected sometimes it's not um often it's collected as half as hazardly often it's left unanalyzed because organizations don't know what to do with it and when it comes to the staff they often don't have time they don't have the training um and they just don't have the structure and the guidance they need to improve their practices and track outcomes and this is the reason for impress we want to make evaluation and data collection easy impress stands for integrated measurement for performance review and enhanced settlement services it's a program led by the YMCA of the national capital region uh three years in length funded by the IRCC so our goal is to help SPOS's evaluate their services and we're building several tools to do this we will provide free bilingual tools to simplify data collection and evaluation this includes an online platform to help you collect data on service quality and on outcomes uh we will enable SPOS to easily gather and analyze client feedback and this will be done through automated dashboards and reports that help you understand your results as soon as data is collected and now I'll pass it over to Noah to explain how we got started thanks Andrew so section two um okay so why do we evaluate our services and why it is essential evaluation helps programs to determine where improvements are needed shows whether the services achieve the desired results and helps programs track down their impact and success next slide please uh through sector consultations we identified several gaps in evaluation capacity organizations reported limited guidance for conducting evaluations no standardized means of collecting client feedback or measuring outcomes and language barriers that made data collection difficult especially with the newcomer community this is where Impress comes in and provides easytouse templates a multilingual online platform customizable surveys and a national training program for 800 SPO staff our goal was to make evaluation accessible and userfriendly for all IRCC funded SPOS's our story actually began with the Wesso project wired evaluation settlement online which paved the way for impress wesso was also an IRCC funded project that strengthened evaluation practices across the sector helping SPOS's improve hybrid and remote service delivery through WESO over 200 organizations and 600 staff learned how to evaluate client needs optimize service quality and use data to drive improvement impress builds on that success expanding from hybrid evaluation to a full performance framework for the sector building on the WSO success Impress was designed with clear objectives first to strengthen newcomer services uh across Canada enhance inclusion and evidence-based decision- making equip service provider organizations with practical tools and training and build a culture of transparency shared learning and accountability these very objectives are translated into deliverables into action next slide please these very objectives are translated into action through our deliverables including multilingual feedback tools an online evaluation platform a performance measurement toolkit and training equity centered tools integrating GBA plus principles and also national training for more than 800 SPO staff across Canada all of these come for free for all our IRCC funded programs so now I'll hand it uh to back to Rola great thank you so much Noa now that you know a little bit about the background of Impress and how it started I'll talk a little bit about how it works in practice and how we collaborate to achieve hopefully in the future soon our final tools for this project as well so of course you saw that there's a lot that goes into you know creating impress and that includes research a lot of research and everything is very evidence-based and that basically you know because of that process what we are doing here is kind of providing a very simple framework to follow and firstly that includes data collection and so for us that's specifically collecting data from different interest holders and that's specifically clients and staff for example and then after you analyze this data So basically you look for different gaps that these interest holders talked about as well as you know positive aspects and then you identify trends based on this data from interest holders and then lastly the most important part actually allow for that decisive action to ensure lasting impact and so that is the continuous improvement cycle however none of this is actually possible without the participatory approach and next slide please and this really includes that data collection portion so basically through these interest holder perspectives such as client and staff as well as our consultants and service provider organizations clients and our advisory committee which I'll talk about a little later they all help us to ensure our project is actually meeting community priorities instead of us assuming what those priorities are and this is really the key uh aspect of participatory approaches as most of you may know and it really gives you know a voice to the population that we are serving and allows for that iterative feedback for our project this not only allows for interest within our partners but also data ownership for SPOS which are service provider organizations is what we're going to be calling them moving forward as well as uh the clients and newcomers themselves and all this really amalgamates to our final tools that are built with our uh national collaboration team so all the interest holders that I was talking about before and as I mentioned there are you know newcomer clients there's SPOS there are also consultants that we work with but also we have our national engagement and outreach partner AMSA they help us reach rural populations as well as organizations uh supporting different equity groups as well and um so they help us with our outreach on that side and then as I mentioned earlier we also have our advisory committee and we have 28 national partners there and uh as you can see in the next slide there's quite a few so I won't go through all of them but they are spread out all over Canada uh through multiple provinces and and territory as well uh and we iter iteratively meet with them to you know gather their feedback and perspective on uh our project and our project materials as we build them as well next slide please and now giving you a little bit of an overview now that you know who is helping us build our tools so as mentioned of course partner and interest holder engagement is very important to us so uh we iteratively engage our community through multiple uh you know through all our phases and all our different materials that we're drafting and this includes the first phase which has already been completed so this is that literature review that we wrote up about the development and methodology of our project itself and we also had focus groups with 104 staff and 30 clients uh to gather feedback on our questionnaires and then in phase two that we have planned uh and it's actually starting in December we will be doing more focus groups with staff and clients uh to gather and finalize our questionnaires as well as pilot the actual surveys with 180 clients as well and all this comes to our final tools as I mentioned and that firstly includes the performance measurement toolkit and this is essentially a downloadable and customizable toolkit where basically SPOS's can fill out you know certain uh different columns or rows and then it actually automatically gets filtered into the remaining templates to allow for easy performance measurement of their services and outcomes and so there will be a guide book for that and some training and then on the other side there's also an online platform um and this platform houses the toolkit but also includes a question bank with all of our questions um from our questionnaires and this essentially allows SPOS's to choose you know questions from this question bank but also customize it with their own if they choose actually send a link to clients to fill out this survey collect that data and actually visualize it on our dashboard on the platform as well and so there will be user guides and training for the platform as well but now to dive into kind of those nitty-gritty details of each of these tools I'll pass it along to Noa to talk about the toolkit thanks Ranella um so uh let's start with the performance measurement toolkit uh next slide please performance measurement is not easy but our goal is to simplify performance measurement through the toolkit the toolkit serves as a step-by-step guide to help service provider organizations make their own logic model their own performance measurement framework and their very own continuous improvement plan through readymade templates which we're going to discuss in more details in the comes in the coming slides so let's start off with some definitions the logic model is this visual representations representation that shows the relationship between your programs resources activities outputs and outcomes the performance measurement framework defines your indicators your data sources and benchmarks and the targets that you aim to achieve while the continuous improvement plan helps you track and refine progress over time we'll get in more details into those three components that make up our main three tools in the toolkit so for in the next slide we show you a prototype for the toolkit this is a very early prototype for the toolkit so it's still under development it it's not a final product we will have we are aiming to have two main components the first component is the toolkit the toolkit includes a quick a quick start on how to use the templates and the different tools and templates that we have that you can download and use like our logic model template our performance measurement framework template and our continuous improvement plan the guide book is a more detailed book for our training that has uh more information about performance measurement how we built the toolkit how to use the tools and additional information and resources related to evaluation of programs so let's start showing you the first template uh that we have today the logic model template this is a template for the logic models it links inputs such as the resources and personnel that you put into your program with outputs which is the what services you provided the count of service the workshops that you actually delivered and the outputs are linked to the outcomes that which measures the real desired impact of the project and whether the project was successful and made the changes that we want while output while outputs the the the client and service counts are essential outcomes should be what guides the project design and determines whether a project was successful our aim is to move for performance measurement from outputbased to be more outcome oriented looking at the final outcome and the success when we talk about outcomes we're really we are really looking at three levels of change as we see here in this uh picture uh that happen over time immediate outcomes intermediate outcomes and long-term or ultimate outcomes first we have the immediate outcomes these are the short-term changes we expect to see in participants as a direct result of our activities they usually happen within the first six months uh the first six months after the service for example participants might show improved knowledge stronger skills or new connections as a result of taking part in the program next we move to the intermediate outcomes or the medium-term outcomes these are the medium change outcomes that often appear between six months and three years after the after the service at this stage we're looking for participants actually applying their new skills and knowledge leading to changes in their behavior or practice finally we reach the ultimate outcomes the long-term big picture changes usually they're not measured and are out of scope because they are not the result of a certain program they are they are a result of a collection of settlement programs that help in the successful integration of the newcomer client in Canada and the last two components of our logic model are the assumptions and external factors assumptions are the conditions needed for the program to work as intended like what you assume should be so that the program runs as you expected and the outcomes are achieved and external factors are the conditions outside the program control that might influence the outcomes or influence the achievements of the program such as economic conditions label market conditions or policy moving on to our next template the second tool we have is the performance measurement framework template the performance measurement framework is usually prepared at the start of the program this is the initial plan of how success looks like and how to measure your indicators in order to uh finally achieve the desired outcomes and the desired intended outcomes in your logic model as you see the template was filled with an example from employment related services the indicators and outcomes come from the IRCC outcomes and measurement guide the template will guide SPOS's to the relevant outcomes and performance indicators to their specific settlement service set by the IRCC uh we will provide SPOS with an inventory of measurable measurable indicators that they can track down to make sure outcomes are achieved after the outcome and indicator there comes the data source uh column this is where will you get the data to measure the indicators some indicators could be collected through the Impress platform and others could be collected from other sources next the baseline column it provides a reference point for comparison this allows us to see if performance is improving over time this could be a baseline that you measure at the beginning of the project or previous cohorts or comparable projects that you had the SPOS's will then fill their target which outlines the desired level of achievement for each indicator it helps set a clear performance goal and then the column of collection frequency how often is our indicators measured complexion date the period the month the quarter when the target is to be achieved and the last column responsible parties who's responsible for collecting the data the next slide after like after we after setting the initial plan our performance measurement template and how we're going to measure our success we need to make sure that we really measured the success and what actual results are achieved and this is what the continuous improvement plan template does for you the template will help SPOS to fill in their actual results for indicators which can come from the platform that Andrew will speak in a minute about or from other sources compare them with targets and benchmarks that they have already set in the performance measurement framework their initial plan and decide what actions needs to be taken who's going to be responsible and the deadline to review the corrective actions now that we have reviewed the plan toolkit templates I'll hand it over to Andrew who's going to tell you more about the second component of our project the platform all right thank you Noah so the second of the two main tools we're building is the survey platform and whereas the performance measurement toolkit we just covered was focused more on making a plan for your programs and making a plan for tracking information the survey platform is focused on helping you collect data that you may not have um so we aim for this to be a userfriendly digital tool that'll help you with every stage of the process so collecting data analyzing using that data in various ways uh and this will be a full service platform where you can make an account go into the survey builder customize your surveys what will distinguish this from other survey builders that you might have used in the past is that this includes a question bank that we have made specifically designed for settlement services and also specifically designed for different areas of settlement services so you'll be able to insert your own questions as well but a lot of the value comes from the fact that you have these questionnaires prepared that you know have been tested and you know are relevant to what IRCC is looking for so just to give an overview of some of the features we want to facilitate easy data collection using our pre-made surveys which will also be multilingual so um clients can respond and read the questions in several different languages we will also provide automated outcome reports um showing you the results as soon as you have data collected so that's something that will help you interpret all this means what you can do with the results um even if you don't necessarily have statistical experts on your team um as well we will have GBA plus integration throughout meaning that everything here has been done through an equity focused lens you will have options to look at different groups and make sure that every group is having their needs met and of course we want to make sure that every step of the way data is secure and client privacy is protected diving in a little more into our question banks this diagram here outlines the different sections of our questionnaire so each of the sections on the far left will have a variety of questions for you to choose from um we chose what kind of questions to include in our survey based on the information that you likely not already have easily accessible so it's quite probable that you are already tracking your spending that you know your employees hours that they're putting in and it's quite likely through IARE or maybe another case management software um that you're already going to be tracking your activities the services you provide the number of clients you serve stuff like that but what you might not have at least not using standardized research methods is a way of measuring client opinions about the quality of the services so the actual customer experience how well you're meeting their needs uh you may also not have an existing way of measuring how easy it is to access and use your services at least from the point of view of the client so we've developed several measures of service quality up there in orange and service convenience in blue here based on previous research to answer these questions as well um while you might be tracking the activities like you said and the clients you serve you may not have means of measuring the impact of your services that is the outcomes and that's the second component of our questionnaire that we intend to provide and it's here at the bottom in green the outcomes we have chosen were also selected for a very particular reason because the IRCC provides guidelines for which outcomes they are looking for and the indicators that can be used to measure these outcomes and the IRCC also runs an annual survey for newcomers called the newcomer outcome survey in which they have measures to assess all these outcomes so all of the outcome measures in our survey will be based on the measures used in the IRCC's newcomer outcome survey this means that not only will you be tracking the outcomes that IRCC is most concerned with you'll be tracking them in the same way as IRCC so this is really the core of the online platform you'll be able to take these questions customize your own survey but you'll know that every step of the way is based on research that we've conducted and is built to fit what IRCC is looking for there is um just one additional feature of our service quality measures that I'd like to highlight so for each question you will have the option to measure both performance and importance so here you can see that there are two rating scales um for each question you can ask about the performance that is how well you did um how good the service was and the other rating scale is how important that aspect of the service was to clients so it's really that simple there's two rating scales uh but why might it be helpful to measure both performance and importance well um imagine that you discover that you're not performing well or at least not as well as you'd like on several different aspects of your service so maybe feedback is positive in general but um clients might be telling you that the written materials aren't easy to understand and the the equipment provided wasn't functioning well so which of these things should you prioritize which is most important to address well for this you can look at your importance measure so the equipment functioning may be something where you have problems so it would probably be on the low performance side if you get low performance scores on that scale uh but maybe clients are also rating it as low in importance so this would fall in the low priority section here uh and it means that maybe you do want to address it but it might not be uh your first priority the clarity of the written materials you're providing to clients that might also be on the low performance side you may be getting low performance scores uh but maybe clients see it as very important so if it's low performance and high importance that's something where you really need to pay attention because your clients care and they're not having their needs met um and so that is why we've included this feature if you measure both importance and performance you can really see um where you need to focus and again once you have collected your data uh once you've asked your clients what you want to know our platform will be able to tell you what some of the results look like before you even need to do any analysis yourself so to demonstrate this I have a prototype of the dashboard here again this is not a final product it's not what they will exactly look like um and I'm not working with real data either but I want to give you an idea of what our tools will be able to do um so there's a number of different areas here that you can look in uh the idea is that you can go as simple or as complex as you like if you just want to take a quick look at okay did we do good in this area did we do good in this area you can do that if you want to dive in deeper you can do that too so let's take a look at client satisfaction for a moment um we have a variety of information in this dashboard we can see the gender distribution how many men women and other um we can see the age distribution how many people fall into different age categories um and you can you can look a little closer so if you hover over you get a bit more information but if you click here it'll actually filter all the other results so right now we're just looking at men uh but also you could just look at women or just look at people 46 to 55 um you can drill in closer if you like uh we also have some measures of client satisfaction on the left here um so the statement I'm happy I decided to use newcomer information services in this case at the YMCA you can switch that out um we see that a lot of people are saying strongly agree which I would take to be a good sign we can also see that people generally strongly agree or agree that they are satisfied with their decision to use the newcomer information services and people um seem to say that it is quite likely um that they are going to recommend these services to friends and family so again we can keep things pretty simple but if we want we can take a closer look we can make it more complex so for example I'm going to select this importance performance matrix here we'll be looking at that rating system that I was telling you about um where you can measure both performance that is you know how well you're doing as well as the importance of that thing for clients so if we go in I have something here that shows a chart with the performance on one axis and the importance on another axis um let's narrow it down a bit let let's just look at accessibility let's see how accessible uh clients think our services are um so you can see for each of these different facets easy to find information service being available you can see how well you did you can see how important clients thought it was uh you can also just look in these individual quadrants so uh again if something's high performance high importance fantastic keep up the good work um but if it's high importance but actually on the lower side of importance or of performance sorry um then that means you should be paying more attention because um if it's important to clients and you're not meeting expectations that's a problem so that's where you need to be um focusing what's useful is you can filter by this as well so let's say I'm in accessibility i want to know are there any aspects of accessibility where we're falling short i can just click needs improvement and then it'll tell me all of the sections where um clients thought it was very important but we weren't meeting those expectations so we can see here um it was easy to find information about the service and how to access it online so if we're not doing so well there we should probably work on making information easy to find uh the service was available in my preferred format um if we're not meeting those expectations maybe we should consider uh maybe we have clients who need to go online or would prefer to go in person we want to offer different options um let's see we can go even further if we want so if we go into uh just an example service quality dimensions here let's look at accessibility as well we're still focused on how accessible our stuff is we can get a breakdown a detailed breakdown of each respondents or not each respondents but each questions u distribution of responses um just to see okay so for example a lot of people are strongly agreeing but we're also seeing you know some disagreement as well so we can look in a little more detail uh you can see there's a lot of options here ranging from very simple features to the more complex um and again this is all fake data it's just an example but this is the sort of thing that you can use the dashboards for it is very good for identifying potential areas where you need to pay more attention and that is it for me i will now pass it back to Reinella great thank you Andrew so as you can see there's a lot that you know goes into our platform and toolkit development but also the payout is quite high there's a lot of amazing features that we have in our platform as well as the toolkit um and so to get to these final tools now I'll talk a little bit about you know the upcoming activities and maybe how you can help as well so as mentioned in our phase two of our o our program overview we have focus groups coming up with 30 SPO clients and 30 SPO staff as well as surveys with 180 clients as well to pilot our questionnaire so for these uh different activities you know we value your perspective and we want your voices to be heard as well and so we would absolutely love for your participation and giving us any contact information uh to help out with either focus group participation or uh you know if you are an SPO that serves newcomers uh in terms of piloting our survey that you know your any clients that you might recommend as well and so we're very open to your participation even just your feedback um or any way that you would like to get involved as well and so the way to kind of do that is um in our next slide you'll see that there's actually a Slido QR code that you can scan with your phone uh camera and you can also go on slido.com and input those numbers um on the website itself to answer this poll but basically the first two questions of our slidoh are just your feedback on our presentation today as well as our impressed tools in general and then the question three is where you'll have a chance to input your contact information or your service provider organizations contact information to participate in you know the focus groups as well as the pilot surveys if you'd like or just kind of get involved in whichever way you see fit um and so yeah we'll just take a couple minutes to basically give you guys the opportunity uh to fill that out um and of course the contact information is fully optional um definitely uh take a couple minutes and let us know um if you have any trouble with the access of it as well and then we'll move on to our question and answer next so hold all your questions for now awesome so a couple participants are still typing we'll give a little bit of more time perfect just about 30 more seconds and then uh we can bring back this uh QR code and um kind of numbered code as well at the end for um anyone to fill out the their remaining questions as well great thank you for putting it in the chat awesome so let's move on um to yeah the question and answer round now for uh your opportunity to engage with us amazing uh so there is a question um Andrew I'm wondering if you'd like to take this one the language question yes so are questions available in a variety of languages what CLB level are the questions is there any preparation or context for clients who may not have experience with writing scales i like this question a lot because we have a good answer for it um so the questions will be available in multiple languages we have English and French of course um but we are planning to translate into six additional languages um and we can maybe put the current list in the chat if anyone's interested um so it will be available in those different languages and they should participants should be able to respond in all those different languages even it's the same survey um we don't know the CLB level we tried to even for the English ones uh the original English and French versions we tried to keep it as simple as possible um but unfortunately weren't able to get like an official rating that said we do have um additional features to help participants who for whom maybe the languages included don't fit any language they speak or they just aren't familiar with the questionnaire stuff like that so um first off we will have some pre-made instructions some pre-made uh guides that would just be like a page you put in the beginning of your survey um to explain how the questions work just like say it's one to five scales uh the basics of that um we also have um low language versions of a lot of our key measures so for service quality and for the outcomes that we're uh helping you measure we have versions that are written in much simpler English with fewer questions um and with images to accompany them um and often simplified uh you know response answers as well sometimes with images as well um so that even if clients don't uh fully understand the language they can still understand what we are trying to ask and communicate uh themselves great thank you Andrew uh and I think there's another question on um are local immigration partnerships eligible and I think Nohi maybe you can start with this one and then I'll add on um so yeah of course local immigration partnerships will be eligible because they are RCC funded and they have uh like the rural uh centers around them and everything so uh there'll be eligible definitely for to access the tools uh and the tools all the tools will be available online for download uh so everyone can uh use them the platform though that's the the part where only IRCC funded programs will be uh will be using only uh if that answers the question thank you Noa um I I will take the next one which is um is it available for download and so we are going to be drafting the tools very soon and it will be available for download in 2027 hopefully and then the platform soon after so it is coming up although there's a little bit of a wait but uh what actually is available for download is our previous uh Wesso toolkit uh and so no talked about this earlier that's what impress was expanded and built upon basically and that is still available for download even though we aren't directly uh you know active in that project but you know that's downloadable and if you need any help with that uh Excel as well we can actually help you um to you know navigate the uh toolkit itself um if you reach out to us through the impress email right in our you on your screen and so that Weso toolkit is available currently and you could just go look up Weso YMCA um and it should come up and then basically it's a downloadable kind of Google drive excel um kind of sheet folder and then you can work off that directly for your own uh service provider organization and so uh the impress tools will be coming up soon though and it is free for all IRCC funded um funded organizations um answering that question of if there is a if there is a cost uh and there's also a question does this framework allow for qualitative data storytelling um and I'll let Andrew take this one yeah so when it comes to qualitative data um we will have a few options uh because a lot of the the questions I showed you are like you know they're quantitative they're continuous um these are previously validated scales but they are you know giving numbers um however at the end of the questionnaire we do have a variety of open-ended questions available as well um so clients will be able to you know write whatever they like um say whatever they like um and express that in an informal way uh without having to put it into a number or anything like that um yeah so we have quite a few options for those open-ended questions um you should probably be able to uh make your own as well um on the platform if there's other things you want to ask um but that would be uh the main feature at this point yeah yes I completely agree uh and just to answer kind of the next couple questions um we've already touched on this a little bit but please feel free to um put in another question um if there's anything more specific you'd like answered but uh the tools will hopefully be completed in 2027 uh in terms of the toolkit and then the platform soon after and uh in terms of uh the cost um it is free for IRCC funded organizations across Canada um lastly there's a question that says do you consider survey fatigue when designing the questions and guidance for SPOS we often hear from clients that they're tired of answering surveys from SPOS and that's why they don't engage um I can I'll let uh maybe Andrew start with this and then I'm sure Noha and I have lots to add as well okay yeah so it's important to emphasize that the questionnaires that we are including in this platform they're really a question bank so it'll be up to you to choose what you want to include um in your survey um so you will be able to decide based on your client's capabilities how much time they have um their language level um and just your experience with the effort they're willing to put in um how much you want to try to stuff in the questionnaire versus keeping it simple keeping it short focusing on what's most important to you um you'll also be able to you know have different versions of the questionnaire so if you have some clients uh that you need to ask different questions uh you can just focus on whatever is most relevant to them so you're not going to be forced to do one big questionnaire um with everything in it one other thing I would highlight is um a feature we're working on is some pre-made very short questionnaires just uh with like you know three or four questions in them uh specifically designed for like immediately after an event or before an event um or before and after if you want to track change over time um just to quickly get the impression of how the client experienced the event and what impact that had um so yes you can customize it to be as long or as short as you want you can base it on your client needs and we also have some features that are just pre-made short short questionnaires that yeah I completely agree with um what Andrew said just to piggyback on that as well uh you know in terms of the consideration factor is that's why we you know have made it such a wide range of being able to customize it the way you see fit in terms of types of questions what they're uh you know asking and how many that that you would like answered and so our question bank is really just a reference and it is tailored to your settlement sector for example um but you know That's just giving an idea of certain examples or kind of other ideas that you can use if you would like um and we understand that each SPO is going to have different requirements depending on the size depending on where you are etc and so you know that's why that customization piece is so important to allow for that tailoring per SPO as well um and yeah I think there's another question um if an organization has several IRCC programs is it possible to create a questionnaire capturing the IRCC outcomes that span multiple programs and get individual program data um Noa maybe you can uh start with this but I think Andrew will will have something to add after so ideally every program will be able to uh collect individual program data for their pro specific program and for the specific outcomes that they achieve uh I'm not it has several great question capturing the RCC outcomes uh so ideally I would say that each program will uh will have their own data collection because it makes more sense this way if you want to know what the outcomes that happened for your employment related services or your uh newcomer information uh center or your NARS so because they have different outcomes and so on so ideally every program will be able to customize the question bank for their particular needs and specific questions and lounge it for the clients um if you want to lounge several like if if the client has taken more than one uh service and you would like to launch two for example surveys at a time you can definitely do that but it can be confusing to the client so uh I think just service by service would be the the most straightforward way to do it and um also we'll will will make the questionnaire uh shorter which will decrease the fatigue that uh someone that there was previous question about so yeah you you you'll be allowed you can do program by program individual programs or if you would like to do it like for several programs at a time you can do that as well yeah so we have um question banks designed for different service areas so we have employment question banks community connection question banks language assessment language learning um needs assessment uh information orientation um so you will be able to say if you just want to assess an individual program it's just uh in one area say it's an employment program you could make a survey um a questionnaire that just has um those employment outcomes and maybe you also want um service quality we have some items specialized for service quality in employment as well but there really isn't anything stopping you from mixing and matching so if you have some clients that have taken u a wide variety of services you can put all of the services uh the questions for all of the services they've taken in the same questionnaire and that is an option as well perfect thank you Andrew uh I'm going to give maybe a couple like a minute uh for any further questions to be put into the question and answer box um but otherwise I can we can hand it off to uh Camila at the end and uh she can help us wrap up but I'll give it a minute before that perfect uh it seems as though there are no other questions so I'll hand it off to Camila to wrap up our presentation today but uh thank you all for being here and for uh attending our presentation camila go ahead thank you thank you all for your questions and uh for your participation today we hope this session has been valuable and informative if you have further questions please reach out to us at impress at ymcotava.ca thank you very much