Blog Post

Exploring the digital messaging capacity of settlement agencies – a draft framework

By: Marco Campana
October 15, 2021

OK. So, I'm not an academic, but consider myself a decent researcher. During some work a few years ago on the digital messaging practices of immigrant and refugee-serving organizations and newcomers, I came across some really interesting research that just struck me as a great model. It was in the form of a Master's degree (which makes me want to write a post about the importance of community, student, grey area, and other non-academic (read: PhD's) research, but that's for another time): ‘As Important to Me as Water’: How Refugees in Rome Use Smartphones to Improve Their Well-being (2017).

St George's research introduced me to the Capability Approach, really interesting work on technology use among refugees, some of their models/frameworks, and her own version of this, the Smartphone Evaluation Framework.

Smartphone Evaluation Framework

Building from this Smartphone Evaluation Framework theory, I proposed a similar Digital Messaging Evaluation Framework. Each section is outlined and explained below, incorporating useful components of other researchers' ideas and approaches.

In this post, I'm sharing my half-baked idea, which I honestly wish I had shared pre-COVID. It could have generated some interesting conversations. Any way, here you go. You can also access this post as a PDF.

Introduction

"The paradox is this: ICTs and particularly the internet are widely regarded as groundbreaking inventions that have changed the way millions of people live their lives, and yet researchers and practitioners in the field of ICT and development often struggle to prove specific impacts of the technology to funders." Dorothea Kleine, ICT4What ?— Using the Choice Framework To Operationalise the Capability Approach To Development

Digital messaging is simple, on the surface, but is just the tip of the iceberg when it comes to digital capacity:

Digital messaging iceberg in settlement services

The following framework comes out of this research (which I'll add to this site soon, when I have a bit more time):

  • Gigler, B.S. (2011). Informational capabilities - the missing link for the impact of ICT on development. E-transform knowledge platform working paper series(1).The World Bank.
  • Gigler, B.S. (2015). Development as freedom in a digital age: experiences from the rural poor in Bolivia. Washington DC: World Bank.
  • Kleine, D. (2010). ICT4WHAT? Using the choice framework to operationalise the capability approach to development. Journal of International Development 22(5), 674-692.
  • Diaz Andrade, A & Doolin, B. (2016). Information and Communication Technology and the Social Inclusion of Refugees. MIS Quarterly. 40. 405-416.
  • Abujarour, S & Krasnova, H & Wenninger, H & Fedorowicz, J & Olbrich, S & Tan, C & Venkatesh, V & Urquhart, C. (2016). Leveraging Technology for Refugee Integration: How Can We Help?
  • Abujarour, S & Krasnova, H. (2017a). Understanding the Role of ICTs in Promoting Social Inclusion: The Case of Syrian Refugees in Germany.
  • Abujarour, S & Krasnova, H & Díaz Andrade, A & Olbrich, S & Tan, C & Urquhart, C & Wiesche, M. (2017b). Empowering Refugees with Technology: Best Practices and Research Agenda.
  • Abujarour, S & Krasnova, H. (2018). E-Learning as a Means of Social Inclusion E-Learning as a Means of Social Inclusion: The Case of Syrian Refugees in Germany.
  • Vivienne St George, T. (2017). ‘As Important to Me as Water ’ : How Refugees in Rome Use Smartphones to Improve Their Well-being.
  • Mergel, I., & Bretschneider, S. I. (2013). A three-stage adoption process for social media use in government. Public Administration Review, 73(3), 390–400.

These researchers started with economist Amartya Sen’s Capability Approach and combined it with McClure's notion of informational capabilities to create a useful framework for the immigrant and refugee-serving sector to build and evaluate settlement agencies' capacity to use technology to serve clients.

McClure's informational capabilities concept refers to:

  1. ICT capability – ability to use ICTs in an effective manner
  2. Information literacy – ability to find, process, evaluate, and use information
  3. Communication Capability – ability to effectively communicate with clients, colleagues and
    others in their professional network
  4. Content Capability – ability to produce and share content with clients and others through the
    network.

These capabilities include the skills, knowledge and attitudes of the Settlement worker. All of these capabilities exist within an Information and Communications Ecology or environment, much of which is outside the control of the individual. This includes agency structure, policies, and even further external forces, such as funding, government policies, and includes the informational capabilities of these actors, as well as those of clients.

Informational capabilities are a person‘s capability to transform their existing informational capital, such as level of access to ICTs (their opportunity structure) into real opportunities to achieve the things they value or need to to. In our case, to serve clients effectively. In our framework, informational capabilities refer to front-line worker's freedom and ability to use ICTs within the institutional and socioeconomic setup of their agency. It is important to emphasize the significant differences between informational capital, ICT capabilities and informational capabilities.

All of this occurs within stages of adoption. It is important to acknowledge that different agencies are at different stages of capability and adoption. Usefully, a couple of researchers have created a model of three phases of adoption of technology (in their case social media) in government. The three stages, are “experimentation, constructive chaos, and institutionalization” of new technologies.

I believe this model, and how it can flow from one stage to the other, is an accurate portrayal of how digital messaging technologies are currently being adopted within Settlement agencies, and can be useful for our analysis.

“In the experimentation phase, individual innovators who have some experience with technology from other work or non-work settings begin using it in the workplace, and the use spreads from worker to worker and thus may be used in a variety of informal ways. In the constructive chaos stage, workers begin to recognize both benefits and risks of the
technology use, which may create tensions within the organization, and organizations respond with attempts to standardize the technology use through practice or policy standards. Often in this stage organizations seek answers that other similar organizations use, or draw upon their past policies related to ICT use to drive current standards. Sometimes reactionary standards are developed in response to misuse, or concerning use, of technology.

Finally, in the institutionalization stage, the agency has developed a set of standards, processes, and enforcement measures to control the use of the technology. Agencies that know about their workers' use of social media are grappling with how to move from the first two categories to the later given the perceived benefits of social media use, while also grappling with the risks and challenges related to social media use.” Mergel, I., & Bretschneider, S. I. A three-stage adoption process for social media use in government.

Additional researchers have taken these approaches and focused on the importance of acquiring informational capabilities — the ability to transform access to ICTs into real opportunities to achieve the things one values. In our case, to serve clients using technologies that they are using or have indicated a preference to use to communicate with service providers. This approach was recently applied to the use of ICT and, more specifically, smartphones among refugees. Vivienne St George, T. (2017). ‘As Important to Me as Water’: How Refugees in Rome Use Smartphones to Improve Their Well-being (2017).

The Digital Messaging Evaluation Framework

Digital messaging evaluation framework

Opportunity Structure (external to people's capabilities – the information and communications ecology)

This is the environment within which front-line workers and managers within are attempting to use digital messaging to serve clients. This structure addresses contextual factors influencing settlement agencies' smartphone use to serve clients. These are conceived of as structural influences which impact on people’s opportunity to use ICTs to expand their capabilities to serve clients.

“Technology is embedded in the wider set-up of institutions, policies, programmes, norms and discourses. As such they need to be analysed as firmly and historically engrained.” The sense of choice vs the use and achievement of choice is heavily influenced by agency capability to see potential of tech and to integrate tech within existing policy, practice, management and human resource frameworks. That in turn is heavily influenced by funder ability to recognize this potential. These are the work, agency and funder information, communications, and funding environments that front-line workers exist within.

Agencies:

  • funding
  • recognition of technology as (not) legitimate service tool/channel
  • leadership digital literacy
  • changing and diverse use of ICT by clients/newcomers
  • data literacy
  • security/encryption literacy
  • including technology in assessment - not asking clients for preferred method of communication at intake
  • changing staff numbers and complement (based on immigration patterns/trends) – how to know the right way to invest in new technology/phones?
  • Lack of funder support/literacy - funder digital literacy and ability to evaluate the request for support of technology in budget, or entire technology-mediated projects

Workers

  • overwhelmed already with day-to-day workload
  • Concerned about more work and boundaries
  • digital, data, security literacy
  • people, not tech focused
  • responsive to client tech use, not proactive – while different clients use different technologies, difficulty prioritizing, and this changes over time
  • don't have tools (including hardware), training and evaluative support to implement tech effectively
  • perception as “para professional” - no regulatory or standards frameworks in place
  • find out about client communications preferences anecdotally
  • management/funding supports vary inconsistently
  • access to training, professional development, orientation and support

Funders:

  • lack of digital literacy in the context of human services
  • pressure to audit rather than serve
  • not clear on what the sector wants in terms of use, support and funding for technology to serve clients
  • internal disconnect about utility of technology in service delivery
  • institutional pressures
  • lack of clarity about how technology is part of and can mediate service delivery rather than being an indirect service tool (eg. Computer on desk as admin tool for funded staff)

Personal assets/resources

In the second box, personal assets and resources are made up of the informational capabilities:

  • ICT capability – ability to use ICTs in an effective manner
  • Information literacy – ability to find, process, evaluate, and use information
  • Communication Capability – ability to effectively communicate with clients, colleagues and others in their professional network
  • Content Capability – ability to produce and share content with clients and others through the network.

In the case of settlement agencies, this is mainly in the form of the existence or lack of personal and agency assets and resources, or that may be hidden, informal and ad hoc. There are many assets that agencies and workers have, but many more challenges and barriers when it comes to using technology to serve clients.

Agencies and Funders:

  • various levels of tech literacy and seeing technology as a legitimate service tool
  • project-based technology integration – siloed, not all staff, not connected to mission/vision/strategic plans
  • few policies, protocols, guidelines, onboarding standards, expectations, management/supervisory clarity, etc. - managers and leaders do not feel competent or capable managing staff using technology to serve clients
  • lack hardware, IT, ICT infrastructure
  • a history of innovative service delivery with scarce resources
  • limited knowledge sharing within agencies and across the sector
  • limited resources, unlimite challenges
  • limited knowledge/understanding and models of how to manage technology for service delivery
  • limited sector/collective advocacy opportunities to funders or policy-makers about ICT use to serve clients

Workers:

  • client centric
    • commitment to clients and their settlement/integration
    • regardless of barriers, they serve clients using technology anyway, sometimes using their own personal devices and accounts
  • varying levels of tech literacy, capacity, interest
  • strong community connections but lack UX/user design knowledge for technology integration – want to be more flexible in using the tech that their clients use
  • limited influence on service delivery methods and innovation in spite of experience
  • limited ways to “pitch” ideas within agencies and to funders
  • some have access to the technology they need/want, some don't – inconsistent implementation across the organization
  • limited exposure to methods/tools/practices of how to use technology to serve clients

Levers for change (institutional processes, policies and programs)

The third box, in the centre of the diagram, refers to the kinds of interventions or support that may be required from government to support agency technology integration, management and use, and agencies to support worker technology use.

Within the levers for change there should be an analysis and recognition of the different stages of technology adoption and where a particular agency is. This will guide what they need to do and supports required to move ultimately to institutionalization of of digital messaging (and other technologies) in service delivery.

Agencies, funders and workers:

  • skills development
  • leadership support
  • space and resources to experiment
  • becoming knowledgeable and competent in user-centred design
  • policy development
  • continuous learning/standardized professional development
  • sharing of “good ideas”/Community of Practice
  • acknowledgement of digital services as legitimate service, not tech, not indirect
  • protocols, guidelines and standards
  • privacy, security, confidentiality standards
  • digital service standards – incorporating best of from other sectors (but also from what is emerging within the sector at the agency and funder levels)
  • hardware access and upgrading (Total Cost of Ownership beyond computers)
  • seeing technology/digital messaging as another service channel that requires the competencies and resources to master it
  • iteration and innovation – the sector can and should have both – both require funder/program innovation
  • Managing in a Computerized Environment 3 (MICE3) – building on what the sector has already told the funder it needs in order to effectively use technology to serve clients (i.e. Their informational capabilities)
  • collaboration and partnerships with the private sector, including civic tech
  • evaluation frameworks – client outcome oriented, not just technology focused – how technology has contributed to outcomes, access and efficiencies
  • simplified processes to pitch new technology use in service delivery
  • agencies move “ask” to funder, funder open to “ask” - better dialogue between sector and funders
  • revisioning the role of the settlement worker – a blended digital/in-person role

Informational/Service capabilities (Affordances) enabled by digital messaging (and technology more broadly)

In the fourth box, we focus on the informational capabilities most appropriate for the settlement sector.

Affordances are neatly organized by the informational capabilities model:

  • ICT capability – ability to use ICTs in an effective manner
  • Information literacy – ability to find, process, evaluate, and use information
  • Communication Capability – ability to effectively communicate with clients, colleagues and others in their professional network
  • Content Capability – ability to produce and share content with clients and others through the network.

(Technology affordance analysis is about unpacking the different attributes of a technology so you can determine its suitability for a particular goal, such as serving newcomers.)

Kleine's “degrees of empowerment” idea is also useful here. Kleine writes: "Individuals were aware of some possibilities the new technology offered them, like email and online chat, but not of others, like Voice over IP. This was precisely because their educational resources (including computer skills) and the dominant discourse… stressed some usages over others. For any piece of research focused on a technology which is new to the respondents, the dimension of 'sense of choice' will play a significant role. The 'use of choice' dimension refers to whether or not an individual actually makes the choice and the 'achievement of choice' refers to whether the outcome matches the choice expressed." For workers and agencies, capabilities are impacted by the actual existence of choice, a worker's or
agency's sense of choice, use of choice and achievement of choice. This is an important dimension to explore within settlement agency context.

The sense of choice vs the use and achievement of choice is heavily influenced by agency capability to see potential of tech and to integrate tech within existing policy, practice, management and human resource frameworks:

Access to possibilities, or the sense of choice, is essential.

In the sector:

  • Combine messaging with other technology, such as Google Translate, mapping, etc. (polymedia)
  • increased technology and information literacy – expansion of sense of what is possible with technology – starting to move from iteration to innovation
  • worker use with client satisfaction provides use case for leadership
  • potential for content sharing, replication and re-use within and across agency
  • more direct learning with clients – example of Chechens showing use of Google Translate as a useful tool to COSTI CSS workers, then adopted by workers within their service framework
  • increased awareness of potential by also limitations of technology to serve clients
  • increased appreciation and capacity for security and encryption
  • beginning to more actively map client journey and where technology intersects and has a role
  • recognizing that front-line workers have the information, knowledge and experience to push upwards to create change and systems movement
  • possibilities for increases in:
    • ICT capability among all settlement actors
    • information and service and referral capability
    • communications capabilities
    • accessibility for clients
    • appreciation for technology possibilities in service delivery (thin wedge)
    • recognition that technology doesn't mean a reduction in human resources
    • recognition of a need to create space for pitches, innovation, policies and guidelines

Outcomes

Overall, we're measuring increases in:

  • informational/ICT capabilities
  • service capabilities
  • integration capabilities with clients

over periods of time and where the agency is at in their digital journey.

The outcomes for workers, agencies, funders are ultimately related to better outcomes for clients. They can be broken into immediate, medium and long-term outcomes, or into the stages of technology adoption: Intrepreneurship and Experimentation, Constructive Chaos, Institutionalization.

Immediate (Intrepreneurship and Experimentation):

  • increased access for and to clients
  • less travel
  • small entry to technology can be leveraged for bigger use later
  • meet clients in the technology channels and platforms they use
  • clients communication preferences addressed
  • increased potential for service scale
  • formalize informal technology use in agencies, legitimize and recognize as service delivery – funder sees as innovation, really just iteration
  • increased appreciation for ethics, standards, technology security and need for encryption

Medium (Constructive Chaos):

  • scale of service – groups, for broadcast or back and forth – is easily created
  • better appreciation of technology as a direct service tool
  • increased technology and information literacy among workers and leadership
  • iteration in service delivery
  • actual increases in scale of service
  • polymedia increases
  • client journey mapping becomes more actively applied
  • Nonprofit Service Canvas or similar pitch approach implemented across sector
  • replication of successful models and approaches using technology to serve clients
  • awareness of potential, but also limitations and downsides, of technology use to serve clients – leading to creation of models, standards, protocols and strategic approaches to integration technology into service delivery

Long-term (Institutionalization):

  • easily replicable models, standards, protocols and approaches within the sector
  • innovation in service delivery
  • enhanced service delivery across channels
  • consistency in digital capacity and offerings to clients
  • adoption of models, standards, protocols and strategic approaches – rolled out across the sector with ongoing training, professional development and capacity-building models agencies and workers can access
  • increase of reaching clients previously unreached or not accessing services
  • professional development includes technology
  • knowledge and content sharing, replication and re-use across agencies bringing efficiencies to digital services
  • Community of Practice develops and is successful – broader than technology, but also encompasses digital capacity

There are likely a number of capacity frameworks out there that can be researched to tease out the specific and actual skill development needed by workers, management and funders. One that has much promise is Nethope's Center for the Digital Nonprofit. Their Digital Nonprofit Skills (DNS)™ framework and Digital Nonprofit Ability (DNA)™ framework expand on the technology adoption for nonprofits, rounding out the specific capabilities agencies and workers need in the digital era.

The the Digital Nonprofit Skills™ (DNS) Assessment which establishes a solid baseline for skills across roles and across organizations. The Digital Nonprofit Ability™ (DNA) Assessment looks at a nonprofit's digital transformation journey to assess their digital readiness.

Leave a Reply

Your email address will not be published.

arrow-circle-upmagnifier