How to evaluate your remote consultation service

In this third part of the CSP's guidance, we look at how to carry out meaningful evaluation when implementing remote consultations or a hybrid model in your practice

Getting started and quick wins

The Covid-19 pandemic necessitated a rapid growth in the implementation of remote physiotherapy delivery options. The maintenance of these options as we move forward requires evidence from evaluation. This is to ensure that the benefits are maximised, risks are reduced and/or mitigated with the outcome and experience remaining positive.

Local evaluations, which are often small-scale, provide information within the context of local service provision, care pathways, and population needs.

When carrying out evaluations:

Decide which aspects of your remote physiotherapy delivery you want to evaluate
Think about the purpose, for example, to inform change in service design, or to inform a business case
Look at previous evaluations for ideas of appropriate methods and measures such as the CSP's innovations database
Consider what information you will need to answer your evaluation question. Are you already collecting this information?
Consider the demographics of your patient population and the accessibility of your service for all patients
Only start collecting more information if it is needed to answer your evaluation question

Evaluation is recommended in the following areas:

Equity

Work collaboratively with patients and the public to evaluate equity of access and participation.
It is of paramount importance to collect data that demonstrates which patients are successfully participating in remote consultation and those that are not. Shared decision making should determine how physiotherapy is delivered, taking into account a person’s needs, communication needs and available resources. Data should be collected to evaluate the suitability of the pathway to existing and potential patients needs (so reflecting the needs of your population).
User-centred design can help to develop and deliver services that people like using, and systems and devices that suit their needs.
No individual should be automatically excluded from digital services because of any presenting condition, medical history, literacy, background, age or other demographic characteristic. It is important that data is collected to evidence this and identify steps that need to be taken to optimise equity of access to services.

Digital exclusion

Data should be collected focused on which patient populations are and are not accessing services with different consultation approaches to identify strategies to improve equity of access to services. To avoid increasing health inequalities remember the importance of collecting data on patients for whom remote physiotherapy delivery is unsuitable, such as patients with poor digital literacy or no access to suitable devices.

Patient reported experience/satisfaction measures (PREMs)

Evaluation is required to understand satisfaction and experience of remote services, in-person services and hybrid services. The measures your service currently uses may be adequate to answer this question. Patient experience questionnaires may need adapting to ask about the type of remote delivery (e.g. telephone, video, email). Ensure data can be collected from patients who do not respond well to remote physiotherapy delivery as well as those who have good outcomes and satisfaction.

Patient reported outcome measures (PROMs)

Evaluation of patient outcomes comparing different consultation approaches in different patient populations.

Staff experience / satisfaction

Impact on staff should form a vital part of evaluation of remote consultations taking in to account their previous experience and preferences, among other factors.
Impact of new working models on staff wellbeing should be evaluated and monitored.

Cost implications

There is no good quality evidence or data comparing the cost of remote and in person physiotherapy despite the perception by some that remote delivery reduces costs. Further evaluation is required to establish accurate costings of virtual and in-person consultations (including start-up costs) to help evaluate economic impact of a hybrid model of service delivery.
As well as the costs of equipment, consider collecting data on how many digital contacts fail either due to technological problems or no-shows/lack of response from the patient? How much non-contact time does each practitioner have? How many patients are managed entirely remotely and how many convert to in-person contact?

Time pressures

Data should be gathered to understand the time cost and any potential savings from different service delivery models and the impact that this has on efficiency of service delivery. Time savings may not be directly experienced by the service that invests the time (e.g. MSK input prevents GP appointment, community input prevents hospital admission) so a review across systems should be undertaken to capture this.

Culture

Leadership should be evaluated, taking into account: proactive, skilled management of change, team dynamics, flexibility and confidence with technology.

Full article content is only available to CSP members. If you are a member you will need to log in.

In this section