Your Comprehensive
Multi-Source Feedback Guide
Multi-source feedback is often perceived as complex, time-consuming, and even confusing. It’s easy to feel overwhelmed when sifting through the abundance of journals, blog articles, and resources available online. Our team has recognised this and worked with industry experts to compile a single resource with all there is to know about MSF.
The comprehensive multi-source feedback guide is the go-to resource for MSF.
This comprehensive guide from Client Focused Evaluation Program (CFEP) Surveys supports clinicians and organisations across the health and care continuum that are considering the benefits of, have adopted, or are in some way participating in multi-source feedback (MSF).
Who is this guide for?
The guide helps clinicians and organisations develop a deeper understanding of how MSF aligns with the Medical Board of Australia’s Professional Performance Framework.1 It offers a breakdown of how MSF can be used to inform professional development at the clinician level, and then more broadly at the organisational level, to drive continuous, data-driven quality improvement to ensure clinicians are working at the top of their scope and together contributing to the ongoing journey towards high-performing health care nationally.
Clinicians and their colleagues
Clinicians participating in MSF (known as ‘MSF candidates’), as well as their clinical colleagues and non-clinical co-workers who may serve as reviewers, will benefit from greater awareness of:
- MSF and its three assessment components (patient and colleague feedback, and
self-assessment) - the survey instruments used (Interpersonal Skills Questionnaire, Colleague Feedback Evaluation Tool, and Self-Assessment Tool), and
- the system and processes of MSF, including data collection, analysis, reporting, debriefing, and the vital reflective period followed by action planning.
MSF for Industry Stakeholders
Healthcare industry stakeholders can use this guide when developing organisation or sector-wide MSF implementation to drive personal development and quality improvement while creating a culture of collaboration, communication, trust and excellence among clinical cohorts.
Some of the stakeholders include:
- Medical colleges
- Continuing professional development (CPD) homes
- Professional and peak body organisations
- Primary Health Networks (PHNs)
- Professional training organisations,and
- Health service provider organisations.
Fast Facts
- 360-degree view of professional performance.
- Ideal for use across a doctor’s career span.
- Wide-reaching benefits across the entire health system.
- Contributes significantly to clinicians’ continuing professional development.
- Contains 3 parts: patient feedback, colleague feedback, self-assessment.
- Participant types: clinician, patients and colleagues (reviewers); debriefer (a coach or supervisor who provides a formal debrief); the supporting medical colleague (SMC).
- Process spans 4 stages: data collection, analysis and reporting, debrief and self-reflection, action planning and CPD allocation.
- Organisations offering MSF as a professional development program need a robust implementation plan.
- MSF should be revisited annually in line with the Medical Board of Australia Professional Performance Framework.
What is multi-source feedback?
Multi-source feedback or MSF is an evidence-based professional development tool that utilise a combination of colleague feedback, patient feedback, and/or a self-assessment to generate insights into the participants professionalism and practice. This provides an opportunity for evidence-based refection to identify strengths and improvement areas.
Clinicians use MSF to collect objective, specific feedback from their patients and clinical and non-clinical colleagues to gain a 360-degree perspective of their performance. Multi-source feedback is sometimes called 360 feedback.
Benefits of MSF
Multi-source feedback has wide-reaching benefits for clinicians, patients, healthcare organisations, medical colleges, CPD homes, and the entire health system.
Clinicians using MSF are exposed to a true professional development exercise to develop interpersonal and professional skills which are also recognised for annual CPD requirements.
Patients benefit from MSF with a greater voice and stronger partnership in the health care, offering greater trust in their providers and a chance to directly improve patient care.
The advantages for healthcare organisations, medical colleges and CPD homes are great. Offering MSF to members and clients provides an evidence-based tool that supports clinical cohorts by boosting their professional performance and reputation while encouraging them to continually seek improvement in their own service delivery.
Feedback and self-assessment
A comprehensive multi-source feedback commonly consists of three parts; colleague feedback, patient feedback, and a self-assessment.
Patient feedback allows patients to give feedback on the humanistic aspects of their care. It’s behaviourally based allowing clinicians to reflect on and refine their interpersonal skills while working with patients.
The Colleague feedback aspect works to understand how the participant is perceived by their peers. The feedback is obtained from both clinical and non-clinical co-workers and is considered a reliable indication of a clinician’s professionalism.
While all three methods are recommended for the greatest insights, MSF can be conducted in part using either patients or colleague feedback and self-assessment. Note that colleague feedback must be completed in combination with the self-assessment.
Who's involved in MSF?
Conducting multi-source feedback requires the involvement of numerous parties. This include:
- The MSF candidate
- Reviewers – including patients, clinical and non-clinical colleagues
- A Debriefer, and
- A Supporting Medical Colleague (SMC)
As the name suggests, the MSF candidate is the person undertaking the MSF, either independently or as part of their training program through a medical college, CPD home or as part of a workplace coordinated program.
The Reviewers provide objective feedback about the candidate’s observable behaviours. The Reviewers can be a clinical or non-clinical co-worker of the candidate who has worked with the MSF candidate in the past two years, or a patient who has experiences clinical care from the candidate.
An MSF Debriefer is medical educator (ME), supervisor, or another formal coaching style role. The Defriefer works with the candidate to work through the multiple data sources they receive, identify strengths and opportunities for improvement, and develop an objective and evidence informed action plan outline actions for personal development, growth and improvement.
The Supporting Medical Colleague (SMC) provides an informal debrief or support to the candidate when they’re conducting their formal debrief to help reflect on strengths and opportunities for improvement.
The Multi-Source Feedback Process
Multi-source feedback is conducted in four stages:
Stage 1: Objective and meaningful data collection comprising patient feedback, colleague feedback and self-assessment
Stage 2: Analysis and reporting resulting feedback of the assessment date and comparators as appropriate
Stage 3: Debrief and self-reflection to identify learning, opportunities and priorities for professional develop, growth and improvement.
Stage 4: Action planning and CPD allocation which translates results into action.
Each stage of the multi-source feedback process is important in maintaining the integrity of feedback collection, data, and reflection. Without strict adherence to the different stages can significantly impact the reliability of the data resulting misleading results from the MSF process.
The stages of MSF are discussed in more detail in ‘The four stages of Multi-Source Feedback’ insights article.
Implementing multi-source feedback
Introducing MSF requires organisational commitment and resourcing. Developing a robust implementation plan with a feasible timeline supported by an effective engagement and communication plan is essential for success5,8.
The process demands both clinical and non-clinical leadership for making decisions about implementing, maintaining, evaluating and monitoring the program.
The following consideration are key during the implementation of MSF:
- Create organisational readiness
- Establish organisational and program support and buy-in
- Approve or facilitate CPD recognition
- Establish the MSF program team
- Determine how the results well be used and who will have access to them
- Identify candidates and reviewers and schedule engagement
- Agree how frequently MSF will be conducted
- Determine the process for facilitated feedback, coaching, supported action planning and review
- Recruits and prepare facilitators/coaches, and
- System learning and continuous improvement
Each of the listed key considerations should be supported with a detailed implementation strategy. This will encourage action and maintain accountability of the implementation goals. Access a full list of suggested implementation strategies in our ‘Implementing Multi-Source Feedback’ article.
Professional Development Opportunities
MSF and other 360-degree type feedback is well established in industry9 and frequently used as an assessment tool to:
- develop insights into individual strengths and opportunities for improvement
- enhance cultural change
- produce summative assessment of performance
- evaluate potential (e.g. careers advice or selection)
- enhance team effectiveness
- identify training needs for the organisation.10
The MSF program combines quantitative and qualitative data, narrative comments, and dedicated debrief and coaching time with a trained facilitator or coach. These elements are then used to develop an objective and constructive action plan that informs personal development, growth and improvement.
When done in the right way for the right purpose, MSF systems have been shown to enhance teamworking, productivity, communication and trust.
Leveraging the expertise of a survey provider like CFEP Surveys in practice assessment, the MSF program aims to normalise the process of professional review and reflection as part of a supportive whole-system learning approach encompassing professional bodies, provider organisations, and health care professionals.
An MSF tool provides an annual cyclical process of reviewing performance, measuring outcomes, and self-evaluation. After completing the action plan, it’s recommended that candidates repeat the virtuous cycle of MSF to engender a culture of excellence, demonstrate leadership, and sustain their lifelong learning and improvement journey.
Evidence to Support MSF
While the content of the MSF instruments may appear to be relatively simple, there is a substantial amount of research, time, money and effort invested in producing the tools. A wide range of research has been conducted on the use of MSF internationally in various clinical settings.
A review of eight systematic reviews demonstrated the evidence base supporting the statistical and psychometric properties of MSF in sufficient to support using the validated tool in the clinical setting.16 The structural validity of the MSF tool has been tested, confirming that MSF performance in terms of response rates, time and costs.21
More recently, a critical review of MSF and its psychometrics noted that publications over the past 50 years or more in the business and health literature support MSF offer quality improvement purposes.
Specific research validating the use of the MSF in the clinical setting covers a range of topics including:
- Validity/reliability
- Feasibility
- Scoring
- Comparison
- Consequences and change
- Barriers and enablers
- Facilitated feedback
- Data
- Performance
- Quality, and
- Academic appointments
For more information about the specific research supporting each topic, download a copy of the full Comprehensive Guide to Multi-Source Feedback (MSF).
Enablers and limiting factors
Like all assessment tools, the MSF tool, has both success factors and limitations. Research into the implementation of MSF has identified these key enablers and their attributed success factors as well as the limitations. 10,38,43,51,52
MSF’s key enablers are:
- Focus on observable behaviours
- Focus on core profession roles i.e. collaborator, communicator, professional
- Use of a Likert scale with neutral midpoint and ‘unable to assess’ options
- Invite free-text comments
- Self-selection of reviewers
- Reporting data and providing feedback
- Understanding the importance of Self-Assessment
- Using a facilitated feedback conversation and coaching techniques to guide self-reflection and action planning
- Creating an action plan for professional development, growth and improvement
- MSF is a formative/quality improvement assessment tool
For more about the success factors of each enabler, check out our ‘Enablers and limitations of Multi-Source Feedback’.
Key limitations of multi-source feedback:
- Culture
- Formative assessment
- Number of respondents
- Realising change
- Resource requirements, and
- Stimuli are daily, real events that are random and different for every observer.
The MSF limitations are explained in more details in our article LINK.
Continuous Quality Improvement (CQI)
The annual cycle of MSF for clinicians is a four-stage process. These stages are founded on the philosophy and principles of continuous quality improvement (CQI).
At its simplest, CQI is a philosophy that encourages all health care providers to continuously ask questions about how they’re performing, can they improve, what do they need to make these improvements, and how will they know if they’ve improved?86,87
CQI generates and reinforces a culture of excellence in which individual health care professionals and service providers continuously strive to better tomorrow than what they were today. The key to creating a culture of excellence is using a structured review and planning approach that provides feedback on the current state, identifies the need for change activities to drive improvement and desired outcomes over time, and provides opportunity to meet that need.
Health care professionals can use the MSF tool to assess their performance (outcomes achieved) and develop a plan for personal development, growth and improvement (CPD and CQI).
The MSF process is based on the principles of CQI for health care professionals (Figure 12).
<INSERT DIAGRAM FROM PAGE 35>
Continuing Professional Development
MSF works towards the Quadruple Aim of health care2 and value-based health care.3 It helps drive continuous, data-driven quality improvement so clinicians can work at the top of their scope and help achieve high-performing health care nationally.
MSF is an important tool in the professional development and quality improvement ‘toolkit’. A combination of data about interpersonal skills and professionalism (communicator, collaborator and professional roles), and data assessing the clinical expert role (application of clinical knowledge and skills and professional values) provides a more comprehensive and holistic view of an individual clinician’s performance.
Common sources of performance data for the clinical expert role91 include:
- clinical record audits
- prescribing patterns
- patient reported measures (PRMs)92 including patient reported outcome measures (PROMs) such as the Patient Activation Measure® or PAM® 19,93 and patient reported experience measures (PREMs)94
- service-related process measures
- assessment of clinical procedures and standards.95
MSF is considered a formative assessment tool which provides assessment data to individual health care professionals and a more rounded picture of opportunities for learning, personal growth and improvement. It is not suitable for summative assessment purposes (i.e. to provide data to organisations for high-stakes decisions about a clinician’s performance).
Administered annually, the MSF tool gives an ongoing point-of-comparison where the MSF candidate’s professional development can be measured year after year.
Optimally, results of the MSF tool would be considered alongside other performance data to guide professional development and CQI. Combining MSF results with other assessments of clinical expertise can give health care professionals a more extensive and authentic overview of their performance, a rich data source for identifying their practice strengths and areas where they could do better, and inform plans for improvement.
Table of Contents
Looking for more detail on any of the topics discuss above?