Comprehensive Guide to
Multi-Source Feedback
Multi-Source Feedback
Overview
What is Multi-Source Feedback and why is it important?
Multi-source feedback, sometimes called 360 feedback, allows individual doctors to obtain feedback from peers, patients and industry associates to provide unique and varied views. These evaluations support doctors in gaining more significant insights into their all-around performance.
MSF systems are deeply established in industry, with 360-degree type feedback routinely used a workplace assessment tool to:
- develop insights into individual strengths and opportunities for improvement
- enhance cultural change
- produce summative assessment of performance
- evaluate potential (e.g.g careers advice or selection)
- enhance team effectiveness, and
- identify training needs for the organisation.
Original work on the development of MSF tools for physicians was undertaken in the United States4,5 and later developed over several years by the College of Physicians and Surgeons of Alberta’s (CPSA) Physician Achievement Review (PAR) program6, and the Medical College of Canada.7,8
MSF is increasingly adopted within continuing professional development and regulatory frameworks worldwide as a method to assess medical performance and quality-assure clinical practice.9 The validity evidence for MSF used within medicine is well established, and MSF is now used in health systems across Canada, Australia, the Netherlands, New Zealand, the United Kingdom, and the United States, with interest growing in other countries.
The MSF program combines quantitative and qualitative data, narrative comments, and dedicated debrief and coaching time with a trained facilitator or coach. These elements are then used to develop an objective and constructive action plan that informs personal development, growth and improvement.
Leveraging CFEP Surveys’ expertise in practice assessment, the MSF program aims to normalise the process of professional review and reflection as part of a supportive whole-system learning approach encompassing professional bodies, provider organisations, and health care professionals.
CFEP Surveys’ MSF tool provides an annual cyclical process of reviewing performance, measuring outcomes, and self-evaluation. After completing the action plan, CFEP Surveys recommends candidates repeat the virtuous cycle of MSF to engender a culture of excellence, demonstrate leadership, and sustain their lifelong learning and improvement journey.
Key roles of a clinician
Communicator
Fill with information from table on page 6 once approved
Collaborator
Fill with information from table on page 6 once approved
Professional
Fill with information from table on page 6 once approved
The multi-source feedback process
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
At its simplest, CQI is a philosophy that encourages all health care providers to continuously ask:16,17
- How am I/we doing?
- Can I/we do better?
- What do I/we need to do to do better?
- How will I/we know I’ve/we’ve improved?
CQI generates and reinforces a culture of excellence in which individual health care professionals and service providers continuously strive to be better tomorrow than what they were today. The key to creating a culture of excellence is using a structured review and planning approach that provides feedback on the current state, identifies the need for change activities to drive improvement and desired outcomes over time, and provides opportunity to meet that need.
Health care professionals can use the MSF tool to assess their performance (outcomes achieved) and develop a plan for personal development, growth and improvement (CPD and CQI).
While data from the MSF tool can identify for the clinician some improvement opportunities in their performance and practice, making the change is often more difficult. It can be especially challenging if the change is beyond the clinician’s reach and requires a broader clinical or system change.
The MSF tool helps to capture and share these challenges to inform learning at the micro system level (the individual clinician and team level – the deliverers of care), the meso system level (service or organisation, training program etc.) or the macro system level (the health sector or local health system).10 When used optimally, the MSF tool underpins a whole-system learning approach to driving patient safety and quality improvement.
Who is involved in the feedback process?
THIS CONTENT IS FROM THE DRAFT GUIDE Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
MSF Candidate
The MSF candidate is the health care professional undertaking the MSF assessment. They may personally decide to undertake MSF, or their participating organisation, professional or peak body or training organisation may nominate them as part of its professional or organisational formative review mechanisms.
The MSF candidate contributes to the MSF process by nominating reviewers (i.e. patient, clinical colleagues or non-clinical co-workers) and completing the Self-Assessment instrument.
Reviewer
The reviewer plays a critical role in providing objective feedback about an MSF candidate’s observable behaviours.
Reviewers include those people who have direct relationships with, and have typically been nominated by, the MSF candidate, such as:
- Patients who have experienced clinical care from the MSF candidate. The patient reviewer provides objective feedback via the Interpersonal Skills Questionnaire instrument.
- Clinical colleagues and non-clinical co-workers who work closely with the MSF candidate. The colleague reviewer provides feedback via the Colleague Feedback Evaluation Tool.
Debriefer
The debriefer has an instrumental role in holding a coaching conversation with the MSF candidate about the MSF feedback. They help the candidate to consider the multiple sources of data provided, identify strengths and opportunities for improvement, and develop an objective and evidence-informed action plan outlining actions for personal development, growth and improvement.
Supporting medical colleague (SMC)
A trusted clinical colleague provides an informal debrief or support to the candidate when they’re conducting their formal debrief to help them reflect on strengths and opportunities for improvement, and when the candidate is planning for change.
Who can use multi-source feedback?
The MSF tool is available to any health care professional operating in any sector within the Australian healthcare system, with variations available for some specialties and their medical colleges. Further, a non-clinical MSF tool is available from CFEP Surveys, to support organisations with a whole-of-practice approach to quality improvement engaging all members of the practice or care delivery team.
Four user types can apply the MSF tool:
- individual health care professionals who wish to adopt an objective approach to personal development, growth and improvement
- health care organisations that have incorporated formative MSF into their healthcare professional or organisational development programs
- medical colleges that have incorporated formative MSF into their integrated professional development programs encompassing clinical education, and professional and career development. As such, they would set and sustain professional standards, and embed the concept of lifelong learning, while promoting objective and evidence-informed professional development, growth and improvement
- CPD homes and registered training organisations that have adopted an integrated approach to clinical education, professional and career development, and similarly incorporated formative MSF into their training programs. Their approach would support health professionals, including clinical trainees, and strengthen and embed evidence-informed personal development, growth and improvement into ongoing professional practice.
What are the benefits of MSF?
MSF is a powerful tool for generating objective, specific feedback about professionalism and practice and for offering an opportunity for evidence-based refection that identifies strengths and areas for improvement. MSF works towards the Quadruple Aim of healthcare21 and value-based healthcare.22
Beneficiaries and their benefits
As shown above, the benefits of the MSF tool are multi-faceted, reflecting the voice, opinion and experience of patients, clinical colleagues, non-clinical co-workers, and the clinician (MSF candidate). These benefits also extend to the broader health system.
Health care professionals
- MSF is a true professional development exercise.
- MSF supports the clinician in developing professionally across their career, particularly in areas of interpersonal skills and their general professionalism as a clinician.
- It’s a great pulse check since they navigate a changeable health landscape (e.g. pandemic response and changes to practice, and systemic change towards person-centred care).
- MSF makes a significant contribution to CPD points required by
a range of colleges. - g with a peer (the supporting medical colleague) creates value in the clinician’s direct professional relationship, and strengthens their professional network and personal support network.
- MSF supports medical colleagues to develop deeper trust in the clinician and, in turn, strengthens their professional standing and reputation.
Patients
- MSF amplifies the voice of the consumer, recognising and valuing the patient as a true partner in care.
- Patient experience is acknowledged. Health consumers feel heard and valued because they play a role in supporting the clinician’s professional development.
- Changes in practice resulting from patient feedback improve the experience of current and future patients, because clinicians are more insightful, better informed and more responsive.
- Trust increases. Demonstrated changes in the clinician’s interpersonal skills may improve patient engagement and activation as a partner in their own health care journey.
Organisations
- Since it’s a validated tool, MSF supports clinical cohorts.
- The MSF tool (supported by significant research) adds rigour to the cohort’s professional performance and reputation.
- Evidence of quality improvement in micro, meso and macro systems and processes (when providing or recommending the use of the MSF tool) sets an example for clinical cohorts to continually seek improvement in their own service delivery.
- When approving CPD for MSF-related activity, the organisation can be confident it is using a validated, ‘fit-for-purpose’ MSF tool and associated reflective exercises and peer support elements. The MSF tool is a full-service model that is good for clinicians and the supporting medical colleagues who in turn learn and reflect on their own professional practice.
Health System
- Candidates are better informed and more likely to work at the top of their scope, which supports superior service delivery across the system.
- Collegiate relationships and networks are strengthened, creating a more robust system, where more information is shared and fewer risks exist (clinicians are more likely to consult a colleague as a result of the trust developed during the MSF process).
- MSF supports the person-centred approach to health care, and is in line with the national move towards person-centred, integrated, value-based care.
- MSF is a standard and validated tool that reduces variability of feedback and increases the value of the reflective and peer support processes – quality measures, delivered in a supportive standardised way.
- MSF can lead to cost savings from improved clinical service delivery (a more engaged, more agile clinical cohort, more responsive to patient need).
MSF for professional development and quality improvement
MSF is an important tool in the professional development and quality improvement ‘toolkit’. A combination of data about interpersonal skills and professionalism (communicator, collaborator and professional roles), and data assessing the clinical expert role (application of clinical knowledge and skills and professional values) provides a more comprehensive and holistic view of an individual clinician’s performance.
Common sources of performance data for the clinical expert role23 include:
- clinical record audits
- prescribing patterns
- patient reported measures (PRMs)24 including patient reported outcome measures (PROMs) such as the Patient Activation Measure® or PAM®12,25 and patient reported experience measures (PREMs)26
- service-related process measures
- assessment of clinical procedures and standards.27
MSF is considered a formative assessment tool which provides assessment data to individual health care professionals and a more rounded picture of opportunities for learning, personal growth and improvement. It is not suitable for summative assessment purposes (i.e. to provide data to organisations for high-stakes decisions about a clinician’s performance). Administered annually, the MSF tool gives an ongoing point-of-comparison where the MSF candidate’s professional development can be measured year after year.
Optimally, results of the MSF tool would be considered alongside other performance data to guide professional development and CQI. Combining MSF results with other assessments of clinical expertise can give health care professionals a more extensive and authentic overview of their performance, a rich data source for identifying their practice strengths and areas where they could do better, and inform plans for improvement.
Alignment with the Medical Board of Australia’s Professional Performance Framework
MSF is a formative workplace assessment tool28,29 for informing personal development, growth and improvement, and so it complements the Medical Board of Australia’s (MBA) Professional Performance Framework.
The Professional Performance Framework supports doctors to practice competently and ethically throughout their career.20 The framework creates a compact between patients and health care professionals that embraces clear expectations of professionalism, and safe and quality care.
This compact safeguards the trust patients have in their doctors. The framework clearly aligns with the three roles assessed with the MSF tool (communicator, collaborator and professional), and with the MSF instruments, which assess interpersonal skills and the patient experience of care, colleague perceptions of professionalism in practice, and self-assessment.
Five pillars of the Professional Performance Framework
Strengthened continuing professional development
All doctors to have a CPD home
•CPD to be relevant to scope of practice
•CPD to be based on personal professional development plans
•50 hours CPD per year, a mix of:
−performance review
−outcome measurement, and
−educational activities.
•CPD home to report to the Board where medical practitioners have not completed their CPD program requirements.
Active assurance of safe practice
Board to identify risks to patient safety and define the principles for screening those at risk
•Increasing age is a known risk factor:
−peer review and health checks for doctors who provide clinical care aged 70 and three yearly after that
−Board will not receive the results of peer review and health screening unless there is a serious risk to patients.
•Professional isolation is a known risk factor:
−education on how to identify and manage this risk
−increasing peer-based CPD for professionally isolated practitioners.
Strengthened assessment and management if practitioner with multiple substantiated complaints
Board to strengthen its assessment and management of practitioners with multiple substantiated complaints
•Board to require practitioners with multiple substantiated complaints to participate in formal peer review.
Guidance to support practitioners
Board to continue to develop and publish clear,relevant and contemporary professional standards including:
−revise Good medical practice: A code of conduct for doctors in Australia
−refine existing and develop new registration standards
−issue other guidance as required.
Collaborations to foster a positive culture
Promote a culture of medicine that is focused on patient safety
•Work in partnership with the profession to reshape the culture of medicine and build a culture of respect
•Encourage doctors to:
−commit to reflective practice and lifelong learning
−take care of their own health and well being
−support their colleagues.
•Work with relevant agencies to promote individual practitioners accessing their data to support practice review and measuring outcomes.
CPD Requirements
The Professional Performance Framework is integrated into the MBA and colleges’ pathways for
medical registration (for both national and international medical graduates) and the ongoing professional development of health care professionals.
Complementary to the Professional Performance Framework is the introduction of a revised CPD registration standard30,31 and health checks for late-career doctors, to support ongoing safe practice.
CPD requirements, effective from January 2023, are:30,31
All doctors are required to nominate a CPD home (an organisation that validates the award of CPD points in line with Professional Performance Framework requirements).
CPD activities must comprise:
educational activities (at least 12.5 hrs)
performance review and outcomes measurement-related activities
(at least 25 hours and a minimum of 5 hours each)
the remaining 12.5 hours across any of the above.
MSF meets two of the important components of the Professional Performance Framework and
associated CPD requirements:
reviewing performance (colleague feedback)
measuring outcomes (patient feedback).
All medical colleges are recognised CPD homes, and independent CPD homes are established to support those clinicians not registered with a college. CPD homes don’t necessarily develop or deliver education and professional development content, although many do. They promote opportunity for professional development, growth and improvement – for example, by adopting, promoting and enabling use of MSF to ensure an objective and evidence-based approach.
CFEP Surveys is a registered CPD home. The MSF tool is a validated tool that not only reviews the clinician’s performance and facilitates self-analysis and self-reflection but also measures outcomes and tracks these over time through annual review rounds. CPD recognition is awarded to MSF candidates on completion and submission of self-refection records.
MSF and CPD
The practice of using qualitative and quantitative data, together with constructive feedback, to guide learning, personal development, growth and improvement is integral to all healthcare professional development programs.
Typically, national professional standards and regulatory frameworks, alongside service standards,32,33 recognise the role that personal performance and practice assessment play in ongoing improvement. The MSF tool provides objective performance data using external sources of data and multiple feedback mechanisms together with self-assessment to inform the development of a personalised action plan.
This plan is data-informed and follows a reflective period where MSF candidates trial and embed changes to clinical practice and service delivery.
The MSF tool fits readily into the annual CDP cycle for all health care professionals, providing objective data they can use when creating and implementing an evidence-informed learning plan and evaluating performance and outcomes.
CFEP Surveys’ MSF tool is consistent with the MBA’s new Continuing Professional Development (CPD) registration standard, effective January 2023.20 This standard requires clinicians to develop and
complete a personalised annual professional development action plan, emphasising performance review (i.e. MSF – colleague feedback component) and measuring outcomes (i.e. MSF – patient feedback component), and self-assessment of CPD activities to inform planning for the following year.
The Australian Health Practitioner Regulation Authority (AHPRA), many of Australia’s medical colleges, and healthcare professional and peak body organisations, have approved the MSF tool, and made available a range of CPD allocations. Please contact CFEP Surveys or your medical college to ask about access to MSF and current CPD allocations for your specialty when completing MSF or components of it (i.e. patient feedback and/or colleague feedback and self-assessment and associated reflection and action planning).
Section 2: The Four Stages of Multi-Source Feedback
- Stage 1 – Objective and meaningful data collection
- Stage 2 – Analysis and reporting
- Stage 3 – Debrief and self-reflection
- Stage 4 – Action planning and CPD allocation
Section 3: How to Implement Multi-Source Feedback
Introducing MSF and applying the MSF tool requires organisational commitment and resourcing. Developing a robust implementation plan with a feasible timeline supported by an effective engagement and communication plan is essential for success.
Chapter 3 clearly outlines the key considerations when introducing an MSF program and provides suggestions for implementation strategies.
Section 4: Multi-Source Feedback Evidence Base
While the content of an MSF tool may appear to be relatively simple, there is a substantial amount of research, time, money and effort invested in producing these ‘fit-for-purpose’ tools. A wide range of research has been conducted on the use of MSF in various clinical settings that support the effectiveness and impact MSF has on quality improvement processes.
Section 5: Enablers and Success Factors of MSF
Lorem ipsum
Section 6: Limitations of Multi-Source Feedback
Chapter 1 summaries Multi-source feedback covering:
Appendices
Appendix A: Glossary