Assessment

The programme of assessment is broadly composed of two elements:

1. Workplace Based Assessments (WPBAs) plus other evidence collected trainees to reflect their learning and evidence their progress

2. Panel-Based Judgements made by a group of senior clinicians who have worked closely with trainees

WPBAs provide a structure for observing the individualised and contextualised application of learning. By providing feedback and encouraging reflection it also helps trainees develop self-regulated learning skills.

The WPBA programme is used throughout training and offers the opportunity for pertinent developmental feedback and the highlighting of concerns at regular intervals when there is a chance to define plans to support learning.

WPBAs are largely formative and trainees choose how to use them to guide their development and to evidence their progress against the ACCS LOs. There is no minimum number required for any of the ACCS placements or the programme as a whole, rather it is up to trainees to work with their supervisors to ensure that they develop a sufficient body of evidence overall.

The assessment blueprint shows how WPBAs fit into the overall programme of assessment; please see the relevant section below.

Acute Care Assessment Tool (ACAT)

The ACAT is designed to assess and facilitate feedback on a doctor’s performance during their practice on the acute medical take, and is used in AM. Any doctor who has been responsible for the supervision of the acute medical take can be the assessor for an ACAT. This tool can also be used to assess other situations where a trainee is interacting with a number of different patients (eg in a day hospital or a business ward round)

Case-based Discussion (CbD)

The CbD assesses the performance of a trainee in their management of a patient to provide an indication of competence in areas such as clinical reasoning, decision-making and application of medical knowledge in relation to patient care. It also serves as a method to document conversations about, and presentations of, cases by trainees. The CbD should focus on a written record (such as written case notes, out-patient letter, discharge summary).

Mini-Clinical Evaluation Exercise (mini-CEX)

This tool evaluates a clinical encounter with a patient to provide an indication of competence in skills essential for good clinical care such as history taking, examination and clinical reasoning. The trainee receives immediate feedback to aid learning. The mini-CEX can be used at any time and in any setting when there is a trainee and patient interaction and an assessor is available.

Direct Observation of Procedural Skills (DOPS)

A DOPS is an assessment tool designed to evaluate the performance of a trainee in undertaking a practical procedure, against a structured checklist. The trainee receives immediate feedback to identify strengths and areas for development.

Multi-Source Feedback (MSF)

This tool is a method of assessing skills such as communication, leadership, team working, reliability etc, across the domains of Good Medical Practice. This provides systematic collection and feedback of performance data on a trainee, derived from a number of colleagues. ‘Raters’ are individuals with whom the trainee works, and includes doctors, administrative staff, and other allied professionals. The trainee will not see the individual responses by raters. Feedback is given to the trainee by the educational supervisor.

Patient Survey (PS)

The PS addresses issues, including the behaviour of the doctor and effectiveness of the consultation, which are important to patients. It is intended to assess the trainee’s performance in areas such as interpersonal skills, communication skills and professionalism by concentrating solely on their performance during one consultation.

Quality Improvement Project Assessment Tool (QIPAT)

The QIPAT is designed to assess a trainee's competence in completing a quality improvement project. The QIPAT can be based on review of quality improvement project documentation or on a presentation of the quality improvement project at a meeting. If possible the trainee should be assessed on the same quality improvement project by more than one assessor.

Teaching Observation (TO)

The TO form is designed to provide structured, formative feedback to trainees on their competence at teaching. The TO can be based on any instance of formalised teaching by the trainee which has been observed by the assessor. The process should be trainee-led (identifying appropriate teaching sessions and assessors).

These are regular, information-rich judgements which are made by a group of senior clinicians who have worked closely with a trainee that are used to regulate the trainee’s progression and remediation (where necessary).

A minimum of one judgement per ACCS placement is required, however regular training faculty meetings throughout the placements are recommended.

Faculty Educational Governance Statements (FEGS) - EM Placement

The group of senior clinicians who have worked closely with the trainee form a faculty who meet regularly to collate information about their performance in the workplace and assess their progress. Together they provide summative recommendations about whether the trainee is progressing adequately and has met the required standard in the ACCS LOs for that placement. These form the basis for the entrustment level decisions at the end of the EM placement.

The purpose is not only to make an assessment of a trainee but to plan on-going training. The FEGS should be documented and communicated to the trainee via their Clinical supervisor who is responsible for initiating the FEGS meetings.

Multiple Trainer Report (MTR) - Anaesthetic Placement

The MTR is the equivalent of the FEGS but carried out during the Anaesthetic placement. It can be performed, like the FEGS, as a collective discussion, or as independent feedback from a selected group of trainers who have worked with the trainee. The MTR captures the views of senior clinical staff, based on observation of the trainee’s performance in practice, providing similar valuable insight into how well they are performing, highlighting areas of excellence and areas where support is required. Like the FEGS, it is used to make entrustment decisions.

Multiple Consultant Report (MCR) - ICM and IM Placements

The MCR is the equivalent of the MTR but with the assessors consisting solely of consultants and is carried out during the ICM and IM placements.

Holistic Assessment of Learning Outcome (HALO)

These are used at the end of the Anaesthetics and ICM placements to assess LO7 Sedation and LO8 ICM. A satisfactorily completed HALO provides evidence that a trainee has achieved the required key capabilities for this LO. For trainees whose parent specialty is Anaesthetics these HALOs count towards stage 1 Anaesthetics training and don’t need to be repeated.

Multi-Source Feedback (MSF)

The MSF provides feedback on trainees that covers areas such as communication and team working. It closely aligns to the Generic ACCS LOs. Feedback should be sought from a wide range of individuals with whom trainees work including non-clinical staff. Feedback should be discussed with trainees by their supervisors.

The minimum number of MSFs required is one per year but repeat MSFs can be undertaken where appropriate.

Reflection

Undertaking regular reflection is an important part of trainees development towards becoming self-directed professional learners. Through reflection a trainees should develop learning objectives related to the situation discussed which should be subsequently incorporated into their PDP. Reflections are also useful to develop ‘self-knowledge’ to help trainees deal with challenging situations.

It is important to reflect on situations that went well in addition to those that did not go so well and trainees should be encouraged to reflect on their learning opportunities and not just clinical events.

The concept of entrustment underpins the assessment process and is the means by which trainers assess whether a trainee has reached the appropriate level to cross thresholds and take on new responsibility with a higher degree of independence.

Decisions about a trainee’s overall progression is based on an assessment of how they are performing in each of their ACCS LOs with an entrustment decision made for each Clinical ACCS LO about what level of supervision they require using the entrustment scale below. The clinical WPBAs also use the same rating scale.

1 - Direct supervisor observation/involvement, able to provide immediate direction or assistance

2a - Supervisor on the ‘shop-floor’ (eg ED, theatres, AMU, ICU), monitoring at regular intervals

2b - Supervisor within hospital for queries, able to provide prompt direction or assistance and trainee knows reliably when to ask for help

3 - Supervisor ‘on call’ from home for queries, able to provide directions via phone and able to attend the bedside if required to provide direct supervision

4 - Would be able to manage with no supervisor involvement (all trainees practice with a consultant taking overall clinical responsibility)

The assessment blueprint below illustrates the possible methods of assessment for each ACCS learning outcome. It is not expected that every method will be used for each one and additional evidence may also be included.

Each of the four ACCS placements has a focus on one or more specific Clinical ACCS LOs with minimum entrustment requirements for each that that need to be achieved by the end of the placement.). Some LOs relate specifically to one placement (e.g. LO7 for anaesthesia). Most however are addressed during multiple ACCS placements. Descriptions of the overall requirements for each placement can be found in the Placement Guidance section here: https://www.accs.ac.uk/training-guidance-1.

The entrustment tables below detail the level that must be reached for each Clinical ACCS LO by the end of each ACCS placement (end of ACCS for LO5) in order for a standard outcome to be achieved at the Annual Review of Competence Progression (ARCP). These entrustment levels are designed to be independently achievable for each placement to which they relate and so the order of trainees’ placements in their overall ACCS programme is not important in this respect.

Further details regarding Clinical ACCS LO5 (procedural skills) can be found in the Assessment of ACCS Practical Skills section: https://www.accs.ac.uk/2021-curriculum/accs-los.

Until the end of placement report is developed on the LLp, you can find a MS Word form version of the report template here: https://www.rcoa.ac.uk/media/25701.

There are a number of procedural skills in which trainees must become proficient to the level expected by the end of ACCS. Trainees must be able to outline the indications for these procedures and recognise the importance of valid consent, aseptic technique, safe use of analgesia and local anaesthetics, minimisation of patient discomfort, and requesting for help when appropriate. For all practical procedures, the trainee must be able to recognise complications and respond appropriately if they arise, including calling for help from colleagues in other specialties when necessary.

ACCS trainees should ideally receive training in procedural skills in a clinical skills lab before performing these procedures clinically, but this is not mandatory. Assessment of procedural skills will be made using the direct observation of procedural skills (DOPS) tool.

When an ACCS trainee has been signed off as being able to perform a procedure independently, they are not required to have any further assessment (DOPS) of that procedure, unless they or their educational supervisor think that this is required (in line with standard professional conduct). This also applies to procedures that have been signed off during other training programmes. They would be expected to continue to record activity in their logbook.

Further details regarding Clinical ACCS LO5 (procedural skills) can be found in the Assessment of ACCS Practical Skills section: https://www.accs.ac.uk/2021-curriculum/accs-los..

The ARCP is the formal process where trainees’ progress is reviewed, usually on an annual basis. This process is used to collate and systematically review evidence about an ACCS trainee’s performance and progress in a holistic way and make decisions about their achievement of expected outcomes and subsequent progression in training.

Evidence

Throughout training, ACCS trainees should engage with the learning process by using their e-portfolio to demonstrate that they are meeting the requirements of the curriculum. The evidence collected on the e-portfolio includes:

• placements in programme

• personal development plans

• logbook data

• evidence of supervisory meetings

• workplace based assessments

• MSFs

• Faculty Governance Statements (EM)

• MCRs (IM, ICM)

• MTR (Anaesthetics)

• HALO (Anaesthetics)

• evidence of reflection

• evidence of interaction with the Programme of Learning

Educational Supervisors Report (ESR)

The evidence provided by a trainee and held within their e-portfolio forms the basis of the ES annual report. The ESR is reviewed by the ARCP panel which has access to all the relevant source material. The panel provides oversight and ensures a nationally consistent approach and standard when making the final summative decision about progression. A satisfactory outcome at the ARCP is required in order to progress through the training programme.

Information flow in the ACCS Programme of Assessment

1) The Training Faculty will deliver a recommendation on each of the clinical ACCS LOs that are relevant to the trainee’s stage of training, i.e. whether they met the required standard for entrustment. This is summarised within a FEG or MCR/MTR and the end of placement Clinical Supervisor Report.

2) The educational supervisor reviews the evidence collated for each of the generic ACCS LOs and offers a judgement on progress in these.

3) The educational supervisor also reviews the FEG, MTR MCRs, WPBAs, multi-source feedback and other relevant data, such as caseload, critical incidents, reflections, and considers and offers insight on flags of concern. This allows for an integrated and individualised collation of diverse evidence.

ACCS ARCP requirements guide

https://www.rcoa.ac.uk/sites/default/files/documents/2021-04/ACCS%20ARCP%202021-2%20Decision%20Aid.pdf

The ARCP requirements guide summarises the evidence that ACCS doctors in training of all parent specialties must provide for ARCP and the standards expected in order to achieve satisfactory ARCP outcome.