FOI23/24 059 Information Regarding 2023 Appeals

Date published: 07/09/2023

I am writing to submit a formal request for information under the Freedom of Information (Scotland) Act 2002. I kindly request that you provide me with the following information regarding 2023 appeals:

1. Regarding "Checking in line with the national standards": Could you kindly provide a comprehensive explanation of the phrase "Checking in line with the national standards" as it pertains to the appeals section? Specifically, I am interested in understanding how the senior examiner ensures that a candidate's script has been accurately marked in accordance with the established national standards, thus minimizing any potential errors in the marking process.

2. Elaboration on the senior examiner's assessment process: I am seeking clarity on how the senior examiner assesses the accuracy of the marking process. Could you elucidate the specific methodology employed by the senior examiner to verify the alignment of the marking with the national standards and the stipulated marking scheme?

3. Distinguishing between clerical checks and marking reviews: Could you provide clarification on the distinction between clerical checks and more comprehensive marking reviews within the appeals process? Specifically, I am interested in understanding whether the appeals process involves both types of assessments and under what circumstances each approach is employed. Could you kindly elucidate whether the appeals process in 2023 primarily entails a clerical assessment or a comprehensive marking review?

4. Scope of script evaluation by a senior examiner: I would appreciate information regarding the extent to which a senior examiner reviews a candidate's script during the appeals process. Is the entire script thoroughly reviewed, or does the review primarily focus on the specific questions or sections that are under appeal?

5. Mitigating bias in the original marking process: Could you elaborate on the strategies and mechanisms in place to ensure impartiality and mitigate bias during the original marking process of all candidates' scripts? Understanding these procedures would provide insights into the overall integrity of the marking process.

6. Addressing unreadable scripts: In the event of an unreadable script, how does the SQA handle the assignment of marks? I would appreciate clarification on the procedure followed when the content of a response is challenging to decipher due to handwriting or other factors.

7. Handling incorrect question numbers: When a candidate has inadvertently provided an incorrect question number in their response, could you explain how markers evaluate such cases? Specifically, how is the content and context of the response taken into consideration to ensure a fair assessment?

8. Ensuring marking accuracy without access to scripts: Given that candidates are unable to request a remark or obtain copies of their scripts, how does the SQA provide assurance that the marking process has been error-free and adheres to the established standards?

9. Impact on scores during appeals: I am seeking clarification on the circumstances under which a candidate's score might change during the appeals process. Could you confirm whether scores would only be altered in cases where there are substantial errors or significant discrepancies between the original marking and the national standards?

10. Request for detailed score breakdown: Is it possible for candidates to request a comprehensive breakdown of their scores, detailing the allocation of marks for individual questions on the examination paper?

11. Examiner access to candidate details: During the original marking process, are examiners privy to candidates' personal details, or is a system in place to ensure anonymity and impartiality throughout the assessment procedure?

12. Potential Impact on Candidate Scores and Mark Reduction in the 2023 Appeals Process: Under which circumstances could a candidate's assessment outcome experience a reduction during the appeals process? Is it conceivable that, in the context of the appeals process for the year 2023, a candidate's overall score might undergo a decrease of 1 or 2 marks?

13. Inquiry Regarding the Appeals Process in 2023: Clarifications on Assessment Review, Score Changes, and Error: If the appeals process for the year 2023 does not involve an extensive re-evaluation of the marking, could you kindly elucidate the recourse available to a candidate who perceives a potential error in the assessment of their script? Moreover, am I correct in understanding that a candidate's score would only undergo modification if a substantial discrepancy of more than 1 or 2 marks is identified? This notion raises a query regarding its applicability across all cases, as it appears perplexing how a candidate's score might decrease, unless it involves a considerable misjudgment or if their work was indisputably overmarked.

14. Assessment of Appeals Involving Coursework: Understanding the SQA's Procedures for Subject-Specific Appeals and Ensuring Alignment with Standards: Could you kindly elucidate the steps taken by the SQA when an appeal involves a subject that includes coursework components? Specifically, how does the appeals process account for the assessment of coursework and ensure fairness in the evaluation?

15. For subjects such as English, where coursework plays a substantial role, does the senior examiner overseeing an appeal thoroughly review the coursework segment of a candidate's work? Could you provide insights into the extent of scrutiny applied to the coursework component during the appeals process?

16. Ensuring that a candidate's coursework and folio align with standards is crucial for maintaining fairness and consistency. Could you elaborate on the measures implemented by the SQA to verify that a candidate's coursework and folio submissions are in line with the established guidelines and national standards?

1. Regarding "Checking in line with the national standards": Could you kindly provide a comprehensive explanation of the phrase "Checking in line with the national standards" as it pertains to the appeals section? Specifically, I am interested in understanding how the senior examiner ensures that a candidate's script has been accurately marked in accordance with the established national standards, thus minimizing any potential errors in the marking process.

For each appeal request, senior appointees (such as the Principal Assessor and senior exam team members) review all the candidate’s externally marked components – each response in each component is reviewed against the marking criteria to ensure that is has been marked appropriately by the original marker. The totalling in each component, and overall, is also checked for any errors that would affect the grade awarded. The purpose of the review is to ensure that the correct grade has been awarded overall. Any deviation from the acceptable marking standard or clerical errors, that affects the grade, are addressed and the grade can be adjusted up or down to correct this.

2. Elaboration on the senior examiner's assessment process: I am seeking clarity on how the senior examiner assesses the accuracy of the marking process. Could you elucidate the specific methodology employed by the senior examiner to verify the alignment of the marking with the national standards and the stipulated marking scheme?

The process is to look at the candidate response and the mark provided for each item alongside the finalised marking instructions and confirm accuracy in application of marks. If the marking was outside acceptable standards, then the senior appointee would review the impact of this and, using the grade boundary cut offs, determine if there had been any error in the original grade awarded to candidate

3. Distinguishing between clerical checks and marking reviews: Could you provide clarification on the distinction between clerical checks and more comprehensive marking reviews within the appeals process? Specifically, I am interested in understanding whether the appeals process involves both types of assessments and under what circumstances each approach is employed. Could you kindly elucidate whether the appeals process in 2023 primarily entails a clerical assessment or a comprehensive marking review?

The appeals process involves both types of review.  For every case senior appointees undertake a clerical check and a more comprehensive check on the application of marking standards. See response to question 1 for more detail.

4. Scope of script evaluation by a senior examiner: I would appreciate information regarding the extent to which a senior examiner reviews a candidate's script during the appeals process. Is the entire script thoroughly reviewed, or does the review primarily focus on the specific questions or sections that are under appeal?

In every case all externally assessed components are reviewed by senior appointees. Again, please see response to question 1.

5. Mitigating bias in the original marking process: Could you elaborate on the strategies and mechanisms in place to ensure impartiality and mitigate bias during the original marking process of all candidates' scripts? Understanding these procedures would provide insights into the overall integrity of the marking process

All SQA appointees, including markers and senior team members, undertake training before carrying out their SQA duties.  This training includes highlighting the risk of implicit bias.  SQA’s e-marking procedures include the suppression of each candidate’s personal data.  Scripts are allocated to markers on a random basis (not full centres).  See also the response to question 11.

6. Addressing unreadable scripts: In the event of an unreadable script, how does the SQA handle the assignment of marks? I would appreciate clarification on the procedure followed when the content of a response is challenging to decipher due to handwriting or other factors.

In instances where a digital image of a script in deemed undecipherable by the appointee the physical script is used. The physical script is reviewed by a senior appointee who determines the marks to be awarded for each question. All markers have the facility to escalate/seek the opinion of senior appointees including the Principal Assessor (SQA’s most senior appointee) for guidance.

7. Handling incorrect question numbers: When a candidate has inadvertently provided an incorrect question number in their response, could you explain how markers evaluate such cases? Specifically, how is the content and context of the response taken into consideration to ensure a fair assessment?

Markers can flag these scripts to senior appointees for guidance and review if it is unclear which questions have been responded to. Senior appointees are fully familiar with all aspects of the assessment and will be able to assess the script accordingly.

For example, in English critical reading – if a learner has written the wrong question number next to their response but the appointee can clearly see that the response relates to another question, then the response will be marked against the appropriate question.

8. Ensuring marking accuracy without access to scripts: Given that candidates are unable to request a remark or obtain copies of their scripts, how does the SQA provide assurance that the marking process has been error-free and adheres to the established standards?

For the original marking, exam scripts and other SQA assessments are marked by SQA appointees, who are practising teachers and lecturers. These markers are provided with comprehensive training before they begin marking, including a check on their ability to apply marking standards against pre-marked scripts. There are also quality assurance processes in place to ensure marking is to a consistent standard throughout the full marking period. These quality assurance measures include ongoing team leader checks and support, as well as system checks.

For the appeals process please see the response to question 1.

9. Impact on scores during appeals: I am seeking clarification on the circumstances under which a candidate’s score might change during the appeals process. Could you confirm whether scores would only be altered in cases where there are substantial errors or significant discrepancies between the original marking and the national standards?

No marks are altered during the appeals process. The only change that can be made is to the candidate’s grade, if this is the outcome having reviewed all the candidate’s externally assessed components (and taking into account any internally assessed components). There are three possible outcomes for appeals – the candidate’s grade can stay the same, go up, or go down.

10. Request for detailed score breakdown: Is it possible for candidates to request a comprehensive breakdown of their scores, detailing the allocation of marks for individual questions on the examination paper?

Candidates can submit a Subject Access Request where they are provided with the mark sheet for their exam.

11. Examiner access to candidate details: During the original marking process, are examiners privy to candidates' personal details, or is a system in place to ensure anonymity and impartiality throughout the assessment procedure?

Most of the marking is from digital image – marking from digital image is completely anonymous – markers are not provided with any learner or centre details. In addition all markers have a contractual obligation to declare centres where they have an interest eg the centre where they teach or where they know learners. A process is in place to ensure that markers cannot mark their declared exception centres. When marking from physical scripts markers do see learner and centre details. The exception process is in place which ensures that no marker marks scripts from learners known to them. In addition, a suite of quality assurance processes are undertaken which ensures that the marking from each marker is monitored to ensure that it is to the national standard – the marking from all markers is sampled and checked.  See also the response to question 8.

12. Potential Impact on Candidate Scores and Mark Reduction in the 2023 Appeals Process: Under which circumstances could a candidate's assessment outcome experience a reduction during the appeals process? Is it conceivable that, in the context of the appeals process for the year 2023, a candidate's overall score might undergo a decrease of 1 or 2 marks?

Please see the response to question 9.  It is possible that a learner’s grade could go down if the assessment components have not been marked to standard, for example a lenient application of marking standards, and the impact of this would result in an overall grade change.

13. Inquiry Regarding the Appeals Process in 2023: Clarifications on Assessment Review, Score Changes, and Error: If the appeals process for the year 2023 does not involve an extensive reevaluation of the marking, could you kindly elucidate the recourse available to a candidate who perceives a potential error in the assessment of their script? Moreover, am I correct in understanding that a candidate's score would only undergo modification if a substantial discrepancy of more than 1 or 2 marks is identified? This notion raises a query regarding its applicability across all cases, as it appears perplexing how a candidate's score might decrease, unless it involves a considerable misjudgment or if their work was indisputably overmarked.

The appeals process does involve an extensive review of marking. Again, please see the response to questions 1, 9 and 12.

14. Assessment of Appeals Involving Coursework: Understanding the SQA's Procedures for Subject-Specific Appeals and Ensuring Alignment with Standards: Could you kindly elucidate the steps taken by the SQA when an appeal involves a subject that includes coursework components? Specifically, how does the appeals process account for the assessment of coursework and ensure fairness in the evaluation?

All externally marked components, whether coursework or exam script, are reviewed as part of the appeals process. Please see responses to question 1 and 2.

15. For subjects such as English, where coursework plays a substantial role, does the senior examiner overseeing an appeal thoroughly review the coursework segment of a candidate's work? Could you provide insights into the extent of scrutiny applied to the coursework component during the appeals process?

Yes, the appointee will review the coursework component with the same rigour as the other assessment components.  Please see responses to questions 1 and 2.

16. Ensuring that a candidate's coursework and folio align with standards is crucial for maintaining fairness and consistency. Could you elaborate on the measures implemented by the SQA to verify that a candidate's coursework and folio submissions are in line with the established guidelines and national standards?

Please see responses to questions 1 and 2.  As an example, in the case of the English Folio, senior appointees will review the folio and ensure that the marker has correctly judged the appropriate marking band with reference to the marking instructions