A Comparative Study of Formative and Summative Assessment Pattern of Clinical Skills for Undergraduate Students in College of Medicine, Dar Al Uloom University, Riyadh View PDF

Assessment of students is an important aspect of learning in medical education and hence deserves attention of medical academicians. Assessment is a fundamental component of both learning and teaching as it frames what students learn and their certification [1].

Formative assessment (FA) is a part of the developmental or ongoing teaching-learning process by providing guidance and continuous feedback to the students on their performance. The feedback given in formative assessments (FAs) informs learners of their present state of learning and provides opportunity to modify learning during the learning process [2].

In contrast, summative assessments (SA) is done at the end of a term or course and are predominantly utilized for grading and certification at the end of a period of study, often without providing feedback to students on their performance [3].

In a study by Scoles PV, et al. (2003) titled “Assessment of clinical skills in medical practice”, he stated that a variety of indirect and direct measures are available for evaluating physicians, but, at present, no single method permits high-stake inferences about clinical skills. Much work is needed to identify the optimal combination of methods to be employed in support of programs to ensure maintenance of competence of practicing [4].

In the MBBS curriculum in Dar Al Uloom University, Riyadh KSA, the assessment is MCQs, & OSPEs/OSCEs based. The evaluation of clinical skills per se has seen a change from being only summative (End of block OSCE) for senior batches, to introducing only formative assessment for years 2 and 3. Year 4 being the transition batch experienced both summative and formative patterns of assessment of their clinical skills.

The aim of this study was to determine the students view about the influence of formative assessment on summative assessment and to come up with the best method of evaluation to achieve the higher benefit from the clinical skills.

An online, self-created, questionnaire through monkey survey was distributed to all the medical undergraduate students in Dar Al Uloom University (from year 2 up till year 5). This questionnaire was validated by pilot study. It was seen that the selected cohort bear ICC r=0.693 which was nearly 70% co-variance among the 9 studied variables and was considered statistically a feasible correlation for sustaining this research.

Study Design

A cross sectional study was conducted in the College of Medicine, Dar Ul Uloom University, Riyadh, Saudi Arabia. The questionnaire was distributed among 230 medical students from year 2 to year 5, both male and female. A well-designed and tested questionnaire of 10 questions was used based on Likert scale and sent through monkey survey to students online.

Inclusion criteria

All students from year 2, 3, 4 and 5 who faced formative and summative assessments

Exclusion criteria

Students from University Preparation Program, students who had withdrawn from courses, students who failed to appear in any kind of assessment.

Objective

• To collect and analyze the feedback of students on formative assessment in terms of its advantages and disadvantages.
• To collect and analyze the feedback of students on summative assessment in terms of its advantages and disadvantages.
• To analyze the comparative feedback of students from different batches on assessment pattern.
• To propose appropriate assessment pattern for the clinical skill assessment of students of College Of Medicine, Dar Al Uloom University.

Ethical considerations

• Approval was obtained from the Research Ethics Committee at Dar Al Uloom University.
• Participant’s anonymity or publishing their data was assured.
• Institutional review board (IRB) approval was obtained for this study prior to study execution.

Statistical Analysis

Intergroup comparison (between group comparisons) was done by student’s t-test and the overall ANOVA by F-test. Moreover, intra class correlation was measured by Chronbach's alpha. Data will be analyzed using the SPSS 22.0 software to generate the results required to formulate an appropriate plan to achieve the most benefit of the clinical skills sessions for undergraduate students.

The questionnaire was sent to 230 medical students from year 2 to year 5 including both male and female, 168 out of 230 students participated in the study and answered the questionnaire. The returned questions were analyzed. 25 (14.9 %) students were from year 2, 28 (16.7 %) students from year 3, 66 (39.3 %) students from year 4 and 49 (29.2 %) students from year 5. Distribution of participating students for each level in academic year 2017/2018 is given in Figure 1.

Students who were assessed by only formative assessment were 53 (31.6%) and belonged to year 2 and 3. Combined method using both formative & summative assessment method were 66 (39.3 %) from year 4 and assessment by summative method alone were 49 (29.2%) participants from year 5. Distribution of assessment among students by academic year is given in Figure 2.

Regarding their performance in the clinical skills at the end of assessment, 94 (56%) students were satisfied while, 46 (27.4 %) of all student were not satisfied with their performance and 28 (16.7%) were neutral in their satisfaction level.

On the other hand, regarding the way of assessment in the clinical skills, 114 (67.9%) students were satisfied while as 50 (29.8 %) students were not satisfied and 4 (2.4%) were neutral in their satisfaction level. Distribution of performance and the way of assessment among students by the academic year is given in Figure 3.

Out of the students who were not satisfied, 22 (44%) were only formatively assessed (continuous assessment with feedback) and 28 (56%) were only summatively assessed (final OSCE).

When asked regarding whether they remember the skill better by the end of semester if they were assessed only by formative assessment, most of the students 101(60.1%) were neutral in their satisfaction level, while 40 (23.8%) students failed to remember them and only 27 (16.1%) remembered their skills well.

When asked regarding whether they remember the skill better by the end of semester if they were assessed only by summative assessment , most of the students 68 (40.5%) were neutral in their satisfaction level while 51 (30.4%) students failed to remember and 49 (29.1 %) students were satisfied and remembered the skills well . Figure 4 shows feedback about remembering the skills by formative or summative assessments separately.

Regarding memorizing the steps of skill in a short duration in formative assessment, 98 (58.3%) students were neutral to feel pressure, while 54 (32.1%) felt pressure to memorize and 16 (9.5%) students were fine to memorize the skill with no pressure.

While as for the summative assessment, memorizing and preparing all the skills together, we found most of the students 112 (66.6%) felt pressure in memorizing all skills while 49 (29.2%) students were neutral and only 7 (4.2%) students were fine to memorize the skill with no pressure. Figure 5 show if the student will felt pressure in memorizing and delivering the skill during the assessment.

In our study we asked the students whether they felt their performance of the clinical skills will improve if we conduct summative assessment at the end of block in addition to formative, most of the students agreed 127(75.6%), 37 (22%) were neutral and 4 (2.4%) disagree

For summative assessment we asked the students whether providing a feedback at the end of assessment would help them improve, most of the students agreed 122 (72.6%), 34 (20.2%) were neutral and 12 (7.1%) disagree. Figure 6 shows improvement the performance of their skills.

Assessment is vital to the education process. The most visible assessments are summative and formative. However, the summative measures what students have learnt and creates accountability for student performance through testing, but assessment also serves a vital formative function through frequent, interactive assessments of student understanding, to identify learning needs and adjust teaching to meet the diversity of the modern classroom. Over the past two decades, an emphasis on formative assessment has emerged in many parts of the world [1] (Table 1).

Table 1:

This study included the students of 2nd to 5th year, among the participants (31.6%) participants were year 2& 3, assessed by formative assessment method in the clinical skills. (39.3%) students were from year 4 and were assessed by combined method using formative & summative assessment method. 29.2% were 5th year students who were assessed by summative assessment method only.

When participants were asked about their performance in the clinical skills at the end of assessment, 56% students were satisfied but considerable number i.e. 27.4% of all student were not satisfied with their performance and 16.7% were neutral in their satisfaction level. This considerable level of dissatisfaction among students related to their clinical skill performance indicated the need of re-evaluation of teaching & assessment methods (Table 2).

Table 2: Satisfaction level on performance in clinical skill- at the end of assessment.

Similarly, in a study titled , “The Assessment of Undergraduate Medical Students’ Satisfaction Levels With the Objective Structured Clinical Examination” by Ahmad Khosravi Khorashad ,found that the majority of the students (94.5%) had a positive attitude toward the OSCE . They mentioned that the OSCE format was a more appropriate type of exam than other methods of testing; however, 79.1% thought that the OSCE format was stressful [5].

In a study by Badyal DK, et al. (2010), Student evaluation of teaching and assessment methods in pharmacology, it was concluded that the implementation of suggestions obtained from students resulted in improvement in their performance [6]. Hence, it is very essential to synchronize teaching and evaluation methods with special requirements of medical students (Table 3).

Table 3: Satisfaction of clinical skills assessment? (Overall).

When asked directly about the satisfaction level related to the assessment method used for evaluation of clinical skills 67.9 % students were satisfied with the way their clinical skills were assessed but a significant percentage 29.8 % were not satisfied (Table 4).

Table 4: Reason of Non satisfaction according to method of assessment used.

When investigated further to know the cause of dissatisfaction, it was found that out of unsatisfied students 56% students feel that it is because they were assessed only by summative method and 44 % feel that only using formative assessment pattern is the cause of their unsatisfactory performance. As the percentage was high, therefore, this response needed further analysis by considering individual method of assessment separately.

Summative Assessment - Feedback analysis results

More than 70% students sense that they cannot remember the clinical skills better if they are assessed only by summative method (Table 5).

Table 5: Remember skill better if assessed only by summative method?

In a research paper titled “Summative and Formative Assessment in Medicine: The Experience of an Anesthesia Trainee” by Sinead M O’Shaughnessy and Pauline Joyce. Summative assessment is regarded as not being ‘student centric’ and is often viewed as embodying all the negative social aspects of assessment.

Summative assessment only allows students to proceed appropriately within the education system [7,8].

In some medical institutions Summative method of assessments carries 80% of credits and formative are given less and the limitations of summative assessments were prominent. Like in an article by Ghosh SK (2014) “Value of assessment of medical students” in The Clinical Teacher, the author mentioned that the majority of students tend to neglect the importance of day-to-day learning in the medical course, and instead concentrate their efforts for the last 2-3 months before the main exams [9]. The continuous assessment tests are mostly carried out at intervals of 1 month; however, it is not mandatory for the students to appear in each of them. It is observed that even if students fail to appear in one, two or even more of these monthly assessment tests, ultimately, they can achieve the cut-off required to appear in the main exam. Among these students, those who are street-smart and intelligent (with some luck of course!) come out with flying colors in the finals. Because of the overwhelming credit awarded to the outcome of one exam (at the end of the course), the performance sheets of these students reflect very high grades, even though they may not have ventured into certain important aspects of the particular subject at all, which were assessed in the monthly tests that they had missed (Table 6).

Table 6: Summative assessment method cause pressure of preparing all the skills together.

When asked about the reason for low performance with only summative assessment 66.66% feel that it is because of pressure of preparing for all skills together.

In an Online article by Reddy K (2016) on wise step website titled “Disadvantages of Summative Evaluation” it is mentioned that the main disadvantages of summative evaluation are that since it focuses on output at the end, in case there are hindrances or difficulties, the learning process at the end can be tough [10]. There is no chance to recover as the results are at the end. This is not an accurate reflection when learning is considered.

Since it is being a single test at the end of the complete session of academics, it makes almost all individuals anxious and disruptive. They face the summative evaluation with nervousness and fear.

Repeated practice test for low-achieving students lowers their self-confidence and self-esteem. The summative evaluation results have a negative effect on low achievers when they are more pronounced for students than for schools or authorities. It is also considered as a limiting process for more able individuals. Anxiety is another reason which is caused in a big bang test especially amongst girls and leads to expanding the gap between higher and low achieving individuals.

Formative Assessment - Feedback analysis results

When investigated further by considering the individual method of assessment more than 80% student sense that they cannot remember the clinical skills better if they are assessed by formative method only.

To evaluate the perception and performance of medical students subjected to formative assessments during integrative clinical skills and asked about the retention of learned skills when they are assessed by formative method, 23.8 % disagreed that they remember better if assessed by only formative method and a large number of students were in dilemma that only formative assessment will be better. Still 16.1 students agreed to that (Table 7).

Table 7: Remember skill better if assessed only by summative method.

Though various research literature elaborated the advantages of formative assessment over summative assessment, like in a study titled “Perception of medical students about formative assessments during clinical courses’ by Labarca et al. (2014) the students outlined that this assessment oriented their study, allowed them to discover their weaknesses and have a perception of the degree of difficulty of the final exam [11]. Over 90% of students that took the formative evaluation improved their academic achievement.

But some researches indicate that formative assessment alone is not beneficial. This finding was supported by a study by N EAlkhateeb, et al. (2019) “Effect of a Formative (OSCE) Objective Structured Clinical Examination on the Clinical Performance of Undergraduate Medical Students in a Summative Examination: A Randomized Controlled Trial” concluded that Single formative-OSCE does not necessarily lead to better performance in subsequent summative-OSCE. This points the need of tailored or combined method of Assessment [12] (Table 8).

Table 8: (a) Do you feel a lot of pressure during formative assessment of memorizing the steps of skill in a short duration?; (b) Do you feel that the pressure of delivering the skill in front of your colleagues in formative assessment affects your learning and performance?

While investigating to find the cause of difficulty in remembering the clinical skill better, more than 30% students mentioned that they feel a lot of pressure during formative assessment of memorizing the steps of skill in a short duration. 57.2 % feel that the pressure of delivering the skill in front of their colleagues in formative assessment affects their learning and performance.

Looking for the drawbacks of formative assessment of clinical skills in the literature some related references, could be found e.g.; In an Online article by Reddy K (2016) titled “Formative Evaluation: Importance, Advantages &Disadvantages”, it is stated that formative evaluation is considered to be a time-consuming process, if they are followed on a monthly, weekly or daily basis as they need frequent gathering of data, analysis, reporting [10]. It is also very important to note that, in order to process with the formative evaluation, well qualified and trained individuals are required so that formative evaluation is carried over successfully till end.

In more recent research, Torrance H (2012) points out the impasse in which formative assessment now finds itself [13]. Despite theoretical development and justification over many years, the practice “is often limited in terms of its scope and its utilization of the full range of possible approaches associated with formative assessment”. It tends “to involve fairly mechanistic forms of activity”.

Further according to Torrance, the emerging problem is that formative assessment is being reduced to regular classroom tests, which are used for monitoring student’s progress, or to a series of techniques for coaching to improve grade and test results [14].

The clinical evaluation, as one of the most important elements in medical education, must measure students' competencies and abilities. The implementation of any assessment tool is basically dependent on the acceptance of students. In a study Hoseini BL, et al. (2013) tried to assess midwifery students' satisfaction with Direct Observation of Procedural Skills (DOPS) and current clinical evaluation methods; they concluded that DOPS method is associated with greater students' satisfaction [15]. Since the students' satisfaction with the current methods was also acceptable, they recommend combining this new clinical evaluation method with the current method, which covers its weaknesses, to promote the students' satisfaction with clinical evaluation methods in a perfect manner.

Combined Summative &Formative Assessment - Feedback analysis results

While enquiring for the solution or preferred assessment method 75.6 % Students were of the view that a combination of formative and summative method of assessment in clinical skills will improve their performance in exams and ultimately in delivering better when they go to hospitals in future (Table 9).

Table 9: Feedback on improvement in performance on use of combined summative & formative Assessment.

In a study of Summative and Formative Assessment in Medicine: The Experience of an Anesthesia Trainee by O'Shaughnessy SM, et al. (2015) stated that, rather than a complete reliance on summative assessment, an increased focus should be placed on formative assessment, which has been historically neglected in medicine [14]. Furthermore, we need to conduct further research to ensure that our assessment methods are an adequate predictor of future performance otherwise we as educational leaders will be failing both our students and the wider community.

Anziani H, et al. (2010) in their research titled “The relationship between formative and summative assessment of undergraduates in oral surgery” concluded that the use of both assessments is beneficial; however, their use together must be considered carefully to avoid confusing students [16].

In one more study almost, similar inference was drawn by Newble DI, et al (1983) [17]. In their study an increase in ward?based learning activities was essential for success in the final examinations. This new clinical examination has thus influenced students’ learning and successfully restored the balance of their learning activities between the clinical and theoretical components of the course.

In a study titled “Differential Effects of Two Types of Formative Assessment in Predicting Performance of First-year Medical Students” by Krasne S, et al. (2016), it is concluded that Formative assessments can be used as effective predictive tools of summative performance in medical school [18].

Similar findings seen in a study titled Trends of undergoing formative assessment in undergraduate medical students by Azam F, et al. (2018), that the formative assessment is an effective tool for improving student’s performance in the summative assessment [19].

A study was done by Hashim Z, et al. (2012) with an objective to assess the importance of an exam skills workshop as a formative tool for final year medical students at the end of their surgical placement at the Lincoln County Hospital. All the students felt that this experience would help them in coping better with the summative OSCE as they now had an idea about what to expect on the day of final exam [20].

As per the student response and literature review, we conclude that both formative and summative methods are essential tools of assessment as they have different advantages, and same holds true with respect to clinical skills evaluation. Therefore, the strategy should be, to implement both together to assess clinical skills of medical students, as it will minimize student concerns and yield better results, preparing them well to become good future clinicians.

The study was approved by the institutional review board (IRB) of college of Medicine, Dar Al Uloom University, Riyadh, Saudi Arabia. IRB Number:-Pro18100004.

The authors declare that there is no conflict of interest regarding the publication of this article.

1. Norcini JJ (2003) Work based assessment. BMJ 326: 753-755.
2. Airasian PW (1988) Measurement driven instruction: A closer look. Educational Measurement: Issues and Practice 7: 6-11.
3. Begum N, Hossain S, Talukder MH (2013) Influence of formative assessment on summative assessment in undergraduate medical students. Bangladesh J Med Educ 4.
4. Scoles PV, Hawkins RE, LaDuca A (2003) Assessment of clinical skills in medical practice. J Contin Educ Health Prof 23: 182-190.
5. Khorashad AK, Salari S, Baharvahdat H, Hejazi S, Lari SM, et al. (2014) he assessment of undergraduate medical students’ satisfaction levels with the objective structured clinical examination. Iran Red Crescent Med J 16: 3088.
6. Badyal DK, Bala S, Kathuria P (2010) Student evaluation of teaching and assessment methods in pharmacology. Indian J Pharmacol 42: 87-89.
7. Black P, Wiliam D (1998) Assessment and classroom learning. Assessment in Education, Principles, Policy & Practice 5: 7-74.
8. Taras M (2005) Assessment-summative and formative-some theoretical reflections. Br J Educ Stud 53: 466-478.
9. Kumar Ghosh S (2014) Resident doctors: keystone of anatomy teaching. The clinical teacher 11: 461-462.
10. Reddy K (2016) Summative evaluation top advantages and disadvantage. Wisestep.
11. Labarca J, Figueroa C, Huidobro B, Wright AC, Riquelme A, et al. (2014) Perception of medical students about formative assessments during clinical courses. Rev Med Chil 142: 1193-1199. 
12. Alkhateeb NE, Al-Dabbagh A, Ibrahim M, Al-Tawil NG (2019) Effect of a formative objective structured clinical examination on the clinical performance of undergraduate medical students in a summative examination: a randomized controlled trial. Indian Pediatr 56: 745-748.
13. Harry Torrance (2012) Formative assessment at the crossroads: Conformative, deformative and transformative assessment. Oxf Rev Educ 38: 1-20.
14. O'Shaughnessy, Sinead M.; Joyce, Pauline (2015) Summative and formative assessment in medicine: the experience of an anaesthesia trainee. Int J High Educ 4: 198-206.
15. Hoseini BL, Mazloum SR, Jafarnejad F, Foroughipour M (2013) Comparison of midwifery students' satisfaction with direct observation of procedural skills and current methods in evaluation of procedural skills in Mashhad Nursing and Midwifery School. Iran J Nurs Midwifery Res 18: 94-100.
16. Anziani H, Durham J, Moore U (2010) The relationship between formative and summative assessment of undergraduates in oral surgery. Eur J Dent Educ 12: 233-238.
17. Newble DI, Jaeger K (1983) The effect of assessments and examinations on the learning of medical students. Med Educ 17: 165-171.
18. Krasne S, Wimmers PF, Relan A, Drake TA (2006) Differential effects of two types of formative assessment in predicting performance of first-year medical students. Adv Health Sci Educ Theory Pract 11: 155-171.
19. Azam F, Shaheen A, Irshad K, Javed N, Ata M (2018) Trends of undergoing formative assessment in undergraduate medical students. JSTMU 1.
20. Hashim Z, Miller A, Fahim N (2012) To assess the importance of an exam skills workshop as a formative. Int J Collab Res Intern Med Public Health 4: 78-85.