Session 8D5 THE USE OF TIMED ASSESSMENT IN ENHANCING LEARNING: A CASE OF ENGINEERING ELECTROMAGNETICS COURSE
B. M. M. Mwiniyiwiwa 1
Abstract ? For a long time Electrical Engineering students at the University of Dar es Salaam had negative attitudes towards Engineering Electromagnetics course. This is because the course is heavily loaded with mathematical concepts. The use of timed assessment was adopted for this course because it encourages the students to study widely in an attempt to cover wide range of concepts in the course. Another reason for using this approach was based on the consideration that students have to perform equally well in other courses. The timed assessment tasks are given frequently to ensure that students’ concentration on the cost is maintained. This paper presents how the use of timed assessment has enhanced students’ understanding of the Engineering Electromagnetics course and contributed to improved performance in the course. Trend analysis was conducted for students’ performance in the timed assessments given to them for the entire period of the course. This study advocates an increased correlation between the continuous assessment (coursework) scores and final examination scores effective from the year when the extensive use of timed assessment was adopted for the course. Index Terms ? Timed Assessment, Assessment Strategies, Study Approaches, Instructional method, Engineering Electromagnetics.
Assessment involves techniques of collecting information for the purpose of making judgment about students’ performance. All course instructors collect assessment information that helps them in making decision about students’ learning and the success of their methods of instruction. The assessment method used would normally depend on the nature of knowledge and skills to be assessed. A number of criteria are used in making decision on what assessment approach to use for the course. These criteria include: capability of the assessment tasks to properly reflect the learning outcomes; representing student performance with sufficient depth to permit confidence conclusions about proficiency and elimination of all relevant sources of bias and distortions that can lead to inaccurate representation of students’ ability . The need for accurate ways of assessing students’ learning outcomes is increasing due to increased competition in scholarships and limited employment opportunities. One has to show evidence of outstanding performance in order to
survive such competitions. In their editorial remarks, correctly asserts that “there is a great deal of pressure from both industry and academic accreditation entities to incorporate a broader set of student learning outcomes and sound assessment techniques into the programmes and courses” . In response to such pressures, a number of assessment procedures have been devised and are increasingly being used in various contexts. However, finding an appropriate assessment tool remains as a major challenge in designing an assessment approach . The difficulty arise because of the diversity of learning objectives, the diversity of what counts as evidence of learning, the varying resources available and the varying assessment contexts . Two kinds of assessment, namely formative and summative, are normally used. Formative assessment involves regular use of various techniques for collecting information about students’ progress in learning which serves as feedback about teaching and learning process . On the other hand, summative assessment is done at the end of the course to determine what the students have achieved from the entire course. The course instructors are mainly concerned with formative kind of assessment since the feedback obtained is useful for them in modifying their instructional approaches. However, the formative assessment becomes useful if the students hold a notion of desirable standard of performance and they use the assessment information to regulate their study approaches. “Effective formative assessment requires that students take a central part in learning (teachers can not learn for them), and unless they come to understand their strengths and weaknesses, and how to deal with them, they will not make improvements in their performance” . This quotation points to the fact that the improvement of teaching and learning process depends how both students and instructor view assessment as an integral part of teaching and learning. For assessment information to bear fruitful results in improving the learning process, there is a need for reflecting on the performance of students. Factors that lead to high achievement should be reinforced while those leading to poor performance should be a focal point of discussion so that a mechanism for better understanding of the concepts may be realized. Instructors may use a range of assessment approaches in making decisions about students’ readiness to learn a new concept, their strengths and weaknesses. These approaches include take home assignments, timed assessments
B. M. M. Mwiniyiwiwa, Department of Electrical Engineering,University of Dar es Salaam, Box 35131 Dar es Salaam, Tanzania, firstname.lastname@example.org J.L. Ndalichako, Department of Educational Psychology, University of Dar es Salaam, Box 35048, Dar es Salaam,Tanzania, email@example.com
International Conference on Engineering Education 8D5-16
August 6 – 10, 2001 Oslo, Norway
(including quizzes, tests, and timed assignments), project ork and portfolio. Each approach has its own merits and demerits. For instance, while the take home assignments provide ample time for students to read widely on the topic, it creates loopholes for cheating in the sense that the assignment may be done by someone else apart from the actual student. Timed assessment tasks encourage students to read thoroughly as they have to grasp most of the concepts before they sit for the assessment task. However, the timed assessment approaches create anxiety among the students that could inhibit them from performing at their best of ability. Such approaches could provide a reliable and valid information about students’ performance if they are used quite often in the course. Portfolio is the method of assessment in which the collection of sample students’ work is made. The main weakness embedded in this kind of assessment is the difficulty in determining what should be included in the collection as well as the criteria that should be used in assessing the portfolios. Generally speaking, the appropriateness of method to be used depends largely on the nature of the course and the approach used to deliver it. In view of the nature of the Engineering Electromagnetics course, extensive use of timed assessment was adopted as a method of determining learning outcomes of the students. The rationale for the choice of assessment is provided in the next section of the paper. This section is then followed by a description on how the timed assessment was implemented. The next section presents the analysis of students’ performance on the course. Finally the discussion of the findings and conclusions are presented. continuous assessment, commonly referred to as coursework score. The coursework constitutes 50 percent of the course grade. At the time of starting the course, the students are thoroughly informed on coursework assessment method. For the past three academic years, the number of students taking this course is between 45 and 49. This number is the annual intake of the Department of Electrical Engineering at the University of Dar es Salaam. The Engineering Electromagnetics course is offered to the students during their second year of study.
IMPLEMENTATION OF TIMED ASSESSMEN T TASKS
The quizzes were constructed to assess both theoretical and computational aspects of the course. The duration for the quizzes were between 20 and 30 minutes while the duration for the tests were between 1.5 and 2 hours. Given the short time given to students, the questions for the quizzes are short and only few as compared to those questions for the tests. As of 2000/2001 academic year, the Faculty of Engineering is following the term system, where there are three teaching terms. The University Examinations are administered at the end of academic year. In the implementation of timed assessment, students were required to write two quizzes and one test per term. The weighting of assessment component of the course was such that 20 percent is allocated to the six quizzes and 30 percent to the three tests and 50 percent to the University Examination. The basis for such weighting was that students would find 20 percent allocated for quizzes as substantial weight to encourage them to work hard consistently towards understanding the course throughout the academic year. The timing of the assessment was evenly spread to ensure that students put consistent efforts in studying for the course. It was further assumed that if students were assessed based on small units they have covered, they would have sufficient time to use deep learning approach as opposed to surface learning approach that is normally done when there is a lot of material to cover . The frequent use of timed assessment gave students an opportunity to monitor their achievement and engage in a corrective action in case they are unsatisfied with their performance. The use of timed assessment has been implemented for three consecutive academic years; 1997/1998, 1998/1999 and 1999/2000. Descriptive statistics were computed for each task and correlations between coursework and final examination was computed for the past five academic years. The following section presents the findings.
THE NATURE OF THE COURSE AND ASSESSMENT APPROACH
The Engineering Electromagnetics course is concerned with fundamental concepts in the aspects of Electrostatics, Magnetostatics, Electromagnetic fields, Electromagnetic waves, Maxwell’s Equations and High Frequency Transmission Lines. This course plays a great role as a foundation for understanding many aspects in the Electrical Engineering and related fields. The Engineering Electromagnetics course is essentially of abstract in nature consisting of enormous vector calculus of several variables. Furthermore, the course requires imaginative skills in such a way that the student is capable of formulating a problem before a solution is sought. The course requires students to have well versed knowledge of all sorts of multivariable calculus because majority of the problems are of multidimensional in nature. In that respect, in most cases, the students are required to formulate the problems in spaces and regions. Given the nature of the course and in an attempt to motivate students to work consistently on the course, the extensive use of timed assessment was adopted as a method of determining students’ learning outcomes . The students were given six quizzes and three tests as part of their International Conference on Engineering Education
ANALYSIS OF S TUDENTS PERFORMANCE
Students performance were analysed to indicate the nature of relationship between continuous assessment and final examination. Secondly, trends in performance for selected
August 6 – 10, 2001 Oslo, Norway 8D5-17
students were analysed in an attempt to show students study approaches as on the basis of assessment feedback. Overview of the General Performance The descriptive statistics were computed for each assessment task that was given to students. The general performance is summarised in terms of means and standard deviations computed for 1998/99 and 1999/00 when the extensive use of timed assessment was adopted for the course. The results are given in TABLE I.
DESCRIPTIVE STATISTICS FOR THE RESULTS OF 1998/99 AND 1999/00 ACADEMIC YEARS
The correlations between coursework and final examination reveal moderate to high relationship between the two measures of student achievement in the course. The timed assessment method presented in this paper was adopted as from 1997/1998 academic year. As it can be observed in TABLE II, the correlaion between the coursework and final examination has increased steadily from 1997/1998 to 1999/2000 academic years. It is evident that the use of timed assessment has enhanced student understanding of the Engineering Electromagnetic content and resulted to better performance for both, cousework and final examination. TABLE II
CORRELATION BETWEEN COURSEWORK AND FINAL EXAMINATION*
Nature of assessment Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 Test 1 Test 2 Test 3 Exam
1998/99 Mean S.D 79.1 10.02 80.2 8.9 58.9 21.5 80.1 15.2 43.6 13.0 84.1 13.7 51.0 17.8 61.8 13.3 47.4 12.4 52.2 13.9
1999/00 Mean S.D 88.2 4.4 95.8 4.1 76.6 21.3 42.0 12.1 70 17.9 63 26.8 52.8 14 77.7 10.6 65.6 10.5 61.0 10.0
Before Timed After Timed Assessment in Place Assessment in Place Year Correlation 95/96 0.36 96/97 0.49 97/98 98/99 99/00 0.40 0.58 0.65
*The correlation between the 2000/2001 coursework and final examination has not been included because the Final Examination is not yet done
TRENDS IN STUDENT PERFORMANCE
Trends in performance across the assessment tasks were examined. The performance of students across various timed assessment tasks, namely six quizzes and three tests revealed a mixed trend. However, two trends featured quite often for the majority of students. The two vivid trends are fluctuating and steady performance. Other trends observed for a small number of students were either consistent improvement or continuous decline in performance. While some students showed fluctuation in their performance, others showed a steady performance. Such trends are shown in FIGURE 1 and FIGURE 2 respectively.
From TABLE I it can be seen that the mean scores for quizzes tend to be higher than those for tests and final examinations. The reason for such an observation is that the quizzes cover a small portion of learning outcomes and since students would normally know the area of coverage for their quizzes, it makes it easier for them to study deeply. In the case of tests and examinations, the coverage is wide. Students with short-term memory are unlikely to remember the factual information and would attempt only questions related to topics that they clearly understood. It can also be observed from TABLE I that the mean score for the quizzes for the two years’ results ranged from 42.0 to 95.8. One reason of such a wide range in terms of mean scores may be the level of difficulty of the quizzes. Another reason for wide range of performance in the quizzes could be poor preparation for the poorly scored cases. Correlation between Coursework and Final Examination The correlation coefficients before and after the use of timed assessment procedure was adopted for the course are given, just to give an impression of how students’ continuous assessment scores correlated with their final examination performance. The correlations between the continuous assessment (coursework) and the final examinations are presented in TABLE II.
80 60 40 20 0 Qz1 Qz2 Qz3 Qz4 Qz5 Qz6 Tst1
Student1 Student2 Student3 Student4
Nature of assessment STUDENTS’ PERFORMANCE FLUCTUATION
The fluctuations in terms of students’ performance may be explained as an attempt to ensure that they perform well. August 6 – 10, 2001 Oslo, Norway
International Conference on Engineering Education 8D5-18
These students tended to relax when they performed very well in one quiz. The relaxation led to somehow a poor performance in the subsequent quiz. In this way, the timed assessment serves as a mechanism for students’ feedback about their learning. In order for students to improve in their performance, they need to have a notion of desired standard or goal, be able to compare the actual performance with the desired performance and to engage into appropriate action to close the gap between the two . Lazy students are likely to have performance with noticeable fluctuations. A sort of steady performance is shown in FIGURE 2. From FIGURE 2 it can be seen that the student performance was not changing significantly. The more less steady performance is normally attributed by the student to maintain consistent efforts in the course. This kind of performance is normally characterized by hard working students.
80 60 40 20 0 Qz1 Qz2 Qz3 Qz4 Qz5 Qz6 Tst1 Tst2 Tst3
student1 student2 student3
Nature of Assessment STUDENTS’ STEADY PERFORMANCE
STUDENTS ’ PERCEPTIONS ABOUT THE M ETHOD OF ASSESSMENT
Students currently enrolled in the course were asked to comment about the method of assessment used in Engineering Electromagnetics course. These students have now completed all the course work requirements. That is, they have already done six quizzes and three tests and they have are about to start writing their final examinations. Students were asked to comment briefly on the frequent timed assessment method that was used for the course. Two main issues emerged from students’ responses. Some students feel that frequent use of timed assessment keeps them vigilant and forces them to explore more about the course. In turn, they find they are learning a lot of concepts in the Engineering Electromagnetics course. The other group of students feel that the assessment method used is too demanding since they also have to study for other courses. The following are some of students’ responses regarding the assessment method used for the course:
Group 1: “This course is too demanding because we have to do several quizzes and tests. We just find ourselves spend most of the time studying for this course but we have other courses that we are taking”. Group 2: “The nature of assessment used for this course forces us to work hard. We spend a lot of time studying for the course. The number of quizzes should be reduced to three and three tests will just be enough”. Group 3: “The many quizzes and tests are given in this course, they provide us with an opportunity to improve our performance in case we have not done well in the initial assessment tasks. My performance in the first two quizzes was terrible. It was quite a headache to me whether I was going to survive the course. I had no choice except changing my approach of studying for the course. I devoted more time working for the course and my performance improved in the subsequent assessments”. Group 4: “The only way to survive in this course is to ensure that you understand all the concepts covered. You know that once some topics are covered you will be required to show evidence of what you have learned. With time, I realised that the best strategy to use in studying for this course is to ensure that once the instructor covers a certain topic I read widely to ensure that I have understood it well before a new topic is started. I am quite happy that I have learned a lot from this course”. It is evident that students are concerned with the frequent use of timed assessment for the course. They generally find the course too demanding as they have to spend a large portion of their time studying for the course. This burden is equally felt by the instructor who has to set and then mark the quizzes. In fact, the quizzes are marked immediately after they have been done so that students and instructor could reflect on the general performance on the quiz before they cover a lot of new materials. The instructor has no teaching assistants who could help in marking the quizzes or in doing reflection of students’ performance. However, despite the burden imposed by this method of assessment, it remains to be the best approach to ensure that students work consistently on the course. Due to the nature of the course, there is a strong belief that, the students’ performance would have been worse if summative approach of assessment would have been adopted.
DISCUSSION OF THE R ESULTS
The results of students’ performance indicate that the use of timed assessment has contributed to increased understanding of students the concepts in the Engineering Electromagnetics course. There is generally an increased correlation between the coursework scores and final examination performance. Students’ attitudes toward the course have changed since the frequent use of assessment encouraged them to study consistently for the course. Such continuous efforts enhance their understanding of the course as a consequence, they perform better and get to like course. August 6 – 10, 2001 Oslo, Norway
International Conference on Engineering Education 8D5-19
The time allocated for doing quizzes is normally short. This is a major challenge to slow writing students with long memory duration as they may not have sufficient time to think through and write down their answers despite that they may know how to answer even all the questions. Nevertheless, this disadvantage is offset during tests and final examinations in which students are given ample time to work on the problems. On the other hand, fast writing students with short memory duration are fevoured by short duration quizzes. In this way, the net effect is that all students are given chance for better performance in the coursework. acted as a catalyst in encouraging students to study widely and get better understanding of the course as a consequence, they could perform better. This paper has also shown how frequent use of assessment may serve as a motivation for students to study hard and have a deep understanding of the concepts. One important point to note is that in order to sustain this mode of assessment effectively, there is a need for the course instructor to have an assistant in marking the students’ assignments. When sufficient human resource is available, the use of frequent timed assessment is ideal for a course like Engineering Electromagnetic in which a large portion of the course content is heavily loaded with mathematical aspects and a lot of imaginations.
CONCLUDING R EMARKS
This paper has provided an analysis of the use of timed assessment for the Engineering Electromagnetics course. The findings of the study revealed that the use of frequent timed assessment has helped the students to study widely for the course thereby enhancing their understanding of Engineering Electromagnetics concepts. Furthermore, the frequent timed assessment method has somehow changed the long lived students’ attitude to thinking that this course is too difficult. However, this method of assessment creates a lot of demands for both, the students and the instructor. Students need to spend a lot of time exploring about the course in terms of intensive revision and enormous problem solving so as for them to grasp the concepts solidly. On the other hand, the instructor has to do the marking in the shortest time possible to ensure that the assessment tasks serves as a useful feedback in improving the teaching and learning of the course. In that case, if feedback is not provided in time, it may not serve its useful purpose of improving the teaching and learning process . The use of frequent timed assessment has provided an opportunity for students to monitor and regulate their learning strategies. In this way, assessment method used has
 Stiggins R. J. “Student-centred classroom assessment”, Columbus, Ohio: Merrill, 1997.  McGourty, J. and Kerns, S, “Using Multisource feedback in the classroom: Computer-Based Approach”, IEEE Transactions on Education , Vol. 43, No. 2, May 2000, pp. 120 – 124.  Angelo T. A. and Cross P., “Classroom assessment techniques”, San Francisco, CA: Jossey-Bass, 1993.  Turns J., Atman C.A. and Adam R. “Concept Maps for Engineering Education: A Cognitively Motivated Tool Supporting Varied Assessment Functions”., IEEE Transactions on Education Vol. 43, No. 2, May 2000, pp. 164-173.  Sadler D. R., “Formative assessment and design of instructional systems”, Instructional Science, Vol 18, 1983, pp. 119-144.  Harlen W. and James M., “Assessment and Learning Differences: Relationship Between Formative and Summative Assessment”, Assessment in Education : Principles, Policy & Practice, Vol. 4, No. 3, 1997, pp. 365-379.  Biggs, J., “Students Approaches to Learning and Studying. Melbourne”, Australian Council for Educational Research, 1987.  Gipps, C., “Quality in Teacher Assessment”, Paul Chapman Publishing Ltd , 1994.  Ramaprasad, A., ”On the definition of feedback”, Behavioral Science Vol. 28, 1983, pp. 4-13.
International Conference on Engineering Education 8D5-20
August 6 – 10, 2001 Oslo, Norway