Innovations in marking and grading of examinations

Conveyor-belt marking

In a convey-belt marking system, each marker specializes in a particular question, rather than mark all the questions in the paper. Therefore, for each script, the marker focuses on only one question.  The model uses the principle of equitable distribution of responsibilities. Markers work in small teams (team work) and specialize in specific question items.

Emphasis is on quality of marking rather than who marks more or faster than others.  The RNEC has identified the following advantages of this examinations marking system:

(i)  There is strong MASTERY of  content of the question in which the marker is    specialized.
(ii)Markers share responsibility  which eliminates or minimizes bias.
(iii)Responsibility is spread from team leaders, chief markers and script markers. It is "team work” in practice.
(iv)Responsibility is shared and compared among and between teams, rather than individual teachers or markers.
(v) There is less fatigue and stress as work is evenly shared out. This contrasts sharply with a competitive model where each marker aims to finish as many scripts as possible each day.
(vi)The system moderates faster markers and motivates slow ones. It is a "win-win” situation. It ensures mutual satisfaction and cross-checking of each others work.
(vii)  Teachers / markers are arranged in small teams of 5 or 6,   which may expand       gradually. This ensures quality assurance.
(viii)  Team leaders benefit from mutual control of the smaller teams. Therefore, control is spread in and between teams. This is in   line with modern management practices.
(ix)Income is evenly spread as emphasis is not based on who is   faster, but who is marking more accurately. Everybody has to be accurate and all are rewarded accordingly.

Thus, the new marking system greatly enhances the quality of student performance, reducing margins of error and thereby boosting grades attained by individual candidates. This in turn boosts grade classes or categories attained by candidates, enabling many to attain first and second divisions, and still many more attaining a "pass mark” (cut of points) which is determined at each level from Primary, Ordinary level and Advanced level. Each level has a "pass-mark” or cut off point. 

The new grading system

The new grading system features harmonized marks processing and grading with the rest of the East African Community Examination Boards. 

a. Primary leaving examinations

In 2008, the grading was changed to numerical grades (1 – 9) and the grades would be added to constitute an aggregate, the aggregates would place candidates in division. The papers are to be marked out of 100 each and then the quality index to be used is the 9 point scale that uses numerals.

Ranging from 1 to 9, where 1 is the highest and 9 the lowest. After the raw marks have been obtained, the grade boundary i.e. values for distinctions 2, credit 6 and pass 8 are determined based on the performance of the candidates and on the previous year’s performance.

A formula is used to determine the other grades. The grades are further subdivided into 4 groups of 1, 2 (distinctions), 3,4,5,6 (credits), 7, 8 (passes) and 9(fail). The grades obtained in all the subjects are added to form Grade aggregate.

Using the grade aggregates, these are further grouped into 4 categories known as divisions to place the pupils in the same ability range as shown below:

Grades                  Qualifiers

1    Excellent
2    Very good
3
4    Credit
5
6
7    Pass
8
9    Fail

Classification of candidates

Aggregate range  Division
3-12    I
13-18    II
19-21   III
22-26   IV
27    Unclassified

b. Ordinary level (o’ level) national examinations

The ordinary level grading is also to be graded on a similar index of the 9 point scale and the grades are further subdivided into smaller groups just like in the primary grading. The difference comes in when determining the divisions as shown in table (ii)

Grades  Qualifiers

1   Excellent
2   Very good
3
4   Credit
5
6
7    Pass
8
9    Fail

Classification of successful candidates

DIVISION I: 7 Credits   +Aggregate

DIVISION II: 5 Credits and 2 passes + Aggregate                                    
DIVISION III: Either 1 Credit +5 Passes or   2 credits + 3 Passes Aggregate in best seven (7)

DIVISION IV:  2 Passes at Grade7
or 4 Passes at grade 8 Aggregate in best seven  

c. Advanced level national examinations
The initial grading for A’ level is similar to the grading in Primary and in O’level whereby numerical grades of 1-9 are used for the papers sat for.

Letter grades of A to E are used where A is the highest pass grade and E the lowest pass grade. There is also F or failure. The letter grades have numerical values attached to them where A has 5 and E has 1, although this is likely to change in 2009 due to the introduction of new subject combinations.

The new changes will include another pass component known as SUBSIDIARY pass which will result from the introduction of General Paper which will have a different weight from the other subjects in the combination. Therefore the grades will range from A to S (subsidiary pass), whereby A will be the highest with a value of 6 and S   the lowest with a value of 1. and F for 0.

Subject weights and points

In order to generate points to be used in selection of candidates to higher Institutions of learning, the value obtained by a candidate in each subject of an option / combination is multiplied by its corresponding subject weight then the results for all the subjects are added together and the total will constitute the points upon which selection will be based as shown in table (iii).

Determining grade boundaries

An Awards Committee composed of subject chief examiners, Subject Officers and other senior Council personnel deliberate on the grade boundaries and fix three major ones mentioned below

which is the minimum level of Distinction;
Which is the minimum level of Satisfaction (credit pass);
Which is the minimum pass at subsidiary level.
The other intervening grades are calculated from the formula below:

If G2, G6, and G8 are fixed,
Then:G1:  (G2-G4) / 2 + G2
G3: (G2-G4) /2 + G4
G4:  (G2-G6) / 2 + G6
G5:  (G2-G4) / 2 + G6
G7:  (G6-G8)/ 2 + G8

After the numerical grades have been determined, the letter grades are easily determined.

Conditions for the award of advanced level certificate
The condition for the award of a certificate in 2008 was for a candidate who has passed in three subjects with at least grade E, plus at least grade D in the practical paper for those taking professional and technical options.

The grade awarded is indicated by letter A, B, C, D, E and F. Grade A is the highest grade of pass with a value of 5 points and Grade E the lowest with a value of 1 point, and, F is Failure.

Introduction of Science Practical Examinations

In 2009, the RNEC has introduced for the very first time, practical examinations in science   (biology, chemistry and physics) based on the new subject combinations.

The RNEC identified relevant skills and issues to learn and implement   which include formulation of policies, guidelines and procedures in Science Practical Examinations;  designing criteria for decision making and problem solving in science practical examinations; how science practical examinations are planned, prepared, organized, conducted, marked,  graded and their relationship to theory papers; assessing  financial, practical and logistical considerations in science practical examinations.

The RNEC trained Advanced Level science teachers who are preparing for and will conduct, mark and grade the practical examinations.

The following areas were highlighted during the training: The level of the examination, the syllabus content and objectives and duration of the examination.

The science process skills, which are basic in practical work, were emphasized.

The skills included manipulation of the equipment and apparatus, observation, measurement, recording readings, tabulation of readings, graphical presentation of results, interpretation and making conclusions.

The RNEC noted that there was positive change in attitude towards practical examinations in science by many of the teachers. Towards the end of the training most of the teachers adjusted to the standard setting procedures and some of the teachers expressed interest to work together as a team and see ways of carrying out practical examinations in science in their schools. 

However, there was evident lack of display of understanding and mastery of skills for marking and grading of practical examinations. Therefore, the RNEC will continue to train teachers in these processes.

Personalized photographs on candidate diplomas
After grading, the RNEC will also for the first time introduce personalized photographs on candidate diplomas.

Here data includes candidates’ finger prints for evidence in case of forgery or theft, especially in cases of impersonation where candidates have been caught trying to seat on behalf of others. It is hoped that this security arrangement will help deter corruption cases related to certificates and diplomas.

ICT development and National examinations

In the assessment process, the RNEC undertakes various massive tasks of registration of students, preparation of examinations, conducting of examinations, marks processing and publication of results, and then orientation of successful students to the next level of education.

These tasks involve processing of large databases and message exchanges through internal communications; and interaction with students, parents or guardians, teachers, heads of institutions and district education officers through announcements, rules and regulations governing the conduct of examinations.

RNEC has a number of ICT equipment including two server machines, a fully wired computer network with fibre optic back borne, an optical mark reading machine, an examination processing application called FAIM running on SQL server and windows 2003 server.

In the last 6 years, the RNEC has been developing its ICT capability in order to deliver its services much more efficiently and effectively.

This also is necessitated by the fact that the number of candidates increases every year, hence ICT capability to deliver and meet the long term needs and expectations of Rwandan primary and secondary schools is essential.

The RNEC has developed a web based tool to enable it improve the coordination and managements of high stake public examinations processes. ICT development at the RNEC has achieved the following targets:

a. Online Registration of candidates, Publication of results and SMS Cell phone based system

A highly secure online system for checking results is desired where students and parents or guardians can access RNEC databases by providing their registration number and then their results are displayed in non modifiable format through web based application or SMS. The candidates are also expected to register online through an interface that is linked to our data base.

The RNEC has built an Online based Results Information System that been used to provide primary and secondary school leaving candidates with online access to their results through cell phones by SMS text and the Internet through the Web and to provide other services such as registration verification by SMS text.

We have installed and configured the system to enable students query the RNEC databases by SMS text for their results as well as through the Web and also to verify their registration status.

The RNEC has built all the necessary capacity needed to run, support and maintain the developed system. The system   includes querying of examination timetables for all RNEC administered examinations.

We have developed a database within RNEC that will enable the collection of statistics on the traffic and usage of the SMS and Internet system. This is important for SMS service provider and for monitoring purposes.

The RNEC has trained staff to develop a strategy of implementation for the marketing and public awareness of the system and entered into a Public-Private Partnership (PPP) with an SMS provider to ensure sustainability.

One of the important features of the system is to conduct system requirements analysis, design and development, deployment, testing and providing a clear maintenance plan of the System comprising of both client and server system components and staff have been trained on its use.

The RNEC has spared no effort to ensure effective and efficient server configurations including the implementation of SMS and Internet interfaces (gateway configuration) between the system and the selected mobile telecommunications operator.

We have established a central database and server software for storage and retrieval of examination results information.

This information has been correlated and aggregated in the manner required by RNEC.

An enhanced website has been in place for use to publish examination results and student registration details and additional promotional information about the System including but not limited to basic information about RNEC.

There exist built-in reporting mechanisms to support statistical analysis of examination results data. Data collected are summarized and made available through the website and other media formats (Tables, Graphic Charts, PDF, Spread sheets, GIS Maps, etc).

Candidates are able to query the database using specific student codes such as registration numbers or other pin numbers to be determined in order to retrieve examination results either by SMS or directly from the website.  Users, who use SMS to receive examination results, are charged a premium rate for the SMS messages by the service provider.

Queries are also made directly into the database through a web interface. 

Information retrieved from the website is also available for downloading. The System has the ability to create historical trend reports (annual reports on primary and secondary schools examination results) using the electronically stored data, to use in planning and reporting on the activities of the sector.

b. Collaboration tool

The RNEC has developed a collaborative tool to enhance communication amongst stakeholders. Below are some of the data/message elements collected by the examinations Council which is utilized by this application. They include:

List of registered students
•  List of withdrawals from exams
•  Rules, regulations and guidelines to conducting exams in centres  and  districts
•  Announcements
• Reports on administering
exams:
Lists of students selected for the next level of study
List of transferred candidates
Urgent Messages to centres and districts
Internal messages by exams council staff
Where possible, voice and video chats for virtual meetings and video conferencing;

General chat

Content format
• The lists are either word processing documents or spread sheets;
• Reports, rules, regulations and guidelines are word processing documents;
•  Announcements, urgent messages and live text chat are in text or html formats;
•  There is also some voiceXML and MPEG or Avi for video chats.
The user categories include:
Systems administrator
• Regular user
• Moderator

Every district officer, Head of department and Chief Executive Officers can be upgraded to moderate his/her group chats.

In addition a user may access the application by providing a user name and password that was previously set at registration by the system administrator. Then users will be authenticated and only information related to their duties is made available to them.

Officers who head examination centres are now able to upload or download their own lists of students and reports, view rules and regulations, announcements, urgent messages, district and RNEC reports and also join chat sessions of their group or the general chat.

District officers are now able to upload or download their own lists of students and reports, view rules and regulations, announcements, urgent messages, district and RNEC reports and also join chat sessions of their group or the general chat and posting announcements and urgent messages.

RNEC officers will be able to upload/download rules and regulations, lists of students and reports, view rules and regulations, announcements, urgent messages, also join chat sessions of their group or the general chat.

c. Selection process

This module enables the RNEC to gather important statistics as indicated below. Schools or institutions can now submit these vacancies online into the RNEC server:

Number of places available at Ordinary and Advanced secondary school levels and at the University level
Number of places available at A level by option for candidates completing O level. Very high degree of security and privacy is required.

d. Examiners tracking system

This module covers the following items using online interfaces that are well designed making use of appropriate technology concepts:

Registration of examiners, giving their personal details.
The exams an examiner sets and also the exam he/she marks
Examiner/teacher’s qualifications
The class a teacher teaches:- A-level, O-level or primary level and the subject that he/she teaches
Team leader’s comments. Very high degree of security and privacy is required.

e. School performance

The RNEC is now able to track the results and examine the performance of schools. This system should be able to generate statistical data on performance of each school.

National examinations and the wider   educational system

a. School infrastructure and other facilities

 One of the major challenges in our education system is school infrastructure. Many of our schools need basic equipment and other facilities that enhance and promote teaching and learning.

School buildings need to be suitable for a learning environment for our children to achieve high performance both at school level and in national examinations.

b. Mastery of the language used as a medium of instruction

Language acquisition affects learning in other subject areas. The results of national examinations show a high level of poor mastery of languages. Language is an important tool as it is used as a medium of instruction and personal study.

National examination results show evidence of a need to improve basic literacy which is a pre- requisite to learning higher level skills and competencies in various subject areas.

Other challenges include lack of family involvement in student learning, absenteeism, poor literacy and numeracy skills, poor learning environment at home and at school, very little time spent on meaningful learning activities, large class sizes, lack of basic school infrastructure and facilities at school, poor discipline at home and school which makes learning unsafe, ignorance of essential life skills especially of critical thinking, problem solving and decision making.  All these affect performance at school level and in national examinations.

c. Teacher training

One of the most crucial challenges facing the performance of students in Rwanda relates to teachers and teacher training. 

There is need for requisite skills and competencies which teachers need in order to use learner centred teaching methods and strategies. Consequently, these affect testing, measurement, assessment and evaluation.

There is also need to understand the relationship between formative (continuous assessment) and summative (final examinations). This is important so that teachers do not emphasize examinations rather than the actual mastery of  the curriculum in form of knowledge and skills.

Also, teachers need to make sense of national goals of education and interpret and translate them into meaningful and positive learning experiences for students for better performance both at school and in national examinations.

d. Curriculum and pedagogical materials 

These aspects are considered together because it is the school that implements and therefore in actual sense manages the curriculum. Before 1994, Rwandan curriculum was very wide in terms of subjects offered at each level to widen the chances of employment.

Following the return of refugees in 1994, it was necessary to reform the curricula to take account among other things of the educational needs, experiences and backgrounds of all Rwandans.

The curriculum reform of 1996 to 1998 has been revised further to take account of current educational and socio-economic challenges.

Those children who have access to reading materials learn more than those who do not, the more books they have, the more they learn.

Thus, availability of pedagogical materials affects the quality of the education system as expressed through learning, achievement and student performance both at school and in national examinations.

e. Formative (continuous assessment) and summative (final examinations)

Large scale implementation especially of the Nine Year basic Education and the newly expanded upper secondary education requires consideration of whether to emphasize formative or continuous assessment as opposed to summative or end of programme examinations.

Expanded access will have to occur along side curriculum development that produces coherent learning from grade 1 to grade

In addition, assessment will continue to play a central role in shaping learning and teaching, measuring learning achievement, diagnosing difficulties and determining who is selected for progression and further studies.

Examinations are important whether they are internal or external because they perform certain functions for the benefit of different stakeholders. Internally, examinations are administered at different periods of the year such as mid-term, end of term, and end of year.

In each case they serve different purposes but the major one is to find out how much of the intended objectives the student will have grasped and to find out whether the teacher’s methodology are working. But several things are accomplished in the process and these include:

(i) Identification of strengths and weaknesses of students for building on strengths, instate remedial sessions for weak students and adjusting the teaching methodology to improve learning in general.

(ii) Provide feedback to teachers and students to close the gap between the attained and desired targets. In this way, the teacher is guided on the next course of action.

(iii). Providing motivation to students

(iv). Estimating potential for future performance

(v). Promotion or selection to the next class

(vi). Provision of statistics for internal use by the schools to find out whether they are accomplishing their objectives

External or public examinations are important in their own way, in view of their advantages which include:

a. Certification: This serves mostly those who are not likely to continue but also those who continue to higher levels and in the employment sector.     

b. Raising standards across all the institutions which is carried out by monitoring the performance of students and keeps the schools on their toes.

c. Curriculum control, because the performance of students is used to serve as an indicator of whether the curriculum was covered or not.

d. Informing students, teachers, parents, of the progress of students

e. Legitimizing particular forms of knowledge. By examining a subject, it gains a high profile in the minds of many students and teachers.

f. Encourages teachers to treat the subject thoroughly in order to cover the prescribed course of study within the given time. This is important because the teacher does not know from which parts of the syllabus questions will be set..

g. Encourages students to read widely and make them get prepared to reach a stated degree of knowledge by a fixed date which is better training for time keeping in everyday life.

They also incite the pupil to get knowledge into a reproducible form and make the students make thorough preparations on most parts of the syllabus because they do not know where questions will be set from.

This may be against the fact that the topics might be uninteresting.

h. Accountability: A role that has become important is that education being a major consumer of public funds, it is fitting that the taxpayer gets an account of how the money is spent. The best way of checking on this is by examining the performance of students where results of the examinations are used to check on the effectiveness of the education system as a whole.

The results are used also in comparing schools, hence comparing their performance and even judging the effectiveness of teachers who on their part, in full view of results, have to take  stock  and appropriate action for more effective instruction (encompassing defining goals, pertinent tasks, providing relevant, understandable and timely feedback, saving the motivational learning context within and outside the classroom).

i. Public examinations are the best way of distributing the few available resources bearing in mind the fact that school - based assessment cannot avoid the bias, since every teacher and school would be having vested interests in the passing of their pupils and their selection to the next level.

Ends