Management of Tutors
OLI's tutors can be regarded as front-end operators in an organization like a bank that has extensive customer services on a day-to-day basis. Like tellers in the bank, OLI's tutors have full support from their managers in the back office. These managers are the course coordinators in the context of OLI who assume the responsibilities of preparing or overseeing the preparation of the course materials, recruiting qualified tutors, planning the study schedule of the course, monitoring tutors' performance, providing back-up support to tutors while they have problems with students or with the course materials and many other tasks such as even directly interacting with students when the tutor is not available.
The majority of the tasks undertaken by OLI's course coordinators relate to the management of tutors. This is why course coordinators have a major role to play in ensuring the quality of the tutoring services that are delivered by tutors. Below we attempt to elaborate further the ways we ensure the quality of our tutor's [sic] services from the perspective of course coordinators.
Tutor Recruitment
About four months before each semester starts, course coordinators and their senior colleagues (the Dean or the Programme Leader) recruit part-time tutors for their respective courses. Experience in distance education is not mandatory, but apart from required academic qualification, we expect the right kind of personality such as care for learners, sympathy for the difficulties of adult students, and maturity to handle adult problems.
Briefing Sessions
After the recruitment process is completed, all new tutors are required to undertake a series of tutor training programmes, as elaborated earlier in this paper. Then the course coordinator concerned will begin a number of briefing sessions with all tutors, be they new or experienced. Usually different sessions are held for new and reappointed tutors separately. The primary objective of the briefing sessions is to introduce and explain to them the nature of the course, the difficulty they may encounter in tutoring students, and most importantly the expectation of the course coordinator in terms of marking the assignments, conducting tutorials, providing ways for students to keep in touch and so on. The session usually lasts three hours.
From time to time during the semester, briefing sessions with all tutors for the course are also held for tutors to share their experience with each other and the course coordinator uses this opportunity to obtain feedback from and provide instructions to the tutors on issues that have not been foreseen and can be rectified immediately.
Debriefing Session
After the final examination of the course has been undertaken by students and before the tutors have completely fulfilled their duties as laid down in their contracts, a debriefing session with the tutors is also usefully held to conclude the course presentation by exchanging tutors' experience and comments on tutoring their students. It is expected that the course coordinator can learn a great deal of the strengths and weaknesses of the course and how effective are the assessment strategies defined by him/her through this kind of debriefing session.
Assessment Meetings
Assessment forms a large part of a tutor's work at the OLI as all the assignments given to students must be marked by their respective tutors. Normally, a tutor is assigned with a tutor group of about 30 to 35. If there are five assignments during the course of study, then the tutor will have to mark about at least 150 assignments. Though tutors are given marking schemes and guidelines on each assignment, it is still not uncommon that there are discrepancies between tutors in proceeding their markings, and awarding marks. Thus it is necessary to standardize the markings across all tutors in order to minimize the differences. Thus an assessment meeting for assignment marking may be needed but is subject to the individual course coordinator's discretion.
For the final examination, it is the Institute's quality assurance policy to have three sessions of such meetings shortly after the examination for each course. First, a coordination meeting will be held by the course coordinator to discuss with appointed script markers on marking a sample of scripts and synchronize their marking approach. Soon after the actual markings are completed, a standardization meeting will be held and chaired by a senior academic (usually the Dean) and attended by both the internal and external examiners of the course to standardize the markings across all script markers. Finally, an award meeting will take place to determine the final grade to each student of the course.
Tutor Monitoring System
As mentioned earlier, OLI's tutors are the direct interface between the Institute and the students. In order to ensure the quality of our delivery system, the performance of the tutors will need to be closely monitored. At the OLI, the monitoring role rests with the individual course coordinators who follow the Institute's guidelines. There are various ways to measure the performance of the tutors and these can be briefly described as follows (Tong, 1992):
- The turnaround time of tutor marking, the quality of marking and the comments on the written assignments will certainly reflect the integrity of the tutors.
- The numbers attending tutorials may reflect how well the tutors have been accepted by the learners. The course coordinator should visit tutorials on a regular basis in order to obtain feedback from the learners' on the tutors and, whenever possible, to personally feel the quality of the tutorials.
- The number of assignments submitted and the examination results may also serve as an indicator on the progress of the learners and the amount of assistance given to them by the tutors.
- The learners dropout rates of certain tutors can also explain the extent of encouragement given to the learners by the tutors. If tutors are in constant touch with the learners, the dropout rates perhaps can be minimized.
Thorpe (1988) indicated that the purpose of monitoring was not only a quality control mechanism, but it also served to expedite immediate and remedial action to areas or problems, as required. The purpose of monitoring is thus seen as a process of standardization and feedback on the performance of the tutors.
Institute's Rules on the Level of Monitoring
The Academic Board of the OLI has set guidelines about the level of monitoring assignment marking. For the first assignment, three scripts will be sampled from each tutor for the course coordinator to monitor and to indicate the quality level of the marking. Level 1 means one script to be monitored on the next assignment; levels 2 and 3 for two and three scripts respectively; and level 4 requires the course coordinator to determine the number of scripts to be monitored next time. Tutors' performance is indicated by different levels of monitoring during the course of study. Meanwhile, the course coordinator needs to take immediate action to improve those who receive the low level of quality signal.
Monitoring Process
By monitoring assignments and checking computer records, course coordinators can ensure that all students are receiving sufficient amount of tutor feedback. If tutors are found to be giving significantly less feedback, they are advised accordingly. Also course coordinators can pick up severe or lenient markers from monitoring the assignment marking. Another method of ensuring the students receive fair marks is to look at the computer printout distribution of marks for each tutor. This printout shows if any tutor is giving particularly high or low marks, or if the distribution of marks is uneven.
Classroom Visits
As part of their job, course coordinators visit tutorial classes conducted by his/her tutors from time to time in order to have first hand experience and feeling about the conduct of the tutors and reactions from students. However it is argued that visits to tutorials should not be considered as a top priority in monitoring the performance of tutors. It is preferred to adopt a relatively informal approach if visits to tutorials have to be carried out (Tong, 1992).
Evaluation of Tutors
For full-time academic staff at the OLI, we have a formal performance appraisal system in place as the number of such staff is quite limited. On the other hand, as the number of part-time tutors is massive, the question of having a formal performance appraisal system for the tutors must be carefully examined because it may be costly and time-consuming to implement such a system. The emphasis of performance appraisal should be on staff development and subjective judgement should be minimized. The main purpose of the performance appraisal system is to produce positive outcomes.
Comments from Tutors
As no formal performance appraisal system for tutors is in place, what the course coordinator can do is to have face-to-face interview with the tutor, which can give some clues about the tutor's ability to handle students, his/her attitude toward the way the education was delivered at a distance and his/her maturity in dealing with adult students. Again, as suggested earlier, this informal appraisal meeting is primarily to help identify the problems and difficulties the tutor has had and for the course coordinator to give advice and encourage the tutor to be committed in the future appointment.
Students' Comments about Tutors
Feedback from students, by means of questionnaires and informal discussions, is an effective way to monitor the performance of tutors. Some OLI course coordinators design their own questionnaires for students to give their feedback about their tutors as well as their experience with the course as a whole, the delivery, and the support they receive from the Institute.
Conclusion
In this paper, we have looked into the quality assurance issues in tutoring in the context of distance education. As tutors play an important role in the teaching and have direct interface between the Institute and students, it is vital for us to have good quality assurance in tutors and their work, namely, tutoring. We started to look into the training of tutors with a formal and rigorous training programme, developed in house to meet our requirements. Then we discussed some of the mechanisms of managing and monitoring tutors' work and performance. Finally, we also suggested some ways to evaluate tutors from a staff development point of view rather than for reappointment decision.
One of the essential elements to ensure good quality of tutoring is to keep close contacts between three parties: the course coordinator, the tutors, and the students. If any improvement can be made in terms of communication between them, the quality of our tutors and their work-tutoring should be levelled up even higher. To this effect, the Institute has recently launched a project on developing a Student Electronic Bulletin Board (EBB). The infrastructure of the EBB has been completed and is now being tested for six months with a group of 30 volunteer students. Initial feedback is positive. By the end of September we shall be able to draw a conclusion whether the EBB can enhance the quality of tutoring.
References
Educational Technology Centre, Open Learning Institute of Hong Kong (1991) Internal evaluation report on the OLI Tutor Orientation and Training course. Hong Kong: Open Learning Institute of Hong Kong.
Open Learning Institute of Hong Kong (1992) Institutional submission. Hong Kong: Open Learning Institute of Hong Kong.
Thorpe M. (1986) 'The human dimension in Open University study', Open Learning, June 1986, 14–20
Thorpe M. (1988) Evaluating open and distance learning. UK: Longman Group
Tong, J. (1992) Managing tutors in distance education. Background papers for the ICDE Pre-conference workshop on 'Personal interaction in distance education', held in Hong Kong on 3–5 November, 1992.
| 
