The Ordinance establishing the OLI also set up its Council of individuals from various Hong Kong business and education sectors and its Academic Board of representatives from the three Schools and senior management. Figure 1 displays the hierarchical structure for academic decision making as well as the external involvement of academic peers. Insert Figure 1

Figure 1. The structure of the OLI

(a list of abbreviations is given at the end of this paper)

A School Committee consists of all academic staff from the School whilst Program Teams involve all staff from a School with the specialisation appropriate to that program. Each team includes a member of the Educational Technology Centre and is supported by its advisory group (APG). The roles of the Course Review Committees (CRC) and the Award Committees are described below.

Quality assurance in the OLI: The degree program

One of the major recommendations from CNAA was that the Institute should endeavour to seek as much external advice on the development of its programs as possible. The original proposals for programs from the PCOLI had been produced by consultants before the arrival of any academics and CNAA expressed concern about both the Institute's ownership of the programs and the manner in which the details had been decided. Subsequently for each of its programs each School established an Advisory Peer Group (APG), consisting of subject specialists, not only from the other Hong Kong tertiary institutions but also from government, commerce and industry where appropriate. The APG discussed with the Program Team, the appropriate direction for the program content to take, in view of the current and future offerings in the subject by the other institutions and the needs of Hong Kong for graduates in that area.

The nature of program development has varied by School and by program. The PCOLI envisaged that the programs in Western Arts and Science and Technology would consist of courses taken primarily from the British Open University (OU). The arguments for this policy included

• the need to present courses in three months from establishment;
• the need to expand the number of course offerings each semester;
• the need to keep staff numbers low, in view of the financial objectives of the Institute;
• the difficulty for suitable courses to be prepared in house, especially when such courses were already in existence.

Whereas there are a number of institutions from which courses could have been taken, the OU was chosen because using only one institution reduces the problems of prerequisites, it guarantees uniformity of style and presentation, and the OU has a large number of internationally recognised courses developed to a very high standard, both in terms of subject content and support for the independent learner. It is only fair to add that the OU had provided assistance and advice to the PCOLI and that advisers to the PCOLI had direct experience of offering OU courses in the region. But it must be re-emphasised that the Institute had to commence operations in a very short time and with relatively very few full time staff. Therefore, whereas some may consider the choice of the OU as one of convenience rather than of deliberate selection through international comparison, the Institute made that choice in the knowledge that the OU is seen as one of the top providers of quality distance education in the world.

The APGs in Western Arts and in Science and Technology were asked to select those OU courses that were considered suitable and appropriate to the program in light of its aims, objectives and structure. Where necessary, the group was encouraged to consider the question of the cultural appropriateness of a particular course in terms of its use in Hong Kong. In general, courses in mathematics were felt to be culturally independent, but concern was expressed about courses in science in which, for example, European flora and fauna would be used in biology. For a course where the APG was unhappy about the approach taken, the group would advise the Program Team as to what needed to be done to the course and that course would then be identified as one requiring adaptation before use.

Where an APG was unable to find sufficient suitable OU courses to meet the aims and objectives of the program, it identified the main requirements of each missing course for the Program Team. The Program Team then needed either to identify a suitable course from another institution, or to prepare a detailed syllabus for a course to be written.

The above approach was not appropriate for the Business program, as neither the OU nor any other institution has suitable undergraduate courses in this discipline. In addition, many of the subject areas would need to cover the Hong Kong situation, which would differ from those found elsewhere. For example, business law, practices in personnel management and accounting have elements and emphases unique to Hong Kong. The PCOLI therefore accepted that the School of Business would need either to carry out major adaptations of courses from elsewhere or to commission the writing of its own courses. This meant that the APG was more involved with the definition of individual course syllabuses than those from the other Schools.

Once a Program Team and APG have agreed upon the structure and content of the program, the Team prepares a Detailed Program Proposal (DPP) outlining the program and its aims, describing course syllabuses, detailing the source of courses and the nature of any adaptation to be undertaken, listing the schedule for course presentation, and indicating the financial consequences of developing and running the program. The DPP is then considered by the School Committee, which must take into account the program's interaction with the other programs in the School, as well as the availability of staff to support it. Upon approval, the DPP is then passed on to the Program Review and Validation Committee (PRVC).

Membership of the PRVC reflects the emphasis placed on external involvement in the Institute's self validation procedures. Three external peers of high standing are members, as well as the OLI Director, AD (Academic), plus representatives from the three Schools. The PRVC's role is to ensure that a School has undertaken appropriate steps to ensure that its program proposals have been prepared according to the established procedures. In considering this, the Committee takes special notice of the notes of the meetings of the APG and Program Team in order to determine which factors were considered and which issues were raised. The PRVC submits a report on the DPP, including a recommendation on whether or not the program should be approved, to the School Committee and Academic Board. Next, Academic Board considers the DPP and the PRVC report with any response from the School Committee. If the Board approves the program, then the DPP is forwarded to Council for final consideration and a decision on whether the program can be offered. The work of the APG and PRVC does not end here. As the Program Team continues with the development of the program, it will call upon the APG to advise it on individual courses as necessary and to discuss with it the need to alter the program as the subject changes, as new distance education courses from other institutions become available and as the Hong Kong context alters. Each program will be reviewed by the PRVC every five years to determine whether suitable progress has been made and to consider whether any amendments need to be made.

Development of courses, or degree modules

As indicated above there are three types of course development, courses imported unchanged, courses imported and adapted, and courses written by or for the Institute. Clearly the development of each type differs, but the procedures by which any course is approved for presentation are the same. As with a program, the Institute involves external peers in the approval of courses. Each course must be reviewed by an External Course Assessor (ECA), who must comment on the quality of the course content in terms of both the treatment of the subject and the design of the course for use by independent learners.

The School Committee establishes a Course Review Committee (CRC) for the course, consisting of academics within the Institute whose own disciplines relate to the course content, plus a member of the Institute's Education Technology Centre (ETC). The CRC prepares a written report for the School Committee, which includes the ECA's report as well as a detailed report by one of the subject experts on the Committee and a report by the ETC member. The CRC's report must consider the soundness of the academic treatment of the subject, a comparison of the standard and content of the course with appropriate courses elsewhere, the contribution to the programs in which it has a defined place, the relevance and appropriateness of the treatment in the context of Hong Kong, and the quality of the course as a learning and teaching tool.

The CRC's report is considered by the School Committee, which on acceptance, forwards the report to the Academic Board and AD (Academic). The latter reviews the course, including the delivery and assessment strategies and then submits a separate report with recommendations to the Academic Board. The Board then decides whether or not to approve the course for presentation.

The development of a course does not end with its first presentation. At the end of each presentation, the coordinator or academic responsible for running the course prepares a report reviewing the problems occurring during the course, the success or failure of any part, and the performance of students. The report will detail any proposed changes to the course content, presentation or structure and will include the report from the External Examiner (see below). In addition, the Education Technology Centre will produce an independent evaluation based on a student survey. Both reports are considered by the School Committee before being passed to the AD (Academic), who summarises the reports from all of the courses for Academic Board. The Board then determines what action, if any, is required before any future presentation is permitted.

Presentation of courses

Whereas the control of quality in course preparation is an important element of the Institute's quality control procedures, it is the actual presentation of the course which provides the essential quality for the individual student. Although considerable effort is taken to design and write the course material for the independent learner, there will inevitably arise difficulties for the individual student which are not catered for by the material. With student numbers on courses in the hundreds, it is not feasible for the coordinator to act as problem solver and adviser for the students. Therefore the Institute employs part time tutors to fulfil this role for each course. These individuals are qualified in the subject, and are usually employed full time elsewhere.

A tutor is normally responsible for 30 students and marks their assignments, holds tutorials and is available to provide immediate assistance over the telephone. Assignments are seen by the Institute as the main source of help to the individual student. Questions are designed not only to provide the student with practice, but also to give tutors the opportunity to identify any weaknesses or errors in the student's understanding of the course content. Tutors are required to write comments on the script to indicate such problems, to provide alternatives, or to expand ideas. Such comments must not simply repeat the correct answers, as provided in the supplied marking scheme, but must aim to guide students towards producing the corrections for themselves. Tutorials are held at regular intervals during a course. The frequency and content of tutorials will depend on the level and size of the course and the nature of the subject itself. First level courses have sessions every 3 or 4 weeks, while at the hig her level this is reduced to every 6 or 8 weeks. Mathematics tutorials are primarily example classes and Arts tutorials often involve guest lecturers. Science courses usually require students to attend practical, laboratory sessions. To supplement tutorials, some courses hold weekly "surgeries" at which one or more tutors are available to meet with students on a first come, first served basis to help with problems. Telephone contact is a special feature of the OLI, as local phone calls in Hong Kong are free and virtually every residence has a telephone. It is therefore relatively easy for a student to contact a tutor about a problem at the time it arises. However, the usefulness of such contact is dependent upon the subject. Discussing mathematical expressions can create confusion. Tutors are encouraged to maintain contact with students over the telephone to ensure that they are keeping to the course schedule, especially in regard to assignment submission.

The work of a tutor contains elements requiring skills new to many tutors, notably using a student's scripts as a teaching opportunity. To assist new tutors, the Institute runs a training program to introduce them to the Institute, distance education and the various components of their work. This training continues during the course with the coordinator holding briefing sessions to discuss the coming features of the course, particular problems or problems which may arise, and to answer questions from the tutors. The coordinator monitors a sample of marked scripts for each assignment from each tutor, to determine whether the marking has been done according to the supplied marking scheme and whether the teaching comments are of the required quality. Tutors are seen as members of a team and the coordinator seeks from them views about how the course and its presentation can be improved.

Unlike most conventional degree programs for which one External Examiner reviews the entire set of courses, at the OLI each individual course must have an External Examiner (EA), a senior academic from a local or overseas institution. Once appointed, the EA receives all of the course material including the assignments and marking schemes. Samples of marked scripts from each tutor are sent during the course to the EA, so that he or she can assess the standard of marking. The EA is a member of the Award Committee, which is firstly responsible for finalising the final examination paper and its marking scheme. Following the final examination, both the coordinator and EA monitor samples of the marked papers in order to ensure uniformity of marking and that it is of the appropriate standard. Upon agreement on the marks, the Award Committee determines the grades. To ensure uniformity of grading across courses within a School, all Award Committees are chaired by the respective Dean. The grades for each cour se require the approval of Academic Board, which receives a report from the EA including any criticisms and recommendations.

It's all fine in theory, but . . .

Whilst the staff of the Institute have established systems and procedures to try to ensure the quality of its material, course presentation and programs, in accordance with the original aims of the PCOLI, there is one constraint which has had a restrictive effect on quality control. The Institute charges students fees, but the Hong Kong Government never expected these fees to fund the Institute during the initial setting up period. However, the Government does expect the Institute to cover its costs increasingly from fees and subsequently has been decreasing its subvention in order to reach zero after four years, ie, in 1993. This situation is unique, we believe. All other open, distance learning organisations that we know enjoy some level of direct or indirect subsidy which enables the fees to be set below the "full cost" level. Even then, the payment of fees is a serious barrier to study for many students and limits the extent to which the education can be said to be truly open to all who want it. At the OLI, students are already paying high fees in relation to average earnings, despite the current level of Government subvention. The Government pays HK$20 million in an annual budget of HK$150 million. OLI students pay HK$7,600 for a 20 credit, year long course. A survey reveals that 73% of the students earn less than HK$10,000 per month and yet the average rate of study is about 15 credits per year and many students are studying 40 credits per year, which is equivalent to the workload of a full time student. These fee levels are planned to rise by 18% later this year as the Institute strives to balance income and expenditure. In raising its fees, the Institute must ensure that the price elasticity of demand does not cause the increase in fees to result in a reduction in the total fee income.

The need to meet the financial objectives of PCOLI has placed considerable pressure on the Institute to minimise costs. The largest single element is salaries and the Institute finds itself having to plan, prepare, present, review and revise courses with relatively few academic and support staff. Subsequently, especially for new courses requiring major adaptation or writing from scratch, course approval procedures have come under pressure to be relaxed as presentation dates approach and the course has not been completed. This situation is worsened by delays in the employment of coordinators. The work put into course review suffers from the high workload on staff, who perhaps do not have the time and energy to review their courses in detail. Ensuring that academic standards are maintained and that new courses are presented to meet student demand are the first priorities. Tutor costs are also a major factor in the Institute's budget and the pressure to keep these costs down indirectly affects the quali ty of tutors' work, especially on assignments.

Clearly the number of courses offered is another major financial factor and Schools have had to keep the number of courses planned per program down to the minimum required to graduate, during the first 5 years at least. This has reduced, if not removed, the number of options within a program and in some cases threatens the ability of the School to meet the program's aims and objectives as set out with the APG. These restrictions are of concern to the staff of the Institute. Standards and quality have not yet seriously been compromised, but if the situation continues, then they will be in danger.

Quality assurance at the Open University

This has been described at length in the literature (eg, Kaye and Rumble, 1981) and the methods of control used are quite different from those used at the OLI. This is, no doubt, a consequence of the greater autonomy enjoyed by the OU, which has a charter to awards its own degrees and which is not subject to external validation by CNAA. The OU does not offer named degree programs, so there is no need for quality assurance at the program level, and the concern for quality is to be found at the level of individual courses.

So where does the quality control of course development occur at the OU? We believe that it occurs mostly within the Course Team (CT) in the form of internal peer review and criticism. The typical CT comprises several subject experts, an educational technologist, an editor, a graphic designer, a TV producer and a course manager and it assumes a collective responsibility for the quality of its products. All its work is iterative, with each text and script going through several draft stages. It might be suspected that members of a CT might (for an easy life) collude with one another to mute criticism. In practice this is rare and members have learned to give, and take, very frank criticism. One might say that the OU course team is a very effective "quality circle" and that the high reputation of the University rests largely on this. At the OU, course development is a very labour intensive process, which can be supported only by a generously funded organisation. The only similarity with the OLI is the use of an External Course Assessor for each course. The OU has no counterpart to the OLI's Course Review Committee.

By contrast, course development at the OLI is usually in the hands of a single course writer, and a course designer. Since it cannot call upon a big team for review, the OLI has perceived the need to formalise the course review process in the manner described above. The other important aspect of quality assurance is the control of exit standards in the form of continuous assessment and examinations. In this respect, there is a close similarity between the OLI and the OU. Indeed, the Institute's procedures are modelled upon those of the OU, and in both institutions an External Examiner is employed on each course.

Comparing east with west

From the students' point of view, the criterion of fitness for purpose is whether the education received enables them to achieve their objective of gaining credit for the courses they have studied. It is interesting to compare the proportions of enrolled students gaining credit from a given course studied at the OLI in Hong Kong and at the OU in the U.K. Table 2 lists a number of courses which have been offered at both establishments in identical forms within the last five years. The overall measure of success, the proportion of enrolled students who passed, is shown together with the proportion presenting themselves for the final examination.