1. The response time test: the time taken to produce a graduate
This test measures the length of time a student is in the sytem [sic] from entry to graduation. The important factors to take into account in applying this test are:
- The number of credits that students need to obtain a particular final qualification.
- The number of credits students are exempted from taking.
- The number of credits students attempt each semester.
- The number of attempts students make at a course before they pass it (the repetition rate).
- The number of terms that elapse between a student's entry and his graduation, including terms in which he was temporarily withdrawn.
Important ratios which can be used as measures of overall efficiency are:
- The ratio of the number of credits attempted in a term to the number of credits assumed to be equivalent to a full course load.
- The ratio of credits passed to credits attempted.
- The ratio of credits passed to the number of credits assumed to be equivalent to a full student course load.
- The ratio of the number of terms in which a student actively studies to the number that have elapsed since first entering the institution.
2. The output-input ratio: The number of graduates as a proportion of the number of students admitted
This test is based on the fact that output is always a function of input. The quantitative measures are in general the easiest to derive. The most common is the output-input ratio: that is, the number of 'graduates' divided by the number of students who entered. This test can be applied within a semester, defining 'graduates' as the number of students who pass a course, or who continue with their studies; or it can be applied in terms of the numbers of students in each cohort who obtain the terminal qualification for which they were registered. Three measures seem to be particularly appropriate:
- The ratio of the number of student courses or credits passed to the number of student courses or credits attempted.
- The ratio of the number of students who were registered at the start of term x who register in term (x + 1), to the number of students who were registered at the beginning of term x less those who graduated at the end of term x.
- The ratio of the number of students in a particular cohort who graduate to the initial number of students in the cohort.
The maximum value which the ratio can have is unity. However, while unity may be an ideal value, educational institutions actually work to an output-input ratio of less than unity, and find it perfectly acceptable to do so. Thus the evaluation, and particularly any judgements based on it, needs to be done on the basis both of the accepted value of the ratio in the educational system as a whole and of the ideal value.
The output-input ratio test fails to identify those students who, while wishing to study a course, do not want to (and never intended to) take the examination; and those who, in taking some of the courses in an academic program, never complete a full degree or diploma or other program.
Qualitative factors also need to be taken into account when drawing conclusions from the quantitative information derived from this test. These factors are much more difficult to determine. However, the following seem to be particularly important:
- The previous educational qualifications of students. Distance teaching institutions vary in their practice with regard to the previous educational qualifications that they require of their applicants. Some institutions (for example, the British Open University) waive entrance requirements altogether, while others require applicants to have the same educational qualifications as those entering comparable TLS. Even so, the quality of the students entering a DLS may be lower than is the case of TLS. For example, although UNED in Costa Rica requires applicants for its degree and diploma programs to have the normal secondary school bachillerato, it does not require its applicants to sit the normal University Entrance Examination, which is a requirement of all conventional university applicants. Thus it seems likely that UNED's students are less academically well prepared than in the case of successful applicants to the conventional universities.
- The number of years elapsed between the end of secondary schooling (or some other educational experience) and the start of the distance teaching course. Two factors are involved here. One is the student's ability to study, which may have been impaired if he or she has been out of the educational system for some time. The other is the student's maturity, which has a bearing on his or her sense of dedication and seriousness towards the requirements which distance education places on the student. Experience in a British Open University pilot study on the admission of students aged 18 to 20 years (the normal minimum age of entry in the OU being 21) showed that the younger students had a very much higher rate of desertion—so much so, in fact, that the project was abandoned. Systems that admit young adults to courses that require a degree of maturity and dedication on the part of the student may in fact reinforce educational failure and disadvantage if the demands made by the system on the student are mor
e than he or she can bear.
- The regional distribution of the students, particularly in respect of the area in which they underwent their schooling. In many countries, students from rural areas are likely to have been educationally deprived relative to those whose schooling took place in urban areas.
- The psychological preparedness of students for studying at a distance. The study techniques required of a student in a DLS place added emphasis on self-reliance and self-discipline, since the student is working at a distance and very often is isolated from both tutors and other students.
- The physical conditions in which students live, particularly those from the lower socio-economic groups, will affect any teaching system that requires home-based study.
Another factor to be considered is the quality of the output relative to the input. The possibility that the 'value added' to the conventional university student is greater than that added to the distance learning student was raised by Carnoy and Levin (1975) and Mace (1978) in relation to studies on the British Open University. Most commentators, however, choose to assume that the value of a degree awarded by a DLS is equivalent to that of any conventional university operating within the same system, assuming always that there are reasonable controls on the overall quality of the educational system at each level. This may or may not be so. Certainly it is very difficult to measure satisfactorily the value added to the student as a result of his studies. Measures of value-added range from those that try to assess the student's gain in knowledge or skills, to those concerned with the economic gain to the individual student and to society. The real problem is to find any wholly satisfactory measures and to e
stablish the extent to which a particular educational experience has had an effect on the student.
3. The correctness of output
This test attempts to evaluate the system's output first in relation to the system's goals, and secondly in relation to the needs and demands of its environments and of its students.
Evaluating output against actual needs has particular relevance for DLS such as the Universidad Nacional Abierta of Venezuela and the Universidad Estatal a Distancia, Costa Rica, both of which aim to provide professional degree and diploma courses (carreras) to meet the needs of their respective countries for trained manpower. Curriculum development emphasises the acquisition of knowledge and skills that will be Of relevance to the student in his career, while at the same time underpinning this with the requisite degree of academic rigour implied by university status. Career orientated goals are not as evident in UNED's Extension Studies Program, and of course, such goals may not apply to all the students registered on a carrera, since some will be studying out of general interest, or to obtain a university qualification without any intention of using it for specific career purposes.
Information is required on the following aspects:
- Needs and expectations.
- The manpower needs, both quantitatively and qualitatively, in terms of requirements (possibly nationally) for (a) the initial training of persons entering a profession, and (b) in-service training for those already practising a profession.
- The personal aims of the students who enter a program. Some students only want to take a few courses, while others wish to gain a formal academic qualification such as a degree or diploma. Some students see their studies as providing them with an initial training for a first career; others may see their studies as leading to a change of career, or to promotion within their existing career structure. A further group of students will take the courses out of general interest. Generally speaking, whatever the expectations of the courses designers are with regard to the needs that they are serving, the heterogeneity of adult students will ensure that the students taking a program will have a variety of expectations in terms of what the program means to them.
- The extent to which needs are being satisfied, or could be better satisfied, by other institutions, and the extent to which such institutions are failing to meet identified needs. Distance teaching is not always an appropriate solution to educational problems. Where manpower needs are very limited the use of high-technology multi-media DLS is unwarranted. It will almost certainly be cheaper to expand conventional campus-based teaching system or to make use of distance teaching methods to expand the 'clientele' of a campus-based institution, as has happened at the University of Waterloo (Leslie, 1979).
- The extent to which the DLS's students achieve their aims, either in terms of successfully completing the courses they set out to take, or in terms of any longer term goals that they may have (such as, for example, entering a profession or being promoted in their job).
Quantitative goals are usually published in planning documents. Generally speaking the most readily quantifiable goals are the numbers of students and graduates forecast for each academic program and the number of courses that have to be developed in each program. So far as student numbers are concerned, two factors are of importance. The first is the level of demand from applicants and the second is the ability of the institution to retain students by minimising dropout. Some institutions offer programs that do not require people to register on them in order to follow them (for example, some courses within UNED's Extension Studies Program). In such cases book sales or audience viewing figures give an indication of the extent of interest in a course.
|
