Aproaches to Evaluation

Approaches to evaluation has involved an examination of the context in which the program occurs, of  the goals, syllabus, and structure of the course, and how these can be planned and developed, as well as analysis of teaching and learning that takes places during the course. The focus throughout has been analysis of the different factor that determine the successful design and implementation of language program and language teaching materials. This overall and interlinked system of elements (i.e., needs, goals, teachers, learners, syllabuses, materials, and teaching) is known as the second language curriculum.

 A number of question need to be answered :

1.  Is the curriculum achieving its goals? 

2.  What is happening in classroom and schools where it is being implemented?

3.   Are those affected by curriculum (e.g., teacher, administrators, students, parents, employers) satisfied with the curriculum?

4.  Have those involved in developing and teaching a language course done satisfactory job?

5.  Does the curriculum compare favorably with others of its kinds?

Curriculum Evaluation is concerned with:

It focus on collecting information about different aspect of a language program in order to understand how the program works, and how successfully it works, enabling different kinds of decision to be made about the program, such as whether the program responds to learners’ needs, whether further teacher training is required for teachers working in program, or whether students are learning sufficiently from it. 

Evaluation may focus on many different aspects of language program:

             Curriculum design : to provide insights about the quality of program planning and organization.

             The syllabus and program content: for example how relevant and engaging it was, how easy or difficult, how successful tests and assessment  procedures were

             Classroom processes : to provide insight about the extent to which a program is being implemented appropriately.

             Materials of instruction : to provide insights about whether specific materials are aiding student learning

             The teacher : for example how they conducted their teaching, what their perceptions were of the program, what they taught

             Teacher training : to asses whether training teachers have received is adequate

             The students : for example what they learned from the program, their perception of it, and how they participated in it

             Monitoring of pupil progress : to conduct formative (in-progress) evaluations of student learning

             Learner motivation : to provide insight about the effectiveness of teachers in aiding students to achieve goals and objectives of the school.The institution : for example, what administrative support was provided, what resources were used, what communication networks were employed

             learning environment : to provide insights about the extent to which students are provided with a responsive environment in terms of their educational need.

             Staff development : to provide insights about the extent to which the school system provides the staff opportunities to increase their effectiveness

             Decision making : to provide insights about how well the school staff-principals, teachers, and others-make decision that result in learner benefits.

                     (Sanders 1992; Weir and Roberts 1994)

Since the 1960s, curriculum evaluation has become of increasing interest to educators and curriculum planners.The scope of evaluation has moved from a concern with test results to the need to collect information and make judgments about all aspects of the curriculum, from planning to implementation.                     

                                                                ( Hewings and Dudley-Evans 1996)

The scope of curriculum evaluation:

Collect information about all aspect of the curriculum

(needs, goals, teachers, learners, syllabuses, materials, teaching)

                                      

Make judgments about all aspect of curriculum

(for example: the method apply is appropriate with the material, students learn sufficiently, teacher training is not required for teacher working in the program   )

Purposes of evaluation

There are 2 major purposes for language program evaluation:

1.  Program accountability

2.  Program development

3   Accountability-oriented evaluation usually examines the effects of a program or project at significant end points of an educational cycle and is usually conducted for the benefit of an external audience or decision maker.

4          Development-oriented evaluation is designed to improve the quality of a program as well as others who are not and may have a teacher-development focus.

                                                                                                                                                 (Weir and Roberts 1994, 5)

The different purposes for evaluation are referred to:

1. Formative

2. Illuminative

3. Summative

A.            FORMATIVE EVALUATION

Evaluation  may be carried out as part of the process of program development in order to find out what is working well, and what is not, and what problem need to be addressed. (It focuses on ongoing development and improvement of the program )

Typical questions that relate to formative evaluation are :

             Has enough time been spent on particular objectives?

             Have the placement tests placed students at the right level in the program?

             How well is the textbook being received?

             Is the methodology teachers are using appropriate?

             Are teachers or students having difficulties with any aspect of the course?

             Are students enjoying the program? If not, what can be done to improve their motivation?

             Are students getting sufficient practice work? Should the workload be increased or decreased.

             Is the pacing of the material adequate?

Information collected during formative evaluation is used to                address

problem  that have been identified                  improve the delivery program   

Example 1 :

             During the implementation of a new primary course in EFL context it is found that rather than using the task-oriented communicative methodology that provides the framework for the course, a number of teachers are resorting to a teacher-dominated drill and practice mode teaching that is not in harmony with the course philosophy (information)

             In order to address this problem a series of Saturday morning workshops are held to identify the kinds of problems teachers are having with the materials.

             Videos are used to model more appropriate teaching strategies and teachers agree to attempt to implement in their classroom some of the techniques they have seen demonstrated and to report back on their experiences at subsequent workshop.

Example 2 :

             A few weeks after a course on integrated skills has started. It is found that there are different perceptions of what the priorities in the course (teachers are spending very different amount of time on different components of the course and emphasizing different things)

             A series of meeting are held to review teachers ’understanding of the course objectives and further clarify the weighting that should be given to different course components.

             Peer observation is suggested as a way for teachers to compare teaching styles  and priorities and to enable them to achieve a consensus concerning teaching practices

Example 3 :

•             A 10-week course on conversation skills has been started for a group of low-level learners. Pronunciation is not a major element of the course because it is assumed that most pronunciation problems will short themselves out after a few weeks.  

•             Four weeks after the course has commenced, teachers report that a number of students have persistent and major pronunciation problems that the course is not addressing.

•             It is decided to refocus one section of the course to include a pronunciation component. Individual diagnostic sessions are held with students who have the most serious pronunciation problem, and laboratory works as well as classroom time is alloted to systematic pronunciation work for the reminder of the course

B.            ILLUMINATIVE EVALUATION

Illuminative evaluation is another kind of evaluation to find out how different  aspects  of  the program work of the teaching learning processes that happen in the program.

Frame work :

1.            How do students to do group work tasks? Do all students participate equally in them?

2.            What type of error-correction strategies do teachers use?

3.            What kind of decisions do teachers employ while teaching?

4.            What type of  teacher - student   interaction pattern typically occur in classes?

5.            What reading strategies do students use with different kinds of texts?

6.            How do students understand the teacher’s intention during a lesson?

7.            Which students in a class are most or least active?

Type of illuminative evaluation is classroom action research / teacher inquiry:

For example:

1.            Teaching a course on reading skills, such as : skimming, scanning, reading for details, surveying a text, critical reading and vocabulary development.

2.            Classroom observation

3.            Group work

 In Classroom action research, something that the teachers have to understand about their classes :

1.            What learning strategies were used by successful learners in their classes?

2.            Do the learners use English outside of the classroom?

3.            Do they feel good about learning English?

In Classroom action research, the teacher got information  by :

•             Using classroom observation

•             Learner journals

•             Interviews

The strategy to be  more effectively facilitate students’ learning

•             How did you go about doing this ?

•             Which way of doing this works best for you ?

C.            SUMMATIVE EVALUATION

It is kind of the evaluation to make decisions about the worth or value of different aspects of curriculum. It takes place after a program has been implemented.

Summative evaluation is concerned with determining the effectiveness of a program, its efficiency, and to some extent with its acceptability. It takes places after the program has been implemented and seeks to answer question such as these:

•             How to effective was the course? Did it achieve the aim?

•             What did the students learn/

•             How well was the course received by students and teacher?

•             Did the materials work well?

•             Were the objective adequate or do they need to be revised?

•             Were the placement and the achievement test adequate?

•             Was the amount of time spent on each unit sufficient?

•             How appropriate were the teaching methods?

•             What  problem were encountered during the course?  

How to measure the effectiveness of the lesson :

•             Mastery of objectives

•             Performance on tests

•             Measures of acceptability

•             Retention rate or reenrollment rate

•             Efficiency of the course

source : Jack C. Richards, Curriculum Development In Language Teaching, resume chapter 9.