designing classroom language test

DESIGNING CLASSROOM LANGUAGE TESTS

TEST TYPES

Defining your purpose will help you choose the right kind of test, and it will also help you to focus on the specific objectives of the test. 

Below are the test types to be examined:

• Language Aptitude Test
• Proficiency Test
• Placement Test
• Diagnostic Test
•  Achievement Test 

1.  Language Aptitude Test

· A language aptitude test is designed to measure capacity or general ability to learn a foreign language.

· Task in MLAT includes: Number learning, phonetic script, spelling clues, word in sentence, and repaired associates.

· There’s no unequivocal evidence that language aptitude test predict communicative success in a language .

· Any test that claims to predict success in learning a language is undoubtedly flawed

2     2. Proficiency Test

· A proficiency test is not limited to any one course, curriculum, or single skill in the language ; rather , it test overall ability.

· It includes: standardized multiple choice items on grammar, vocabulary, reading comprehension, and aural comprehension.

· Proficiency test are almost always summative and norm-referenced.

· They are usually not equipped to provide diagnostic feedback.

· Their role is to accept or to deny someone’s passage into the text stage of a journey.

3    3. Placement Test

· The ultimate objective of a placement test is to correctly place a student into a course or level.

· A placement test usually includes a sampling of the material to be covered in the various courses in a curriculum.

· In a placement test , a student should find the test material neither too easy nor too difficult but appropriately challenging.

· The English as a Second Language Placement Test ( ESLPT ) at San Francisco State University has three parts, part 1: student read a short article and then write a summary essay. Part 2: student write a composition in response to an article. Part 3: multiple choice; students read an essay and identify grammar errors in it.

4     4.Diagnostic Tests

· A diagnostic test is designed to diagnose specified aspects of a language.

· A diagnostic test can help a student become aware of errors and encourage the adoption of appropriate compensatory strategies.

· A typical diagnostic test of oral production was created by Clifford prator (1972) to accompany a manual of English pronunciation.

- Test-takers are directed to read a 150-word passage while they are tape recorded.

- The test administrator then refers to an inventory of phonological items for analyzing a learner’s production.

-After multiple listening, the administrator produces a checklist for errors in five separate categories.

-Stress and rhythm.

-Intonation,

-Vowels,

-Consonants, and

 -Other factors.

5    5. Achievement Test

· An achievement test is related directly to classroom lessons, units, or even a total curriculum.

· Achievement tests should be limited to particular material addressed in a curriculum within a particular time frame and should be offered after a course has focused on the objectives in question.

· The primary role of an achievement test is to determine whether course objectives has been meet – end of a period of instruction.

· Achievement test are often summative because they are administered at the end f a unit or term of study. But effective achievement tests can serve as useful wash back by showing the errors of a students and helping them analyze their weaknesses and strengths.

· Achievement test range from five-or ten-minutes quizzes to three hour final examinations, with an almost infinite variety of item types and formats.

· Some practical steps in constructing classroom tests:

-     Assessing Clear, Unambiguous Objective.

Before giving a test; examine the objectives for the unit you’re testing. Your first task in designing a test, then, is to determine appropriate objectives.

Example: “ students will recognize and produce tag question, with the correct grammatical from and final intonation pattern, in simple social conversation.“

-    Drawing Up Test Specification

Test specifications will simply comprise

- A broad outline of the test

-What skill you will test

-What the items will look like

This is an example for test specification based on the objective stated above:

“Student will recognize and produce tag questions, with the correct grammatical form and final intonation pattern, in simple social conversation.”

Test specification

1. Speaking (5 minutes per person, previous day)

Format: oral interview, T and S

Task: T ask question to S

2. Listening (10 minutes)

Format: T makes audiotape in advance, with one other voice on it

Tasks: a. 5 minimal pair items, multiple choice

3. Reading (10 minutes)

Format: cloze test items (10 total) in a story line

Task: fill inn the blanks

4. Writing (10 minutes )

Format: prompt for a topic: why I like /didn’t like a recent TV sitcom

Task: writing a short opinion paragraph

- Devising Test Task

As you devise your test items, consider such factors as

-How students will perceive them (face validity)

- The extent to which authentic language and contexts are present

- Potential difficult caused by cultural schemata

- Designing Multiple-Choice Test Items

There are a number of weaknesses in multiple-choice items:

- The technique tests only recognition knowledge.

- Guessing may have a considerable effect of test scores.

- The technique severely restricts what can be tested.

- It is very difficult to write successfully items.

- Wash back may be harmful.

v Cheating may be facilitated

Some important jargons in Multiple-Choice Items:

1. Multiple-choice items are all receptive, or selective, that is, the test taker chooses from a set of responses rather than creating a response. Other receptive items types include true-false question and matching lists.

2. Every multiple-choice item has a stem, which presents several option or alternatives to choose from.

SCORING, GRADING AND GIVING FEEDBACK

A. Scoring

As you design a classroom test, you must consider how t test will be scored and graded. Your scoring plans reflect the relative weight that you place on each section and items in each section. The integrated skills class that we have been using as an example focuses on listening and speaking skills with some attention to reading and writing. Three of your nine objectives target reading and writing skills. How do you assign scoring to the various components of this test?

Because oral production is a driving force in your overall objectives, you decide to place more weight on the speaking (oral interview) section than on the other three sections. Five minutes is actually a long time to spend in a one-on-one situation with a student, and some significant information can be extracted from such a session. You therefore designate. 40 percent of the grade to the oral interview. You consider the listening and reading sections to be equally important, but each of them, especially in this multiple choice format, is of less consequence than the oral interview. So you give each of them a 20 percent weight. That leaves 20 percent for the writing section, which seems about right to you given the time and focus on writing in this unit of the course.

Your next task is to assign scoring for each item. This may take a little numerical common sense, but it doesn’t require a degree in math. To make matters simple, you decide to have a 100-point test in which.

-    The listening and reading items are each worth 2 points.

-   The oral interview will yield four scores ranging from 5 to 1, reflecting fluency, prosodic features, accuracy of the target grammatical objectives, and discourse appropriateness. To weight these scores appropriately, you will double each individual score and then add them together for a possible total score of 40.

-   The writing sample has two scores: one for grammar/mechanics (including the correct use of so and because) and one for overall effectiveness of the message, each ranging from 5 to 1. Again, to achieve the correct weight for writing, you will double each score and add them, so the possible total is 20 points.

Here are your decisions about scoring your test:

                                    Percent of                                                       Possible Total

                                    Total Grade                                                   Correct

Oral Interview                  40%            4 scores, 5 to 1 range x 2        = 40    

Listening                          20%            10 items @ 2 points each        = 20

Reading                            20%            10 items @ 2 points each        = 20

Writing                             20%            2 scores, 5 to 1 range x 2        = 20

Total                                                                                                        100

At this point you may wonder if the interview should carry less weight or the written essay more, but your intuition tells you that these weights are plausible representations of the relative emphases in this unit of the course.

After administering the test once, you may decide to shift some of these weights or to make other changes. You will then have valuable information about how easy or difficult the test was, about whether the time limit was reasonable, about your student’s affective reaction to it, and about their general performance. Finally, you will have an intuitive judgment about whether this test correctly assessed your students. Take note of these impressions, however no empirical they may be, and use them for revising the test in another term.

B. Grading

Grading doesn’t mean just giving “A” for 90-100, and a “B” for 80-89.

It’s not that simple . How you assign letter grades to a test in a product of

- The country, culture, and context of the English classroom,

- Institutional expectations (most of them unwritten),

- Explicit and implicit definitions of grades that you have set forth,

- The relationship you have established with the class, and

- Student expectations that have been engendered (cause) in previous test and quizzes in the class.

C. Giving feedback

Feedback should become beneficial wash back. Those are some examples of feedback:

1. A letter grade

2. A total score

3. Four sub scores (speaking, listening, reading, writing)

4. For the listening and reading sections

- An indication of correct/incorrect responses

- Marginal comments

5. For the oral interview

- Scores for each element being rated

- A checklist of areas needing work

- A post-interview conference to go over the results

6. On the essay

- Scores for each element being rated

-A checklist of areas needing work

- Marginal and end –of-essay comments, suggestions

- A post-test conference to go over work

- A self-assessment

7. On all or selected parts of the test, peer checking of results

8. A whole-class discussion of results of the test

9. Individual conferences with each student to review the whole test

Ways of Assessing Learners

Progress and achievement can also be assessed through written work (done in class or for homework), class activities such as simulations or oral presentations, group projects, and self assessment by the learners themselves. Clear assessment criteria are needed. If the results are to be included in the overall course assessment of the learners, the criteria should be understood and agreed upon by all the teachers involved.

CONCLUSION

There are five kinds of test types: Language aptitude tests, proficiency tests, placement tests, diagnostic tests, and achievement tests. Every test must be a wonderfully innovative instrument that will garner the accolades of the colleagues and the admiration of the students.

In the test, we have some practical steps to test construction, they are: assessing clear and unambiguous objectives, drawing up test specifications, devising test tasks, and designing multiple-choice test items.

Evaluation can fulfill two functions: assessment and feedback. Assessment is a matter of measuring what the learners already know. Any assessment should also provide positive feedback to inform teachers and learners about what is still not known, thus providing important input to the content and methods of future works.

 reference :  http://desvieta.blogspot.co.id/2010/04/designing-classroom-language-tests.html
 http://febbyeni.blogspot.co.id/2013/06/designing-classroom-language-test.html

Principles language assessment

Assessment has profited from the advances that have taken place in statistical analysis over the last century and it is now considered important for assessors to check their tests empirically as well as rationally. In addition, the upsurge in interest in the results of tests and examinations by governments, parents and candidates has led to a multi-million pound testing industry and contemporaneously there is a huge increase in the numbers of scholars researching different aspects of testing and assessment.

Although trends in testing, as in other fields, change over time, some principles of assessment are permanent and are not overly affected by current fashions. They need to be held in mind by assessors whenever they construct a test, whether this be a class quiz, a class essay or an end-of-year examination. (Here the terms 'test' and 'assessment' are used interchangeably.) The most important of these principles fall under three heads: validity, reliability and washback. Whilst the following discussion focuses specifically upon language assessment, the principles of testing apply more generally.

Since the definition of assessment has been understood, the next items for teachers to comprehend are principles of language assessment. To design good assessment, teachers should pay attention to validity, reliability, practicality,authenticity, and wash back. Each of them is further explained below.

A. Practicality

An effective test is practical. This means that it

• Is not excessively expensive,
• Stays within appropriate time constraints,
• Is relatively easy to administer, and
• Has a scoring/evaluation procedure that is specific and time-efficient.

A test that is prohibitively expensive is impractical. A test of language proficiency that takes a student five hours to complete is impractical-it consumes more time (and money) than necessary to accomplish its objective. A test that requires individual one-on-one proctoring is impractical for a group of several hundred test-takers and only a handful of examiners. A test that takes a few minutes for a student to take and several hours for an examiner too evaluate is impractical for most classroom situations.

B. Reliability

A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test should yield similar result. The issue of reliability of a test may best be addressed by considering a number of factors that may contribute to the unreliability of a test. Consider the following possibilities (adapted from Mousavi, 2002, p. 804): fluctuations in the student, in scoring, in test administration, and in the test itself.

• Student-Related Reliability

He most common learner-related issue in reliability is caused by temporary illness, fatigue, a “bad day,” anxiety, and other physical or psychological factors, which may make an “observed” score deviate from one’s “true” score. Also included in this category are such factors as a test-taker’s “test-wiseness” or strategies for efficient test taking (Mousavi, 2002, p. 804).

• Rater Reliability

Human error, subjectivity, and bias may enter into the scoring process. Inter-rater reliability occurs when two or more scores yield inconsistent score of the same test, possibly for lack of attention to scoring criteria, inexperience, inattention, or even preconceived biases. In the story above about the placement test, the initial scoring plan for the dictations was found to be unreliable-that is, the two scorers were not applying the same standards.

• Test Administration Reliability

Unreliability may also result from the conditions in which the test is administered. I once witnessed the administration of a test of aural comprehension in which a tape recorder played items for comprehension, but because of street noise outside the building, students sitting next to windows could not hear the tape accurately. This was a clear case of unreliability caused by the conditions of the test administration. Other sources of unreliability are found in photocopying variations, the amount of light in different parts of the room, variations in temperature, and even the condition of desks and chairs.

• Test Reliability

Sometimes the nature of the test itself can cause measurement errors. If a test is too long, test-takers may become fatigued by the time they reach the later items and hastily respond incorrectly. Timed tests may discriminate against students who do not perform well on a test with a time limit. We all know people (and you may be include in this category1) who “know” the course material perfectly but who are adversely affected by the presence of a clock ticking away. Poorly written test items (that are ambiguous or that have more than on correct answer) may be a further source of test unreliability.

C. Validity

By far the most complex criterion of an effective test-and arguably the most important principle-is validity, “the extent to which inferences made from assessment result are appropriate, meaningful, and useful in terms of the purpose of the assessment” (Ground, 1998, p. 226). A valid test of reading ability actually measures reading ability-not 20/20 vision, nor previous knowledge in a subject, nor some other variable of questionable relevance. To measure writing ability, one might ask students to write as many words as they can in 15 minutes, then simply count the words for the final score. Such a test would be easy to administer (practical), and the scoring quite dependable (reliable). But it would not constitute a valid test of writing ability without some consideration of comprehensibility, rhetorical discourse elements, and the organization of ideas, among other factors.

• Content-Relate Evidence

If a test actually samples the subject matter about which conclusion are to be drawn, and if it requires the test-takers to perform the behavior that is being measured, it can claim content-related evidence of validity, often popularly referred to as content validity (e.g., Mousavi, 2002; Hughes, 2003). You can usually identify content-related evidence observationally if you can clearly define the achievement that you are measuring.

• Criterion-Related Evidence

A second of evidence of the validity of a test may be found in what is called criterion-related evidence, also referred to as criterion-related validity, or the extent to which the “criterion” of the test has actually been reached. You will recall that in Chapter I it was noted that most classroom-based assessment with teacher-designed tests fits the concept of criterion-referenced assessment. In such tests, specified classroom objectives are measured, and implied predetermined levels of performance are expected to be reached (80 percent is considered a minimal passing grade).

• Construct-Related Evidence

A third kind of evidence that can support validity, but one that does not play as large a role classroom teachers, is construct-related validity, commonly referred to as construct validity. A construct is any theory, hypothesis, or model that attempts to explain observed phenomena in our universe of perceptions. Constructs may or may not be directly or empirically measured-their verification often requires inferential data.

• Consequential Validity

As well as the above three widely accepted forms of evidence that may be introduced to support the validity of an assessment, two other categories may be of some interest and utility in your own quest for validating classroom test. Messick (1989), Grounlund (1998), McNamara (2000), and Brindley (2001), among others, underscore the potential importance of the consequences of using an assessment. Consequential validity encompasses all the consequences of a test, including such considerations as its accuracy in measuring intended criteria, its impact on the preparation of test-takers, its effect on the learner, and the (intended and unintended) social consequences of a test’s interpretation and use.

• Face Validity

An important facet of consequential validity is the extent to which “students view the assessment as fair, relevant, and useful for improving learning” (Gronlund, 1998, p. 210), or what is popularly known as face validity. “Face validity refers to the degree to which a test looks right, and appears to measure the knowledge or abilities it claims to measure, based on the subjective judgment of the examines who take it, the administrative personnel who decode on its use, and other psychometrically unsophisticated observers” (Mousavi, 2002, p. 244).

D. Authenticity

An fourth major principle of language testing is authenticity, a concept that is a little slippery to define, especially within the art and science of evaluating and designing tests. Bachman and Palmer (1996, p. 23) define authenticity as “the degree of correspondence of the characteristics of a given language test task to the features of a target language task,” and then suggest an agenda for identifying those target language tasks and for transforming them into valid test items.

E. Washback

A facet of consequential validity, discussed above, is “the effect of testing on teaching and learning” (Hughes, 2003, p. 1), otherwise known among language-testing specialists as washback. In large-scale assessment, wasback generally refers to the effects the test have on instruction in terms of how students prepare for the test. “Cram” courses and “teaching to the test” are examples of such washback. Another form of washback that occurs more in classroom assessment is the information that “washes back” to students in the form of useful diagnoses of strengths and weaknesses. Washback also includes the effects of an assessment on teaching and learning prior to the assessment itself, that is, on preparation for the assessment.

F. Applying Principles to the Evaluation of Classroom Tests

The five principles of practicality, reliability, validity, authenticity, and washback go a long way toward providing useful guidelines for both evaluating an existing assessment procedure and designing one on your own. Quizzes, tests, final exams, and standardized proficiency tests can all be scrutinized through these five lenses.

1. Are the test procedures practical?
2. Is the test reliable?
3. Does the procedure demonstrate content validity?
4. Is the procedures face valid and “biased for best”?
5. Are the test tasks as authentic as possible?
6. Does the test other beneficial washback to the learner?

Reference :

Brown, H. Douglas. 2004. Language Assessment: Principles and Classroom Practices. New York: Longman.
http://blogrizkirmd.blogspot.co.id/2014/01/principles-of-language-assessment.html