Classroom-focused testing / assessment

Need for training in different language testing = and assessment activities:

- Were there differences between regions of Europe?

Detailed findings regarding the respondents’ nee= d for training in different assessment activities are reported below. Included are only those respondents who worked in Europe and replied to the questions in Part 1 of the questionnaire (Teachers’ Questionnaire) are included here (n =3D 491 – 566, depending on the question / activity).

Classroom-foc= used testing / assessment

Preparing your own classroom tests <= /p>

M= ost of the respondents reported a need for further training in this activity although = most of them had received previous training. Typically, they needed more advanced training despite the fact that they already had had some training (see the corresponding findings / charts for previous training on the activities).

V= ery significant differences were found between the regions (chi square =3D 43.8= , df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not nee= d any further training. In contrast, hardly any Baltic respondent chose the ̵= 6;no need for training’ option.

Using ready-made tests from textbook packages or from other sources

<= o:p>

D= epending on the region, some 30-60% of the respondents reported a need for training = in this activity – typically, more advanced training.<= /p>

S= ignificant differences were found between the regions (chi square =3D 28.3, df =3D 12, p =3D .005). Th= e Northern and Western Europeans needed less training than others, whereas the Eastern= and South-Eastern and Baltic respondents needed training somewhat more often th= an others.

<= o:p>

Interpreting test results (classroom or external tests)

A= s with ‘preparing your classroom tests’, some 50-60% of the respondents reported a need for more advanced training, and only a minority needed no training.

V= ery significant differences were found between the regions (chi square =3D 46.6= , df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not nee= d any further training, whereas the South-Eastern European respondents needed more training compared with the others.

Giving feedback to students based on information from tests / assessment

A= bout half of the respondents reported they need more advanced training; some 20-30% reported no need at all.

V= ery significant differences were found between the regions (chi square =3D 41.4= , df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training, whereas the Baltic respondents need= ed somewhat more training compared with the others.

Using self / peer assessme= nt

About 20% reported no need for further training in this activi= ty; the most common response was ‘need more advanced training’ which was chosen by about 50% of the respondents.

Significant differences were found between the regions (chi square =3D 25.2, df =3D 12, p =3D .014). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training.

Using informal, continuous, non-test type of assessment

Some 40-50% reported a need for advanc= ed training in this activity, whereas about 20% felt no further need for train= ing.

Significant differences were found between the regions (chi square =3D 28.4, df =3D 12, p =3D .005). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training, whereas the South-Eastern Europeans needed training more often.

Using the European Language Portfolio, an adaptation of it or some other portfoli= o

<= o:p>

U= sually, about 90% or more of the respondents wanted to get more training in the use= of portfolios, either basic or advanced.

V= ery significant differences were found between the regions (chi square =3D 42.3, df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training, whereas the South-Eastern Europeans needed training more often.

<= o:p>

Purposes of t= esting

To give grades<= /span>

&nbs= p;

Need for training varied b= ut in general about half of the respondents felt some need for further training. Compared with the activities listed above, more respondents reported no need for training (40-50%).

V= ery significant differences were found between the regions (chi square =3D 36.4= , df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training, whereas respondents from the Baltic region needed training more often than the others.

&nbs= p;

To find out what needs to be taught / learned next

Need for training varied b= ut in general about 60-70% of the respondents felt some need for further training= .

V= ery significant differences were found between the regions (chi square =3D 36.0= , df =3D 12, p =3D .000). In= particular, the Western European respondents differed from the other in that so many of them did not need any further training, whereas respondents from the Baltic region and South-Eastern Europe, in particular, needed training more often = than the others.

To place students onto courses, programmes, etc

Need for training varied b= ut in general about 60-70% of the respondents felt some need for further training= .

V= ery significant differences were found between the regions (chi square =3D 46.6= , df =3D 12, p =3D .000). In= particular, the Western European and also the Northern European respondents differed fr= om the other in that so many of them did not need any further training – which probably stems from the fact that only some 40% of the North Europeans and somewhat over 50% of the Western Europeans were engaged in this activit= y in the first place. The Central, Eastern and South-Eastern European respondent= s, in particular, reported need for training more often than the others.

&nbs= p;

To award final certificates (from school / programme; local, regional or natio= nal level)

Need for training varied b= ut in general about 60-70% of the respondents felt some need for further training= .

V= ery significant differences were found between the regions (chi square =3D 41.1= , df =3D 12, p =3D .000). In= particular, the Western European and also the Northern European respondents differed fr= om the other in that so many of them did not need any further training. The Ba= ltic and Eastern European respondents, in particular, reported need for training= more often than the others.

Content and c= oncepts

Receptive skills (reading/ listening)

&nbs= p;

Need for training varied i= n that the Western Europeans clearly differed from everybody else; for all other regions, close to 80% of the respondents needed further training, in particular, advanced training. Most of the respondents reported being engag= ed in this activity so the demand for more education is understandable; it is interesting to note that there is this demand for further training despite = the fact that most respondents had had a fair amount of previous training in th= is activity.<= /o:p>

V= ery significant differences were found between the regions (chi square =3D 46.2= , df =3D 12, p =3D .000). Th= e difference is almost entirely due to the Western European respondents, most of whom did not need any further training in this activity. The only other group which = was somewhat different from the rest were the respondents from the Baltic region who needed further training a bit more often than the others.

&nbs= p;

Productive skills (speaking/writing)

&nbs= p;

The need for training in t= he assessment of the productive skills was very similar to the need expressed = for the receptive skills above. The Western Europeans clearly differed from everybody else as did the Baltic respondents, to a lesser degree; for all o= ther regions, close to 80% of the respondents needed further training, in particular, advanced training. Most of the respondents reported being engag= ed in this activity so the demand for more education is understandable; it is interesting to note that there is this demand for further training despite = the fact that most respondents had had a fair amount of previous training in th= is activity.<= /o:p>

V= ery significant differences were found between the regions (chi square =3D 53.8= , df =3D 12, p =3D .000). Th= e difference is mostly due to the Western European respondents, most of whom did not need any further training in this activity. The only other group which was diffe= rent from the rest were the respondents from the Baltic region who needed further training more often than the others.

Microlinguistic aspects (eg grammar/vocabulary)

The need for training in t= he assessment of grammar and vocabulary was very similar to the need expressed= for the receptive and productive skills above. The Western Europeans clearly differed from everybody else, as did the Baltic respondents, to a lesser degree; for all other regions, about 75% of the respondents needed further training, in particular, advanced training. Most of the respondents reported being engaged in this activity so the demand for more education is understandable; it is interesting to note that there is this demand for fur= ther training despite the fact that most respondents had had a fair amount of previous training in this activity.

V= ery significant differences were found between the regions (chi square =3D 42.2= , df =3D 12, p =3D .000). Th= e difference is mostly due to the Western European respondents, most of whom did not need any further training in this activity. The only other group which was diffe= rent from the rest were the respondents from the Baltic region who needed further training more often than the others.

<= o:p>

Integrated language skills=

Typically, about 80% of the respondents needed trainin= g in this activity, usually more advanced type of training.

Very significant differences were found between the regions (chi square =3D 40.0, df =3D 12, p =3D .000). The Western Europeans needed furth= er training significantly less often than the others, whereas the Baltic and Central European respondents felt more often need for training in this acti= vity

Aspects of culture<= /p>

Regions varied, but typically, about 70-80% of the respondents needed training in this activity.

Very significant differences were found between the regions (chi square =3D 58.0, df =3D 12, p =3D .000). Again, the Western Europeans needed further training significantly less often than the others. The Baltic and South-Eastern European respondents also different from the others in their pattern of need.

<= o:p>

Establishing reliability of tests / assessment

Most of the respondents who replied to this question reported they need further training in this activity (80-90%), which is not= surprising given the fact that about half of them were engaged in the activity but the amount of training varied. Typically, the need focused on more advanced rat= her than basic training.

Significant differences were found between the regions (chi square =3D 25.0, df =3D 12, p =3D .015). The number of Western European respondents who needed no further training was unexpectedly high, as was the number of North Europeans who needed basic training, whereas the number of North European who needed more advanced training was lower than could be expected.

&nbs= p;

Establishing validity of tests / assessment

The pattern of responses is very similar here to the p= attern in the previous activity (establishing reliability): most respondents wanted more training, in particular advanced training – here the need for mo= re advanced training appears to be even more pronounced.

Significant differences were found between the regions (chi square =3D 28.5, df =3D 12, p =3D .005). The number of Western European respondents who needed no further training was unexpectedly high, as was the number of North Europeans who needed basic training, whereas the number of North European who needed more advanced training was lower than could be expected.

Using statistics to study the quality of tests / assessment

The pattern of responses is very similar to the one on establishing reliability above which is probably not surprising as finding = out about the reliability of the assessments is one of the most common reasons = for using statistics in language testing / assessment.

External tests and examinations (regional= or national)

Taking part in rating oral or written performances

S= ignificant differences were found between the regions (chi square =3D 34.3, df =3D 12, p =3D .001). Cl= early fewer of the Western European respondents needed training than could be expected = (many of them were engaged in this activity but most of them had already been trained) and, conversely, more South-Eastern European respondents needed it= .

Using statistics to study the quality of tests / assessment

Interestingly, the majority of the respondents reported a need for further training= in this activity although only a majority of them reported being engaged in it (perhaps they would like to do this if they had the skill to do this; most = of them had not received previous training in using statistics).

S= ignificant differences were found between the regions (chi square =3D 30.8, df =3D 12, p =3D .002). Ag= ain, the Western Europeans differed from the others in that fewer of them reported a= ny need for training. A similar trend was found for the Northern European respondents. In contrast, both the Southern and South-Eastern European resp= ondents reported need for training in this external exam activity.

Writing items/test tasks for an examination body

60-80% of the respondents, depending on the region, re= ported they need some further training in writing items (about half of the respond= ents to Questionnaire 1 were or had in the past been engaged in this activity, a= nd only 20-40% had received any training). It is entirely possible that althou= gh this activity referred to item writing for a school-external test or examination, some respondents who did not actually write items for an exam board expressed their need for training in item writing in general by answe= ring this question and stating they need training.

Very significant differences were found between the regions (chi square =3D 40.1, df =3D 12, p =3D .000). A = somewhat similar pattern can be observed here as in the previous situation (using statistics): the Western and to some extent the Northern European responden= ts needed training in fewer cases than the Southern and South-Eastern responde= nts in particular.

Reviewing items/ test tasks for an external examination

A very similar pattern of responses can be observed for this activity as for the previous one (item writing). Also, the current engagement and previous training were very simi= lar. As for item writing, reviewing items designed by somebody else is an activi= ty that can take place outside a test or exam organization, so some of express= ions of need for training my stem from a more general need for training.

Very significant differences were found between the regions (chi square =3D 47.6, df =3D 12, p =3D .000). Co= mpared to the item writing activity above, the differences were more varied: the West= ern Europeans were clearly split into those who either did not at all need furt= her training and those who needed more advanced training. A somewhat similar pattern could be observed for the Northern Europeans. The Baltic, South-Eas= tern and Southern European respondents, in contrast, needed such training more o= ften than could be expected.

&nbs= p;

Acting as an interviewer / interlocutor in an oral test or examination<= /span>

Most of the respondents re= ported that they would like to have further training in this activity (70-80%), although only about half of them were currently engaged in the activity. Ag= ain, the more general need for education in being an interviewer or interlocutor= in oral tests may play a part in the high figure for more training.=

Very significant differences were found between the regions (chi square =3D 37.6, df =3D 12, p =3D .000). Th= e difference is mostly due to the Western European respondents who felt the need for fur= ther training less often than the others. The only other significant differences were the unexpectedly low number of South-Eastern Europeans who chose the option ‘no need’.

&nbs= p;

Defining assessment criter= ia

&nbs= p;

Most of the respondents re= ported that they would like to have further training in this activity (70-90%), although ‘only’ 50-60% of them were currently engaged in the activity. Again, the more general need for education in defining assessment criteria may play a part in the high figure for more training.

Very significant differences were found between the regions (chi square =3D 38.4, df =3D 12, p =3D .000). Mo= stly the differences are due to the West European respondents who needed training le= ss often and the Baltic respondents who needed it more often than the others. However, also the answer pattern of the South-Eastern and Southern European respondents was somewhat different from the others.