ENLTA / EALTA survey on
language assessment
Need for training:
Which were the activitie=
s or
areas of testing where the perceived need was the greatest in different par=
ts
of
<= o:p>
<= o:p>
<= o:p>
I=
n the
following we report to what extent the respondents from different regions o=
f
<= o:p>
<= o:p>
|
1 =3D Northern Europe ( 2 =3D Baltic region ( 3 =3D Western Europe (Ireland, UK, France=
, Belgium, Netherlands) 4 =3D Central Europe ( 5 =3D South-Eastern Europe ( 6 =3D Eastern Europe ( 7 =3D Southern Europe ( |
<= o:p>
<= o:p>
T=
able 1 and
2 below summarise the findings. Both tables present the ten assessment
activities per region for which the highest number of respondents reported =
need
for further training. No atte=
mpt is
made here to distinguish basic and more advanced training, rather they are
treated together (for a breakdown by region of those who needed basic vs.
advanced training, see the section ‘Charts & descriptions, activi=
ty
by activity’ elsewhere in this website).
<= o:p>
B=
ecause of
the fact that it was possible for the respondents to answer only some of the
questions in the survey questionnaire and leave others unanswered, it is not
straightforward to estimate the proportion of respondents who reported a ne=
ed
for further training in the assessment activities listed in the questionnai=
re.
For that reason, two sets of results are presented here.
<= o:p>
T=
able 1 is
based on the percentages of respondents from each region who responded to at
least one question in Part 1 of the questionnaire. These respondents thus h=
ad a
chance to express their need for training; if they left an activity unanswe=
red
the interpretation may be that they did not want training in that particular
activity, otherwise they would have responded to it. However, because the a=
bove
is an assumption only and may underestimate the number / proportion of
respondents who needed further training, another set of results is also
presented. (Table 1 also reports the results for non-European respondents, =
just
out of curiosity.)
<= o:p>
T=
able 2 is
based on the percentages calculated from the number of respondents who actu=
ally
answered to the particular activity and stated whether they needed no train=
ing,
basic training or more advance training in the activity described. The
resulting percentages are thus higher than those in Table 1. It is difficul=
t to
say which ones are more accurate and thus it is useful to look at both tabl=
es
together (and also compare the figures with the findings presented in Part =
1 of
the EALTA survey report by Hasselgren et all. (2004). Note that further details on the
respondents’ need for training is found in this website under
‘Charts & descriptions, activity by activity’ for the train=
ing
needs.
<= o:p>
T=
able 3
gives the exact percentages for all assessment activities that are reported=
for
only the top ten activities in Table 2 (Table 3 is thus based on the same c=
alculation
of proportions as Table 2).
<= o:p>
T=
o give an
example on how to read the information in the tables, the first entry in Ta=
ble
1 indicates that 70% of the Northern European (Finnish, Swedish, Norwegian,
Danish and Icelandic) respondents who accessed the Teachers’
Questionnaire stated that they would like to receive either basic or advanc=
ed
training in the use of the European Language Portfolio or any other type of
portfolio. The second entry in the first row indicates that 63% of them nee=
ded
training in using self and peer assessment. Sometimes, if the percentage of
respondents was exactly the same for two or more activities, the cells in t=
he
table have been merged to indicate that the activities presented share the =
same
rank.
<= o:p>
<= o:p>
<= o:p>
Which are the activities=
in
which the need for training appears to be the most frequent?
C=
ertain
trends emerge from Table 1 and 2. Firstly, when we look at the magnitude of=
the
figures, those in Table 1 are lower than in Table 2, simply because they we=
re
calculated differently (see above). Typically, the percentages are 15 ̵=
1;
20 units higher in Table 2. Whichever table we look at, it is evident that =
the
demand for education in these selected assessment activities is high, often
over 80 or 90%.
<= o:p>
S=
econdly,
it can be seen that respondents from certain regions reported a need for
training more often than respondents from other regions. As already establi=
shed
elsewhere in this report, the lower level of need for training by the Weste=
rn
European teachers in our survey also manifests itself clearly in this analy=
sis
of the needs. Even the most ‘popular’ topics for training recei=
ve
50% (or 70%, depending on the table) ratings. In contrast, teachers from the
Baltic countries showed very high level of ‘interest’ in the top
ten activities. The other regions do not differ very clearly from each other
when we look at the percentages in the two tables (in particular, in Table =
2).
<= o:p>
T=
hirdly, we
can see which particular activities are singled out, in each region, as the
main targets for training. In which
activities did the respondents (who were mainly teachers) want training? How
much variation was there across
<= o:p>
I=
t is clear
from both Tables that there is a great demand for training in the use of portfolio type of assessment. It =
is the
number one activity in four or five of the seven European regions, and appe=
ars
in the top ten list for all the other regions, t=
oo,
except for non-European respondents. Several other classroom-focused assess=
ment
activities also appear on the list, such as use of self and peer assessment, continuous, non-test type of assessment, =
and
interpretation of test results, in particular. Giving feedback and maki=
ng
one’s own tests can also be seen among the top ten activities but less
often.
<= o:p>
T=
he second
clear cluster of activities where teachers feel the need for more training =
are the concepts and tools for studying and establishi=
ng the
quality of tests and assessments: =
reliability,
validity, and the use of statistics. In most cases, the percentage of
teachers who would like to get more training in these matters is only sligh=
tly
lower than the percentage of those who need training in the classroom-related
activities.
<= o:p>
O=
f the
purposes and content of assessment, integrated
testing and testing receptive and productive skills were also among the
high-ranking activities.
<= o:p>
O=
f the
activities associated with external test and examinations, defining criteria in particular but also writing and reviewing
items appeared on the list a few times. It is mentioned elsewhere in this
report that it is not altogether clear how the respondents have interpreted=
the
meaning of ‘defining criteria’; it may be that part of the inte=
rest
in receiving training in this activity may be that it can often be an activ=
ity
in which the teachers are engaged anyway, outside the context of external
testing.
Table 1: Top 10 activiti=
es in
terms of need for education
(percentages calculated from the total nu=
mber
of respondents, for the region, who entered Part 1 of the questionnaire and
answered at least on of the questions; the percentages in this table may th=
us
underestimate the need for training in general and for the particular
assessment activities)
<= o:p>
|
|
1st |
2nd |
3rd |
4th |
5th |
6th |
7th |
8th |
9th |
10th |
|
|
Portfolio 70% |
Self-assm.
63% |
Reliability 62% |
Validity 62% |
Statistics 61% |
Culture 59% |
Interpret results 58%<= o:p> |
Test productive 58% |
Continuous assm. 57% |
Feedback & Test
receptive 57% |
|
Baltic region |
Portfolio & Test productive skills=
77% |
Feedback & Self- Reliability & Vali=
dity
& Statistics & Define criteria 75% |
Make own tests &
Interpret results & Test grammar/vocab & Integrated assm. 72% |
|||||||
|
|
Reliability 54% |
Validity 53% |
Statistics 49% |
Portfolio 47% |
Self-assm. 47% |
Continuous assm. 43% |
Define criteria 41% |
Statistics (external tests) 41% |
Make own tests 40% |
Integrated assm. 38% |
|
|
Reliability &Valid=
ity 67% |
Make own tests 64% |
Interpret results &
Statistics 62% |
Integrated assm. 60% |
Portfolio 58% |
Feedback & Test
receptive & Test productive 56% |
||||
|
South-eastern |
Portfolio & Interpret results 75% |
Validity & Statist=
ics 71% |
Reliability 70% |
Self-assm.
& Define criteria 69% |
Rating (external) &=
; Statistics (external)<= o:p> 68% |
Make own tests & Continuous assm. & Culture & Interview
(external) 67% |
||||
|
|
Portfolio 73% |
Statistics 70% |
Interpret results &=
; Continuous assm. 67% |
Reliability & Define criteria 65% |
Culture & Validity=
64% |
Integrated assm. 63% |
Make own tests & Statistics (external) =
& Item writing (external=
) 62% |
|||
|
|
Portfolio 62% |
Make own tests &
Reliability & Define criteria 60% |
Statistics (external)<= o:p> 58% |
Interpret results &
Validity & Interview (external) 56% |
Test receptive & T=
est
productive & Integrated assm. & Statistics & Item writing (external)
& Item review (external) 53% |
|||||
|
non-Eur. countries |
Interpret results 63% |
Make own tests & C=
ontinuous
assm. & Validity & Statist=
ics
(external) 61% |
Self-assm.
& Reliability & Statistics & Item writing (external=
) 58% |
<= o:p>
Table 2: Top 10 activiti=
es in
terms of need for education
<= o:p>
(percentages a=
re calculated
from the number of the respondents who replied to the activity in question =
and
stated whether they needed no training, basic training or more advanced
training in the activity)
<= o:p>
|
|
1st |
2nd |
3rd |
4th |
5th |
6th |
7th |
8th |
9th |
10th |
|
|
Portfolio 90% |
Self-assessm. 84% |
Reliability 83% |
Statistics 83% |
Validity 85% |
Test culture 77% |
Interpret results & Integrated assessment 75% |
Test productive 75% |
Continuous assessm. 75% |
|
|
Baltic region |
Integrated assessm. 94% |
Portfolio 92% |
Define criteria (ext.) 92% |
Test productive 90% |
Self-assessment & Reliability & Validity & Statistics<= o:p> 90% |
Own tests & Test grammar / vocab. =
90% |
||||
|
|
Validity 75% |
Reliability 74% |
Portfolio 72% |
Statistics 71% |
Self-assessm 66% |
Statistics (external) 63% |
Define criteria (ext.) 61% |
Continuous assessm. 60% |
Integrated assessm. 54% |
Write items (ext.) 54% |
|
|
Validity 95% |
Reliability 92% |
Integrated assessm. 89% |
Interpret results & Statistics 84% |
Own tests 83% |
Portfolio 81% |
Test productive 78% |
Test receptive 76% |
Placement 76% |
|
|
South-eastern |
Portfolio 95% |
Interpret results 88% |
Statistics 88% |
Validity 87% |
Reliability 87% |
Self-assessm. 86% |
Define criteria (ext.) 85% |
Continuous assessm. 85% |
Statistics (ext.) 84% |
Rating (external) 83% |
|
|
Portfolio 91% |
Statistics 83% |
Continuous assessm. 83% |
Interpret results 82% |
Reliability 81% |
Validity 81% |
Define criteria (ext.) 80% |
Test culture 80% |
Integrated assessm. 80% |
Statistics (ext.) 78% |
|
|
Portfolio 90% |
Reliability 90% |
Statistics (ext.) 90% |
Define criteria (ext.) 87% |
Validity 86% |
Statistics & Write items (external) & Review items (extern=
al) 83% |
Integrated assessm. 80% |
Own tests 79% |
<= o:p>
Table 3: Need for assess=
ment
training – activities & percentages by region
<= o:p>
|
Percentage of the respondents who reported a need
for training (either basic or advanced) in the activity (percentages calculated from the total of those who replied someth=
ing
to the activity in question, as in the case of Table 2) |
|||||||
|
Activity: |
Northern
Europe |
Baltic
region |
Western Europ=
e |
Central
Europe |
South-Eastern
Europe |
Eastern Europ=
e |
Southern
Europe |
|
Classroom-focused testing / assessment |
|
|
|
|
|
|
|
|
Preparing your own classroom tests |
67.4 |
89.5 |
50.0 |
82.7 |
78.2 |
74.0 |
79.4 |
|
Using ready-made tests from textbook packages or
from other sources |
43.4 |
58.8 |
28.4 |
37.5 |
52.4 |
50.7 |
44.4 |
|
Interpreting test results (classroom or external
tests) |
75.3 |
87.2 |
51.0 |
84.2 |
88.4 |
81.6 |
75.8 |
|
Giving feedback to students based on information
from tests / assessment |
71.9 |
85.4 |
44.4 |
74.4 |
76.5 |
71.1 |
63.3 |
|
Using self / peer assessment |
84.4 |
89.7 |
66.0 |
74.3 |
86.4 |
73.3 |
72.4 |
|
Using informal, continuous, non-test type of
assessment |
74.6 |
86.5 |
60.0 |
73.0 |
85.0 |
82.7 |
71.0 |
|
Using the European Language Portfolio, an adapta=
tion
of it or some other portfolio |
89.7 |
92.3 |
71.6 |
81.1 |
95.0 |
90.5 |
90.3 |
|
Purposes of testing |
|
|
|
|
|
|
|
|
To give grades |
59.3 |
78.4 |
36.2 |
66.8 |
64.7 |
64.9 |
51.5 |
|
To find out what needs to be taught / learned ne=
xt |
59.8 |
81.6 |
43.5 |
75.0 |
74.7 |
68.5 |
69.0 |
|
To place students onto courses, programmes, etc<=
/span> |
52.8 |
75.7 |
38.6 |
75.7 |
72.5 |
70.7 |
62.5 |
|
To award final certificates (from school /
programme; local, regional or national level) |
63.7 |
78.9 |
42.2 |
72.2 |
64.1 |
75.7 |
66.7 |
|
Content and concepts<=
/span> |
|
|
|
|
|
|
|
|
-=
receptive skills (reading/ listening) |
73.6 |
84.6 |
43.6 |
76.3 |
75.3 |
75.7 |
75.0 |
|
-=
productive skills (speaking/writing) |
75.1 |
90.0 |
44.7 |
78.4 |
78.0 |
74.3 |
77.0 |
|
- microlinguistic as=
pects
(eg grammar/vocabulary) |
65.7 |
89.5 |
42.9 |
75.0 |
74.4 |
72.0 |
71.0 |
|
-=
integrated language |
75.3 |
94.4 |
54.3 |
88.6 |
80.7 |
79.5 |
80.0 |
|
-=
aspects of culture |
77.3 |
84.2 |
48.3 |
73.5 |
82.9 |
79.7 |
66.7 |
|
Establishing reliability of tests / assessment <=
/span> |
83.3 |
89.7 |
74.2 |
92.1 |
86.6 |
81.1 |
90.0 |
|
Establishing validity of tests / assessment |
82.2 |
89.7 |
74.7 |
94.6 |
86.7 |
80.8 |
86.2 |
|
Using statistics to study the quality of tests /
assessment |
82.5 |
89.7 |
70.7 |
84.2 |
87.8 |
83.1 |
82.8 |
|
External tests and examinations (region=
al or
national) |
|
|
|
|
|
|
|
|
Taking part in rating oral or written performanc=
es |
68.4 |
80.6 |
50.0 |
66.7 |
83.1 |
75.3 |
70.0 |
|
Using statistics to study the quality of tests /
assessment |
68.6 |
84.2 |
62.8 |
72.7 |
84.1 |
78.1 |
89.7 |
|
Writing items/test tasks for an examination body=
|
65.5 |
84.6 |
53.9 |
60.0 |
81.7 |
77.0 |
82.8 |
|
Reviewing items/ test tasks for an external
examination |
63.9 |
89.2 |
52.2 |
60.0 |
81.7 |
72.0 |
82.8 |
|
Acting as an interviewer / interlocutor in an or=
al
test or examination |
67.6 |
82.1 |
49.4 |
69.4 |
81.9 |
75.7 |
78.1 |
|
Defining
assessment criteria |
74.6 |
92.1 |
61.1 |
73.5 |
85.4 |
80.0 |
87.1 |
S=
ee also
the section ’Charts & descriptions, activity by activity’ t=
hat
concerns the need for training expressed by the respondents.
<= o:p>