ENLTA/EALTA survey on language testing and assessment, Report Part 2
Overall need for training in language testing
and assessment by region
Detailed
statistical analyses
The analyses reported here are based on the Chi-Square
Tests. The SPSS-programme that was used on compute the chi-squares defines them in the following way:
“The Chi-Square Test procedure tabulates a
variable into categories and computes a chi-square statistic. This
goodness-of-fit test compares the observed and expected frequencies in each
category to test either that all categories contain the same proportion of
values or that each category contains a user-specified proportion of values.”
To interpret the chi-square output, please pay
attention to the following points.
(1) The
smaller table ‘Chi-Square Tests’ shows if there is a statistically significant
relationship between the two variables that are studied. The first row (Pearson
Chi-Square) displays the Chi-Square value and the significance level of the
finding (Asymp. Sig.). If the significance level is
smaller than .05, there is a significant relationship between the two
variables, i.e. the observed frequencies in the ‘Crosstab’
table are not based on chance. The number of observations in each cell of the
table should be more than 5; if there are too many cells with fewer than 5
observations, the chi-square test / value is not reliable (note the extra row
of text immediately after the ‘Chi-Square Tests’ table).
(2) The
bigger table ‘Crosstab’ displays how the respondents
in the two or more background categories replied to the question concerning
their need to receive training on the particular assessment activity or
concept. ‘Count’ shows the number of actual responses in each category and
‘Expected Count’ shows what the expected number should have been if the
distribution of responses were based on chance alone, given the total number of
respondents in the two or more groups compared.
(3) The percentage
row displays the percentage of the observed responses in each category (i.e. it
is based on the ‘Count’), and is useful in interpreting and describing the
results in practice.
(4) Standardized
Rediduals in the last row in each cell are useful in locating
where exactly the obversed overall relationship /
difference (idenfied by the significant chi-square
value) takes place. Roughly speaking, if the standardized residual is bigger
than +2.0 or smaller than -2.0, then the difference between the observed value (Count) and the expected value in that cell is
significant. Note that if the statistical significance level of the chi-square
is not very strong (i.e. it is only somewhat smaller than .05), the
standardized residuals may not be outside the +/- 2.0 range for any of the
cells in the table, and thus, it is difficult to say what exactly is the source
for the significant overall chi-square value. However, in such cases, too, it
is probably the cells with the highest standardized residuals which contribute
the most to the overall significant results (see e.g. the table for the
activity ‘Giving feedback’ below).
Crosstabs
Case
Processing Summary
|
|
Cases |
|||||
| Valid |
Missing |
Total |
||||
| N |
Percent |
N |
Percent |
N |
Percent |
|
|
Regions of Europe * Teachers - all
need for assessment education combined (mean) (Banded) |
581 |
63,6% |
333 |
36,4% |
914 |
100,0% |
Regions of Europe * Teachers - all need
for assessment education combined (mean) (Banded) Crosstabulation
|
|
|
Teachers
- all need for assessment education combined (mean) (Banded) |
Total |
|||
| no or little need on average |
basic need
on average |
advanced need
on average |
||||
| Regions of Europe |
Northern Europe |
Count |
34 |
97 |
57 |
188 |
| Expected Count |
30,4 |
86,4 |
71,2 |
188,0 |
||
| % within
Regions of Europe |
18,1% |
51,6% |
30,3% |
100,0% |
||
| Std. Residual |
,6 |
1,1 |
-1,7 |
|
||
| Baltic region |
Count |
1 |
22 |
19 |
42 |
|
| Expected Count |
6,8 |
19,3 |
15,9 |
42,0 |
||
| % within
Regions of Europe |
2,4% |
52,4% |
45,2% |
100,0% |
||
| Std. Residual |
-2,2 |
,6 |
,8 |
|
||
| Western Europe |
Count |
30 |
49 |
29 |
108 |
|
| Expected Count |
17,5 |
49,6 |
40,9 |
108,0 |
||
| % within
Regions of Europe |
27,8% |
45,4% |
26,9% |
100,0% |
||
| Std. Residual |
3,0 |
-,1 |
-1,9 |
|
||
| Central Europe |
Count |
3 |
23 |
16 |
42 |
|
| Expected Count |
6,8 |
19,3 |
15,9 |
42,0 |
||
| % within
Regions of Europe |
7,1% |
54,8% |
38,1% |
100,0% |
||
| Std. Residual |
-1,5 |
,8 |
,0 |
|
||
| South-Eastern Europe |
Count |
9 |
34 |
45 |
88 |
|
| Expected Count |
14,2 |
40,4 |
33,3 |
88,0 |
||
| % within
Regions of Europe |
10,2% |
38,6% |
51,1% |
100,0% |
||
| Std. Residual |
-1,4 |
-1,0 |
2,0 |
|
||
| Eastern Europe |
Count |
13 |
29 |
36 |
78 |
|
| Expected Count |
12,6 |
35,8 |
29,5 |
78,0 |
||
| % within
Regions of Europe |
16,7% |
37,2% |
46,2% |
100,0% |
||
| Std. Residual |
,1 |
-1,1 |
1,2 |
|
||
| Southern Europe |
Count |
4 |
13 |
18 |
35 |
|
| Expected Count |
5,7 |
16,1 |
13,3 |
35,0 |
||
| % within
Regions of Europe |
11,4% |
37,1% |
51,4% |
100,0% |
||
| Std. Residual |
-,7 |
-,8 |
1,3 |
|
||
|
Total |
Count |
94 |
267 |
220 |
581 |
|
| Expected Count |
94,0 |
267,0 |
220,0 |
581,0 |
||
| % within
Regions of Europe |
16,2% |
46,0% |
37,9% |
100,0% |
||
Chi-Square Tests
|
|
Value |
df |
Asymp. Sig.
(2-sided) |
|
Pearson Chi-Square |
38,310(a) |
12 |
,000 |
|
Likelihood Ratio |
40,400 |
12 |
,000 |
|
Linear-by-Linear Association |
9,890 |
1 |
,002 |
|
N of Valid Cases |
581 |
|
|
a 0
cells (,0%) have expected count less than 5. The minimum expected count is 5,66.
Crosstabs
Case
Processing Summary
|
|
Cases |
|||||
| Valid |
Missing |
Total |
||||
| N |
Percent |
N |
Percent |
N |
Percent |
|
|
Regions of Europe * Teachers -
total nbr of activities etc with need for basic
education (Banded) |
416 |
45,5% |
498 |
54,5% |
914 |
100,0% |
Regions of Europe
* Teachers - total nbr of activities etc with need
for basic education (Banded) Crosstabulation
|
|
|
Teachers
- total nbr of activities etc with need for basic
education (Banded) |
Total |
|||||
| 5 or
fewer basic |
5-10 basic |
10-15 basic |
15-20 basic |
20 or more basic |
||||
| Regions of Europe |
Northern Europe |
Count |
13 |
45 |
34 |
24 |
32 |
148 |
| Expected Count |
14,9 |
41,3 |
32,0 |
29,2 |
30,6 |
148,0 |
||
| % within
Regions of Europe |
8,8% |
30,4% |
23,0% |
16,2% |
21,6% |
100,0% |
||
| Std. Residual |
-,5 |
,6 |
,4 |
-1,0 |
,3 |
|
||
| Baltic region |
Count |
4 |
6 |
6 |
11 |
10 |
37 |
|
| Expected Count |
3,7 |
10,3 |
8,0 |
7,3 |
7,6 |
37,0 |
||
| % within
Regions of Europe |
10,8% |
16,2% |
16,2% |
29,7% |
27,0% |
100,0% |
||
| Std. Residual |
,1 |
-1,3 |
-,7 |
1,4 |
,9 |
|
||
| Western Europe |
Count |
8 |
15 |
10 |
10 |
10 |
53 |
|
| Expected Count |
5,4 |
14,8 |
11,5 |
10,4 |
11,0 |
53,0 |
||
| % within
Regions of Europe |
15,1% |
28,3% |
18,9% |
18,9% |
18,9% |
100,0% |
||
| Std. Residual |
1,1 |
,1 |
-,4 |
-,1 |
-,3 |
|
||
| Central Europe |
Count |
4 |
10 |
7 |
9 |
1 |
31 |
|
| Expected Count |
3,1 |
8,6 |
6,7 |
6,1 |
6,4 |
31,0 |
||
| % within
Regions of Europe |
12,9% |
32,3% |
22,6% |
29,0% |
3,2% |
100,0% |
||
| Std. Residual |
,5 |
,5 |
,1 |
1,2 |
-2,1 |
|
||
| South-Eastern Europe |
Count |
7 |
12 |
13 |
18 |
17 |
67 |
|
| Expected Count |
6,8 |
18,7 |
14,5 |
13,2 |
13,9 |
67,0 |
||
| % within
Regions of Europe |
10,4% |
17,9% |
19,4% |
26,9% |
25,4% |
100,0% |
||
| Std. Residual |
,1 |
-1,5 |
-,4 |
1,3 |
,8 |
|
||
| Eastern Europe |
Count |
5 |
16 |
13 |
9 |
13 |
56 |
|
| Expected Count |
5,7 |
15,6 |
12,1 |
11,0 |
11,6 |
56,0 |
||
| % within
Regions of Europe |
8,9% |
28,6% |
23,2% |
16,1% |
23,2% |
100,0% |
||
| Std. Residual |
-,3 |
,1 |
,3 |
-,6 |
,4 |
|
||
| Southern Europe |
Count |
1 |
12 |
7 |
1 |
3 |
24 |
|
| Expected Count |
2,4 |
6,7 |
5,2 |
4,7 |
5,0 |
24,0 |
||
| % within
Regions of Europe |
4,2% |
50,0% |
29,2% |
4,2% |
12,5% |
100,0% |
||
| Std. Residual |
-,9 |
2,1 |
,8 |
-1,7 |
-,9 |
|
||
|
Total |
Count |
42 |
116 |
90 |
82 |
86 |
416 |
|
| Expected Count |
42,0 |
116,0 |
90,0 |
82,0 |
86,0 |
416,0 |
||
| % within
Regions of Europe |
10,1% |
27,9% |
21,6% |
19,7% |
20,7% |
100,0% |
||
Chi-Square Tests
|
|
Value |
df |
Asymp. Sig.
(2-sided) |
|
Pearson Chi-Square |
29,733(a) |
24 |
,194 |
|
Likelihood Ratio |
33,065 |
24 |
,103 |
|
Linear-by-Linear Association |
,125 |
1 |
,724 |
|
N of Valid Cases |
416 |
|
|
a 5
cells (14,3%) have expected count less than 5. The minimum expected count is 2,42.
Crosstabs
|
|
Cases |
|||||
| Valid |
Missing |
Total |
||||
| N |
Percent |
N |
Percent |
N |
Percent |
|
|
Regions of Europe * Teachers -
total nbr of activities etc with need for advanced
education (Banded) |
511 |
55,9% |
403 |
44,1% |
914 |
100,0% |
Regions of Europe *
Teachers - total nbr of activities etc with need for
advanced education (Banded) Crosstabulation
|
|
|
Teachers
- total nbr of activities etc with need for
advanced education (Banded) |
Total |
|||||
| 5 or
fewer advanced |
5-10 advanced |
10-15 advanced |
15-20 advanced |
20 or more advanced |
||||
| Regions of Europe |
Northern Europe |
Count |
10 |
24 |
22 |
32 |
70 |
158 |
| Expected Count |
9,9 |
23,2 |
23,8 |
33,4 |
67,7 |
158,0 |
||
| % within
Regions of Europe |
6,3% |
15,2% |
13,9% |
20,3% |
44,3% |
100,0% |
||
| Std. Residual |
,0 |
,2 |
-,4 |
-,2 |
,3 |
|
||
| Baltic region |
Count |
3 |
5 |
6 |
7 |
19 |
40 |
|
| Expected Count |
2,5 |
5,9 |
6,0 |
8,5 |
17,1 |
40,0 |
||
| % within
Regions of Europe |
7,5% |
12,5% |
15,0% |
17,5% |
47,5% |
100,0% |
||
| Std. Residual |
,3 |
-,4 |
,0 |
-,5 |
,4 |
|
||
| Western Europe |
Count |
9 |
23 |
18 |
18 |
21 |
89 |
|
| Expected Count |
5,6 |
13,1 |
13,4 |
18,8 |
38,1 |
89,0 |
||
| % within
Regions of Europe |
10,1% |
25,8% |
20,2% |
20,2% |
23,6% |
100,0% |
||
| Std. Residual |
1,5 |
2,7 |
1,3 |
-,2 |
-2,8 |
|
||
| Central Europe |
Count |
2 |
5 |
2 |
9 |
20 |
38 |
|
| Expected Count |
2,4 |
5,6 |
5,7 |
8,0 |
16,3 |
38,0 |
||
| % within
Regions of Europe |
5,3% |
13,2% |
5,3% |
23,7% |
52,6% |
100,0% |
||
| Std. Residual |
-,2 |
-,2 |
-1,6 |
,3 |
,9 |
|
||
| South-Eastern Europe |
Count |
5 |
10 |
12 |
16 |
43 |
86 |
|
| Expected Count |
5,4 |
12,6 |
13,0 |
18,2 |
36,9 |
86,0 |
||
| % within
Regions of Europe |
5,8% |
11,6% |
14,0% |
18,6% |
50,0% |
100,0% |
||
| Std. Residual |
-,2 |
-,7 |
-,3 |
-,5 |
1,0 |
|
||
| Eastern Europe |
Count |
0 |
6 |
8 |
20 |
34 |
68 |
|
| Expected Count |
4,3 |
10,0 |
10,2 |
14,4 |
29,1 |
68,0 |
||
| % within
Regions of Europe |
,0% |
8,8% |
11,8% |
29,4% |
50,0% |
100,0% |
||
| Std. Residual |
-2,1 |
-1,3 |
-,7 |
1,5 |
,9 |
|
||
| Southern Europe |
Count |
3 |
2 |
9 |
6 |
12 |
32 |
|
| Expected Count |
2,0 |
4,7 |
4,8 |
6,8 |
13,7 |
32,0 |
||
| % within
Regions of Europe |
9,4% |
6,3% |
28,1% |
18,8% |
37,5% |
100,0% |
||
| Std. Residual |
,7 |
-1,2 |
1,9 |
-,3 |
-,5 |
|
||
|
Total |
Count |
32 |
75 |
77 |
108 |
219 |
511 |
|
| Expected Count |
32,0 |
75,0 |
77,0 |
108,0 |
219,0 |
511,0 |
||
| % within
Regions of Europe |
6,3% |
14,7% |
15,1% |
21,1% |
42,9% |
100,0% |
||
Chi-Square Tests
|
|
Value |
df |
Asymp. Sig.
(2-sided) |
|
Pearson Chi-Square |
40,710(a) |
24 |
,018 |
|
Likelihood Ratio |
45,022 |
24 |
,006 |
|
Linear-by-Linear Association |
2,793 |
1 |
,095 |
|
N of Valid Cases |
511 |
|
|
a 6
cells (17,1%) have expected count less than 5. The minimum expected count is 2,00.