National Centre for Vocational Education
Research
|
About the research
Damian Oliver, National Centre for Vocational Education Research
Lower-level
qualifications (certificate I and II programs) provide little or no immediate
return to the individual in terms of increased wages. However, lower-level qualifications
are intended to prepare students who would otherwise not be capable of enrolling
in and completing a higher-level qualification or making a successful
transition into the workplace, because of their ability, social circumstances,
or previous educational experiences. The aim of this report is to test whether
lower-level qualifications serve a broader purpose by functioning as a
‘stepping stone’ to further study or into the labour market.
The critical part of the methodology is the
selection of the comparison group. Using data from the Longitudinal Surveys of
Australian Youth (LSAY), the research matches certificate I and II graduates to
other young people who share similar characteristics but who have neither
completed nor are undertaking study or training at a higher level. The report
compares their further study, training, employment and overall wellbeing
outcomes two years after graduation and at age 26. The findings do not relate
to certificate I or II qualifications completed as part of an apprenticeship or
traineeship.
§ Two years after completing a certificate I or
II qualification, young males are more likely to have undertaken an
apprenticeship or traineeship, when compared with other individuals with
similar background characteristics.
§ After two years, young female certificate I and
II graduates are more likely to be employed and to have undertaken an
apprenticeship or traineeship when compared with other similar females.
§ At age 26, the benefits of completing a
certificate I or II qualification are still apparent for males but at the same
age, females in the control group have caught up to their counterparts who are
certificate I and II graduates.
§ The benefits of completing a certificate I or
II qualification are strongest amongst the most disadvantaged learners within
the pool of certificate I and II graduates.
Tom Karmel
Managing Director, NCVER
Managing Director, NCVER
Tables 6
Introduction 7
Background 9
Methodology 11
Data 11
Constructing the control and treatment groups 11
The propensity scores 13
Results 15
Outcomes after two years 15
Discussion 19
References 21
Appendices
A 22
B 24
Tables
1 Estimated
completion rates for qualifications at certificate I and
above, commencing 2007 10
above, commencing 2007 10
2 Treatment and
control group counts by wave 13
3 Summary of
outcomes after two years in the treatment and control
groups (males) 15
groups (males) 15
4 Summary of
outcomes after two years in the treatment and control
groups (females) 16
groups (females) 16
5 Summary of
outcomes at age 26 in the treatment and control
groups (males) 17
groups (males) 17
6 Summary of
outcomes at age 26 in the treatment and control
groups (females) 17
groups (females) 17
7 Average treatment
effect after two years for most disadvantaged
and least disadvantaged graduates (males) 18
and least disadvantaged graduates (males) 18
8 Average treatment
effect after two years for most disadvantaged
and least disadvantaged graduates (females) 18
and least disadvantaged graduates (females) 18
9 Summary of
treatment effects of a certificate I/certificate II 19
A1 Enrolments by
qualification level and various characteristics,
students under 25 years, 2010 (%) 22
students under 25 years, 2010 (%) 22
A2 Outcomes by AQF
qualification level, students under 25 only,
2010 (%) 23
2010 (%) 23
B1 Variables used in modelling completion of a certificate
I or
certificate II 25
certificate II 25
B2 Outcome variables 26
B3 Model estimates
from the propensity scores regression, males (for outcomes after two years) 27
B4 Model estimates
from the propensity scores regression, females
(for outcomes after two years) 28
(for outcomes after two years) 28
B5 Average
distribution of completing a certificate I/certificate II,
before and after propensity score matching (males) 30
before and after propensity score matching (males) 30
B6 Mean maths and
reading scores, before and after propensity score matching (males) 31
B7 Average
distribution of completing a certificate I/certificate II,
before and after propensity score matching (females) 32
before and after propensity score matching (females) 32
B8 Mean maths and
reading scores, before and after propensity score matching (females) 33
Introduction
A number of studies have consistently shown that
while higher-level vocational education and training (VET) qualifications
generate positive economic returns, the economic benefit for an individual
completing a lower-level qualification is negligible (Long & Shah 2008;
Leigh 2008; Karmel & Nguyen 2007; Ryan 2002). However, these studies have typically
been careful in their conclusions not to completely dismiss the value of
completing a lower-level certificate, noting that this may be a ‘stepping stone’
or springboard to further study (see, for example, Long & Shah 2008, p.42;
Harris, Rainey & Sumner 2006) or have other less tangible benefits, such as
improved self-esteem or foundations skills like literacy and numeracy (Dawe
2004). The underlying assumption is that certificate I and II programs prepare
students who would otherwise not be capable of enrolling in and completing a
higher-level qualification or making a successful transition into the
workplace, because of their ability, social circumstances, or previous
educational experiences.
The purpose of this report is to examine
whether, in the absence of immediate positive economic returns, certificate I
and II programs really do provide a springboard to higher study, aid the
transition into the workforce, or improve general wellbeing. In this report, we
exclude traineeships that involve a certificate I or II qualification from our
consideration of lower–level qualifications. Other studies have examined
outcomes from traineeships (Karmel, Blomberg & Vnuk 2010; Cully &
Curtain 2001). Instead, we concentrate on certificate I and II qualifications
that do not involve a contract of training, such as foundation programs,
bridging courses, pre-apprenticeships and pre-vocational courses.
Lower–level qualifications are designed and
promoted as being targeted toward disadvantaged or discouraged learners, yet we
find that a sizeable proportion of lower-level VET students display
characteristics that suggest they are neither particularly disadvantaged nor
discouraged. Using data from the National Centre for Vocational Education
Research (NCVER) National VET Provider Collection, we also find that completion
is very important in determining whether or not a certificate I or II
qualification confers any benefit, especially in relation to further study.
Poor targeting could obscure positive outcomes from lower-level
qualifications among the type of students for whom they are intended.
When we turn to data from the Longitudinal
Surveys of Australian Youth (LSAY), we find further confirmation that young
people who complete lower-level VET qualifications are not so different from
other young people. The broad profile of lower-level VET graduates in the LSAY
sample means that, based on their characteristics, some of these graduates
could have just as easily completed an apprenticeship or traineeship, some a
university qualification, while others most resemble young people who have not
completed any post-school qualification. Therefore we restrict the scope of the
control group to focus on young people who have the most to gain from
completing a lower-level qualification. Using the information on who completes
a lower-level qualification, we match each certificate I or II graduate with
someone who hasn’t completed a certificate I or II (or any other post-school
qualification) but who shares similar background characteristics. We do this
using an econometric technique called propensity score matching.
When certificate I and certificate II graduates
are paired with similar non-graduates, we find that after two years both males
and females are more likely to have completed or be undertaking an
apprenticeship or traineeship. Females are also more likely to be employed and
males are on average happier if they have completed a certificate I or
certificate II. Over a slightly longer period, to age 26, the benefits for
males of completing a certificate I or certificate II solidify, and males
remain more likely to have completed or be undertaking an apprenticeship or
traineeship and are more likely to have completed a certificate III or higher
qualification. However, the benefits for females are not as apparent at age 26.
We attribute this to the different occupational labour markets and training
paths typically open to males and females.
The structure of this report is straightforward.
The following section provides background on lower-level qualifications,
including their place in the qualifications framework, the characteristics of
students who undertake them, completion rates and the pay-offs from completion.
Next, we provide an explanation of how we have constructed our treatment and
control groups and a brief, non-technical overview of the propensity score
matching methodology. In the results section, we compare the treatment and
control groups against a range of further study and labour market outcomes. We
conclude with a discussion of the results and some of the policy implications.
Background
Within the Australian Qualifications Framework
(AQF), lower-level qualifications (certificates I and II) exist to provide
individuals with a path to further study or entry into the workforce. The
purpose of a certificate I is to equip individuals with ‘basic functional
knowledge and skills to undertake work, further learning and community
involvement’ (AQF Council 2011, p.25). A certificate II ‘qualifies individuals
to undertake mainly routine work and as a pathway to further learning’ (AQF
Council 2011, p.28). The typical volume of learning for both a certificate I
and a certificate II is between six months and a year full-time, although it
may be possible to complete particular courses more quickly.
The current Australian Qualifications Framework
introduced ten levels of learning and certificates I and II correspond to the
first two levels. Certificate I and II qualifications are by definition at a
lower level than other VET qualifications and higher education qualifications.
There is no formal equivalence in the framework between certificate I or II
qualifications and the Senior Secondary Certificate of Education (commonly
known as Year 12). This was a deliberate decision of the council, in
recognition that Year 12 may fit within any of a number of levels, depending on
the subjects chosen by the individual student. Lim and Karmel (2011) support
this decision. They found that compared with Year 12, a certificate II does not
produce equivalent further study or employment outcomes and it is questionable
whether even a certificate III can be considered a vocational equivalent to Year
12.
Data from the
National VET Provider Collection shows a sharp division between lower-level
qualifications and higher-level qualifications, even among enrollees under the
age of 25 years. Compared with young people enrolled in higher-level VET
qualifications, young people enrolled in certificate I and II qualifications
are typically younger, more likely to be an early school leaver, more likely to
be Indigenous and more likely to have a disability. Half of all enrolments in certificate
I and II programs are from students still attending school. Where students
participating in VET in Schools are eligible to complete an AQF qualification,
it will typically be at the certificate I or II level. Of those who are not at
school, most are early school leavers (although a third of certificate II
students have completed Year 12). Appendix A contains more detail on the
characteristics of certificate I and II students.
Fewer than one in five certificate II students
and one in 20 certificate I students are enrolled as part of an apprenticeship
or traineeship. More common within lower-level certificates is the
pre-apprenticeship, a course designed as a pathway into an apprenticeship,
particularly in the traditional trades. Foley and Blomberg (2011, p.22)
estimate that most (58%) pre-apprenticeship activity is at the certificate II
level, corresponding to approximately 38 000 course enrolments in 2009,
predominantly in the engineering and related technologies field of education.
Most of the remaining pre-apprenticeship activity (39.4% or approximately 26 000
enrolments in 2009) is at the certificate I level. Most certificate I
pre-apprenticeships are in the architecture and building field of education.
Based on data from the 2010 National VET Provider Collection (author’s calculations),
pre-apprenticeships account for one in three (32.6%) certificate I enrolments
and one in five (20.3%) certificate II enrolments.
In the introduction, we
referred to a number of studies that show no positive economic returns from
certificate I and II qualifications. These findings are consistent with the
results of the latest NCVER Student Outcomes Survey (SOS, NCVER 2011a), which
show that certificate I and II graduates are less likely to be employed than
graduates of higher-level qualifications. If there is merit in a lower-level
qualification, it is mainly that it functions as a stepping stone or
springboard to further study (see, for example, Long & Shah 2008, p.42;
Harris, Rainey & Sumner 2006).
Bearing this in mind, it is not sufficient just
to commence a lower-level qualification — completion really matters. Recent
research by NCVER (Karmel & Fieger 2012) indicates that certificate I and
II students who complete their qualification are 2.82 times more likely to
enrol in further study than those who do not complete. This difference is much
larger than for certificate III and IV students (2.09) and diploma and above
students (1.65). Likewise, certificate I and II students are more likely to be
employed if they complete their qualification. Further, the completion pay-off
in terms of employment after training is higher among certificate I and II
students (1.25) than certificate III and IV students (1.23) and diploma and
above students (1.12). However, fewer than one in four students commencing a certificate
II qualification in 2007 completed the qualification. Fewer than one in five
students commencing a certificate I qualification in 2007 completed. As table 1
shows, the proportion is higher when the population is restricted to students
aged 25 years and under and without a post-school qualification, but the
completion rates for certificate I or II qualifications are consistently lower
than those for other qualification levels.
AQF qualification
|
Estimated
qualification completion rate
|
|
All
students
|
Full-time
students aged 25 years and under, with no post-school qualifications
|
|
Certificate I
|
17.2
|
30.3
|
Certificate II
|
21.2
|
30.1
|
Certificate III
|
32.5
|
42.0
|
Certificate IV
|
31.4
|
32.4
|
Diploma and above
|
32.6
|
36.5
|
Total
|
27.2
|
35.6
|
Source: NCVER
(2011b, tables 3 and 4).
Low completion rates are a persistent
problem. Evidence from previous NCVER research (Stanwick 2005) is that only
about 40% of certificate II graduates and 28% of certificate I graduates under
25 years of age went on to complete a further qualification at the same or
higher level.
Therefore, the treatment we are most interested
in is completion of a certificate I or II qualification. Because of the
limitations of the data sources, existing studies have not been able to identify
the characteristics of certificate I and II graduates who do complete a further
qualification, or compare the outcomes over time for certificate I and II graduates
with other young people with similar characteristics. This project is able to
overcome the limitations by using LSAY data from cohorts from the 1995 (Y95)
and 1998 (Y98), which we do in the next section.
Methodology
Having established that young people who
undertake and complete lower-level qualifications differ from other young
people, we cannot simply compare the outcomes of certificate I and certificate
II graduates with other young people who do not share the same background. To
do so would ignore the influence of family background, academic ability and
personal attributes. We need some way of taking into account the
characteristics of certificate I and II graduates. To do that, we use an
econometric technique called propensity score matching.
Propensity score matching is an attempt to
unlock the counterfactual by matching each person who has undergone the
treatment (completing a certificate I or II) with someone who has similar
characteristics but who has not undergone the treatment, and then comparing the
outcomes for the two groups. Propensity score matching is well suited in
situations like the current one, where there is a relatively small proportion
of cases that have undergone the treatment and a large pool of diverse cases
that have not.
To set up the propensity score matching requires
preparation, which is described in the following sections. First, the LSAY data
are described, with a breakdown of the treatment and control categories. Next,
the propensity scores are calculated and presented. Once we have the propensity
scores, we describe the technique for finding suitable matches. Once a matched
sample that is balanced on the relevant background characteristics is in place,
we then compare the average outcomes for the two groups.
Data
The results
presented in the background section drew on the National VET Provider Collection
and the NCVER Student Outcomes Survey. LSAY is a good complement to this. LSAY
is a longitudinal study that first surveys students (in the case of the Y95 and
Y98 cohorts) in Year 9 and interviews them successively for a further 11 years.
At the end of the survey, the median age of respondents is 26. In the first
wave of the survey, students undertake a short test of their reading and
mathematics ability. They also complete questions relating to their family
background. We combine data from the Y95 and Y98 cohorts, the two most recent
complete LSAY cohorts. Combining two cohorts maximises the number of responses,
which is especially important considering the low proportion of young people
completing lower-level certificates.
Constructing the control and treatment groups
We are interested
in the role of certificate I and II programs as pathways into further study and
employment. As we saw in the previous section, completing a certificate I or II
greatly increases the likelihood of commencing another qualification. We
therefore examine the first qualification completed
by participants after leaving school, up to age 26, when the survey ends. (For
Y95 and Y98 cohorts, LSAY did not collect information on qualifications
completed by participants while still at school. This means that lower-level
qualifications completed as part of VET in Schools activity are not within
scope.)
To take account of timing, we construct a
treatment and control group for each wave of each cohort. This important step
means that the post-treatment periods are the same for the treatment and
control groups. For each year, the treatment group comprises any respondent who
completed a certificate I or II qualification in that year, provided the
respondent had not previously completed any qualification and that the qualification
was not completed as part of an apprenticeship or traineeship. We emphasise
completion of a certificate I or II as the criterion for inclusion in the
treatment group because of the previous research that has demonstrated that the
payoff is large and also for the pragmatic reason that it is much more
straightforward to identify graduates in the survey.
The control group comprises all other
respondents present in that wave who:
§ are not in school
§ have not already completed a higher post-school
qualification
§ are not currently studying for a higher
qualification.
Implicit in this decision is the assumption that
a certificate I and II qualification is inappropriate for anyone who could
otherwise gain direct entry into a certificate III or higher qualification. In
this sense, we have applied a narrower scope for the control group than if we
had only excluded graduates of higher qualifications. We did this in response
to the policy rationale of lower-level qualifications as a pathway to higher
qualifications.
The control group also includes:
§ respondents who have completed a certificate I
or II in that year as part of a traineeship, since traineeships may be thought
of as an alternative pathway directly into employment
§ anyone who was studying a certificate I or II
qualification in that year but who did not complete the qualification (provided
they had not already completed a higher qualification).
The control group excludes respondents who
complete a certificate I or II in a subsequent wave, to avoid any inappropriate
matching because of sequencing issues in the survey.
The other restrictions are that there must be
observations two years before completing the qualification (so that we can
observe unemployment history before the respondent commenced the qualification)
and observations two years after completing the qualification (so that we can
observe outcomes). We can only observe a smaller number of cases at age 26
because, over the longer period, attrition further erodes the sample.
Table 2 shows the number of cases in the
treatment and control groups before matching. The cases are broken down by
wave. Both LSAY95 and LSAY98 began with a survey of Year 9 students and lasted
for 12 waves in total. Because we want to observe outcomes two years after
completing the lower-level qualification, there are no treatment or control
cases from wave 11 or wave 12. There are no treatment cases from wave 1 or wave
2, because we include in our propensity score model a measure of unemployment
experience two waves before completing the qualification (to be sure that any
incidence of unemployment occurred before the qualification was commenced).
Most of the treatment cases come from waves 4, 5, and 6. The control group for
each wave is as already described. Respondents who remain in the survey but who
completed a post-school qualification in a prior wave, or who complete a
certificate III or higher qualification in the current wave, or who are
studying for a certificate III or higher qualification in the current wave are
excluded from the control group. The total number of cases declines over time
because of sample attrition.
Wave
|
Treatment
|
Control
|
Excluded
|
Total1
|
3
|
47
|
971
|
8 183
|
9 201
|
4
|
95
|
1 276
|
7 104
|
8 475
|
5
|
179
|
3 787
|
4 834
|
8 800
|
6
|
96
|
4 033
|
3 958
|
8 087
|
7
|
57
|
3 044
|
3 661
|
6 762
|
8
|
22
|
2 386
|
3 338
|
5 746
|
9
|
20
|
2 024
|
3 118
|
5 162
|
10
|
13
|
1 715
|
2 948
|
4 676
|
Total
– all waves in scope
|
529
|
19 236
|
37 144
|
56 909
|
Note: 1
Declining counts reflect the influence of attrition.
Source: LSAY,
Y95 and Y98 cohorts.
The propensity scores
Propensity
scores reflect the predicted probability of undergoing a treatment, in this
case completing a certificate I or II as the first qualification after leaving
school. The following variables were included in the model:
§ State
§ Size of local area
§ School type
§ Academic achievement and ability
-
Highest
school level
-
Reading
score
-
Maths
score
-
Tertiary
entrance rank (where applicable)
§ Family background & personal
characteristics
-
Indigenous
status
-
Disability
status
-
Parental
occupational status
-
Highest
level of parental education
-
Students’
country of birth
-
Parents’
country of birth
§ Motivation factors
-
Views
learning as fun
-
Treated
fairly in class
-
Views self
as successful student
-
Believes
school is useful to later life
§ Unemployment history, t-2
§ Sample characteristics
-
Survey
wave
-
Cohort.
We describe
the statistical methodology for calculating the propensity scores and present
the model estimates in appendix B. Background characteristics largely have the
expected influence. For example, respondents who have one parent from a
non-English speaking country are more likely to be certificate I or II
graduates, as are male respondents whose parents work in jobs with low
occupational status. Respondents with a disability are more likely to complete
certificate I or II qualifications. We also find that academic achievement
plays a role. For both males and females, lower maths scores and tertiary
entrance ranks (where present) increase the probability of completing a
certificate I or II. Females who do not think that school will help them later
in life but who view themselves as successful students are more likely to
complete a certificate I or II as their initial qualification than those who do
not, once other factors are controlled for. Different factors are at play for
males. Male students who think they were treated fairly in class but who do not
view learning as fun are more likely to complete a certificate I or II than
other respondents. For both males and females, a period of unemployment
increases the likelihood of completing a certificate I or II qualification. Because
of missing data on some covariates, the number of treatment cases with a
propensity score reduces to 249 males and 236 females.
Our approach is to match the treatment group of certificate
II completers to a sub-sample of the control group using the propensity scores.
By matching cases based on the propensity scores, we assume that we have
addressed selection issues, and any differences in outcomes can be attributed
to the effect of the treatment. However, we have only controlled for observable
covariates and not on any unobserved covariates. This is known as the
conditional independence assumption.
Matching was performed using the Stata program
by Leuven and Sianesi (2003). Callipers apply a restriction to the process by
only allowing a match if the control’s propensity score is within a certain
distance. Radius matching matches a treatment case to all control cases with
propensity scores within a certain distance. We adopt the most straightforward
matching method. Each treatment case is matched to its nearest neighbour in the
control group, which is made up of all other respondents, whether they have
completed a higher qualification or no qualification at all. A calliper of 0.008
is applied, meaning that a control case will not be matched to a treatment case
if the difference between the propensity scores is greater than 0.008. The
calliper value was selected as it is one-quarter of the standard deviation of
the propensity scores (Rosenbaum & Rubin 1985). To simplify, the same
calliper was used for all propensity score matches. Control cases are not
replaced once matched. Matching was done separately for males and females, for
each cohort and for each period. In total, 12 cases (seven males and five
females) were off support, meaning that no remaining case in the control group
had a propensity score within the calliper range. Removing treated cases off
support removes results in 242 matched cases for males and 231 matched cases
for females. Results are then weighted using the most recent weight for each
treated case.
Following matching, the treatment sample and the
matched control sample are much more balanced in relation to the observed
covariates. As the chi-square tests in tables B5 and B7 and the t-tests in tables
B6 and B8 demonstrate, only a few of the selection variables remain unbalanced.
This was confirmed by chi-square tests, which show no significant differences
between the treatment and control groups for 13 out of 15 covariates for the
male propensity score model and three out of 15 covariates for the female
propensity score model.
Results
Having achieved a balanced control sub-sample,
we can now compare the outcomes for respondents who completed a certificate I
or II after finishing school to the outcomes for respondents with similar
characteristics. We look at three broad outcomes:
§ completion of a certificate III or higher
qualification, completion of or current engagement in an apprenticeship or
traineeship; this is a test of the ‘stepping stone’ idea
§ employment, hourly wage and occupational status;
this is to determine if lower-level qualifications assist people make positive
transitions into the labour market
§ overall life satisfaction; this tests whether
lower-level qualifications might have less observable benefits such as boosted
self-esteem and improved general wellbeing.
Outcomes are examined two years after completing
the qualification, and at age 26.
Outcomes after two years
Because we have addressed selection bias using
propensity score matching, it suffices to compare the outcomes using paired
t-tests (with the results weighted to reflect attrition in the treatment
category). Results are presented separately for males and females.
By nearly ten percentage points, male
certificate I and certificate II graduates are more likely to have completed
or be undertaking an apprenticeship or traineeship than other similar
individuals. On average, they are also more satisfied with their life overall.
A higher proportion of male certificate I/II graduates than other similar
individuals are employed, and a higher proportion have completed or are undertaking
a certificate III or higher qualification. (This is almost entirely due to the
higher participation in apprenticeships.) Male certificate I and certificate II
graduates are in jobs with similar occupational status but earn slightly
less than other similar individuals. This finding is likely due to the effect
of lower training wages for people undertaking an apprenticeship. Full details
are shown in table 3.
Treatment
|
Control
|
Difference
|
T
|
P
> |t|
|
|
Completed or undertaking a certificate
III or higher1
|
29.5%
|
23.2%
|
6.3%
|
1.46
|
0.146
|
Completed or undertaking an
apprenticeship or traineeship1
|
28.9%
|
19.2%
|
9.8%
|
2.35
|
0.019
|
Employed1
|
83.5%
|
82.1%
|
1.4%
|
0.35
|
0.726
|
Overall life satisfaction1
|
4.5
|
4.4
|
0.1
|
1.72
|
0.086
|
Average hourly wage2
|
$17.05
|
$20.19
|
-$3.14
|
-1.47
|
0.144
|
Occupational status3
|
31.9
|
31.4
|
0.5
|
0.29
|
0.770
|
Notes: 1
N = 242
2
N = 178
3
N = 191
Source: LSAY,
Y05 and Y98 cohorts.
After two years, female certificate I
and II graduates are also more likely to be undertaking or have completed an
apprenticeship or apprenticeship when compared with other similar individuals,
by a similar margin to males (8.4 percentage points). Female certificate I and
II graduates are more likely to be employed (79.7% compared with 72.1% in the
control group). There are also benefits in terms of completing or undertaking a
certificate III or higher qualification, overall life satisfaction and hourly
wage, but the differences are not statistically significant. Females in the
treatment group work in jobs with slightly lower occupational status, but the
difference is not statistically significant. The outcomes are summarised in
table 4.
Treatment
|
Control
|
Difference
|
T
|
P
< |t|
|
|
Completed or undertaking a certificate
III or higher1
|
31.0%
|
25.1%
|
5.9%
|
1.32
|
0.187
|
Completed or undertaking an
apprenticeship or traineeship1
|
18.7%
|
10.3%
|
8.4%
|
2.20
|
0.028
|
Employed1
|
79.7%
|
72.1%
|
7.5%
|
1.69
|
0.092
|
Overall life satisfaction1
|
4.5
|
4.4
|
0.1
|
1.21
|
0.228
|
Average hourly wage
|
$28.03
|
$26.47
|
$1.56
|
0.16
|
0.871
|
Occupational status
|
36.0
|
38.3
|
-2.3
|
-1.04
|
0.300
|
Notes: 1
N = 231
2
N = 142
3
N = 154
Source: LSAY,
Y05 and Y98 cohorts.
Outcomes at age 26
To assess whether the benefits of
completing a certificate I or II are short-term or long lasting, we repeated
the entire process (construction of treatment and control group, propensity
score calculation, and matching) looking only at respondents who remain in the
survey until age 26. Respondents are still matched on a year-by-year basis.
Because of attrition, we have approximately half the number of treated cases
available for matching.
Table 5 shows the results for males at age 26.
The pathway into apprenticeship effect that was evident after two years has
strengthened. By between 11 and 13 percentage points, males in the treatment
group are more likely to have completed a certificate III or higher
qualification, more likely to be undertaking a certificate III or higher
qualification, and more likely to have completed or be undertaking an
apprenticeship or traineeship. All three outcomes are related, since
apprenticeships typically involve undertaking a qualification at certificate
III level. There are small, positive non-significant differences between the
treatment group and the control group in the proportion employed, and in their average
life satisfaction and occupational status. As occurred in the results after two
years, there is a small, negative non-significant difference in the average
hourly wage, which can be easily explained by the lower training wages received
by respondents currently undertaking an apprenticeship.
Treatment
|
Control
|
Difference
|
T
|
P
> |t|
|
|
Completed a certificate III or higher1
|
32.0%
|
20.7%
|
11.3%
|
1.85
|
0.066
|
Completed or undertaking a certificate
III or higher1
|
38.0%
|
25.0%
|
13.1%
|
2.17
|
0.032
|
Completed or undertaking an
apprenticeship or traineeship1
|
30.2%
|
18.7%
|
11.5%
|
2.23
|
0.028
|
Employed1
|
92.2%
|
89.3%
|
2.9%
|
0.78
|
0.436
|
Overall life satisfaction1
|
4.5
|
4.5
|
<0.1
|
0.09
|
0.929
|
Average hourly wage2
|
$29.76
|
$31.65
|
-$1.89
|
-0.99
|
0.325
|
Occupational status3
|
39.0
|
37.1
|
1.9
|
0.77
|
0.444
|
Notes: 1
N = 126
2
N = 99
3
N = 114
Source: LSAY,
Y05 and Y98 cohorts.
There is a very different picture
looking at the results for females (table 6). The benefits that were present
after two years have disappeared by age 26. On average, participation in the
treatment group leads to poorer outcomes, although the differences are
relatively small and only in one instance (occupational status) does the
difference approach conventional levels of statistical significance. However,
the reason for the difference between the two sets of results is because
females in the control group have ‘caught up’ and now have employment and
further education and training outcomes that are comparable with females who
complete lower-level qualifications. This suggests that over time, a variety of
alternative paths are open to females to improve their education and labour
market chances.
Treatment
|
Control
|
Difference
|
T
|
P
< |t|
|
|
Completed a certificate III or higher1
|
36.3%
|
38.6%
|
-2.3%
|
-0.40
|
0.690
|
Completed or undertaking a certificate
III or higher1
|
41.3%
|
46.0%
|
-4.7%
|
-0.81
|
0.421
|
Completed or undertaking an
apprenticeship or traineeship1
|
16.9%
|
14.1%
|
-2.8%
|
-0.59
|
0.554
|
Employed1
|
80.0%
|
81.2%
|
-1.1%
|
-0.18
|
0.854
|
Overall life satisfaction1
|
4.5
|
4.6
|
-0.1
|
-1.07
|
0.286
|
Average hourly wage
|
$25.90
|
$31.84
|
-5.94
|
-1.50
|
0.139
|
Occupational status
|
43.1
|
48.4
|
-5.3
|
-1.73
|
0.087
|
Notes: 1
N = 130
2
N = 70
3
N = 88
Source: LSAY,
Y05 and Y98 cohorts
Distribution
To gauge for whom certificate I and II
qualifications have the greatest impact, we devised a simple experiment. We
took the matched sample for looking at outcomes and ranked it by the propensity
score of the treated cases. We then divided into two, splitting the top 50%
(those most likely to complete a certificate I or certificate II, based on
their background characteristics), and the bottom 50% (those least likely to
complete a certificate I or certificate II). Recall that there was a range of
characteristics that predicted whether someone would complete a certificate I
or certificate II, but the strongest across both sexes included lower maths
scores, lower tertiary entrance rank (where present), having a disability,
having one parent from a non-English speaking country and experiencing
unemployment before undertaking the qualification. Splitting the sample in this
way provides an indication of whether the boost from completing a certificate I
or certificate II is larger for these individuals, which would suggest targeting
qualifications could lead to more efficient policy outcomes.
The results, shown in tables 7 and 8, provide
partial support for the targeted thesis. However, the results further confirm
that the benefits are experienced differently for males and females. For males,
the benefit is primarily related to further participation in training through
apprenticeships. For females, the employment benefit is stronger among those
most likely to complete a certificate I or certificate II, whereas the training
pathway is more evenly distributed.
Most
disadvantaged4
|
Least
disadvantaged4
|
|
Completed or undertaking a certificate
III or higher1
|
11.6%*
|
8.4%
|
Completed or undertaking an
apprenticeship or traineeship1
|
14.6%*
|
4.8%
|
Employed1
|
1.7%
|
1.0%
|
Overall life satisfaction1
|
0.1
|
0.1
|
Average hourly wage2
|
-$2.79
|
-$3.50*
|
Occupational status3
|
1.9
|
-0.9
|
Notes: 1
N = 121
2
N = 96
3
N = 89
4
‘Most disadvantaged’ category comprises the top 50% of treated cases, ranked by
propensity score.
‘Least disadvantaged” comprises the bottom 50% of treated cases, ranked by propensity score.
‘Least disadvantaged” comprises the bottom 50% of treated cases, ranked by propensity score.
*
p < 0.1
Source: LSAY,
Y05 and Y98 cohorts.
Most
disadvantaged4
|
Least
disadvantaged4
|
|
Completed or undertaking a certificate
III or higher1
|
5.5%
|
6.4%
|
Completed or undertaking an
apprenticeship or traineeship1
|
9.1%*
|
7.3%
|
Employed1
|
12.8%**
|
6.9%
|
Overall life satisfaction1
|
0.1
|
<0.1
|
Average hourly wage2
|
$4.40
|
-$2.12
|
Occupational status3
|
-2.3
|
-2.8
|
Notes: 1
N = 116
2
N = 77
3
N = 71
4
‘Most disadvantaged’ category comprises the top 50% of treated cases, ranked by
propensity score.
‘Least disadvantaged” comprises the bottom 50% of treated cases, ranked by propensity score.
‘Least disadvantaged” comprises the bottom 50% of treated cases, ranked by propensity score.
*
p < 0.1
**
p< 0.05
Source: LSAY,
Y05 and Y98 cohorts.
Discussion
Most of the
evidence published to date has shown poor outcomes for certificate I and II
graduates. To overcome some of the limitations of previous studies, we used
propensity score matching to test the treatment effect of a certificate I or
certificate II qualification. However, we have also constructed a very narrow
control group, excluding anybody who has completed a certificate III or higher
qualification as well as anybody who is already studying for a qualification at
certificate III level or above.
Table 9 summarises the treatment effects of
certificate I and certificate II qualifications for young people. Initially, both
males and females exhibit generally positive outcomes after completing a
certificate I or certificate II. However, by age 26, the gap between the
treatment group and the control group remains for males but has disappeared for
females.
Outcome
|
After
two years
|
At
age 26
|
||
Males
|
Females
|
Males
|
Females
|
|
Completed a certificate III or higher
|
NA
|
NA
|
+
|
-
|
Completed or undertaking a certificate
III or higher qualification
|
+
|
+
|
+
|
-
|
Completed or undertaking an
apprenticeship or traineeship
|
+
|
+
|
+
|
-
|
Employed
|
+
|
+
|
+
|
-
|
Hourly wage
|
-
|
+
|
-
|
-
|
Occupational status
|
+
|
-
|
+
|
-
|
Life satisfaction
|
+
|
+
|
+
|
-
|
Note: Shading
indicates statistically significant difference (p<0.1).
Thus, in the longer run we see
positive outcomes for males undertaking lower-level qualifications but the
benefits are not apparent for females. We suggest that lower-level
qualifications work as a pathway into apprenticeships in the traditional
trades. This not only explains the large difference at age 26 between the
treatment and control groups in the proportion of males undertaking or having
completed an apprenticeship but also the proportion who have completed or are
undertaking a certificate III or higher qualification (since apprenticeships in
the traditional trades are typically at the certificate III level).
In the short-term, outcomes for females are
improved by completing a certificate I or certificate II qualification. Two
years after completing the qualification, female certificate I and certificate
II graduates are more likely to be employed and are more likely to be
undertaking or have completed an apprenticeship or traineeship. By 26 years,
this benefit has disappeared, not because outcomes for females in the treatment
group deteriorate but because females in the control group catch up. We suggest
this reflects the multiple paths into occupations typically held by females,
such as traineeships, direct entry into VET study at higher levels and
employment without any further post-school study. By age 26, the impact of
these different paths has evened out. This finding is consistent with Karmel
and Liu (2011), who found that the best pathway for females is clearly completion
of Year 12 followed by university study, whether they have a relatively high or
low academic orientation. It should be borne in mind that the labour market
during the survey period (1995—2006 for the Y95 cohort and 1998—2009 for the
Y98 cohort) was relatively strong. The employment benefits of lower-level
qualifications for females could be more persistent during times of higher
unemployment.
Overall, our findings suggest that lower-level
qualifications offer distinctive benefits to young males and females, provided
they do not have an alternative viable pathway into higher study or training.
For males, the advantage conferred by lower-level qualifications is as a
pathway into apprenticeships. The benefits are enduring, still apparent at age
26. For females, the benefits of lower-level qualifications appear more general
and pre-vocational in nature. There is a short-term boost to employment levels
(and participation in apprenticeships and traineeships) but the advantage is no
longer apparent at age 26. At least in good economic times, alternative
pathways for females (such as traineeships, direct entry into certificate III
level study or higher, or finding employment without completing any post-school
study) provide comparable outcomes over the longer-term.
References
AQF (Australian Qualifications Framework)
Council 2011, Australian Qualifications Framework,
AQF Council, Adelaide.
Cully, M & Curtain, R 2001, ‘New
apprenticeships: an unheralded labour market program’, Australian Bulletin of Labour, vol.27, no.3, pp.204—15.
Dawe, S 2004, Moving on from enabling courses: why do some students remain in
enabling courses? NCVER, Adelaide.
Foley, P & Blomberg, D 2011, Pre-apprenticeship training activity,
NCVER, Adelaide.
Harris, R, Rainey, L & Sumner, R 2006,
Crazy paving or stepping stones? Learning
pathways within and between VET and higher education, NCVER, Adelaide.
Karmel, T, Blomberg, D & Vnuk, M 2010,
The effectiveness of the traineeship
model, NCVER, Adelaide.
Karmel, T & Fieger, P 2012, The value of completing a qualification,
NCVER, Adelaide.
Karmel, T & Liu, S 2011, Which paths work for which young people?
NCVER, Adelaide.
Karmel, T & Nguyen, N 2007, The value of completing a vocational
education and training qualification, NCVER, Adelaide.
Leigh, A 2008, ‘Returns to education in
Australia’, Economic Papers, vol.27,
no.3, pp.233—49.
Leuven, E & Sianesi, B 2003, PSMATCH2:
Stata module to perform full Mahalanobis and propensity score matching, common
support graphing, and covariate imbalance testing, viewed November 2011, <http://ideas.repec.org/c/boc/bocode/s432001.html>.
Lim, P & Karmel, T 2011, The vocational equivalent to Year 12,
NCVER, Adelaide.
Long, M & Shah, C 2008, Private returns to vocational education and
training qualifications, NCVER, Adelaide.
McMillan, J & Jones, FL 2000, ‘The
ANU3_2 scale: a revised occupational status scale for Australia’, Journal of Sociology, vol.36, no.1,
pp.64—80.
McMillan,
J, Beavis, A & Jones, FL 2009, ‘The AUSEI06: a new socioeconomic index for
Australia’, Journal of Sociology, vol.45,
no.2, pp.123—49.
NCVER (National Centre for Vocational
Education Research) 2011a, Australian
vocational education and training statistics: student outcomes 2010, NCVER,
Adelaide.
——2011b, Australian vocational education and training statistics: students and
courses 2010, NCVER, Adelaide.
Rosenbaum, P 2002, Observational studies,
Springer, New York.
Rosenbaum, P & Rubin, D 1985, ‘Constructing
a control group using multivariate matched sampling methods that incorporate
the propensity score’, The American
Statistician, vol.39, no.1, pp.33—8.
Ryan, C 2002, Individual returns to vocational education and training qualifications:
their implications for lifelong learning, NCVER, Adelaide.
Stanwick, J 2005, Australian Qualifications Framework lower-level qualifications: pathways
to where for young people, NCVER, Adelaide.
Appendix A
Certificate I
|
Certificate II
|
Higher-level
qual.
|
|
Student
characteristics
|
|||
Male
|
67.0
|
55.5
|
56.6
|
Female
|
32.8
|
44.4
|
43.3
|
Not
known
|
0.2
|
0.1
|
0.1
|
20 to
24 years
|
20.7
|
20.1
|
49.1
|
19
years and under
|
79.3
|
79.9
|
50.9
|
Indigenous
|
15.0
|
7.6
|
3.1
|
Not
indigenous
|
74.5
|
85.6
|
90.4
|
Not
known
|
10.5
|
6.8
|
6.5
|
With a
disability
|
11.3
|
6.1
|
4.4
|
Without
a disability
|
58.2
|
67.1
|
86.0
|
Not
known
|
30.5
|
26.8
|
9.6
|
Still
attending school
|
50.5
|
49.6
|
11.1
|
Not
attending school
|
43.1
|
45.7
|
82.4
|
Not
known
|
6.4
|
4.7
|
6.5
|
Highest
school level of those not at school:
|
|||
Year
12
|
22.9
|
35.4
|
58.2
|
Year
11
|
12.2
|
14.3
|
13.5
|
Year
10
|
24.5
|
27.8
|
20.8
|
Year 9
or lower
|
29.5
|
17.2
|
3.8
|
Did
not attend school
|
0.9
|
0.4
|
0.3
|
Not
known
|
10.1
|
4.9
|
3.4
|
Course
is part of an apprenticeship or traineeship
|
3.4
|
16.3
|
44.2
|
Course
is not part of an apprenticeship or traineeship
|
96.6
|
83.7
|
55.8
|
Course
characteristics
|
|||
Field
of study
|
|||
03 -
Engineering and related technologies
|
13.8
|
17.5
|
21.1
|
04 -
Architecture and building
|
21.2
|
7.8
|
12.2
|
08 -
Management and commerce
|
12.7
|
26.7
|
22.0
|
11 -
Food, hospitality and personal services
|
13.5
|
21.2
|
11.6
|
12 -
Mixed field programmes
|
29.8
|
8.5
|
2.3
|
All
other fields of education
|
9.1
|
18.3
|
30.5
|
Course
is part of a pre-apprenticeship
|
32.6
|
20.3
|
na
|
Course
is not part of a pre-apprenticeship
|
67.4
|
79.7
|
na
|
Total (N)
|
141 475
|
401 122
|
1 182 722
|
Source: National
VET Provider Collection 2010, custom data.
Qualification
|
Higher
level
|
Lower level
|
||
Certificate II
|
Certificate I
|
Total
|
||
Employed
before training
|
||||
Employed
|
71.6
|
57.6
|
34.8
|
53.5
|
Not employed
|
25.8
|
40.0
|
62.2
|
44.0
|
Not stated
|
2.5
|
2.4
|
3.0
|
2.5
|
Employed
after training
|
||||
Employed
|
78.8
|
64.5
|
50.1
|
61.9
|
Not employed
|
19.1
|
34.1
|
46.4
|
36.3
|
Not stated
|
2.1
|
1.4
|
3.5
|
1.8
|
Employed
or in further study
|
||||
Emp or in FS
|
88.9
|
79.6
|
71.7
|
78.1
|
Not Emp or in FS
|
9.0
|
18.7
|
25.0
|
19.9
|
Not stated
|
2.1
|
1.7
|
3.3
|
2.0
|
Satisfied
with overall quality of training
|
||||
Agree
|
86.8
|
89.0
|
89.9
|
89.1
|
Disagree
|
4.6
|
3.6
|
1.6
|
3.2
|
Neither agree nor disagree
|
7.0
|
5.6
|
5.6
|
5.6
|
Not stated
|
1.7
|
1.9
|
2.9
|
2.1
|
Achieved
reason for study
|
||||
Wholly or partly
|
86.5
|
84.0
|
80.3
|
83.4
|
No
|
4.3
|
5.8
|
7.0
|
6.0
|
Don’t know yet
|
8.3
|
8.8
|
10.5
|
9.1
|
Not stated
|
0.9
|
1.4
|
2.2
|
1.5
|
Total
|
100.0
|
100.0
|
100.0
|
100.0
|
Total (N)
|
148 422
|
67 402
|
14 846
|
82 248
|
Source: NCVER
2010 Student Outcomes Survey, custom data.
Appendix B
Statistical methodology
The logistic
model used to estimate the probability of completing a lower-level VET
qualification follows Lim and Karmel (2011), who modelled Year 12 completion
also using the Y95 and Y98 LSAY cohorts. Table B1 provides a summary of the
variables entered into the model. In addition, the Y95 or Y98 cohort was
entered as a random effect. The regression was weighted using the most recent
weight calculated for each respondent.
The motivation
items were selected on the following basis. In both cohorts, respondents were
asked in wave one 20 items about how they felt about school. A factor analysis
identified four separate factors:
§ the first associated with liking learning
§ the second associated with a positive view of
self as student
§ the third associated with receiving fair
treatment at school
§ the fourth associated with viewing school as
useful for life as an adult.
Variable
|
Values
|
Background
|
|
State
|
ACT (reference
category)
|
NSW
|
|
Vic.
|
|
Qld
|
|
SA
|
|
WA
|
|
Tas.
|
|
NT
|
|
School type
|
Government
|
Catholic
|
|
Independent
|
|
Respondent’s highest school level
|
Year 10 or below
|
Year 11
|
|
Year 12
|
|
Parental occupational status
|
Measured based on ASCO, reported in wave
1, using ANU3 scale (McMillan & Jones 2000).
Higher score is taken. Results then
divided into quartiles.
Fifth category comprises cases where
there is no occupational status
(either because no parent is in the labour force or because of missing data). |
Parents’ highest education level
|
4 Completed university
|
3 Technical or trade qualification
|
|
2 Completed secondary school
|
|
1 Some secondary school
|
|
9 Missing information from both parents
|
|
Students’ country of birth
|
1 if Australia
|
2 if another English speaking country
|
|
3 if a non-English speaking country
|
|
Parents’ country of birth
|
1 if neither parent born in a non-English
speaking country
|
2 if one parent born in a non-English
speaking country
|
|
3 if both parents born in a non-English
speaking country
|
|
Indigenous
|
1 if Indigenous, 0 otherwise
|
With a disability
|
1 if has a disability, 0 otherwise
|
Size of local area:
|
1 Metropolitan
|
2 Regional
|
|
3 Rural or remote
|
|
9 Missing locality information
|
|
Ability
|
|
Reading score
|
Score between 0 and 20, measured in wave
1
|
Maths score
|
Score between 0 and 20, measured in wave
1
|
TER
|
Tertiary entrance rank (0–100)
|
TER missing
|
No TER, because respondent did not
complete Year 12, was not otherwise eligible for a TER, or because TER is
missing.
|
Motivation
|
|
Views learning as fun
|
Five-point scale between -2 and 2
|
Treated fairly in class
|
Five-point scale between -2 and 2
|
Views self as successful student
|
Five-point scale between -2 and 2
|
School is useful to later life
|
Five-point scale between -2 and 2
|
Unemployment
history (t-2)
|
Respondent had at least one episode of
unemployment in the 12-month period two years before the qualification could
have been completed.
|
Notes: ASCO = Australian Standard Classification of
Occupations.
Table B2 Outcome
variables
Variable
|
Values
|
Completed a certificate III or higher
qualification
|
1 Completed a certificate III or higher
qualification
0 Has not completed a certificate III or
higher qualification
|
Completed or undertaking an
apprenticeship or traineeship
|
1 Completed or undertaking an
apprenticeship or traineeship
0 Has never commenced an apprenticeship
or traineeship or commenced but did not complete
|
Employed
|
1 Employed
0 Not employed (unemployed or not in
labour force)
|
Hourly wage
|
Deflated to 1995 dollars
|
Occupational status
|
ANU Occupational status scale based on
ASCO (McMillan & Jones 2000) or ANZSCO (McMillan, Bevis & Jones 2009).
Values range from 0 to 100.
|
Life satisfaction
|
Five-point scale. Response to question ‘How
satisfied are you with … your life overall’.
|
The propensity scores
The logistic
model used to determine the propensity scores is:
where 
represents the design matrices for the intercept and the independent
variables, 
the
regression co-efficients for the intercept and independent variables, 
, the design matrices for the random cohort
factor, 
, represents the random regression
co-efficients for cohort, further we assume 
, and 
, and 
. Further, we note that 
, that is, 
and 
are
uncorrelated.
represents the design matrices for the intercept and the independent
variables, the
regression co-efficients for the intercept and independent variables, , the design matrices for the random cohort
factor, , represents the random regression
co-efficients for cohort, further we assume , and , and . Further, we note that , that is, and are
uncorrelated.
The probability
of completing a certificate I or certificate II as the first qualification
after leaving school is determined using:
Table B3 Model estimates from
the propensity scores regression, males (for outcomes after two years)
Parameter
|
Categories
|
Estimate
|
Standard error
|
Pr
> |z|
|
Intercept
|
-1.520
|
2.486
|
0.541
|
|
Time
|
-1.520
|
2.486
|
0.541
|
|
Time^2
|
-0.430
|
0.757
|
0.570
|
|
State
|
ACT
|
reference category
|
||
NSW
|
0.708
|
0.168
|
<0.001
|
|
Vic.
|
0.982
|
0.178
|
<0.001
|
|
QLD
|
0.716
|
0.137
|
<0.001
|
|
SA
|
0.905
|
0.078
|
<0.001
|
|
WA
|
1.056
|
0.101
|
<0.001
|
|
Tas.
|
0.634
|
0.432
|
0.142
|
|
NT
|
1.031
|
0.349
|
0.003
|
|
School type
|
Government
|
reference category
|
||
Catholic
|
-0.145
|
0.028
|
<0.001
|
|
Independent
|
-0.481
|
0.265
|
0.070
|
|
Highest school level
|
Year 12
|
reference category
|
||
Year 11
|
-0.319
|
0.284
|
0.260
|
|
Year 10 or below
|
-0.740
|
0.196
|
0.000
|
|
Reading score
|
0.029
|
0.019
|
0.115
|
|
Maths score
|
-0.081
|
0.003
|
<0.001
|
|
Tertiary entrance rank
|
-0.011
|
0.006
|
0.048
|
|
Missing tertiary entrance rank
|
-0.409
|
0.370
|
0.269
|
|
Parental occupational status
|
Bottom quartile
|
reference category
|
||
Second quartile
|
-0.003
|
0.121
|
0.979
|
|
Third quartile
|
-0.289
|
0.191
|
0.129
|
|
Top quartile
|
-0.334
|
0.097
|
0.001
|
|
None (unemployed or missing)
|
0.263
|
0.034
|
<0.001
|
|
Parents’ highest education level
|
University
|
reference category
|
||
Technical or trade qualification
|
0.104
|
0.092
|
0.255
|
|
Completed secondary school
|
0.323
|
0.110
|
0.003
|
|
Some secondary school
|
0.592
|
0.086
|
<0.001
|
|
No information available
|
-0.161
|
0.118
|
0.174
|
|
Students’ country or birth
|
Australia
|
reference category
|
||
Another English-speaking country
|
-0.464
|
0.224
|
0.038
|
|
Non-English speaking country
|
-0.794
|
0.343
|
0.021
|
|
0.521
|
0.246
|
0.034
|
||
Parents’ country of birth
|
Neither born in a non-English
speaking country
|
reference category
|
||
One born in a non-English speaking
country
|
0.494
|
0.069
|
<0.001
|
|
Both born in a non-English speaking
country
|
0.228
|
0.457
|
0.619
|
|
Motivation factors
|
Views learning as fun
|
-0.134
|
0.001
|
<0.001
|
Treated fairly in class
|
0.130
|
0.077
|
0.090
|
|
Views self as successful student
|
-0.035
|
0.084
|
0.677
|
|
Thinks school will help with life as
adult
|
0.043
|
0.037
|
0.248
|
|
Indigenous
|
-0.156
|
0.457
|
0.732
|
|
With a disability
|
0.731
|
0.122
|
<0.001
|
|
Size:
|
Metropolitan
|
reference category
|
||
Regional
|
0.318
|
0.052
|
<0.001
|
|
Rural or remote
|
0.415
|
0.111
|
<0.001
|
|
Missing location information
|
-0.064
|
0.202
|
0.752
|
|
Unemployed, t-2
|
0.861
|
0.082
|
<0.001
|
|
Parameter
|
Categories
|
Estimate
|
Standard error
|
Pr
> |z|
|
Intercept
|
0.251
|
0.913
|
0.783
|
|
Time
|
-0.576
|
0.287
|
0.045
|
|
Time^2
|
0.019
|
0.022
|
0.389
|
|
State
|
ACT
|
reference
category
|
||
NSW
|
0.771
|
0.434
|
0.076
|
|
Vic.
|
0.419
|
0.105
|
<.0001
|
|
QLD
|
0.285
|
0.090
|
0.002
|
|
SA
|
0.274
|
0.030
|
<.0001
|
|
WA
|
0.588
|
0.077
|
<.0001
|
|
Tas.
|
0.156
|
0.147
|
0.290
|
|
NT
|
0.347
|
0.200
|
0.083
|
|
School type
|
Government
|
reference
category
|
||
Catholic
|
-0.384
|
0.011
|
<.0001
|
|
Independent
|
0.446
|
0.051
|
<.0001
|
|
Highest school level
|
Year 12
|
reference
category
|
||
Year 11
|
-0.326
|
0.564
|
0.563
|
|
Year 10 or below
|
-0.565
|
0.402
|
0.160
|
|
Reading score
|
-0.021
|
0.019
|
0.261
|
|
Maths score
|
-0.050
|
0.025
|
0.047
|
|
Tertiary entrance rank
|
-0.010
|
0.001
|
<.0001
|
|
Missing tertiary entrance rank
|
-0.462
|
0.032
|
<.0001
|
|
Parental occupational status
|
Bottom quartile
|
reference
category
|
||
Second quartile
|
-0.400
|
0.125
|
0.001
|
|
Third quartile
|
0.090
|
0.187
|
0.629
|
|
Top quartile
|
-0.307
|
0.005
|
<.0001
|
|
None (unemployed or missing)
|
0.219
|
0.493
|
0.656
|
|
Parents’ highest education level
|
University
|
reference
category
|
||
Technical or trade qualification
|
-0.175
|
0.253
|
0.489
|
|
Completed secondary school
|
-0.348
|
0.040
|
<.0001
|
|
Some secondary school
|
0.091
|
0.095
|
0.336
|
|
No information available
|
-0.331
|
0.036
|
<.0001
|
|
Students’ country or birth
|
Australia
|
reference
category
|
||
Another English-speaking country
|
0.080
|
0.190
|
0.676
|
|
Non-English speaking country
|
0.110
|
0.078
|
0.158
|
|
-0.439
|
0.577
|
0.447
|
||
Parents’ country of birth
|
Neither born in a non-English
speaking country
|
reference
category
|
||
One born in a non-English speaking
country
|
0.284
|
0.091
|
0.002
|
|
Both born in a non-English speaking
country
|
0.195
|
0.299
|
0.513
|
|
Motivation factors
|
Views learning as fun
|
0.042
|
0.036
|
0.247
|
Treated fairly in class
|
0.033
|
0.056
|
0.549
|
|
Views self as successful student
|
-0.153
|
0.024
|
<.0001
|
|
Thinks school will help with life as
adult
|
0.198
|
0.046
|
<.0001
|
|
Indigenous
|
-0.430
|
0.036
|
<.0001
|
|
With a disability
|
0.215
|
0.057
|
0.000
|
|
Size:
|
Metropolitan
|
reference
category
|
||
Regional
|
0.001
|
0.175
|
0.997
|
|
Rural or remote
|
0.043
|
0.090
|
0.637
|
|
Missing location information
|
-0.076
|
0.059
|
0.199
|
|
Unemployed, t-2
|
0.521
|
0.026
|
<.0001
|
|
Propensity score models (and matching routines)
were re-estimated for those in employment only and with outcomes after two
years, for all respondents in the treatment and control groups with outcomes at
age 26, and for employed respondents only with outcomes at age 26. These were
similar to the initial models. Details are available upon request to the
author.
Sample balance
Demographic
|
Distribution
before matching (%) |
P
> Χ2
|
Distribution
after matching (%) |
P
> Χ2
|
State
|
||||
ACT
|
1.31
|
0.253
|
28.8
|
0.046
|
NSW
|
2.50
|
56.1
|
||
Vic.
|
3.20
|
56.3
|
||
QLD
|
2.06
|
48.8
|
||
SA
|
3.28
|
44.0
|
||
WA
|
3.73
|
48.0
|
||
Tas.
|
2.21
|
34.8
|
||
NT
|
2.86
|
19.1
|
||
Indigenous
status
|
||||
Non-indigenous
|
2.68
|
0.014
|
49.5
|
0.554
|
Indigenous
|
3.47
|
42.4
|
||
Disability
status
|
||||
Not with a disability
|
2.59
|
<
0.001
|
49.4
|
0.309
|
With a disability
|
6.52
|
40.4
|
||
Highest
school level
|
||||
Year 12
|
2.32
|
<
0.001
|
47.7
|
0.440
|
Year 11
|
3.31
|
51.2
|
||
Year 10 or below
|
3.39
|
54.4
|
||
TER
status
|
||||
TER
|
1.89
|
<
0.001
|
45.8
|
0.199
|
No TER
|
3.19
|
51.9
|
||
School
type
|
||||
Government
|
2.95
|
0.04
|
52.65
|
0.046
|
Catholic
|
2.33
|
45.0
|
||
Independent
|
1.41
|
34.1
|
||
Size
of local area
|
||||
Metropolitan
|
2.30
|
0.001
|
51.4
|
0.457
|
Regional
|
3.11
|
48.6
|
||
Rural or remote
|
3.14
|
51.4
|
||
Size information missing
|
3.05
|
28.5
|
||
Parents’
highest education level
|
||||
Some secondary school
|
2.48
|
0.203
|
49.5
|
0.218
|
Secondary school
|
2.34
|
57.1
|
||
Trade or technical qualification
|
3.01
|
45.6
|
||
University qualification
|
2.73
|
45.6
|
||
Missing information for both parents
|
2.91
|
58.0
|
||
Parental
occupational status
|
||||
Bottom quartile
|
2.93
|
<
0.001
|
48.6
|
0.732
|
Second quartile
|
2.61
|
50.1
|
||
Third quartile
|
2.01
|
53.9
|
||
Top quartiles
|
1.95
|
43.5
|
||
Both parents missing occupational status
|
4.21
|
52.6
|
Demographic
|
Distribution
before matching (%) |
P
> Χ2
|
Distribution
after matching (%) |
P
> Χ2
|
Student’s
country of birth
|
||||
Australia
|
2.71
|
0.004
|
49.1
|
0.510
|
Another English speaking country
|
1.66
|
67.5
|
||
A non-English speaking country
|
1.21
|
47.7
|
||
Missing country of birth
|
4.71
|
59.9
|
||
Parents’
countries of birth
|
||||
Neither born in a non-English speaking
country
|
2.70
|
0.027
|
49.3
|
0.689
|
One parent born in a non-English speaking
country
|
3.30
|
50.5
|
||
Both parents born in a non-English
speaking country
|
2.23
|
56.6
|
||
Unemployment
history
|
||||
Period of unemployment, t-2
|
5.12
|
<
0.001
|
48.7
|
0.597
|
No period of unemployment, t-2
|
1.64
|
51.1
|
Before
matching
|
After
matching
|
|||||
Treatment
|
Control
|
p
> |t|
|
Treatment
|
Control
|
p
> |t|
|
|
Maths test
|
10.6
|
11.9
|
<
0.001
|
10.9
|
11.1
|
0.520
|
Reading test
|
11.5
|
12.0
|
0.056
|
11.6
|
12.0
|
0.267
|
TER (where recorded)
|
53.1
|
59.3
|
0.022
|
54.9
|
53.4
|
0.682
|
Demographic
|
Distribution
before matching (%) |
P
> Χ2
|
Distribution
after matching (%) |
P
> Χ2
|
State
|
||||
ACT
|
2.30
|
0.028
|
28.1
|
0.003
|
NSW
|
4.14
|
59.0
|
||
Vic.
|
2.85
|
57.3
|
||
QLD
|
2.51
|
52.1
|
||
SA
|
2.67
|
49.5
|
||
WA
|
3.02
|
34.9
|
||
Tas.
|
2.39
|
39.2
|
||
NT
|
2.39
|
24.1
|
||
Indigenous
status
|
||||
Non-indigenous
|
3.09
|
0.923
|
50.5
|
0.802
|
Indigenous
|
2.71
|
47.0
|
||
Disability
status
|
||||
Not with a disability
|
3.04
|
0.030
|
49.8
|
0.456
|
With a disability
|
5.05
|
60.4
|
||
Highest
school level
|
||||
Year 12
|
2.87
|
<
0.001
|
48.9
|
0.780
|
Year 11
|
2.96
|
51.4
|
||
Year 10 or below
|
3.87
|
52.6
|
||
TER
status
|
||||
TER
|
3.48
|
0.005
|
47.6
|
0.392
|
No TER
|
2.49
|
51.5
|
||
School
type
|
||||
Government
|
3.22
|
0.022
|
52.1
|
0.149
|
Catholic
|
2.03
|
48.2
|
||
Independent
|
3.76
|
39.1
|
||
Size
of local area
|
||||
Metropolitan
|
3.14
|
0.048
|
47.4
|
0.286
|
Regional
|
2.92
|
55.3
|
||
Rural or remote
|
3.17
|
51.2
|
||
Size information missing
|
2.79
|
25.9
|
||
Parents’
highest education level
|
||||
Some secondary school
|
3.88
|
0.062
|
57.1
|
0.396
|
Secondary school
|
2.91
|
52.1
|
||
Trade or technical qualification
|
2.63
|
48.2
|
||
University qualification
|
3.22
|
44.6
|
||
Missing information for both parents
|
2.71
|
48.3
|
||
Parental
occupational status
|
||||
Bottom quartile
|
3.60
|
0.021
|
53.2
|
0.601
|
Second quartile
|
2.27
|
55.6
|
||
Third quartile
|
3.31
|
46.9
|
||
Top quartiles
|
2.33
|
46.2
|
||
Both parents missing occupational status
|
4.03
|
47.0
|
Demographic
|
Distribution
before matching (%) |
P
> Χ2
|
Distribution
after matching (%) |
P
> Χ2
|
Student’s
country of birth
|
||||
Australia
|
3.05
|
0.197
|
50.5
|
0.652
|
Another English-speaking country
|
3.31
|
58.2
|
||
A non-English speaking country
|
3.83
|
43.1
|
||
Missing country of birth
|
2.88
|
39.1
|
||
Parents’
countries of birth
|
||||
Neither born in a non-English speaking
country
|
2.92
|
<
0.001
|
51.2
|
0.487
|
One parent born in a non-English speaking
country
|
3.74
|
49.8
|
||
Both parents born in a non-English
speaking country
|
3.94
|
43.2
|
||
Unemployment
history
|
||||
Period of unemployment, t-2
|
2.46
|
< 0.001
|
54.6
|
0.023
|
No period of unemployment, t-2
|
4.60
|
44.9
|
Before
matching
|
After
matching
|
|||||
Treatment
|
Control
|
p
> |t|
|
Treatment
|
Control
|
p
> |t|
|
|
Maths test
|
10.5
|
11.4
|
<
0.001
|
10.7
|
11.4
|
0.045
|
Reading test
|
12.1
|
12.9
|
<
0.001
|
12.4
|
12.7
|
0.483
|
TER
|
57.6
|
61.0
|
0.169
|
59.9
|
57.6
|
0.518
|
No comments:
Post a Comment