Quantitative analysis of a Māori and Pacific admission process on first-year health study

Background Universities should provide flexible and inclusive selection and admission policies to increase equity in access and outcomes for indigenous and ethnic minority students. This study investigates an equity-targeted admissions process, involving a Multiple Mini Interview and objective testing, advising Māori and Pacific students on their best starting point for academic success towards a career in medicine, nursing, health sciences and pharmacy. Methods All Māori and Pacific Admission Scheme (MAPAS) interviewees enrolled in bridging/foundation or degree-level programmes at the University of Auckland were identified (2009 to 2012). Generalised linear regression models estimated the predicted effects of admission variables (e.g. MAPAS Maths Test; National Certificate in Educational Achievement (NCEA) Rank Score; Any 2 Sciences; Followed MAPAS Advice) on first year academic outcomes (i.e. Grade Point Average (GPA) and Passes All Courses) adjusting for MAPAS interview year, gender, ancestry and school decile. Results 368 First Year Tertiary (bridging/foundation or degree-level) and 242 First Year Bachelor (degree-level only) students were investigated. NCEA Rank Score (estimate 0.26, CI: 0.18-0.34, p< 0.0001); MAPAS Advice Followed (1.26, CI: 0.18-1.34, p = 0.0002); Exposure to Any 2 Sciences (0.651, CI: 0.15-1.15, p = 0.012); and MAPAS Mathematics Test (0.14, CI: 0.02-0.26, p = 0.0186) variables were strongly associated with an increase in First Year Tertiary GPA. The odds of passing all courses in First Year Tertiary study was 5.4 times higher for students who Followed MAPAS Advice (CI: 2.35-12.39; p< 0.0001) and 2.3 times higher with Exposure to Any Two Sciences (CI: 1.15-4.60; p = 0.0186). First Year Bachelor students who Followed MAPAS Advice had an average GPA that was 1.1 points higher for all eight (CI: 0.45-1.73; p = 0.0009) and Core 4 courses (CI: 0.60-2.04; p = 0.0004). Conclusions The MAPAS admissions process was strongly associated with positive academic outcomes in the first year of tertiary study. Universities should invest in a comprehensive admissions process that includes alternative entry pathways for indigenous and ethnic minority applicants.


Background
Worldwide, tertiary institutions are attempting to widen participation to historically underserved populations including indigenous and ethnic minority students [1] Often driven by social inclusion and social accountability policies, universities have devised a number of strategies to increase diversity. Within an indigenous and ethnic minority health workforce context, a pipeline approach is recommended to address well-known barriers to accessing and succeeding in university-level studies. A pipeline approach often includes early exposure interventions aimed at raising aspirations and academic preparation for a career in health [2][3][4]; addressing educational disadvantage via the provision of bridging/ foundation programmes [5,6] and improving student performance by providing comprehensive support programmes [7][8][9]. Given the highly competitive context of health professional programme selection, it is also recommended that universities provide more flexible and inclusive selection and admission policies for students from underserved populations [1,10].
Universities have a choice of selection tools that can be used to inform student admission including prior academic performance, interview scores and results from aptitude tests. Both cognitive and non-cognitive tools are used by universities when selecting students; however it is arguable that prior academic performance remains a dominant tool for medical selection in many universities [11]. Given this reality, indigenous and ethnic minority students are required to aim to achieve a high level of academic performance within the pathways used for future selection into medical or health professional programmes of study [12]. Unfortunately, students from underserved populations are less likely to receive access to science-rich subjects and are more likely to leave high school with lower qualifications than their peers [5,10,13]. Providing an admissions process that can determine whether indigenous and ethnic minority applicants are academically (and socially) ready to achieve success in pre-medical degree pathways and the provision of alternative entry pathways is recommended for tertiary institutions committed to widening participation [14,15].
An extensive body of research identifies the tertiary conditions and factors that impact on academic success within the first year of study at university [16][17][18][19][20]. Indicators of prior academic performance such as: secondary school grade point averages [21]; secondary school factors including markers of socio-geographic status (e.g. school decile) [22]; and student characteristics (e.g. autonomy, confidence, motivation, control) [17,23] have been identified as important factors impacting on academic performance in the first year of study. In addition, factors associated with the environment of the tertiary institution also impact on student engagement; such factors include: opportunities for teachers and students to engage with each other [18]; levels of institutional support to provide environments conducive to learning [20]; and the provision of academic, social and personal support [16].
To date, few studies have explored the effect of equitytargeted admission processes on the academic performance of indigenous and ethnic minority students in their first year of tertiary study. As a result, tertiary institutions have little empirical evidence to understand the effect of equity-targeted selection processes and whether such initiatives are likely to support a widening participation agenda.
This article explores the predictive effect of admission variables associated with an equity-targeted admission process on academic outcomes for Māori (the indigenous peoples of Aotearoa New Zealand) and Pacific (a heterogeneous composite of peoples with Pacific nation ancestry born and/or living in New Zealand) applicants applying under the Māori and Pacific Admission Scheme (MAPAS) to the Faculty of Medical and Health Sciences (FMHS) at the University of Auckland (UoA).

FMHS entry pathways
Admission into FMHS health professional programmes is generally via direct entry into First Year Bachelor level undergraduate study for those applicants who meet the necessary entry requirements [24]. The FMHS also offers a one-year, MAPAS-specific bridging/foundation programme, the Certificate in Health Sciences (CertHSc) through which Māori and Pacific students who achieve a CertHSc GPA above B+ can gain alternative entry into First Year Bachelor undergraduate study. Hence, Māori and Pacific First Year Tertiary students within FMHS could either be enrolled in the CertHSc bridging foundation programme, or, the first year of bachelor level study ( Table 1). The first year of bachelor level study also acts as a 'pre-medical' year prior to admission into the FMHS Medical programme in year 2. Table 1 provides definitions of the Certificate in Health Sciences, First Year Tertiary, and First Year Bachelor terms used within this study (Table 1).

Māori and Pacific Admission Scheme (MAPAS)
MAPAS operates an equity-targeted admissions process for applicants with indigenous Māori and Pacific ancestry. The process aims to gather a broad range of information about Māori and Pacific applicant preparation for tertiary health study. The December interview process involves a Multiple Mini Interview (MMI), an English test and a mathematics test.
The MMI is an alternative form of admission interview that aims to reduce interviewer bias by consisting of a number of short interview stations with multiple interviewers. The MMI has been shown to be reliable, The first year in which a student enrols in a form of tertiary study at the level of a bachelor degree qualification acceptable and feasible in a variety of tertiary health study contexts [25]. In building on the original pilot of the MMI [26], other studies have taken advantage of the intended benefit of the flexibility of station development in their own contexts [27,28]. Whilst the original authors aimed to assess suitability of applicants as health professionals, the MAPAS MMI aims to assess Māori and Pacific applicant preparation for and potential to succeed in FMHS programmes. In the MAPAS context, the MMI has been redeveloped to include four 8-min stations assessing career aspirations; academic preparation; family support and student information. The MAPAS mathematics and English testing are used in addition to the MAPAS MMI to objectively assess academic numeracy and literacy skills. Using MMI and testing information, two assessments are made about: 1) potential to succeed within the CertHSc, and 2) potential to succeed within the Bachelor of: Health Sciences; Science (Biomedicine) 1 ; Nursing; or Pharmacy. Potential to succeed is assessed as: pass, borderline or fail (objective testing) for the English and mathematics testing and few, some, or major concerns (subjective testing) for each MMI station. A MAPAS Recommendations Team reviews the combination of results and provides a provisional MAPAS recommendation (advice regarding the applicant's recommended best starting point given

Methodology
This study used a Kaupapa Māori Research (KMR) approach, broadly defined and responsive to Pacific research methodologies [29,30]. This approach recognises that issues associated with power, privilege and agency within society are hypothesised to act similarly on both Māori and Pacific students [31,32]. In this instance KMR aims to: ensure research outputs are positive for Māori and Pacific students; explicitly challenge 'victim blame' or 'cultural deficit' analyses that may blame Māori or Pacific students for educational failure; and provide a structural analysis to promote institutional change targeting Māori and Pacific student success [14,33]. This research was led by senior Māori and Pacific researchers with input from a FMHS advisory group.

Study design
The predictive effect of MAPAS admission process variables on academic outcomes in the first year of tertiary study was explored. Applicant data were obtained from the MAPAS admissions database and the university's centralised student data management system for all MAPAS interviewees (2008 -2011) who subsequently enrolled in relevant tertiary health programmes (2009 -2012) within the FMHS at the UoA. Approval to complete this research was granted by the University of Auckland Human Participant Ethics Committee (Ref 8110). As per ethics protocols, written informed consent was not required for this research project due to the use of secondary administrative data sources. All secondary data obtained from these datasets were de-identified by an independent research member with no student contact or teaching responsibilities and data analysis occurred via a coding system. Two student cohorts are identified: First Year Tertiary Students i.e. students enrolled in either the CertHSc or the first year of a bachelor programme in the year following their MAPAS interview; and First Year Bachelor Students i.e. students enrolled in a bachelor programme in either the first or second year following their MAPAS interview (may include CertHSc graduates).   have been linked to higher levels of deprivation associated with reduced access to, and outcomes from, tertiary education [34] ( Table 2). Admission predictor variables include: MAPAS Testing results (%); MMI Station results (Some or Major Concerns (SMC) versus Few Concerns (FC)); Provisional December Recommendation (CertHSc, Bachelor, Not FMHS); secondary school results including New Zealand's NCEA Rank Score 3 (out of 320); Level 3 NCEA Subject Credits (number of credits achieved in English, biology, chemistry, physics, mathematics); Exposure to Any 2 Sciences of senior biology, chemistry or physics (yes, no) 4 ; Followed MAPAS Advice (yes, no); and Final January Recommendation made in January (CertHSc, Bachelor, Not FMHS).

Demographic variables include: Year of Admission
Academic outcome variables include: Grade Point Average (GPA) Eight Courses, 0-9 (i.e. GPA achieved across a total of eight courses over the year); GPA Core 4 Courses, 0-9 (i.e. GPA achieved across four core courses 5 taken in the first year of bachelor study that are specifically assessed for selection into second year medicine at the UoA); Passes All Courses, yes/no (i.e. across total of eight courses); Passes All Core 4 Courses, yes/no (i.e. across the four core courses).

Statistical analysis
All downloaded data were recorded in Microsoft Office Excel spread sheets. Statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC, USA). Continuous variables were presented as mean and standard deviation (SD); categorical variables as frequencies (n) and percentages (%) ( Tables 2 and 3). Generalised linear and logistic regression models were used to estimate the predicted effects of individual admission variables on academic outcomes (i.e. GPA and Passes All); adjusting for pre-defined demographic variables (i.e. MAPAS interview year, gender, ancestry and school decile) (Tables 4, 5, 6 and 7). Admission variables that showed significant single predictive effect (i.e. MAPAS Maths Test, NCEA Rank Score, Any 2 Sciences and Followed MAPAS Advice) were included in the multiple regression analyses to determine their joint effects on the academic outcomes of interest (Tables 8 and 9). All statistical tests were two-sided at 5 % significance level.

Descriptive variables
A total of 368 students were identified in the First Year Tertiary cohort. Of these, 37 % were Māori, 57 % Pacific and 6 % had Both Māori and Pacific ancestry. Two thirds were female (67 %), the mean age was 19.2 years (SD 4.2 %) and 70 % or more came from a secondary school with a medium or low school decile (representing more deprived communities). The First Year Bachelor cohort had a total of 242 students with a similar demographic profile to First Year Tertiary students (Table 2).

Predictor variables Mathematics and english testing
The First Year Tertiary cohort had a mean percentage mark for the mathematics test of 79.0 % (SD 18.3 %) and 68.4 % (SD 13.6 %) for the English test. This represents a borderline-fail result for bachelor-level study and a pass result for CertHSc-level study as the best starting point of entry across both assessments. The First Year Bachelor cohort had a slightly higher mean mark for both the mathematics (80.4 %, SD 18.3 %) and English tests (70.6 %, SD 12.8 %) (Table 3). L3 subject missing data includes those NCEA applicants who did not enrol in that particular subject *Adjusted for MAPAS interview year, gender, ancestry and school decile. For GPA (a continuous outcome variable), its mean change associated with the change in alinear predictor was estimated with 95 % confidence interval. For a continuous predictor variable, this gave the difference in means with either 20 point (NCEA Rank Score) or 10 % (MAPAS Maths percentage mark) increase in the predictor. For a categorical predictor, this gave the difference in means between the current and reference categories (i.e. yes vs. no). The null hypothesis was that there was no change in the mean response (i.e. Δ = 0) **NCEA = National Certificate in Educational Achievement, CIE = Cambridge International Exam, IB = International Baccalaureate a FC Few concerns, SMC Some or major concerns  (Table 3).

School results
The average NCEA rank score (out of a total of 320) was 190.5 (SD 51.3) for First Year Tertiary and 201.8 (SD 52.7) for First Year Bachelor students. Both averages fall below requirements for guaranteed entry within FMHS (set at a rank score between 210 -250 depending on the programme). The average number of subject credits for both cohorts were 0.3-3.4 credits below requirements for guaranteed entry (i.e. 16 -18 subject credits depending on programme) ( Table 3). At least two thirds of all students admitted into either the CertHSc or bachelor programmes had taken two or more science subjects in their final year of secondary school (Table 3).

MAPAS recommendations
For First Year Tertiary students, MAPAS recommended CertHSc to 72 % of all students, followed by Bachelor (26 %) and Not FMHS (2 %). For First Year Bachelor students, 58 % were recommended to start at the CertHSc level, followed by 39 % Bachelor and 3 % Not FMHS (Table 3).

Followed MAPAS advice
Over 83 % of all students followed MAPAS advice regarding the best starting point for success with only 12 -17 % of students from each cohort not following their final MAPAS recommendation (Table 3).

Outcome variables GPA All eight courses and core 4 courses
The average GPA for all eight courses (out of a total of 9) was 4.  (Table 2).

Multiple regression analysis
First year tertiary -GPA  L3 subject missing data includes those NCEA applicants who did not enrol in that particular subject a FC Few concerns *Adjusted for MAPAS interview year, gender, ancestry and school decile. For GPA (a continuous outcome variable), its mean change associated with the change in alinear predictor was estimated with 95 % confidence interval. For a continuous predictor variable, this gave the difference in means with either 20 point (NCEA Rank Score) or 10 % (MAPAS Maths percentage mark) increase in the predictor. For a categorical predictor, this gave the difference in means between the current and reference categories (i.e. yes vs. no). The null hypothesis was that there was no change in the mean response (i.e. Δ = 0) **NCEA = National Certificate in Educational Achievement, CIE = Cambridge International Exam, IB = International Baccalaureate

First year tertiary -passes All courses
The odds of passing all eight courses was 5.4 times higher for those students who followed MAPAS advice versus those students who did not (CI: 2.36-12.39; p < 0.0001) ( Table 8). The odds of passing all eight courses was 2.3 times higher for those students who had exposure to Any 2 Sciences versus those students who did not (CI: 1.15-4.61; p = 0.019) ( Table 8).

First year bachelor -GPA
For every 20 point increase in NCEA Rank Score, the GPA achieved by First Year Bachelor students increased by an average of 0.4 for all 8 courses (CI: 0.30-0.50; p < 0.0001) and for Core 4 courses (CI: 0.26-0.50; p < 0.0001) ( Table 7). Students who followed MAPAS advice had on average a GPA that was 1.1 points higher than students who did not follow MAPAS advice for all eight courses (CI: 0.45-1.73; p = 0.0009) and Core 4 courses (CI: 0.60-2.04; p = 0.0004) ( Table 8). times higher for those students who followed MAPAS advice versus those students who did not (Table 9).

Discussion
Our findings confirm that the MAPAS admissions process is strongly associated with positive academic outcomes in the first year of tertiary study. Our results reinforce the evidencebase showing a strong association between secondary school performance via NCEA rank score (a marker of the quality of grades achieved) and positive tertiary academic outcomes [35]. The existing literature base has also been extended, given our identification of a strong association between exposure to two or more senior science subjects (a marker of breadth of knowledge) and first year academic outcomes. Similar to other studies, our findings show that the number of credits achieved within NCEA subjects appear to be less strongly correlated with tertiary outcomes [35]. Overall, our findings suggest that there is value in providing a comprehensive admissions process for indigenous and ethnic minority students applying under equity targeted admission programmes. Students admitted into tertiary institutions under targeted admission programmes have been shown to experience peer/educator stigma and 'everyday racism'. Demonstrating the effectiveness of targeted admission programmes may assist some indigenous and ethnic minority students to override this societal (and potentially internalised) stigma to receive the benefits that targeted admission programmes have to offer.
Increasing the odds of passing all first year courses has relevance for all students. This is important for applicants pursuing medicine as even small increments in first year bachelor GPA, particularly within the Core 4 courses used for medical selection, may have a profound impact on potential selection [12,19]. A student's progress towards completion of total point requirements within their degree has been shown to improve student retention and increase the likelihood of degree completion [36]. Aligning MAPAS admission to a comprehensive process focussed on achieving equity in access and performance is likely to have  contributed to the recent increase in numbers and improved performance observed for Māori and Pacific students within the FMHS [5,37]. Our data suggests secondary schooling is yet to demonstrate the ability to prepare Māori and Pacific students adequately for tertiary health professional study. Both teaching and subject selection are critical factors. Māori and Pacific students and their families are not to blame for the observed inequities in secondary education. Rather, Māori and Pacific students and their families  should receive greater support to navigate NCEA subject selection and ensure that students achieve the right number and quality of credits [38]. This is consistent with international evidence showing that indigenous and ethnic minority students are less likely to receive highquality careers or university advice [38,39] and in some instances may be actively discouraged from pursuing a health professional career [2]. Based on our findings, it appears that the secondary education sector is failing to ensure that indigenous and ethnic minority students are 'university-ready' for health-professional study. Unfortunately, this is not a new issue [5,14,40,41] and nor is it unique to New Zealand [3,42]. Action by secondary schools and educators to address their own role in the creation and maintenance of ethnic inequities in academic outcomes is recommended [43]. Likewise, tertiary institutions are expected to be part of the solution [44]. Pechenkina & Anderson (2011) call for "more effective institutional response to the lack of adequate preparation of indigenous students… via greater investment in the pipeline and provision of transitioning programmes" (p. [5][6]. Our findings further support the delivery of bridging/foundation programmes targeting indigenous and ethnic minority students.

Strengths
This study explores a unique application of the MMI within an equity-targeted context [14,26]. Although we identified varied associations between individual MMI stations and academic outcomes, we believe that our overall findings support maintaining the MMI within the MAPAS admissions process. This reflects the strong association observed between following MAPAS advice (a predictor variable that is determined by the combined assessment of all results) and higher academic outcomes.
Using both cognitive (e.g. NCEA school results, MAPAS Maths and English test) and non-cognitive (e.g. MMI results) tools for student selection within the total MAPAS admission process supports a widening participation agenda and is consistent with recommendations to use more inclusive selection tools [10,[45][46][47]. This is particularly important when assessing the potential of alternative admission or older applicants who may possess maturity shown to be positively associated with tertiary programme completion [36,48].

Limitations
This study has a number of limitations. The analysis relied on secondary data and is therefore limited by the quality of data sources. However, combining central university and MAPAS datasets has reduced the potential for data misclassification by using verified ancestry and increased the admission variables available for analysis [49,50]. Our research was limited to first-year outcomes due to resource and time constraints. Ideally, the effect of predictor variables on long-term outcomes across all FMHS programmes should be examined. Comparing academic outcomes across all ethnic groups may also highlight issues of disadvantage and privilege [51]. This research is in progress and is drawing on the methods developed within this study. We acknowledge that combining Māori and Pacific data is not ideal from an indigenous rights or Pacific-centric perspective. However, this is consistent with our methodological approach as it maximises statistical power (to aid student success) and supports a structural critique of the effect of 'society' on 'ancestry' [14]. As the quantum of Māori and Pacific data increases, further research should investigate Māori-specific and Pacific-specific predictors of academic success.

Conclusion
Tertiary institutions committed to widening participation should prioritise the funding and delivery of a comprehensive, flexible and inclusive admissions process that includes alternative entry pathways for indigenous and ethnic minority applicants [10,52,53].  (7). 10.1186/s12939-015-0133-7. or https://www.fmhs.auckland.ac.nz/en/faculty/for/futureundergraduates/maori-and-pacific-admission-scheme.html 3 The National Certificate of Educational Achievement (NCEA) is the major assessment method used in New Zealand secondary schools. The NCEA Rank Score reflects the best 80 credits at Level 3 or higher, over a maximum of five approved subjects. It reflects a system of Grade Point Average and is used by the UoA to assist with admission to limited entry programmes 23. Shulruf B, Hattie J, Tumen S. New Zealand's standard-based assessment for secondary schools (NCEA): implications for policy makers. Asia Pacific Journal of Education. 2010;30 (2). 4 Exposure to a minimum of two final year secondary school science subjects is recommended for success within the CertHSc (alongside English and mathematics rich subjects). This variable includes secondary school results from NCEA, International Baccalaureate (IB) and Cambridge International Examinations (CIE). 5 The Core 4 courses include: CHEM110 (Chemistry of the living world), POPLHLTH 111 (Population Health), MEDSCI 142 (Biology for Biomedicine Science: Organ Systems) and BIOSCI 107 (Biology for Biomedicine Science: Cellular Processes and Development).