Southern States Enterprise University (SSEU), located in Birmingham, Alabama has an overall graduation rate of 23 percent, retaining only 6 out of 26 Master of Business Administration (MBA) students in the last year (National Center for Education Statistics [NCES], 2014). Ranked at number 17, the university is one of the top 25 institutions with the worst graduation rates (O’Shaughnessy, 2011). SSEU has implemented the traditional method of recruiting MBA candidates wherein prospective students are accepted based on their “Graduate Mathematical Admissions Test” (GMAT) scores and undergraduate GPAs.
Over 1,000 institutions nationwide use GMAT scores and undergraduate GPAs to recruit their students but only a fraction of these students persist to graduation (Deis & Kheirandish, 2010; Fish & Wilson, 2007; Loucopoulos et al, 2007). The purpose of this study is to determine if granting GMAT waivers for MBA candidates with substantial leadership background is more effective in predicting student persistence and graduation than the traditional method of requiring GMAT scores and undergraduate GPAs.
Levels of Measurement
Measurement is an organized and replicable process wherein events or objects are calculated and classified according to their attributes (Creswell, 2009; Weiner, 2007). Assigned with numerical values, these attributes fall into one of four classifications: a) nominal, b) ordinal, c), interval, and d) ratio (Weiner, 2007; Frankfort-Nachmias & Nachmias, 2008). Reliability is the degree to which the measurement method will produce consistent results after repeated applications (Creswell, 2009; Weiner, 2007; Frankfort-Nachmias & Nachmias, 2008).
Likewise, validity is the degree to which the method or the instrumentation is effective in “describing or quantifying what is designed to measure” (Weiner, 2007, p. 7). Variations in replicated studies “may be due to a mere chance, inconsistency, or a change in the “underlying event” (Weiner, 2007, p. 8). It is crucial for the researcher to utilize instrumentations that produce reliable and valid results to ensure that future replicated studies with other populations under similar conditions will generate similar results (Creswell, 2009; Weiner, 2007).
Based on the decreasing retention and graduation rates at SSEU, the institution’s senior executives and administrative leaders are considering taking a different approach in predicting the success of their MBA candidates. The study will examine the persistence and graduation rates of cohort A, wherein in lieu of their GMAT scores and undergraduate GPAs, candidates with at least 5 years of leadership experience will be admitted with GMAT waivers. Likewise, the study will also examine the persistence and graduation rates of cohort B, wherein GMAT scores and undergraduate GPAs are the determining factors when admitting MBA candidates.
Nominal scales. Relevant to any study on MBA students’ performance and completion are the participants’ gender. Female MBA students have lower GMAT scores than their male counterparts but there is a significant increase in their overall school performance (Hancock, 1999). In the proposed study, the researcher will distribute MBA questionnaires (Figure 1 in Appendices) to collect demographic data, represented by nominal scales. The nominal scales will record the students’ gender: 01= male and 02=female.
Ordinal measures. The “Leadership Skills Inventory”(LSI), (Table 1 in the Appendices section) will generate responses with ordinal measures: a) not true, b) seldom true, c) occasionally true, d) somewhat true, and e) very true (Parker, Flin, McKinley, & Yule, 2013; Sage Publications, 2010). The objective of the LSI is to determine leadership behavior (Parker, Flin, McKinley, & Yule, 2013). Likewise, the study will utilize the “Study Skills and Habits Questionnaire” or SSHQ (Table 2 in the Appendices section) to interpret academic skills and behaviors, generating responses with ordinal measures: a) rarely or never like me, b) not often like me, c) sometimes like me, d) fairly like me, and e) a lot or always like me (Queen’s University, n.d.).
Interval measures. The MBA questionnaire will utilize an interval scale to record the participants’ leadership experience. The number of years of experience in leadership will be organized into 5-year increments: 001=1 to 5 years of leadership experience, 002=6 to 10 years of leadership experience, 003=10-15 years of leadership experience, 004=15 to 20 years of leadership experience, and 005=20 or more years of leadership experience. Likewise, the participants’ ages will be recorded with interval measures: 01=ages 23 to 30; 02= ages 31-40; 03= ages 41-50, and 04= ages 51 and beyond.
Ratios. The GMAT scores and undergraduate GPAs for participants in cohort B will be recorded as ratios:
Table 3. GMAT Scores and Percentiles
GMAT Integrated Reasoning Score
≥ 1 ≤ 4.34 0% to 37.5%
≥ 4.35 ≤ 6.34 37.6% to 67.5%
≥ 6.34 ≤ 7.75 67.6% to 89%
≥ 7.76≤ 9 89.1% to 100%
GMAT Verbal Score
≥ 1 ≤ 27.5 0% to 45.5%
≥ 27.6 ≤ 37.5 45.6% to 83.5%
≥ 37.6 ≤ 40 83.6% to 90%
≥ 41≤ 55 91% to 100%
≥ 1 ≤ 22 0% to 9%
≥ 22.1 ≤ 37.5 10% to 42%
≥ 37.6 ≤ 45 43% to 70%
≥ 46≤ 55 72% to 89%
≥ 56≤ 65 90% to 100%
Note: GMAT exam scores from GMAT Handbook provided by
Graduate Management Admission Council (GMAC)
MBA questionnaire. The researcher will distribute MBA questionnaires to members of cohort A, which will identify their gender, ages, and the number of years they served as leaders. Furthermore, the survey will consist of questions from the LSI and SSHQ. Most importantly, the questionnaire will be instrumental in determining the number of years MBA candidates have had in leading or managing groups of individuals.
Ensuring Content, Empirical, and Construct Validity
Validity is the primary concern in all measurement programs, evaluating how “empirical evidences are aligned with scientific theories, which can then support interpretations, inferences, and actions” (Young, 2007, p. 3).
Content validity. Content validity examines the development of a test (Thorn & Deitz, 1989). In this study, the researcher will examine the LSI and SSHQ instrumentations. Content validity ensures that “no phenomenon in the investigation is left out” (Frankfort-Nachmias & Nachmias, 2008, p. 150). To ensure content validity, all measurements in the current study were, at one point, used by other researchers, to determine student success. This validity, specifically, sampling validity, ensures that previous researchers have already administered these instrumentations with a group of college candidates to predict their success (Parker, Flin, McKinley, & Yule, 2013; Sage Publications, 2010). Furthermore, several institutions including Queen’s University utilize LSI when admitting MBA candidates (Queen’s University, n.d.).
Empirical validity. Empirical validity examines “the relationship between the instrument and the measured outcomes” (Frankfort-Nachmias & Nachmias, 2008, p. 150). With its integration of the LSI and SSHQ surveys, the MBA questionnaire will generate scores that represent the candidates’ years of experience as leaders. This component is vital when comparing the persistence and graduation rates of candidates with GMAT waiver due to substantial amount of leadership experience against the persistence and graduation rates of candidates who are accepted in the university based on their GMAT scores and undergraduate GPAs. The LSI and the SSHQ measurement tools are the most appropriate tools to evaluate the candidates’ knowledge and experience associated with business administration (Parker, Flin, McKinley, & Yule, 2013; Queen’s University, n.d.; Sage Publications, 2010).
Construct validity. Construct validity examines the relationship between the measurement tool and the underpinnings of theoretical frameworks (Dijksterhuis, Jozwiak, Braat, & Scheele, 2013). The current study will utilize Gardner’s Multiple Intelligence (MI) theory. Instrumentations used for predicting student success consistently yield large amounts of data about the students’ cognitive and non-cognitive traits (Lopez & Patron, 2012). Likewise, Gardner’s theory covers seven intelligences associated with cognitive and non-cognitive abilities. In the current study, Gardner’s theory can help the researcher examine the “preferred intelligence” of the candidates, which coincides with the measurement tools used to examine the candidates’ leadership and academic skills (Al-Balhan, 2006).
Unapplied validity and rationale. Since the researcher’s subjective evaluation is not included in the study, face validity will have no application in the current study (Frankfort-Nachmias & Nachmias, 2008; Dijksterhuis, Jozwiak, Braat, & Scheele, 2013). In addition, the study will not utilize any pre-test or post-test outcomes, in which facial validity would be most appropriate (Frankfort-Nachmias & Nachmias, 2008).
Ensuring reliability. In any study, inadequate measurement tools result to serious consequences (Babbar, 1995). To predict student success, “a degree to which a measurement meets the test of universality and other generally accepted standards in higher education, depends on the measurement instruments and the process of attaining measurement outcomes” (Babbar, 1995, p. 72). Since the study will adopt instrumentations that are widely used in higher education institutions, it will not have new attributes or contexts, ensuring reliability (Frankfort-Nachmias & Nachmias, 2008).
Measurement Instruments: Strengths and Limitations
Sampling validity is present in all instrumentations that will be implemented in this study. For cohort B, the strength of the GMAT scores and undergraduate GPAs is comprehensive in nature, covering mathematical, written, and critical thinking skills of MBA candidates. However, adult learners, who have not been in university classrooms for several years and in extreme cases for over 10 years, may not have the interest and the ability to supply these components. These components may even serve as a deterrent that would prevent them from taking interest in pursuing their graduate degrees. The researcher hopes that the number of years of substantial experience in leading and managing individuals will be a stronger predictor for retention and graduation than the traditional GMAT and undergraduate GPAs.
Rationale behind the proposed scale. The MBA questionnaire, which collects data on the participants’ demographics and number of leadership years and integrates both LSI and SSHQ, will serve as the most comprehensive admissions instrument to predict student success than the traditional methods because it measures the candidates’ cognitive and non-cognitive skills. Furthermore, it can be administered to new graduate students and to adult learners who have not attended post-secondary institutions for several years.
Justification of the scale, its reliability, and validity. Test-retest reliability can be applied to the survey (Wu, 2012, p. 10). A split-half reliability can also be applied, “that is dividing the survey into two halves to which two sets of answers will have similar responses” (Wu, 2012, p. 10). Furthermore, the researcher has the confidence that the MBA questionnaire for cohort A, when examined by two different observers under similar conditions, will still generate similar responses since the characteristics and attributes of MBA candidates are similar. Finally, the MBA questionnaire can be administered to all MBA candidates, who have or have not taken the GMAT or who have not attended post-secondary institutions for several years.
Test: Norm or criterion referenced. The MBA questionnaire is a type of criterion-referenced test (CRT) in which MBA candidates can demonstrate their knowledge and skills in the field (The National Center for Fair and Open Testing [Fairtest]. The survey will examine both cognitive and non-cognitive skills. Furthermore, the survey can help recruiters determine if a candidate has “achieved specific skills or concepts” (Popham, 1975 as cited in Huitt, 1996).
Population for the scale and test. The MBA questionnaire with the LSI and the SSHQ contents will be administered to all MBA candidates applying at SSEU for the Fall semester of 2014. Currently, there are 30 provisional MBA candidates, who have applied at the university by filling out the online application available at the institution’s website. Of these 30 candidates, 15 candidates with more than 5 years of leadership experience will be given the GMAT waivers while the other 10 candidates will be admitted to the institution based on their GMAT scores and undergraduate GPAs.