“Measuring Diversity”: New Guide from NSF, AAAS, Helps Universities Evaluate Their Efforts

With many U.S. universities working to recruit and support minority graduate students, the AAAS, in partnership with the National Science Foundation, has issued a new guide offering detailed advice and practical tools to help administrators evaluate the effectiveness of their efforts.

“Measuring Diversity: An Evaluation Guide for STEM Graduate Program Leaders” has a simple premise: A high rate of students from underrepresented minority groups leave graduate-level studies before completing advanced degrees, but especially at a time of rising global competition, that represents a significant and costly loss of talent and potential.

“

title="This link will open in a new window"

target="_blank">Measuring Diversity

,” issued by AAAS and based on work with the National Science Foundation Alliance for Graduate Education and the Professoriate (AGEP), is available in paperback or online. The 83-page guide offers a detailed framework for evaluating graduate level programs in science, technology, engineering, and mathematics, along with practical tools that focus on key areas of evaluation

“We live in a time when there’s a premium on innovation. We live in a time of rising global competition,” says Shirley Malcom, the head of Education and Human Resources at AAAS and a co-editor of the guide. “We need to cultivate and support our homegrown talent. That means we need a way to take the pulse of our graduate education programs in science, engineering, and related fields to make the programs more efficient and more effective.”

“This publication emphasizes the importance of not only collecting the right data, but also using it,” said Jessie DeAro, the AGEP program director. “The evaluation of graduate education programs is a critical component for achieving our overall goal of a U.S. STEM workforce that reflects the diversity of the U.S. population. Evaluation can provide answers to important questions such as what is and is not working for different graduate student populations in order to make improvements to graduate education programs.”

“Measuring Diversity” doesn’t tell universities how to keep under-represented minorities in school. Rather, it offers a framework and tools for assessing the strengths and weaknesses of graduate programs. By adapting ideas from the guide, universities also would be able to generate the evaluation data frequently sought by funding organizations, policymakers, and accreditation groups.

Initiated six years ago, the guide features contributions from 25 authors representing universities that have received AGEP grants from NSF. It is intended as a resource for university administrators—including vice presidents, provosts, and deans—and faculty members. It focuses on how to assess the graduate school experience, especially at the Ph.D level, among African Americans, Hispanic Americans, American Indians, native Alaskans, and native Hawaiians and other Pacific Islanders.

The guide was edited by Yolanda George, deputy director of AAAS Education and Human Resources; Patricia Campbell, president of

title="This link will open in a new window"

target="_blank">Campbell-Kibler Associates Inc.

; and Malcom. The work was funded by NSF.

Statistics show that graduate students from under-represented minorities encounter greater challenges than their white and Asian colleagues.

According to

title="This PDF will open in a new window"

target="_blank">NSF data

, for example, the median time to complete a Ph.D after entering a program is 7.7 years for white and Asian graduate students. For Hispanic students, the median is 8.3 years, and for both African American and native Alaskan/American Indians, the median time to completion is 9.9 years.

This is a critical disparity, Malcom said. “This extended time to receive a degree is costly, and it comes when budgets are tightening. In addition, the longer the time it takes to receive a degree, the greater the opportunity cost to the student. For example, as time passes, there’s an increased likelihood of dropping out. There’s greater accumulation of debt. If you finish your degree and go to work sooner, some of these costs can be avoided.”

Based on studies from the Council of Graduate Schools, current estimates of Ph.D completion range from less than 40% for African Americans in the physical sciences to about 70% for international students, Malcom and AAAS Chief Executive Office Alan I. Leshner say in a letter introducing the guide.

Those who don’t complete the graduate studies “represent a tremendous loss of talent, especially considering the profile of entering students and the sizable investment being made in their education,” Leshner and Malcom write. “The tremendous challenges that we face now and into the future as a country and a planet can only be addressed by preparing the best minds to attack the biggest problems.”

The authors of “Measuring Diversity” emphasize a related point: “The under-representation of these racial/ethnic groups has serious implications for the nation’s ability to compete in a global economy driven by innovations in science and technology.”

A critical problem, Malcom suggests, is that graduate programs often don’t provide sufficient support for students from various cultures. Women and minority grad students are like canaries in a coal mine, she says—when they struggle, it suggests deeper system problems that are likely to affect all students.

“In undergraduate education, you hear a lot of concern about the retention of minority students,” she said. “But in graduate education, if a student leaves, it’s not seen as an issue of retention. There’s an assumption that it’s a question of quality—the student wasn’t good enough. But that’s not necessarily the case.

“High-quality graduate programs must work to become more efficient and even more effective.”

The first chapter of “Measuring Diversity” sets the context and framework for evaluating science-related diversity programs; the second chapter offers an overview of indicators, asserting the need for both quantitative and qualitative data. Subsequent chapters zoom in on suggested ways to evaluate recruitment strategies and admissions; retention and Ph.D completion; faculty mentoring; preparation for the professoriate; and other issues.

In addition, the guide and its related Web site offer four evaluation tools:

• An interactive tool to help institutions look at the progress they’ve made—and need to make—toward diversity goals.

• An online tool focused on “enrollment yield”—the proportion of applicants admitted who actually enroll in a graduate program—which can help administrators assess their recruitment and admissions practices.

• A tool for calculating an approximate rate of student retention, a “quick-and-dirty” way of evaluating trends without the high cost of computing exact retention rates.

• A tool for helping institutions to monitor the progress of individual students toward their Ph.D.

“Especially in these economic times, graduate programs need to move to more data‑based decision making,” said Campbell. “While this is true in general, it is of particular importance for efforts to increase diversity. These free tools are designed to help administrators and faculty use data to make better decisions for all students, with a particular emphasis on under‑represented ones.”

Campbell and Malcom are scheduled to make a “Measuring Diversity” presentation on 7 June at the

title="This link will open in a new window"

target="_blank">joint annual meeting

being held by the Division of Human Resource Development within the NSF’s Directorate for Education and Human Resources.

Edward W. Lempinen

25 May 2011

News Source : “Measuring Diversity”: New Guide from NSF, AAAS, Helps Universities Evaluate Their Efforts