Harvard Medical School has offered a voluntary diversity-training program every year since 2000. Immediately after each session, organizers asked how much volunteers liked learning about the importance of diversity for organizations. People said they did, but organizers did not investigate whether participants actually changed their behaviors, attitudes, or performance.

Increasingly, organizations are finding that "consciousness raising" alone isn't enough. In 2003, research by Thomas Kochan at Massachusetts Institute of Technology showed that having diverse teams on board could indeed improve a company's performance and boost innovation - if diversity is managed effectively. (See "Diversity, The New Business Case" on p. 42.) Others have since confirmed the findings in a variety of settings. As a result, companies are increasingly testing strategies for measuring whether their diversity programs are helping employees work together more effectively and even contributing to the bottom line, and fixing them if they're not.

"One of the top-priority questions in diversity over the last five years is over measuring results and being able to talk about what gains are made, because a lot of people just slapped on diversity programs that didn't work," says Amri Johnson, executive committee member for the National Association for Blacks in Bio, and also executive vice president at Cook Ross, a healthcare cultural competency consulting firm in Silver Spring, Md.

"Managing diversity well is key to making it work for you," notes Verna Ford, an executive consultant with Novations Group, a consulting and training company in Boston.

Measuring the success of diversity initiatives must begin by setting goals for these initiatives - for example, reducing turnover. Moreover, keeping track of employee recruitment, retention, and promotion is essential, says Daphne Mobley, Wyeth vice president of diversity. In academia, tenure is a key metric, according to Beth McGee, faculty diversity officer at Case Western Reserve University.

One effective strategy for identifying problems is to analyze employee satisfaction surveys demographically, for example by gender, ethnicity, or tenure at the company, says Deborah Dagit, executive director of diversity and work environment at Merck in Whitehouse Station, NJ. "For companies who are really serious about diversity, they look at a lot of slices and dices of their data," she explains. "Have we got a population ... that is feeling disenfranchised? Are they feeling disengaged, are they less likely to recommend this as a great place to work?" The company can then conduct focus groups to identify root causes and do something about them, she adds.

"You want to get rid of disparities in perceptions between how different groups of people perceive the organization, where nobody is less happy," says Anita Rowe, a partner in Gardenswartz & Rowe, a diversity consulting firm in Los Angeles.

In academia, surveys can ask employees if their lab space distributions are fair and if start-up packages are equitable for their departments. Surveys can also ask students and young faculty if they are getting appropriate mentoring and if they need more help with grant-writing or in other areas, McGee says.

Experts agree that leadership's commitment to a diversity initiative is the most important metric for success. "The diversity manager should never be the one solely responsible for diversity. The entire leadership team is supposed to be," Ford says. "You have to get backing from the top, [from people] who are willing to hold everyone accountable for implementing diversity goals and plans," Rowe adds.

AstraZeneca requires all its managers to specify diversity objectives, and achievement of these objectives is reflected in their performance evaluations, notes Orlando Ceaser, the company's senior director of diversity. "To help managers meet these objectives, we provide specific training on ways to more effectively promote and protect diversity among their teams," he explains.

Managers can also be evaluated on whether their efforts yield lower costs and more effective, better-quality results. "It's proper to provide incentives for them as well," Ford says, noting that many companies offer bonuses related to how well employees contribute to the company's overall diversity efforts.

The impact of diversity could also be measured in the customers a company draws or the students a university enrolls, Ceaser says.

"Are you missing opportunities for markets, hitting all the demographics with your marketing strategies, retaining customers? What are your sales like around the country, and what are they like in regions that are more diverse? Are you culturally sensitive when it comes to recruiting patients for trials?" Johnson asks.

Most companies do not divulge data on employee diversity, nor about what they measure. Two out of five companies see this information as proprietary and keep it thoroughly confidential. Slightly more than one-third releases the information, but only to concerned groups, such as employees, vendors, or customers. Only about one-fourth makes the information public, Novations found.

When it comes to academia, programs that measure and demonstrate the success of diversity initiatives are roughly 10 years behind those in industry, says Harry Reed, head of the office of workforce diversity at Lawrence Berkeley National Laboratory and laboratory ombudsman.

"In the private sector, employment lies under the discretion of managers, so employment's much more flexible," Reed says. In academia, "the decision lives at the department level as to who gets hired and who gets tenure. Are people hired because of their accomplishments or for other reasons? What they're after and why is not as transparent, making it hard to systematically change barriers to diversity embedded in the culture."

To make the academic hiring process more transparent, diversity specialists can work with search committees to assess an institution's advertisements and to help draw up templates of questions that search committees can ask, McGee says.

Just as industry offers incentives to managers for supporting diversity, schools can work along similar lines, McGee adds, by providing incentives to deans to diversify schools or curricula. Such incentives might include an increased budget from the central administration, or improvements to labs or libraries.

Measuring the success of a school's diversity initiatives from students' perspectives requires a sustained effort that starts early. "We have to begin surveying students at the beginning of their first year, then in midyears, then near the end of a student's time at a school, and then even after they graduate, regarding perceptions of their environment, of what they learned educationally and culturally, how they grew socially and in their intellectual abilities to think and problem solve, and [then] put diversity in that context," says Damon Williams, assistant vice provost at the University of Connecticut. "Measuring core academic outcomes such as retention rates, graduation rates, academic performance" is important, adds Williams.

A perpetually tricky issue when assessing the success of diversity programs lies with all the confounding factors involved, Rowe notes. "There are many other variables that can impact results you measure, with the changing economy and changing population, [and] fluctuations of all kinds in the job market. Perhaps staff is retained not because of diversity efforts but because the economy is lousy," she adds.

To properly assess the success of a diversity initiative, Rowe continues, "you need to be in it for the long haul. It's not just fixing one problem or holding a training seminar. It takes time to see where people are getting excluded or not used to their best, or even what you should be measuring."

This year, Harvard expanded its diversity-training program from four to 16 hours to make it more comprehensive. Organizers distributed follow-up evaluations three months later. They learned that leaders and managers want more training on how to create a welcoming workplace environment and to help staff communicate and collaborate more effectively.

"By better measuring how successful our program is, we can improve it further, and ... encourage others to come," says Robert Amelio, director of Harvard Medical School's center for workplace learning and performance. "Now we can work better together."