Managers on a Mission: Med schools begin to quantify research, teaching, and patient care

Graphic: Leza Berardone The hallowed halls of medical academia may shudder at the thought of their denizens' life's work in research, teaching, and clinical care being reduced to numbers. But several administrators at the 125 medical schools in the United States feel they can implement a management system that measures outcomes in research, teaching effort, and clinical-work revenue/cost--and still preserve academic excellence. In fact, the system could bring, they say, some sorely needed accountability to their institutions. Fiscal management has not been the strong suit of medical schools for many years, administrators say, and the bite managed care took out of a once-comfortable profit margin has exacerbated this situation. "If tuition and state money are not sufficient for the educational mission, you have choices. You can raise tuition. You can go to the state. Or you say we're making profit on clinical [activities]--let's subsidize education. That's what happened in the past. But the margin on that clinical money is going away, because of managed care. The source of cross-subsidy is gone," says Robert T. Watson, senior associate dean for educational affairs at the University of Florida Medical School at Gainesville. That school, along with the University of Maryland School of Medicine and Loyola University of Chicago's Stritch School of Medicine, has been reporting the value or productivity of its research, teaching, and patient care to faculty members and basing its reports on metrics. The three have also been using those numbers to make management decisions and to determine faculty compensation at the department level through a system known as mission-based management. Robert Jones, executive director of the American Association of Medical Colleges, describes mission-based management as two new reporting systems: "Productivity is one of the two. The other reporting system is an integrated financial statement that breaks out the cost of the various missions [research, teaching, and patient care]. [It gives] statement revenue and expenses on a mission." The reports, according to Jones, are not detailed. They vary depending on the overall mission of the school. But they ultimately determine if each mission [research, teaching, or patient care] is self-supporting or not. The reports can be used to determine the productivity of an individual faculty member, or an entire department. But, Jones explains, they are not necessarily used to measure the value of a specific research project. "Our focus is to look at several different measures and get an aggregated metric, but that's only the half of it," says Jones, whose organization, together with the Healthcare Group, a division of Computer Sciences Corp., based in El Segundo, Calif., is offering consulting services to medical schools interested in mission-based management. The two worked with the University of Florida, University of Maryland, and Loyola University. In February, they announced that five medical schools would be entering mission-based management programs. They are Creighton University, East Carolina University, the University of California, San Diego, the University of Texas, Galveston, and West Virginia University. The ultimate goal of mission-based management, according to Jones, is "to change the culture of the school so that [it is] using that data. We're talking about transforming the ways schools are managed. We have to preserve what's good about [an academic] culture. But we can't get away from the fact that our institutions are involved in several different businesses. Research is a business. Educating graduate students is a business." Darrell Kirch Darrell Kirch, dean of the Medical College of Georgia in Augusta uses the term "mission-based budgeting." Kirch, whose institution is just in the planning stages of this process, explains that "it addresses the problem of these complex intertwined funding streams that have become nearly impossible to unravel. There's little alignment between clinical, research, and educational [revenue sources] that come together to make up the [faculty member's] salary. It's an attempt to make sure the funding streams stay aligned with the mission they intend to support." Florida's Watson explains that the idea is to use the measures or metrics to establish a budget for each mission. "Then you link productivity back to the budget and back to the mission." He also says that as a result of mission-based management, his institution went from seeing the loss of $1 million a month to seeing a positive variance. The University of Pennsylvania's Alan L. Hillman, who says his school is at least five years from implementing either the measures or the entire management tool, said that before managed care, doctors believed in the ethics of fee-for-service medicine. "Some thought it went hand in hand with the Hippocratic oath. ... But once they made that leap, they widely adapted to it. Pandora's box was opened. Compensation was tied to all kinds of behavior," said Hillman, also a professor of medicine and health care policy at Penn's School of Medicine and the Wharton School in Philadelphia. At the medical schools where some component of this management system is either being anticipated or used, the main concern is the same: data integrity. It's highly important that the measures, which vary depending on the medical school, are as accurate as possible and that they are used in context--as a management aide among other factors for the department chair to determine how productive a faculty member is in each of the three missions. "Each of the measurements used individually is imperfect, but together they can give a good picture," explains Jones. Patient care, for some time now, has been measured by assigning medical procedures "relative value units" (RVUs), based how much effort is exerted during the procedure, including time, technical skill, mental effort and judgment, physician stress, and total work. The RVUs also include how much is spent in rent, support staff and supplies, and malpractice. Research, which has just recently begun to be quantified in a similar way, is evaluated by outcome, whereas teaching is measured by how the individual faculty member spends time. The measurement may include time spent on labs, lectures, and mentoring. In research, factors used to compose metrics could involve the number of grants a faculty member gets, the frequency of grants from federal agencies such as the National Institutes of Health, grant dollars, number and prestige of publications, and the number of citations those publications get, according to Jones. Another metric, known as dollar density per square foot, includes figures such as square feet of lab space used. Those numbers are hard to pinpoint, according to Arthur Garson at Baylor College of Medicine in Houston. Lab space "is something we have to continually look at. If we are using a research metric, we have to make sure it is absolutely correct," notes Garson, who explained that although Baylor has been measuring performance for about four years, the institution is still two to three years away from tying pay to performance. Although the distribution of the reports to the faculty members have coincided with a 12 percent decrease in "unfunded" research, he emphasizes that there is no evidence of a causal relationship. Garson adds that when Baylor does implement its plan, variations in pay will be distributed directly to the department chair, who will decide--based on objective criteria--how the money is distributed. The form of mission-based management at the University of Florida, and eventually at Baylor, gives the department chair a lot of power, but less leeway, according to Watson. The data is given to the chair, who may interpret it based on his knowledge of what the individual faculty member is doing. "Information doesn't take over the responsibility of the department chair to manage. It just provides another context. The chair and faculty member's annual conversation sometimes takes place without the benefit of information. It's difficult to manage. This gives information. The measures may look poor if the outcomes are long term. The chair doesn't have to take that data slavishly," says Jones. Women scientist are more vulnerable than men under proposed management changes, says Catherine J. Didion of the Association for Women in Science. Also acknowledging the fallibility of the data, Garson comments, "We want to make sure the metrics are modifiable so that the chair could adapt them to the individual performance." The chair could be flexible if the faculty member has not published in a while but is working on a project that has promise. Some have argued if the data is left to interpretation, faculty members who don't enjoy a good relationship with the chair could lose out. "Women don't have the publication rate of men, but the citation rate shows that they are [equally creative]. A lot is left up to interpretation here. Women are more vulnerable when this is the case because they aren't part of the 'informal socializing' that men are," says Catherine J. Didion, executive director of the Association for Women in Science, Washington, D.C. Jones acknowledged that any information in the hands of certain people could be dangerous. But, he points out, in order for mission-based management to work, institutions are going to have to rely on the chair. "Do we withhold information, because all chairs are not up to the task? No, we improve the manageability of the chair." Watson notes that at the University of Florida, if a chair is not managing properly it becomes clear to everyone involved very quickly. In fact, since the departments end up supporting each other or paying for a department that's not pulling its weight, peer pressure becomes a powerful tool. The controversy is the fear that in applying metrics or the entire system of mission-based management something precious and much more subjective will get lost. The results of much research, especially in its infancy, are intangible, Didion points out. "That's the tension. Academics look at themselves as artists--The Nobel laureate argument is always brought up. Nobel laureates, if you measured them, would not come out as so productive. The problem is, research is a business. If we are going to manage it well, we need some measures that a business has. We're not taking that artistry way," asserts Jones. Watson, however, is impatient with the Nobel laureate argument. "In the past, nobody collected data of any significance. So if you published, you got promoted. No one cared whether the research had any outcomes at all." In fact, some academics say they would welcome the measurements. At the Joan and Sanford I. Weill Medical College of Cornell University, where measurements of research and teaching are just being considered, Lisa Staiano-Coico considers herself productive in her research in wound repair and cell migration. "I think it's good to have a productivity measure, because I'm quite productive. I wouldn't have a hard and fast number to determine my career. If you are a highly productive faculty member, you would want to be rewarded. There needs to be some flexibility so you don't have arbitrary numbers determining somebody's career at an institution," says Staiano-Coico, who is also senior associate dean at the medical college in New York City. Also, Staiano-Coico points out, measuring each of the three missions will mean that teaching could be valued as much as research and clinical work by faculty members. "The faculty are enthusiastic about having teaching counting more toward tenure and their productivity." Kirch says faculty are aware of the "downward pressures" on support, especially in the clinical area at the Medical College of Georgia. "Faculty want to work where there are incentives. They want to have confidence in the tools and measures that are being used." In addition, proponents say they don't see these measurements having any impact on the type of research that academics pursue. " ... They pick the research project that garners support now." Nothing's perfect, including this system of measurements. Watson concludes, "You do worry that it gets to an extreme in an academic institution. 'Are they punching a time clock? Do you get paid to think?' But if we're not covering their salaries, it's a burden for the institution." Katherine Q. Woodbury is a freelance writer based in Philadelphia.

Managers on a Mission: Med schools begin to quantify research, teaching, and patient care

Interested in reading more?

Become a Member of