Streetwise Professor

October 24, 2010

Getting Schooled

Filed under: Economics,Politics — The Professor @ 10:03 am

At the risk of writing something that could be taken as special pleading, I’ll weigh in on the WSJ article “Putting a Price on Professors.”  The risk exists because the article is about state policy towards professors in Texas, and specifically discusses the University of Houston.  I plead guilty on all counts.

The article describes recent state laws that are intended to improve academic performance, specifically by increasing professorial accountability and efficiency, in part by quantifying faculty performance, with the implication that these quantitative measures will eventually be inputs to hiring, pay,  and curricular decisions.

At the outset, I should express my broad sympathy with these objectives.  As to the means, well, just let me say that with respect to the metrics for evaluating faculty performance, those pushing these metrics want to improve education in the worst way, and they’ve found it.

This is in fact a great illustration of the perils of high powered incentive systems in multi-task environments, something first analyzed rigorously by Holmstrom and Tirole Milgrom in JLEO in 1991*.  The basic problem with high powered incentives, in which compensation and perks are strongly related to measured performance, is that these incentives can cause serious distortions when some important aspects of performance are very hard to measure.  Those subject to this incentive system tend to devote excessive effort to the measured activities that determine compensation, and too little effort to the unmeasured–but potentially valuable–activities that do not affect compensation because they cannot be measured with any precision.

A canonical example from education is teaching to the test.  When teacher performance is evaluated primarily by student test performance, teachers have an incentive to teach to the test only, and stint on any non-test related instruction, even though said instruction may be quite beneficial and valuable to the students.

In many ways, this multi-tasking problem is captured by Einstein’s aphorism that not everything that counts can be measured, and that not everything that can be measured counts.  The Holmstrom-Tirole Milgrom corollary is that counting only those things that can be measured means that nobody will produce the things that count but can’t be measured, and are likely to produce things that can be measured but don’t really count.

Higher education is rife with measurability issues.  It is easy, for instance, to count student enrollments and graduation rates.  It is far harder to determine how much knowledge those students have received, especially in higher level and more abstract courses in non-quantitative areas.   It is difficult to monitor and measure faculty engagement with students outside the classroom.   Research is also extremely hard to evaluate.  Yes, you can count papers.  Yes, you can rank journals.  But both are noisy measures–potentially very noisy measures–of research “quality.”

So, it is much harder to measure some crucial dimensions of faculty quality and performance than others.  If you want to have research universities, and faculty who engage with students outside the classroom, etc., high powered incentive systems are not the way to achieve it.

If you take a look at WSJ article’s description of the spreadsheet at Texas A&M used to evaluate faculty contribution, you might be reminded of old Soviet incentive systems.  When the Soviets would try to measure the performance of nail makers by the weight of nails produced, the factories would churn out small numbers of huge, heavy nails.  Not liking this, the planners changed the incentive system to base compensation on the number of nails produced, so the factories produced lots of little nails.  Analogously, if you reward faculty for teaching big sections of basic courses, small specialized electives will disappear, or will be staffed by the least influential (usually, most junior) faculty who are less capable (on average) of teaching them.  If you include grant monies as a primary criterion, faculty members will spend more time grant grubbing and less time doing other things that are also important to the educational and research missions.  Students are frequently not in the best position to evaluate the value or utility of what they are being taught, so making faculty salary and promotion highly dependent on student teaching evaluations tends to skew teacher efforts towards entertainment and achieving popularity, rather than delivering knowledge and constructive (and sometimes painful) feedback.

This is not to say that low powered incentive systems are costless.  All of the criticisms of the modern university have a basis in fact.  Some faculty respond to low powered incentives by mailing it in, or teaching classes that are personally satisfying (or easy) but which are not useful for students.

But in this, as in everything else, there are trade-offs.  If you don’t like things the way they are . . . be careful what you ask for.  Changes intended to address one problem–e.g., faculty sloth/moral hazard–can create other problems that are far more costly.

The modern university is characterized by low powered incentives.  Universities are almost exclusively not-for-profit.  Internal reward systems have low incentive power.  Before rushing in to change that system wholesale, it is worthwhile to stand back and consider why these arrangements have developed, and more importantly, survived.  The well-documented problems with for-profit universities provide a valuable cautionary tale, and a useful contrast to the criticisms of the traditional university.

As in most things, rather than trying to engineer from above (in legislatures or governors’ offices) to achieve superior results, it is better to encourage an environment in which competition not just in price, but in organizational form and internal management and governance, can flourish.  The practices that survive will not be perfect, because perfection is not an option in economics, but it is likely that they will have attributes that are, as a whole, superior to those that do not.

* Error caught by “Anonymous.”

Print Friendly


  1. As in most things, rather than trying to engineer from above (in legislatures or governors’ offices) to achieve superior results, it is better to encourage an environment in which competition not just in price, but in organizational form and internal management and governance, can flourish. The practices that survive will not be perfect, because perfection is not an option in economics, but it is likely that they will have attributes that are, as a whole, superior to those that do not.

    Exactly. Stand well back and let the market rip. Let individual customers decide for themselves what sort of academic instruction they value the most; those that don’t offer a valuable service go bust.

    Comment by Tim Newman — October 24, 2010 @ 1:55 pm

  2. [...] Go read Craig Pirrong’s post on the WSJ article at Streetwise Professor.  That’s an [...]

    Pingback by Earning My Keep…. « Organizations and Markets — October 24, 2010 @ 2:03 pm

  3. A similar controversy is raging in high school education…esp. wrt Michelle Rhee’s DC reforms. The teacher’s union is broadly in support of performance pay – but the crucial question for them is how exactly “performance” will be measured. Rhee’s solution was to simply use test score results. The union argues this will simply distort educational quality and make teachers grade monkeys……
    On a related note, I believe a lot of inefficiency in higher education exists due to the presence of an extraordinary number of “administrative” jobs which are better paid but involve nothing more than random paper pushing.

    Comment by Surya — October 24, 2010 @ 3:12 pm

  4. TYPO: Milgrom not Tirole

    Comment by anonymous — October 24, 2010 @ 4:42 pm

  5. A spontaneous order for education? Sounds great to me. Decentralized free markets work best in economics, so why not for education?

    Comment by Troy Camplin — October 25, 2010 @ 1:23 am

  6. [...] that taxpayers are getting their money’s worth from state universities.  Acknowledging that his comments “could be taken as special pleading,” a blogger who is also a Texas state employee providing university students with advanced [...]

    Pingback by Could be special pleading on higher education « Knowledge Problem — October 25, 2010 @ 8:25 am

  7. Speaking from a students perspective, the ability to research professor’s academic and professional history definitely aided my course decisions. Granted LinkedIn helped more than anything the state put out there (not to mention this blog).

    Take the MBA market in Houston. UH is ~$30K, HBU is ~$40, Tulane is ~$70, Rice is ~$80, and UT is ~$90. It’s kind of shocking that a public program is the most expensive (and one is the least expensive). All of these program’s have excellent faculty, but by and large, is UT’s 3X better than UH’s? Not a chance. Even if you could measure the ‘quality’ of professors, there is still no justification for runaway education costs (regardless of whether or not MBAs are thought of as ‘profit centers’ for their respective schools).

    Comment by Jack — October 25, 2010 @ 12:50 pm

  8. Jack–I have no problems with giving students more information re faculty background. What would be a problem going forward is crack-brained incentive schemes. And the cost thing is clearly an issue. I can assure you that my salary, and that of most colleagues has increased far more slowly than tuition. It’s something of a puzzle as to where all the money goes.

    The ProfessorComment by The Professor — October 25, 2010 @ 4:58 pm

  9. Understood, as a recent student I can attest to the flippant nature of faculty reviews too. Teaching to the test has bastardized this state’s education system already (TAKS is quite possibly the worst thing to happen to Texas’ public schools), I shudder at the thought of that being expanded to higher education.

    If my tuition was increasing 10% yoy and my professor’s were getting across the board 10% raises, I would be happy with that. The free market dictates that the most talented professors will (all other things being equal) flock to the programs that compensate them best, but yes, this black hole since deregulation is intriguing.

    Comment by Jack — October 26, 2010 @ 1:36 pm

  10. One of the other bits of data lacking in the WSJ piece is that while they report on the rate of tuition increase a public universities over the past decade or two, they do not report on how much state support of public universities has been cut over that period. I suspect that, in many cases, the rate of tuition increase is proportional (in absolute value) to the decrease in state support.

    Comment by Mike — October 28, 2010 @ 8:40 pm

  11. Andrew,

    > Georgians once again outperform Russians in development of a modern society.

    Wait, you are confusing Russia and Swaziland:

    List of countries by GDP (PPP) per capita

    44 Estonia 24,450
    45 Lithuania 24,356
    46 Russia 23,589
    47 Poland 22,783
    48 Hungary 22,635
    49 Latvia 21,810

    99 Ukraine 7,298

    103 Bhutan 6,591
    104 Armenia 6,544
    108 Angola 6,006
    109 Georgia 5,833
    110 Mongolia 5,374
    113 Swaziland 5,161

    Comment by vladislav — April 11, 2014 @ 3:34 am

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress