Streetwise Professor

November 30, 2009

Providing Incentives to Collect Data

Filed under: Climate Change,Uncategorized — The Professor @ 8:00 pm

Commentor Scott raises the issue of the incentive to collect data, particularly large data sets, if it is imperative to make said data public.  This is a very important issue that I addressed a bit in my first post on the CRU fiasco, but it’s worth a couple of additional thoughts.

Here are some ways to provide incentives:

  • Subsidies, in the form of grants or prizes.  This is already done to some degree.  Receipt of such funding to support data collection–i.e., the subsidization of a public good–should be contingent on making this data freely available.
  • The provision of complementary services for a fee, such as consulting on the use of the data, or creation of customized data sets.  This is the source of revenues for many providers of open source software.
  • Licensing or sale of the data for a reasonable fee.  This has been quite effective for many databases widely used by finance and economics academics (and industry), notably the CRSP stock and bond data (but others as well).
  • Conditioning access to data on formal recognition in all working papers and published papers, perhaps including the creation of a “data authorship” category of something of the sort.

In brief, a variety of mechanisms can provide incentives to create information/data.  A variety of mechanisms are utilized in markets for other information goods.  The just listed are some of the most well known.

Intellectual property rights (e.g., granting rights of exclusive use, trade secrets) are other means by which creators can capture a stream of benefits, thereby giving them an incentive to produce information goods.  These may be the efficient arrangement in some settings, but I am dubious that that is the case in the sciences and social sciences.  Science depends on replication, which is incompatible with “I’d show you but I’d have to kill you” secrecy.

I also had an additional thought regarding how to evaluate scholarship for hiring, promotion, tenure, and salary decisions in a research university in an open source, non-journal dominated system.  I mentioned the idea of using citations as a main metric, and evaluating the “quality” of citations based on characteristics of the works in which a citation occurs–such as the number of times the citing author’s work is itself cited.  This is essentially a links-based metric.  Major search engines, notably Google, utilize such metrics in ranking web sites to determine display order.  The concepts underlying search engine ranking algorithms could perhaps be adapted to rank scholarly impact as well.

The creation of a data set that is utilized by other scholars could be an input to the ranking algorithm, providing another incentive to invest in their creation.

Presumably the CRU employees associated with the creation of a climate data set would have achieved a stratospheric impact factor/ranking under such a system because myriad other scholars wold have used the data.  But of course, this would have come at a (private) cost: they couldn’t have controlled the conclusions of the work done with their data (which the emails suggest was an important consideration to them).  But that private cost is swamped by the benefit of permitting open access to other researchers, so the proper response is: tough luck to you.

Print Friendly, PDF & Email


  1. one disadvantage of the ‘link-based’ evaluation system: it encourages group-think even more than the current systme. In my research I have come across brilliant papers that have been rarely cited, simply because they made unpopular suggestions. This is probably more true in the humanities than the natural sciences. The real solution, of course, is to get the state out of the business of (higher) education altogether. Have those who benefit from research pay for it. I don’t see any logical reason to treat research differently from sandwiches or newspapers. Maybe its the romantic notion of the ivory tower residents pluckily working on basic research we all need… Hardly supported by evidence. Research, like any other human activity, can be wasteful when not conducted in a system of free exchange.

    Comment by dc — December 9, 2009 @ 10:01 am

  2. dc–thanks for your comment. No system is invulnerable to groupthink. I don’t think, however, that a link-based system would be more vulnerable to it than the current, more hierarchical system. Yes, there can be social pressures that induce people not to recognize heterodox work (even if it is brilliant), but the journal-based system makes it easier to control discourse and punish those defecting from the consensus.

    Link-based systems can be gamed (Google is very secretive about its ranking systems for just that reason). But all-in-all I think that lower entry barrier systems are more likely to encourage the dissemination of the different than the current, high entry barrier system. It is closer to the free exchange system that you mention.

    The ProfessorComment by The Professor — December 9, 2009 @ 10:12 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress