Risk, uncertainty, and value judgements in science policy

Yesterday my colleagues and I at Michigan State University and Kellogg Biological Station had a reading group to discuss Pielke's The Honest Broker. We read chapters 4-6 for today, which are titled, "4) Values; 5) Uncertainty; and 6) How science policy shapes science in policy and politics."

We talked about whether science is a good tool in making decisions. Certainly it can be good for informing decisions, such as if there's a tornado coming and you need to know whether you should evacuate. Unfortunately, as we saw in one of my previous posts, sometimes scientific assessments of risk and uncertainty do NOT translate well into action. Pielke agrees with this perspective. He thinks that science just adds smoke and mirrors to debates that are really about core values. So unless the situation under debate is one with low uncertainty and highly shared values (a tornado is coming, we should evacuate), we need more recognition of the underlying values of a debate (see: the climate change debate).

Pielke repeatedly refers to two works by Dan Sarewitz, who is one of my professors at Arizona State and regarded by many as a science policy guru. The first article is "How science makes environmental controversies worse" (2004). The second is "Science and Environmental Policy: An Excess of Objectivity" (2000). Both are worth a thorough reading: one thing I've discovered in grad school is that I sometimes read the same article months, or a year, apart, and find revelatory new nuggets of knowledge each time I read it.

The "Excess of Objectivity" book chapter is an insightful commentary on how science can actually impede the political process, by focusing on always disputable and uncertain facts while ignoring underlying value conflicts in highly politicized environmental issues. The “excess of objectivity” refers to the incompatibility of multiple fields of science, and how while each field claims objectivity, they drive controversy and muddy the political waters.

"How science makes environmental controversies worse" makes the same core argument, using a set of different examples from the 2000 election results, to climate change, to genetically modified food (another good case study is the debate over nuclear waste: see this editorial). This discussion reminded me of an article I read during my first weeks of grad school, "Value Judgments and Risk Comparisons. The Case of Genetically Engineered Crops" (2003) by Paul Thompson, who is an environmental and agricultural philosopher at MSU.

I wrote up an analysis of it that I think highlights the issues of value, risk, and uncertainty in environmental controversies pretty well: Thompson focuses on the inherent value judgments that scientists make about genetically engineered (GE) crops and environmental risk. He aims to identify the values behind the GE debate, rather than taking a philosophical or scientific position in the debate. He focuses on a relatively small aspect of this debate, which are claims for and against a comparative evaluation of the environmental risk of GE vs. traditional (non-GE) crops. This is the standard metric used by scientists and federal agencies to assess the risk of GE crops. Thompson’s argument is that risk assessments are inherently based on value-based judgments; the science itself cannot settle a claim about environmental risks.

He shows that the current regulatory system ironically puts the burden of proof on anti-GE activists, who are “in the position of needing to justify special treatment for this class of plants” (emphasize added, Thompson, 2003, p. 11). This gap charges the largely non-scientific public with demonstrating the scientific credibility of their value system, against the grain of the values held by the scientific community, which of course causes further problems on multiple levels. Thompson identifies several other challenges in the regulation of GE crops based on the current framework.

Risk assessments, especially environmental risk assessments, depend on value-based judgments of how much and what types of risk are “acceptable,” despite attempts to scientifically quantify this risk. The definitions of risk by the scientists and activists are essentially incompatible for comparing the risks of GM vs. non-GM crops, or even defining the concept of environmental risk. This highlights very clearly that science, rather than aiding the decision-making process, can complicate and add uncertainty to political debates.

On a related note, I'm headed to Boston today to attend the Science and Democracy Network conference! I'm really excited to talk to like-minded scholars about our work, and make some great connections.

4) Science and Public Value

Image Source.

A friend of mine asked about my last post on the "co-production" of knowledge, "who then 'owns' the research or is it always a public resource after co-production?”

Great question, and one that scholars have been struggling with especially in light of patents on genes and other biotechnology, such as genetically modified foods. This is generally referred to as "intellectual property" or "intellectual property rights" (IPR). Patents are supposed to protect the inventor and fuel innovation, but the case lately has been an increasingly convoluted fight over patent law, with "patent sharks" prowling for unclaimed discoveries that they can later sue companies for using. The figure below demonstrates some of the craziness in just smart phones:

Image Source.

But what happens when a drug company asks an indigenous tribe about their medicinal plants, and then goes on to patent and produce the medicinal compound? Or when patients donate their DNA to a study, only to be charged later for a test or treatment because a biotech company has patented the blueprint of the gene that causes cancer? Who should "own" that knowledge?

These are questions that modern governments are dealing with for the first time due to technological advances. Public research organizations are dealing with them as well: for example, the public agricultural research system that is largely responsible for last century's "Green Revolution" now must be more cautious about what agricultural technologies they can use, because of all the patents. Richard Jefferson is someone who understands this problem and is creating innovative solutions that benefit poor countries. He started a company that promotes "open source biology" by patenting discoveries in agricultural science, but then making those discoveries public. Some excerpts from this paper:

Most critically, we must democratize these abilities, both to measure and to respond, in order to diversify agro-ecosystems and environments and decentralize the problem-solving capability. We will achieve this by fostering scientific method and harnessing local knowledge and commitment in communities that have previously been ignored or treated as passive recipients of help. (p. 38)

At the start of the twenty-first century, science is at a critical juncture. Four centuries of inquiry, discovery, and invention have created a base of knowledge that has the potential to provide people everywhere, in all circumstances, with nourishment, improved health, and longer life. But the institutional mechanisms that ostensibly exist to encourage the application of science to practical problems are today hindering that very process. The norms that have evolved around gate-keeping have created new clergy, new impediments and new inefficiencies. Without a systemic change, science’s promise will not be available for those who most need it, and the promise of a truly diverse, robust and fair innovation culture may elude us. (p. 40)

This all boils down to a question of science and the public good. The "social contract of science" is an unspoken agreement that science, in the end, will produce public good. As the environmental movement often points out, science sometimes produces public bads. Or it doesn't produce the hoped-for goods. For example, “there are 6000 patents that invoke ‘plant breeding’ and ‘drought resistance’ yet none of them has yet resulted in an improved commercial variety” (Clark et al., p. 10). Agricultural extension programs do unique boundary work that is affected by both private and public interests. The private sector is crucial to developing new, useful technologies for farmers. Agricultural research institutions must increasingly embrace their role as a mediator between the private realm of gene patents and their goal of developing agricultural technologies for the public good.

More broadly, many of my colleagues at ASU's Consortium for Science, Policy, and Outcomes are working on this issue of science and public value. A recent issue of the journal Minerva featured their work, and a short review is available here. Also, this very-readable report by a British think-tank called Demos takes a Science & Technology Studies perspective on this topic. They tackle head-on provocative questions that I've been exploring throughout this blog:

Science has major social benefits and thus ‘public value’. Yet crucially, as recent controversies have underlined, this value cannot be assumed and taken as automatic, no matter what scientific research is done, or under what conditions. We need therefore to shift from noun to adjective, by asking not only: what is the public value of science? But also, what would public value science look like? (p. 29)

3) Public participation in science: co-production of knowledge

Image source.

An important theme of Science & Technology Studies is the "co-production of knowledge." When I first started my graduate studies at ASU, this word threw me for a loop because it packs so much meaning into one phrase, but I guess that's the purpose of academic jargon! The co-production of knowledge means, according to Sheila Jasanoff, "the proposition that the ways in which we know and represent the world (both nature and society) are inseparable from the ways we choose to live in it" (Jasanoff, 2004, p. 2).

Jasanoff's definition incorporates how knowledge shapes both science and social order (sometimes simply described as the co-production of science and policy or society). How we represent knowledge-- through charts and graphs, DNA samples, lie detector tests, brain scans, environmental impact assessments, maps, etc.-- has implications for not just science, but society. For example, a map of climate change vulnerability can show the scientific results of a study, but it also holds implicit values and political implications.

The creators of this climate change vulnerability map chose not to include most developed countries in their assessment, which certainly has impacts of how we view climate change. For example, climate change is sometimes viewed as a problem mostly facing developing countries, and this map reinforces this. Co-production of knowledge and social order goes beyond just media representations of science; it is deeply important for our political system, and how we make decisions based on science.

We often think of science as a top-down, self-regulating hierarchy of experts. But when science gets used in decision-making, it must conform to the ideals of democracy. Science cannot dictate policy decisions, but can be a useful political tool. Thus, incorporating the "co-production" perspective of science and society can make the role of science more clear in these situations, rather than the muddled role it currently takes. For example, Roger Pielke Jr. shows how the Intergovernmental Panel on Climate Change (IPCC) ignored the "co-production of knowledge" and instead engaged in what he calls "stealth advocacy" of policy. If the IPCC was more upfront about the political implications of their climate change assessments, they would act as more of an "honest broker" of the co-production of science and policy.

Several scholars have examined climate change knowledge from a co-production perspective. Vogel et al. (2007) examine the connections between co-production and science communication. Commenting on many of the topics previously discussed in this blog, they use the case study of food security and climate vulnerability assessments in southern Africa, and how crossing the science-practice boundary through stakeholder engagement resulted in more useful assessments that could be utilized by local organizations and governments. By recognizing the needs of stakeholders, the knowledge gained from these assessments can be used in more democratic ways.

Lemos and Morehouse (2005) look at the case study of NOAA's regional integrated science assessment (RISA) program (coincidentally, the Great Lakes basin now has a GLISA program). The Southwest RISA used a process of stakeholder dialogues to produce a regional assessment of climate change impacts that was relevant and useful to end users. The authors write,

"Co-production of science and policy in the context of integrated assessment activities requires substantial commitment to the three components we have identified: interdisciplinarity, stakeholder participation, and production of knowledge that is demonstrably usable." (Lemos & Morehouse, 2005, p. 66)

In this case, the RISA aimed to produce scientific knowledge about climate change that would be useful for decision-makers like farmers and policy-makers. Without the stakeholder participation, the researchers would not have known how the information would be applied, and how to shape their research based on this. For a more detailed description of other RISA climate change programs, see this report.

For the climate change and agriculture project I'm working on with Michigan State University Extension and Kellogg Biological Station, we are trying to follow the model of stakeholder participation from the start. This is based partly on the work of our colleagues in the Southeast climate RISA and participatory process used to create their AgroClimate website. There are also great examples of using participatory focus groups and community dialogues in forestry and bioethics. Personally, I can tell you that something I've already learned is how important social science research is to this process. The natural sciences can tell us a lot about our world, but social science helps tell us what decision-makers need for knowledge and support. The results are often surprising and definitely eye-opening for those of use entrenched in academia. And this is why co-production of knowledge is important! Recognizing that the process of knowledge production (aka science) is just as important as the end results, and that how the end results are used is often pervasively social and political, STS can lend us some valuable insights for practical results.

Science policy communication failure costs lives

Image source.

A recent issue of Science magazine features a news article about seven scientists in Italy who are facing manslaughter charges for not predicting the danger of an earthquake that killed 308 people. The scientists were part of a risk committee of earth scientists who testified that incipient tremors were not evidence of an oncoming earthquake in 2009. According to Science, “They agreed that no one can currently predict precisely when, where, and with what strength an earthquake will strike” (3 June 2011, p. 1135). These are all accurate statements, from a scientific point of view. But the problem lies in translating these statements for decision-makers and stakeholders, which includes people in the town of L’Aquila, Italy.

The lead scientist “maintained that he and his scientific colleagues had a responsibility to provide the ‘best scientific findings’ and that it is ‘up to politicians’ to translate the scientific findings into decisions” (Science, 3 June 2011, p. 1136). This is the linear model of science policy at its worst, literally costing lives because of the mismatch of science and policy risk management paradigms, or as Cash et al. (2006) describe, the “loading dock” model of simply delivering scientific results and hoping that the public sphere will pick them up and use them. To the scientists, risk and uncertainty are quantifiable metrics that are difficult to translate into social action. To decision-makers and the public, risk is a socially mediated, multidimensional value that depends on more than just probabilities. Uncertainty has been a traditional sticking point in earth science and policy topics such as climate change. However, Cash et al. (2006) demonstrate how bringing together scientists and decision-makers from the beginning helped improve the utility of climate models for end-users. They write, “Scientists began to understand that managers were comfortable making decisions under uncertainty, and managers began to understand the concerns scientists had about making scientific claims in the face of uncertainty” (Cash et al., 2006, p. 482). This was clearly not the case with the Italian scientists and decision-makers.

At first glance, this case provokes outcry from scientists afraid of losing the public’s trust and being put on trial, literally. While it may be presumptuous to actually put scientists on trial for a failure to dialogue with decision-makers, this puts into question the implicit “social contract of science” that has justified basic scientific research since the end of WWII. Sheila Jasanoff told a group of ASU graduate students last spring that, “Scientists have become arrogant, and have not explained to the people why they deserve support... The Enlightenment was not a historical event. It is a process, a mission, a continuous duty to explain yourself” (personal communication, 11 February 2011; not an exact quote, but very close). Jasanoff lays out an alternative claim to the linear model of science policy that she calls “technologies of humility” (2003). In contrast to calls for “more science” to reduce uncertainty, Jasanoff writes that, “what is lacking is not just knowledge to fill the gaps, but also processes and methods to elicit what the public wants, and to use what is already known” (2006, p. 240). The abstract of her paper states, “governments should reconsider existing relations among decision-makers, experts, and citizens in the management of technology. Policy-makers need a set of ‘technologies of humility’ for systematically assessing the unknown and the uncertain” (Jasanoff, 2003, p. 223). Jasanoff and other Science and Society scholars have been writing about the failures of the linear science policy model in predicting risk since the 1980s, when the risk-management paradigm began to crumble in the wake of seemingly “unpredictable” human-technology-based disasters like Chernobyl. Today we face critical policy issues from climate change to toxic chemicals that fundamentally depend upon and understanding of environmental science, but just understanding the science is not enough. We need a new model of science policy that incorporates the needs of decision-makers and stakeholders from the start, not after it’s too late.

Sources:

Cartlidge, E. (3 June 2011). “Quake Experts to Be Tried For Manslaughter.” Science, 332, p. 1135-1136.

Cash, D.W., Borck, J.C., & Patt, A.G. (2006). “Countering the Loading-Dock Approach to Linking Science and Decision Making.” Science, Technology, & Human Values, 31, p. 465-494.http://sciencepolicy.colorado.edu/students/envs_5100/Cashetal2006.pdf

Jasanoff, Sheila (2003). “Technologies of Humility: Citizen Participation in Governing Science.” Minerva, 41. 223-244.http://sciencepolicy.colorado.edu/students/envs_5100/jasanoff2003.pdf

Further reading:

Sarewitz, D., Pielke, Jr., R.A., & Byerly, R. (editors) (2000). Prediction: Science, Decision Making and the Future of Nature. Washington, DC: Island Press. Available at: Google books, Amazon.com