Hostname: page-component-7b9c58cd5d-7g5wt Total loading time: 0 Render date: 2025-03-16T13:05:14.369Z Has data issue: false hasContentIssue false
  • English
  • Français

Experimenter demand effects in economic experiments

Published online by Cambridge University Press:  14 March 2025

Daniel John Zizzo
Daniel John Zizzo*
Affiliation:
School of Economics and CBESS, University of East Anglia, Norwich NR4 7TJ, UK
*
e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Experimenter demand effects refer to changes in behavior by experimental subjects due to cues about what constitutes appropriate behavior. We argue that they can either be social or purely cognitive, and that, when they may exist, it crucially matters how they relate to the true experimental objectives. They are usually a potential problem only when they are positively correlated with the true experimental objectives’ predictions, and we identify techniques such as non-deceptive obfuscation to minimize this correlation. We discuss the persuasiveness or otherwise of defenses that can be used against demand effects criticisms when such correlation remains an issue.

Keywords

Experimenter demand effectsExperimental designExperimental instructionsSocial desirabilitySocial pressureFramingMethodology

JEL classification

C91: Laboratory, Individual Behavior C92: Laboratory, Group Behavior B41: Economic Methodology
Type
Research Article
Information
Experimental Economics , Volume 13 , Issue 1 , March 2010 , pp. 75 - 98
Copyright
Copyright © Economic Science Association 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

I wish to thank participants to a presentation at Bologna Forli’ and Piers Fleming, Nick Bardsley, Dan Benjamin, Pablo Branas-Garza, Tim Cason, Gary Charness, Andrew Colman, John Duffy, Dirk Engelmann, Paul Ferraro, Shaun Hargreaves Heap, Alexander Koch, Charlie Plott, Jen Shang and two anonymous referees for useful feedback and references on the topic. Partial financial support from the Nuffield Foundation (SGS/35859) is gratefully acknowledged. The usual disclaimer applies.

References

Abbink, K., & Hennig-Schmidt, H. (2006). Neutral versus loaded instructions in a bribery experiment. Experimental Economics, 9, 103–121.CrossRefGoogle Scholar
Ackert, L. F., Charupat, N., Church, B. K., & Deaves, R. (2006). An experimental examination of the house money effect in a multi-period setting. Experimental Economics, 9, 5–16.CrossRefGoogle Scholar
Alm, J., McClelland, G. H., & Schultze, W. D. (1992). Why do people pay taxes? Journal of Public Economics, 48, 21–38.CrossRefGoogle Scholar
Andreoni, J., & Miller, J. (2002). Giving according to GARP: An experimental test of the consistency of preferences for altruism. Econometrica, 70, 737–753.CrossRefGoogle Scholar
Ariely, D., Loewenstein, G., & Prelec, D. (2003). Coherent arbitrariness: Stable demand curves without stable preferences. Quarterly Journal of Economics, 118, 73–105.CrossRefGoogle Scholar
Asch, S. E. (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychology Monographs, 70, 1–70.CrossRefGoogle Scholar
Bacharach, M., Guerra, G., & Zizzo, D. J. (2007). The self-fulfilling property of trust: An experimental study. Theory and Decision, 63, 349–388.CrossRefGoogle Scholar
Baldry, J. C. (1986). Tax evasion is not a gamble. Economics Letters, 22, 333–335.CrossRefGoogle Scholar
Ball, S., Eckel, C., Grossman, P.J., & Zane, W. (2001). Status in markets. Quarterly Journal of Economics, 116, 161–188.CrossRefGoogle Scholar
Bardsley, N. (2005). Experimental economics and the artificiality of alteration. Journal of Economic Methodology, 12, 239–251.CrossRefGoogle Scholar
Bardsley, N. (2008). Dictator game giving: Altruism or artefact? Experimental Economics, 11, 122–133.CrossRefGoogle Scholar
Binmore, K., Shaked, A., & Sutton, J. (1985). Testing noncooperative bargaining theory: a preliminary study. American Economic Review, 75, 1178–1180.Google Scholar
Blass, T. (1999). The Milgram paradigm after 35 years: Some things we now know about obedience to authority. Journal of Applied Social Psychology, 29, 955–978.CrossRefGoogle Scholar
Bosman, R., & van Winden, F. (2002). Emotional hazard in a power-to-take experiment. Economic Journal, 112, 147–169.CrossRefGoogle Scholar
Branas-Garza, P. (2006). Poverty in dictator games: Awakening solidarity. Journal of Economic Behavior and Organization, 60, 306–320.CrossRefGoogle Scholar
Branas-Garza, P. (2007). Promoting helping behavior with framing in dictator games. Journal of Economic Psychology, 28, 477–486.CrossRefGoogle Scholar
Burnham, T. C. (2003). Engineering altruism: a theoretical and experimental investigation of anonymity and gift giving. Journal of Economic Behavior and Organization, 50, 133–144.CrossRefGoogle Scholar
Camerer, C. F. (2003). Behavioral game theory: experiments in strategic interaction. Princeton: Princeton University Press.Google Scholar
Casari, M., & Cason, T. N. (2009). The strategy method lowers measured trustworthy behavior. Economics Letters, 103, 157–159.CrossRefGoogle Scholar
Cason, T. N., & Sharma, T. (2007). Recommended play and correlated equilibria: an experimental study. Economic Theory, 33, 11–27.CrossRefGoogle Scholar
Chou, E., McConnell, M., Nagel, R., & Plott, C. R. (2009). The control of game form recognition in experiments: Understanding dominant strategy failures in a simple two person “guessing” game. Experimental Economics, 12, 159–179.CrossRefGoogle Scholar
Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: Wiley.Google Scholar
Davis, D. D., & Holt, C. A. (1993). Experimental economics. Princeton: Princeton University Press.CrossRefGoogle Scholar
Eckel, C. C., & Grossman, P. J. (2000). Volunteers and pseudo-volunteers: the effect of recruitment method in dictator experiments. Experimental Economics, 3, 101–120.CrossRefGoogle Scholar
Eckel, C., & Holt, C. A. (1989). Strategic voting in agenda-controlled committee experiments. American Economic Review, 79, 763–773.Google Scholar
Ellingsen, T., & Johannesson, M. (2008). Pride and prejudice: the human side of incentive theory. American Economic Review, 98, 990–1008.CrossRefGoogle Scholar
Fehr, E., Kirchsteiger, G., & Riedl, A. (1993). Does fairness prevent market clearing? An experimental investigation. Quarterly Journal of Economics, 108, 437–459.CrossRefGoogle Scholar
Festinger, L. (1957). A theory of cognitive dissonance. Evanston: Peterson Row.CrossRefGoogle Scholar
Fischbacher, U., Gachter, S., & Fehr, E. (2001). Are people conditionally cooperative? Evidence from a public goods experiment. Economics Letters, 71, 397–404.CrossRefGoogle Scholar
Fleming, P., Townsend, E., Lowe, K. C., & Ferguson, E. (2007). Social desirability effects on biotechnology across the dimensions of risk, ethicality and naturalness. Journal of Risk Research, 10, 9891003.CrossRefGoogle Scholar
Forsythe, R., Rietz, T., Myerson, R., & Weber, R. (1996). An experimental study of voting rules and polls in three-candidate elections. International Journal of Game Theory, 25, 335–383.CrossRefGoogle Scholar
Frank, B. L. (1998). Good news for the experimenters: Subjects do not care about your welfare. Economics Letters, 61, 171–174.CrossRefGoogle Scholar
French, J. R. P. Jr., & Raven, B. (1959). The bases of social power. In Cartwright, D. (Ed.), Studies in social power (150-167). Ann Arbor: Research Center for Group Dynamics University of Michigan.Google Scholar
Gale, J., Binmore, K. G., & Samuelson, L. (1995). Learning to be imperfect: the ultimatum game. Games and Economic Behavior, 8, 856–890.CrossRefGoogle Scholar
Golding, S. L., & Lichtenstein, E. (1970). Confession of awareness and prior knowledge of deception as a function of interview set and approval motivation. Journal of Personality and Social Psychology, 14, 213–223.CrossRefGoogle Scholar
Guerra, G., & Zizzo, D. J. (2004). Trust responsiveness and beliefs. Journal of Economic Behavior and Organization, 55, 25–30.CrossRefGoogle Scholar
Harrison, G. W. (2007). House money effects in public good experiments: Comment. Experimental Economics, 10, 429–437.CrossRefGoogle Scholar
Harrison, G. W., & Johnson, L. T. (2006). Identifying altruism in the laboratory. In Davis, D. & Isaac, R. M. (Eds.), Research in experimental economics: Vol. 11. Experiments investigating fundraising and charitable contributors (pp. 177–223). Amsterdam: Elsevier.CrossRefGoogle Scholar
Heinemann, F., Nagel, R., & Ockenfels, P. (2004). The theory of global games on test: Experimental analysis of coordination games with public and private information. Econometrica, 72, 1583–1599.CrossRefGoogle Scholar
Hoffman, E., McCabe, K., Shachat, K., & Smith, V. (1994). Preferences, property rights, and anonymity in bargaining games. Games and Economic Behavior, 7, 346–380.CrossRefGoogle Scholar
Huck, S., Normann, H.-T., & Oechssler, J. (2004). Two are few and four are many: number effects in experimental oligopolies. Journal of Economic Behavior and Organization, 53, 435–446.CrossRefGoogle Scholar
Jones, S. R. G. (1984). The economics of conformism. Oxford: Blackwell.Google Scholar
Jones, S. R. G. (1992). Was there a Hawthorne effect? American Journal of Sociology, 98, 451–468.Google Scholar
Kirkpatrick, L. A., & Ellis, B. J. (2004). An evolutionary-psychological approach to self-esteem: multiple domains and multiple functions. In Brewer, M. B. & Hewstone, M. (Eds.), Self and social identity (pp. 52–77). Oxford: Blackwell.Google Scholar
Lei, V., Noussair, C. N., & Plott, C. R. (2001). Nonspeculative bubbles in experimental asset markets: Lack of common knowledge of rationality vs. actual irrationality. Econometrica, 69, 831–859.CrossRefGoogle Scholar
Levitt, S. D., & List, J. A. (2007). What do laboratory experiments measuring social preferences reveal about the real world? Journal of Economic Perspectives, 21, 153–174.CrossRefGoogle Scholar
List, J. A. (2007). On the interpretation of giving in dictator games. Journal of Political Economy, 115, 482–493.CrossRefGoogle Scholar
Loomes, G. (1999). Some lessons from past experiments and some challenges for the future. Economic Journal, 109, F35–F45.CrossRefGoogle Scholar
Markman, A. B., & Moreau, C. P. (2001). Analogy and analogical comparison in choice. In Gentner, D., Holyoak, K. J. & Kokinov, B. (Eds.), The analogical mind: perspectives from cognitive science (pp. 363399). Cambridge: MIT Press.Google Scholar
Mayo, D. G. (1996). Errorand the growth of experimental knowledge. Chicago: Chicago University Press.CrossRefGoogle Scholar
Mayo, E. (1933). The human problems of an industrial civilization. New York: MacMillan.Google Scholar
Milgram, S. (1974). Obedience to authority: an experimental view. New York: Harper and Row.Google Scholar
Noussair, C. M., & Powell, O. (2009). Peaks and valleys: experimental asset markets with non-monotonic fundamentals. Journal of Economic Studies (forthcoming).Google Scholar
Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783.CrossRefGoogle Scholar
Orne, M. T. (1973). Communication by the total experimental situation: why it is important, how it is evaluated, and its significance for the ecological validity of findings. In Pliner, P., Krames, L., & Alloway, T. M. (Eds.), Communication and affect (pp. 157–191). San Diego: Academic Press.CrossRefGoogle Scholar
Ortmann, A., & Hertwig, A. (2002). The costs of deception: evidence from psychology. Experimental Economics, 5, 111–131.CrossRefGoogle Scholar
Plott, C. R., & Levine, M. E. (1978). A model of agenda influence on committee decisions. American Economic Review, 68, 146–160.Google Scholar
Rosnow, R.L., & Rosenthal, R. (1997). People studying people: artifacts and ethics in behavioral research. New York: Freeman.Google Scholar
Sarin, R. K., & Weber, M. (1993). Effects of ambiguity in market experiments. Management Science, 39, 602–615.CrossRefGoogle Scholar
Schotter, A., & Sopher, B. (2003). Social learning and coordination conventions in intergenerational games: an experimental study. Journal of Political Economy, 111, 498–529.CrossRefGoogle Scholar
Shang, J., & Croson, R. (2009). Field experiments in charitable contribution: the impact of social information on the voluntary provision of public goods. Economic Journal, 119, 1422–1439.CrossRefGoogle Scholar
Shanks, D.R., & St. John, M. F. (1994). Characteristics of dissociable human learning systems. Behavioral and Brain Sciences, 17, 367–447.CrossRefGoogle Scholar
Sitzia, S., & Zizzo, D. J. (2009). In search of product complexity effects in experimental retail markets. Theory and Decision (forthcoming).Google Scholar
Smith, V. L. (1982). Microeconomic systems as an experimental science. American Economic Review, 72, 923–955.Google Scholar
Smith, V. L. (1994). Economics in the laboratory. Journal of Economic Perspectives, 8, 113–131.CrossRefGoogle Scholar
Solnick, S. (2007). Cash and alternate methods of accounting in an experimental game. Journal of Economic Behavior and Organization, 62, 316–321.CrossRefGoogle Scholar
Stewart, N., Chater, N., Stott, H. P., & Reimers, S. (2003). Prospect relativity: how choice options influence decision under risk. Journal of Experimental Psychology: General, 132, 23–46.CrossRefGoogle ScholarPubMed
Stocco, A., & Fum, D. (2008). Implicit emotional biases in decision making: the case of the Iowa gambling task. Brain and Cognition, 66, 253–259.CrossRefGoogle ScholarPubMed
Tan, J. H. W. (2006). Religion and social preferences: an experimental study. Economics Letters, 90, 6067.CrossRefGoogle Scholar
van Huyck, J., Gillette, A. B., & Battalio, R. (1992). Credible assignments in coordination games. Games and Economic Behavior, 4, 606–626.CrossRefGoogle Scholar
Zizzo, D. J. (2003). Verbal and behavioral learning in a probability compounding task. Theory and Decision, 54, 287–314.CrossRefGoogle Scholar
Zizzo, D. J. (2008). Experimenter demand effects in economic experiments. Social Science Research Network discussion paper, July.CrossRefGoogle Scholar