Hostname: page-component-7b9c58cd5d-7g5wt Total loading time: 0 Render date: 2025-03-15T22:19:53.671Z Has data issue: false hasContentIssue false
  • English
  • Français

A penny for your thoughts: a survey of methods for eliciting beliefs

Published online by Cambridge University Press:  14 March 2025

Karl H. Schlag ,
James Tremewan  and
Joël J. van der Weele
Karl H. Schlag*
Affiliation:
University of Vienna, Vienna, Austria
James Tremewan*
Affiliation:
University of Vienna, Vienna, Austria
Joël J. van der Weele*
Affiliation:
Department of Economics, Center for Experimental Economics and political Decision making (CREED), University of Amsterdam, Roeterstraat 11, 1018WB Amsterdam, The Netherlands Tinbergen Institute, Amsterdam, The Netherlands Center for Financial Studies, Frankfurt, Germany
*
[email protected]
[email protected]
[email protected]; [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Incentivized methods for eliciting subjective probabilities in economic experiments present the subject with risky choices that encourage truthful reporting. We discuss the most prominent elicitation methods and their underlying assumptions, provide theoretical comparisons and give a new justification for the quadratic scoring rule. On the empirical side, we survey the performance of these elicitation methods in actual experiments, considering also practical issues of implementation such as order effects, hedging, and different ways of presenting probabilities and payment schemes to experimental subjects. We end with a discussion of the trade-offs involved in using incentives for belief elicitation and some guidelines for implementation.

Keywords

Belief elicitationSubjective beliefsScoring rulesExperimental design

JEL classification

C91: Laboratory, Individual Behavior D83: Search • Learning • Information and Knowledge • Communication • Belief • Unawareness C83: Survey Methods • Sampling Methods
Type
Original Paper
Information
Experimental Economics , Volume 18 , Issue 3 , September 2015 , pp. 457 - 490
Copyright
Copyright © 2014 Economic Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, F (1987). Discovering personal probabilities when utility functions are unknown. Management Science, 33(4), 542–544. 10.1287/mnsc.33.4.542CrossRefGoogle Scholar
Andersen, S., Fountain, J., Harrison, G. W. and Rutström, E. E. (2014). Estimating Subjective Probabilities. Journal of Risk and Uncertainty, 48(3), 207–220.CrossRefGoogle Scholar
Armantier, O, & Treich, N (2009). Subjective probabilities in games: A solution to the overbidding puzzle. International Economic Review, 50(4), 1079–1102. 10.1111/j.1468-2354.2009.00560.xCrossRefGoogle Scholar
Armantier, O, & Treich, N (2013). Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging. European Economic Review, 62, 17–40. 10.1016/j.euroecorev.2013.03.008CrossRefGoogle Scholar
Artinger, F., Exadaktylos, F., Koppel, H., & Sääksvuori, L. (2010). Applying Quadratic Scoring Rule transparently in multiple choice settings: a note. Technical report, Jena Economic Research Paper.Google Scholar
Beach, L, & Phillips, L (1967). Subjective probabilities inferred from estimates and bets. Journal of Experimental Psychology, 75(3), 354–359. 10.1037/h0025061CrossRefGoogle ScholarPubMed
Beach, LR, & Wise, JA (1969). Subjective probability revision and subsequent decisions. Journal of Experimental Psychology, 81(3), 561–565. 10.1037/h0027919CrossRefGoogle Scholar
Becker, G, DeGroot, M, & Marschak, J (1964). Measuring utility by a single-response sequential method. Behavioral Science, 9(3), 226 10.1002/bs.3830090304CrossRefGoogle ScholarPubMed
Berg, JE, Rietz, TA, & Dickhaut, JW (2008). On the performance of the lottery procedure for controlling risk preferences. Handbook of Experimental Economics Results, 1, 1087–1097. 10.1016/S1574-0722(07)00115-1CrossRefGoogle Scholar
Blanco, M, Engelmann, D, Koch, AK, & Normann, H-T (2010). Belief elicitation in experiments: Is there a hedging problem?. Experimental Economics, 13(4), 412–438. 10.1007/s10683-010-9249-1CrossRefGoogle Scholar
Brier, GW (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;22.0.CO;2>CrossRefGoogle Scholar
Cervera, JL, Muñoz, J Berger, JO, Dawid, AP, & Smith, AFM (1996). Proper scoring rules for fractiles. Bayesian statistics 5, Oxford: Oxford University Press 513–519.CrossRefGoogle Scholar
Costa-Gomes, M. A., Huck, S. and Weizsacker, G. (2012). Beliefs and actions in the trust game: creating instrumental variables to estimate the causal effect. WZB Discussion Paper, 2012–302.Google Scholar
Croson, RTA (2000). Thinking like a game theorist: Factors affecting the frequency of equilibrium play. Journal of Economic Behavior & Organization, 41(3), 299–314. 10.1016/S0167-2681(99)00078-5CrossRefGoogle Scholar
Dawes, RM, McTavish, J, & Shaklee, H (1977). Behavior, communication, and assumptions about other people’s behavior in a commons dilemma situation. Journal of Personality and Social Psychology, 35(1), 1 10.1037/0022-3514.35.1.1CrossRefGoogle Scholar
De Finetti, B (1965). Methods for discriminating levels of partial knowledge concerning a test item. British Journal of Mathematical and Statistical Psychology, 18(1), 87–123. 10.1111/j.2044-8317.1965.tb00695.xCrossRefGoogle Scholar
De Finetti, B (1970). Logical foundations and measurement of subjective probability. Acta Psychologica, 34, 129–145. 10.1016/0001-6918(70)90012-0CrossRefGoogle Scholar
De Finetti, B (1974). Theory of probability, New York: Wiley.Google Scholar
Delavande, A, Giné, X, & McKenzie, D (2011). Measuring subjective expectations in developing countries: A critical review and new evidence. Journal of Development Economics, 94(2), 151–163. 10.1016/j.jdeveco.2010.01.008CrossRefGoogle Scholar
Ducharme, W, & Donnell, M (1973). Intrasubject comparison of four response modes for “subjective probability” assessment. Organizational Behavior and Human Performance, 10, 108–117. 10.1016/0030-5073(73)90007-XCrossRefGoogle Scholar
Engelmann, D, & Strobel, M (2000). The false consensus effect disappears if representative information and monetary incentives are given. Experimental Economics, 260(2000), 241–260. 10.1023/A:1011472501737CrossRefGoogle Scholar
Erev, I, Bornstein, G, & Wallsten, T (1993). The negative effect of probability assessments on decision quality. Organizational Behavior and Human Decision Processes, 55, 78–94. 10.1006/obhd.1993.1025CrossRefGoogle Scholar
Fischer, GW (1982). Scoring-rule feedback and the overconfidence syndrome in subjective probability forecasting. Organizational Behavior and Human Performance, 29(3), 352–369. 10.1016/0030-5073(82)90250-1CrossRefGoogle Scholar
Fountain, J, & Harrison, GW (2011). What do prediction markets predict?. Applied Economics Letters, 18(3), 267–272. 10.1080/13504850903559575CrossRefGoogle Scholar
Friedman, D (1983). Effective scoring rules for probabilistic forecasts. Management Science, 29(4), 447–454. 10.1287/mnsc.29.4.447CrossRefGoogle Scholar
Gächter, S, & Renner, E (2010). The effects of (incentivized) belief elicitation in public goods experiments. Experimental Economics, 13(3), 364–377. 10.1007/s10683-010-9246-4CrossRefGoogle Scholar
Garthwaite, PH, Kadane, JB, & O’Hagan, A (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100(470), 680–701. 10.1198/016214505000000105CrossRefGoogle Scholar
Gigerenzer, G, & Hoffrage, U (1995). How to improve bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684–704. 10.1037/0033-295X.102.4.684CrossRefGoogle Scholar
Gneiting, T, & Raftery, AE (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477), 359–378. 10.1198/016214506000001437CrossRefGoogle Scholar
Good, I (1952). Rational decisions. Journal of the Royal Statistical Society. Series B, 14(1), 107–114.CrossRefGoogle Scholar
Grether, D. (1981). Financial incentive effects and individual decision-making. California Institute of Technology, Working Paper 401.Google Scholar
Guerra, G, & Zizzo, DJ (2004). Trust responsiveness and beliefs. Journal of Economic Behavior & Organization, 55(1), 25–30. 10.1016/j.jebo.2003.03.003CrossRefGoogle Scholar
Hao, L, & Houser, D (2012). Belief elicitation in the presence of novice participants: An experimental study. Journal of Risk and Uncertainty, 2, 161–180. 10.1007/s11166-011-9133-1CrossRefGoogle Scholar
Harrison, GW, Martínez-Correa, J, & Swarthout, J (2014). Eliciting subjective probabilities with binary lotteries. Journal of Economic Behavior and Organization, 101, 128–140. 10.1016/j.jebo.2014.02.011CrossRefGoogle Scholar
Harrison, G. W., Martínez-correa, J., Swarthout, J. T., & Ulm, E. R. (2013a). Scoring rules for subjective probability distributions. Manuscript, Georgia State University.Google Scholar
Harrison, GW, Martinez-Correa, J, & Swarthout, T (2013). Inducing risk neutral preferences with binary lotteries: A reconsideration. Journal of Economic Behavior and Organization, 94, 145–159. 10.1016/j.jebo.2012.09.008CrossRefGoogle Scholar
Harrison, GW, & Rutström, EE (2009). Expected utility theory and prospect theory: One wedding and a decent funeral. Experimental Economics, 12(2), 133–158. 10.1007/s10683-008-9203-7CrossRefGoogle Scholar
Heinemann, F, Nagel, R, & Ockenfels, P (2009). Measuring strategic uncertainty in coordination games. Review of Economic Studies, 76, 181–221. 10.1111/j.1467-937X.2008.00512.xCrossRefGoogle Scholar
Hoffmann, T. (2013). The Effect of Belief Elicitation on Game Play. Manuscript, Mannheim University.Google Scholar
Hollard, G., Massoni, S. and Vergnaud, J. (2010). Subjective beliefs formation and elicitation rules: Experimental evidence. Centre d’Economie de la Sorbonne Working Paper, 2010.88.Google Scholar
Holt, C (2006). Markets, games and strategic behavior, Boston: Pearson/Addison-Wesley.Google Scholar
Holt, CA, & Laury, S (2002). Risk aversion and incentive effects. The American Economic Review, 92(5), 1644 10.1257/000282802762024700CrossRefGoogle Scholar
Hossain, T, & Okui, R (2013). The binarized scoring rule. The Review of Economic Studies, 80(3), 984–1001. 10.1093/restud/rdt006CrossRefGoogle Scholar
Huck, S, & Weizsäcker, G (2002). Do players correctly estimate what others do?: Evidence of conservatism in beliefs. Journal of Economic Behavior & Organization, 47(1), 71–85. 10.1016/S0167-2681(01)00170-6CrossRefGoogle Scholar
Hurley, T, & Shogren, J (2005). An experimental comparison of induced and elicited beliefs. Journal of Risk and Uncertainty, 30(2), 169–188. 10.1007/s11166-005-6565-5CrossRefGoogle Scholar
Hurley, T. T. M., Peterson, N., & Shogren, J. J. F. (2007). Belief elicitation: An experimental comparison of scoring rule and prediction methods. Manuscript, University of Minnesota.Google Scholar
Jaffray, J, & Karni, E (1999). Elicitation of subjective probabilities when the initial endowment is unobservable. Journal of Risk and Uncertainty, 8, 5–20. 10.1023/A:1007706009628CrossRefGoogle Scholar
Jenkinson, D. (2005). The elicitation of probabilities—A review of the statistical literature. Manuscript, University of Sheffield.Google Scholar
Jensen, FA, & Peterson, CR (1973). Psychological effects of proper scoring rules. Organizational Behavior and Human Performance, 9(2), 307–317. 10.1016/0030-5073(73)90054-8CrossRefGoogle Scholar
Jose, VRR, & Winkler, RL (2009). Evaluating quantile assessments. Operations Research, 57(5), 1287–1297. 10.1287/opre.1080.0665CrossRefGoogle Scholar
Kadane, J, & Winkler, R (1988). Separating probability elicitation from utilities. Journal of the American Statistical Association, 83(402), 357–363. 10.1080/01621459.1988.10478605CrossRefGoogle Scholar
Karni, E (2009). A mechanism for eliciting probabilities. Econometrica, 77(2), 603–606. 10.3982/ECTA7833Google Scholar
Karni, E, & Safra, Z (1995). The impossibility of experimental elicitation of subjective probabilities. Theory and Decision, 38, 313–320. 10.1007/BF01362238CrossRefGoogle Scholar
Koessler, F, Noussair, C, & Ziegelmeyer, A (2012). Information aggregation and belief elicitation in experimental parimutuel betting markets. Journal of Economic Behavior & Organization, 83(2), 195–208. 10.1016/j.jebo.2012.02.017CrossRefGoogle Scholar
Kothiyal, A, Spinu, V, & Wakker, P (2011). Comonotonic proper scoring rules to measure ambiguity and subjective beliefs. Journal of Multi-Criteria Decision Analysis, 101, 101–113.Google Scholar
Krawczyk, M. (2011). Overconfident for real? Proper scoring for confidence intervals. Manuscript, University of Warsaw.Google Scholar
Lambert, N, Pennock, D, & Shoham, Y (2008). Eliciting properties of probability distributions: The highlights. ACM SIGecom Exchanges, 7(3), 1–5. 10.1145/1486877.1486886CrossRefGoogle Scholar
Lindley, DV, Tversky, A, & Brown, RV (1979). On the reconciliation of probability assessments. Journal of the Royal Statistical Society. Series A (General), 142(2), 146 10.2307/2345078CrossRefGoogle Scholar
Lipkus, I, Samsa, G, & Rimer, B (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 37–44. 10.1177/0272989X0102100105CrossRefGoogle ScholarPubMed
Machina, MJ, & Schmeidler, D (1992). A more robust definition of subjective probability. Econometrica, 60(4), 745–780. 10.2307/2951565CrossRefGoogle Scholar
Manski, C (2002). Identification of decision rules in experiments on simple games of proposal and response. European Economic Review, 46, 880–891. 10.1016/S0014-2921(01)00222-7CrossRefGoogle Scholar
Manski, C (2004). Measuring expectations. Econometrica, 72(5), 1329–1376. 10.1111/j.1468-0262.2004.00537.xCrossRefGoogle Scholar
Manski, CF (2006). Interpreting the predictions of prediction markets. Economics Letters, 91(3), 425–429. 10.1016/j.econlet.2006.01.004CrossRefGoogle Scholar
Matheson, J, & Winkler, R (1976). Scoring rules for continuous probability distributions. Management Science, 22(10), 1087–1096. 10.1287/mnsc.22.10.1087CrossRefGoogle Scholar
McCarthy, J (1956). Measures of the value of information. Proceedings of the National Academy of Sciences of the United States of America, 42(9), 654–655. 10.1073/pnas.42.9.654CrossRefGoogle ScholarPubMed
McKelvey, R, & Page, T (1990). Public and private information: An experimental study of information pooling. Econometrica, 58(6), 1321–1339. 10.2307/2938318CrossRefGoogle Scholar
Miller, N, Resnick, P, & Zeckhauser, R (2005). Eliciting informative feedback: The peer-prediction method. Management Science, 51(9), 1359–1373. 10.1287/mnsc.1050.0379CrossRefGoogle Scholar
Nyarko, Y, & Schotter, A (2002). An experimental study of belief learning using elicited beliefs. Econometrica, 70(3), 971–1005. 10.1111/1468-0262.00316CrossRefGoogle Scholar
Offerman, T., & Palley, A. B. (2013). Lossed in Translation: An Off-the-Shelf Method to Recover Probabilistic Beliefs from Loss-Averse Agents. Manuscript, University of Amsterdam.CrossRefGoogle Scholar
Offerman, T, Sonnemans, J, & Schram, A (1996). Value orientations, expectations and voluntary contributions in public goods. The Economic Journal, 106(437), 817–845. 10.2307/2235360CrossRefGoogle Scholar
Offerman, T, Sonnemans, J, Van de Kuilen, G, & Wakker, PP (2009). A truth serum for non-bayesians. Review of Economic Studies, 76(4), 1461–1489. 10.1111/j.1467-937X.2009.00557.xCrossRefGoogle Scholar
Palfrey, TR, & Wang, SW (2009). On eliciting beliefs in strategic games. Journal of Economic Behavior & Organization, 71(2), 98–109. 10.1016/j.jebo.2009.03.025CrossRefGoogle Scholar
Phillips, LD, & Edwards, W (1966). Conservatism in a simple probability inference task. Journal of Experimental Psychology, 72(3), 346–354. 10.1037/h0023653CrossRefGoogle Scholar
Prelec, D (2004). A bayesian truth serum for subjective data. Science, 306, 462–466. 10.1126/science.1102081CrossRefGoogle ScholarPubMed
Price, P (1998). Effects of a relative-frequency elicitation question on likelihood judgment accuracy: The case of external correspondence. Organizational Behavior and Human Decision Processes, 76(3), 277–297. 10.1006/obhd.1998.2807CrossRefGoogle ScholarPubMed
Ramsey, F. (1926). Truth and Probability. In Braithwaite, R. B. (Ed.), The foundations of mathematics and other logical essays (pp. 156–198). New York (1931): Harcourt.Google Scholar
Roby, T. B. (1964). Belief states: A preliminary empirical study. Technical Documentary Report, Decision Sciences Laboratory.CrossRefGoogle Scholar
Rutström, EE, & Wilcox, NT (2009). Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test. Games and Economic Behavior, 67(2), 616–632. 10.1016/j.geb.2009.04.001CrossRefGoogle Scholar
Savage, LJ (1954). The foundation of statistics, New York: Wiley.Google Scholar
Savage, LJ (1971). Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336), 783–801. 10.1080/01621459.1971.10482346CrossRefGoogle Scholar
Schervish, M (1989). A general method for comparing probability assessors. The Annals of Statistics, 17(4), 1856–1879. 10.1214/aos/1176347398CrossRefGoogle Scholar
Schlag, K. H. and Tremewan, J. (2014). Simple belief elicitation. SSRN Working paper 2449224.CrossRefGoogle Scholar
Schlag, K. H., & van der Weele, J. J. (2012). Incentives for interval elicitation. Manuscript, Vienna University.Google Scholar
Schlag, KH, & van der Weele, JJ (2013). Eliciting probabilities, means, medians, variances and covariances without assuming risk neutrality. Theoretical Economics Letters, 03(1), 38–42. 10.4236/tel.2013.31006CrossRefGoogle Scholar
Schmalensee, R (1976). An experimental study of expectation formation. Econometrica, 44(1), 17–41. 10.2307/1911378CrossRefGoogle Scholar
Schotter, A, & Trevino, I (2014). Belief elicitation in the laboratory. Annual Review of Economics, 6, 103–128. 10.1146/annurev-economics-080213-040927CrossRefGoogle Scholar
Schum, DA, Goldstein, IL, Howell, WC, & Southard, JF (1967). Subjective probability revisions under several cost-payoff arrangements. Organizational Behavior and Human Performance, 2, 84–104. 10.1016/0030-5073(67)90012-8CrossRefGoogle Scholar
Seghers, R. C., Fryback, D. G., & Goodman, B. C. (1973). Relative variance preferences in a choice-among-bets paradigm. Technical report, DTIC Document.Google Scholar
Selten, R (1998). Axiomatic characterization of the quadratic scoring rule. Experimental Economics, 62, 43–62. 10.1023/A:1009957816843CrossRefGoogle Scholar
Selten, R, Sadrieh, A, & Abbink, K (1999). Money does not induce risk neutral behavior, but binary lotteries do even worse. Theory and Decision, 46, 211–249. 10.1023/A:1005038628305CrossRefGoogle Scholar
Shuford, E, Albert, A, & Massengill, HE (1966). Admissible probability measurement procedures. Psychometrika, 31(2), 125–145. 10.1007/BF02289503CrossRefGoogle ScholarPubMed
Smith, A (2013). Estimating the causal effect of beliefs on contributions in repeated public good games. Experimental Economics, 16(3), 414–425. 10.1007/s10683-012-9345-5CrossRefGoogle Scholar
Smith, C (1961). Consistency in statistical inference and decision. Journal of the Royal Statistical Society. Series B, 23(1), 1–37.CrossRefGoogle Scholar
Sonnemans, J., & Offerman, T. T. T. (2001). Is the quadratic scoring rule behaviorally incentive compatible?. Manuscript, University of Amsterdam.Google Scholar
Staël von Holstein, C-AS (1970). Measurement of subjective probability. Acta Psychologica, 34, 146–159. 10.1016/0001-6918(70)90013-2CrossRefGoogle Scholar
Toda, M (1951). Measurement of intuitive probability by a method of game. Japanese Journal of Psychology, 22, 29–40. 10.4992/jjpsy.22.29Google Scholar
Toda, M. (1963). Measurement of subjective probability distribution. Report No. 3(3), State College, Pennsylvania, Institute for Research, Division of Mathematical Psychology.Google Scholar
Trautmann, S. T. and van de Kuilen, G. (2014). Belief elicitation: A horse race among truth serums. The Economic Journal. doi:10.1111/ecoj.12160.CrossRefGoogle Scholar
Tversky, A, & Koehler, DJ (1994). Support theory: A nonextensional representation of subjective probability. Psychological Review, 101(4), 547 10.1037/0033-295X.101.4.547CrossRefGoogle Scholar
Vlek, C (1973). The fair betting game as an admissible procedure for assessment of subjective probabilities. British Journal of Mathematical and Statistical Psychology, 26(1), 18–30. 10.1111/j.2044-8317.1973.tb00503.xCrossRefGoogle Scholar
Vlek, CAJC (1973). Coherence of human judgment in a limited probabilistic environment. Organizational Behavior and Human Performance, 9(460–481), 460–481. 10.1016/0030-5073(73)90065-2CrossRefGoogle Scholar
Wang, H, Dash, D, & Druzdzel, MJ (2002). A method for evaluating elicitation schemes for probabilistic models. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 32(1), 38–43. 10.1109/3477.979958CrossRefGoogle ScholarPubMed
Wang, SW (2011). Incentive effects: The case of belief elicitation from individuals in groups. Economics Letters, 111(1), 30–33. 10.1016/j.econlet.2010.11.045CrossRefGoogle Scholar
Whitcomb, KM, Önkal, D, Benson, PG, & Curley, SP (1993). An evaluation of the reliability of probability judgments across response modes and over time. Journal of Behavioral Decision Making, 6(4), 283–296. 10.1002/bdm.3960060406CrossRefGoogle Scholar
Wilcox, N. T. and Feltovich, N. (2000). Thinking like a game theorist: Comment. University of Houston Department of Economics working paper.Google Scholar
Winkler, R (1996). Scoring rules and the evaluation of probabilities. Test, 5(1), 1–60. 10.1007/BF02562681CrossRefGoogle Scholar
Winkler, R, & Murphy, A (1968). ”Good” probability assessors. Journal of Applied Meteorology, 7, 751 10.1175/1520-0450(1968)007<0751:PA>2.0.CO;22.0.CO;2>CrossRefGoogle Scholar
Winkler, R, & Murphy, A (1970). Nonlinear utility and the probability score. Journal of Applied Meteorology, 9, 143–148. 10.1175/1520-0450(1970)009<0143:NUATPS>2.0.CO;22.0.CO;2>CrossRefGoogle Scholar
Winkler, R, & Murphy, A (1979). The use of probabilities in forecasts of maximum and minimum temperatures. The Meteorological Magazine, 108(1288), 317–329.Google Scholar
Wolfers, J. and Zitzewitz, E. (2006). Interpreting prediction market prices as probabilities. NBER Working Paper 12200.CrossRefGoogle Scholar
Yaniv, I, & Foster, D (1995). Graininess of judgement under uncertainty: An accuracy-informativeness trade-off. Journal of Experimental Psychology: General, 124(4), 424–432. 10.1037/0096-3445.124.4.424CrossRefGoogle Scholar
Yaniv, I, & Foster, D (1997). Precision and accuracy of judgmental estimation. Journal of Behavioral Decision Making, 10, 21–32. 10.1002/(SICI)1099-0771(199703)10:1<21::AID-BDM243>3.0.CO;2-G3.0.CO;2-G>CrossRefGoogle Scholar
Zizzo, DJ (2009). Experimenter demand effects in economic experiments. Experimental Economics, 13(1), 75–98. 10.1007/s10683-009-9230-zCrossRefGoogle Scholar