Many prior studies have identified that subjects in experiments demonstrate preferences for fair allocations. We present an experimental study designed to test whether a similar concern for fairness manifests itself when the decision maker is choosing among differing probabilistic allocation mechanisms that will all generate an ex post unfair allocation by assigning an indivisible prize to one individual. This investigation is inspired by Karni and Safra (Econometrica, 70, 263-284, 2002) in which a structure for preferences for fairness in such an environment was developed. Here we use this model to design experiments that allow us to test for the presence of concern for fairness in individual choice behavior and examine some factors that may affect the intensity of the concern for fairness.