In the context of eliciting preferences for decision making under risk, we ask the question: "which might be the 'best' method for eliciting such preferences?". It is well known that different methods differ in terms of the bias in the elicitation; it is rather less well-known that different methods differ in terms of their noisiness. The optimal trade-off depends upon the relative magnitudes of these two effects. We examine four different elicitation mechanisms (pairwise choice, willingness-to-pay, willingness-to-accept, and certainty equivalents) and estimate both effects. Our results suggest that economists might be better advised to use what appears to be a relatively inefficient elicitation technique (i.e. pairwise choice) in order to avoid the bias in better-known and more widely-used techniques.