In this paper, we show the importance of accounting for heterogeneity among sample firms in stochastic frontier analysis. For a fairly homogenous sample of German savings and cooperative banks, we analyze how alternative theoretical assumptions regarding the nature of heterogeneity can be modeled and the extent to which the respective empirical specifications affect estimated efficiency levels and rankings. We find that the level of efficiency scores is affected in the case of both cost and profitmodels. On the cost side especially, level and rank correlations show that different specifications identify different banks as being best or worst performers. Our main conclusion is that efficiency studies in general and bank efficiency studies in particular should account for heterogeneity across sample firms. Especially when efficiency measures are employed for policy purposes, a careful choice of models and transparency regarding maximization methods are essential to be able to make inferences about managerial behavior.