Deposit systems for one-way beverage containers are widely supported by green activists and have been implemented in several countries. This paper analyzes whether such deposit systems can optimally internalize the externalities that result when consumers dump these containers. It is shown that two major problems arise in a competitive market. First, the proceeds from bottle deposits tend to reduce the price of beverages in a competitive environment and therefore lead to a departure from a first best allocation. Second, the system usually requires producers and vendors to run a system for taking back and recycling used containers, whose cost vendors can only partly shift to consumers who return their bottles. While a deposit system alone is never optimal, the paper proposes tax-deposit systems that can implement a first-best allocation.