The end of the milk quota system in Europe has resulted in substantial structural changes to the dairy industry, calling its future into question, especially in mountainous areas. This study analyzes the internal and contextual factors that influence dairy restructuring in French mountainous areas. Three complementary logit models are used to specify farmer decisions (e.g., dairy production exit, stability, and growth). Original and exhaustive geo-located administrative datasets are combined to create a farm longitudinal dataset and characterize farm-restructuring patterns on a fine geographical scale. Farm size, corporate legal status, and specialization are positively associated with farm growth. Viable diversification strategies through either farm processing or agro-environmental scheme contracting are also highlighted. Farm sustainability is supported by the economic success of the local industry and the propensity to adopt extensive practices. With respect to public policy, dairy policy appears to have the strongest impact. Thus, the liberalization of dairy policy threatens the future of dairy farming, particularly in areas with low milk density and no production under labels. This work calls for a policy that promotes the collective dimension of dairy farm performance. The current rural development policy alone may be insufficient to support farms’ long-term sustainability.