Climate change has increased rainfall uncertainty, leading to greater production risks in agriculture. We examine the gender-differentiated labor impacts of droughts using unique individual-level panel data for agricultural households in India over half a decade. Accounting for unobserved heterogeneity across individuals, we find that women’s workdays are 19% lower than men’s when a drought occurs, driven by the former’s lack of diversification to the non-farm sector. Women are less likely to work outside their village and migrate relative to men in response to droughts and are consequently unable to cope fully with the adverse agricultural productivity shock. We find suggestive evidence in support of social costs emanating from gender norms that constrain women’s access to non-farm work opportunities. The results highlight the gendered impact of climate shocks, potentially exacerbating extant gender gaps in the labor market.