This paper combines local temperature treatment effects with a quantitative macroeconomic model to assess the potential for global reallocation between agricultural and non-agricultural production to reduce the costs of climate change. First, I use firm-level panel data from a wide range of countries to show that extreme heat reduces productivity less in manufacturing and services than in agriculture, implying that hot countries could achieve large potential gains through adapting to global warming by shifting labor toward manufacturing and increasing imports of food. To investigate the likelihood that such gains will be realized, I embed the estimated productivity effects in a model of sectoral specialization and trade covering 158 countries. Simulations suggest that climate change does little to alter the geography of agricultural production, however, as high trade barriers in developing countries temper the influence of shifting comparative advantage. Instead, climate change accentuates the existing pattern, known as “the food problem,” in which poor countries specialize heavily in relatively low productivity agricultural sectors to meet subsistence consumer needs. The productivity effects of climate change reduce welfare by 6-10% for the poorest quartile of the world with trade barriers held at current levels, but by nearly 70% less in an alternative policy counterfactual that moves low-income countries to OECD levels of trade openness.