How adipocytes contribute to the physiological control of stem cells is a critical question towards understanding the link between obesity and multiple diseases, including cancers. Previous studies have revealed that adult stem cells are influenced by whole-body physiology through multiple diet-dependent factors. For example, nutrient-dependent pathways acting within the Drosophila ovary control the number and proliferation of germline stem cells (GSCs). The potential role of nutrient sensing by adipocytes in modulating stem cells in other organs, however, remains largely unexplored. Here, we report that amino acid sensing by adult adipocytes specifically modulates the maintenance of GSCs through a Target of Rapamycin-independent mechanism. Instead, reduced amino acid levels and the consequent increase in uncoupled tRNAs trigger activation of the GCN2-dependent amino acid response pathway within adipocytes, causing increased rates of GSC loss. These studies reveal a new step in adipocyte-stem cell crosstalk.