The Integrated Stress Response (ISR) helps metazoan cells adapt to cellular stress by limiting the availability of initiator methionyl-tRNA for translation. Such limiting conditions paradoxically stimulate the translation of ATF4 mRNA through a regulatory 5' leader sequence with multiple upstream Open Reading Frames (uORFs), thereby activating stress-responsive gene expression. Here, we report the identification of two critical regulators of such ATF4 induction, the noncanonical initiation factors eIF2D and DENR. Loss of eIF2D and DENR in Drosophila results in increased vulnerability to amino acid deprivation, susceptibility to retinal degeneration caused by endoplasmic reticulum (ER) stress, and developmental defects similar to ATF4 mutants. eIF2D requires its RNA-binding motif for regulation of 5' leader-mediated ATF4 translation. Consistently, eIF2D and DENR deficient human cells show impaired ATF4 protein induction in response to ER stress. Altogether, our findings indicate that eIF2D and DENR are critical mediators of ATF4 translational induction and stress responses in vivo.