There is growing evidence that macroautophagic cargo is not limited to bulk cytosol in response to starvation and can occur selectively for substrates, including aggregated proteins. It remains unclear, however, whether starvation-induced and selective macroautophagy share identical adaptor molecules to capture their cargo. Here, we report that Alfy, a phosphatidylinositol 3-phosphate-binding protein, is central to the selective elimination of aggregated proteins. We report that the loss of Alfy inhibits the clearance of inclusions, with little to no effect on the starvation response. Alfy is recruited to intracellular inclusions and scaffolds a complex between p62(SQSTM1)-positive proteins and the autophagic effectors Atg5, Atg12, Atg16L, and LC3. Alfy overexpression leads to elimination of aggregates in an Atg5-dependent manner and, likewise, to protection in a neuronal and Drosophila model of polyglutamine toxicity. We propose that Alfy plays a key role in selective macroautophagy by bridging cargo to the molecular machinery that builds autophagosomes.