Scribble (Scrib) module proteins are major regulators of cell polarity, but how they influence membrane traffic is not known. Endocytosis is also a key regulator of polarity through roles that remain unclear. Here we link Scrib to a specific arm of the endocytic trafficking system. Drosophila mutants that block AP-2-dependent endocytosis share many phenotypes with Scrib module mutants, but Scrib module mutants show intact internalization and endolysosomal transport. However, defective traffic of retromer pathway cargo is seen, and retromer components show strong genetic interactions with the Scrib module. The Scrib module is required for proper retromer localization to endosomes and promotes appropriate cargo sorting into the retromer pathway via both aPKC-dependent and -independent mechanisms. We propose that the Scrib module regulates epithelial polarity by influencing endocytic itineraries of Crumbs and other retromer-dependent cargo.