The power and ease of Drosophila genetics and the medical relevance of mosquito-transmitted viruses have made dipterans important model organisms in antiviral immunology. Studies of virus-host interactions at the molecular and population levels have illuminated determinants of resistance to virus infection. Here, we review the sources and nature of variation in antiviral immunity and virus susceptibility in model dipteran insects, specifically the fruit fly Drosophila melanogaster and vector mosquitoes of the genera Aedes and Culex. We first discuss antiviral immune mechanisms and describe the virus-specificity of these responses. In the following sections, we review genetic and microbiota-dependent variation in antiviral immunity. In the final sections, we explore less well-studied sources of variation, including abiotic factors, sexual dimorphism, infection history, and endogenous viral elements. We borrow from work on other pathogen types and non-dipteran species when it parallels or complements studies in dipterans. Understanding natural variation in virus-host interactions may lead to the identification of novel restriction factors and immune mechanisms and shed light on the molecular determinants of vector competence.