One constant refrain in evaluations and reviews of decentralization is that the results are mixed. But given that decentralization is a complex intervention or phenomenon, what is more important is to generate evidence to inform implementation strategies. We therefore synthesized evidence from the literature to understand why, how and under what circumstances decentralization influences health system equity, efficiency and resilience. In doing this, we adopted the realist approach to evidence synthesis and included quantitative and qualitative studies in high-, low- and middle-income countries that assessed the the impact of decentralization on health systems. We searched the Medline and Embase databases via Ovid, and the Cochrane library of systematic reviews and included 51 studies with data from 25 countries. We identified three mechanisms through which decentralization impacts on health system equity, efficiency and resilience: 'Voting with feet' (reflecting how decentralization either exacerbates or assuages the existing patterns of inequities in the distribution of people, resources and outcomes in a jurisdiction); 'Close to ground' (reflecting how bringing governance closer to the people allows for use of local initiative, information, feedback, input and control); and 'Watching the watchers' (reflecting mutual accountability and support relations between multiple centres of governance which are multiplied by decentralization, involving governments at different levels and also community health committees and health boards). We also identified institutional, socio-economic and geographic contextual factors that influence each of these mechanisms. By moving beyond findings that the effects of decentralization on health systems and outcomes are mixed, this review presents mechanisms and contextual factors to which policymakers and implementers need to pay attention in their efforts to maximize the positive and minimize the negative impact of decentralized governance.