As data become more and more ubiquitous, so too do data visualizations, which increasingly circulate online and are an important means through which non-experts get access to data. This paper addresses the factors that affect how people engage with data visualizations, a relatively under-researched focus in visualization research to date. Drawing on qualitative, empirical research with users, we identify six factors that affect engagement: subject matter; source/media location; beliefs and opinions; time; emotions; and confidence and skills. In drawing attention to these factors, we bring HCI concerns together with approaches to media audience research, to identify new themes for visualization research. In particular, we argue that our findings have implications for how effectiveness is conceived and defined in relation to data visualizations and how this varies depending on how, by whom, where and for what purpose visualizations are encountered. Our paper aims to extend the horizons of visualization research, in its focus on factors that affect engagement and how these suggest new definitions of effectiveness.