Category Archives: event data

A Pretty Big Omission: More on Event Data

In my recent post on coming round to the contribution of quant research in IR, I mentioned that I was interested to see what emerges out of the ICEWS event data system. But despite my falling scepticism towards quant approaches, the critical theory I imbibed over the years made me roll my eyes pretty hard when I read that the ICEWS project doesn’t collect data on the US ‘by design’. That’s what you get when your research project is funded by Lockheed Martin I guess: data-collection driven by the need to have information about the scary outside world. It strikes me that missing data for the US is going to create some problems for those that want to use the data-base to analyse the relationship between protest and democracy, economic development or a host of other factors.

This seems like a variant of sampling on the dependent variable: gathering data from the rest of the world, as the rest of the world is where threats to the US (are presumed to) originate from. I’m not sure whether the database includes information on US domestic terrorism, but if it doesn’t then that’s a huge omission – especially as some studies find a positive correlation between economic development and terror (both origin and targeting).

Changing My Mind III: Quantitative Research Doesn’t Have Much to Add to IR

For the first few years I studied international relations I was of the view that quantitative approaches didn’t have a great deal to add to the subject. This view was not, I should note, based on a general antipathy towards the natural sciences or a fear of numbers – both of which are common in the social sciences and the humanities. Instead, I thought that the most important issues in international relations were theoretical and related to issues in philosophy of social science. It also seemed to me that the quantitative research that I was aware often missed the point of the issue it was intending to address, operationalising concepts in a manner that was unconvincing and thus reducing fairly subtle processes to crude measures. It also seemed to be atheoretical and involved throwing a load of different variables into the hopper to see what would come out the other side.

I realised, however, that many of the issues that I was interested in could not be resolved through theory alone. It might seem strange that I ever believed (implicitly) that they could, but a lot of debate in IR and the humanities more broadly seems to involve the evaluation of a set of claims based on a set of theoretical desiderata alone. This is a rut, and its a rut that a lot of areas of scholarship fall into. Engagement with philosophy of science and social theory is important, but it can’t adjudicate between competing empirical claims about topics such as inequality, conflict, democratisation and state formation.

Reading more statistical research, I realised that there was much more in the way of theoretically sophisticated, historically informed quant research in IR than I had credited. I started to take quantitative contributions much more seriously, but interrogate them in more detail. Every statistical model makes theoretical assumptions and those assumptions can be questioned and problematised just like those of any other theoretical claim. Getting into the nuts and bolts of quantitative research on topics ranging from economic growth and inequality, to democratisation and economic development, to strategic rivalries between states gave me a more informed understanding of the contributions and limits of quantitative research in world politics. The most significant problems are that first, the entities social science works with are not stable over historical time, the social world is never really in equilibrium, and so the relationships between variables cannot be expected to be constant across time and space. The second problem is the issue of causal complexity, the way in which causal factors may interact and combine in specific configurtions to produce certain outcomes. The third problem is the lack of reliable data and measures based on well-operationalised concepts.

Many quantitative scholars and methodologists acknowledge these problems, however, and have attempted to devise ways to address these tricky issues. Braumoeller and Wimmer come to mind scholars who have made major contributions to world politics recently through the intelligent, theoretically informed use of quantitative methods. Paul Schrodt has made some searing criticisms of status quo quantitative practice in the study of politics and attempted to push analytical techniques forward in the discipline. His contributions to efforts to make event data useful for research in politics and international relations are very interesting, and I’ll be watching out for what he and others do with the huge new ICEWS database. It’s possible that with data at this level of granuality, quantitative scholarship can move beyond a focus on broad structural correlates of outcomes and towards a greater focus on political processes. Of course, there are pitfalls and obstacles, but I’m more optimistic about the prospects than I was before I really started to engage with this area of research.