Policy-making in the Council: the need for interviews

Researchers have typically aimed to classify three factors when analyzing policy-making: a. the policy positions of the main actors (inputs), b. the details of the negotiations (processes), and c. the resolution of controversies through decision outcomes, i.e., outputs (Thomson et al., Reference Thomson, Arregui, Leuffen, Costello, Cross, Hertz and Jensen2012). We contend that interview-based data has become necessary because the other types discussed below usually fail to cover all three simultaneously. This is particularly true for the Council, given its secretive nature and public consensus norms. Indeed, it is no coincidence that the methods we discuss here have found extensive application in foreign policy analysis, a field with similar diplomatic customs (e.g., Bueno de Mesquita, Reference Bueno de Mesquita2009).Footnote ¹ While interviews are a key source of information in legislative studies, too (and more generally in the social sciences, e.g., Mosley, Reference Mosley2013; Rubin, Reference Rubin2021), they are typically employed as a source of qualitative rather than quantitative data.

Indeed, scholars have also utilized ‘more traditional’ types of data to explain policy-making in the institution: formal outputs (typically voting records and/or official statements) and official documents. The former have seen a proliferation of usage since the early 2000s, particularly aimed at identifying common patterns of alliance between member states (e.g., Mattila and Lane, Reference Mattila and Lane2001) or determining the predictors of their behavior (e.g., Hosli, et al., Reference Hosli, Mattila and Uriot2011). One consistent trend across all studies has been the documentation of a ‘culture of consensus’ in the Council, where making disagreement public is taboo (e.g., Heisenberg, Reference Heisenberg2005). Therefore, opposing votes are relatively rare, and may significantly underestimate actual instances of disagreement (e.g., Arregui, Reference Arregui2008).

Official documents, on the other hand, have primarily been utilized to analyze the Council’s procedural dimension. This body of literature has focused on determining the predictors of the EU’s legislative speed and efficiency (e.g., König, Reference König2007), or the intensity of its agenda (e.g., Toshkov, Reference Toshkov2011). Other contributions have examined the agenda-setting power of the Presidency (Vaznonytė, Reference Vaznonytė2020) or the predictors of ministerial involvement in legislative dossiers (e.g., Häge, Reference Häge, Naurin and Rasmussen2012). While these examples are just a subset of many, they demonstrate that this literature has concerned itself with processes rather than outcomes. In contrast, the aforementioned studies using formal outputs usually model said outcomes, and to a lesser extent, inputs. Thus, both approaches only offer partial perspectives, and expert interviewsFootnote ² have emerged as crucial for capturing data about the whole legislative process.Footnote ³

This method’s main advantages lie in its flexibility. Researchers can tailor their questions to gather precise data, unconstrained by what the EU institutions publish in official documents. Interviews are thus a very direct method of measuring ‘real’ preferences, relatively unaffected by the pressures that may affect final votes (Bailer, Reference Bailer, Martin and Saalfeld2014). Additionally, interactions between academics and policymakers can yield beneficial insights and inspire new research questions. Finally, even if aimed at the collection of systematic quantitative data, these interviews almost inevitably generate valuable qualitative evidence, which can help explain the underlying mechanisms behind the correlations under study (see e.g., Thomson et al., Reference Thomson, Stokman, Achen and König2006, for various examples). However, expert assessment has drawbacks, too. It is costly and time-consuming, requiring extensive stays in Brussels. Accessing experts can be difficult and raises ethical questions, as they are public sector workers and their time is funded by public money. Scholars must therefore ensure that interviews are well-targeted and conducted only when necessary, thoroughly preparing with publically available information before meeting experts. Most notably, these issues pose serious challenges to replicability, further complicated by ethical concerns around sharing interview transcripts (see Kern and Mustasilta, Reference Kern and Mustasilta2023).

Furthermore, interview data is subject to individual bias, stemming from both interviewers and interviewees. This is absent in formal outputs (which have no interpretative component) and minimized in document analysis (though hand-coding by researchers is sometimes employed). Scholars have gone to great lengths to mitigate this risk, through extensive triangulation with documentary evidence and additional interviewees. Despite these remarkable efforts, though, obtaining confidence measures for interview data remains impossible. These challenges notwithstanding, interview-based quantitative data enable scholars to pursue a breadth of research agendas that is simply unthinkable using other sources, all the while systematizing a literature that has traditionally suffered from an over-reliance on case studies. In particular, said data enable researchers to quantify the distances between actors’ preferences, compare them to legislative outcomes, or compute the intensity of relationship networks, thus measuring important concepts like bargaining success and network capital. This opens up new research avenues by allowing for the testing of said concepts’ predictors, which other sources struggle to replicate due to the aforementioned flaws. Moreover, these datasets provide access to information unavailable elsewhere, leading to unprecedented insights into the explanatory mechanisms underlying Council negotiations, as well as fresh perspectives on its democratic legitimacy.

Legislative bargaining datasets: a review

We identify five interview-based datasets that focus on the EU policy-making process: DEU, NDEU, INTEREURO, EMU Positions, and NCEU. This section provides an overview of their construction, as well as a snapshot of the research questions they have been instrumental in addressing. Table 1 below provides a comparative description of their main features, including basic statistics such as the number of legislative proposals covered.Footnote ⁴

Table 1. Dataset descriptions

The first and broadest effort to codify the EU legislative process is the Decision-Making in the European Union (DEU) dataset (Thomson et al, Reference Thomson, Stokman, Achen and König2006; Thomson et al., Reference Thomson, Arregui, Leuffen, Costello, Cross, Hertz and Jensen2012; Arregui and Perarnaud, Reference Arregui and Perarnaud2022). In its latest version of three (DEU III), it covers 20 years of EU policy-making (1999–2019), documenting 141 legislative proposals. The dataset, theoretically rooted in rational choice institutionalism, employs spatial analysis (e.g., Bueno de Mesquita and Stokman, Reference Bueno de Mesquita and Stokman1994) as its guiding principle. Through 494 face-to-face interviews with experts, the DEU collects data on the initial bargaining positions of member states (MSs), the EC and the EP, along with issue salience, allowing for the testing of predictors of legislative outcomes. The extensive timeframe and depth of research make DEU the largest existing dataset on EU decision-making. Its centrality is demonstrated by its influence on other data-collection endeavors, often serving as a complementary source or validity test. DEU itself is also validated using official documents.

Through its longevity and depth, it has contributed to many findings. Notably, scholars have sought to explain specific preferences, coalitions, and cleavages in Council voting (e.g., Kaeding and Selck, Reference Kaeding and Selck2005) and to understand the prevalence of consensus norms within the institution (e.g., König and Junge, Reference König and Junge2009). Others have examined the impact of Council rules and procedures on voting behavior (e.g., Arregui, Reference Arregui2008) and the predictors of negotiating success within the institution (e.g., Aksoy, Reference Aksoy2010; Golub, Reference Golub2012). Additionally, there is growing interest in the salience of domestic preferences in Council voting (e.g., Franchino and Wratil, Reference Franchino and Wratil2019)

Furthermore, DEU has enabled more focused studies, analyzing the behavior of specific MSs (e.g., Kirpsza, Reference Kirpsza2020; Bicchi and Arregui, Reference Bicchi and Arregui2023) or examining the impact of the EU’s Eastern enlargement (e.g., Thomson, Reference Thomson2009). It has also allowed for the analysis of specific institutional features, such as the effectiveness of the Presidency (e.g., Warntjen, Reference Warntjen2008), the impact of national government changes (e.g., Scherpereel and Perez, Reference Scherpereel and Perez2015), the influence of divergent preferences within delegations representing coalition governments (Kostadinova and Kreppel, Reference Kostadinova and Kreppel2022), and transparency levels in decision-making (Cross, Reference Cross2014).

The dataset has also been utilized to understand negotiation processes beyond the Council, namely interinstitutional negotiations. Scholars have examined the Council’s bargaining power relative to the Commission and the Parliament (e.g., Costello and Thomson, Reference Costello and Thomson2013; Kreppel, Reference Kreppel2018), as well as the dynamics of trilogues (e.g., Rasmussen and Reh, Reference Rasmussen and Reh2013). Finally, the DEU’s detailed information has enabled the analysis of competing bargaining models (e.g., Thomson et al., Reference Thomson, Stokman, Achen and König2006; Schneider et al., Reference Schneider, Finke and Bailer2010), allowing researchers to explain the occasionally idiosyncratic predictions resulting from their application (Wøien Hansen, Reference Wøien Hansen2014).

The second study is the National Decision-Making in the European Union (NDEU) dataset (Schneider and Baltz, Reference Schneider and Baltz2003). Developed concurrently with the first version of DEU (1997–1999), the NDEU employs a similar methodology but focuses on pre-negotiations between national governments and domestic interest groups for a subset of cases. Through fieldwork in four EU countries, NDEU researchers gathered data on the domestic negotiations that shape the national positions presented in the Council. While it lacks the breadth and depth of DEU, NDEU complements it by offering insights into actors beyond the MSs, highlighting often overlooked sources of power in the literature (namely the domestic interests, both political and business-related, that shape MS preference formation).

This dataset has contributed to a relatively limited number of publications, primarily fulfilling its purpose of demonstrating that interest groups can influence legislative outcomes in their favor during national preference formation (Schneider and Baltz, Reference Schneider and Baltz2003). Further research showed that government bureaucracies retain significant discretion in the decision-making process (Schneider and Baltz, Reference Schneider and Baltz2005) and that traditional étatiste attitudes in preference-shaping only yield to clientelistic or corporatist patterns when vital private interests are involved (Schneider et al., Reference Schneider, Finke and Baltz2007).

The third study constitutes a systematic analysis of the influence of non-state lobbying and interest groups on the EC, collected between 2008 and 2010. The INTEREURO dataset (Bernhagen et al., Reference Bernhagen, Dür and Marshall2014) uses a data-gathering process similar to DEU, modeling actor preferences based on interviews with 70 policy experts. This dataset models the positions of the MSs and of the EU’s institutions, as well as those of over 1,000 different non-state actors, mostly sectoral interest groups (though individual ones are typically involved in only a few legislative files). The resulting literature groups these actors according to the broader stakeholders they represent, usually business vs. non-business. Finally, further data is provided on the technical expertise of each non-state actor and their tactics.

Some of INTEREURO’s contributions are technical, as it has been employed to refine mapping and measurement techniques in the broader literature on interest groups (e.g., Bernhagen et al., Reference Bernhagen, Dür and Marshall2014). Derived empirical publications have examined the efficacy of lobbyists’ tactics (e.g., Börang and Naurin, Reference Börang and Naurin2015), their alignment with citizens/policymakers in terms of issue salience (Beyers et al., Reference Beyers, Dür and Wonka2018), the interaction between business and non-business groups (Beyers and de Bruycker, Reference Beyers and de Bruycker2017), and their relative success (Dür et al., Reference Dür, Bernhagen and Marshall2015). Finally, this dataset has been used to explore the extent to which interest groups contribute to the politicization of EU debates (Wonka et al., Reference Wonka, De Bruycker, De Bièvre, Braun and Beyers2018), to map their relationships with the EP’s party groups (e.g., de Bruycker, Reference de Bruycker2016), and to analyze predictors of their positional proximity with the Commission (Bernhagen et al., Reference Bernhagen, Dür and Marshall2015).

The fourth study analyses sectorial policy-making, specifically examining the contested economic reforms following the Eurozone crisis (2010–2015). Using DEU’s process, the Economic and Monetary Union (EMU) Positions dataset (Wasserfallen et al., Reference Wasserfallen, Leuffen, Kudrna and Degner2019) models positions and saliences for all 28 MS governments and six EU institutions for 47 issues taken from 10 highly controversial dossiers. Unlike in DEU, the EMU Positions’s authors carry out extensive document analysis (examining around 5,000 sources) before conducting their interviews. This innovation reduces the number of meetings needed, potentially increasing accessibility for future researchers. However, the method’s reliance on media reports may limit its applicability to highly mediatized policy domains, which may not be representative of EU policy-making in general.

Empirically, the dataset has been employed to evaluate bargaining success (Lundgren et al., Reference Lundgren, Bailer, Dellmuth, Tallberg and Tarlea2019) and model the dimensions of conflict (Lehner and Wasserfallen, Reference Lehner and Wasserfallen2019) during the Eurozone reforms, as well as to explore topics such as the relevance of economic and political factors in MS preference formation (Târlea et al., Reference Târlea, Bailer, Degner, Dellmuth, Leuffen, Lundgren and Wasserfallen2019) and the effects of Franco-German cooperation on negotiations (Degner and Leuffen, Reference Degner and Leuffen2019). Finally, Finke & Bailer (Reference Finke and Bailer2019) utilize EMU Positions to test bargaining models during economic crises.

The fifth and final dataset, the Negotiations in the Council of the European Union (NCEU), adopts a distinctive approach by focusing on the network analysis of cooperation and conflict within the Council (Naurin et al., Reference Naurin, Johansson and Lindahl2020). With seven waves of interviews, the NCEU dataset provides extensive coverage over time (2003–2021) across 11 different working groups and committees in the Council. Researchers reconstruct the most common patterns of cooperation among MSs, employing Policy Network Theory (e.g., Leifeld and Schneider, Reference Leifeld and Schneider2012) and the stochastic actor-oriented model (Snijders et al., Reference Snijders, van de Bunt and Steglich2010) as their framework. A ‘Network Capital’ index is then computed to identify the most influential negotiators in the bargaining process.

This approach has allowed scholars to construct a network model of EU decision-making, examining how interinstitutional rules influence intra-institutional politics (e.g., Häge and Naurin, Reference Häge and Naurin2013) and providing insights into power dynamics among MSs in the Council (e.g., Naurin, Reference Naurin2015). The dataset has also been applied to more sectoral studies, particularly focusing on MS outside the Eurozone (e.g., Naurin and Lindahl, Reference Naurin and Lindahl2010) and coalition-building before and after the 2004 enlargement (Naurin and Lindahl, Reference Naurin, Lindahl, Naurin and Wallace2008). Recent research using NCEU data has explored correlations between MS cooperation and policy compliance (Johansson, Reference Johansson2018), the factors behind the formation of policy networks (Huhe et al., Reference Huhe, Naurin and Thomson2018), and gender balance within the Council’s working groups and committees (e.g., Naurin et al., Reference Naurin, Naurin and Alexander2019).

Extant gaps in the literature: what these datasets cannot do

As observed in the introduction, EU decision-making is a complex and somewhat secretive governance process involving a wide range of political actors. Interview-based datasets have demonstrated their value by overcoming data collection challenges, offering snapshots of decision-making processes, and developing tools to measure bargaining mechanisms and outcomes. The extensive literature reviewed in this study underscores their significance. They have enabled the formulation of numerous original hypotheses on EU intra- and interinstitutional dynamics, advancing scholars’ understanding of the Union’s decision-making dynamics to unprecedented levels over the past two decades. Thanks to them, researchers have been able to explain the mechanisms behind Council negotiations and its interactions with legislative partners, map network relations between its internal actors and understand the determinants of their bargaining success, as well as address questions about its democratic legitimacy. Scholars have also begun to explore the datasets’ potential for interoperability. NDEU especially was designed to work in conjunction with the DEU, but the latter has also been combined extensively with NCEU (e.g., Huhe et al., Reference Huhe, Naurin and Thomson2018). Overall, though, this capacity remains underexploited.

Despite this progress, claiming a complete understanding of EU negotiation mechanisms remains premature, and interview-based datasets have their own blind spots. For instance, they do not capture the agenda-setting stage, where powerful interests can prevent certain issues from even being discussed in Council. This may potentially bias analyses of actors’ influence (Degner and Leuffen, Reference Degner and Leuffen2019). Moreover, they do not allow for the modeling of interinstitutional and cross-actor coalitions. Namely, these datasets tend to be underpowered to capture more complex alliances between MSs, EP political groups and sectoral lobbying actors (Princen, Reference Princen2012). Efforts like NDEU and INTEREURO represent initial attempts to address these critiques by focusing on different stakeholders. However, comprehensive research including all actors continues to face significant challenges in data collection.

Likewise, our datasets generally struggle to model the differences between the various Council configurations and related policy areas. Theoretical work suggests that these operate as distinct realms with varying norms that impact bargaining outcomes (Lewis, Reference Lewis2010). However, detailed examinations are scarce, and have relied on alternative sources (Bailer et al., Reference Bailer, Mattila and Schneider2015; Häge, Reference Häge2016). Again, the primary challenge lies in data-gathering. Conducting interview-based studies that yield enough statistical power for the analysis of each configuration would demand extremely extensive fieldwork, making it unrealistic for a single research team. Scholars interested in this area may thus need to innovate by identifying instrumental variables enabling indirect comparisons between configurations. Finally, researchers still struggle to accurately measure individuals’ impact on EU decision-making, usually treating them as representatives of group interests. Exceptions exist (e.g., Bailer, Reference Bailer2004), but officials’ personal power is often overlooked despite its likely importance. The challenge here is bias. The evaluation of individuals’ effectiveness is highly subjective, much more so than the information collected in the existing datasets.

Future developments and conclusion

Computing power advancements and the advent of Artificial Intelligence (AI) present exciting opportunities for political scientists. Furthermore, post-Lisbon Treaty, the Council has begun to broadcast its meetings. Both these avenues provide robust data mining options, potentially complementing or supplanting interviews. In this section, we review current research and envision future applications, contrasting these methods with expert interviews.

The DICEU dataset (Wratil and Hobolt, Reference Wratil and Hobolt2019) is the most notable existing collection effort utilizing these new methods. Specifically, the authors have transcribed and hand-coded Council meeting videos to analyze delegation approval rates through text analysis. Initially focused on ECOFIN Council deliberations from 2010 to 2015, this innovative dataset has only yielded two articles so far, due to its youth. Hobolt and Wratil (Reference Hobolt and Wratil2020) found that government responsiveness in Council negotiations depends on national issue salience, while Wratil et al. (Reference Wratil, Wäckerle and Proksch2022) compared the impact of public opinion on Europhile and Eurosceptic governments.

DICEU offers significant potential, with notable strengths. It analyses public statements directly, thus reducing the risk of bias inherent to secondary sources. The data collection process is also cheaper than sending researchers to Brussels. However, interview data remains superior in other ways. First, video analysis lacks flexibility, as scholars are limited to the analysis of what diplomats are willing to say publicly, while much negotiation occurs in-camera. Additionally, public statements may not represent true preferences, as delegates may seek to conceal their intentions. To their credit, Wratil & Hobolt (Reference Wratil and Hobolt2019) validate DICEU extensively,Footnote ⁶ and their position estimates generally match the DEU’s, but the dataset needs further testing across policy areas. Second, video analysis still incurs important costs, given its reliance on employing research assistants to hand-code statements. AI-driven text analysis could provide a solution, and the DICEU’s research team is already working on this, as well as expanding the dataset.

AI advancements could also significantly impact the study of the Council, and pioneers (e.g., Franchino and Mariotto, Reference Franchino and Mariotto2013; Cross and Hermansson, Reference Cross and Hermansson2017; Laloux, Reference Laloux2021) have already applied text similarity algorithms to compare legislative texts and assess bargaining success. Although these early efforts are praiseworthy and yield interesting results, the software is relatively unsophisticated, focusing on word similarity without considering the weight of amendments. Recent language-learning models (LLMs) offer improvements and can categorize text or even rank statements on a policy axis (e.g., economic left-right, social liberal-conservative, see Le Mens and Gallego, Reference Le Mens and Gallego2023). This enhances cost-efficiency and allows for larger sample sizes than traditional methods.

Two issues remain: bias and accuracy. Software may exhibit political bias from its training data or creators. This can be mitigated by using different LLMs for the same analysis, though a bias common to all models may exist. Accuracy is a larger concern, especially with complex texts spanning various political axes. Experts recommend extensive ex-post validation (e.g., Grimmer and Stewart, Reference Grimmer and Stewart2013), using human coding or expert interviews. A further problem is the availability of documents. The Council rarely publishes individual MS positions, and even when it does, they are not collected in a single document. Advanced software can only analyze available documents, not locate non-existent ones. Thus, humans must compile positions using current methods—official documents and/or interviews—slowing the process and reducing the sample size, lessening AI’s main advantages. So far, then, both videos and LLMs complement rather than replace interviews, and the latter will always maintain their uses, even if limited in future to the beginning (as a mechanism for developing new research questions) or the end (as a validation tool) of the research process. Any innovation enhancing efficiency in EU studies is a positive development, although direct contact with policymakers will remain vital for the discipline.

In conclusion, this overview has traced the evolution and future potential of interview-based research on the Council, focusing on five datasets and the research questions they have helped answer. These datasets have enabled scholars to link EU inputs, outputs and processes in ways other sources cannot, particularly in understanding the predictors of bargaining success and coalition formation within the institution. However, this data collection method remains costly, and certain research questions present an insurmountable challenge for it. We have therefore outlined potential new advances that could bridge these gaps, and look forward to further developments allowing us to reach a more comprehensive understanding of the Council. We also hope that this review can serve as inspiration for research on other institutions, particularly international ones where access to traditional data is limited. To our knowledge, the methods discussed here have only been applied systematically to the Council, but they could be adapted to other key decision-making bodies, both within the EU (e.g., the Parliament) and beyond.