Habeas Corpus? Cultural Keywords, Statistical Keywords, and the Role of a Corpus in their Identification

Abstract

At the end of World War II, a young officer from the British army was granted immediate demobilization to return to Cambridge in order to complete his degree, which had been interrupted by the war. He had enlisted in the army in 1941 at the age of 20 and had led a unit of four tanks as part of the Guards Armoured Division during the battle for Normandy. After successfully completing his undergraduate degree on his return to Cambridge, he took a job as a tutor with the Workers' Educational Association in the hope that it would also allow him time to write novels and literary criticism. Within 10 years, at the age of 35, he had finished his first major work of criticism. This was to become one of the most influential works of criticism in English of the latter half of the twentieth century, and along with other of his books was to play a decisive role in founding and shaping the academic field of Cultural Studies in the United Kingdom. The man was Raymond Williams, and the book was Culture and Society 1780–1950.

The manuscript as delivered in 1956 to the publisher Chatto and Windus by a then relatively unknown academic – a tutor in adult education – was considered too long, and an important appendix in which Williams discussed words which he considered significant in framing debates about culture and society was left out. Even so, his introduction to Culture and Society carries the subtitle: The Key Words – ‘Industry’, ‘Democracy’, ‘Class’, ‘Art’, ‘Culture’. Indeed, his encounter with the history of these words in the Oxford English Dictionary in the basement of the public library of Seaford more or less primed the book and became a cornerstone of his method of literary and cultural analysis. Some twenty years later, in 1976, these very words Industry, Democracy, Class, Art, and Culture, along with the excised appendix and further notes became the basis of a self-standing work, Keywords: A vocabulary of Culture and Society (revised and expanded in 1983), in which Williams provided two- or three-page accounts of 131words that he considered crucial to our understanding of culture and society, as well as the complicated relations between them. In this did both Culture and Society and Keywords not only inaugurate and help to shape the field of cultural studies, but they also prompted a particular and continuing thread of work in that field on the vocabulary of culture and society.

There have, indeed, been two substantial sequels within the tradition of enquiry that Williams inaugurated: New Keywords: A Revised Vocabulary of Culture and Society¹ and Keywords for Today: A 21st Century Vocabulary.² Both books draw their inspiration directly and openly from Williams's original Keywords and see their purpose as building on his initial definitions and purpose in the light of social and cultural change over the intervening decades. Of course, the notion that some particular words seem able to sum up a culture, a time, a place, or a substantial body of writing was not unique to Williams: the general idea has a long history and was shared by other scholars. Stubbs³ points to antecedent traditions in Europe – Schlüsselwørter in Germany and mots clef in France (both of which translate fairly neatly as ‘keywords’).^4,5,6 Indeed, Conrad's narrator in Under Western Eyes – a teacher of languages, no less – when faced with a bewildering document finds himself wishing if some ‘keyword’ might not be found – ‘a word that could stand at the back of all the words covering the pages; a word which, if not truth itself, may perchance hold truth enough to help to help the moral discovery which should be the object of every tale’.⁷

But there is a whole other tradition of work in which the notion of keyword is also crucial in a quite different way, and that is corpus linguistics – the analysis of patterns of language in very large bodies (corpora) of text, primarily to clarify questions relating to the nature of meaning and ultimately the nature of language itself. Its radical point of departure is perhaps best summed up by the statement of the British linguist, J.R. Firth¹⁰ (p. 11): ‘you shall know a word by the company it keeps’. Or as Wittgenstein¹¹ (p. 80, 109) puts it: ‘the meaning of a word is its use in the language’. In order to study ‘use in the language’ or ‘the company’ kept by a word, corpus linguistics examines extremely large bodies of data (on the grounds that the frequency of occurrence of many words is quite small, so you need very large corpora to catch regularities in their behaviour.) This perceived need to work with very large corpora was given extra impetus by the advent of digital and computing technology: thus, as currently exercised, corpus linguistics uses computational analysis often supported by statistics. An important element of computational and statistical method in corpus linguistics is the isolation of those words which have particular salience – keywords – in their respective contexts in a corpus.

Basically, a software programme (favoured ones are AntConc, see Antony,¹² or Wordsmith, see Scott¹³) is used to sort the words of a corpus into a list ranked, for example, by frequency (or, for that matter, by alphabetical order), and a statistical procedure (such as Log-Likelihood score or the Chi-squared or t-test) is applied to determine if the frequency of a particular word in the target corpus when compared with the frequency of that same word in another corpus (selected for reference or comparative purposes) is relatively and proportionally different. If the statistical procedure suggests that the difference in relative frequency is in the technical sense significant,¹⁴ then the word in question is deemed to be a keyword in the target corpus under scrutiny.

From its radical point of departure, corpus linguistics has become a major trend in the systematic study of language. Indeed, some of its adherents credit it with bringing about the equivalent of a Copernican revolution in linguistics – a shift from the introspective study of linguistic intuitions to the empirical observation of linguistic behaviour, and a shift from deductions about the nature of linguistic structure to building instead cumulative generalisations from large bodies of data. It has been widely adopted in allied fields such as literary studies, language teaching, and forensic linguistics, with one particular tool of corpus linguistics – the identification of statistically significant keywords – being adopted in several detailed studies. (See, for example, Bondi and Scott.¹⁷ For some critical reflection on the kinds of metric used in isolation of key words, see also: Gabrielatos and Marchi¹⁸; and Pojanapunya and Watson Todd).¹⁹

There is, therefore, at first sight a very marked difference between the computational and statistical approach to keywords of corpus linguistics and the method adopted by Raymond Williams which underlay the publication of Keywords: A Vocabulary of Culture and Society. And yet – although very different in approach – might there not remain a tantalising possibility that the computational and statistical procedures of corpus linguistics could lend some further investigatory power to the somewhat intuitive and frankly interpretative approach of cultural studies? After all, did not the linguist J.R. Firth (often cited as an inspiration for corpus linguistics) talk of ‘sociologically important words which one might call focal or pivot words’ some forty years before Williams, preparing the ground for some kind of productive rapprochement between corpus linguistics and cultural studies.

In this respect, of special interest, therefore, is a corpus linguistic study, Keywords in the Press: The New Labour Years,¹⁸ that does identify keywords on a statistical basis but then sets out precisely to reconcile their identification of keywords on the basis of their statistical significance with the highly interpretative approach of Williams. Their first sentence sets the scene: ‘This book reports on a research project which attempts to combine Raymond Williams’ influential notion of keywords … with corpus linguistics’ (p.1). They aim to do so by integrating the quantitative rigour of using statistical significance to identify keywords in large bodies of text with Williams's more intuitive approach to identifying keywords on the basis that they are, as he says, either ‘significant binding words in certain activities and their interpretation; … [or]… significant, indicative words in certain forms of thought’⁹ – especially when exploring areas of meaning pertinent to the interrelationships of culture and society.

Each of their keywords is subjected to detailed commentary leading to conclusions of the following kind. Choice, for example, in the Blair years is associated with ‘a broad political philosophy of market-based services’ (p. 90). Global becomes ‘a more powerful sociopolitical keyword as it makes globalization, which could be a contested concept if it were foregrounded, invisible’ (p. 138). Indeed, ‘at its most extreme, it seems that global … shares with other keywords … the ability to sum up an assumed set of semantic features which are paradoxically difficult to capture or enumerate’ (p. 139). Its keyness or statistical over-presence in the corpus ‘linguistically suggests the existence of globalization as a process, thus presenting globalization as already existing’ (p. 138). In the case of reform, ‘there is a tendency to presuppose that it is desirable and, if not an absolute good like democracy and freedom, at least to be welcomed and possibly to be packaged in an acceptable way (p. 112). And further ‘it is in the non-countable unmodified form that we see the tendency towards reform as a desirable outcome of political power’ (p. 113). The use of spin ‘reflects the very great amount of attention that was paid to the New Labour communication methods and the distrust of these methods in the hands of communications advisors, who rapidly became known in all contexts as spin doctors (pp. 64–5). Respect in the Blair corpus was ‘a new moral virtue … a policy area alongside jobs, wages and housing, with practical and legislative actions that could be taken to increase this desirable and tangible asset’ (p. 185). At the same time, it seems ‘to have developed in the same way as other keywords in this study, becoming more like a shorthand label for a complex idea, but one which is both assumed and slippery’ (p. 186). Lastly terror is used ‘more in connection with coercion and intimidation (i.e. terrorism) than it is in connection with discussion of people’s emotional states’ (p. 162). However, they add, more generally, that ‘the word form terror acts as a form of shorthand that includes all sorts of activities, some of them at a remove from acts of terror that might cause terror. As a consequence, terror becomes ever more encompassing and indeterminate. The possible consequence of this is that the use of terror as a term goes unnoticed and unchallenged’ (p. 162). As the authors comment in their conclusions, a central conclusion to be drawn from their analyses is that (the emergent meanings of) their keywords ‘can be shown to develop into a shorthand for a vague, but implicitly complex set of assumptions’ (p. 196).

The study, published as part of the Bloomsbury series, Research in Corpus and Discourse, has been well received, with generally approving reviews.²¹ It has been described as: ‘a thorough and comprehensive piece of research’ (Gomez-Jimenez, p. 111); ‘a refreshing contribution to the body of research at the interface of stylistics, discourse analysis and corpus linguistics (Wiegand); 'a very insightful study of the language of newspaper reporting … (which) … successfully merges the fields of critical stylistics and corpus linguistics’ (Fotiadou, 186). Its methodological contributions were found to be particularly welcome and attracted special mention: ‘Research methodology is one of the strengths of this book’ (Gomez-Jimenez, 108); ‘The methodology for analysis is sound, systematic, explicit, carefully reflected and transparent so that the book will be very useful as a guideline for similar studies’ … ‘[I]t can also be used as a methodological template for future analyses of such keywords’²¹; ‘[T]he book has potential methodological and theoretical implications for … corpus linguistics, discourse analysis, and stylistics’ (Wiegand); and finally, ‘[I]t is valuable that they offer a complex and flexible analysis, but it is even more valuable that they explain the methodology adopted … in great detail … This results in a reliable and replicable method of analysis that will doubtless serve other scholars in the field carrying our similar research’ (Gomez-Jimenez, p. 109).

It is worth noting that, while the book's methodology is welcomed by reviewers for its transparency, reliability, and replicability, the beneficiaries of this methodology are envisaged to be mainly scholars in the fields of stylistics, discourse analysis, and corpus linguistics. Strangely absent from the reviews is any sense of the methodology's possible contribution to cultural studies, media studies, or sociology. And yet that is precisely the avowed impetus of the study.

What then do Jeffries and Walker bring as linguists when, as they say, they ‘revisit Williams’ keywords’ (p. 5)? They claim to bring objectivity and even more importantly ‘rigour, retrievability and replicability’ (p.16). According to their account, Williams himself – despite his fifteen pages of closely argued introduction – ‘gives no explanation of where the list [of keywords] came from’. In this regard, they believe that his approach relied simply on intuition and interpretation, reflecting ‘his own personal and political biases’ (p. 5), too easily affected by aspects of his own background as a ‘white male Marxist’ (p. 5).^22,23, ^22,23 Never mind that this bald description effaces the varied facets of Williams's identity as – for instance – working class by origin, Welsh, wartime artillery officer and commander of a tank unit in the battle for Normandy, novelist, and Cambridge academic.

Let us start with one of the most basic decisions at the outset of the study – that of building a corpus. Their aim is to see ‘to what extent the language of broadsheet journalism showed signs of reflecting the ideological landscape of the UK under Blair’s government’ (p. 18). Even allowing for the fact that this question itself presupposes that the outline of the ideological landscape is somewhere evident in advance so that the language of broadsheet journalism can be compared with it, what then will count as representative of the language of broadsheet journalism? Jeffries and Walker choose three newspapers as the basis of their 14.8-million-word corpus: The Guardian, The Independent, and The Times. These titles, however, span a narrow spectrum of broadsheet opinion, roughly from the political centre to the left. A large circulation right-wing newspaper The Daily Telegraph was not included on the grounds that it 'was not available across the whole ten-year period of the Blair premiership’ (p. 24). No mention is made of possible alternatives to the Telegraph, or – for that matter – additions from across the political spectrum such as The Economist, The Financial Times, The Daily Mail, and The Daily Express. Already, from the outset we have a partial – one might even say politically biased – selection of broadsheet journalism, one that was limited, in part at least, apparently on pragmatic grounds, by access to material. It would be useful – on the grounds of replicability, for example – to know how access to the raw data was affected. But all we are told is: ‘We sourced our data from an online database of newspapers’ (p. 24). No information is provided about this database, nor is it even named. Already, however, one kind of problematic auxiliary assumption is in play: that the political persuasion of a broadsheet newspaper (and/or their preferred readership) does not affect its deployment of sociopolitically significant keywords.

Of course, once the corpus has been designed, other kinds of restriction or filtering must also come early into play: ‘we were selective about the articles we retrieved, and included only news articles that dealt with political or current affairs’ (p. 24). Thus, ‘We achieved this in part by selecting only those articles which contained specific search terms’, such as Labour AND/OR Blair AND/OR Government (p. 25). What counts as an article, however, is not made clear. Initially they refer to news reporting and news stories; later in their analysis, however, the scope of what counts as an article widens imperceptibly to include news commentary. So, for instance, in their discussion of Spin in the Blair corpus, they refer to ‘the creativity and versatility with which the political sense of spin was taken up by commentators’, demonstrating ‘not just a scathing critique of New Labour by the commentariat, but also a playful delight in turning the tables and satirizing the government’s style of language’ (p. 65).

However, a corpus that is restricted to news reports will be very different from one that includes op-ed pieces and news editorials. There are clear generic differences between these kinds of writing in terms of evaluation, structure, and argument which will inevitably affect the kinds of semantic load placed on the keywords that occur within them. And unless we know the kinds of generic constraints exercised in the selection of articles, the process is difficult to replicate. So, a second unexamined and problematic auxiliary assumption finds its way into their study: that different genres of newspaper writing do not affect the deployment of sociopolitical keywords.

Yet further methodological problems, however, arise in the movement from selecting the shape of the corpus to the identification of sociopolitical keywords. As the authors openly acknowledge, this process is for them partly quantitative and partly qualitative. In order to identify words that are statistically salient in the corpus under study (the target corpus), it needs to be compared with another corpus (the reference corpus). A common practice in corpus linguistics is to compare the target corpus with a large reference corpus of general English. Should, for instance, we wish to know if the word realistic is salient in a corpus of television drama reviews, then we compare its relative frequency in the television corpus with the corpus of general English. If the relative frequency is the same, then its frequency would seem not to be salient in the drama reviews. But what happens if we compare the relative frequency of realistic in a large enough corpus of television drama reviews with an equivalently sized corpus of reviews of television game shows, and it turns out to be relatively much more frequent in drama reviews than in game show reviews? We might then wish to conclude that realistic is a keyword for drama reviews when compared with other kinds of review, especially if a statistical test seems to confirm its statistical significance.²⁴ It is clear that salience – or more exactly, statistical significance – rests ultimately on the choice of a reference corpus and is limited to the terms of the comparison.

Given, however, that their interest is in the development of specific meanings in their chosen sociopolitical keywords during the New Labour years, Jeffries and Walker require some yardstick against which to measure these changes. For this they turn not so much to an analysis of the reference corpus (which could provide a baseline of usage immediately preceding the Blair years) but rather to the Oxford English Dictionary Online (OED). Thus, their detailed discussion of each keyword begins with a summary of the meanings as defined by the OED. Take, for example, their commentary on one of their keywords, Terror:

The presence – and the problems – of auxiliary assumptions in scientific work has been much discussed since first articulated as the Duhem-Quine thesis – and increasingly so since the emergence of the replication or replicability crisis in the behavioural sciences. If auxiliary assumptions are implicit and unrecognised in the design of a research method, then these unstated assumptions make it difficult for subsequent researchers to reproduce the method in the way the original researchers applied it. (See Ting and Montgomery²⁵). In the fields of experimental psychology or social psychology, for instance, failures of replicability (now a source of much concern) can be traced in part to ‘insufficient specification of the conditions necessary or sufficient to obtain the results’.²⁶

Failures of replicability can also be traced to problems of statistical method, particularly the use of p-values to determine if the measured effects are consequential or inconsequential. We will return to this point in Section 8. below under the heading ‘Methodology, Rigour, and the Pitfalls of Statistics’. Suffice it to say at this stage that the arbitrary methodological decisions with their attendant auxiliary assumptions that we have identified above cannot but raise doubts about the scientific rigour and replicability claimed for their approach by Jeffries and Walker. But in addition to problems of methodological rigour, there are also problems of conceptual rigour, especially as they pertain to the admittedly difficult term ‘ideology’.

Given that the declared aim of Jeffries and Walker in identifying keywords is informed by a ‘wish to characterize a period in British political history by the words of the period and in doing so question, perhaps even challenge, the ideology that they represent’ (p. 4), it is somewhat surprising that no clear ideology of the New Labour years emerges from their extensive discussion of the keywords which they identify. On the contrary, a common characteristic discerned by them in the keywords that they study is that under New Labour these words become ‘relatively empty of meaning’ (p.195) – although paradoxically at the same time they also manage to ‘presume a package of semantic features’ (p. 195) with the effect that they appear ‘to stand for a complex set of semantic components which the reader is presumed to understand’ (p. 195). Indeed, all six keywords ‘can be shown to develop into a shorthand for a vague, but implicitly complex set of assumptions’ (p. 196). As a set, these sociopolitical keywords, they say, ‘seem to do important ideological work in serious discussions about political decision-making and yet appear to be relatively empty of meaning … The uneasy feeling that many voters report of being unsure what the political elites really stand for may well be partly due to this kind of political language which we feel we should understand, but know that we do not’ (p. 197).

Apart from the fact that on the surface this account seems to subvert the recognised distinction in mainstream linguistics between the inherent meaning of words and propositions – normally dealt with in the field of semantics – and the implied meanings of utterances in context – normally dealt with in pragmatics – it is hard to comprehend how keywords can be simultaneously ‘empty of meaning’ but ‘stand for a complex set of semantic components’. But this characteristic of shorthand emptiness of meaning identified by Jeffries and Walker is in their account the main thread that binds these keywords together in their ideological work. And so, empty of meaning, their keywords apparently do not interlink in any positive way: they do not combine to help shape a compelling narrative; they do not associate with each other to underpin arguments in favour of a particular way of life; they do not apparently offer a subject position of any kind, favourable or otherwise, to the reader or writer, the speaker or hearer.

Part of the difficulty here may well lie with Jeffries and Walker's theory of ideology; but given its central role in the argument of the book, surprisingly little space is devoted to defining or explaining it. Their initial definition occurs in a two-page section entitled Ideation and Ideology: ‘Ideology can be seen as referring to sets of values and/or beliefs that are held by a group of people or by a society as a whole’ (p. 10). As a definition, however, this is so general that it has little to distinguish it from definitions of ‘world view’ or even ‘culture’. See, for example, Williams: ‘the culture of a group (…) is the peculiar and distinctive “way of life” of the group (…), the meanings, values and ideas embodied in institutions, in social relations, in systems of beliefs, in mores, customs, in the uses of objects and material life’ (Keywords, p. 90); or, as the Cambridge Dictionary puts it: ‘the way of life, especially the general customs and beliefs, of a particular group of people at a particular time’.

Jeffries and Walker attempt to delimit the operation of ideology and its relationship to language by introducing the notion of ‘text world’ and ‘ideation’. Ideation is their term, broadly, for the representational function of language and its capacity to ‘make present’ in language the world of objects, persons, events, and processes, by virtue of selection from within the available linguistic systems of vocabulary and grammar – the lexicogrammar. In this model of language, most utterances draw on the capacities of ideation. But not all utterances are ideological. For Jeffries and Walker, utterances become ideological when values become attached to statements about the world as ideationally encoded by lexicogrammatical selection. To illustrate, they use the example of notices which they associate with 1950s/1960s London boarding houses: No blacks. No Irish. No dogs. This text becomes ideological for Jeffries and Walker in the way it creates equivalence through grammatical parallelism between particular elements within the text – blacks, Irish, dogs – and thus lowers the status of some classifications of human beings to that of animals. ‘The text presents a world view which has an attached ideology’ (p. 11). And so, in this way ‘Ideology (…) comes into play where ideational processes in texts produce text-conceptual worlds which have values attached to them’ (p. 11) by – for example – structural equivalence. Nonetheless, this definition of ideology – as values attached by equivalent lexicogrammatical selections – remains extremely broad. As it stands, it would apply to Martin Luther King's “I have a dream” peroration in his speech from the steps of the Lincoln Memorial or for that matter to the Beatitudes in Christ's ‘Sermon on the Mount’ in Matthew 5: 3–12.

In a further attempt to narrow their definition, they note that ‘Ideologies can be attached more or less implicitly/explicitly’ (p. 11). And ‘ideologies that are expressed explicitly are usually self-consciously and sincerely held views’ (which to some extent thankfully lets Martin Luther King and the Evangelist off the bad ideological hook). However, ‘[s]ome ideologies, which are usually implicit rather than explicit in texts, become so embedded in a culture that they seem to be common sense and are therefore “naturalized.” Such ideologies can be, at times, difficult to spot and as a result are more difficult to argue with’ (p. 12). It is this buried, naturalised ideology, lying hidden in the sociopolitical keywords of the New Labour period, that Jeffries and Walker set off to track down in order to display how a pattern of local adjustment of lexical meaning is common to these everyday words (choice, reform, spin, etc.) and ‘that turn out to be significant markers of the naturalized ideology of the Blair years’ (p. 14).

There are honourable antecedents for the ‘ideology-as-the-naturalization-of-meaning’ thesis: one of its clearest and theoretically subtle expositions can be found in Roland Barthes (1957/1972)²⁷ Mythologies – a kind of ‘key myths’ of post-war bourgeois French society.²⁸ Alongside detailed commentary by Barthes around specific cases of the naturalisation of meaning (he dubbed it ‘the privation of history’), he identified several other ideological manoeuvres, such as tautology (‘Brexit means Brexit’ would be a recent example), inoculation (‘a few bad apples’, as an institutional response to the presence of perpetrators within their ranks, and so not admitted to be a systemic problem), and the statement of fact (perhaps better known now, since Trump, as ‘a statement of alternative fact’), even while naturalisation remains for Barthes the overarching principle. Since the publication of Mythologies in English in 1972, however, and spanning roughly from the 70s to the 90s, an important body of work developed across the human and social sciences which placed the concept of ideology front and centre in cultural, media, and communication studies, and to a lesser extent in applied or critical linguistics, drawing variously on Marxism, classical sociology, linguistics, semiology, and psychoanalysis. To mention only a few: Althusser²⁹; Coward and Ellis³⁰; Fowler & Kress³¹; Pecheux³²; Volosinov³³; Stuart^34,35; Thompson^36,37, ^36,37; Eagleton³⁸; and Van Dijk.³⁹

Surprisingly, none of this substantial body of work is mentioned, referenced, or discussed by Jeffries and Walker, leaving them to rely simply on asserting the ‘naturalization of meaning’ thesis which – pace Barthes (also not referenced), who devotes a whole book to a theoretically sophisticated exposition and exemplification of it – leaves Jeffries and Walker with only a weak and undeveloped definition of ideology. In this way, their two-page discussion of ideation and ideology manages to bypass three decades of sustained theoretical exploration and enquiry, and several important questions about the nature of ideology are thereby overlooked. First of all, who and what is ideology for? Who benefits and who loses in the power struggles of meaning? In Jeffries and Walker's account, the ideology of the Blair years is condensed into six keywords that circulate in a narrow segment of the broadsheet press. Whose words are they? And who are they for? As we pointed out earlier, the filtering and flattening of the source database of newspapers into an anonymous corpus obscures: (1) whether these words are used in reporting a discursive agenda set within the inner councils of the Labour Party, in policy documents, in Parliamentary debate, or in corporate boardrooms; or (2) whether these words are being used instead to encode the comments and reactions of leader writers and columnists – ‘the commentariat’. What is their purpose – their evaluative accent: to propose a policy, as in ‘reform’ or critique a practice, as in ‘spin’?

Secondly, if these six words project a unified ideology, where are the interlocking networks of meaning – the presupposed or implicated schemes, scripts, and tropes – that constitute the semantic and pragmatic cross currents that bind them together? Jeffries and Walker's corpus linguistic methodology has no doubt worked effectively to map the collocational and lexicogrammatical environment of the individual words considered on an individual basis, but there is little sense of how these words might work in concert (or in contradiction) ‘as meanings in the service of power’³⁴ working through and articulating positions at a particular historical conjuncture.

Thirdly, their account of the naturalising ideology of the Blair era seems to presuppose that their six keywords project a unitary and unified ideology. Jeffries and Walker may not be clear about the parameters and laylines of this ideology – because, according to them, its very terms are emptied of meaning – but there is little or no sense in their study of the fault lines, resistances, inconsistencies, or of the struggles within the force field of ideology as it works out in process. Is this ideology of the Blair years simply a dominant ideology? If so, where was its subordinate? In the case of war on terror, for instance, its use, although pervasive, was incidentally and often called into question by the use of quotation marks, by attributing it to a named source, e.g. Bush's ‘call for a “war on terror”’, or by citing it as an object of criticism, e.g. ‘Gen. Sir Mike Jackson condemns “war on terror”’ (See Montgomery³⁸). For this reason, there was, he noted, ‘an unevenness in practice to the way terms and patterns become mobilized’.³⁸ (p.131)

At the very least, public discourse, for instance, is very different from private discourse (although the exact boundaries between them may not always be easy to determine) and is subject accordingly to differing kinds of validity judgements. And the nature of these claims and judgements about validity are in any case subject to change and transformation. There are grounds to believe, for instance, that the public sphere, originally conceptualised by Habermas in ideal terms of consensus, rationality, and coordination has become increasingly agonistic and belligerent. (See, for example, Higgins and Smith⁴⁵). Indeed, the tenor, key, or tone of the public sphere is as important as the ideational component of its ideologies so that the belligerence, or authenticity, or sincerity of its actors comes to seem as important as the truth or otherwise of their statements. (See, for example, Montgomery.⁴⁶)

Finally, however, there must exist the very possibility that some sociopolitical key words may not be at all, in any sense, statistically significant. Words that are numerically or mathematically salient by virtue of their frequency in a body of data may simply constitute evidence of an ideological surface structure but one that rests on a deep structure whose components may be generatively but not be statistically significant.

To work an informal example: if the public discourse of the last fifteen years since the financial crisis of 2008 were examined for keywords, the following would have claims for sociopolitical significance – if not continuously throughout the period, at least for part of this period:

Austerity, Global/Globalization, Order, Democracy, Free/Freedom, Market, Rules, Control, Sovereign/Sovereignty, Private/Privatisation, Culture, Immigrant/Immigration, Radical/radicalized, Climate, Money, Spending, Public, Individual, Legal/illegal.

Amongst these candidate keywords of sociopolitical significance, there are collocational tendencies that develop such as illegal + immigrant/immigration, free + market. And their behaviour as lexical items may well call for statistical scrutiny. Some of these words during this period may, moreover, develop particular kinds of semantic prosody that statistics and concordancing may help to reveal. Culture, for instance, while taking on some of the semantic load of academic definitions⁴⁷ begins to assume – for reasons that are difficult to explain – a quite negative semantic prosody as in collocations such as cancel culture and culture wars, but most particularly in the frame culture of _________, where the frame is almost invariably completed by a negative term (misogyny, racism, bullying, violence).

At the same time, from the perspective of a theory of ideology, what is most important is the way in which some of these candidate keywords of sociopolitical (and cultural?) significance articulate together in clusters or chains of meaning such as the following:

Law, Order, Control, Rules, Regulations, Individual, Sovereignty, Free, Freedom

And become inflected in phrases such as:

free market, freedom of the individual, rules-based international order, take back control in which freedom (of the market and the individual) to exercise sovereignty (of the self or the nation) from rules and regulations (especially if these can be regarded as set elsewhere) is an absolute good.

Of course, in practice the meaning of many of these candidate keywords of sociopolitical significance is unstable, multi-accentual, and may be invoked in quite different ways in different discursive formations. Even to those of a liberal persuasion, not all rules and regulations are a threat to freedom of the individual or the smooth operation of the market: it may be necessary, for instance, even for those with a proclivity for rule breaking, to legislate to prevent online harms in an age of social media or the scope and impact of Artificial Intelligence (AI). Indeed, as Schlesinger⁴⁸ has pointed out, the digital economy has prompted the development of what he calls a ‘neo-regulatory framework’ where ‘regulation’, for instance, may be invoked covertly for reasons of national security but traded off against ‘innovation’ in uneven and contradictory ways.

But although these chains and clusters of meaning may well be fragments of ideology, which – like Gramsci's notion of common sense – consist of ‘an ambiguous, contradictory and multiform concept’, they may nonetheless be governed by an underlying structure capable of generating and holding in place this apparently ‘chaotic aggregate of disparate conceptions’. For underlying the chaotic surface, an elementary structure may be discerned of the kind in which the (autonomous, sovereign) self is always potentially under threat from the (alien, limiting, regulating) other; nor is this structure susceptible to, or the outcome of, statistical scrutiny since neither self nor other would emerge in any catalogue of statistical keywords. Indeed, in the search for keywords, they would most likely be filtered out at an early stage of the inquiry as close relatives of grammatical items and therefore to be excluded along with items such as here, there, and everywhere.

Ideology, of course, is a contested concept. (Williams devoted one of his longer entries in Keywords to it. It receives extended treatment in Bennett et al. New Keywords.¹ But, interestingly, MacCabe and Yanacek's Keywords for Today² dispenses with it altogether.) And from a high point of interest in the last decade of the twentieth century, the focus has shifted away from ideology to be replaced by a concern with discourse – which seemed, under the influence of Foucault, to offer new ways of understanding the relationships between power, language, truth, and representation. More recently, under the pressure of political events, even the notion of discourse has to some extent retreated from the forefront of academic concern in cultural and media studies to be replaced by a focus on populism. It is an unfortunate irony that, as Eagleton remarks, ‘the concept of ideology should be out of fashion among intellectuals at just the time when it was flourishing in reality’.⁴⁹ (p.xiii)

For all that, ideology (rather like culture) is a term that has increasingly found a place in everyday, public policy debate where ‘ideological’ now stands in a dichotomous relation to ‘pragmatic’, or even ‘common sense’. None of these developments provide good reasons for Jeffries and Walker to avoid the term. But in using it repeatedly in the narrow sense of ‘naturalized ideology’, the term has to carry a heavy burden in their argument: some greater degree of elaboration of its theoretical implications and antecedents would have been welcome. As it stands, their version of ideology has to serve a complex theoretical role, which the notion of the naturalization of meaning seems ill-equipped to fill. After all, many kinds of linguistic expression – especially, for example, successful metaphors – become automatised and emptied of their original meaning. Who now remembers that ‘to fathom’, ‘to sound out’, ‘to lose one’s bearings’, ‘swinging the lead’, and ‘taking soundings’ began life as precise nautical expressions now that the original literal meaning has long since slipped from view? And a whole theory of poetic language – Russian and Czech Formalism – was built up around the notion of defamiliarising and de-automatising everyday language.⁵⁰ The naturalization (or automatization) of meaning is not the singular province of ideology.

But there is a more fundamental problem in Jeffries and Walker's approach: one which concerns the very epistemological status of their statistical method. For, at the heart of their approach is an appeal (not unique to them but common in corpus linguistics) to statistics. Keywords, for them, are not just salient in their corpus but deemed by them to be statistically significant. And without the application of statistics to their corpora, they would lack any explanation of where their list of keywords came from – except that they had deemed them sociopolitically significant by a process not as clear as that adopted by Williams in identifying culturally significant keywords. Moreover, statistics do more for Jeffries and Walker than solve the practical problem of initially identifying keywords; they provide a fundamental guarantee of the overall scientific rigour, objectivity, and replicability of their research. This is the dimension they offer as a complement to, and improvement on, Williams. For unlike Williams, whose list reflects his own ‘personal and political biases’, and whose selection they regard as difficult to replicate, their own initial list of keywords is the outcome of a computational and mathematical process of sorting and ranking, where an impersonal statistical machinery (used, it must be said, countless number of times across the applied behavioural sciences) assumes control. So, applying the well-tried statistical formulae to the same data, any researcher should be able to achieve the same result, free of the perturbations and noise of personal bias.

We have already set out in detail in Section 6. some of the auxiliary assumptions that implicitly come into play at various points in their research design – in the choice of database, the selection of newspapers, the selection of the reference corpus – all of which will affect their their attempt to delineate the ideology of the New Labour years. At a deeper level, however, there is a problem with the statistical method itself, with the very notion of statistical significance, or at least with the way in which it is commonly applied in many applied behavioural studies.

On the surface, then, all seems well in terms of objectivity and statistical rigour. However, the crisis in the behavioural sciences over issues of replicability dating from around 2015 (see, for instance, Open Science Collaboration²⁴) has brought the apparent robustness of tests of statistical significance into question, to such a degree that the American Statistical Association (ASA), the largest international association of statisticians, was prompted to release a public statement on p-values in 2016 and followed it up with two special issues in its official journal, The American Statistician.⁵¹ Targeting the typical applied researcher, the 2016 statement attempts in precise terms to define clearly the concept of p-value and also advises against some common but problematic uses. Their statement defines the p-value as ‘the probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value’.⁵²

In short, neither p-value, nor its associated Log-Likelihood value, can be used as a guarantee of statistical robustness or that the results are correct and that they were not reached by chance. In various ways, therefore, Jeffries and Walker's special appeal to statistical method as a touchstone of rigour is undermined by the developed understanding of the leading professional association of statisticians. Indeed, Jeffries and Walker's overarching claim to objectivity, transparency of method, and replicability as signature virtues of their whole approach must be seen as unsustainable on various counts. They rely on auxiliary assumptions that are not openly acknowledged or explicated; they override their own statistical indicators in favour of un-explicated interpretive procedures; and they seem unaware of the now-recognised limitations of the cornerstone of their statistical method, viz. statistical significance itself.

There is, however, in conclusion, a curious paradox at work in Jeffries and Walker's Keywords in the Press: The New Labour years. They set out to complement the work of Raymond Williams as described by him in his book Keywords: A vocabulary of culture and society. But Jeffries and Walker's approach recasts what Williams was doing not so much as the analysis of cultural keywords but rather as, in their terms, the isolation of sociopolitical keywords, identified – as we have seen – initially by statistical method. These sociopolitical keywords, claim Jeffries and Walker, provide an outline of the ideology of the Blair years, in the same way that Williams ‘tried to capture something about the ideology of the post-war years, with the aim of challenging that ideology and contesting the meaning of the keywords he discussed’.

Instead, the words on which Williams concentrated were words which in their polysemous history showed variable and conflicting currents of meaning around questions of culture and society. He selected them because in their history and in their current, sometimes contradictory usage, they might help advance our understanding of those very questions that lay at the heart of Culture and Society and the Long Revolution.⁵⁷ To see Williams's Keywords as a form of ideology critique is fundamentally a betrayal of the truly radical edge that informs his approach. His keywords were intended as tools for thinking with, and his tools for thinking with were the words themselves, the uses to which they have been put, and the uses to which they might be put in the struggle to achieve the long revolution and a better future. In this, his keywords collectively – along with the successor volumes – have the quality, in Bauman's paraphrase of Santayana, ‘of a knife with the edge pressed against the future’.⁵⁸ (p.12)