[Editor’s Note: This is the seventh entry in an eight-part Showcase on new ideas in administrative law theory. The introductory post is available here.]
–Sarah Nason, Prifysgol Bangor University
Studies examining empirical dimensions of administrative law have grown up in parallel too, but largely disconnected from, theoretical work. Some suggests that contemporary preoccupation both with theory and empiricism is part of a trend away from traditional doctrinal scholarship in administrative law, perhaps due to dissatisfaction with the limits of common law method.
Elsewhere I have developed a constructivist methodology that attempts to meaningfully bring together administrative law theory and empirical evidence. The facts relied upon have largely been those about who issues cases and defends them, who their lawyers are, the topics of claims and their outcomes. Here I begin to examine how we can use judgments as empirical evidence, and what this might contribute to administrative law theory.
The problem with using case law as facts is that the judgments subject to analysis will usually be limited and self-selected. The method also has to assume that the reasons for deciding cited in reported cases are indeed the reasons for deciding. There have been attempts to overcome these concerns, including by analysing a full set of cases from a particular time-period, or those brought against a particular type of defendant, and presenting this alongside other corroborating information. Social-scientific techniques of content analysis have been used to draw replicable inferences about case law by interpreting and labelling judgments. However, this method does not eradicate the need for evaluation; a choice still needs to be made about how to code the data.
The next evolution is automated content analysis; using Machine Learning and Natural Language Processing techniques to classify textual information. Two basic approaches are; identifying the frequency of words or phrases (n-grams) in a corpus of text; and, topic modelling, using algorithms to discover the themes that pervade a large and potentially otherwise unstructured, collection of documents.
N-Gram Analysis (Frequency of Words)
In my research I have constructed ‘corpora’ of substantive judgments in England and Wales High Court judicial review (from 1983 up to and including 2017). Examining trends in the frequency with which particular words and phrases (n-grams) are used across these corpora matches other data about the topics of judicial review; high numbers of tax and planning cases, longer-term increases in reference to prisoners, immigration and the European Union. On the other hand, looking at the frequency with which words associated with grounds of judicial review occur one finds barely any reference to modern doctrinal concepts such as ‘intensity’, ‘anxious scrutiny’, or ‘deference’. References to ‘proportionality’ and ‘proportionate’ have, however, increased steadily from the 1990s up to and including 2017. But ‘jurisdiction’, ‘jurisdictional’ and ‘illegality’ have also enjoyed a resurgence especially from 2014/2015. Whilst falling in the earlier 2000s, uses of the words ‘reasonable’, ‘reasonableness’ and ‘Wednesbury’ are also on the increase. Conversely, doctrinal phrases including ‘relevant (and ‘irrelevant’) considerations’, ‘proper’ (and ‘improper’) purposes’ are rarely ever used. Similarly, the phrases ‘error of fact’ and ‘error of law’ barely occur (though the latter is enjoying an increase in the 2010s alongside the elderly statesman ‘ultra vires’). The corpora provide additional data showing how ‘fairness’ has almost entirely replaced ‘procedural propriety’ and ‘natural justice’; and that ‘consultation’ is among the terms increasing most significantly in use since the 1980s.
The research also developed a test that can determine with 89% accuracy whether a judgment is an Administrative Court judicial review, or statutory appeal judgment. Revealingly, differences in the legal grounds cited are not significant enough to form part of this test; though maladministration and consultation feature notably in judicial review but not in statutory appeals.
The findings reinforce a growing view that at least some general doctrinal analytical tools playing a central role in structuring textbooks may actually have little to contribute in terms of identifying concrete solutions to legal problems. Nevertheless, the perennial force of Wednesbury reasonableness, jurisdiction, and ultra vires remains evident, and indeed more prominent than proportionality-based reasoning in the most recent years.
Topic-Modelling
Using a corpus of 1,200 Administrative Court judicial review judgments from 2001 to 2017 (a sample of approx. 25% of reported cases), I developed a topic-modelling analysis where a topic is a cluster that groups together words more likely to appear with one another across a corpus, utilizing patterns of word occurrence to break up a corpus into components, identifying the vocabulary associated with each topic. This cuts across all the judgments in the corpus showing how consistently words appear together wherever they appear, not just in individual case judgments. The eight most prominent topics were:
| Topic A (planning policy): housing, plan, development, policy, local, planning, affordable, core, strategy, policies |
| Topic B (prisons & parole hearing decisions): board, claimant, risk, parole, prison, prisoner, mr, decision, oral, hearing |
| Topic C (professional discipline): panel, interim, order, conditions, article, case, practice, practise, suspension, orders |
| Topic D (access to justice & legal services): services, legal, contract, law, time, tender, criteria, lsc, process, solicitors |
| Topic E (children, family and young persons): care, act, section, mr, children, authority, court, treatment, child, may |
| Topic F (indefinite leave to remain): policy, leave, secretary, decision, claimant, case, state, ilr, application, period |
| Topic G (asylum regulations & rights): member, article, state, regulation, asylum, application, decision, united, rights, right |
| Topic H (deportation & mental health): claimant, claimants, detention, secretary, state, decision, risk, deportation, case, mental |
Implications for Administrative Law: Facts, Theory and Doctrine
It would have been naïve to think that topic-modelling could have revealed a judicial review cake layered by a recognisable set of grounds, resolving prominent debates around say whether proportionality has indeed supplanted reasonableness review (though I admit I hoped that it might!). Nevertheless, in my view the implications of these machine learning methods support the case for a more contextualised subject-matter specific approach to administrative law doctrine and theory. They also cause us to reconsider the comparative perspectives and projects of common law scholars, legal practitioners, and members of the judiciary, and how these might be more constructively connected.
Suggested Citation: Sarah Nason, Showcase–New Directions in Administrative Law Theory: Administrative Law Theory and Empirical Research, Int’l J. Const. L. Blog, Sept. 21, 2019, at: http://www.iconnectblog.com/2019/09/showcase–new-directions-in-administrative-law-theory:-administrative-law-theory-and-empirical-research
Comments