John Newton Darroch, 1930–2024

Abstract

John Darroch was born in England in Melksham, Wiltshire, to George Darroch and Phyllis Lacey on 22 October 1930 and died, aged 93 years, on 15 April 2024.

He attended grammar school where he excelled in the classroom and at sports, and subsequently gained admission to study civil engineering at the University of Bristol. Recognising his talent, his mathematics teacher persuaded him to postpone university and take the exam for the Cambridge Open Scholarship to study mathematics.

John was successful and studied for 4 years, specialising in theoretical physics and completing the Diploma in Mathematical Statistics under the supervision of Dennis Lindley. He studied briefly with R.A. Fisher in this time, although this appears not to have been a significant factor in his future career choices. It was also there that he met his future wife, Elisabeth Pennington. He completed 2 years of national service in the RAF, at the rank of Pilot Officer, teaching mathematics, and in 1955 he and Elisabeth set sail for Cape Town to take up his new position as a lecturer at the University.

John arrived at the University of Cape Town ‘with no thought of doing any research’. Within a few months, an enquiry from the professor of biology awakened his instinct for research, culminating in his 1958 Biometrika paper on capture-recapture experiments. He enrolled in a Ph.D. by Publication program at Cape Town, and was subsequently awarded the degree on the basis of his series of three Biometrika papers on capture-recapture (Darroch 1958, 1959, 1961). This seminal contribution proposed models and provided maximum likelihood estimates for a number of capture-recapture settings, and provided a basis for much of the development that followed. It was through this work, undertaken without supervision and with only rudimentary access to the literature, that John developed his first-principles approach to research.

John's academic career flourished. He and Elisabeth returned to England to take up a lectureship at the University of Manchester, where he supervised George Seber for a period introducing him to a problem in capture-recapture. Keen to escape the cold winters, John and his young family moved to Adelaide where he took up a senior lectureship at the University of Adelaide. This was followed by a position at the University of Michigan, Ann Arbor, but John and his family were drawn back to Adelaide. In 1966 he was offered and accepted the Inaugural Chair in Statistics at Flinders University, a position he held until his retirement in 1996.

John is best known for his contributions to statistical methodology, especially in the area of multivariate categorical data. His work was recognised in 2005 through the award of the SSA Pitman Medal. Writing in support of the award, Stephen Fienberg observed: Someone once remarked to me that there are few statisticians who have a major new idea that could be said to influence the development of our field. Clearly John Darroch has had at least four such breakthrough ideas in the area of categorical data alone. But he has also worked on other important problems in statistics (e.g., canonical correlations) and its application (e.g., statistics and the law). John has thought clearly and wisely about a diverse collection of problems and has had a truly major impact on the field of statistics.

John's interest in categorical data, commencing with his 1962 paper on interactions in multi-factor contingency tables (Darroch 1962), spanned several decades and resulted in many important contributions. This included the development of the generalised iterative proportional scaling algorithm jointly with Douglas Ratcliff (Darroch & Ratcliff 1972), which, with its generalisations, continues to find applications because of its simplicity and convergence properties. His joint work with Terry Speed and Steffen Lauritzen on Markov fields and log-linear interactions for contingency tables (Darroch, Lauritzen & Speed 1980) was a landmark in the development of models for categorical data and set the stage for a broader class of graphical models that revolutionised Bayesian statistics. It is interesting to note that John had separately developed some of these ideas a few years earlier, but that work was never published. Similarly, John was a pioneer of Markov Chain Monte Carlo, developing Markov Chains based on a very simple neighbourhood structure to estimate exact P-values in contingency table analysis. This work was presented in various seminars but also not published.

John's other widely recognised work is in the area of quasi-stationarity distributions for absorbing Markov processes. Together with Eugene Seneta, he made ground-breaking advances to the theory (Darroch & Seneta 1965, 1967).

In addition to the major contributions highlighted above, John's productive research career resulted in a number of other insightful papers on a range of topics on statistical methodology. Throughout this body of work, a number of themes are apparent.

First, his insights were often highly original, elegant and relevant. This is, no doubt, attributable in part to the intellectual independence arising from his early experience at Cape Town. From working in virtual isolation, John recognised the advantage of not being constrained by the thoughts of others and carried this forward in his future work. When considering a new research topic, he would deliberately not read the existing literature until he had ‘thought long and hard about the problem’.

Second, although John was a highly capable mathematician in his own right, he valued conceptual insight above mathematical technicality. His intuition was such that by choosing the right concept, the mathematical justification would typically follow. He thought critically about his own ideas and was not seduced by the apparent elegance of a concept without rigorous justification. As such, his research was typically an ideal blend of creativity, conceptual clarity and mathematical rigour.

Finally, John's methodological research covered a diverse range of topics but a common strand through much of this work was his interest with dependence among variates: that is, the genesis and interpretation of complex relationships, how such relationships can be represented in models and methods of inference. He thought deeply on these problems and developed a sophisticated view that properly accounted for the multiple sources of dependence that will typically be present in any observable association.

John's interests and influence extended beyond academic publishing; of particular note is his involvement in the Splatt Royal Commission in 1983. The subject of the Royal Commission was to enquire into the murder conviction of Edward Splatt in 1978, and John assisted the Commission defence barrister with the statistical aspects of the case. He identified serious flaws in the logic of the prosecution arguments that he articulated by applying probabilistic logic and Bayes' theorem. The report of the Royal Commission indicated acceptance of his arguments and also overturned the conviction of Edward Splatt. The Splatt Royal Commission became the subject of John's presidential address to the Statistical Society of Australia. The text appeared in the February 1985 SSAI newsletter and a version was later published in the Professional Statistician (Darroch 1987). John was subsequently introduced by Terry Speed to Sir Richard Eggleston, an eminent judge, Chancellor of Monash University and author of the book ‘Evidence, proof and probability’. John greatly valued the ensuing interaction that involved extensive correspondence and resulted in papers at two international conferences.

Although best known for his research, John made many important contributions to the profession, most notably, his contribution to statistical education. His initial appointment at the University of Cape Town was as their first ever lecturer in probability and statistics and it was his role to introduce that material to the curriculum. When he arrived at the University of Adelaide 7 years later, there was a single statistics subject available. Within a few months, John introduced a second offering in mathematical statistics at third year and a fourth-year subject on Markov Chains. Both subjects were an innovative synthesis of the most up-to-date material available at the time. These subjects inspired Eugene Seneta to further study with John, leading eventually to their previously mentioned work on quasi-stationary distributions. In his role as the Inaugural Chair in Statistics at Flinders University, John was responsible for the design and implementation of the statistics program. At its zenith, the program made available a comprehensive range of subjects up to honours level to a generation of statisticians. The program encompassed many of the recent developments of the time, and of particular note was the treatment of linear statistical models in terms of coordinate-free geometry. John worked hard for many years to maintain the program against the tide of declining enrolments. He recognised the need to establish a service teaching role for the discipline, a task made even more difficult by the inexplicable opposition from the then Head of Mathematical Sciences. Supervision of graduate students was also a great source of satisfaction to John over his career. He was a role model, mentor and friend to his students, many of whom progressed to highly successful careers.

John at his desk in his Flinders University office, May 1995. Source: unknown

John contributed to the profession more broadly and held, at various times, the positions of the SA Branch President and national President of the Statistical Society of Australia and President of the International Biometric Society, Australasian Region. Within the University, he contributed in a number of roles, including as the Head of the School of Mathematical Sciences. In an environment at times driven by ego, he brought much needed reason and civility through his unimpeachable courtesy and steadfast adherence to principle.

Beyond his professional life, John had wide and varied interests. He enjoyed many physical activities. He maintained his involvement in hockey and athletics from his early years at school until his move to Cape Town, where they were replaced with hiking on Table Mountain and swimming and body-surfing. These were replaced over the years by tennis, sailing, gliding, hang-gliding and wind-surfing. He had a lifelong love of music, literature and travel. Pottery was another of his long-term hobbies and he took great pleasure in sharing his carefully made pieces with friends and family.

All who knew John would attest to his modest, unassuming and impeccably polite manner. He was articulate and could hold conversation on a wide range of topics. He genuinely enjoyed interaction with people from all walks of life. In the days before outsourcing, it was standard practice for tradespeople employed by the University to take their tea breaks in whichever tea room was nearby their worksite. John came to know many of them well and it was common to see him speaking to them with the same level of interest and engagement as he did with his academic colleagues.

John's colleagues, students and visitors will remember his family home in the Adelaide foothills at Glen Osmond. John and Elisabeth would from time to time entertain various groups, and many memorable evenings were spent overlooking a spectacular view of the city. Many of John's colleagues and collaborators also became his lifelong friends. Academic visitors would often stay with him and Elisabeth when in Adelaide. For many years following their retirement, John and his close friends from Flinders University, Miloslav Jiřina, Jerry Kautsky and Ananth Rao, formed the nucleus of a small group of former colleagues who would meet each month for lunch.

When he retired in 1996, John decided to make a clean break from statistics. He gave away all of his books and journals and eschewed the suggestion of an emeritus appointment at the University. He and Elisabeth spent the next few years travelling, visiting families and friends and indulging their love of the arts. Sadly, Elisabeth was diagnosed with cancer early in 2000 and passed away later that year. Unfortunately, this was not the first loss suffered by John and his family, having been devastated by the death of his oldest son, Alastair, in an accident in 1981. John's children Richard and Helen were a great source of comfort following Elisabeth's death.

Fortuitously, John met Gloria Acton in 2002 at a meeting of the Jung Society and they married the following year. John and Gloria shared many interests and were clearly devoted to each other and their families. They spent many happy years travelling together, visiting family and enjoying music, opera and literature until his health made it no longer possible.

John is remembered with great affection and enormous respect by those fortunate enough to have known him. His modest and unassuming nature belies a lifetime of outstanding contribution and influence.