Data rescue: discovery and recovery of historic climate observations

On the fortieth anniversary of the establishment of the Royal Meteorological Society's History Special Interest Group, a symposium was held on 11 October 2023 to discuss the data rescue and preservation activities that are currently underway in the United Kingdom. The Met Office's National Meteorological Library and Archive is a unique resource and should be the first port of call for anyone engaged in the analysis of historic climate data. The manager of the archive, Catherine Ross, gave the first presentation of the meeting. The collection is unique and Catherine described the importance of preserving the original sources of observations and metadata for both current and future researchers. The main theme of Catherine's talk was the value of metadata for assessing the reliability of meteorological observations. A vivid example was presented using the barogram sheets recorded onboard the Royal Navy vessel HMS Prince of Wales, which showed erratic behaviour on 24 May 1941. Contextual information indicated the reason for the unusual trace: the ship was engaging fire at the time with the German battleship Bismarck.

A theme that recurred throughout the meeting was the importance of placing modern climatological conditions in a long-term context. The longest series discussed at the meeting was that kept at the Radcliffe Observatory in Oxford since 1766, and Stephen Burt described the advances that are being made in the digitization and processing of the sub-daily pressure observations from the observatory. In his book ‘Oxford Weather and Climate since 1767’, co-authored with Tim Burt, the meteorological record-keeping practices at the observatory and the construction of the long temperature series were described in detail. Stephen's talk given at the symposium focused on the importance of the pressure series for understanding long-term change in the atmospheric circulation locally and regionally, for example, through assimilation into reanalysis datasets (see https://www.ecmwf.int/en/about/media-centre/focus/2023/fact-sheet-reanalysis). A notable feature of the Radcliffe Observatory's meteorological history is the homogeneity of the recording practices. Changes to the instruments and their location were few over a 200+ year period and the observatory holds the title for providing the UK's longest unbroken instrumental series at a single site. Indeed, we learnt that the Newman standard barometer (no. 1220) that was installed in 1838 is still in use today.

Over the last decade, the recruitment of volunteers to digitize historic meteorological observations via the Zooniverse platform has proved to be hugely successful. Indeed, the pressure data from the Radcliffe Observatory are currently being digitized using that platform (https://www.zooniverse.org/projects/edh/weather-rescue-oxford). The citizen-science theme was continued in Ed Hawkins's talk about the Rainfall Rescue project. In just 16 days in spring 2020 thanks to the efforts of more than 16 000 volunteers, 5.28 million monthly rainfall observations were digitized using quadruplicate data entry. These data have now been incorporated into the Met Office's gridded dataset and have extended the start date of that series back to 1836. In addition, the number of stations used in the gridding during the nineteenth century has increased considerably to provide more reliable, detailed local information. Focus has now moved on to the monumental task of digitizing the historic daily rainfall data for 1850–1960, which with 100 million observations presents a challenge even to the significant efforts of the many volunteers on Zooniverse.

Attention in the meeting was not only concentrated on British data series. Two talks described work that is currently being undertaken on historic data that will contribute to the new global climate datasets being developed under the NERC-funded GloSAT project. In Michael Taylor's talk we learnt about the efforts that have been undertaken to extend the CRUTEM monthly global terrestrial temperature dataset back to the late eighteenth century. The infilling of gaps in station series, particularly over the 1961–1990 base period, have increased the number of series incorporated into the dataset. Again, the value of metadata was noted in Michael's talk, particularly for guiding the correction for exposure-bias in the early temperature data. His talk concluded with a description of the art-science project titled ‘A Climate Mural for our Times’ commissioned by Norwich City Council (https://crudata.uea.ac.uk/cru/climate-mural), which depicts the temperature record from 65 Ma years BCE through to the year 2200 in art form.

Continuing the global theme, Praveen Teleti's talk described the results from two digitization projects that have recovered historic meteorological observations taken onboard ships: the World War Two recovery project has recovered hourly data from over 28 000 logbook images taken by the US Navy Pacific Fleet; and the Weather Rescue at Sea citizen science project recruited 3500 volunteers to digitize four-hourly observations recorded from ships in the 1860–1870s. Praveen reminded us of the historical context of weather record-keeping in two vivid examples, from HMS Erebus and Terror. These ships were pivotal in nineteenth-century exploration and the logbooks, from which the weather data have now been extracted, offer only a glimpse of the hardships endured by the crews in their voyages to Antarctica and the Arctic.

Mark McCarthy from the UK Met Office concluded the meeting with a talk that described the benefits of data rescue from a user's perspective. Mark outlined the role of the National Climate Information Centre in collating and presenting data from a variety of sources to provide a definitive and reliable source of climate information. This information is continually improved through the refinement of data processing and the recovery of historic data. Considerable progress was made in the mid-twentieth century by researchers engaged in the construction and homogenization of historic data series, and today's researchers continue to benefit from that work. However, we can now capture and analyse orders of magnitude more historic data than was possible by our predecessors and in his talk, Mark stressed the importance of preserving those data for future generations of researchers.

The next 40 years will see further considerable change to our climate, but the importance of contextualising contemporary conditions in a long historical timeframe will remain as important as ever. Julian Mayes, in his opening remarks to the meeting, described the purpose of the Society's History Group as furthering the study of the history of meteorology and physical oceanography. The sharing of information about the nature of historic record-keeping and the methods by which the observations were taken will remain essential to future data rescue initiatives.