Using knitr and LaTeX for literate laboratory notes

Many years ago I worked in a lab that hired a new student. His assignment seemed to be easy: to reproduce the results of another student and to expand on them. The work in question included mathematical modeling, computer simulations, and so on. To his dismay, the new student found out that the programs used undocumented libraries, the scripts had incomprehensible options, and the models had unstated assumptions. Deciphering all this turned out to be very difficult. While the original author was willing to help, he could not do much: the author started to forget the details of his research soon after graduation. This experience had a profound influence on me. The question of whether I would be able to understand my own research in a decade or two became an obsession. Being a scientist, I asked myself how the problem was solved elsewhere. In experimental and applied sciences the research may cost millions of dollars. To preserve it, the researchers are required to keep detailed logs of their activity in the laboratory notes. The notes include the details of the experiments and their results, and also the hypotheses tested. There are courses [14] and books [6] about laboratory notes. It is fascinating to study laboratory notes of great scientists, for example, Linus Pauling. Pauling’s notes comprise 46 notebooks spanning from 1922 to 1992, digitized by Special Collections & Archives of the Oregon State University [13]. His beautiful notes have a definite aesthetic value. I would argue that the concept behind the practice of laboratory note keeping is somewhat akin to the concept of literate programming [8]. Knuth understood that code is just part of a programmer’s output. The programmer’s thoughts about these programs are even more important. Similarly, an important insight for science is that papers, preprints, presentations are not the research, but “an advertisement of the research” [15]. We must preserve the research itself [11, 15]. Laboratory notes are the means for this preservation. Classic laboratory notes are physical notebooks like those of Linus Pauling. Unfortunately, this format has a number of flaws: