Energy Technology Expert Elicitations for Policy: Workshops, Modeling, and Meta-Analysis

Characterizing the future performance of energy technologies can improve the development of energy policies that have net benefits under a broad set of future conditions. In particular, decisions about public investments in research, development, and demonstration (RD&D) that promote technological change can benefit from (1) an explicit consideration of the uncertainty inherent in the innovation process and (2) a systematic evaluation of the tradeoffs in investment allocations across different technologies. To shed light on these questions, over the past five years several groups in the United States and Europe have conducted expert elicitations and modeled the resulting societal benefits. In this paper, we discuss the lessons learned from the design and implementation of these initiatives in four respects. First, we discuss lessons from the development of ten energy-technology expert elicitation protocols, highlighting the challenge of matching elicitation design with a particular modeling tool. Second, we report insights from the use of expert elicitations to optimize RD&D investment portfolios. These include a discussion of the rate of decreasing marginal returns to research, the optimal level of overall investments, and the sensitivity of results to policy scenarios and selected metrics for evaluation. Third, we discuss the effect of combining online elicitation tools with in-person group discussions on the usefulness of the results. Fourth, we summarize the results of a meta-analysis of elicited data across research groups to identify the association between expert characteristics and elicitation results.