CEC04-03 Tools to assess the quality of of kinetic publications

Abstract

Assessment of the quality of studies is an important step when reviewing publications. Concerning reviewing studies, a long tradition exists in the medical field with the Cochrane Reviews which started in 1993 when the Cochrane Collaboration Group was built. The focus of Cochrane reviews is on efficacy of medicines, therapeutic interventions and diagnostic procedures by performing systematic reviews. The collaboration group developed methodologies for systematic reviews, among tools to assess the “quality” of a study, and they are developed further until today.

Quality assessment (called also: critical appraisal, and risk of bias assessment) refers to the assessment of the methodological quality of a study, on which the confidence in the results of the study is based.

The principles of assessing the methodological quality of a study have extended to further fields, e.g. in toxicology where systematic reviews are performed to assess the safety of a substance in current use e.g. as a food additive.

The tools which were developed are primarily meant and applicable for the assessment of effect studies in humans and also animals, particularly the integration of both streams of evidence (Hannes et al., 2020; NTP OHAT, 2019).

Soliman et al. 2022 have published an inventory of quality markers which can be used to evaluate pharmacokinetic studies in humans. However, unfortunately, until today no appraisal tools are published which can be used for animal studies. No tools are also available in Guidelines addressing aspects of kinetic studies of OECD and ICH.

Thus, our group, a WG in the German Society of Toxicology, started to develop a tool suitable for assessing the quality of animal toxicokinetic studies.

The tool contains elements of quality assessment as in other tools, however adjusted to the aim to assessing toxicokinetic studies. The sections of the tool cover Study Design and Planning, Study Performance, Animal Housing and Feeding Conditions, Sample Collection and Chemical Analysis, Toxicokinetic Analysis and Data Reporting, and an Overall Evaluation. Every section is covered by several questions (sub-items) for which grades have to be given (3 for acceptable without restrictions, 2 for acceptable with restrictions, and 1 for existing with major flaws or not existing). Not applicable is also foreseen if the question is not relevant in the context of the study. The questions are given different weights, depending on their importance between 4 and 1. The final evaluation is performed by adding up the points (grades x weight), resulting in a final sum. The sum is related to the sum which could be reached if all questions were assessed as acceptable without restrictions. The percentage of the theoretical maximum score can be used as a benchmark for assessing the quality of the study and the confidence which can be given to the results.