Editorial on IVD cellular assay validation

Abstract

The article in this issue of Cytometry B on “Standardization of Flow Cytometric Detection of Antigen Expression” by the NCI clinical cytometry group formerly headed by Maryalice Stetler-Stevenson and the NIST group headed by Lili Wang is deserving of not only an accompanying editorial, but special attention by all those readers intending to work in clinical cytometry for the coming decade, as it describes an important future component of diagnostic cellular analysis (Tain et al., 2023). Specifically, the ability to measure the antigen (or probe target) expression on well-characterized cell populations will be a vital component of not only monitoring of patients with malignancy, as discussed in the article by Tian et al. herein (Tain et al., 2023) but also monitoring of a variety of immune responses or therapeutically altered defined cell populations. A few predictions as to what true antigen quantitation will provide: (1) treatment of sickle cell disease will be adjusted through a standardized measurement of the level of hemoglobin F in F cells (Hgb F containing RBCs) in order to retard the sickling process (De Souza et al., 2023); (2) patients with severe infection, cytokine storms and certainly sepsis will be monitored for a combination of activation markers (CD64, CD169, HLA-Dr and others) on neutrophils, monocytes and other cell types for informative and actionable changes regarding the immune status (Bourgoin et al., 2020; Davis et al., 2006; Davis & Bigelow, 2005; Ortillon et al., 2021; Schiff et al., 1997); (3) Rapid assays for the genetic expression of newly induced targets (CAR-T cells, adenovirus insertion of other targeting receptors, etc.).

The paper also compares two commonly advocated quantitation methods, PE labeled beads to derive average or median antibody binding capacity (ABC) per cell (Davis et al., 1998) vs. single point transformation or ratiometric comparison of the targeted cell population to the CD4 expression on helper T cells using an assumed 40,000 CD4 mAb binding sites per cell (Degheidy et al., 2016; Wang et al., 2016). Other technical variables the paper convincingly observed is that purified 1:1 PE:antibody preparations give better precision than regular off-the-shelf PE-labeled antibody lots, even if the measured F/P ratio of the off the shelf preparation is close to 1.00. Not surprisingly the study provides quantitative evidence that clone selection does matter and different clones with the same reported target antigen specificity can give variable results, up to nearly a two-fold difference in ABC units and this difference was in no way correctable using the reported F/P ratio of the antibody lot. While the use of 1:1 PE:antibody preparation along with spectrally matched beads for ABC quantitation gave acceptable imprecision with a CV between four instruments of ~5%. The surprising finding is that ratiometric one-point transformation of data using a secondary cell population, in this case CD4+ lymphocytes (based upon CD45 light scatter gating) gave the most superior performance in terms of low imprecision across multiple instrument platforms as CVs of ~1%. Interlaboratory precision being the holy grail of antigen quantitation, which involves getting the same result independent of instrument or reagent with a variance of ~1%. The study again brings to question regarding any assumption on “normal” antigen expression levels, particularly in light of what is know about gene expression variations between individuals and the various inhibitory and potentiating inducers of gene expression.

Based upon my experience developing the Leuko64 assay designed to measure neutrophil CD64 expression as a potential indicator of infection or sepsis that used an internal spectrally matched bead for ratiometric one-point transformation using software-based, lot-specific value assignments of ABC units to the beads could achieve the same level of imprecision (CVs of replicates of range of 1%) (Wong et al., 2015) similar to that reported herein by Tian et al. Furthermore, the paper provides one of the first peer-reviewed validations of stabilized control material as a long-term quality control tool for antigen quantitation thus providing, as can also be done with stable Fc-binding capture beads, the ability to generate long-term Levey–Jennings plots for monitoring the antibody fluorescence intensity throughout the lifetime of these critical reagents. Appropriately these teams from NIST and NCI have adhered to assay validation protocols outlined in the CLSI H62 document (CLSI, 2021). This now being 2 years after the H62 publication, which is now the de facto Cytometry B reviewers' guideline for judging completeness of assay validation papers. In doing so, the clinical and standards arms of the U.S. government are showing their recognition that such guidance is critical to support future clinical therapies and their recognition of the need for well-validated tools to support future clinical therapies, which are heavily dependent upon the expression level of surface antigens or receptors on defined viable cell populations either using in vivo blood samples or pre-infusion ex vivo manufactured cell products in order to more accurately judge therapeutic efficacy. In a similar vein, one would hope that the FDA will soon endorse the approach articulated the CLSI H62 document to ensure widespread adoption of these consensus best practices for the validation of flow cytometric methods. The EU IVDR regulatory guidelines for both IVD device manufacturers and clinical laboratories currently provides clear guidance on validation of laboratory developed tests (LDTs), certainly more so than the current often obscure combination of U.S. regulations that face IVD manufacturers and the current regulatory uncertainty threatening clinical flow cytometry laboratories, who offer LDTs as their most important service to hematology and oncology patients, recently announced by the FDA (FDA, 2023; NIST, 2023).

The point being made clear by this NIST/NCI collaborative paper, as well as many other works using spectrally-matched bead calibrators, is that highly reproduceable antigen expression measurements can be reliably obtained even across different flow cytometric platforms. However, an issue of accuracy still remains, likely due to the lack of traceability of these measurements to any “truth” or standard. As this paper points out, the previous assumption of 40,000 CD4 molecules on helper T cells needs further validation. Traceability is now a major premise for the complete validation of a clinical laboratory assay in the EU under the current IVDR with that goal being certified by the Joint Committee for Traceability in Laboratory Medicine (JCTLM, www.jctlm.org), who review and approve either a reference material or reference procedures and its acceptable traceability to specific clinical analytes. This would include any clinical assay reporting quantitative units of specific molecules per cell. Yes, one could use Fc capture beads or beads with a calibrated level of the fluorochrome attached to the antibody, once those have been traceably calibrated, but like cells, what is the means of traceability to some equivalent of molecules per cell? Fortunately, the solution for this final step of reliable quantitative cytometry is being evaluated as we go to press through the NIST Flow Cytometry Standards Consortium, which fortunately the FDA is part of this consortium of government, academic, and private manufacturers with an interest in IVD quantitative flow cytometry products. The NIST Flow Cytometry Consortium's goals are to: (1) develop reference standards including reference materials, reference data, reference methods, and measurement service for assigning the Equivalent Number of Reference Fluorophores (ERF) to calibration microspheres and assessing the associated uncertainties and utilities; (2) develop candidate reference standards including biological reference materials, reference data, reference methods; and (3) design interlaboratory studies based on candidate reference materials to support the development of best practices, standard methods, and documentary standards (https://www.nist.gov/programs-projects/quantitative-flow-cytometry-measurements). The current first large validation study by the consortium will provide validation of the NIST ERF concept of molecular quantitation through fluorescence traceability. The NIST approach promises to provide a means for clinical assays to have a recognized or at least NIST-endorsed route to cite traceability to regulators or at least those in the EU where IVD companies and clinical flow cytometry labs are not forced to guess the rules of the regulatory games.