Structural analysis and intelligent classification of clinical trial eligibility criteria based on deep learning and medical text mining

Objective:

To enhance the efficiency, quality, and innovation capability of clinical trials, this paper introduces a novel model called CTEC-AC (Clinical Trial Eligibility Criteria Automatic Classification), aimed at structuring clinical trial eligibility criteria into computationally explainable classifications.

Methods:

We obtained detailed information on the latest 2,500 clinical trials from ClinicalTrials.gov, generating over 20,000 eligibility criteria data entries. To enhance the expressiveness of these criteria, we integrated two powerful methods: ClinicalBERT and MetaMap. The resulting enhanced features were used as input for a hierarchical clustering algorithm. Post-processing included expert validation of the algorithm’s output to ensure the accuracy of the constructed annotated eligibility text corpus. Ultimately, our model was employed to automate the classification of eligibility criteria.

Results:

We identified 31 distinct categories to summarize the eligibility criteria written by clinical researchers and uncovered common themes in how these criteria are expressed. Using our automated classification model on a labeled dataset, we achieved a macro-average F1 score of 0.94.

Conclusion:

This work can automatically extract structured representations from unstructured eligibility criteria text, significantly advancing the informatization of clinical trials. This, in turn, can significantly enhance the intelligence of automated participant recruitment for clinical researchers.