Fine-tuning large language models for predicting organic micropollutant rejection by polyamide membranes

Abstract

Large language models (LLMs) have demonstrated cross-domain application capabilities due to the emergent intelligence derived from billion-scale parameters. However, their potential in modeling specific membrane processes requires further investigation. In this study, we selected the rejection of organic micropollutants (OMPs) by polyamide membrane, a typical membrane process, to evaluate whether LLMs can accurately predict its performance. The results show that base LLMs, including gpt-4o, 4o-mini, and 4.1-mini, were unable to predict the OMP rejection process before fine-tuning, with predictive accuracy even inferior to a linear regression model. However, the predictive performance of LLMs improved substantially after fine-tuning, which enhanced the mechanistic understanding of size exclusion, enabling the prediction of OMP rejection. Notably, larger LLMs did not outperform smaller ones, indicating that model size alone does not guarantee superior performance, likely due to the limited parameter ratio that can be fine-tuned. Although the fine-tuned LLMs still slightly underperformed compared to conventional machine learning models, they demonstrated considerable promise for regression-based applications. Further analysis revealed that the lack of domain-specific knowledge in LLMs limits the effectiveness of transfer learning during fine-tuning. Moreover, converting structured data into text sequences leads to information loss and redundancy, preventing LLMs from effectively capturing the influence of input features. Therefore, model architecture, data sufficiency, and task compatibility are suggested to be prioritized for the successful application of LLMs. This study demonstrates the potential of fine-tuned LLMs for regression-based modeling and highlights feasible strategies for improvement, providing valuable insights for advancing their practical deployment in membrane research.