The ethical evaluation of large language models and its optimization

The utilization of large language models (LLMs)has experienced tremendous growth in the past few years, bringing numerous benefits and conveniences. Yet, this expansion has also underscored ethical concerns, including issues such as hallucinations, toxic content, biased data and other unintended consequences. While the governance of these risks has garnered attention, a comprehensive and rigorous analysis of ethical evaluation connected to LLMs remains lacking. Against the background, this paper conducts an analysis of 105 assessment tools developed by governmental agencies, academic institutions, research groups, and technology corporations. The findings reveal a convergence emerging of these assessment principles, primarily focusing on data ethic, bias, discrimination and fairness, safety, robustness, human preferences alignment, particular ethical scenarios, responsibility, transparency and interpretability, and public participation. The study also presents the limitations of current ethical assessments paired with a critical analysis. This involves considering the collaboration between various institutions while taking into account the general public, the necessity of incorporating multidimensional real-world ethical contexts and related datasets, and the importance of integrating worldwide AI ethics guidelines with the ethical evaluation of LLMs. Such optimization can be incorporated into future evaluation efforts, aligning the technical advancements of LLMs with ethical considerations.