Zhang Jiacheng, Liu Zheli, Xiao Guangwen, Nie Lihai, Wang Yongchang, Shi Liang, Jin Meihong
Online available: 2025-12-26
[Objective] To address the fragmentation of value-alignment evaluation systems for large language models, the insufficient coverage of Chinese-specific values, the scarcity of high-quality deep evaluation data, and the lagging evaluation methodologies, this study constructs a methodological framework and toolset for value-alignment assessment tailored to large language models.[Methods] We propose an integrated methodological framework that unifies value rules, evaluation data, and intelligent technologies. Under this framework, we design a three-dimensional evaluation system encompassing “capability–task–indicator,” carry out data collection, augmentation, and expert annotation, and build a systematic deep-evaluation scoring dataset. Ultimately, through pre-training, instruction fine-tuning, and expert-feedback training, we develop a value-alignment evaluation model.[Results] The constructed evaluation model achieves an accuracy of 98.57%, enabling automated assessment of value-alignment levels in large language models. Empirical findings show that domestic models exhibit overall higher alignment than foreign ones, though common issues remain, including insufficient incorporation of red cultural resources, factual and hallucinatory misinformation, weakened ideological expression, over-censorship, and limited dynamic adaptability.[Limitations] The study primarily targets text-based large language models, and its applicability to multimodal models requires further validation. In addition, the evaluation outputs are presented in three tiers—high, medium, and low—leaving room for improvement in interpretability.[Conclusion] This research contributes to improving a value-alignment assessment and governance system with Chinese characteristics, ensuring the healthy development of large language models within a safe, trustworthy, and controllable framework. It also provides essential technical support for effectively implementing mainstream values in China’s economic development and social governance.