PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
Constructed PKU-SafeRLHF dataset with decoupled helpfulness and harmlessness annotations, covering 19 harm categories and 3 severity levels, to improve LLM safety alignment.
Jiaming Ji, Donghai Hong, Borong Zhang et al.