BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
Introduces BeaverTails dataset with disentangled helpfulness and harmlessness annotations, using PPO-Lagrangian for safety RLHF, improving AI safety alignment with 33k QA and 36k preference pairs.
Jiaming Ji, Mickel Liu, Juntao Dai et al.