cs.CL 2411.13676

Hymba: A Hybrid-head Architecture for Small Language Models

Hymba employs a hybrid-head architecture combining transformer attention and state space models, achieving superior efficiency and performance with fewer parameters, surpassing comparable small models.

Xin Dong, Yonggan Fu, Shizhe Diao et al.

2024-11-21 109 citations 30
cs.IR 2411.13322

Scaling Laws for Online Advertisement Retrieval

Proposes offline R/R* metric and scaling law for online ad retrieval models, enabling cost-effective performance prediction.

Yunli Wang, Zhen Zhang, Zixuan Yang et al.

2024-11-20 59
cs.CL 2411.10636

Mitigating Extrinsic Gender Bias for Bangla Classification Tasks

Proposes RandSymKL, combining symmetric KL and cross-entropy, to mitigate extrinsic gender bias in Bangla classification tasks, achieving 90.66% accuracy and bias reduction.

Sajib Kumar Saha Joy, Arman Hassan Mahy, Meherin Sultana et al.

2024-11-16 63