Harnessing Large Language Models Over Transformer Models for Detecting Bengali Depressive Social Media Text: A Comprehensive Study

Depression is among the most prevalent mental illnesses globally, yet diagnosis remains challenging due to stigma, resource limitations, and subjective assessment methods. Recent advances in NLP and machine learning have enabled automated detection of depressive symptoms from social media texts, with models like Support Vector Machines, Naive Bayes, and deep neural networks demonstrating varying degrees of success. Transformer-based models such as BERT and its multilingual variants have further improved accuracy by capturing contextual semantics. However, most prior work focused on English or resource-rich languages, leaving low-resource languages like Bengali underexplored. The emergence of large-scale pre-trained models such as GPT-3 and GPT-4 has revolutionized NLP, offering unprecedented transfer learning capabilities. This study builds on these developments, aiming to adapt LLMs for Bengali depression detection, a language with limited annotated datasets and unique linguistic features. By combining traditional models, deep learning, transformers, and LLMs, the research seeks to establish a comprehensive framework for low-resource mental health analysis, addressing linguistic, cultural, and technical challenges.

The core challenge is to accurately classify Bengali social media texts into depressive or non-depressive categories, despite linguistic complexities and data scarcity. Existing models struggle with subtle expressions, sarcasm, and cultural nuances. Moreover, low-resource settings hinder the effective training of large models from scratch. The problem is compounded by the need for models that can perform well with minimal labeled data, leveraging zero-shot and few-shot learning. Developing a robust, scalable, and interpretable system capable of early depression detection in Bengali social media content is crucial for public health initiatives. Addressing these issues requires innovative model architectures, transfer learning strategies, and high-quality datasets, which this study aims to provide.

This work introduces several innovations: 1) a fine-tuned DepGPT model based on GPT-3.5, optimized for Bengali depression detection; 2) a comprehensive comparison of deep learning, transformer, and large language models in low-resource scenarios; 3) the creation of a high-quality, expert-annotated Bengali social media depression dataset (BSMDD); 4) application of zero-shot and few-shot learning techniques to low-resource language tasks. These innovations enable models to achieve high accuracy with limited labeled data, overcoming resource constraints. The integration of expert translation, annotation, and advanced fine-tuning strategies distinguishes this work from prior studies, providing a scalable solution adaptable to other low-resource languages and mental health applications.

• �� Data collection: Extract social media posts from Reddit and X, focusing on depression-related subreddits and general tweets.
• �� Annotation: Experts translate and label posts as depressive or non-depressive, ensuring high-quality, culturally relevant labels.
• �� Preprocessing: Remove duplicates, short texts (<30 words), English characters, hashtags, URLs, and symbols; tokenize, remove stop words, and apply stemming.
• �� Dataset split: Divide into 80% training and 20% testing, maintaining class balance.
• �� Model training: Implement traditional classifiers (SVM, Naive Bayes), deep models (LSTM, Bi-LSTM, GRU), transformer models (BERT, BanglaBERT, SahajBERT), and large models (GPT-3.5, GPT-4, DepGPT).
• �� Fine-tuning: Adapt GPT-3.5 to depression detection task via supervised learning, optimizing hyperparameters.
• �� Evaluation: Use accuracy, precision, recall, F1-score; compare zero-shot and few-shot performance across models.

The dataset comprises 28,000 balanced samples, split into training and testing sets. Models are trained on 80% of data, tested on 20%. DepGPT, fine-tuned from GPT-3.5, achieves an accuracy of 97.96%, F1-score of 98.04%. Baseline models like GPT-3.5 Turbo and Alpaca LoRA 7B show lower performance in zero and few-shot scenarios. The experiments include ablation studies on embedding strategies, model architectures, and learning paradigms. Evaluation metrics focus on accuracy, F1-score, recall, and precision, validating the effectiveness of the proposed approach. The results demonstrate the superiority of DepGPT in low-resource settings, confirming its potential for real-world deployment.

DepGPT achieves 97.96% accuracy and 98.04% F1-score, outperforming all baselines, especially in zero-shot and few-shot contexts. SahajBERT and Bi-LSTM with FastText embeddings also perform well, validating the effectiveness of combining pre-trained transformers with traditional embeddings. The models generalize effectively across linguistic variations, confirming the robustness of the approach. These findings highlight the transformative potential of large models in low-resource mental health detection, setting new benchmarks for Bengali NLP tasks.

The developed models can be integrated into social media monitoring tools for early depression screening, providing real-time alerts to mental health authorities. They can assist in public health surveillance, identifying high-risk regions or populations. In clinical settings, such tools can support mental health professionals by pre-screening large volumes of social media data, enabling targeted interventions. Future integration with multimodal data (voice, images) could further enhance detection accuracy and user engagement, making mental health support more accessible and scalable.

Despite high performance, models may misclassify nuanced or sarcastic expressions, limiting reliability in complex contexts. Dataset size and diversity remain limited, affecting generalization. High computational costs for training and deployment pose practical challenges. Cultural and linguistic variations within Bengali-speaking populations may reduce model accuracy. Further research is needed to improve interpretability, reduce bias, and validate models in real-world clinical environments.