DocTalkBN: A Novel Dataset of Expert Telemedicine Conversations in Bengali

Recent advances in NLP have significantly impacted healthcare, enabling automated analysis of clinical notes, medical literature, and patient-provider dialogues. Datasets like MTS-Dialog and MEDIQA-Chat have facilitated progress but are limited to structured texts or synthetic dialogues, lacking the spontaneity of real conversations. For low-resource languages such as Bengali, the scarcity of large-scale, authentic clinical dialogue data remains a major obstacle. Existing resources focus on entity recognition or summarization from forums or articles, which do not reflect natural speech patterns or contextual complexity. This gap hampers the development of culturally and linguistically appropriate medical AI systems, especially for telemedicine applications in underserved regions. The need for authentic, multimodal datasets that capture real doctor-patient interactions in low-resource settings is urgent to improve model robustness, interpretability, and safety.

The core challenge lies in collecting high-quality, authentic medical dialogues in Bengali that encompass spontaneous speech, socio-cultural nuances, and multimodal cues. Existing datasets are either limited in scale, restricted to text, or synthetic, failing to represent real clinical interactions. This limits models' ability to understand colloquial language, contextual shifts, and sensitive topics, which are crucial for safe and effective AI-assisted healthcare. Moreover, manual annotation is costly and time-consuming, especially for low-resource languages. Addressing these issues requires innovative data collection pipelines that leverage existing media, combine automatic filtering with human validation, and ensure data privacy and clinical relevance. Overcoming these hurdles is essential to develop AI systems capable of nuanced clinical reasoning and safe decision-making in Bengali-speaking populations.

The key innovations include: 1) constructing the first large-scale, authentic Bengali medical dialogue dataset from televised telemedicine programs, capturing spontaneous speech and socio-cultural context; 2) employing a hybrid pipeline combining LLM-based filtering, speech recognition, and multi-round human validation to ensure data quality; 3) developing a structured dialogue reconstruction process with Gemini-3-Flash, enabling precise speaker segmentation and turn delineation; 4) designing three downstream benchmarks—triage, safety, and NER—using expert-guided annotation and LLM-assisted curation, facilitating comprehensive evaluation of clinical reasoning models. These innovations address the limitations of prior datasets, which lacked authenticity, multimodality, and scale, especially in low-resource languages.

• �� Data collection: sourced from publicly available telemedicine videos, applying Qwen3-30B-Instruct to filter relevant content. • Subtitle extraction: obtained auto-generated Bangla subtitles, processed to remove noise and artifacts. • Preprocessing: segmenting long transcripts, normalizing text, removing advertisements, and anonymizing personal data using BNLP Toolkit. • Validation: multiple expert annotators reviewed transcripts for fidelity, clarity, and medical content accuracy. • Dialogue structuring: Gemini-3-Flash identified speakers, segmented turns, and classified interactions into doctor-patient or host-doctor exchanges. • Annotation: multi-task labels generated for triage, safety, and entity recognition, with iterative human validation to ensure correctness. • Benchmark creation: used LLM prompts to generate downstream datasets, followed by expert review and correction, ensuring clinical relevance and data quality.

Experiments employed 80/10/10 splits across training, validation, and testing sets. Models like GPT-4o, Llama-3-70B, and encoder-based baselines (BanglaBERT, mmBERT) were fine-tuned or prompted for each task. Evaluation metrics included macro-F1, precision, and recall. Hyperparameters such as batch size 16, 100 training epochs, and early stopping were used. Ablation studies examined the impact of multimodal features and validation procedures. Results demonstrated that models trained on DocTalkBN significantly outperformed baselines on all tasks, with Llama-3-70B achieving F1 0.463 on triage, and GPT-4o reaching F1 0.976 on advice safety, confirming the dataset’s robustness and practical value.

The dataset’s scale and authenticity led to notable improvements in downstream tasks. Llama-3-70B achieved an F1 of 0.463 in triage, indicating effective clinical reasoning. GPT-4o excelled in advice safety, with F1 0.976, demonstrating high reliability in safety assessments. Entity recognition models like Qwen-3.5-Flash achieved strict F1 scores around 0.743, validating the dataset’s utility for structured information extraction. These results highlight the importance of real-world, multimodal data in training models capable of nuanced understanding and decision-making in low-resource clinical settings.

The dataset enables training robust models for telemedicine, remote diagnostics, and AI-powered clinical decision support, especially in Bengali-speaking regions. It supports developing culturally sensitive, safe, and interpretable AI systems that can assist doctors and empower patients. Long-term, the dataset paves the way for multilingual, multimodal healthcare AI, bridging disparities and improving access to quality care globally.

The data, sourced from televised programs, may not fully represent diverse patient populations or complex cases. Automatic speech recognition introduces noise, affecting transcript quality. Model performance on rare or highly sensitive cases remains limited, necessitating further data collection, multi-region validation, and clinical expert involvement to enhance safety and generalization.