International Journal of Innovative Research in Computer and Communication Engineering
ISSN Approved Journal | Impact factor: 8.771 | ESTD: 2013 | Follows UGC CARE Journal Norms and Guidelines
| Monthly, Peer-Reviewed, Refereed, Scholarly, Multidisciplinary and Open Access Journal | High Impact Factor 8.771 (Calculated by Google Scholar and Semantic Scholar | AI-Powered Research Tool | Indexing in all Major Database & Metadata, Citation Generator | Digital Object Identifier (DOI) |
| TITLE | MedQLoRA: Quantized Low-Rank Adaptation for Efficient Medical Text Summarization |
|---|---|
| ABSTRACT | Medical text summarization plays a crucial role in reducing the time required for healthcare professionals and researchers to review large volumes of medical documents. Recent Large Language Models (LLMs) have achieved promising results in text summarization tasks; however, their fine-tuning often requires substantial computational resources, memory, and storage, limiting their deployment in resource-constrained environments. To address this challenge, this work proposes MedQLoRA, a quantized low-rank adaptation framework for efficient medical text summarization. The proposed approach leverages Quantized Low-Rank Adaptation (QLoRA) to fine-tune pre-trained language models while significantly reducing the number of trainable parameters and GPU memory requirements. This study evaluates the effectiveness of MedQLoRA on medical text summarization datasets and compares its performance with conventional Low-Rank Adaptation (LoRA) and full fine-tuning approaches. In addition to standard evaluation metrics such as ROUGE and BERTScore, the study analyzes training efficiency, memory consumption, and inference speed to provide a comprehensive assessment of practical deployment feasibility. The major contributions of this work are threefold: (1) the development of a memory-efficient QLoRA-based framework for medical text summarization, (2) a systematic comparison of QLoRA, LoRA, and full fine-tuning under resource-constrained settings, and (3) the establishment of practical guidelines for selecting efficient adaptation strategies in healthcare natural language processing applications. |
| AUTHOR | M. PAVANI SAI, K. VASANTHA Department of Computer Science and Engineering, St. Mary's Women's Engineering College, Guntur, Andhra Pradesh, India |
| VOLUME | 186 |
| DOI | DOI: 10.15680/IJIRCCE.2026.1407016 |
| pdf/16_MedQLoRA Quantized Low-Rank Adaptation for Efficient Medical Text Summarization.pdf | |
| KEYWORDS | |
| References | [1] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” Proc. ICLR, 2022. [2] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” Proc. NeurIPS, 2023. [3] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” Journal of Machine Learning Research, vol. 21, 2020. [4] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, et al., “Scaling Instruction-Finetuned Language Models,” arXiv:2210.11416, 2022. [5] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-Efficient Transfer Learning for NLP,” Proc. ICML, 2019. [6] X. L. Li and P. Liang, “Prefix-Tuning: Optimizing Continuous Prompts for Generation,” Proc. ACL-IJCNLP, 2021. [7] C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” Proc. ACL Workshop on Text Summarization Branches Out, 2004. [8] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating Text Generation with BERT,” Proc. ICLR, 2020. [9] T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer, “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale,” Proc. NeurIPS, 2022. [10] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” Proc. ACL, 2020. |
