Unveiling the Power of Large Language Models: A Comparative Study of Retrieval-Augmented Generation, Fine-Tuning and Their Synergistic Fusion for Enhanced Performance

Budakoglu G., Emekci H.

IEEE Access, vol.13, pp.30936-30951, 2025 (SCI-Expanded) identifier

  • Publication Type: Article / Article
  • Volume: 13
  • Publication Date: 2025
  • Doi Number: 10.1109/access.2025.3542334
  • Journal Name: IEEE Access
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC, Directory of Open Access Journals
  • Page Numbers: pp.30936-30951
  • Keywords: fine-tuning, hybrid models, Large language models (LLMs), performance optimization, retrieval-augmented generation (RAG)
  • TED University Affiliated: Yes

Abstract

Large language model optimization for a particular application is both crucial and challenging in natural language processing. This paper compares two salient techniques for this Retrieve-augmented generation and fine-tuning along with a new hybrid method that combines both. In this work, we investigate the effectiveness of various methods using on SQuAD (the Stanford Question Answering Dataset), MS MARCO (Microsoft MAchine Reading Comprehension) and SQL CREATE TABLE statements. RAG is used because it enriches model responses with external data without much computational load during inference. Fine-tuning updates model parameters for better contextual accuracy. Our hybrid model balances these two techniques in terms of both accuracy and efficiency. While fine-tuning entails semantic precision, RAG is more resource efficient. The hybrid approach while it may not offer surpassing results over fine-tuning-offers a balanced solution in scenarios where the application demands both efficiency and accuracy. These findings represent the trade-off involved in the LLM optimization and offer scope for further studies and practical applications.