PyTorchKR
- 12/18~24의 주요 ML 논문에 소개된 RAG 기술에 대한 서베이 논문을 정리해보았습니다.
- LLM의 활용이 늘어나며, RAG에 대한 연구들 또한 계속되고 있습니다.
- 1부에서는 RAG 기술의 패러다임들을, 2부에서는 주요 구성요소들에 대해서 알아보았습니다. 번외편에서는 저자들이 GitHub 저장소에 공개된 발표 슬라이드(영문)와 참고 기술/코드를 정리한 내용을 소개하려고 합니다.
hoxy... RAG 기술 현황 1편과 2편을 읽고 오셨나요? 아직 읽지 않으셨다면 1편부터 읽으시는 것을 추천드립니다!
RAG 기술 전체 보기
관련 논문 및 코드 목록
Augmentation Stage
Pre-training
1.Improving language models by retrieving from trillions of tokens [paper][code]
2.Few-shot Learning with Re-trieval Augmented Language Models [paper]
3.Toolformer: Language Models Can Teach Themselves to Use Tools[paper]
4.Copy is all you need[paper]
5.In-context learning with retrieval augmented encoder-decoder language model[paper]
6.Shall we pretrain autoregressive language models with retrieval?[paper]
7.Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP[paper]
Fine-tuning
1.Dense Passage Retrieval for Open-Domain Question Answering[paper]
2.UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation[paper][code]
3.Distilling knowledge from reader to retriever for question answering[paper]
4.RA-DIT: Retrieval-Augmented Dual Instruction Tuning[paper]
5.Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection[paper]
6.Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation[paper]
7.Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data [paper] [code]
8.Replug: Retrieval-augmented black-box language models [paper]
9.Augmentation-Adapted Retriever Improves Generalization of Language
Models as Generic Plug-In [paper][code]
Inference
1.Generalization through Memorization: Nearest Neighbor Language Models[paper]
2.DEMONSTRATE–SEARCH–PREDICT:
Composing retrieval and language models for knowledge-intensive NLP [paper][code]
3.Keyword Augmented Retrieval: Novel framework for Information Retrieval integrated with speech interface. [paper]
4.Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. [paper][code]
5.Generate rather than Retrieve: Large Language Models are Strong Context Generators [paper] [code]
6.In-Context Retrieval-Augmented Language Models [paper]
Augmentation Source
Unstructured Data
1.UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation[paper][code]
2.From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL [paper]
3.Copy is all you need [paper]
Structured Data
1.FABULA: Intelligence Report Generation Using Retrieval-Augmented Narrative Construction [paper]
2.Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation [paper]
3.KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases [paper]
4.Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT [paper]
LLM Generated Content
1.Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory [paper]
2.DEMONSTRATE–SEARCH–PREDICT:
Composing retrieval and language models for knowledge-intensive NLP [paper]
3.Recitation-augmented language models[paper]
4.Generate rather than Retrieve: Large Language Models are Strong Context Generators [paper]
5.Self-Knowledge Guided Retrieval Augmentation for Large Language Models [paper]
Augmentation Process
Once Retrieval
1.Retrieval-augmented generation for knowledge-intensive nlp tasks [paper]
2.UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation [paper]
3.Augmented Large Language Models with Parametric Knowledge Guiding [paper]
4.Learning to Retrieve In-Context Examples for Large Language Models.[paper]
5.Few-shot Learning with Re-trieval Augmented Language Models [paper]
6.Replug: Retrieval-augmented black-box language models [paper]
7.Recitation-augmented language models[paper]
Iterative Retrieval
1.DEMONSTRATE–SEARCH–PREDICT:
Composing retrieval and language models for knowledge-intensive NLP [paper][code]
2.Retrieve-and-Sample: Document-level Event Argument Extraction via Hybrid Retrieval Augmentation [paper]
3.Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy[paper]
4.RETRIEVAL-GENERATION SYNERGY AUGMENTED LARGE LANGUAGE MODELS [paper]
Recursive Retrieval
1.Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions [paper][code]
2.Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models [paper]
Adaptive Retrieval
1.Active Retrieval Augmented Generation[paper][code]
2.Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [paper]
3.In-context learning with retrieval augmented encoder-decoder language model [paper]
더 읽어보기
서베이 논문 저자들의 GitHub 저장소
발표 슬라이드 (PDF/영문)
RAG_Slide_ENG.pdf (8.8 MB)
원본 서베이 논문
파이토치 한국 사용자 모임 이 정리한 이 글이 유용하셨나요? 회원으로 가입 하시면 주요 글들을 이메일
로 보내드립니다! (기본은 Weekly지만 Daily로 변경도 가능합니다.)
아래
쪽에 좋아요
를 눌러주시면 뉴스 발행에 힘이 됩니다~ 