This Dify workflow creates a Retrieval-Augmented Generation system: upload documents, chunk and embed them, then let users ask questions with answers grounded in the actual document content, complete with source citations.
TL;DR — Key Takeaways
- RAG Document Q&A System with Dify — Beginner level
- Tool: Dify
- 5 steps with detailed instructions
- Prerequisites: Dify account, documents to upload, OpenAI API key for embeddings
Use Case
Legal teams reviewing contracts, HR departments with policy documents, technical teams with API documentation.
Prerequisites
Dify account, documents to upload, OpenAI API key for embeddings
Workflow Diagram
RAG Document Q&A System with Dify
Step-by-Step Guide
-
Create Knowledge Base
In Dify, create a new Knowledge Base. Upload your documents (PDF, DOCX, TXT, MD). Set chunking to 300 tokens with 50 token overlap. Choose the embedding model (OpenAI text-embedding-3-small is cost-effective).
-
Configure Retrieval
Set retrieval mode to 'semantic search' with top-K=5. Enable reranking if available (Cohere reranker improves accuracy significantly). Set a similarity threshold of 0.7 to filter irrelevant chunks.
-
Create Q&A App
Create a new Completion App. Connect your knowledge base. Set the prompt: 'Answer the user's question using ONLY the provided context. If the answer is not in the context, say I cannot find this information. Always cite the source document and page number.'
-
Add Citation Display
Enable Dify's citation feature. When the model generates an answer, it will show which chunks were used. Configure the UI to display source filenames and relevant passages.
-
Deploy and Monitor
Deploy as a web app or API. Set up logging to track: query volume, answer accuracy (user feedback), and unanswered questions (to identify knowledge gaps). Add new documents based on gaps found.
Tags
This workflow answers:
- How to set up rag document q&a system with dify
- What tools and access you need
- How to troubleshoot common issues