Tutorial: Build a Knowledge Base Q&A Assistant

Using “build a Q&A assistant based on internal policy documents for the organization” as the example, this tutorial walks the full flow — create the Knowledge Base, verify retrieval, orchestrate a Chatflow, debug and test, publish and go live. It is for developers and admins building their first conversational app on LanSphere.

Scenario and Value

An organization’s rules and management measures usually scatter across many documents, and day-to-day inquiries lean heavily on manual answering:

• Employees flip through multiple files when they have a question — hard to find, slow to read;
• HR and administration answer the same questions over and over, consuming huge effort;
• Different people answer differently, inviting disputes;
• After a new document is issued, old answers keep circulating — information lags.

Put the policy documents into a Knowledge Base and let a Chatflow answer from it: employees ask self-service questions anytime, answers are traceable and consistent, and HR and administration are freed from repetitive Q&A.

What You Will Build

A “Policy Q&A Assistant” Chatflow app: employees ask in a conversation; the app retrieves relevant content from the policy Knowledge Base, the LLM composes the answer and marks the cited sources; when nothing is found, it says so plainly instead of making things up. In the end, publish the app to the App Square and list it in the Lansenger client’s “AI Assistant Marketplace,” so every employee can use it inside Lansenger.

The overall chain: Knowledge Base (policy documents) → Chatflow (Input → Knowledge Base → LLM → Reply) → Publish (App Square + Lansenger AI Assistant).

Recommended Flow

Preparation

Step 1: Create the Knowledge Base

1. Go to “left menu → Knowledge Base,” click “Create Knowledge Base,” and pick the “Simple” build mode (standard creation).
2. Basic info: Fill in the name (required, up to 30 characters — e.g., “Org Policy Knowledge Base”) and description (optional).
3. Upload files: Upload the policy PDF files.
4. Chunking strategy: Pick the preset “General Document Q&A.” This strategy is Parent-Child Chunking · Paragraph with max child chunk 512 — for policy clauses organized by paragraph, no manual tuning needed.
5. Parameter confirmation: With Parent-Child Chunking, the indexing method is locked to “Vector Indexing”; pick an Embedding Model (required), and a Rerank Model as well for better retrieval precision.
6. Confirm creation: The page shows per-file progress; once everything succeeds, click “Enter Knowledge Base.”

Note: Once Vector Indexing is chosen, you cannot switch back to Keyword Indexing; files appended to this Knowledge Base later automatically inherit the chunking mode from creation — unchangeable.

Step 2: Verify the Knowledge Base with a Retrieval Test

Do not rush onto the canvas yet — run a Retrieval Test first to confirm “questions find the right content”:

1. Enter the Knowledge Base you just created and open the Retrieval Test (as prompted on the page).
2. Enter a few typical questions, e.g., “How many days of annual leave?” and “What is the travel accommodation allowance?”
3. Inspect the retrieved chunks, match scores, and source documents:
   • Retrieved chunks do contain the answer → the Knowledge Base is usable; move on;
   • Retrieved chunks are irrelevant or empty → fix the Knowledge Base chunking first (see “Verify and Tune” at the end) — do not carry problems onto the canvas.

Tip: Try several phrasings at this stage (colloquial phrasing, abbreviations) — that is how employees really ask.

Step 3: Orchestrate the Chatflow

1. Click “New Project,” pick the app type “Chatflow,” and fill in the app name (e.g., “Policy Q&A Assistant”).
2. On the canvas of the “App Development” tab, the “Input” node exists by default as the conversation entry.
3. Click to add a “Knowledge Base” node, connected after “Input”:
   • Associate Knowledge Base: Pick the “Org Policy Knowledge Base” you just created;
   • Retrieval method: Pick “hybrid retrieval” (semantic + keyword weighted — friendlier to clause numbers and proper nouns);
   • Top K: Keep the default 3;
   • With a Rerank Model configured, enable it for better precision.
4. Click to add an “LLM” node, connected after the “Knowledge Base” node. Pick the chat model, pass the Knowledge Base node’s retrieval results in as context, and write the prompt:

You are the organization's policy Q&A assistant, answering employees' questions about rules and regulations based on Knowledge Base retrieval results.

[Knowledge Base retrieval results]
(Insert the Knowledge Base node's output variable here)

Follow these rules strictly:
1. Answer only from the retrieval results above. Do not use any information beyond them.
2. When the retrieval results contain no answer, reply clearly: "No relevant provision was found in the current policy documents,"
   and suggest the employee consult HR and administration. Never fabricate clauses or numbers.
3. Mark the cited source at the end of the answer, in the format: Source: relevant section of "File Name."
4. Answer in bullet points, in concise, formal language; for questions unrelated to policies, politely explain that this assistant only answers policy questions.

Tip: Insert the context in the prompt via the canvas’s variable picker as the Knowledge Base node’s output variable — the placeholder syntax depends on the actual variable name.

5. Click to add a “Reply” node, connected after the “LLM” node, with its content referencing the LLM node’s output. A Chatflow outputs messages to the user through “Reply” nodes.

Step 4: Configure Conversational Features

Enable the following in the Chatflow’s feature configuration (as prompted on the page):

Step 5: Debug with Typical Questions

1. Use end-to-end debugging for multi-turn conversation tests, inspecting each node’s input, output, and errors every turn; past runs are in “Debug History.”
2. Cover two kinds of cases per the table — what should be answered must be answered right; what should be declined must be declined:

3. Multi-turn follow-up test: ask “How many days of annual leave?” first, then follow up with “What about family visit leave?” — confirm multi-turn memory works and context does not cross.

Step 6: Save and Publish

1. Save: Click “Save” — the draft is visible only to you.
2. Publish to App Square: After saving the latest content, click the “Save” button’s drop-down → “Publish to App Square.” Once listed, everyone on the platform (including Viewers) can see and use it — good for a small-scale pilot and feedback collection first.
3. Publish as a Lansenger AI Assistant (Admin role or above required):
   • Go to the app details page → “Publishing Channels” tab and pick the “Lansenger AI Assistant” channel;
   • Account linked to Lansenger: confirm in the pop-up to publish directly; not linked: follow the pop-up to the Lansenger authorization center — publishing runs automatically once QR-code authentication passes;
   • The assistant title defaults to the app name (within 10 characters) and the description defaults to the app description (within 20 characters) — mind the length limits when naming;
   • After publishing, the button changes to “Published” and is greyed out — no repeated publishing; the system automatically creates the app and bot instance on the Lansenger side and lists it in the Lansenger client’s “AI Assistant Marketplace.”

The employee-side experience: see the assistant’s avatar, title, and description in the Lansenger “AI Assistant Marketplace” and click to enter; first use requires data authorization — chatting starts only after approval; declining makes it unusable, and re-entering re-pushes the authorization card.

Verify and Tune

After going live, review real questions in the “Conversation Logs” tab, mark answers Like/Dislike, add annotated replies, and tune per the table:

Acceptance Checklist

• Policy questions answered accurately, consistent with the source text, with cited sources marked at the end;
• Questions with no answer in the Knowledge Base declined clearly — no fabricated clauses or numbers;
• Multi-turn follow-ups stay coherent, and Citation & Attribution displays properly;
• The assistant is visible in the Lansenger “AI Assistant Marketplace,” and end users can chat normally after data authorization.

Ideas for Extension

• Enable “File Upload” so employees can temporarily upload a file into the Q&A;
• Enable “Suggested Questions” to recommend next questions from the current answer, improving the continuous experience;
• Add an “Intent Recognition” node on the canvas to route policy inquiries and other matters to different branches;
• Integrate the Q&A capability into internal systems via “Publishing Channels → API Access.”