Semantic Knowledgebase Search

This workflow, titled “Semantic Search”, enables users to search for information within their private knowledgebase by leveraging advanced language models and semantic search techniques. It is designed to scan across all scheduled domains, documents, and Q&A sections, automating the retrieval of the most relevant information in response to user queries.

User Interaction and Welcome Message

When a user opens the chat interface, the workflow triggers a welcome message:

• Message Widget displays:

  👋 Welcome to the Private Knowledgebase Search Tool!
  I’m here to help you search through documents in your private knowledgebase 📚. I’ll scan all scheduled domains, private documents, and Q&A sections to find the information you need.
  Simply enter your query, and let’s get started on finding the answers! ✨🔍

This friendly message helps orient users and guides them to enter their search query.

Query Processing and Expansion

1. User Input:
   The user submits a query via the chat input field.

2. Query Expansion:

   • The query is sent to a Query Expansion component.
   • Powered by an OpenAI language model (specifically, gpt-4o-mini), this component generates up to three paraphrased or semantically similar queries.
   • The purpose is to increase the chances of retrieving all relevant documents, even when the initial query wording is ambiguous or limited.

Document Retrieval

• The expanded queries are passed to a Document Retriever.
• This component searches the user’s private knowledgebase, including scheduled domains, documents, and Q&A sections.
• It pulls up to 10 of the most relevant documents, focusing on content within <H1> headers to maximize context relevance.

Results Presentation

• The retrieved documents are fed into a Document Widget, which formats and presents them in a chat-friendly way.
• The final compiled results are displayed back to the user in the chat interface.

Workflow Diagram

flowchart LR
    A[Chat Opened] --> B[Welcome Message]
    B --> C[User Query Input]
    C --> D[Query Expansion\n(OpenAI LLM)]
    D --> E[Document Retriever]
    E --> F[Document Widget]
    F --> G[Chat Output]

Benefits and Use Cases

• Automation: The workflow automates semantic search, saving manual effort and ensuring users always receive a friendly, guided experience.
• Scalability: By expanding queries and searching across all relevant sources, the workflow provides robust coverage, making it suitable for large or complex knowledgebases.
• Accuracy: Leveraging LLMs for paraphrasing reduces the risk of missing information due to how a query is worded.
• User Experience: Immediate feedback and clear instructions make the tool user-friendly, even for non-technical audiences.

Typical use cases:

• Internal knowledge management for support teams
• Company-wide FAQ and document search portals
• Automated assistants for private or proprietary datasets

By integrating semantic search with LLM-powered query expansion, this workflow ensures users can efficiently access relevant knowledge, boosting productivity and information discovery.