​
What is a Data node?

A Data node is a knowledge base that you can use to upload documents via our API.
The files uploaded to the Knowledge Base node will be index in a Vector Store.
All created knowledge bases are automatically synced to the Knowledge Base Dashboard.

​
Data Node Settings

If you click the gear icon in the node, you will see the available settings.

​

Available configurations for the Search Parameters:

  • Output Format: Choose between chunks, pages, and docs.
  • Metadata Filter Strategy: Choose between Strict Filter, Loose Filter, and No Filter.
  • Query Strategy: Choose between Semantic, Keyword, and Hybrid.
  • Top Results: Number of search results ranked by relevance.
  • Max Characters: Limits the number of characters sent to the LLM.
  • Answer Multiple Questions: Get the answers from multiple questions in parallel.
  • Advanced Q&A: Handle questions to compare or summarize documents.
  • Rerank: Get more precise information retrieval
  • Query Transformation: Get more precise information retrieval

Available configurations for the Upload Parameters:

  • Embeddings Model: Specifies the embedding model used to convert text into vectors.
  • Chunking Algorithm: Determines how the text is chunked before processing. “Sentence” means the text will be split at sentence boundaries rather than arbitrary character limits. “Naive” splits the text at arbitrary points.
  • Chunk Overlap: Defines how much consecutive chunks overlap in terms of characters. Overlapping ensures contextual continuity when processing text segments. A higher value means more repeated content across chunks.
  • Chunk Length: Sets the maximum number of characters per chunk. A larger chunk size retains more context, while a smaller size improves efficiency but may lose some context.
  • **Advanced Data Extraction: **Allows for more sophisticated extraction techniques, useful for data that includes images, tables, graphs.
  • Text in images (OCR): Optical Character Recognition (OCR) will be applied to extract text from images before processing.
  • Embeddings API Key: Use your own embedding model for processing.

​
Upload Files via API

When you first create a Data node, you will see a button to upload data to the Vector Store API. You can find the API documentation here.

​
How to use the Data Node

After uploading your data, you can query it with using natural language with an input node. Connect its output to an LLM for further processing or to an Output Node to get the extracted data.