AI Glossary Article Generator
Generate in-depth, SEO-optimized glossary articles by leveraging AI and real-time web research. This flow analyzes the top-ranking content and writing styles, e...
4 min read
The Keyword Frequency Evaluator is a component designed to analyze one or more texts and identify the most significant keywords based on their frequency and other configurable criteria. This makes it particularly useful for extracting core topics, performing content analysis, or preparing data for downstream AI tasks like summarization, clustering, or search.
What Does This Component Do?
This component takes in a list of text inputs (such as documents, messages, or URL records), processes them, and outputs a ranked list of the most frequent and relevant keywords found across the input data. It provides several options to fine-tune the extraction process, such as excluding stopwords, focusing on keywords that appear in multiple texts, and controlling the size and length of extracted keywords.
Inputs
The component offers the following configurable inputs:
| Input Name | Type | Default Value | Description |
|---|---|---|---|
| Input | List of Texts | - | The main text(s) to analyze. Accepts UrlRecord, Message, or Document types. |
| Exclude Stopwords | Boolean | true | If enabled, common stopwords (like “the”, “and”, etc.) are excluded from the results. |
| Just Intersecting | Boolean | true | If enabled, only keywords appearing in more than one input text are returned. |
| Max Keywords | Integer | 50 | The maximum number of keywords to return in the final output. |
| Min Frequency | Integer | 3 | Minimum number of occurrences a word must have to be considered a keyword. |
| Min Word Length | Integer | 3 | Minimum character length a word must have to qualify as a keyword. |
| Qgrams | Multi-select (1-6) | 2, 3, 4 | The size(s) of word sequences (n-grams) to consider when extracting keywords. |
Key Features
- Flexible Input: Works with different types of text records, supporting batch processing.
- Advanced Filtering: Fine-tune the output by excluding stopwords, setting minimum word length, and frequency thresholds.
- Control Output Size: Limit the number of returned keywords to focus on the most prominent terms.
- N-gram Support: Extract multi-word phrases (Q-grams), not just single keywords, for richer semantic insight.
- Intersecting Keywords: Optionally, focus on keywords that are common across multiple documents for comparative analysis.
Output
The component produces the following output:
- Top Keywords:
A list of the most significant keywords (as aMessagetype), based on the extraction logic and the set parameters. This output can be used for further processing or visualization in your AI workflow.
Example Use Cases
- Document Summarization: Quickly identify the main topics in a set of documents.
- Content Clustering: Use extracted keywords to group similar documents or messages.
- Search and Indexing: Generate index terms for efficient retrieval in information systems.
- Trend Analysis: Track recurring themes across communication logs or dataset snapshots.
Why Is This Useful?
The Keyword Frequency Evaluator streamlines the process of extracting meaningful terms from large or multiple bodies of text. By offering detailed configuration, it adapts to various needs — from simple keyword extraction to sophisticated comparative analyses. As part of an AI workflow, it enables downstream components to work with condensed, information-rich representations of your text data, improving efficiency and interpretability.
