Schedules

FlowHunt's Schedules feature lets you periodically crawl domains and YouTube channels, keeping chatbots updated with fresh information. Choose crawl type, frequency, and URL to automate data retrieval. Ideal for maintaining relevant AI-driven interactions.

Schedules

Schedules allow you to crawl your domains and YouTube channels periodically. Crawling means that AI will go over your website or YouTube channel, and record the information for later use.

This essential tool keeps your flows and chatbots smart, relevant, and always freshly informed. Simply set it and forget it. Your chatbot will learn at scale and always provide users with up-to-date information.

To use the information from Schedules in your Flow, use the Document Retriever component.

How to add a Schedule

  • Navigate to the Schedules screen and click Add Schedule:
How to add Schedule in Flowhunt
  • A pop-up will appear:
Add new Schedule in Flowhunt
  • Pick the type of crawl:
    • Domain crawl – Goes through your entire domain, learning from every page.
    • Single URL crawl – Good for pages that need daily crawling or for adding external sources in a controlled way.
    • Sitemap crawl – Learns information about your pages, linked media, and files to understand your site better and crawl it more efficiently.
    • YouTube channel – Index the closed captions from your YouTube videos.
  • FlowHunt uses video captions to index the content of YouTube videos. However, YouTube’s auto-generated captions aren’t always accurate. If you want to ensure your Chatbot has the right information, we suggest you take the time to create the captions yourself, or at least check the auto-generated ones.

    YouTube’s auto-generated captions aren’t always accurate. To ensure your Chatbot has the right information, check the auto-generated captions or take the time to create the captions yourself.

  • Pick the frequency – how often is the schedule repeated:
    • Daily
    • Weekly
    • Monthly
    • Yearly

Remember that crawling costs credits, and larger schedules can get quite pricey. When choosing the frequency, consider how often you update the data and how important it is for the flow to have the latest information. You probably won’t need your whole domain crawled daily, but you might benefit from daily crawls of frequently updated URLs.

  • Input the URL. Based on the type of crawl, you will want to use these URL structures:
    • Domain crawl: https://www.example.com
    • Single URL crawl: https://www.example.com/blog/article1
    • Sitemap crawl: https://www.example.com/page-sitemap.xml
  • Click Add New Schedule.

Managing Schedules

Your schedule has been created and will show as pending. To see the progress, refresh the page:

Schedules overview in Flowhunt

The crawling shouldn’t take more than a couple of minutes. In this example, we’re doing a single URL crawl, which takes less than five seconds. If your schedule runs into an error, just hover over the red ‘Error’ status tag to learn more.

The Schedule can always be edited or deleted. Once finished, you don’t need to wait for the next scheduled time. If you need to, you can re-run the schedule manually by clicking the repeat icon next to the status tag:

You can easily check the crawled URLs, ensuring all the necessary information is gathered. To do so, click Show found URLs. This will show you a list of all the URLs crawled within the schedule. To see more about a specific URL, click Details:

To go even deeper, click Show URL Tree in the detail of a URL. This will allow you to see the page structure by headlines. Clicking one will reveal the content:

Frequently Asked Questions

  • What is the Schedules feature?

    Schedules allow you to power your Chatbots and Flows with your knowledge sources by periodically crawling websites or single URLs.

  • How frequent should the crawling be?

    It depends on your use case and needs. However, crawling bigger (100+) domains too often can get expensive quickly. We recommend only using daily schedules for single URLs that you know change frequently. You can also run a schedule manually whenever you need to.

  • How do I check if the crawling was successful?

    The status and number of crawled URLs in the list inform you of any possible errors. To see the exact URLs crawled, click Show Found URLs. You can go even deeper by clicking Show URL Tree in the detail of a URL. This will reveal the page structure and all the content.

  • How do I make the Flow use information from my Schedules?

    Use the Document Retriever component to allow the Flow to look up information in crawled URLs.

Discover how a Webpage Content GAP Analysis can boost your SEO by identifying missing elements in your content. Learn to enhance your webpage's ranking with actionable insights and competitor comparisons. Visit FlowHunt for more details.

Webpage Content GAP Analysis

Boost your SEO with FlowHunt's Webpage Content GAP Analysis. Identify content gaps, enhance ranking potential, and refine your strategy.

Discover FlowHunt's AI-driven templates for chatbots, content creation, SEO, and more. Simplify your workflow with powerful, specialized tools today!

Templates

Discover FlowHunt's AI-driven templates for chatbots, content creation, SEO, and more. Simplify your workflow with powerful, specialized tools today!

Generate perfect SEO titles effortlessly with FlowHunt's Web Page Title Generator. Input your keyword and let AI create optimized titles for you!

Web Page Title Generator Template

Generate perfect SEO titles effortlessly with FlowHunt's Web Page Title Generator. Just input a keyword and get top-performing titles in seconds!

Learn from the top-ranking content on Google. This Tool will generate high-quality, SEO-optimized content inspired by the best.

Top Pages Content Generator

Generate high-quality, SEO-optimized content by analyzing top-ranking Google pages with FlowHunt's Top Pages Content Generator. Try it now!

Our website uses cookies. By continuing we assume your permission to deploy cookies as detailed in our privacy and cookies policy.