Skip to main content
Use this guide when your website, sitemap, repository, or files change and the chatbot must follow. Old content gives old answers. SiteGPT keeps content fresh in three ways:
  • Auto-sync reads a website, sitemap, or GitHub import again on a schedule.
  • Sitemap scan checks a sitemap for added and removed URLs.
  • Resync refreshes the items you select, right now.

​
Before you start

  • Auto-sync needs the Growth plan or above. A sitemap scan schedule needs the Scale plan or above. Which schedules you can pick depends on your plan. See Plans and limits.
  • The Super Admin, Admin, or Manager role on the chatbot.

​
What refreshes automatically

The dashboard cannot schedule links. With the CLI or the API, you can give a single document, such as a link, its own refresh schedule. See knowledge documents update-config in the CLI command reference.

​
Set an auto-sync schedule

You set the schedule when you add a website, sitemap, or GitHub repository. Use the Auto Sync Frequency field in the form. See Add your website and Connect GitHub. Auto Sync Jobs lists only imports that have a sync schedule. If you added an import with Never (Manual only), it is not in the list. To give it a schedule, add it again with an Auto Sync Frequency and turn on Skip pages that are already trained. To change the schedule of an import in Auto Sync Jobs:
1

Open the sync jobs

In the chatbot sidebar, select Auto Sync Jobs.
2

Open the job

Select the job for the website, sitemap, or repository.
3

Open the settings

Select the settings icon.
4

Pick a frequency

In Sync Frequency, pick a frequency. The change applies from the next scheduled run.

​
When scheduled runs happen

The Growth plan includes Monthly. The Scale plan and above also include Weekly. The dashboard also lists Daily, but no plan includes daily auto-sync. If your content changes more often than your schedule, resync it after each change. A changed schedule does not start a run right away. To refresh now, use Resync.

​
What auto-sync does

  • It reads the website, sitemap, or repository again.
  • It adds new pages it finds and refreshes the pages it finds again.
  • It never deletes content. A page that was removed from your website stays in the chatbot. See Remove pages that no longer exist.
  • It uses your page quota for new pages. Refreshing a page that is already trained works even when you have no pages left. Its page count changes only if the page got longer or shorter.

​
Scan a sitemap for new and removed pages

A sitemap scan reads your sitemap again and compares it with the pages SiteGPT has:
  • It adds URLs that are new in the sitemap.
  • It deletes URLs that are no longer in the sitemap.
  • It deletes pages only when it read the whole sitemap. If part of the sitemap cannot be read, for example a linked sitemap fails to load, the scan adds new URLs but deletes nothing. The next scan tries again.
  • It does not refresh pages that did not change in the sitemap. Use auto-sync or resync for that.
To set a scan schedule, use Auto Scan Frequency when you add the sitemap. To change it later, open the job in Auto Sync Jobs, select the settings icon, and pick a Scan Frequency. Scan Frequency and Scan now show only for sitemap jobs, and only sitemaps with a sync schedule are listed in Auto Sync Jobs. A daily scan runs every day at 00:00 UTC. A weekly scan runs every Sunday at 00:00 UTC. A monthly scan runs on the 1st of each month at 00:00 UTC. To scan now, open the job, select the settings icon, and select Scan now. A message confirms that the scan started. New pages show up as they train. Scan now works even if no scan schedule is set. If a job has the same frequency for sync and scan, only the sync runs on that schedule. The scan does not run, so removed pages stay. To remove pages on a schedule, pick a scan frequency that differs from the sync frequency.

​
Resync content now

Resync fetches the selected items again with their saved settings. It does not look for new pages.
1

Open the list

Go to Website Links > Links List or Files & Data Sources > Files List.
2

Select the items

Select the check box next to each link or file.
3

Resync

Select Resync N Selected and confirm.
To resync every item with one status, use the resync button on a status card, such as Failed. To find new pages on a website without a schedule, add the website again with Skip pages that are already trained turned on.

​
Remove pages that no longer exist

Website auto-sync does not remove pages that you deleted from your site. A resync of a deleted page marks it Failed with a 404 error. Remove those pages by hand:
  1. In Links List, select the pages that no longer exist.
  2. Select Delete N Selected and confirm.
For a sitemap import, a sitemap scan removes URLs that you took out of the sitemap. For uploaded files and connected app files, delete old files in Files List.

​
Stop a schedule

  1. In Auto Sync Jobs, open the job.
  2. Select the settings icon.
  3. Select Delete Sync Job and confirm.
Deleting a sync job stops its sync and scan schedules, and the job leaves the list. The content it added stays in the chatbot. To remove the content, delete it in Links List or Files List.

​
Check that it works

In Auto Sync Jobs, open a job. The job details show Last Run, Next Run, and Total Runs. After a run, ask the chatbot about content you changed.