How it works
When you add content from URLs, sitemaps, websites, or YouTube channels, you can enable automatic syncing to:- Detect new pages or videos
- Update modified content
- Remove deleted content
- Keep your chatbot’s knowledge current
Sync frequencies
Choose how often to sync your content:Daily sync
Updates content every 24 hours. Best for:- Frequently updated blogs
- News sites
- Documentation that changes often
- Active YouTube channels
Weekly sync
Updates content every 7 days. Best for:- Marketing websites
- Product pages
- Knowledge bases with moderate updates
- Educational content
Monthly sync
Updates content every 30 days. Best for:- Static websites
- Infrequently updated content
- Large content libraries
- Cost-conscious implementations
Manual only
No automatic syncing. You control when to update content manually.Setting up auto-sync
For new content
When adding content:1
Add your content source
Go to Content and add URLs, sitemap, website, or YouTube channel.
2
Choose sync frequency
Select Daily, Weekly, Monthly, or Manual from the dropdown.
3
Complete setup
Finish adding your content. Auto-sync will begin based on your selected frequency.
For existing content
To enable auto-sync on existing content:1
Go to auto-sync jobs
Navigate to Content > Auto-Sync Jobs in your dashboard.
2
Find your content
Locate the content source you want to auto-sync.
3
Update frequency
Click the sync frequency dropdown and select your preferred schedule.
4
Save changes
The new sync schedule takes effect immediately.
Managing sync jobs
View all sync jobs
The Auto-Sync Jobs page shows:- All content sources with auto-sync enabled
- Current sync frequency
- Last sync date and time
- Next scheduled sync
- Sync status (successful, failed, in progress)
Filter by source type
Filter jobs by content type:- URLs
- Sitemap
- Website
- YouTube
Filter by frequency
View jobs by sync schedule:- Daily
- Weekly
- Monthly
- Manual only
Sync statistics
Track sync performance:- Total jobs - Number of auto-sync jobs configured
- Active jobs - Jobs with automatic syncing enabled
- Successful - Jobs that completed successfully
- Running - Jobs currently in progress
Sync job details
Click on any sync job to view:Job information
- Content source URL or identifier
- Sync frequency
- Date created
- Last modified
Sync history
- All previous sync runs
- Start and end times
- Status (completed, failed, in progress)
- Pages/videos processed
- Changes detected
Sync results
- New content added
- Updated content
- Removed content
- Errors encountered
Troubleshooting
Sync failures
If a sync job fails: Check source availabilityEnsure the website or content source is accessible and hasn’t moved. Review error messages
Click on the failed sync to see specific error details. Verify permissions
Confirm you have access to the content source (e.g., YouTube channel is public). Check rate limits
Some websites may block frequent crawling. Consider reducing sync frequency.
Missing content
If new content isn’t appearing: Wait for next syncContent updates only during scheduled syncs or manual triggers. Check sync status
Verify the last sync completed successfully. Review content filters
Ensure your content isn’t being filtered out by URL patterns or exclusions. Trigger manual sync
Force an immediate sync to test if content is detected.
Slow syncs
If syncs take too long: Reduce scopeLimit the number of pages or videos being synced. Increase frequency
More frequent syncs process fewer changes each time. Check content size
Large pages or videos take longer to process. Review server response
Slow source websites delay sync completion.
Best practices
Choose appropriate frequencies
- Daily - Only for content that changes daily
- Weekly - Good balance for most use cases
- Monthly - Sufficient for stable content
- Manual - For complete control over updates
Monitor sync health
Check your Auto-Sync Jobs page regularly to:- Identify failed syncs
- Review sync patterns
- Optimize sync schedules
- Remove unnecessary jobs
Optimize content sources
- Use sitemaps instead of full website crawls when possible
- Limit URL lists to essential pages
- Exclude irrelevant sections of your website
- Remove outdated content sources
Balance freshness and costs
More frequent syncing:- Keeps content more current
- Uses more processing resources
- May hit rate limits
- Increases costs on higher-tier plans
Pricing
Auto-sync availability depends on your plan:- Daily sync - Available on Pro and Enterprise plans
- Weekly sync - Available on all paid plans
- Monthly sync - Available on all paid plans
- Manual only - Available on all plans including Free