> ## Documentation Index > Fetch the complete documentation index at: https://sitegpt.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Add Content to Chatbot > Unified endpoint to add content to a chatbot. Supports multiple source types: - **URLS_LIST**: Add a list of URLs directly - **WEBSITE**: Crawl a website with configurable depth and filters - **SITEMAP**: Extract URLs from an XML sitemap - **YOUTUBE**: Add YouTube video or channel transcripts This endpoint provides full control over crawling behavior, scraping options, and auto-sync configuration. ## OpenAPI ````yaml post /v1/chatbots/{chatbotId}/links openapi: 3.0.0 info: title: SiteGPT API Collection description: >- This collection contains the documentation of all the API endpoints. ## Getting started guide To start using the SiteGPT APIs, you need to - You must use a valid API Key to send requests to the API endpoints. You can get your API key from SiteGPT's [billing dashboard](https://sitegpt.ai/billing). If you don't see it there, contact admin to generate one for you. - The API returns request responses in JSON format. When an API request returns an error, it is sent in the JSON response as an error key. ## Authentication SiteGPT uses API keys for authentication. You can generate a SiteGPT API key in the [billing dashboard](https://sitegpt.ai/billing). You must include an API key in each request to the Postman API with the Authorization request header. Header Name : Authorization Header Value : Bearer {{your-api-key}} ### Authentication error response If an API key is missing, malformed, or invalid, you will receive an HTTP 401 Unauthorized response code. ## Need some help? In case you have any questions, feel free to contact us. license: name: SiteGPT Terms and Conditions url: https://sitegpt.ai/terms version: 1.0.0 contact: name: Bhanu Teja P email: bhanu@sitegpt.ai url: mailto:bhanu@sitegpt.ai servers: - url: https://sitegpt.ai/api security: - bearerAuth: [] tags: - name: Chatbot x-displayName: Chatbots description: >- APIs to manage different chatbots. APIs include `Fetching All Chatbots`, `Creating New Chatbot`, Fetching One Chatbot`, `Updating Chatbot Appearance` and `Deleting Chatbot`. x-mint: icon: message-bot - name: Chatbot Appearance x-displayName: Appearance description: APIs to update appearance of the chatbot. x-mint: icon: palette - name: Chatbot Custom Content x-displayName: Custom Content description: APIs to modify custom content of the chatbot. x-mint: icon: file-pen - name: Chatbot Follow Up Prompts x-displayName: Follow-up Prompts description: >- APIs to manage `Follow up Prompts` of a particular chatbot. APIs include `Fetching All Follow up Prompts`, `Adding New Follow up Prompt`, `Fetching One Follow up Prompt`, `Updating Follow up Prompt` and `Deleting Follow up Prompt`. x-mint: icon: message-arrow-up - name: Chatbot Icons x-displayName: Icons description: APIs to manage (add/update/delete) icons related to a particular chatbot. x-mint: icon: icons - name: Chatbot Links x-displayName: Content & Training description: >- APIs to fetch internal links of any url and to add certain links to a particular chatbot. x-mint: icon: book-open - name: Chatbot Messages x-displayName: Messages description: >- APIs to send a message to a particular thread or start a new conversation with in a particular chatbot. x-mint: icon: messages - name: Chatbot Personas x-displayName: Personas description: APIs to manage personas of the chatbot. x-mint: icon: user-tie - name: Chatbot Prompts x-displayName: Instructions description: APIs to manage prompts of the chatbot. x-mint: icon: message-text - name: Chatbot Conversation Starters x-displayName: Conversation Starters description: >- APIs to manage `Conversation Starters` of a particular chatbot. APIs include `Fetching All Conversation Starters`, `Adding New Conversation Starter`, `Fetching One Conversation Starter`, `Updating Conversation Starter` and `Deleting Conversation Starter`. x-mint: icon: bolt - name: Chatbot Settings x-displayName: Settings description: >- APIs to update different settings of the chatbot. APIs include `General Settings`, `User Data Settings`, `Chat Mode Settings`, `Localization Settings` and `Advanced Settings`. x-mint: icon: gear - name: Chatbot Threads x-displayName: Threads description: >- APIs to manage threads related to a particular chatbot. APIs include `Fetching All Threads`, `Fetching One Thread` and `Update Thread`. x-mint: icon: message-lines - name: Whitelabel x-displayName: Whitelabel (Deprecated) description: >- APIs to manage whitelabel users. APIs include `Create Whitelabel User` and `Update Whitelabel User Quota`. x-mint: icon: tag paths: /v1/chatbots/{chatbotId}/links: parameters: - name: chatbotId description: Id of the chatbot in: path schema: type: string required: true post: tags: - Chatbot Links summary: Add Content to Chatbot description: >- Unified endpoint to add content to a chatbot. Supports multiple source types: - **URLS_LIST**: Add a list of URLs directly - **WEBSITE**: Crawl a website with configurable depth and filters - **SITEMAP**: Extract URLs from an XML sitemap - **YOUTUBE**: Add YouTube video or channel transcripts This endpoint provides full control over crawling behavior, scraping options, and auto-sync configuration. operationId: add-content-to-chatbot-v1 requestBody: required: true description: Request body with source type and configuration content: application/json: schema: oneOf: - type: object description: Add URLs from a list required: - sourceType - config properties: sourceType: type: string enum: - URLS_LIST config: type: object required: - urls properties: urls: type: array minItems: 1 maxItems: 1000 items: type: string format: uri description: List of URLs to add (max 1000) scrapeOptions: $ref: '#/components/schemas/ScrapeOptions' syncFrequency: $ref: '#/components/schemas/SyncFrequency' - type: object description: Crawl a website required: - sourceType - config properties: sourceType: type: string enum: - WEBSITE config: type: object required: - url - maxLinks properties: url: type: string format: uri description: Starting URL for the website crawl recursionDepth: type: integer minimum: 1 maximum: 5 default: 3 description: How many levels deep to crawl (1-5) maxLinks: type: integer minimum: 1 description: Maximum number of pages to scrape includePaths: type: array items: type: string description: Only include URLs matching these path patterns excludePaths: type: array items: type: string description: Exclude URLs matching these path patterns allowedDomains: type: array items: type: string description: Allow crawling these additional domains scrapeOptions: $ref: '#/components/schemas/ScrapeOptions' syncFrequency: $ref: '#/components/schemas/SyncFrequency' - type: object description: Extract URLs from a sitemap required: - sourceType - config properties: sourceType: type: string enum: - SITEMAP config: type: object required: - url - maxLinks properties: url: type: string format: uri description: URL of the XML sitemap maxLinks: type: integer minimum: 1 description: Maximum number of URLs to extract from sitemap includePaths: type: array items: type: string description: Only include URLs matching these path patterns excludePaths: type: array items: type: string description: Exclude URLs matching these path patterns scrapeOptions: $ref: '#/components/schemas/ScrapeOptions' syncFrequency: $ref: '#/components/schemas/SyncFrequency' scanFrequency: $ref: '#/components/schemas/SyncFrequency' - type: object description: Add YouTube video or channel required: - sourceType - config properties: sourceType: type: string enum: - YOUTUBE config: type: object required: - url properties: url: type: string format: uri description: YouTube video or channel URL syncFrequency: $ref: '#/components/schemas/SyncFrequency' examples: URLS_LIST: value: sourceType: URLS_LIST config: urls: - https://sitegpt.ai - https://sitegpt.ai/pricing - https://sitegpt.ai/docs WEBSITE: value: sourceType: WEBSITE config: url: https://sitegpt.ai recursionDepth: 3 maxLinks: 100 includePaths: - /docs - /blog excludePaths: - /admin syncFrequency: WEEKLY SITEMAP: value: sourceType: SITEMAP config: url: https://sitegpt.ai/sitemap.xml maxLinks: 500 syncFrequency: DAILY scanFrequency: DAILY YOUTUBE: value: sourceType: YOUTUBE config: url: https://youtube.com/watch?v=dQw4w9WgXcQ syncFrequency: NEVER responses: '200': description: OK - Content added successfully content: application/json: schema: type: object properties: success: type: boolean message: type: string data: type: object properties: ingestJobId: type: string description: ID of the ingestion job created documentsQueued: type: integer description: Number of documents queued for processing examples: Success: value: success: true message: Added links to the chatbot successfully data: ingestJobId: job_abc123 documentsQueued: 5 '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Validation Error: value: success: false message: Failed to add links to the chatbot data: null error: code: REQUEST_VALIDATION_FAILED message: Request body is not valid details: - path: - config - recursionDepth message: Number must be greater than or equal to 1 '401': description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Authentication Error: value: success: false message: Failed to add links to the chatbot data: null error: code: API_KEY_NOT_VALID message: Authorization header does not contain valid API key '403': description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Forbidden Error: value: success: false message: Failed to add links to the chatbot data: null error: code: CHATBOT_ADD_LINKS_FORBIDDEN message: You are not authorized to add links to this chatbot '404': description: Not Found content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Not Found Error: value: success: false message: Failed to add links to the chatbot data: null error: code: CHATBOT_OWNER_NOT_FOUND message: Chatbot owner does not exist for the chatbot '429': description: Too Many Requests - Quota Exceeded content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Quota Exceeded: value: success: false message: Failed to add links to the chatbot data: null error: code: CHATBOT_LINKS_LIMIT_REACHED message: >- You have reached your pages quota limit (500 pages). Current usage: 500 pages. '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: Server Error: value: success: false message: Failed to add links to the chatbot data: null error: code: CHATBOT_ADD_LINKS_FAILED message: Some technical error has occurred !! components: schemas: ScrapeOptions: title: ScrapeOptions type: object description: Options for web scraping properties: onlyMainContent: type: boolean default: true description: >- When enabled, only the main content of the page is extracted, excluding headers, footers, sidebars, and navigation elements includeSelectors: type: array items: type: string description: CSS selectors to include when scraping (e.g., "main", ".content") excludeSelectors: type: array items: type: string description: CSS selectors to exclude when scraping (e.g., "footer", ".sidebar") headers: type: object additionalProperties: type: string description: >- Custom HTTP headers to use when fetching pages (e.g., {"Authorization": "Bearer token"}) SyncFrequency: title: SyncFrequency type: string description: Frequency for automatic content re-sync (requires feature flag) enum: - NEVER - DAILY - WEEKLY - MONTHLY default: NEVER ErrorResponse: type: object description: Error Response properties: success: type: boolean description: Indicates that this is error response message: type: string description: Overall error message regarding the operation performed data: type: object description: Data returned from the API. This will be `null` for errors nullable: true error: type: object description: Object with the error details properties: code: type: string description: Based on this error code, you can identify what is the error message: type: string description: The message indicating the reason for the error details: type: array description: More details on the error. Applicable only for Validation errors nullable: true items: type: object properties: path: type: array description: >- Specifies the path in request body where error occurred. If the error field is nexted, then this array length will be more than 1. items: oneOf: - type: string description: Specifies the name of the field - type: integer description: Specifies the position in an array message: type: string description: Reason for the error of the path mentioned before securitySchemes: bearerAuth: type: http scheme: bearer ````