Skip to main content
PUT
Python

Authorizations

x-api-key
string
header
required

API key for authentication

Path Parameters

id
string
required

Ingestion setting ID for the web crawl data source

Body

application/json
name
string
required

Name of the web crawl data source

Maximum string length: 200
rrule
string

Recurrence rule (RFC 5545 RRULE). Empty string or omission removes the existing schedule.

load_more_enabled
boolean

Omission preserves the saved value; false disables clicking without clearing selectors or the limit. Null is invalid.

load_more_selectors
string[]

Ordered CSS selectors. Omission preserves them; an empty array clears them and uses built-in controls. Null is invalid.

load_more_max_clicks
integer

Maximum click attempts per page. Omission preserves the saved value, or 10 for legacy settings. Null is invalid.

Required range: 1 <= x <= 100

Response

Successfully updated web crawl setting

id
string
required

Web crawl ingestion setting ID

name
string
required

Name of the web crawl ingestion setting

start_url
string
required

Start URL of the web crawl

max_depth
integer
required

Maximum crawl depth

html_only
boolean
required

When true, only HTML files will be downloaded

use_browser
boolean
required

Whether to use a headless browser for crawling

load_more_enabled
boolean
required

Whether to click load-more controls. Existing settings without this field are disabled.

load_more_selectors
string[]
required

Ordered CSS selectors. An empty array uses built-in load-more controls.

load_more_max_clicks
integer
required

Maximum click attempts per page. Defaults to 10.

Required range: 1 <= x <= 100
path_filters
string[]

Path filters for crawling

content_pattern
string[]

Content patterns for filtering

max_num_files
integer | null

Maximum number of files to crawl

no_canonical_check
boolean

Whether to disable duplicate filtering by canonical URL

file_extensions
string[]

File extensions to include (e.g. ".pdf", ".docx")

rrule
string

Recurrence rule (RFC 5545 RRULE)