Import large websites from one sitemap

Shipped

November 9, 2026

Area

Knowledge

Available on

Every plan

If your website has thousands of pages, getting all of it into your chatbot's knowledge was harder than it should have been. The sitemap picker stopped at 1,000 addresses, Import All did nothing on a sitemap with more than 500 new pages, and a knowledge base that passed 1,000 pages could refuse to list or open them. All three are fixed, and imports now run on a steadier queue behind the scenes.

What's new

The sitemap picker loads up to 10,000 pages. Paste your sitemap and Studio lists up to 10,000 of its addresses, ten times more than before. They appear 1,000 at a time, with a button to show the next 1,000. Pages already in your knowledge base move to the bottom of the list, so the next pages to add are always at the top.

Import All works for big sitemaps. On a saved sitemap, Import All now sends new pages in batches of 500, one after another. A progress note shows which pages are going out, and Studio asks you to keep the page open until every batch has been sent. If one batch fails, the batches already sent are kept and only the pages that didn't go in stay on the list.

Large knowledge bases open normally. A knowledge base with more than 1,000 pages or documents now lists them and opens each one as usual.

Steadier imports. Website imports now wait in a queue that keeps going if our servers restart, and that takes turns between accounts. Your import keeps moving even when someone else is importing thousands of pages at the same time.

How to use it

To add pages from a sitemap:

1. Open your knowledge base in Studio and add a website.
2. Choose From Sitemap, enter your Sitemap URL (or just your domain) and fetch the addresses.
3. Filter by path if you only want some sections, then select up to 500 pages and click Add Selected.
4. Come back for the next 500. The pages you've already added sit at the bottom, so the top of the list is what's left.

To import everything from a saved sitemap:

1. Find the sitemap in your knowledge base's sitemap list.
2. Click Import All.
3. Keep the page open while the batches are sent. Once they're all sent, the pages are read and added in the background.

Good to know

• 500 at a time. You can select up to 500 pages in the picker per round. Import All handles the batching for you.
• Keep the page open while sending. Refreshing or closing the page while Import All is still sending stops the batches that haven't gone out yet. Pages already sent carry on.
• Over 10,000 pages? The picker shows the first 10,000 addresses. Add the rest from the Multiple URLs tab, or use a smaller sitemap for each section of your site.
• Your plan's knowledge limit still applies. Importing more pages doesn't raise your storage. If you reach your limit, Studio tells you before adding more.

To keep imported pages up to date after this, use sitemap sync. For pasting lists of addresses, see bulk website import.