Product update · Knowledge

Import hundreds of web pages at once, and pick exactly which

Add many pages from a list or your sitemap in one go, filter the sitemap by path, and keep working while the pages import in the background.

Gopi Krishna Lakkepuram

· 3 min read

Shipped
Area
Knowledge
Available on
Every plan

Your website already answers most of your customers' questions. Getting those pages into your chatbot's knowledge used to mean waiting while Studio added them one at a time, with the dialog open, before you could do anything else. Now you hand over the whole list and get on with your day.

What's new

  • Bulk import that runs in the background. Paste a list of URLs or pick pages from your sitemap, and Studio adds them all in one go. The dialog closes straight away. Pages appear in your Websites list and are read one after another in the background.
  • Progress you can see, and stop. While pages import, a banner shows "Processing websites... X of Y done". Changed your mind? Click Stop Import and the remaining pages are cancelled.
  • Filter your sitemap by path. Big sitemaps mix the pages you want with ones you don't. Open Filter by path to narrow the list. Use Include paths to keep pages such as /services/ and Exclude paths to drop pages such as /blog/, each with a match type: Starts with, Ends with, Contains, Exact match or Wildcard. Click Apply to select every matching page.
  • Page titles filled in for you. You no longer wait for Studio to look up each page's title before importing. Titles are filled in automatically once each page has been read.
  • Rename any page later. Choose Edit Name from a website's actions menu to give it a clearer name.

How to use it

  1. Open Sources, choose the source you want to add to, and go to the Websites tab.
  2. Click Add Website.
  3. Choose Multiple URLs and paste one address per line, then click Process URLs. Or choose From Sitemap, enter your sitemap address or your domain, and fetch its pages.
  4. On a sitemap, open Filter by path to include or exclude sections, then click Apply.
  5. Review the selection and click Add Selected.
  6. Keep working. The Websites list updates as each page is read.

Good to know

  • Up to 500 pages per import. You can select up to 500 pages at a time from this dialog. For bigger sitemaps, see large sitemap imports.
  • Storage limits still apply. Imported pages count toward your plan's knowledge storage. If a source runs out of space, the remaining pages aren't added.
  • Some pages can't be read. Pages that build their content in the visitor's browser may come through empty. For those, add the key information as text instead.
  • Duplicates are skipped. Pages already in the source are marked so you don't add them twice.
  • Adding new pages later. When your site grows, sitemap sync shows which sitemap pages are new so you can import just those.

Explore Hyperleap AI

An AI front desk that answers customer questions, collects contact details, and books appointments on your website and messaging channels.