Question 1

What is the Web Scraping tool?

Accepted Answer

The Web Scraping tool is a two-tab developer utility on Mikdan Tools. The Schema Manager lets you create and manage reusable scraping configurations (providers) that describe how to fetch structured data from external websites. The Content Browser lets you search and paginate through scraped results using those providers.

Question 2

Do I need an account to use this tool?

Accepted Answer

Yes. The Web Scraping tool requires you to be signed in. Authentication ensures that your provider configurations are scoped to your account and that requests to the scraping backend are authorised on your behalf.

Question 3

What is a scraping provider?

Accepted Answer

A provider is a saved configuration that describes a website you want to scrape. It includes the base URL, a search path template with a {searchQuery} placeholder, CSS selectors for extracting data fields, category mappings, and optional proxy settings. Providers are managed in the Schema Manager tab.

Question 4

Are my provider configurations and scraped results stored permanently?

Accepted Answer

Provider configurations are stored server-side and tied to your user account. Scraped results, however, are never persisted — they only exist in memory for the current browsing session. No scraped content is written to any database or browser storage.

Question 5

How do I switch between the Content Browser and the Schema Manager?

Accepted Answer

Click the tab labels at the top of the tool interface. The active tab is also reflected in the URL (?tab=browser or ?tab=schema), so you can bookmark or share a direct link to either tab. Your browser's Back and Forward buttons also navigate between tab states.

Question 6

Can I use a proxy when scraping with a provider?

Accepted Answer

Yes. Each provider has an optional proxy setting in the Options section of the Provider Form. Enable the proxyEnabled toggle and provide a valid http:// or https:// proxy URL. The proxy is applied server-side — it is never exposed to the browser.

Question 7

What happens if a scraping provider is disabled?

Accepted Answer

Disabled providers (where the isEnabled option is turned off) are hidden from the Content Browser's provider selector. They still appear in the Schema Manager so you can re-enable or delete them. Disabling a provider is a non-destructive way to pause scraping without losing the configuration.

Web Scraping

Sign in to use the Web Scraping Tool

Related Tools

JSON Formatter

Base64 Encoder

Base64 Decoder

Decode Gzip Base64