About mpliphi-crawler

mpliphi-crawler reads public websites for Mpliphi customers, so the AI agent each business sets up can answer questions about that business. It only visits a site when a Mpliphi customer adds it.

What it reads

Public pages and linked PDFs. It does not log in, fill in forms, add to carts or follow links to other websites. It reads slowly, about one page a second, and waits longer if your robots.txt sets a Crawl-delay.

It respects robots.txt

To stop it reading your site, add:

User-agent: mpliphi-crawler
Disallow: /

To let it read your site when other bots are blocked, add:

User-agent: mpliphi-crawler
Allow: /