// parseless — free wordpress plugin
AI crawlers don’t read your HTML. Stop serving it to them.
ParseLess detects 14 AI crawlers and serves them clean, token-efficient Markdown on the same URL — while humans and Googlebot still get your normal site. Free, open source, and built by someone you can hire.
Free forever · 50+ active installs · 5-minute setup
// measured, not invented
Benchmarks from a real content page — with the methodology published.
The AI-SEO category is full of unfalsifiable claims. These numbers come from the ParseLess readme measurements, and the test setup is documented so anyone can reproduce them.
Page weight served to AI crawlers
Before: 79 KBWith ParseLess: 4 KB95% smaller (~20×)Tokens per page
Before: ~19,800 (HTML)With ParseLess: ~975 (Markdown)~95% fewer tokensTime to first byte (TTFB)
Before: 251 msWith ParseLess: 115 ms54% fasterDatabase queries per bot request
Before: Full page renderWith ParseLess: Cached Markdown~60–80% fewerPeak PHP memory per bot request
Before: Full theme stackWith ParseLess: Markdown only~60% lowerMethodology: figures taken from the ParseLess readme benchmarks on a real content page. Payload measured as transferred bytes; token counts via standard LLM tokenizers; TTFB averaged over repeated requests; database queries and peak memory measured server-side with standard WordPress profiling. The full test setup — theme, page, and measurement method — is documented in the plugin readme.
Numbers vary by theme. Elementor and Divi sites see the largest savings; lightweight themes see less.
// why this matters now
AI crawlers are already a fifth of verified bot traffic — and they almost never send visitors back.
Third-party context, cited with sources and dates — these are not my measurements:
of verified bot traffic was AI crawlers in May 2026, and the share keeps growing.
Cloudflare, May 2026 ↗pages crawled per referral for ClaudeBot in Q1 2026 — versus ~4.9:1 for traditional Google search. Other major AI bots range from ~118:1 to nearly 50,000:1.
Crawl-to-referral analyses, Q1 2026 ↗of AI crawler requests are search-purpose. The rest is extraction: your content feeding models while nothing comes back.
Cloudflare, 2026 ↗Cloudflare began default-blocking AI training bots on new ad-supported domains, with pay-per-use crawling fees. The free ride is ending.
Cloudflare, September 2026 ↗Block them, or let them bleed you?
The market is splitting into two camps: block AI crawlers entirely, or keep paying to serve full HTML pages to bots that were never going to render them. ParseLess is the third path — serve them exactly what they consume, at a fraction of the cost to you, on your terms.
// what parseless does
One plugin. Same URL. Two versions of your site.
Everything below is verifiable in the plugin listing and the source code.
Same URL, no cloaking games
Humans, Googlebot, and everyone else get your normal HTML. Verified AI crawlers get a Markdown version of the exact same content — nothing hidden, nothing different in substance.
14 named AI crawlers detected
GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, out of the box. The detection list is editable, and a blocked-bots list lets you refuse crawlers entirely.
Respects your SEO settings
Honors noindex directives from Yoast, Rank Math, and Genesis, and automatically skips private, draft, and password-protected content.
Fast by design
Markdown responses are transient-cached and cleared on save — bots always get fresh content without re-rendering your theme on every hit.
Private by default
Logging is off by default. When enabled, IPs are stored as salted SHA-256 hashes — never raw — with configurable 7/30/90/365-day retention and one-click purge. No external service, no data leaves your server.
Built for developers
Five filters to customize detection, output, and behavior. And yes: curl yoursite.com/post/?format=md | claude "summarize this" just works.
Four endpoints AI agents already know how to use
/llms.txtA curated map of your site, written for AI agents
/llms-full.txtThe full content, pre-digested
/botfood-sitemap.xmlA sitemap crawlers can actually use
/okf/A machine-friendly content feed
// what you can’t see can still bill you
A brand-new, empty WordPress site logged 164 bot-like requests in 30 days.
While testing ParseLess, I pointed it at a fresh WordPress install with no content and no backlinks. Here is what showed up:
Most site owners have no idea this is happening, because nothing in WordPress shows them. ParseLess makes it visible. But visibility is only useful if someone reads the results and knows what to do about them.
That’s where I come in.
The plugin is free and always will be. If you want someone to read what’s crawling your site, separate real AI agents from noise and scanners, and turn that into a crawling and content strategy — that’s consulting. That’s what I do.
// work with the developer
ParseLess is free. My time is not.
I build WordPress + AI systems for a living. ParseLess is what I give away — these are the engagements I take on.
AI Crawler Visibility Audit
A structured review of who is actually crawling your WordPress site: server logs, ParseLess data, and bot analytics. You get a written report naming your real crawler mix, what it costs you, and a prioritized action plan.
- →Server log and bot traffic analysis
- →ParseLess setup and configuration
- →Written report: who crawls you, what it costs
- →Crawl policy recommendations — serve, throttle, or block
- →One follow-up session to review findings
AI-Readiness Setup for WordPress
Full GEO/AEO implementation on your WordPress site: ParseLess configuration, llms.txt strategy, structured content, machine-readable endpoints, and measurement so you know whether AI platforms can actually use your content.
- →ParseLess install, tuning, and verification
- →llms.txt and endpoint strategy
- →Content structure review for AI extraction
- →Measurement setup and baseline report
Custom WordPress + AI Development
The same engineering that went into ParseLess, applied to your problem: custom plugins, AI integrations, content pipelines, and automation built to survive production.
- →Architecture design and documentation
- →Custom plugin or integration build
- →Error handling, caching, and observability
- →Full handover with operational docs
Fractional WordPress + AI Lead
Ongoing senior technical guidance for agencies and site owners who need answers on AI: crawler policy, vendor evaluation, architecture review, and implementation oversight, without a full-time hire.
- →Weekly architecture reviews
- →Async availability for technical decisions
- →Code and vendor review
- →Team upskilling sessions
// faq
Common questions.
What is ParseLess?
+
Does ParseLess change what human visitors see?
+
Which AI crawlers does ParseLess detect?
+
Will ParseLess improve my rankings in ChatGPT or Perplexity?
+
How much bandwidth and server load does it actually save?
+
Is ParseLess privacy-friendly?
+
Does it work with Yoast, Rank Math, or other SEO plugins?
+
Who builds ParseLess, and can I hire them?
+
Let the machines read Markdown. Let me handle the rest.
Install ParseLess free from WordPress.org, or book a call and have the person who wrote it look at what is actually crawling your site.
Free plugin. No sales pitch on the call — you explain your setup, I tell you what makes sense.