
The 95% Problem: What KetoBible Taught Me About Bot Traffic
A real-world look at what happens when 95% of your website traffic is automated, and why moving to a headless stack was the only permanent fix.
I bought KetoBible.net because I liked the content. It was a massive library of recipes and nutrition articles, built on WordPress, monetised through AdSense. For a while, it worked. Then the bots found it.
By the time I decided to migrate it, the site was receiving thousands of hits a month. On paper, that looked like a successful site. In reality, almost none of those visitors were human.
When I pulled the server logs, the truth was stark: roughly 95% of the traffic hitting the site was automated. These weren't customers. They were AI crawlers training on the recipes, scrapers harvesting the content for spam sites, and vulnerability probes cycling through the WordPress install looking for a way in.
Because the traffic was automated, the ad impressions were worthless. The ad revenue had effectively gone to zero. But the site was still paying for shared hosting, and it was still paying more every time the bot volume grew.
The Throttling Wall
Shared and managed hosting plans price for human visitors. When bots arrive - and they always do - the CPU and bandwidth get eaten by traffic that will never buy anything.
The host's answer isn't to help you block the bots; it's to protect their own servers. Eventually, I received the email every WordPress owner dreads: throttle the site or upgrade the plan. I was paying to serve traffic that was never going to convert, and the cost was rising.
Plugins didn't fix it. Security plugins and caching plugins act at the server level. They might mitigate the damage, but they don't stop the requests from hitting the server in the first place. The CPU was still being burned. The database was still being queried.
The Permanent Fix: Headless
I migrated KetoBible onto a headless stack - Next.js on Vercel, behind Cloudflare, with Payload CMS and a Postgres database for the content.
The migration itself was straightforward. The important part was the architecture that replaced the old one.
Cloudflare WAF rules, rate limiting, and bot fingerprinting now sit in front of everything, filtering automated traffic at the edge before it ever reaches the application. Legitimate visitors get the page; bots get blocked or challenged. The origin server only ever sees real humans and legitimate SEO crawlers like Google, BingBot, etc...
There is no plugin layer to maintain. No PHP to update. No database bloat from abandoned plugins. The features the site needs - search, category filtering, the recipe pages - are all built into the code, not bolted on with plugins that carry the risk of breaking the site when they update.
A lot of small website owners in this situation actually pay an agency to handle the updates, backups and mitigate security threats as they arise - reacting to problems...not driving solutions.
The Result
The bot problem is gone. Cloudflare filters the automated traffic at the edge, so the origin server stays quiet. The throttle notices stopped the day after migration because the webiste was no longer on a shared hosting plan - on a shared server trying to keep up with the traffic spikes from malicious visits.
The hosting bill dropped to the cost of the domain. Vercel's free tier plus Cloudflare covers everything the site does... and as the site grows - both in content and in visitor numbers, upgrading is a value added decision.
Page speed improved, because static pages from a CDN load faster than a WordPress install rendering pages on demand from a shared server. Core Web Vitals came out at 100 across the board. And the content team - which is me - can still edit everything from a Payload admin dashboard that works the same way WordPress did, just without the plugin management underneath it.
This is the same architecture I now offer to clients. The KetoBible migration is the proof that it works.
Related posts
Is your site being scraped?
ScrapeShield analyses your traffic to reveal bots, scrapers, and AI crawlers in real time. See who is accessing your content, how often, and from where — then decide what to do about it.
No spam - ever..! Just the latest articles, fixes, and advice - direct to your inbox.



