Recently, I was reviewing how I generate my blog. I have noticed that my site seems to get a lot of bot traffic. I would rather not see this traffic, so I thought I should investigate how I might limit them to reduce the amount of traffic. So, a bit of searching and thinking, and this is what I have decided for now. The Idea How do I prevent AI bots from scanning my page and using up my bandwidt
Build a robot.txt file based on a data file using Hugo
Adam.S·Dev.to··1 min read
D
Continue reading on Dev.to
This article was sourced from Dev.to's RSS feed. Visit the original for the complete story.