Bots visiting my site, maxing out my usage plan
Answered
Boxer posted this in #help-forum
BoxerOP
I don’t understand where, why or how these bots are running on my site, I have an SSL, I am hosted on Vercel using nameservers, the packages I am using in the codebase are very standard, why is this happening to me and how do I fix it???
Answered by B33fb0n3
it's happening for all of us. To be a little bit more protected against that change it like the following:
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
47 Replies
BoxerOP
Domain: filmaxcinemahub.com
@Boxer I don’t understand where, why or how these bots are running on my site, I have an SSL, I am hosted on Vercel using nameservers, the packages I am using in the codebase are very standard, why is this happening to me and how do I fix it???
it's happening for all of us. To be a little bit more protected against that change it like the following:
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
Answer
@B33fb0n3 it's happening for all of us. To be a little bit more protected against that change it like the following:
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
BoxerOP
Damn, any hope they reset my image optimization usage if I slap cloudflare in the middle and send them a support email or something? These bots maxed my shit from the time I went to sleep till now. Ridiculous
@Boxer Damn, any hope they reset my image optimization usage if I slap cloudflare in the middle and send them a support email or something? These bots maxed my shit from the time I went to sleep till now. Ridiculous
you can try to contact the support and maybe they can do something about this. Here you can visit the vercel suppot: #vercel-help
@B33fb0n3 it's happening for all of us. To be a little bit more protected against that change it like the following:
Your current setup:
Domain (Nameservers) -> Vercel
Your future setup:
Domain (Nameservers) -> Cloudflare -> Vercel
Like that, cloudflare can filter all potential bot traffic and it's completely free 🙂
BoxerOP
Do I have to move all of my other DNS stuff to cloudflare as well to do this or just the nameservers?
@Boxer Do I have to move all of my other DNS stuff to cloudflare as well to do this or just the nameservers?
You need to move all DNS stuff to cloudflare. But don't worry, that happens automatically.
When you add a new domain to cloudflare it'll check which DNS entries are available and you can choose if you want to use them
When you add a new domain to cloudflare it'll check which DNS entries are available and you can choose if you want to use them
BoxerOP
It just pulls those entries from Vercel or something??
If it tried to pull from my registrar there is nothing there lol
@Boxer It just pulls those entries from Vercel or something??
Yea, it checks the current DNS entries and displays you the entries. Even if you DNS entries are not on vercel, somewhere they are and cloudflare will know where they are 🙂
@B33fb0n3 Yea, it checks the current DNS entries and displays you the entries. Even if you DNS entries are not on vercel, somewhere they are and cloudflare will know where they are 🙂
BoxerOP
I made another thread on here because I am seeing issue with my SSL cert#2 even with the A+ rating, as soon as I figure out why I am pulling out of Vercel and using cloudflare because this is crazy but thank you for your help
BoxerOP
can you zoom out a bit I have no idea where I can do this
oh nvm i see
@Boxer do you happen to know how they are named?
bc some of them are crawlers most of the times
cloudflare will most likely not block crawlers
i don't think me visiting my website is causing these many requests
(all of these went trough btw)
BoxerOP
Yeah the one hitting mine is claudebot
yep, it's a crawler
@Boxer this will help blocking it
it's in italian tho
BoxerOP
I just turned on challenge attack mode and it stopped. Maxed out my image optimization though
yeah, challenge attack is not supposed to be on at all times
i suggest looking at that guide
bc it seems like it is VERY aggressive towards a lot of ppl
BoxerOP
I mean my site is fucked anyways
I’ll probably have to get a whole new domain and can the one I have
@Boxer I mean my site is fucked anyways
wdym it's fucked anyway?
BoxerOP
It’s being blacklisted, falsely being reported as a malicious site by a few security vendors I even got this email back from one of the vendors today;
We are not allowed to re-classify the site as there is a law enforcement request involving copyrighted material being used inappropriately
I am literally pulling info from a public api and rendering it, that is all lmao.
We are not allowed to re-classify the site as there is a law enforcement request involving copyrighted material being used inappropriately
I am literally pulling info from a public api and rendering it, that is all lmao.
BoxerOP
I’m about to put this identical codebase on a fresh domain and have 0 problems
Just a normal ass site build with normal shit lol
Idk how a security vendor would even get a notice from “law enforcement” day 1 of me owning a domain but I haven’t been notified, Vercel hasn’t and neither has my registrar
I just wanted to publish stuff to my portfolio 😭😭😭
@Boxer It’s being blacklisted, falsely being reported as a malicious site by a few security vendors I even got this email back from one of the vendors today;
We are not allowed to re-classify the site as there is a law enforcement request involving copyrighted material being used inappropriately
I am literally pulling info from a public api and rendering it, that is all lmao.
It's bc of the content, no matter the apis, you do not own the right to those films
@Diamond Master It's bc of the content, no matter the apis, you do not own the right to those films
BoxerOP
What’s because of the content?
@Boxer What’s because of the content?
maybe the images?
i think that is causing problems
BoxerOP
I mean it’s not a commercial app, nothing is monetized, all the data comes from a public api that complies with DMCA, this is fair use.
@Boxer I mean it’s not a commercial app, nothing is monetized, all the data comes from a public api that complies with DMCA, this is fair use.
yeah well, nobody said that those companies are good at their job either
@Diamond Master yeah well, nobody said that those companies are good at their job either
BoxerOP
Yeah I can’t disagree there lol