Stopping pen testing bad actor from causing Internal Server Error
Unanswered
New Zealand posted this in #help-forum
New ZealandOP
A bad actor has been waking us up in the night by vulnerability scanning us. Is there any config changes to prevent this? We really don't need relative urls or anything. If I visit the website through website it's a 404, but curl repros with an Internal Server Error. I believe it's happening before the middleware layer so we can't block it through that, although I believe it's something that next should handle. I imagine this should be serving a 404 error still. I haven't found any resources online.
Failed to handle request for /\%20../%20../%20../%20../%20../%20../etc/passwd/
TypeError: Invalid URL
at new URL (node:internal/url:775:36)
at parseRelativeUrl (/app/node_modules/next/dist/shared/lib/router/utils/parse-relative-url.js:16:68)
at parseUrl (/app/node_modules/next/dist/shared/lib/router/utils/parse-url.js:15:55)
at requestHandlerImpl (/app/node_modules/next/dist/server/lib/router-server.js:115:54)
at Server.requestListener (//app/node_modules/next/dist/server/lib/start-server.js:141:19)
at Server.emit (node:events:514:28)
at parserOnIncoming (node:_http_server:1143:12)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) {
code: 'ERR_INVALID_URL',
input: '/\\\%20../%20../%20../%20../%20../%20../etc/passwd/',
base: 'http://n/'
}
I've managed to repro locally on 2 different next servers with the following command
Failed to handle request for /\%20../%20../%20../%20../%20../%20../etc/passwd/
TypeError: Invalid URL
at new URL (node:internal/url:775:36)
at parseRelativeUrl (/app/node_modules/next/dist/shared/lib/router/utils/parse-relative-url.js:16:68)
at parseUrl (/app/node_modules/next/dist/shared/lib/router/utils/parse-url.js:15:55)
at requestHandlerImpl (/app/node_modules/next/dist/server/lib/router-server.js:115:54)
at Server.requestListener (//app/node_modules/next/dist/server/lib/start-server.js:141:19)
at Server.emit (node:events:514:28)
at parserOnIncoming (node:_http_server:1143:12)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) {
code: 'ERR_INVALID_URL',
input: '/\\\%20../%20../%20../%20../%20../%20../etc/passwd/',
base: 'http://n/'
}
new URL("/\\\%20../%20../%20../%20../%20../%20../etc/passwd/") of course throwsI've managed to repro locally on 2 different next servers with the following command
curl -k http://localhost:3000/\\\\\\%20../%20../%20../%20../%20../%20../etc/passwd/11 Replies
Asian black bear
This can only be mitigated using a proxy such as Cloudflare and rate limiting.
Nothing else you can do about it otherwise.
New ZealandOP
I imagine vercel.app would have a reverse proxy / firewall like Near said. Does it repro for anyone when deploying locally?
FYI I also don't repro in Postman, only curl. I get a 404 as desired in postman. Perhaps the browser and postman are doing some url encoding to escape the characters, which makes it valid enough? This is the function that throws https://github.com/vercel/next.js/blob/13f8fcbb6b7754469012a8cf7853fac542bf91a3/packages/next/src/shared/lib/router/utils/parse-relative-url.ts
I'm not a doctor, and I recognize stopping all vulnerability scanning errors is a losing battle, but it feels like invalid supplied urls is a valid failure case that Next should handle as a Bad Request without having to rely on a reverse proxy. Especially if it happens before you could even prevent it in the middleware layer
FYI I also don't repro in Postman, only curl. I get a 404 as desired in postman. Perhaps the browser and postman are doing some url encoding to escape the characters, which makes it valid enough? This is the function that throws https://github.com/vercel/next.js/blob/13f8fcbb6b7754469012a8cf7853fac542bf91a3/packages/next/src/shared/lib/router/utils/parse-relative-url.ts
I'm not a doctor, and I recognize stopping all vulnerability scanning errors is a losing battle, but it feels like invalid supplied urls is a valid failure case that Next should handle as a Bad Request without having to rely on a reverse proxy. Especially if it happens before you could even prevent it in the middleware layer
this is indeed a bug of either your code or nextjs, problem is that i can't even reproduce it. it's working as intended here.
New ZealandOP
Very odd, with my curl command as well? Postman doesn't repro for me. Thank you for trying.
I should've clarified I'm on 14.2 but I'll test latest and 14.2 on a fresh repro tomorrow. Hopefully it's a bug that's already fixed 🤷♂️
I should've clarified I'm on 14.2 but I'll test latest and 14.2 on a fresh repro tomorrow. Hopefully it's a bug that's already fixed 🤷♂️
i was on 14.2.11. didnt try curl.
New ZealandOP
I was able to repro it on a fresh
It makes sense I guess that it requires i18n from this, https://github.com/vercel/next.js/blob/39833fdf7133eafb495f65600a829ba167569cf1/packages/next/src/server/lib/router-server.ts#L167-L172
I've confirmed setting
Thank you Joulev and Near for the input.
npx create-next-app@latest if and only if the i18n section of next.config.mjs is populated. I reproed it with both App Router and Pages Router, again, only managed to repro it with curl./** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
i18n: {
locales: ["en-US"],
defaultLocale: "en-US",
},
};
export default nextConfig;It makes sense I guess that it requires i18n from this, https://github.com/vercel/next.js/blob/39833fdf7133eafb495f65600a829ba167569cf1/packages/next/src/server/lib/router-server.ts#L167-L172
I've confirmed setting
localeDetection: false, is a workaround. Perhaps we'll try that along with exploring some AWS WAF configs until we need i18n support.Thank you Joulev and Near for the input.
New ZealandOP