Fetch not working correctly in middleware.ts (in production)
Answered
Sun bear posted this in #help-forum
Sun bearOP
I have a fetch call in my middleware that is used for obtaining a user region by calling my api route on my backend server. In development this is working fine, but in production this fetch call throws an error.
My fetch function inside middleware looks like this:
I have also tried to call this endpoint in postman to see if it is the problem with backend but that does not seem to be the issue if you try to do it. (Since the app is deployed you can try to ping it yourself)
I will leave my middleware and the entire error below in github gists:
Error: connect ECONNREFUSED 127.0.1.1:9000 <- notice that the error looks like the fetch is trying to call a local domain even though it is pointed at an absolute one (like in the example below).My fetch function inside middleware looks like this:
const res = await fetch(
`https://srv520376.hstgr.cloud:9000/store/regions`,
{
next: {
revalidate: 3600,
tags: ["regions"],
},
},
)I have also tried to call this endpoint in postman to see if it is the problem with backend but that does not seem to be the issue if you try to do it. (Since the app is deployed you can try to ping it yourself)
I will leave my middleware and the entire error below in github gists:
middleware.ts: https://gist.github.com/Aleksa1312/00c5d7f601de377f146ab06ed940ab19
error.log: https://gist.github.com/Aleksa1312/96ebfffd160efcde819eadc01eada7a2
Answered by Sun bear
using service name directly (
http://backend:9000 in middleware) as url has fixed the issue.71 Replies
Is this deployed to vercel or self hosted?
Sun bearOP
it is self hosted via vps
Are you using nginx?
Sun bearOP
I am using caddy
One sec lemme test locally.
Sun bearOP
there might be a cors problem if you ping the url I am using from localhost
but that is essencially the same as getting a response
ye
Im getting a respone without issue while its built
import { type NextRequest, NextResponse } from "next/server"
export async function middleware(request: NextRequest) {
const fetchRequest = await fetch(
`https://srv520376.hstgr.cloud:9000/store/regions`,
{
next: {
revalidate: 3600,
tags: ["regions"],
},
},
)
console.log(fetchRequest)
return NextResponse.next()
}
export const config = {
matcher: ["/((?!api|_next/static|favicon.ico).*)"],
}I got a feeling that your reverse proxy solution is tampering somehow.
Sun bearOP
srv520376.hstgr.cloud:443 {
tls email@gmail.com
reverse_proxy http://frontend:8000
}
srv520376.hstgr.cloud:9000 {
tls email@gmail.com
reverse_proxy http://backend:9000
}I am just redirecting requests from port 9000 to my backend and requests on 443 to frontend
@Jboncz I got a feeling that your reverse proxy solution is tampering somehow.
Sun bearOP
I just don't get why it would work in dev but not production
Agreed, like I said im not having the issue so I dont think its a nextjs issue.
Sun bearOP
what do you mean by a route handler?
like a next.js api route or?
Yes
Sun bearOP
I don't have any api routes
so what is providing the response from https://srv520376.hstgr.cloud:9000/store/regions
Sun bearOP
my express server on port 9000
Is express seeing the request hit it?
I know its a dumb question just making sure im asking everything lol
Sun bearOP
not when it is called by middleware
Sun bearOP
I only have rewrites and remotePatterns for images:
async rewrites() {
return [
{
source: "/products",
destination: "/search",
},
{
source: "/home",
destination: "/",
},
]
},@Sun bear not when it is called by middleware
Sun bearOP
I meant only when the production middleware calls for it. I get when you try to ping it:
xxx.xxx.xxx.xxx - - [04/May/2024:20:01:25 +0000] "GET /store/regions HTTP/1.1" 200 2315 "-" "Next.js Middleware"Trying to think what could be causing it.
Im not sure to be honest with you. Nextjs as far as I know wouldnt rewrite the url your hitting on build.
take a step back and recreate the issue using minimum viable test case. Remove everything from your middleware except that fetch.
Rebuild, and navigate, see what happens.
Can you look at caddys logs and see where its redirecting traffic?
Sun bearOP
I will take a look, but it will take time to rebuild :D
I will add a message if i find someting
thank you for helping
okay, since its takes awhile to rebuild check caddys logs first.
Im assuming they have an option to log what all its forwarding.
if you could enable it and see what the request looks like before its hits caddy for reverse proxy vs what it rewrites it to.
Try to put console logs in the express server.
Gotta debug using logs on production.
Gotta debug using logs on production.
He said its not even hitting the express server.
Which is why I was leaning more of the caddy direction
Ooou I see. Yea it makes sense.
You can try to host it through vercel once as well.
Maybe some server configurations are the root cause?
You can try to host it through vercel once as well.
Maybe some server configurations are the root cause?
Thats the dangers of self hosting you gotta manage it end to end. 😂
Sun bearOP
I am so confused... I have created a very minimal build where the only request sent to backend is the one from middleware. In this build the middleware call is not even mentioned in the caddy nor express logs. Running GET on the same address the middleware is supposed to call using Postman fetches the data normally and logs the data both in caddy and express logs. The only clue I have is that the frontend throws an error
And in this error it is like the middleware is trying to call a local api address 127.0.1.1:9000 which is not what is in the fetch call 🤦♂️
}
[TypeError: fetch failed] {
cause: [Error: connect ECONNREFUSED 127.0.1.1:9000] {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.1.1',
port: 9000
}
}And in this error it is like the middleware is trying to call a local api address 127.0.1.1:9000 which is not what is in the fetch call 🤦♂️
Have u tried different browsers? (grasping at straws here)
Sun bearOP
I will try chrome now but that is probably not the issue
not working in chrome neither.
Can u once try giving the fetch function the backend url directly?
I mean remove the 9000-->backend redirect logic.
Just for testing if the reverse proxy has sm issue
I mean remove the 9000-->backend redirect logic.
Just for testing if the reverse proxy has sm issue
Whats weird is that it doesnt happen in dev
Sun bearOP
The backend is not exposed externally through docker network so it does not exist to the middleware since it is ran on edge...
I see
Maybe some configuration with docker?
Not working as intended in prod
Not working as intended in prod
Sun bearOP
I am using the official Dockerfile as in the next.js docs
Could it be that somehow middleware sees that the request is located on the same network
| host:
| host:
https://srv520376.hstgr.cloudso it tries to shortcut by calling localhost or something?
127.0.1.1:9000
Ooou maybe
Can u somehow map localhost:9000 to backend?
Sun bearOP
maybe I can add 127.0.1.1:9000 to reverse_proxy?
Yea
Sun bearOP
yes I will try that
Sun bearOP
This cannot be fixed by just adding this:
Since I cannot really add a non public domain to the reverse proxy. But I am pretty sure the shortcutting is at fault. Do you have any idea what is causing the shortcutting: next.js, caddy or docker?
127.0.1.1:9000 {
reverse_proxy http://backend:9000
}
host.docker.internal:9000 {
reverse_proxy http://backend:9000
}Since I cannot really add a non public domain to the reverse proxy. But I am pretty sure the shortcutting is at fault. Do you have any idea what is causing the shortcutting: next.js, caddy or docker?
I dont think its next js.
Mp docker
Mp docker
Sun bearOP
using service name directly (
http://backend:9000 in middleware) as url has fixed the issue.Answer
@Sun bear using service name directly (`http://backend:9000` in middleware) as url has fixed the issue.
I knew it! Glad to see its fixed!