Docker runtime variables
Answered
Dwarf Hotot posted this in #help-forum
Dwarf HototOP
Hi,
It's probably not fully NextJS related but it gives me headaches, here is my problem.
I managed to make my next application load
I had to use
Output is set to
Dev mode works flawlessly. A locally built docker image also works without any problem.
The problem is when I use Gitlab CI. I tried, Docker-in-docker, buildah and kaniko, it always resulted in my environment variable being not read/used.
What am I missing here?
It's probably not fully NextJS related but it gives me headaches, here is my problem.
I managed to make my next application load
process.env variables at runtime, using Zod for validation.I had to use
unstore_noStore() function to make it work.Output is set to
standalone.Dev mode works flawlessly. A locally built docker image also works without any problem.
The problem is when I use Gitlab CI. I tried, Docker-in-docker, buildah and kaniko, it always resulted in my environment variable being not read/used.
What am I missing here?
Answered by Ray
unstable_noStore() should be used in where you are trying to access the variable53 Replies
@Dwarf Hotot Hi,
It's probably not fully NextJS related but it gives me headaches, here is my problem.
I managed to make my next application load `process.env` variables at runtime, using Zod for validation.
I had to use `unstore_noStore()` function to make it work.
Output is set to `standalone`.
Dev mode works flawlessly. A locally built docker image also works without any problem.
The problem is when I use Gitlab CI. I tried, Docker-in-docker, buildah and kaniko, it always resulted in my environment variable being not read/used.
What am I missing here?
could you show your
https://github.com/vercel/next.js/blob/canary/examples/with-docker/Dockerfile
dockerfile or try this examplehttps://github.com/vercel/next.js/blob/canary/examples/with-docker/Dockerfile
Dwarf HototOP
Hi @Ray, here is my
Dockerfile, approximately the same than the given example :FROM node:18-alpine AS base
# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
# Install dependencies based on the preferred package manager
COPY package.json package-lock.json* ./
RUN \
if [ -f package-lock.json ]; then npm ci; \
else echo "Lockfile not found." && exit 1; \
fi
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
# build prisma client
RUN npx prisma generate
# build the app
RUN npm run build
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ENV NODE_ENV production
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=builder /app/public ./public
# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
RUN rm .env
USER nextjs
EXPOSE 3000
ENV PORT 3000
# set hostname to localhost
ENV HOSTNAME "0.0.0.0"
# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD ["node", "server.js"]And here is the
config.ts where I load process.envimport { unstable_noStore } from 'next/cache';
import z from 'zod';
const envSchema = z.object({
DATABASE_URL: z.string().trim().url().min(1),
STEAM_API_KEY: z.string().trim().length(32),
STEAM_API_USER_URL: z.string().trim().url().min(1),
STEAM_API_GAMESERVERS_URL: z.string().trim().url().min(1),
GAME_SERVER_HOSTNAME: z.union([z.string().trim().min(1), z.string().ip({ version: "v4" }).min(1)]),
GAME_SERVER_PORT: z.number().int().optional().default(27015),
DISCORD_LINK: z.string().url().optional()
});
// enable runtime environment variables
// otherwise process.env is frozen during build
unstable_noStore();
const envServer = envSchema.safeParse(process.env);
if (!envServer.success) {
console.error(envServer.error.issues);
process.exit(1);
}
export const Config = envServer.data;I do have a bunch of
console.log if database connection fails, and it never get logged...printenv from the container :where do you store the env variable? in .env?
@Dwarf Hotot Click to see attachment
look like the env variable are there but the server couldn't read it?
Dwarf HototOP
Yes something like that. But I don't understand why it works when I docker build and push locally and let portainer pull the image and start the container.
When the image is built and pushed on the ci, it doesn't work anymore.
I tried to see if there is some sort of cache somewhere, I double checked the Sha of the image and commits, everything seems consistent...
@Dwarf Hotot When the image is built and pushed on the ci, it doesn't work anymore.
have you setup the variable on ci?
Dwarf HototOP
No, I didn't, but why should I when I pass the envfile/envvars through portainer at runtime ?
how do you pass it at runtime?
Dwarf HototOP
I can give it a try.
Dwarf HototOP
When testing remotely built image ony workstation I do a docker run with a --env-file
On portainer, I set the variable in a docker compose
I think it is because they build the image with the env file and variable when you run it
@Ray this?
Dwarf HototOP
I followed this documentation https://nextjs.org/docs/pages/building-your-application/configuring/environment-variables#runtime-environment-variables
this is why I used unstable_noStore
but it's not in a component
maybe I messed up things while trying to validate env vars with zod?
@Ray I think it is because they build the image with the env file and variable when you run it
Dwarf HototOP
I'm not sure to understand that point
Dwarf HototOP
yes I do
but I don't think I use the runtimeConfig
unstable_noStore() should be used in where you are trying to access the variableAnswer
Dwarf HototOP
this is set in the next.config.js
or use it inside a function
Dwarf HototOP
but isn't static it when I import my
config.ts?@Dwarf Hotot but isn't static it when I import my `config.ts`?
I don't think so because its runtime variable?
Dwarf HototOP
I'll give it a try anyway
Dwarf HototOP
ok, thanks
Dwarf HototOP
again @Ray you nailed it!
the way cache works is a mystery for me then
😆
Dwarf HototOP
the most difficult is to understand why it can work in dev and not in prod, and moreover not in a docker built image on gitlab 🤣
the good part is that I'm making progress ^^
thanks!
@Dwarf Hotot the most difficult is to understand why it can work in dev and not in prod, and moreover not in a docker built image on gitlab 🤣
yea a lot of new stuff in app router 😆
Dwarf HototOP
I took a look at how t3 manages env vars and zod validation, I may give it a try!
Dwarf HototOP
I ended up with using
@t3-oss/env-nextjs which obviously comes with t3. But I didn't want to scafold t3 and refactor all my code. So that's a good compromise (instead of having noStore used everywhere)@Dwarf Hotot I ended up with using `@t3-oss/env-nextjs` which obviously comes with t3. But I didn't want to scafold t3 and refactor all my code. So that's a good compromise (instead of having `noStore` used everywhere)
yep, I use that library even the project is not bootstrap from t3 stack
Dwarf HototOP
I spoke too soon, I forgot one
noStore somewhere which made me thought @t3-oss/env-nextjs could do the trick@Dwarf Hotot I spoke too soon, I forgot one `noStore` somewhere which made me thought `@t3-oss/env-nextjs` could do the trick
I think this should work
export function getEnvServer() {
unstable_noStore()
return envSchema.safeParse(process.env)
}Dwarf HototOP
copied/pasted 🙂
Dwarf HototOP
works like a charm, thanks @Ray I owe you a ðŸº