Next.js Discord

Discord Forum

Help w Docker compose: a database + webapp running together and drizzle isn't pushing the schemas

Answered
LuisLl posted this in #help-forum
Open in Discord
Maybe this is not even possible or recommended at all, I'm fairly new to Docker and I wanna self-host an app that's contained in a single Docker compose file.
I'm using drizzle and I'm trying to generate the schemas and push them to the database before my Next.js app starts so I can pre-generate static pages that require the database to be up and running, with the necessary schemas.

## If I was working locally
Like normal local server running on my machine I would do this:
//.env
DATABASE_USER=my_user
DATABASE_PASSWORD=my_password
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_NAME=my_database
DATABASE_URL=postgresql://my_user:my_password@db:5432/my_database

//compose.yaml
services:
  db:
    container_name: only_db
    image: postgres:latest
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
      POSTGRES_DB: ${DATABASE_NAME}
    ports:
      - "5432:5432"
    restart: always
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

1. pnpm install
2. docker-compose up --build
3. pnpm drizzle-kit generate
4. pnpm drizzle-kit push
---- At this point my database will be up and ready with the tables created
5. pnpm build
---- Next build would have access to the database at build time and will generate static pages
6. pnpm start
---- All works perfectly if i use docker-compose to run my database only

## Issue when having both my app and database in my docker-compose
- I would like the migrations to be done before i request a page that needs the database
- PROBLEM: the database is up, I can reach it from my web app but drizzle generate and drizzle push aren't working, database is empty so It fails saying my tables don't exist.
- PROBLEM: if i make my pages dynamic it builds, but when I go to a page that needs the database tables to pull data it breaks too

I hope someone can help me!
Answered by LuisLl
I was able to make the migrations run in the same docker compose, now I have tree containers running: db, migrations (depends on db) and web (depends on migrations and db), still the tables aren’t up at build time even if the migrations container runs before the web app container (supposedly). Anyway I’ll share what I did on my repo:
View full answer

42 Replies

This is my compose.yaml with the two services: web and database
services:
  db:
    container_name: template_postgres_db
    image: postgres:latest
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
      POSTGRES_DB: ${DATABASE_NAME}
    ports:
      - "5432:5432"
    restart: always
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - my_network

  web:
    build: .
    ports:
      - "3000:3000"
    env_file:
      - ./env.production
    depends_on:
      - db
    networks:
      - my_network

volumes:
  postgres_data:

networks:
  my_network:
    name: my_network
    driver: bridge


Here is my Dockerfile
FROM node:18-alpine AS base

# Install pnpm globally
RUN npm install -g pnpm

# Stage 1: Install dependencies
FROM base AS deps
WORKDIR /app
COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile
COPY . .

RUN pnpm db:generate
RUN pnpm db:push

# Stage 2: Build the application
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN pnpm run build

# Stage 3: Production server
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/public ./public
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static

EXPOSE 3000
CMD node server.js


And here my .env.production
NODE_ENV=production
# Database
DATABASE_USER=my_user
DATABASE_PASSWORD=my_password
DATABASE_HOST=db
DATABASE_PORT=5432
DATABASE_NAME=my_database
DATABASE_URL=postgresql://my_user:my_password@db:5432/my_database
@Arinji could you take a look when you have time? Thanks
If what I'm trying to do isn't even recommended or possible also would appreaciate some advide, this is just me trying to self host a full app contained in a single docker-compose.
ok so
@LuisLl
dockerfile is meant purely to build an image
it shld not depend on anything external like a db
what you could instead do is put it all in your docker compose
services:
  db:
    container_name: template_postgres_db
    image: postgres:latest
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
      POSTGRES_DB: ${DATABASE_NAME}
    ports:
      - "5432:5432"
    restart: always
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - my_network

  migrations:
    build: .
    command: ["pnpm", "db:generate && pnpm db:push"]  # Run Drizzle migrations
    depends_on:
      db:
        condition: service_healthy  # Wait for DB to be ready
    env_file:
      - ./env.production
    networks:
      - my_network

  web:
    build: .
    ports:
      - "3000:3000"
    env_file:
      - ./env.production
    depends_on:
      migrations:
        condition: service_completed_successfully  # Wait for migrations
    networks:
      - my_network

volumes:
  postgres_data:

networks:
  my_network:
    name: my_network
    driver: bridge
im not on my pc to test this... i just told chatgpt to update your compose file xD
it looks correct, but you shld confirm
@Arinji im not on my pc to test this... i just told chatgpt to update your compose file xD it looks correct, but you shld confirm
Yea I tried so many ways let me try that one. Chat GPT got sick of me 😭
xD
To check for conditions isn't necessary that there's a healthcheck command in the database?
good point, missed that completely
healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USER} -d ${DATABASE_NAME}"]
      interval: 5s
      timeout: 5s
      retries: 5
enjoy
that is confirmed xD
I wrote it already but thanks:p
kk
@Arinji kk
Oh yea I got this error a lot before too:

Attaching to migrations-1, web-1, postgres_db
migrations-1  |
migrations-1  | > template@0.1.0 db:generate /app
migrations-1  | > pnpm drizzle-kit generate              
migrations-1  |
migrations-1  | undefined                                 
migrations-1  |  ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command "drizzle-kit" not found
migrations-1  |  ELIFECYCLE  Command failed with exit code 254.                                                                                                         
migrations-1 exited with code 254
Gracefully stopping... (press Ctrl+C again to force)
I moved drizzle kit from devDependencies to dependencies so it's included in the build but i think it's not inclused since I'm not using it during the build really
hmm maybe
sorry no clue myself... havent needed to run migrations like this, i usually just run them directly from my golang app
Yea I think I'm over complicating things here
@Arinji sorry no clue myself... havent needed to run migrations like this, i usually just run them directly from my golang app
I've tried everything in my capabilities as for now, maybe it's possible but I'll stop fighting the tools as a noob and do what B33f told me (sorry B33f if you read this, should've listened :c), he runs the Database on its own docker container, separate docker-compose. Definitely separate from the app itself. And the database can be reached externally by the app..
@Arinji Thank you btw:meow_peek:
@LuisLl I've tried everything in my capabilities as for now, maybe it's possible but I'll stop fighting the tools as a noob and do what **B33f ** told me (sorry B33f if you read this, should've listened :c), he runs the Database on its own docker container, separate docker-compose. Definitely separate from the app itself. And the database can be reached externally by the app..
you will need to remove the running migration command from Dockerfile as that runs during build and the postgres service won't be up during build. Running postgres in diff docker compose setup will increase complexity of adding networks.
Create a new .sh file which waits for postgres to be up and then runs the migrations and then starts your application with pnpm start(or equivalent)
#!/bin/sh

# Wait for Postgres to be ready
until pg_isready -h postgres -p 5432 -U postgres
do
  echo "Waiting for postgres..."
  sleep 2
done

echo "PostgreSQL started"
pnpm prisma db push
pnpm start
in your CMD in dockerfile, you need to refer to this file
don't forget to install postgresql-client for pg_isready command
(feels weird being at the other side asking for help, I'll give myself a point ✅ when I solve it, jk lol)
Update: Still doesn't seem to push the migrations.
postgres_db  | 2025-03-04 03:34:46.974 UTC [7689] ERROR:  relation "todos" does not exist at character 60
postgres_db  | 2025-03-04 03:34:46.974 UTC [7689] STATEMENT:  select "id", "description", "created_at", "completed" from "todos" order by "todos"."id"
web-1        |  ⨯ error: relation "todos" does not exist                         
web-1        |     at async (.next/server/chunks/158.js:7:36596)
web-1        |     at async m (.next/server/app/(core)/todos/page.js:2:7144) 
....


I'll try the guide from Lee Rob, I didn't before because it looked complex but what if that's the only way? lol
Also Lee has this migration.ts file but I can't find where he's importing it:
import dotenv from 'dotenv';
import path from 'path';
import { migrate } from 'drizzle-orm/postgres-js/migrator';
import { client, db } from './drizzle';

dotenv.config();

async function main() {
  await migrate(db, {
    migrationsFolder: path.join(process.cwd(), './lib/db/migrations'),
  });
  console.log(`Migrations complete`);
  await client.end();
}

main();


Will keep the thread updated anyway
The tutorial showcases the app already having access to the database. I assume he made sure to have the database up and the database tables created before recording, but I'm not sure since he seems to be ding it live :/
In the READme he says this:
Both the Next.js app and PostgreSQL database will be up and running in Docker containers. To set up your database, you could install npm inside your Postgres container and use the Drizzle scripts, or you can use psql:
> docker exec -it myapp-db-1 sh
> apk add --no-cache postgresql-client
> psql -U myuser -d mydatabase -c '
> CREATE TABLE IF NOT EXISTS "todos" (
>   "id" serial PRIMARY KEY NOT NULL,
>   "content" varchar(255) NOT NULL,
>   "completed" boolean DEFAULT false,
>   "created_at" timestamp DEFAULT now()
> );'
I could not find where the migrations happened in the process (because it probably didn't), coudn't find anything related to pushing migrations or creating tables in the Dockerfile, docker-compose.yaml or the deploy.sh script he pulls into the VPS and runs (this script does everything from catting the ENV variables to a .env file, pulling the actual repo from github, installing docker, docker-compose, run docker-compose, etc.. and then running the docker-compose file)

### What I had to do was this:
1. I went into my VPS: ssh root@ip.
2. I pulled the deploy.sh script from github, just that file and executed it.
----- This script did all the set up like I said above
3. Once both containers were up (my Next app and Postgres db), I went into my Postgres container, inside the database itself with docker exec -it myapp-db-1 psql -U myuser -d mydatabase (change user and password accordingly)
4. I literally executed raw SQL:
CREATE TABLE IF NOT EXISTS "todos" (
  "id" serial PRIMARY KEY NOT NULL,
  "content" varchar(255) NOT NULL,
  "completed" boolean DEFAULT false,
  "created_at" timestamp DEFAULT now()
);

5. Now my app works when I navigate to pages that require the database

I still would like to know why drizzle commands do not have access to the db connection when I run docker compose and the containers are building...
I was able to make the migrations run in the same docker compose, now I have tree containers running: db, migrations (depends on db) and web (depends on migrations and db), still the tables aren’t up at build time even if the migrations container runs before the web app container (supposedly). Anyway I’ll share what I did on my repo:
Answer
These are the important files
I think you are just complicating things
FROM ...
# rest of your config
RUN apk add --no-cache postgresql-client

# do all the building and stuff here 
RUN chmod +x run_server.sh
CMD ["/run_server.sh"]



#!/bin/sh
until pg_isready -h postgres -d 5432 -U postgres
do 
  echo "Waiting for Postgres to start"
  sleep 2
done

echo "Postgresql Started. Running migrations..."
node ./scripts/run-migrations.js
echo "Ran migrations. Starting the server..."
pnpm start
I tried a script and failed multiple times but now I'm understanding more how these things work so I should give it another try