Help w Docker compose: a database + webapp running together and drizzle isn't pushing the schemas
Answered
LuisLl posted this in #help-forum
LuisLlOP
Maybe this is not even possible or recommended at all, I'm fairly new to Docker and I wanna self-host an app that's contained in a single Docker compose file.
I'm using drizzle and I'm trying to generate the schemas and push them to the database before my Next.js app starts so I can pre-generate static pages that require the database to be up and running, with the necessary schemas.
## If I was working locally
Like normal local server running on my machine I would do this:
1. pnpm install
2. docker-compose up --build
3. pnpm drizzle-kit generate
4. pnpm drizzle-kit push
---- At this point my database will be up and ready with the tables created
5. pnpm build
---- Next build would have access to the database at build time and will generate static pages
6. pnpm start
---- All works perfectly if i use docker-compose to run my database only
## Issue when having both my app and database in my docker-compose
- I would like the migrations to be done before i request a page that needs the database
- PROBLEM: the database is up, I can reach it from my web app but
- PROBLEM: if i make my pages dynamic it builds, but when I go to a page that needs the database tables to pull data it breaks too
I hope someone can help me!
I'm using drizzle and I'm trying to generate the schemas and push them to the database before my Next.js app starts so I can pre-generate static pages that require the database to be up and running, with the necessary schemas.
## If I was working locally
Like normal local server running on my machine I would do this:
//.env
DATABASE_USER=my_user
DATABASE_PASSWORD=my_password
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_NAME=my_database
DATABASE_URL=postgresql://my_user:my_password@db:5432/my_database
//compose.yaml
services:
db:
container_name: only_db
image: postgres:latest
environment:
POSTGRES_USER: ${DATABASE_USER}
POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
POSTGRES_DB: ${DATABASE_NAME}
ports:
- "5432:5432"
restart: always
volumes:
- postgres_data:/var/lib/postgresql/data
volumes:
postgres_data:
1. pnpm install
2. docker-compose up --build
3. pnpm drizzle-kit generate
4. pnpm drizzle-kit push
---- At this point my database will be up and ready with the tables created
5. pnpm build
---- Next build would have access to the database at build time and will generate static pages
6. pnpm start
---- All works perfectly if i use docker-compose to run my database only
## Issue when having both my app and database in my docker-compose
- I would like the migrations to be done before i request a page that needs the database
- PROBLEM: the database is up, I can reach it from my web app but
drizzle generate
and drizzle push
aren't working, database is empty so It fails saying my tables don't exist.- PROBLEM: if i make my pages dynamic it builds, but when I go to a page that needs the database tables to pull data it breaks too
I hope someone can help me!
Answered by LuisLl
I was able to make the migrations run in the same docker compose, now I have tree containers running: db, migrations (depends on db) and web (depends on migrations and db), still the tables aren’t up at build time even if the migrations container runs before the web app container (supposedly). Anyway I’ll share what I did on my repo:
42 Replies
LuisLlOP
This is my compose.yaml with the two services: web and database
Here is my Dockerfile
And here my .env.production
services:
db:
container_name: template_postgres_db
image: postgres:latest
environment:
POSTGRES_USER: ${DATABASE_USER}
POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
POSTGRES_DB: ${DATABASE_NAME}
ports:
- "5432:5432"
restart: always
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- my_network
web:
build: .
ports:
- "3000:3000"
env_file:
- ./env.production
depends_on:
- db
networks:
- my_network
volumes:
postgres_data:
networks:
my_network:
name: my_network
driver: bridge
Here is my Dockerfile
FROM node:18-alpine AS base
# Install pnpm globally
RUN npm install -g pnpm
# Stage 1: Install dependencies
FROM base AS deps
WORKDIR /app
COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile
COPY . .
RUN pnpm db:generate
RUN pnpm db:push
# Stage 2: Build the application
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN pnpm run build
# Stage 3: Production server
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/public ./public
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
EXPOSE 3000
CMD node server.js
And here my .env.production
NODE_ENV=production
# Database
DATABASE_USER=my_user
DATABASE_PASSWORD=my_password
DATABASE_HOST=db
DATABASE_PORT=5432
DATABASE_NAME=my_database
DATABASE_URL=postgresql://my_user:my_password@db:5432/my_database
@Arinji could you take a look when you have time? Thanks
If what I'm trying to do isn't even recommended or possible also would appreaciate some advide, this is just me trying to self host a full app contained in a single docker-compose.
ok so
@LuisLl
dockerfile is meant purely to build an image
it shld not depend on anything external like a db
what you could instead do is put it all in your docker compose
services:
db:
container_name: template_postgres_db
image: postgres:latest
environment:
POSTGRES_USER: ${DATABASE_USER}
POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
POSTGRES_DB: ${DATABASE_NAME}
ports:
- "5432:5432"
restart: always
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- my_network
migrations:
build: .
command: ["pnpm", "db:generate && pnpm db:push"] # Run Drizzle migrations
depends_on:
db:
condition: service_healthy # Wait for DB to be ready
env_file:
- ./env.production
networks:
- my_network
web:
build: .
ports:
- "3000:3000"
env_file:
- ./env.production
depends_on:
migrations:
condition: service_completed_successfully # Wait for migrations
networks:
- my_network
volumes:
postgres_data:
networks:
my_network:
name: my_network
driver: bridge
im not on my pc to test this... i just told chatgpt to update your compose file xD
it looks correct, but you shld confirm
it looks correct, but you shld confirm
@Arinji im not on my pc to test this... i just told chatgpt to update your compose file xD
it looks correct, but you shld confirm
LuisLlOP
Yea I tried so many ways let me try that one. Chat GPT got sick of me 😭
xD
LuisLlOP
To check for conditions isn't necessary that there's a healthcheck command in the database?
good point, missed that completely
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USER} -d ${DATABASE_NAME}"]
interval: 5s
timeout: 5s
retries: 5
enjoy
that is confirmed xD
LuisLlOP
I wrote it already but thanks:p
kk
@Arinji kk
LuisLlOP
Oh yea I got this error a lot before too:
Attaching to migrations-1, web-1, postgres_db
migrations-1 |
migrations-1 | > template@0.1.0 db:generate /app
migrations-1 | > pnpm drizzle-kit generate
migrations-1 |
migrations-1 | undefined
migrations-1 | ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL Command "drizzle-kit" not found
migrations-1 | ELIFECYCLE Command failed with exit code 254.
migrations-1 exited with code 254
Gracefully stopping... (press Ctrl+C again to force)
I moved drizzle kit from devDependencies to dependencies so it's included in the build but i think it's not inclused since I'm not using it during the build really
hmm maybe
sorry no clue myself... havent needed to run migrations like this, i usually just run them directly from my golang app
LuisLlOP
Yea I think I'm over complicating things here
@Arinji sorry no clue myself... havent needed to run migrations like this, i usually just run them directly from my golang app
LuisLlOP
I've tried everything in my capabilities as for now, maybe it's possible but I'll stop fighting the tools as a noob and do what B33f told me (sorry B33f if you read this, should've listened :c), he runs the Database on its own docker container, separate docker-compose. Definitely separate from the app itself. And the database can be reached externally by the app..
LuisLlOP
@Arinji Thank you btw

@LuisLl I've tried everything in my capabilities as for now, maybe it's possible but I'll stop fighting the tools as a noob and do what **B33f ** told me (sorry B33f if you read this, should've listened :c), he runs the Database on its own docker container, separate docker-compose. Definitely separate from the app itself. And the database can be reached externally by the app..
you will need to remove the running migration command from Dockerfile as that runs during build and the postgres service won't be up during build. Running postgres in diff docker compose setup will increase complexity of adding networks.
Create a new
Create a new
.sh
file which waits for postgres to be up and then runs the migrations and then starts your application with pnpm start
(or equivalent)#!/bin/sh
# Wait for Postgres to be ready
until pg_isready -h postgres -p 5432 -U postgres
do
echo "Waiting for postgres..."
sleep 2
done
echo "PostgreSQL started"
pnpm prisma db push
pnpm start
in your
CMD
in dockerfile, you need to refer to this filedon't forget to install
postgresql-client
for pg_isready
command@Yi Lon Ma you will need to remove the running migration command from Dockerfile as that runs during build and the postgres service won't be up during build. Running postgres in diff docker compose setup will increase complexity of adding networks.
Create a new `.sh` file which waits for postgres to be up and then runs the migrations and then starts your application with `pnpm start`(or equivalent)
LuisLlOP
So what I'm trying to do is doable and it's just a matter of skill issue? If so, I can live with that, my curiousity wanted to try it lol
I'm using Drizzle so I assume it's gonna be the same for both ORMs, I'll try that, thanks
I'm using Drizzle so I assume it's gonna be the same for both ORMs, I'll try that, thanks
LuisLlOP
Update: Still doesn't seem to push the migrations.
I'll try the guide from Lee Rob, I didn't before because it looked complex but what if that's the only way? lol
Also Lee has this
Will keep the thread updated anyway
postgres_db | 2025-03-04 03:34:46.974 UTC [7689] ERROR: relation "todos" does not exist at character 60
postgres_db | 2025-03-04 03:34:46.974 UTC [7689] STATEMENT: select "id", "description", "created_at", "completed" from "todos" order by "todos"."id"
web-1 | ⨯ error: relation "todos" does not exist
web-1 | at async (.next/server/chunks/158.js:7:36596)
web-1 | at async m (.next/server/app/(core)/todos/page.js:2:7144)
....
I'll try the guide from Lee Rob, I didn't before because it looked complex but what if that's the only way? lol
Also Lee has this
migration.ts
file but I can't find where he's importing it:import dotenv from 'dotenv';
import path from 'path';
import { migrate } from 'drizzle-orm/postgres-js/migrator';
import { client, db } from './drizzle';
dotenv.config();
async function main() {
await migrate(db, {
migrationsFolder: path.join(process.cwd(), './lib/db/migrations'),
});
console.log(`Migrations complete`);
await client.end();
}
main();
Will keep the thread updated anyway
LuisLlOP
I followed Lee Rob Self Hosting tutorial. [GitHub repo](https://github.com/leerob/next-self-host)
https://www.youtube.com/watch?v=sIVL4JMqRfc&t=589s
https://www.youtube.com/watch?v=sIVL4JMqRfc&t=589s
LuisLlOP
The tutorial showcases the app already having access to the database. I assume he made sure to have the database up and the database tables created before recording, but I'm not sure since he seems to be ding it live :/
In the READme he says this:
Both the Next.js app and PostgreSQL database will be up and running in Docker containers. To set up your database, you could install npm inside your Postgres container and use the Drizzle scripts, or you can use psql:
> docker exec -it myapp-db-1 sh
> apk add --no-cache postgresql-client
> psql -U myuser -d mydatabase -c '
> CREATE TABLE IF NOT EXISTS "todos" (
> "id" serial PRIMARY KEY NOT NULL,
> "content" varchar(255) NOT NULL,
> "completed" boolean DEFAULT false,
> "created_at" timestamp DEFAULT now()
> );'
LuisLlOP
I could not find where the migrations happened in the process (because it probably didn't), coudn't find anything related to pushing migrations or creating tables in the
### What I had to do was this:
1. I went into my VPS:
2. I pulled the
----- This script did all the set up like I said above
3. Once both containers were up (my Next app and Postgres db), I went into my Postgres container, inside the database itself with
4. I literally executed raw SQL:
5. Now my app works when I navigate to pages that require the database
I still would like to know why drizzle commands do not have access to the db connection when I run docker compose and the containers are building...
Dockerfile
, docker-compose.yaml
or the deploy.sh
script he pulls into the VPS and runs (this script does everything from catting the ENV variables to a .env
file, pulling the actual repo from github, installing docker, docker-compose, run docker-compose, etc.. and then running the docker-compose file)### What I had to do was this:
1. I went into my VPS:
ssh root@ip
.2. I pulled the
deploy.sh
script from github, just that file and executed it.----- This script did all the set up like I said above
3. Once both containers were up (my Next app and Postgres db), I went into my Postgres container, inside the database itself with
docker exec -it myapp-db-1 psql -U myuser -d mydatabase
(change user and password accordingly)4. I literally executed raw SQL:
CREATE TABLE IF NOT EXISTS "todos" (
"id" serial PRIMARY KEY NOT NULL,
"content" varchar(255) NOT NULL,
"completed" boolean DEFAULT false,
"created_at" timestamp DEFAULT now()
);
5. Now my app works when I navigate to pages that require the database
I still would like to know why drizzle commands do not have access to the db connection when I run docker compose and the containers are building...
LuisLlOP
I was able to make the migrations run in the same docker compose, now I have tree containers running: db, migrations (depends on db) and web (depends on migrations and db), still the tables aren’t up at build time even if the migrations container runs before the web app container (supposedly). Anyway I’ll share what I did on my repo:
Answer
These are the important files
I think you are just complicating things
FROM ...
# rest of your config
RUN apk add --no-cache postgresql-client
# do all the building and stuff here
RUN chmod +x run_server.sh
CMD ["/run_server.sh"]
#!/bin/sh
until pg_isready -h postgres -d 5432 -U postgres
do
echo "Waiting for Postgres to start"
sleep 2
done
echo "Postgresql Started. Running migrations..."
node ./scripts/run-migrations.js
echo "Ran migrations. Starting the server..."
pnpm start
LuisLlOP
I tried a script and failed multiple times but now I'm understanding more how these things work so I should give it another try