Too many connections to db
Answered
Николя posted this in #help-forum
НиколяOP
Hello everyone, I’m making a website for watching anime, I have a problem that at some point the Too many connections to db error appears, the deadline for submitting the site to the university is here, but I still can’t solve this problem, can anyone - can you help? I use connection bullets and then call the connected database in api routes, please tell me if anyone knows how to fix this problem
Answered by B33fb0n3
Yes you are right. There was a lot of traffic in that page and I never had performance issues or traffic issues. Because the 80gb are not at vercel. They are on s3. And s3 is cheap. Also to request the media I am using a cdn. So they are compressed and served on the correct size to save bandwidth. The cdn also caches the media. Like that vercel just does the image compression and hosting of the pages for me
34 Replies
Madeiran sardinella
Singleton?
@Madeiran sardinella Singleton?
НиколяOP
If necessary, I can show the db.js code and one of my api routes
Madeiran sardinella
Yep, it can help
НиколяOP
It`s my db.js
This is one of the api routes
// module.exports = connection;
const mysql = require('mysql');
// Создание пула соединений
const pool = mysql.createPool({
host: 'localhost',
user: 'root',
password: 'SonyaRyKo2023',
database: 'anime',
connectionLimit: 500, // Максимальное количество соединений в пуле
queueLimit: 0, // Максимальное количество запросов в очереди (0 - без ограничений)
acquireTimeout: 10000,
waitForConnections: true,
});
// Проверка соединения (опционально)
pool.getConnection((err, connection) => {
if (err) {
console.error('Error connecting to MySQL database:', err);
} else {
console.log('Connected to MySQL database');
connection.release(); // Освобождаем соединение обратно в пул
}
});
module.exports = pool;This is one of the api routes
import { NextResponse } from "next/server";
import pool from "@/db";
export async function GET() {
try {
// Получаем соединение из пула
const connection = await new Promise((resolve, reject) => {
pool.getConnection((err, connection) => {
if (err) {
reject(err);
} else {
resolve(connection);
}
});
});
// Выполняем запрос с использованием полученного соединения
const results = await new Promise((resolve, reject) => {
connection.query('SELECT * FROM series WHERE ReleasData <= NOW()', (err, results) => {
// Освобождаем соединение обратно в пул после выполнения запроса
connection.release();
if (err) {
reject(err);
} else {
resolve(results);
}
});
});
// console.log(results);
return NextResponse.json(results);
} catch (error) {
return NextResponse.json(
{ message: error },
{
status: 500
}
);
}
}@Николя It`s my db.js
// module.exports = connection;
const mysql = require('mysql');
// Создание пула соединений
const pool = mysql.createPool({
host: 'localhost',
user: 'root',
password: 'SonyaRyKo2023',
database: 'anime',
connectionLimit: 500, // Максимальное количество соединений в пуле
queueLimit: 0, // Максимальное количество запросов в очереди (0 - без ограничений)
acquireTimeout: 10000,
waitForConnections: true,
});
// Проверка соединения (опционально)
pool.getConnection((err, connection) => {
if (err) {
console.error('Error connecting to MySQL database:', err);
} else {
console.log('Connected to MySQL database');
connection.release(); // Освобождаем соединение обратно в пул
}
});
module.exports = pool;
This is one of the api routes
import { NextResponse } from "next/server";
import pool from "@/db";
export async function GET() {
try {
// Получаем соединение из пула
const connection = await new Promise((resolve, reject) => {
pool.getConnection((err, connection) => {
if (err) {
reject(err);
} else {
resolve(connection);
}
});
});
// Выполняем запрос с использованием полученного соединения
const results = await new Promise((resolve, reject) => {
connection.query('SELECT * FROM series WHERE ReleasData <= NOW()', (err, results) => {
// Освобождаем соединение обратно в пул после выполнения запроса
connection.release();
if (err) {
reject(err);
} else {
resolve(results);
}
});
});
// console.log(results);
return NextResponse.json(results);
} catch (error) {
return NextResponse.json(
{ message: error },
{
status: 500
}
);
}
}
You need to close the connections, after you used them. You can choose, what and how you want to do that:
1. Close the connection after using it
2. Create a idle timeout so the connection will be closed automatically after a specific time of not using the connection
I prefer 2., but you can still use 1.
Right now, you are either not closing them, or open to many at the same time, so it hits the limits
1. Close the connection after using it
2. Create a idle timeout so the connection will be closed automatically after a specific time of not using the connection
I prefer 2., but you can still use 1.
Right now, you are either not closing them, or open to many at the same time, so it hits the limits
@B33fb0n3 You need to close the connections, after you used them. You can choose, what and how you want to do that:
1. Close the connection after using it
2. Create a idle timeout so the connection will be closed automatically after a specific time of not using the connection
I prefer 2., but you can still use 1.
Right now, you are either not closing them, or open to many at the same time, so it hits the limits
НиколяOP
Yeah, I thought that I was releasing them because I was using pool, could you please tell me how to do option 2?
@Николя Yeah, I thought that I was releasing them because I was using pool, could you please tell me how to do option 2?
you are using mysql and mysql access a system variable called
wait_timeout. Change this to a number of seconds after the idle session will be closed. The default is 8 hours. For serverless enviorements (like vercel & nextjs) I like to use a timeout of 1 - 15 minutes depending of the project.you can see your timeouts in the console to see if you done everything correctly using the
show variables like "%timeout%"; commandof course you can also directly set them
but keep in mind to save it also in the configuration. Else you will hit the limits again
set global wait_timeout=3; (3 seconds)but keep in mind to save it also in the configuration. Else you will hit the limits again
@B33fb0n3 of course you can also directly set them set global wait_timeout=3; (3 seconds)
but keep in mind to save it also in the configuration. Else you will hit the limits again
НиколяOP
Yeah, I think I understand, I’ll definitely try this method today, thank you very much for the advice
Please tell me, I’m wondering, in option 1, when I should close connections, what’s wrong with me? The second option seems much more interesting to me, but I still wonder what’s wrong with 1, I seem to have closed the connections in the code above
@Николя Please tell me, I’m wondering, in option 1, when I should close connections, what’s wrong with me? The second option seems much more interesting to me, but I still wonder what’s wrong with 1, I seem to have closed the connections in the code above
You are on a serverful env or serverless env like vercel?
НиколяOP
serverful, I just need to deploy by tomorrow morning, I'm thinking of installing it on a regular vps and that's it
@B33fb0n3 You are on a serverful env or serverless env like vercel?
НиколяOP
And can you tell me which one is generally better? And which one is easier?I thought about using VPS, but perhaps there are easier ways?
I understand that the questions may be stupid or simple, but I’m just a beginner in this area and don’t know some of the details, so it’s very interesting to hear the opinion of a professional
I understand that the questions may be stupid or simple, but I’m just a beginner in this area and don’t know some of the details, so it’s very interesting to hear the opinion of a professional
Hm yeah, then the release should work.. I am not that hard into serverful environments. If I would be a beginner I would hold to bullet proof structure like vercel. Yes, it’s serverless, but you will also have way less problems and also the advantages of derverles
@B33fb0n3 Hm yeah, then the release should work.. I am not that hard into serverful environments. If I would be a beginner I would hold to bullet proof structure like vercel. Yes, it’s serverless, but you will also have way less problems and also the advantages of derverles
НиколяOP
Sounds good, do you think there will be a free version? Since it turns out I will need to upload the db video somewhere
The only problem why I don’t use Vercel is that I don’t know what’s wrong with their traffic and where to download the database and video
Yea, you can either use the database from vercel or use your own server for database. Like I like to use vercel just for hosting and serving the internet files like html, css, js and of course the bundled versions of it. The rest like a database is on my own server and for file management I like to use cloud computing like s3 for storing files and a cdn to get and cache these files like that you pay nearly nothing. Maybe you pay for your own server, but I guess you already have it 😊
@B33fb0n3 Yea, you can either use the database from vercel or use your own server for database. Like I like to use vercel just for hosting and serving the internet files like html, css, js and of course the bundled versions of it. The rest like a database is on my own server and for file management I like to use cloud computing like s3 for storing files and a cdn to get and cache these files like that you pay nearly nothing. Maybe you pay for your own server, but I guess you already have it 😊
НиколяOP
I’m now thinking of installing it on google cloude, because it’s free for 3 months and you can even deploy it serverless, also store videos and a database there, I don’t have the server itself yet, because I can’t decide where to get it, initially I wanted to launch it I have a local server, but I changed my mind, I’m afraid my laptop won’t last that long hahahaha. I would upload the video to google cloude storage as well as the database to google sql, and the project itself to versel, but I’m worried that there will be little traffic there for the free version
@B33fb0n3 Yea, you can either use the database from vercel or use your own server for database. Like I like to use vercel just for hosting and serving the internet files like html, css, js and of course the bundled versions of it. The rest like a database is on my own server and for file management I like to use cloud computing like s3 for storing files and a cdn to get and cache these files like that you pay nearly nothing. Maybe you pay for your own server, but I guess you already have it 😊
НиколяOP
And paid file storage on AWS would probably cost me a little too much, I have 50 videos of 400 MB each, it would be a little expensive
@Николя I’m now thinking of installing it on google cloude, because it’s free for 3 months and you can even deploy it serverless, also store videos and a database there, I don’t have the server itself yet, because I can’t decide where to get it, initially I wanted to launch it I have a local server, but I changed my mind, I’m afraid my laptop won’t last that long hahahaha. I would upload the video to google cloude storage as well as the database to google sql, and the project itself to versel, but I’m worried that there will be little traffic there for the free version
НиколяOP
Now my database is only located locally
I ended up having this setup:
Hosting: vercel free plan
Database: own server (from netcup)
File Storage: s3
CDN for Media: Bunnycdn
Maybe that’s also a solution for you. It’s pretty cheap
Hosting: vercel free plan
Database: own server (from netcup)
File Storage: s3
CDN for Media: Bunnycdn
Maybe that’s also a solution for you. It’s pretty cheap
@Николя And paid file storage on AWS would probably cost me a little too much, I have 50 videos of 400 MB each, it would be a little expensive
That costs nothing. It’s like one or five cents. I uploaded ~80gb and paid nearly 1$ as far as I know. And it was a high traffic page with a reach of > 8 Million
@Николя Now my database is only located locally
A local fevelopement database is pretty nice. You can test everything and if something goes wrong, nothing happens to the production database. Very much recommended
@B33fb0n3 A local fevelopement database is pretty nice. You can test everything and if something goes wrong, nothing happens to the production database. Very much recommended
НиколяOP
And have you bought anything from vercel? More specifically, I mean traffic (bandwidth)
@Николя And have you bought anything from vercel? More specifically, I mean traffic (bandwidth)
No I haven’t yet. I hope I will never need to pay something to vercel, because I think vercel is pretty expensive…
@B33fb0n3 No I haven’t yet. I hope I will never need to pay something to vercel, because I think vercel is pretty expensive…
НиколяOP
Yes, I agree, but did you have problems with traffic then? Because if you uploaded an 80GB video and there was a lot of traffic, then there must have been a lot of traffic, if I understand everything correctly and don’t confuse anything
@Николя Yes, I agree, but did you have problems with traffic then? Because if you uploaded an 80GB video and there was a lot of traffic, then there must have been a lot of traffic, if I understand everything correctly and don’t confuse anything
Yes you are right. There was a lot of traffic in that page and I never had performance issues or traffic issues. Because the 80gb are not at vercel. They are on s3. And s3 is cheap. Also to request the media I am using a cdn. So they are compressed and served on the correct size to save bandwidth. The cdn also caches the media. Like that vercel just does the image compression and hosting of the pages for me
Answer
Nothing more 🙂
@B33fb0n3 Yes you are right. There was a lot of traffic in that page and I never had performance issues or traffic issues. Because the 80gb are not at vercel. They are on s3. And s3 is cheap. Also to request the media I am using a cdn. So they are compressed and served on the correct size to save bandwidth. The cdn also caches the media. Like that vercel just does the image compression and hosting of the pages for me
НиколяOP
Oh, looks intresting, thanks so much bro, i will try
happy to help