Some good old system design
Answered
Dovekie posted this in #help-forum
DovekieOP
Hello everyone, how you doing ?
So I want to create a little app that consists of 2 main parts system-design-wise - and I'd like to know where you think I should host it (could be multiple platforms).
Code Stack: NextJS,
FIRST PART: Your good old CRUD app where users can create, read, update stuff. Nothing crazy here.
SECOND PART: A job that's reading rows from a SQL table. Then for each row it performs some requests on an API, then based on that result it updates 2 other tables and it also might need to save media (pictures, 20s videos) on some blob storage (think S3 or something). This job will be running regularly since its function is to keep stuff up to date. Now, how many entires could the job table have ? Think 10-100. And then for each of these entries we might need to perform another 100-200 requests to check if data needs updating. Then 80% of the time the data will need updating and 50% of the time we will also need to put some sort of media into a blob storage. I want the system to be up to date every 5m. Ofc if the app is succesfull things will need to change in the future but that's it for now.
MY TAKE:
For the first part. Vercel. $20 / month and we're well below reaching any limits
For the second part:
AWS EC2 for the job since Lamba wont work due to it running continuously and sometimes having to deal with media (20s videos).
RDS for a SQL (or DynamoDB) that's going to be getting getting updated regularly.
S3 for Blob storage
So I want to create a little app that consists of 2 main parts system-design-wise - and I'd like to know where you think I should host it (could be multiple platforms).
Code Stack: NextJS,
FIRST PART: Your good old CRUD app where users can create, read, update stuff. Nothing crazy here.
SECOND PART: A job that's reading rows from a SQL table. Then for each row it performs some requests on an API, then based on that result it updates 2 other tables and it also might need to save media (pictures, 20s videos) on some blob storage (think S3 or something). This job will be running regularly since its function is to keep stuff up to date. Now, how many entires could the job table have ? Think 10-100. And then for each of these entries we might need to perform another 100-200 requests to check if data needs updating. Then 80% of the time the data will need updating and 50% of the time we will also need to put some sort of media into a blob storage. I want the system to be up to date every 5m. Ofc if the app is succesfull things will need to change in the future but that's it for now.
MY TAKE:
For the first part. Vercel. $20 / month and we're well below reaching any limits
For the second part:
AWS EC2 for the job since Lamba wont work due to it running continuously and sometimes having to deal with media (20s videos).
RDS for a SQL (or DynamoDB) that's going to be getting getting updated regularly.
S3 for Blob storage
Answered by tafutada777
there are three options, AWS, GCP and Azure. if you’re familiar with AWS, you should go with AWS. if you have plans to use Open AI APIs, Azure would be nice. i use GCP k8s btw. i don’t recommend multiple platforms as the other days i got network trouble from GCP to Azure, which made me crazy.
5 Replies
do you mean where to host the first part, Next.js, on Vercel or AWS?
actually, Vercel deploys js codes on AWS lambda(except for Edge Runtime), so you should host it on the same AWS VPN, so I/O got faster.
DovekieOP
Im wondering where I should host the entire thing actually, given the requirenments I mentioned above
there are three options, AWS, GCP and Azure. if you’re familiar with AWS, you should go with AWS. if you have plans to use Open AI APIs, Azure would be nice. i use GCP k8s btw. i don’t recommend multiple platforms as the other days i got network trouble from GCP to Azure, which made me crazy.
Answer
DovekieOP
Copy that. Thank you very much for your input!