CORS problem to upload on S3
Unanswered
American Chinchilla posted this in #help-forum
American ChinchillaOP
Hello,
Has anyone managed to upload files to an S3 via the nextjs frontend?
I don't have access to my provider's CORS, but it is configured as Allow , but I still have CORS problems:
I tried with an Allow CORS extension, but no result. Has anyone already had this problem and managed to solve it?
Has anyone managed to upload files to an S3 via the nextjs frontend?
I don't have access to my provider's CORS, but it is configured as Allow , but I still have CORS problems:
Access to fetch at 'https://restream.s3.swiss-ba*03.infom*k.com/restream/uploads/videos/tz82z5c1p*8d46z6c6zkmm_how-m*ess-right-half*s-lego-legobuild-1080-y*ube.me.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ebedb2dd44e548e18b0daed966a164da%2F20241001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241001T082725Z&X-Amz-Expires=3600&X-Amz-Signature=a90663f4aeb2*c075aef06d10&X-Amz-SignedHeaders=host&response-content-type=video%2Fmp4&x-id=GetObject' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.I tried with an Allow CORS extension, but no result. Has anyone already had this problem and managed to solve it?
30 Replies
@American Chinchilla Hello,
Has anyone managed to upload files to an S3 via the nextjs frontend?
I don't have access to my provider's CORS, but it is configured as Allow , but I still have CORS problems:
js
Access to fetch at 'https://restream.s3.swiss-ba*03.infom*k.com/restream/uploads/videos/tz82z5c1p*8d46z6c6zkmm_how-m*ess-right-half*s-lego-legobuild-1080-y*ube.me.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ebedb2dd44e548e18b0daed966a164da%2F20241001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241001T082725Z&X-Amz-Expires=3600&X-Amz-Signature=a90663f4aeb2*c075aef06d10&X-Amz-SignedHeaders=host&response-content-type=video%2Fmp4&x-id=GetObject' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
I tried with an Allow CORS extension, but no result. Has anyone already had this problem and managed to solve it?
I like to configure my CORS inside my S3 like this:
Like that everyone can upload to it as long as they have a policy. So create a upload policy for your nextjs frontend client with your backend. You might want to take a look at this: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html
Like that you can upload files directly and securly from your client to the s3 server
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"POST",
"GET"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": [
"x-amz-server-side-encryption",
"x-amz-request-id",
"x-amz-id-2",
"ETag"
],
"MaxAgeSeconds": 3600
}
]Like that everyone can upload to it as long as they have a policy. So create a upload policy for your nextjs frontend client with your backend. You might want to take a look at this: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html
Like that you can upload files directly and securly from your client to the s3 server
American ChinchillaOP
Unfortunately I am on a Backup service from Swiss-Backup which seems to be on an Amazon S3, but I do not have access to the CORS configuration of the instance.
But logically I think that the allowOrigin must be configured on * since from the backend I can reach it to upload files.
I was able to have a pre-signed url for my bucket in S3, but when I upload a file I have a problem:
But logically I think that the allowOrigin must be configured on * since from the backend I can reach it to upload files.
I was able to have a pre-signed url for my bucket in S3, but when I upload a file I have a problem:
Presigned URL generated successfully: https://restr*m.s3.us-east-1.amazonaws.com/how-ma*-ytshorts.savetube.me.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ebedb2dd*1%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241001T085432Z&X-Amz-Expires=120&X-Amz-Signature=830d868*e905129&X-Amz-SignedHeaders=host&x-id=PutObject
create:1 Access to fetch at 'https://restream.s3.us-east-1.amazonaws.com/how-many-*horts.savetube.me.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ebedb*001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241001T085432Z&X-Amz-Expires=120&X-Amz-Signature=830d*41fe905129&X-Amz-SignedHeaders=host&x-id=PutObject' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
page.jsx:41
PUT https://res*r*am.s3.us-east-1.amazonaws.com/how-m*chris-lego-legobuild-1080-yt*e.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ebedb2*%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241001T085432Z&X-Amz-Expires=120&X-Amz-Signature=830*1fe905129&X-Amz-SignedHeaders=host&x-id=PutObject net::ERR_FAILED
uploadToS3 @ page.jsx:41
await in uploadToS3
handleVideoUpload @ page.jsx:62my generatePRe-signed :
and my post function S3 :
async generatePresignedUrl(fileName, contentType) {
const params = {
Bucket: process.env.NEXT_PUBLIC_BUCKET,
Key: fileName,
ContentType: contentType,
};
try {
const command = new PutObjectCommand(params);
// Generate the presigned URL with a 60 seconds expiration
const signedUrl = await getSignedUrl(this.s3Client, command, { expiresIn: 120 });
console.log('Presigned URL generated successfully:', signedUrl);
return signedUrl;
} catch (error) {
console.error('Error generating presigned URL:', error);
throw error;
}
}and my post function S3 :
async function uploadToS3(file) {
try {
const signedUrl = await S3Service.generatePresignedUrl(file.name, file.type);
const response = await fetch(signedUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': 'application/json'
},
});
if (!response.ok) {
throw new Error('Failed to upload the file to S3');
}
return
} catch (error) {
console.error('Error uploading file to S3:', error);
throw error;
}
}if so, u should upload to your route handler first, and let that upload it to S3
This eliminates all Cors and headers issues
@gin if so, u should upload to your route handler first, and let that upload it to S3
yes, it would eliminate this issue, but he will also hit the limits of nextjs bodysize. And I guess that's not what he want
@B33fb0n3 1 MB 💀
uhmm
and u cant change that? lmao
@gin and u cant change that? lmao
you can, but I don't want to think about DDoS attacks or other potential stuff, that can be done like that. Uploading it from the client directly prevent all that (no policy form the server = no upload)
if he manages to fix his s3 config sure
i dont even know why we discussing if the author didnt reply
lol
ima head out
@gin i dont even know why we discussing if the author didnt reply
haha yea, kinda stupid. But we can't do anything against it, when OP don't want to talk with us 🤷♂️
American ChinchillaOP
In fact I do video uploads, certainly can go up to 20GB, before I used a chunk system to send my videos to the backend which then sent the complete video to the S3.
But now that I have a lot of users, it makes big performance problems while I could directly send to the S3 from the client.
But now that I have a lot of users, it makes big performance problems while I could directly send to the S3 from the client.
I want to talk, I didn't see it right away haha
@American Chinchilla In fact I do video uploads, certainly can go up to 20GB, before I used a chunk system to send my videos to the backend which then sent the complete video to the S3.
But now that I have a lot of users, it makes big performance problems while I could directly send to the S3 from the client.
yea... get the policy from your backend, sent it back to the client, so the client can directly upload it to s3
@B33fb0n3 yea... get the policy from your backend, sent it back to the client, so the client can directly upload it to s3
American ChinchillaOP
That is to say that the getSignedUrl must come from the server?
@American Chinchilla That is to say that the getSignedUrl must come from the server?
it should yea. At the end it will look like this:
Client -- request upload --> nextjs server -- generates url & policy -- send back to client --> Client -- selects file & send with policy --> s3 -- saves it --|
Client -- request upload --> nextjs server -- generates url & policy -- send back to client --> Client -- selects file & send with policy --> s3 -- saves it --|
@B33fb0n3 it should yea. At the end it will look like this:
Client -- request upload --> nextjs server -- generates url & policy -- send back to client --> Client -- selects file & send with policy --> s3 -- saves it --|
American ChinchillaOP
Do you have an example or a tutorial because I have used the presigned on the server side and sent it on the client side to use it I still have my error :
Access to fetch at 'https://s3.swiss-backup03.infomaniak.com/restream' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.@American Chinchilla Do you have an example or a tutorial because I have used the presigned on the server side and sent it on the client side to use it I still have my error :
Access to fetch at 'https://s3.swiss-backup03.infomaniak.com/restream' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
yea, this is one example for that: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html
As you can see you need way more stuff from the policy than just the presigned url
As you can see you need way more stuff from the policy than just the presigned url
American ChinchillaOP
Since this morning I've been trying to pre-sign a link from the backend to the frontend (nextjs) via my adonisjs server so that I can upload videos to my Swiss backup S3 Bucket. I knew I had to generate a pre-signed url with the policy. But when I try to use the link from the frontend and do a POST fetch on it, it gives me an error:
Can you see an error in my backend method or have I got something wrong?
Access to fetch at 'https://s*ss-bac*p03.infomaniak.com/res*ffm' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Can you see an error in my backend method or have I got something wrong?
import { S3Client } from '@aws-sdk/client-s3'
import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
static async generateS3UploadPolicy(fileName: string, fileType: string) {
const bucket = process.env.S3_BUCKET
const region = process.env.S3_REGION
const accessKeyId = process.env.S3_ACCESS_KEY_ID
const secretAccessKey = process.env.S3_SECRET_ACCESS_KEY
const endpoint = process.env.S3_ENDPOINT
const s3Client = new S3Client({
region,
credentials: {
accessKeyId,
secretAccessKey,
},
endpoint,
})
const key = `restream/uploads/${DateTime.now().toFormat('yyyyMMdd_HHmmss')}_${fileName}`
const params = {
Bucket: bucket,
Key: key,
Conditions: [
['starts-with', '$key', 'restream/uploads/'],
{ acl: 'private' },
['content-length-range', 0, 10485760],
['eq', '$Content-Type', fileType],
],
Fields: {
key,
'Content-Type': fileType,
'acl': 'private',
},
Expires: 120 * 60,
}
try {
return await createPresignedPost(s3Client, params)
} catch (error) {
throw new Error(`Failed to generate S3 upload policy: ${error.message}`)
}
}Note that when I use the flydrive drive for adonisjs in S3 mode, the upload works very well. But my goal is to be able to upload from the frontend to the S3.
@American Chinchilla Since this morning I've been trying to pre-sign a link from the backend to the frontend (nextjs) via my adonisjs server so that I can upload videos to my Swiss backup S3 Bucket. I knew I had to generate a pre-signed url with the policy. But when I try to use the link from the frontend and do a POST fetch on it, it gives me an error:
js
Access to fetch at 'https://s*ss-bac*p03.infomaniak.com/res*ffm' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Can you see an error in my backend method or have I got something wrong?
localhost won't sent any
The header will be automatically send with, when it's activated when beeing in a local enviorement. It looks like that's why it's working fine when you use the flydrive drive for adonisjs in s3 mode
Access-Control-Allow-Origin with the request. To still be able to try everything, you can use extensions like this: https://chromewebstore.google.com/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlfThe header will be automatically send with, when it's activated when beeing in a local enviorement. It looks like that's why it's working fine when you use the flydrive drive for adonisjs in s3 mode
@American Chinchilla solved?