Next.js Discord

Discord Forum

Help sharing cache across Docker containers

Unanswered
Chippiparai posted this in #help-forum
Open in Discord
ChippiparaiOP
My team is using NextJS with Docker... it's working great in a single container, but I'm struggling to figure out how to have a shared cache across multiple containers. Does anyone have a good experience doing this?

Can we just mount the .next folder on a shared volume? Can you persist the cache across multiple builds so that it doesn't have to regenerate ALL the images every time we redeploy?

I also see there is the ability to create a custom cache-handler.js file that could work with S3 (which I think would be a good solution) but I'm struggling to figure out the details there and what I'm doing isn't working. Is there a good example of that? How would that work across builds? Does that work with images?

3 Replies

ChippiparaiOP
Good to know: I was looking at https://nextjs.org/docs/app/building-your-application/deploying#configuring-caching however there is better information about this API at https://nextjs.org/docs/app/api-reference/next-config-js/incrementalCacheHandlerPath

I will experiment interfacing this with S3 and report back if I get anywhere.
ChippiparaiOP
Ok, got somewhere with it, but more testing needed.

Running locally, it slows down the site quite a bit. A different AWS product like DynamoDB or ElastiCache might be a better choice, albeit more expensive. Behind CloudFront it might be fine too. It also doesn't seem to be actually caching like it should. I.e. if I make a change in my CMS, I see that change almost immediately when I refresh.

This cache-handler applies to fetch-cache only and does NOT include the image cache, so I will pursue other avenues for that.

// cache-handler.js
const {
    uploadToS3,
    getFromS3,
    deleteFromS3,
    listKeysTagsFromS3,
} = require("./s3-helpers");

module.exports = class CacheHandler {
    constructor(options) {
        this.options = options;
    }

    async get(key) {
        try {
            const data = await getFromS3(key);
            return data ?? null;
        } catch (error) {
            console.log(error);
        }
    }

    async set(key, data, ctx) {
        try {
            if (data) {
                await uploadToS3(
                    key,
                    JSON.stringify({
                        ...data,
                        lastModified: Date.now(),
                        tags: ctx?.tags,
                    })
                );
            }
        } catch (error) {
            console.log(error);
        }
    }

    async revalidateTag(tag) {
        try {
            const cacheList = await listKeysTagsFromS3();
            // Iterate over all entries in the cache
            for (const key in cacheList) {
                // If the value's tags include the specified tag, delete this entry
                if (cacheList[key]?.includes(tag)) {
                    console.log("Deleting", key);
                    deleteFromS3(key);
                }
            }
        } catch (error) {
            console.log(error);
        }
    }
};
// s3-helpers.js
const {
    S3Client,
    PutObjectCommand,
    GetObjectCommand,
    DeleteObjectCommand,
    ListObjectsV2Command,
} = require("@aws-sdk/client-s3");

const BUCKET_NAME = "next-cache-test";

const s3Client = new S3Client({
    region: "us-east-1",
    credentials: {
        accessKeyId: "yourId",
        secretAccessKey: "yourKey",
        sessionToken: "yourToken",
    },
});

async function uploadToS3(key, data) {
    const params = {
        Bucket: BUCKET_NAME,
        Key: key,
        Body: data,
    };

    try {
        await s3Client.send(new PutObjectCommand(params));
        console.log("Successfully uploaded data to S3 bucket");
    } catch (error) {
        console.error("Error while uploading to S3:", error);
    }
}

async function getFromS3(key) {
    const params = {
        Bucket: BUCKET_NAME,
        Key: key,
    };

    try {
        const response = await s3Client.send(new GetObjectCommand(params));
        console.log("Successfully fetched data from S3 bucket");
        return response.Body.transformToString();
    } catch (error) {
        console.error("Error while fetching from S3:", error);
        console.error("key:", key ?? "unknown");
    }
}