Next.js Caching Issue
Answered
Schneider’s Smooth-fronted Caima… posted this in #help-forum
Schneider’s Smooth-fronted CaimanOP
I am experiencing an issue with Next.js where my content is not being updated, even after adding more blog articles in Payload CMS v3 (using MongoDB).
It shows fine with
Is this something to do with "steaming"?
Where
Here is the code for /src/app/page.tsx:
and here is the code for /src/components/RecentBlogPosts.tsx:
It shows fine with
pnpm run dev on localhost, perhaps because caching is turned off? But when I add new articles in Payload it doesn't show up for some reason.Is this something to do with "steaming"?
Where
<RecentBlogPosts /> should allow for that?Here is the code for /src/app/page.tsx:
import RecentBlogPosts from '@/components/RecentBlogPosts'
import React from 'react'
export default function page() {
return (
<div className="bg-red-300 text-2xl">
hello world
<RecentBlogPosts />
</div>
)
}and here is the code for /src/components/RecentBlogPosts.tsx:
import { getPayloadHMR } from '@payloadcms/next/utilities'
import React from 'react'
import config from '@/payload.config'
export default async function RecentBlogPosts() {
const payload = await getPayloadHMR({ config })
const blogPosts = await payload.find({
collection: 'blog-post',
})
return (
<div>
<h2>Recent Blog Articles</h2>
<div className="grid grid-cols-3 gap-10">
{blogPosts.docs.map((post: any) => {
return (
<div key={post.id} className="bg-gray-200 p-4">
<h3>{post.title}</h3>
<p>{post.description}</p>
</div>
)
})}
</div>
</div>
)
}Answered by Arinji
export const dynamic = "force-dynamic"
put that in the top line of your page
put that in the top line of your page
238 Replies
Dang @Schneider’s Smooth-fronted Caiman your using payload v3 for a prod site?
export const dynamic = "force-dynamic"
put that in the top line of your page
put that in the top line of your page
Answer
@Arinji Dang <@313298495417221121> your using payload v3 for a prod site?
Schneider’s Smooth-fronted CaimanOP
More for getting familiar, I think it’s a very exciting new CMS looking forward to v3 full release
@Arinji export const dynamic = "force-dynamic"
put that in the top line of your page
Schneider’s Smooth-fronted CaimanOP
Ty I will try that
@Arinji export const dynamic = "force-dynamic"
put that in the top line of your page
Schneider’s Smooth-fronted CaimanOP
The top of the component file or top of page.tsx?
@Arinji i use v2 xD
Schneider’s Smooth-fronted CaimanOP
Who is your hosting provider?
self hosted
i work in a server hosting compay
Schneider’s Smooth-fronted CaimanOP
Express is required for v2, right? Only v3 supports JAMstack hosting?
so we have a bunch of servers to use xD
@Arinji mhm, express
Schneider’s Smooth-fronted CaimanOP
Yeah the issue I am faced with is media upload
It is not persistent on Vercel unfortunately
yea :(
anyways did the export work?
Schneider’s Smooth-fronted CaimanOP
And also Sharp doesn’t work on Vercel
@Arinji anyways did the export work?
Schneider’s Smooth-fronted CaimanOP
Trying now
It works now 🙂 ty!
@Arinji export const dynamic = "force-dynamic"
put that in the top line of your page
Schneider’s Smooth-fronted CaimanOP
So does that mean I lose all forms of caching?
yea
what i do is wrap in unstable cache
and give it a tag
and in payload, when you change the data, you revalidate the tag
@Schneider’s Smooth-fronted Caiman
@Arinji what i do is wrap in unstable cache
Schneider’s Smooth-fronted CaimanOP
Is it all done on this file? page.tsx
inside the component @Schneider’s Smooth-fronted Caiman
@Arinji inside the component <@313298495417221121>
Schneider’s Smooth-fronted CaimanOP
Ty, sorry just to clarify how would that look exactly? Is this the proper way to do it or kinda a temporary solution for the time being 🙂 Payload or Next would most likely make this easier?
@Arinji https://nextjs.org/docs/app/api-reference/functions/unstable_cache
Schneider’s Smooth-fronted CaimanOP
Reason I’m asking is because I noticed this: “Warning: This API is unstable and may change in the future. We will provide migration documentation and codemods, if needed, as this API stabilizes.”
I also seem to recall “Streaming” as a feature in Nextjs not sure if that’s another way to do it, where certain components update automatically when changed, no refresh required afaik
@Schneider’s Smooth-fronted Caiman Reason I’m asking is because I noticed this: “Warning: This API is unstable and may change in the future. We will provide migration documentation and codemods, if needed, as this API stabilizes.”
eh, the fun part abt nextjs, is that unstable cache is oen of the best and most used and most stable feature of nextjs caching
@Arinji eh, the fun part abt nextjs, is that unstable cache is oen of the best and most used and most stable feature of nextjs caching
Schneider’s Smooth-fronted CaimanOP
Oh really, OK haha I will try to figure it out. Pretty easy to implement for each desired component that you do not want to cache?
unstable cache is for aggressive caching
@Arinji unstable cache is for aggressive caching
Schneider’s Smooth-fronted CaimanOP
So the rest of the site gets cached but the recent posts component stays fresh?
anything you wrap with unstable cache gets cached
@Arinji anything you wrap with unstable cache gets cached
Schneider’s Smooth-fronted CaimanOP
too advanced for me to understand lol
looks like a lot of work to fix caching
not really
Schneider’s Smooth-fronted CaimanOP
I was hoping there would be a simple way like
<RecentBlogPosts cache='false'/>lets start again, i think i got confused by your questions
What do you want to cache, what do you want to stay un cached.
Schneider’s Smooth-fronted CaimanOP
Well, the issue at the very beginning was that the RecentBlogPosts were not showing up, and the reason for that was Next.js caching
export const dynamic = "force-dynamic" fixed that, but removed all caching from the websiteimport { getPayloadHMR } from '@payloadcms/next/utilities'
import React from 'react'
import config from '@/payload.config'
export default async function RecentBlogPosts() {
const payload = await getPayloadHMR({ config })
const blogPosts = await payload.find({
collection: 'blog-post',
})
return (
<div>
<h2>Recent Blog Articles</h2>
<div className="grid grid-cols-3 gap-10">
{blogPosts.docs.map((post: any) => {
return (
<div key={post.id} className="bg-gray-200 p-4">
<h3>{post.title}</h3>
<p>{post.description}</p>
</div>
)
})}
</div>
</div>
)
}so you want the blogPosts to be cached
only
am i correct?
Schneider’s Smooth-fronted CaimanOP
yeah
ok so wrap only blogPosts with unstable cache
and give it a tag of "blog-posts"
and remove the export dynamic
ping me once done, show me updated code
@Arinji tsx
import { getPayloadHMR } from '@payloadcms/next/utilities'
import React from 'react'
import config from '@/payload.config'
export default async function RecentBlogPosts() {
const payload = await getPayloadHMR({ config })
const blogPosts = await payload.find({
collection: 'blog-post',
})
return (
<div>
<h2>Recent Blog Articles</h2>
<div className="grid grid-cols-3 gap-10">
{blogPosts.docs.map((post: any) => {
return (
<div key={post.id} className="bg-gray-200 p-4">
<h3>{post.title}</h3>
<p>{post.description}</p>
</div>
)
})}
</div>
</div>
)
}
Schneider’s Smooth-fronted CaimanOP
just curious, how did you get this code? lol
i can even see your env file
Schneider’s Smooth-fronted CaimanOP
hmmm, I thought the repo was private
LOL what
Schneider’s Smooth-fronted CaimanOP
you wrote it out? 😮
@Schneider’s Smooth-fronted Caiman you wrote it out? 😮
copy pasing also works :D
Schneider’s Smooth-fronted CaimanOP
oh OK, forgot that was up there hahaha
no worries lol
make the changes i said, and show me your unstable cache component so i can check the syntax
ping me once done :D
@Arinji make the changes i said, and show me your unstable cache component so i can check the syntax
Schneider’s Smooth-fronted CaimanOP
so i did give it an attempt but not sure I am following, when I try to wrap with this:
I get syntax errors for some reason
import { unstable_cache } from 'next/cache';
const getCachedUser = unstable_cache(I get syntax errors for some reason
unstable cache syntax is goofy
one sec
Schneider’s Smooth-fronted CaimanOP
kk 🙂
TopData = await unstable_cache(
async () => {
const dbData = await db
.collection("views")
.aggregate([ ])
.toArray();
return dbData;
},
["key"],
{ tags: ["homepage", "hiddenArticles"] }
)();@Schneider’s Smooth-fronted Caiman imma explain what it means
so here, TopData is whats gonna get the data, first param for unstable cache is an async callback which returns some data, second param is keyparts, you can keep it empty for this usecase, so just an empty array, and the final param are options, so for you usecase copy the tags part, and have "blog-posts" in there
Unstable cache returns a function, so to make it a normal variable we call the function at the end and await it
Unstable cache returns a function, so to make it a normal variable we call the function at the end and await it
and voila, unstable cache
@Arinji tsx
TopData = await unstable_cache(
async () => {
const dbData = await db
.collection("views")
.aggregate([ ])
.toArray();
return dbData;
},
["key"],
{ tags: ["homepage", "hiddenArticles"] }
)();
you dont need to
await unstable_cache if you dont call it again@ᴉuɐpɹɐɐ you dont need to `await` unstable_cache if you dont call it again
its an async function doe xD
Schneider’s Smooth-fronted CaimanOP
hmmm
I feel like I should watch this first https://www.youtube.com/watch?v=VBlSe8tvg4U
the return of the unstable_cache is an async function but unstable_cache itself is not an async funciton.
Here is the anatomy
Here is the anatomy
const fetchData = async () => {}
const fetchDataCached = unstable_cache(fetchData)
const data = await fetchDataCached()
const datb = await unstable_cache(fetchData)()@Schneider’s Smooth-fronted Caiman I feel like I should watch this first https://www.youtube.com/watch?v=VBlSe8tvg4U
def, its a good vid, dont watch other caching vids
most are outdated now
@ᴉuɐpɹɐɐ
@Arinji most are outdated now
Schneider’s Smooth-fronted CaimanOP
yeah, i take it this has a lot to do with App Router
any reason you went with Payload v2? will you upgrade to v3? or waiting for stable release before doing so
@Arinji <@194128415954173952>
screenshot the full function
@ᴉuɐpɹɐɐ screenshot the full function
TopData = unstable_cache(
async () => {
const dbData = await db
.collection("views")
.aggregate([
{
$group: {
_id: { $toObjectId: "$article" },
ipViews: { $sum: 1 },
},
},
{
$lookup: {
from: "articles",
localField: "_id",
foreignField: "_id",
as: "articleInfo",
},
},
{ $unwind: "$articleInfo" },
{
$match: {
"articleInfo.disabled": false,
"articleInfo.unlisted": false,
},
},
{
$addFields: {
totalViews: { $sum: ["$ipViews", "$articleInfo.InitialViews"] },
categoryId: { $toObjectId: "$articleInfo.Category" },
},
},
{ $sort: { totalViews: -1 } },
{ $limit: 5 },
{
$lookup: {
from: "categories",
localField: "categoryId",
foreignField: "_id",
as: "categoryInfo",
},
},
{ $unwind: "$categoryInfo" },
{
$project: {
totalViews: 1,
title: "$articleInfo.title",
url: {
category: "$categoryInfo.publicUrl",
article: "$articleInfo.link",
},
},
},
])
.toArray();
const parsedData = dbData
.map((article) => {
const parse = TopArticleSchema.safeParse(article);
if (parse.success) return parse.data;
})
.filter((article): article is TopArticleSchemaType => !!article);
return parsedData;
},
["key"],
{ tags: ["homepage", "hiddenArticles"] }
)();
}barely made the limit xD
@Schneider’s Smooth-fronted Caiman any reason you went with Payload v2? will you upgrade to v3? or waiting for stable release before doing so
it wasnt there when we started using v2, we are gonna shift when they add full support to the lexical editor
@ᴉuɐpɹɐɐ there is an `()` in the end
i mean yea, i wanted to call it instantly
thats why the await :D
did i not explain that
"Unstable cache returns a function, so to make it a normal variable we call the function at the end and await it"
@Arinji did i not explain that
oh sorry because your snippet ends with
} and not () i missed itno worries lol
i was just cutting my actual function short so its not a huge snippet
but still, remove the orphan
} maybetook out the orphan as per your request
we wont let orphans live in this forum post 🗣️ 🗣️ 🗣️
Schneider’s Smooth-fronted CaimanOP
@Arinji your portfolio is really amazing! and only 16 years old, wow
very impressive stuff
how long have you been doing web dev for?
@Arinji eh 2 yrs now
Schneider’s Smooth-fronted CaimanOP
you pick up very quick!
any programming experience before that?
not really, i actually hated html when it was taught in school
like the very basics
who would have known i would grow up and start using it for my future lol
pls keep the convo on-topic then we can move to #off-topic
:D
Original message was deleted
ask me in off topic, once we fix your caching issues
so did the unstable cache stuff make sense?
@Arinji so did the unstable cache stuff make sense?
Schneider’s Smooth-fronted CaimanOP
not yet, but I'm here at the timestamp where it's talked about:
https://youtu.be/VBlSe8tvg4U?si=goDdiumNqDuzI4Cj&t=458
https://youtu.be/VBlSe8tvg4U?si=goDdiumNqDuzI4Cj&t=458
It shows fine with pnpm run dev on localhost, perhaps because caching is turned off? But when I add new articles in Payload it doesn't show up for some reason.
Did you try adding
export const dynamic = 'force-dynamic'that did indeed work
to show recent posts
therefore solved?
@ᴉuɐpɹɐɐ therefore solved?
Schneider’s Smooth-fronted CaimanOP
but I think that i lose all forms of cache by using that
for the entire site afaik
not the entire site
jsut the page
@Schneider’s Smooth-fronted Caiman but I think that i lose all forms of cache by using that
can your cms calls an endpoint after each update/new post? is there a feature for that in your cms?
@Arinji jsut the page
Schneider’s Smooth-fronted CaimanOP
oh, but if I have Payload connected to every page, then basically the same thing lol
@ᴉuɐpɹɐɐ can your cms calls an endpoint after each update/new post? is there a feature for that in your cms?
Schneider’s Smooth-fronted CaimanOP
I believe so
@Schneider’s Smooth-fronted Caiman I believe so
also did you use generateStaticParams yet?
Schneider’s Smooth-fronted CaimanOP
Payload seems to be pretty flexible, more than any other CMS afaik
@ᴉuɐpɹɐɐ also did you use generateStaticParams yet?
Schneider’s Smooth-fronted CaimanOP
no
143 messages already. Please keep things concise and come to a solution soon.
forum getting heated fr fr lol
@ᴉuɐpɹɐɐ also did you use generateStaticParams yet?
not needed here :/
@joulev 143 messages already. Please keep things concise and come to a solution soon.
Schneider’s Smooth-fronted CaimanOP
is there a message quota we gotta keep it under?
its just a case of basic caching a function
and adding revalidation for it
@Schneider’s Smooth-fronted Caiman is there a message quota we gotta keep it under?
its just a general thing
but we shld take it seriously
@Schneider’s Smooth-fronted Caiman is there a message quota we gotta keep it under?
No. But if after 150 messages things are still going in circles, it will just be a huge waste of time to all parties involved
So please, keep things concise and come to a conclusion soon
@joulev So please, keep things concise and come to a conclusion soon
sry my bad for not reading the discussion. Arinji is getting closer to the conclusion. please let him continue
:D lol, lets finish this up shall we
@joulev So please, keep things concise and come to a conclusion soon
Schneider’s Smooth-fronted CaimanOP
https://nextjs.org/learn/
I have gone through the entire course, both for React Foundations and Learn Next.js
but afaik, this cache issue is not mentioned
I have gone through the entire course, both for React Foundations and Learn Next.js
but afaik, this cache issue is not mentioned
its normal behaviour
lets jus focus on fixing the cache for your component
sounds good?
Schneider’s Smooth-fronted CaimanOP
OK
import { getPayloadHMR } from '@payloadcms/next/utilities'
import React from 'react'
import config from '@/payload.config'
export default async function RecentBlogPosts() {
const payload = await getPayloadHMR({ config })
const blogPosts = await payload.find({
collection: 'blog-post',
})
return (
<div>
<h2>Recent Blog Articles</h2>
<div className="grid grid-cols-3 gap-10">
{blogPosts.docs.map((post: any) => {
return (
<div key={post.id} className="bg-gray-200 p-4">
<h3>{post.title}</h3>
<p>{post.description}</p>
</div>
)
})}
</div>
</div>
)
}Update this with the stuff i sent
@Arinji this
Schneider’s Smooth-fronted CaimanOP
ty, i am still not quite understanding so i am still going through the Vercel video on caching
any idea why
wouldn't it be better to have
any idea why
run dev removes caching altogether? (is that on purpose for some reason?)wouldn't it be better to have
run dev emulate the same thing run build on production would see? that way we are not surprised when things don't work the same way after it has been taken liveSchneider’s Smooth-fronted CaimanOP
14
14.+?
like whats after 14 lol
Schneider’s Smooth-fronted CaimanOP
afaik Payload v3 beta only works with Next14, it might even be canary, lemme double check
yeah, its
"next": "14.3.0-canary.28",that's just the version that was added when using
pnpx create-payload-app@betaany idea why run dev removes caching altogether? (is that on purpose for some reason?)
wouldn't it be better to have run dev emulate the same thing run build on production would see? that way we are not surprised when things don't work the same way after it has been taken live
Next.js only removes static site caching but not data caches. unstable_cache should still work on dev
@ᴉuɐpɹɐɐ > any idea why run dev removes caching altogether? (is that on purpose for some reason?)
> wouldn't it be better to have run dev emulate the same thing run build on production would see? that way we are not surprised when things don't work the same way after it has been taken live
Next.js only removes static site caching but not data caches. unstable_cache should still work on dev
Schneider’s Smooth-fronted CaimanOP
but as a simple example,
{Date.now()} is not cached on dev, but it is cached on buildthat might be because your page is statically generated
Schneider’s Smooth-fronted CaimanOP
yeah thats what I mean, why would dev cache differently? (it's done that way on purpose?)
why is static site caching removed? its not removed, it was never there to begin with. the dev server is only to make dynamic calls at request time while the static pages needs a different configuration to be rendered.
when the vid abt caching was released, the dev server worked differently, it didnt cache anything. everything was dynamic, now its much more similar to actual prod
all pages are rendered dynamically in
run dev but you still have the benefit of the data cache, which are the fetch() and the unstable_cache()@Arinji when the vid abt caching was released, the dev server worked differently, it didnt cache anything. everything was dynamic, now its much more similar to actual prod
Schneider’s Smooth-fronted CaimanOP
I tested and getting same result as video on caching tho (from 3 months ago)
https://payload-mongodb.vercel.app/
https://payload-mongodb.vercel.app/
this time will update on dev every refresh
but it stays the same on prod
but it stays the same on prod
using
{Date.now()}if you have failure to render on recent build then it wont show latest data
@Schneider’s Smooth-fronted Caiman tldr, dont go into cache in this forum
its a long process
the forum will go over 200, if we try to explain caching and its history to you atm
@Schneider’s Smooth-fronted Caiman this time will update on dev every refresh
but it stays the same on prod
try screenshotting the build log of your most recent build in vercel
@ᴉuɐpɹɐɐ try screenshotting the build log of your most recent build in vercel
Schneider’s Smooth-fronted CaimanOP
196.27 MB for build cache, wow, alright
your
/ is statically rendered at build time. therefore it doesnt change for every request1715704250683 -> today at 4pm gmt +0
@ᴉuɐpɹɐɐ your `/` is statically rendered at build time. therefore it doesnt change for every request
Schneider’s Smooth-fronted CaimanOP
can i enable that same build cache behavior for
run dev?so i am never surprised by cache
no but you can get used to it
i told you the signs of a statically rendered page
and how to check them
@ᴉuɐpɹɐɐ no but you can get used to it
Schneider’s Smooth-fronted CaimanOP
there must be a reason for it tho, right?
run build and run dev having different cache behaviourand i just told you that
Schneider’s Smooth-fronted CaimanOP
oh, sorry, where did you explain it?
dynamically rendered routes requires a server to process request and return SSR'd react page
statically rendered routes only requires a CDN to just give you the "built" pages / "built" apis / the .html
statically rendered routes only requires a CDN to just give you the "built" pages / "built" apis / the .html
while unintuitive,
npm run dev is basically a dev server to process request and return SSR'd react page. It will NOT statically render the page at every changes.its not a different cache behavior, its just how page rendering works
one thing they might do is to "simulate" it but it will never be made the same
one thing they might do is to "simulate" it but it will never be made the same
Schneider’s Smooth-fronted CaimanOP
ahhh, I see I see
so this is more to do with how CDN works for cached behaviour on production?
since we don't have a CDN on localhost,
so this is more to do with how CDN works for cached behaviour on production?
since we don't have a CDN on localhost,
npm run dev will act differently than npm run build which is more 1-to-1 with productionCDN is an oversimplification
but you get the point
Schneider’s Smooth-fronted CaimanOP
that's interesting.. I wonder if Nextjs is working on any sort of work around for that? where you see same cache behaviour on dev as you do for production?
once you do
dynamically rendered page will become a serverless function
static pages will become .html-like data that is stored on the disk
npm run build, the result for static pages and dynamically rendered page will go through a different folder and place.dynamically rendered page will become a serverless function
static pages will become .html-like data that is stored on the disk
@Schneider’s Smooth-fronted Caiman that's interesting.. I wonder if Nextjs is working on any sort of work around for that? where you see same cache behaviour on dev as you do for production?
it can try to "simulate" but its not the highest priority right now. and it will never be 100% has the same cache behavior. not only that, people may not like it since it requires you to know how to "reset" the statically cached dev pages
Schneider’s Smooth-fronted CaimanOP
npm run build is kinda annoying to do every time you wanna check for cachebc there is no hot reload with build, takes a while to do every time you wanna check
thats why best i can say to you is just to 'imagine' what it looks like, and npm run build once in a while
just imagine your page stuck in a snapshot of your current access
just imagine that nothing changes after you refresh it
Schneider’s Smooth-fronted CaimanOP
and like you said npm run build may behave different than a build on production, so there is that aspect of caching too lol
these days its not that large of a difference if we are exclusing static site generation
Schneider’s Smooth-fronted CaimanOP
caching seems to be a tricky thing, unexpected behaviours sometimes, i should really get to know it better to prevent surprises
you just missed a big gap in static vs dynamic rendering, every other cache works fine in
devSchneider’s Smooth-fronted CaimanOP
oh i see
so if I'm using a CMS like Payload, maybe I should avoid using static rendering altogether?
no?
just develop as normal
@ᴉuɐpɹɐɐ just develop as normal
Schneider’s Smooth-fronted CaimanOP
OK
the static build cache will be reset everytime you make a new deployment
so it will looks like the last time you saw it on local
npm run build@ᴉuɐpɹɐɐ the static build cache will be reset everytime you make a new deployment
Schneider’s Smooth-fronted CaimanOP
but i am not re-deploying every time I add content in Payload
your dynamically rendered routes will update that just fine
but your statically rendered routes will need cache invalidation (without redeployment).
this is why i suggested if your CMS can make a request to your next.js server thatd be nice.
Schneider’s Smooth-fronted CaimanOP
I actually plan to have a lot of control in Payload: the menu links, the footer links, the text on all pages, the color of the background
I don't know if there is a single thing that will stay "static" when I am done with the website, everything is dynamic in that sense where Payload can control it from within /admin (and without re-deployment)
I don't know if there is a single thing that will stay "static" when I am done with the website, everything is dynamic in that sense where Payload can control it from within /admin (and without re-deployment)
@ᴉuɐpɹɐɐ this is why i suggested if your CMS can make a request to your next.js server thatd be nice.
Schneider’s Smooth-fronted CaimanOP
I will see if this is doable, ty
but my server is actually JAMstack hosting, I don't use Express
next.js is a server. you can utilize that to handle request from outside
If the issue is resolved, kindly mark a answer as a solution to show to other contributors/helpers that the issue is resolved
@ᴉuɐpɹɐɐ next.js is a server. you can utilize that to handle request from outside
Schneider’s Smooth-fronted CaimanOP
fyi @ᴉuɐpɹɐɐ @Arinji I asked on Payload Discord and they recommended this:
https://payloadcms.com/docs/hooks/collections#afterchange
https://payloadcms.com/docs/hooks/collections#afterchange
i'll lock this one. further questions/discussions please open a new thread