I currently have a system in place using a library called rate-limiter-flexible and server actions.

In practice it looks something like this:

// generate image

"use server"

import { RateLimiterMemory } from "rate-limiter-flexible"
import { getIPAddress } from "utils"
import { getSession } from "your-auth-logic"

const rateLimiter = new RateLimiterMemory({
  points: 10, // number of points each user has
  duration: 60 * 60 * 24 // time it needs to pass for user's points to reset (1 day in seconds)
})

export async fucntion generateAiImage() {
  const isLoggedIn = await getSession()

  const ip = await (getIPAddress()) ?? "127.0.0.1" // get the users ip or use localhost for testing in dev


  if (typeof ip !== "string") {
    // ... handle if the ip was not retrived successfully
  }
  
  try {
    // if the user is logged in only consume 1 point giving him 10 images a day
    if (isLoggedIn) await rateLimiter.consume(ip, 1) 
    // if user is not logged in consume 5x the points giving a normal user 2 images per day
    else await rateLimiter.consume(ip, 5)
  } catch(e) {
    console.log(e)
    // ... handle the error in case a user has reached his limit
  }

  // ... your generate image logic
}

We can use this util function to get the users IP address in server actions:

// utils.ts

"use server"

import { headers } from "next/headers"

export async function getIPAddress() {
  return headers().get("x-forwarded-for")
}

I find this solution easier to implement than upstash and middleware and so far it has been working great without issues. I am still waiting to test it on a more serious load application though, but if that becomes a problem the library does provide a RateLimiterRedis solution aswell.

I'm not sure if the memory will work perfectly, have you deployed it?

it works when deployed to vercel and correctly logs my ip address

I need a solution which works in network layer basically

assuming I Need to scale to 10-20K users, is it a good idea to store it on memory only?

and is the memory shared between different serverless function instances

You will have to test this out, but adding Redis later on shouldn't be a problem or expensive.

As for if the memory is shared between instances of the serverless function it seems so.

I have tried spamming the function from my computer until I was blocked, then from my phone which was connected to the same router as my computer (so they had the same IP address) and the phone was also blocked.

Won’t that memory be lost once there are no requests?

I don't know 😅

Test it out

Oh ok, KV seems to get good with $0.5/million reqs

Oh wait, write requests are expensive at $5/million

Any suggestions from where should I get Redis/KV?

My current solution is buying a VPS on Hetzner and hosting everything in Docker. You can get a really cheap one for less than $3.4/mo. That is where I host all my postgres, redis, express, mailservers and such. If the app get's slow you can rescale it, but be careful as it does not scale infinitely like the one vercel and cloud providers provide.

I basically need something that scales, so I can't get a VPS

Cloudflare KV is a lil expensive

probably AWS if you want good pricing as vercel just "drop ships" their database with improved DX. Vercel is always an option if you want something more expensive but "just works".

I use dynamoDB as primary database, I'll see how using it for that will work out

DynamoDB Will cost $3.25, cloudflare KV will cost $5.5 for million reqs