Next.js Discord

Discord Forum

Make page data available to LLMs for indexing

Unanswered
Britannia Petite posted this in #help-forum
Open in Discord
Britannia PetiteOP
Has anyone experience with submitting data to LLMs (e.g. OpenAI) to scrape and potentially show our website in their answers? We are a ecommerce platform and would like to be referenced by common LLMs in their results where it makes sense.

We generated a txt file for each product page so that it is easily accessible for any LLM scraper.

My question is: Is there a strategy or way to best submit these txts files to e.g. OpenAI? Thinking like Google Search Console for OpenAI? I do not know if they constantly scrape pages and look for e.g. robots.txt files to start discovering a website. I want to make it as easy as possible for LLMs to read and fetch our content.

This is more a curiosity post and I would like to get some feedback from someone who has done this in the past.

Thanks for any pointers or hints!

1 Reply

Britannia PetiteOP
Anyone 🙂 ?