Make page data available to LLMs for indexing
Unanswered
Britannia Petite posted this in #help-forum
Britannia PetiteOP
Has anyone experience with submitting data to LLMs (e.g. OpenAI) to scrape and potentially show our website in their answers? We are a ecommerce platform and would like to be referenced by common LLMs in their results where it makes sense.
We generated a txt file for each product page so that it is easily accessible for any LLM scraper.
My question is: Is there a strategy or way to best submit these txts files to e.g. OpenAI? Thinking like Google Search Console for OpenAI? I do not know if they constantly scrape pages and look for e.g. robots.txt files to start discovering a website. I want to make it as easy as possible for LLMs to read and fetch our content.
This is more a curiosity post and I would like to get some feedback from someone who has done this in the past.
Thanks for any pointers or hints!
We generated a txt file for each product page so that it is easily accessible for any LLM scraper.
My question is: Is there a strategy or way to best submit these txts files to e.g. OpenAI? Thinking like Google Search Console for OpenAI? I do not know if they constantly scrape pages and look for e.g. robots.txt files to start discovering a website. I want to make it as easy as possible for LLMs to read and fetch our content.
This is more a curiosity post and I would like to get some feedback from someone who has done this in the past.
Thanks for any pointers or hints!
1 Reply
Britannia PetiteOP
Anyone 🙂 ?