Any ideas for LLM selection for RAG

Question

We've been developing an enterprise RAG that answers questions based on sensitive data. So, Is there any better LLM which perform well for RAG and that can be hosted on AWS servers.

qwertyCoder · Accepted Answer

There's many sites that show rankings of LLM's have you looked at these? They often rank LLM's by task so you can see which task is most relevant. Example: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

softwaremagician · Answer

LLMs in a public cloud dealing with sensitive data seems to be a receipt for trouble. even worse, trouble that is hard to predict and that may span way out of your company's intended scope. I suggest you rethink carefully the risks and re-evaluate how to protect the private data during training and after the LLM is ready to be deployed. Think about how pro.pt exploitation could make the LLM spill out sensitive data from other subjects, or out of context, and the risks for the data owners of the private data. (I see litigation coming hard to your door)

Any ideas for LLM selection for RAG

About

Software Engineer

Members