RAG Data Security: What to Share in a Pilot, and What to Keep Back
RAG data security in a pilot starts with sharing less. A small, well-chosen sample of everyday documents is enough to learn whether a RAG works; content with personal data, commercial terms or legal holds can wait. Before anything is uploaded, ask where the sample will live, how personal data is masked, whether it trains anything, and what happens at the end.
Key takeaways
- You don't need your whole knowledge base to learn whether a RAG works — a small sample is enough.
- Share everyday procedures and guidance; think twice about personal data and contracts; keep legal holds back.
- A small sample is easier to approve, easier to review and quicker to set up.
- Ask five questions before uploading: storage, masking, training, logging and what happens at the end.
- Choose a deployment model that matches your data boundary: hosted, dedicated or your own cloud.
You do not need to hand over your whole knowledge base to learn whether a RAG system works. A small, well-chosen sample is enough — and it is the easiest way to keep RAG data security simple during a pilot. The question is which documents to share, which to think twice about, and what to ask before anything leaves your hands.
Why start with a small sample?
A team we worked with began with a small sample of their own documents. A small set is easier to approve, easier to review and quicker to set up, and it still shows how the system handles your real content. It also keeps the security conversation proportionate: approving a few hundred everyday documents is a very different request from approving the whole estate.
The goal of the sample is to be representative, not complete. Include the mix your staff actually work with — procedures, long PDFs, scanned pages, old versions — so the test reflects reality. The method behind this kind of pilot is described in try before you build.
What should you share, and what should you keep back?
| Share | Think twice | Keep back for now |
|---|---|---|
| Procedures, policies, guidance and manuals your staff already use | Documents with names, contact details or other personal data | Anything under legal hold, deal-sensitive or subject to strict confidentiality |
| Internal knowledge that is widely available inside the company | Contracts and commercial terms | Data your rules do not allow to leave your environment |
A pilot answers one question: will this work on our documents for our people? You can answer it with everyday content, and save the sensitive material for a deployment that has already earned trust.
Which data-security questions should you ask a partner?
Before uploading anything, ask every partner the same five questions — and ask for answers you can check.
- Where will the sample be stored, and who can see it? The answer should name the environment and the people with access.
- Is personal data masked, and how do we check that? Masking should happen before content reaches a model, and you should be able to see it working.
- Is our content used to train anything? The answer should be a clear no, in writing.
- What is logged, and can we see the log? Every question, answer and source should be recorded, and the record should be available to you.
- What happens to the sample when the pilot ends? Deletion should be explicit and confirmed.
How does SphereIQ handle a pilot sample?
SphereIQ can run hosted by Sphere, in a dedicated single-tenant environment, or in your own cloud, so a pilot can stay inside the boundary your security team requires. Answers are built only from documents each person is allowed to read, personal data is masked by policy, and every question and answer is written to an audit trail. SphereIQ does not train models on your data.
If your rules require the data never to leave your own environment, the pilot can run there too — see self-hosted LLMs in your VPC. For how access rules travel with every document, see RAG access control, and for what the audit record should let you prove, see the AI audit trail.
A simple way to decide
When in doubt about a document, leave it out of the first sample. You can always widen the pilot once the basics — access, masking and logging — have been checked on content nobody is worried about. The pilot is there to answer whether the system works on your knowledge, not to test how much risk your security team will accept.
Frequently asked questions
How much data do we need to share for a RAG pilot?
Should personal data be included in a pilot?
Is our content used to train models?
Where does the sample live during the pilot?
Agree the sample, then see it work.
In a walkthrough we help you choose a safe, representative sample and show how SphereIQ handles access, masking and logging on it.