Most AI data privacy at work problems do not start with a hacker. They start with a helpful employee who copies a block of text, drops it into a chat window and asks for a summary. The text might be a client email thread, a spreadsheet of names and phone numbers, or an intake note about someone’s mother. The question this post answers is simple: what should never go into that box, what can go in after you clean it up, and how to turn that into a rule your team will actually follow.
I work as Solutions Architect at a home health agency, and I am building Nuraflow, software that holds intake, scheduling and care records for home care and hospice. So I spend a lot of time thinking about where sensitive text travels after someone hits enter. This is the guide I would hand a new hire on day one.
Why pasting into AI is different from searching
When you type a search, you send a few words. When you paste into an AI tool, you often send the whole document, and it is stored on the provider’s servers for some period, under rules set by the account you are signed into. Three things decide what happens to it.
- Training. Some consumer plans let the provider use your chats to improve its models, depending on a setting. Anthropic, for example, gives Claude Free, Pro and Max users a choice, and its consumer terms update says data is kept for five years if you allow training, versus 30 days if you do not.
- Retention for safety. Even when content is not used for training, it can be logged. OpenAI’s API data controls page says API data has not been used for training since March 1, 2023 unless you opt in, but abuse monitoring logs are kept for up to 30 days by default.
- Leakage. NIST’s Generative AI Profile (NIST AI 600-1, July 2024) lists data privacy as one of twelve generative AI risks, describing leakage and unauthorized disclosure of health, location and other personal data. It also notes that models have revealed information from their training data during adversarial attacks, which it calls data memorization.
None of that means the tools are unsafe. It means the text you paste leaves your control, and the protection you get depends on the account, not on how careful the person typing feels.
The account matters more than the brand
The single biggest lever is whether your team uses a consumer account or a business account. Anthropic’s update says the consumer changes do not apply to services under its Commercial Terms, including Claude for Work, Claude for Education, Claude for Government and API use. OpenAI’s API documentation draws a similar line for API traffic. That split is the reason I tell business owners to stop debating which chatbot is “more private” and start asking which plan their staff are signed into.
| Question | Personal account on a free or paid consumer plan | Company-managed business or API account |
|---|---|---|
| Who controls the settings? | Each employee, individually | An admin, for everyone |
| Can chats be used for training? | Often, depending on a toggle the employee may never have opened | Generally not by default, per the provider’s commercial terms |
| What happens when the employee leaves? | The chat history leaves with them | The company keeps or deletes it |
| Can you sign a contract for regulated data? | Not as an individual on a personal login | Sometimes, on specific plans and endpoints |
That last row matters for healthcare. OpenAI’s data controls page describes using BAA-eligible API endpoints for PHI once a customer has executed its Business Associate and Healthcare Addendum and the organization is set up for it. That is a specific, contracted configuration. It is not the same as an employee opening a chat tab on a personal login.
The red list: never paste these
These are the categories I treat as off limits in any AI tool that your organization has not specifically approved and contracted for that kind of data. If a task needs one of these, the task belongs in your system of record, not in a chat window.
Health information about a real person
Under HIPAA, a vendor that creates, receives, maintains or transmits protected health information on behalf of a covered entity is a business associate, according to the definition in 45 CFR 160.103. Under 45 CFR 164.502(e), the covered entity may share PHI with a business associate only after getting satisfactory assurances, documented in a written contract, that the vendor will safeguard it. That contract is the Business Associate Agreement, and it has to exist before PHI goes anywhere near the tool. A nurse pasting a visit note into a personal chatbot to “tidy up the wording” has sent PHI to a vendor with no BAA. I covered the website side of this in my guide to a HIPAA compliant website for a home health agency, and the same logic applies here.
Passwords, API keys and access tokens
It happens all the time with technical work: paste a config file or an error log to get help debugging, and the file contains a database password or a live key. Treat any secret that has touched a chat as exposed and rotate it. The fix is cheap. The habit of not checking is expensive.
Government IDs and financial account numbers
Social Security numbers, driver’s license numbers, bank and card numbers. There is almost no legitimate drafting task that needs the real number. Replace it with “XXX” and the AI will do the same job.
Trade secrets and confidential client material
The federal definition of a trade secret in 18 U.S.C. 1839(3) includes the requirement that the owner has taken reasonable measures to keep the information secret. My reading, as a non-lawyer: if your pricing model, unreleased product plans or source code routinely get pasted into personal AI accounts, you are making that argument harder for yourself. The same caution applies to anything you received under an NDA. Your client did not agree to share it with your AI vendor.
HR and personnel records
Performance reviews, disciplinary notes, medical leave details, salary data. These are high harm if exposed and rarely need real names to draft or summarize.
The yellow list: paste only after you strip it
A lot of useful work sits in the middle. You want help rewriting a complaint response, summarizing a messy email thread or turning meeting notes into tasks. The text is not secret, but it has people in it. Here the rule is: remove the identifiers first, then paste.
For healthcare teams, HIPAA already gives you a ready-made checklist. The safe harbor de-identification method in 45 CFR 164.514(b) lists 18 types of identifiers to remove, including names, geographic units smaller than a state, all date elements except year that relate to the individual, phone numbers, email addresses, medical record numbers, account numbers and full face photos. It also requires that you have no actual knowledge the remaining information could identify the person. Even outside healthcare, I find that list the most practical scrubbing guide there is.
A before and after makes it concrete. Before:
Maria Lopez, 214-555-0182, called on March 3 about her father Robert, 84, discharged from the hospital after a hip replacement. Wants weekday morning visits in Garland.
After:
A daughter called about her father, in his 80s, recently discharged after a hip replacement. Wants weekday morning visits. Draft a warm reply explaining next steps for an assessment.
The AI writes an equally good reply from the second version. That is the test I use: if the output would be just as useful without the identifying details, the details should not go in.
One honest caveat. Stripping names is not the same as anonymizing. A rare diagnosis plus a small town plus a date can still point to one person, which is exactly why the safe harbor list includes geography and dates. When in doubt, generalize further or do not paste.
The green list: usually fine
- Your own public content: published blog posts, website copy, public job ads.
- Generic questions with no company data: “How do I write a polite overdue invoice email?”
- Templates and outlines you will fill in later, outside the tool.
- Code or formulas with secrets and real data removed.
- Already de-identified or synthetic sample data you created for testing.
A 30-second check before anyone pastes
- Which account am I in? Company-approved business account, or a personal login?
- Is there a real person in this text? Names, phone numbers, addresses, dates tied to them.
- Is there anything secret? Passwords, keys, pricing, client files, anything under NDA.
- Would the output be just as good without it? If yes, remove it.
- Would I be comfortable if this chat were read aloud to the client? If not, stop.
Turning this into a one-page AI policy
Rules that live in one manager’s head do not survive a busy Tuesday. Here is the order I would write it in for a small team.
- Name the approved tools and accounts. One or two, on business plans, managed by an admin. Everything else is “ask first”.
- Publish the red, yellow and green lists above, with one example from your own work in each.
- Set the toggles centrally. If you must allow personal accounts for now, show people where the training and history settings are and check them together.
- Say what to do after a mistake. Delete the chat, rotate any exposed secret, and tell one named person the same day. Make it safe to report.
- Review it every quarter. Provider terms change, as the Anthropic update shows. Re-read the policies of the tools you approve.
That fourth step matters more in healthcare. Under the HIPAA breach definition in 45 CFR 164.402, an impermissible disclosure of PHI is presumed to be a breach unless a risk assessment shows a low probability that it was compromised. Your privacy officer needs to hear about it quickly to make that call, so hiding a mistake is the worst outcome.
When not to use AI for the task at all
Sometimes the honest answer is to skip it. If scrubbing a document takes longer than writing the reply yourself, write the reply. If the task is really a lookup inside your CRM or EMR, a chat window is the wrong tool, and the better fix is a proper workflow. I made a similar argument in why most small businesses don’t need AI yet: organize the system first, then add AI where it helps.
Also, do not lean too hard on vendor promises. The FTC has said that model-as-a-service companies must honor their privacy and confidentiality commitments and may be liable if they do not. That is useful, but enforcement happens after the fact. It does not un-send the file your employee pasted. The risk also grows once AI can act on your systems rather than just chat, which I wrote about in AI agents vs chatbots for small business.
Frequently asked questions
Is it safe to paste customer emails into ChatGPT or Claude?
It depends on the account and the content. On a company-managed business plan with clear terms, a customer email with no sensitive details is usually reasonable. On a personal consumer account, remove names, contact details and anything confidential first, and check the training setting.
Does turning off training make a chat private?
No. It changes whether the content is used to improve models. Providers can still retain chats for a period for safety and abuse monitoring, as OpenAI’s and Anthropic’s documentation both describe. Treat it as reducing risk, not removing it.
Can a home health agency use AI with patient data?
Only through a vendor and configuration covered by a Business Associate Agreement, set up deliberately by the agency, not by individual staff. Until that exists, keep patient details out and use de-identified text.
What if someone already pasted something they shouldn’t have?
Delete the chat, rotate any passwords or keys involved, and report it internally the same day. If it involved PHI, your privacy officer needs to assess it under the breach rules. If it involved client data under contract, check what the contract requires.
Want help writing yours?
Most teams don’t need a 20-page policy. They need an approved tool, three short lists and a clear person to call when something goes wrong. If you want help setting that up, or choosing where AI fits safely in your operations, tell me how your team works today.
