In many companies, someone is already using an AI assistant to draft emails, summarize a contract or review code. The problem is not using it, but using it without rules. In 2023, the UK's NCSC warned that whatever is typed into a public assistant is visible to the provider and will almost certainly be used at some point to develop the service. Its recommendation: do not include sensitive information in those queries.
The joint guidance Engaging with Artificial Intelligence (January 2024), led by Australia's cyber security centre with CISA, the FBI, the NSA, the UK NCSC and other agencies, makes the same point: what is given to these systems may end up in their training data and inform responses to users outside the organization.
This article summarizes the risks, what to ask a provider and a one-page use policy you can adapt. It closes with a checklist.
The reference frameworks
- NIST AI RMF 1.0 (January 2023): a voluntary framework for managing AI risk, with four functions: govern, map, measure and manage. NIST says it is being revised.
- NIST AI 600-1 (July 2024): its generative AI profile. It defines 12 risks unique to or exacerbated by this technology.
- OWASP Top 10 for LLM Applications 2025: written for those who build applications on language models, although several of its risks reach any user. OWASP published a 2026 edition in August 2026 with different numbering; this article cites the 2025 edition.
- ISO/IEC 42001:2023 (December 2023): specifies requirements for establishing, implementing, maintaining and continually improving an AI management system. According to ISO, it serves organizations of any size that develop, provide or use AI-based products or services.
Six concrete risks
1. Leaking confidential or personal data
This is the most immediate risk. OWASP describes it in LLM02:2025: personal, financial and health data or confidential business information can be exposed. NIST AI 600-1 adds that models may leak, generate or correctly infer sensitive information about individuals, even data nobody gave them, by stitching together different sources.
2. Hallucinations
NIST calls them confabulation: erroneous or false content stated with confidence, which can mislead the reader. OWASP (LLM09:2025) adds overreliance: trusting the output without verifying it. The joint guidance cites a 2023 case: a federal judge in New York reportedly found at least six hallucinated cases in a legal brief, and the lawyer attributed them to research done with an AI assistant.
3. Intellectual property
NIST AI 600-1 points to two sides. Outward, AI makes it easier to produce or replicate copyrighted or trademarked content without authorization, and the legal status of generated content is still being debated. Inward, it makes it easier to expose trade secrets. The joint guidance calls for training staff on what must not be entered, such as personal data or the organization's intellectual property.
4. Bias
NIST describes the amplification of historical and societal biases, and performance disparities between groups or languages that can lead to discrimination or ill-founded decisions. The NCSC also warns that these models can be biased. This matters most when AI takes part in decisions about people: screening applicants, performance reviews, credit.
5. Vendor dependence
NIST AI 600-1 includes value chain risk: a limited number of models is reused and third-party components are integrated, and when something fails it can be hard to tell where it came from. The joint guidance recommends knowing the provider's availability commitments, putting incident responsibilities in the contract and knowing what happens to your data when the relationship ends.
6. Shadow AI
This is the use of tools nobody approved: personal accounts, browser extensions or AI features inside software you already use. Without an inventory you don't know what data left or where it went. NIST AI 600-1 recommends adding generative AI to the system inventory, keeping lists of approved providers and extending acceptable use policies to contractors and consultants.
If you connect an assistant to your email or files, there is one more risk: OWASP (LLM01:2025) describes indirect prompt injection, which arrives from websites or files. It recommends giving the model minimum permissions and requiring human approval for privileged operations.
What to ask the provider
Before approving a tool, ask for written answers. The questions come from the joint guidance, the NCSC and OWASP.
Question | Why |
|---|---|
Do you use my inputs to train the model? Can I opt out? | The joint guidance asks you to find out and to consider private versions; OWASP recommends allowing opt-out |
How long do you keep prompts, files and responses, and how are they deleted? | OWASP calls for clear retention, usage and deletion policies |
What happens to my data when the contract ends? | It is usually in the privacy policy or terms, according to the joint guidance |
Where is it stored and processed? | Cloud services may move data between regions |
Who at the provider or its partners can see it? Is it mixed with other customers' data? | These are questions the NCSC raises |
Is there single sign-on, phishing-resistant MFA, central account management and usage logs? | The joint guidance calls for phishing-resistant MFA, least privilege and input logging |
What availability do you commit to, and who does what in an incident? | The joint guidance asks for this in the contract |
How did you develop and test the system? | The joint guidance favors vendors that are transparent about it |
Read the terms and privacy policy of the plan you are buying, not those of the free version. The NCSC says they need to be thoroughly understood before asking sensitive questions.
The framework in Chile
Chile's National Artificial Intelligence Policy was published in 2021 with three pillars: enabling factors, development and adoption, and governance and ethics. According to the Ministry of Science, the progress of generative AI prompted an update of the governance and ethics pillar in 2023.
There is no AI law yet. The bill regulating artificial intelligence systems (bulletin 15869-19, merged with 16821-19) was passed by the Chamber of Deputies on October 13, 2025. As of November 4, 2026, it is in its second reading in the Senate, awaiting the first report of the Future Challenges Committee.
Law 21.719, which amends Law 19.628 on personal data, does address this. It defines data processing as any operation that allows personal data to be collected, processed, stored, communicated, transmitted or used: pasting customer data into an external assistant falls within it. Its article 15 bis regulates processing through a third-party processor, who must follow the controller's instructions and may not use the data for any other purpose. Article 8 bis gives data subjects the right to object to decisions based on automated processing that significantly affect them, with exceptions. The law takes effect on December 1, 2026. On October 28, 2026, the Senate passed a bill (bulletin 18.623-07) that postpones it to October 15, 2027, and it is in its second reading in the Chamber; until it is published, the date in force is December 1, 2026. This summarizes legal texts; it is not legal advice.
Generative AI use policy (one page)
Copy it, fill in the brackets and adjust it. Keep it to one sheet: a policy nobody reads does not work.
Generative AI use policy · [Company] · version [1.0] · [date]
- Scope. Applies to employees, contractors and consultants who use tools that generate text, images, audio or code, including AI features in software we already use.
- Approved tools. Only tools in the approved tools register are used, with company accounts, never personal ones. To add one, ask [IT owner], who puts the provider questions to the vendor before approving it.
- Data never entered into a tool not approved for it: personal data of customers, employees or applicants; sensitive data, such as health, biometric or socioeconomic data; passwords, API keys and tokens; unpublished financial information; documents under a confidentiality agreement; our own source code; security configurations and vulnerabilities.
- Human review. A person reviews every output before it is sent, published or run, and checks figures, quotes, legal references and code. Whoever uses it is accountable for it.
- Decisions about people. AI does not decide alone on hiring, evaluation, credit or other matters that affect a person. A person decides and can explain the basis.
- Intellectual property. No third-party material we have no right to use is entered. Before publishing generated content, check that it does not reproduce others' works or trademarks.
- Connections and permissions. No assistant is connected to email, files or internal systems without approval from [IT owner] and with minimum permissions. Sending, deleting or paying requires human confirmation.
- Incidents. If you entered something you shouldn't have or see an odd result, tell [owner] the same day. Reporting is not punished.
- Owners. Policy owner: [management]. Tools register and permissions: [IT]. Personal data and contracts: [legal or data protection lead]. Reviewed every [six] months or when a tool changes.
The register can be a spreadsheet: tool and plan purchased, permitted uses, permitted data, owner, approval date and next review date. NIST AI 600-1 suggests also recording human oversight roles and considerations for intellectual property and personal or confidential data.
Checklist
# | Action | Owner |
|---|---|---|
1 | Inventory of AI tools in use, including personal accounts and AI features in existing software | IT |
2 | Provider questions answered in writing for each tool | IT and legal |
3 | Terms and privacy policy of the purchased plan read and filed | Legal |
4 | One-page policy approved and communicated | Management |
5 | Approved tools register published, with an owner | IT |
6 | Company accounts with MFA on every approved tool | IT |
7 | Short training: what data is not entered and how to verify outputs | Management and IT |
8 | Uses involving personal data reviewed against Law 21.719 | Legal |
9 | Assistants connected to email or files with minimum permissions and human approval | IT |
10 | AI incidents included in the response plan | IT and management |
11 | Policy review scheduled | Management |
The joint guidance asks that the incident response plan cover issues arising from, or affecting, AI systems. If you don't have one, you can start from the ransomware response plan.
Sources
- NCSC · ChatGPT and large language models: what's the risk? (March 14, 2023).
- ACSC, CISA, FBI, NSA, NCSC-UK and others · Engaging with Artificial Intelligence (January 2024), via New Zealand's NCSC.
- NIST · AI Risk Management Framework.
- NIST · AI 600-1, Generative Artificial Intelligence Profile (July 2024).
- OWASP · Top 10 for LLM Applications 2025 and 2026 edition.
- OWASP · LLM01:2025 Prompt Injection.
- OWASP · LLM02:2025 Sensitive Information Disclosure.
- OWASP · LLM09:2025 Misinformation.
- ISO · ISO/IEC 42001:2023.
- Chile's Ministry of Science · National Artificial Intelligence Policy.
- Chilean Senate · Legislative record of bulletin 15869-19.
- BCN · Law 21.719.
- Chilean Senate · Legislative record of bulletin 18.623-07.
- Chilean Senate · Letter to the Chamber with the approved bill, bulletin 18.623-07.
At Astrum, cybersecurity is our core line of work. If you want to review how your company's data is protected when using these tools, let's talk.