Enterprise AI Security: How to Protect Confidential Data While Embracing AI

As enterprises actively adopt AI technologies to sharpen their competitive edge, the data security risks that AI introduces cannot be overlooked. From employees feeding confidential data into public AI services to large language models potentially leaking sensitive information from training datasets, AI security has become an issue every organization must take seriously. This article provides a comprehensive examination of the cybersecurity challenges enterprises face in the AI era, risk assessment frameworks, protective strategies, and how to build a secure and trustworthy enterprise AI environment.

Key Security Risks in Enterprise AI Applications

企業在使用 AI 服務時面臨的資安風險可以分為幾個層面。首先是「資料外洩風險」：當企業使用第三方雲端 AI 服務（如 ChatGPT、各類雲端 API）時，輸入的資料會被傳送到外部伺服器處理。如果員工在不知不覺中將客戶個資、商業機密、財務資料、程式碼等敏感資訊輸入這些服務，就可能造成資料外洩。各家 AI 服務商對於「使用者輸入是否會用於模型訓練」的政策並不一致，且會隨方案等級（免費／個人付費／企業版）、部署區域與契約條款而不同，也可能隨版本更新而改變。因此不應以「業界普遍如何」來推定，正確做法是針對貴公司實際使用的每一項服務與方案，查閱其官方資料使用與訓練政策的當期版本，並在企業合約中明確約定不得用於訓練、資料保留期限與刪除義務，再輔以管理控制（如封鎖未核准的 AI 網站、在閘道端做敏感資料偵測）而非僅依賴廠商聲明。

The second category is "model security risk": large language models themselves can become targets of attack. Prompt Injection refers to attackers using carefully crafted inputs to trick an AI model into bypassing its safety constraints, performing unintended actions, or leaking sensitive information contained in system prompts. Model Extraction involves issuing a large volume of queries to replicate a model's behavior. Adversarial Attacks exploit subtle input modifications to deceive an AI model into making incorrect judgments.

The third category is "supply chain risk": the AI models, frameworks, and libraries that enterprises rely on may contain known or unknown security vulnerabilities. Open-source models, while generally more transparent, may also be compromised with backdoors. An attack on any link in the AI supply chain can have downstream effects on every enterprise that depends on those services.

第四是「合規風險」：隨著各國 AI 相關規範陸續發展（如歐盟《人工智慧法》、台灣的《個人資料保護法》），企業在使用 AI 時需檢視是否符合適用的法律要求。以 AI 處理個人資料若未落實告知、目的特定、必要範圍與安全維護等要求，可能涉及行政責任與民事賠償；具體適用條件與可能後果，需依個案資料類型、企業角色與現行條文由法務判斷。此外，AI 系統的決策過程缺乏透明度（黑盒問題），在需要說明理由的場景（如金融信貸審核、人資篩選）中容易引發爭議，這類用途在部分法域被歸類為高風險而有額外義務。實際適用範圍與作業要求，仍應以主管機關最新公告及貴公司法務認定為準。

Building an Enterprise AI Security Framework

Effective enterprise AI security requires action across three dimensions simultaneously: organizational, technical, and process. At the organizational level, enterprises should establish clear AI usage policies that define what types of data employees may and may not enter into AI tools. Regular security awareness training ensures that employees understand AI-related security risks and proper usage practices. Establishing a cross-functional AI governance committee responsible for setting and overseeing AI security standards is also essential.

At the technical level, data classification and access control are the most fundamental protective measures. Enterprise data should be tiered by sensitivity, with corresponding AI usage restrictions applied to each tier. For example, the most highly confidential data should only be processed within an on-premise AI environment, while general-level data may be handled by cloud services that have passed a security evaluation. Implementing fine-grained access controls ensures that employees can only access the AI capabilities and data required for their specific roles.

Data masking and anonymization techniques can automatically replace sensitive information — such as names, national ID numbers, and credit card numbers — with anonymized substitutes before the data enters an AI system, thereby protecting privacy without compromising the effectiveness of AI analysis. Encryption ensures the security of data both in transit and at rest.

For AI systems that connect to enterprise knowledge bases using technologies such as RAG, strict retrieval permission controls must be enforced — ensuring that the AI system can only access documents a given user is authorized to view when generating responses, and preventing the AI system from being used to circumvent existing document access management.

On-Premise Deployment: Best Practices for Enterprise AI Security

對於資安要求嚴格的企業，地端 AI 部署是降低資料外送風險的主要選項之一。在地端部署模式下，模型推論與檢索都在企業自有環境中執行，因此可以有效阻斷「提示內容與檢索到的內文被送往第三方推論服務」這條最主要的外送路徑。

不過地端不等於沒有外流面。盤點威脅模型時，至少還有幾條路徑需要單獨處理：模型權重與相依套件的下載來源（供應鏈風險，需驗證雜湊值與來源可信度）；系統與模型的更新機制是否需要對外連線；可觀測性與錯誤回報工具是否把提示內容送往外部 SaaS；備份與異地備援的存放位置與加密狀態；Agent 或外掛可呼叫的外部 API 會帶出哪些欄位；以及內部具權限人員的濫用與誤用。務實做法是為 AI 系統畫出完整資料流圖，逐條標記企業邊界的位置與對應控制，再據此判斷殘餘風險是否可接受——地端改變的是風險組成，而不是把風險歸零。

The security configuration of an on-premise AI environment should include: network isolation — deploying the AI system within an internal network segment isolated from external networks to prevent unauthorized external access; authentication and authorization — implementing multi-factor authentication and role-based access control (RBAC) to ensure only authorized personnel can use the AI system; and audit logging — recording all AI system usage, including query content, documents accessed, and responses generated, to support after-the-fact investigation and compliance auditing.

Model security is another critical focus area for on-premise deployments. Enterprises should regularly update AI models and related software to patch known vulnerabilities; apply content filtering and security checks to both model inputs and outputs to prevent prompt injection attacks and sensitive information leakage; and implement model version management to enable rapid rollback to a secure version whenever an issue is identified.

AI Security Monitoring and Continuous Improvement

AI security is not a one-time effort — it is a dynamic, ongoing process of continuous monitoring and improvement. Enterprises should establish security monitoring mechanisms for their AI systems to detect anomalous usage patterns in real time (such as bulk data extraction or unusual query patterns) and configure automated alerting rules accordingly.

Regular security assessments and penetration testing can proactively identify vulnerabilities in AI systems. Red team exercises — in which simulated attackers attempt various attacks against the AI system — are a particularly effective security assessment method. For systems that use large language models, it is also important to periodically test whether the model can be manipulated into producing unsafe outputs.

Establishing an AI security incident response plan is equally critical. When a data breach or AI system attack occurs, enterprises need well-defined handling procedures — covering incident detection, impact assessment, containment measures, root-cause analysis, and follow-up remediation. Adhering to industry-standard security frameworks such as ISO 27001 and the NIST AI RMF can help enterprises build a systematic AI security management program.

Regulatory Compliance and AI Governance

各國的 AI 監管規範正在快速發展。歐盟《人工智慧法》（EU AI Act）普遍被視為第一部跨產業、全面性的 AI 專法，採風險分級架構，對被歸類為高風險的用途（如用於信用評估、人員招募、執法等場景）設有較嚴格的安全、資料治理與透明度義務；其各項義務有分階段生效的時程安排，且實作細則與標準仍在陸續發布，適用與否需依貴公司是否落入其管轄範圍逐案判斷。

在台灣，AI 治理的主管機關是數位發展部。政府部門端已有可參照的規範文件：行政院於 2023 年通過並函頒《行政院及所屬機關（構）使用生成式AI參考指引》，就公務使用生成式 AI 的資料處理與人工審核等事項提供原則性指引；資通安全管理法體系下的資通安全責任等級分為 A、B、C、D、E 五級，不同等級對應不同的資安應辦事項，會影響公部門與特定關鍵基礎設施提供者的 AI 系統建置要求。至於《個人資料保護法》，其修正與相關子法的研擬進度、以及是否針對 AI 處理另設要求，請以全國法規資料庫與主管機關的最新公告為準，不宜以預期中的修法方向作為規劃依據。實際適用範圍與作業要求，仍應以主管機關最新公告及貴公司法務認定為準。

延伸查詢：全國法規資料庫; 數位發展部

When adopting AI, enterprises should assess applicable regulatory requirements at the outset to ensure that their AI systems are designed and used in compliance with the law. This includes establishing a lawful basis for data processing, providing notice and obtaining consent for the use of personal data, ensuring the transparency and explainability of AI decisions, and safeguarding data subject rights. Building a robust AI governance framework not only reduces compliance risk but also strengthens the confidence of customers and partners in the enterprise's AI initiatives.