At Tier Zero Security, our consultants are actively engaged in Artificial Intelligence (AI) related researches and are involved in the AI community in New Zealand. Our AI Red Teaming services evaluate your AI-integrated solutions holistically, examining not only AI-specific vulnerabilities such as prompt injection, insecure outputs, hallucinations, and sensitive information leakage, but also how these vulnerabilities may affect or be leveraged to exploit other system components like APIs, databases, and access controls.
We assess the full attack surface of your AI solution from an offensive security perspective, analysing how AI interacts with surrounding infrastructure such as APIs and databases. This includes evaluating the effectiveness of custom guardrails to ensure they align with intended use and are resilient to bypass attempts. Our approach evaluates the business impact, providing context for vulnerabilities in your solution. Engagements can be structured as one-off assessments or ongoing support embedded into your AI development lifecycle, enabling secure and responsible AI deployment from design to production.
Examples of scenarios where our AI Red Teaming can assist include:
- M365 Copilot integration review: Ensuring appropriate access controls are in place within your M365 environment to prevent unauthorised access and data leakage.
- Chatbot integration: Verifying that your chatbot or conversational AI cannot be misused to cause harm to your business, such as through data leakage or model hallucinations.
- Agentic AI integration: Assessing the security of tools and plugins available to the AI, ensuring they cannot be exploited or abused.
- AI development environment security review: Evaluating the security of your AI development environment. A compromised model during training (e.g. through data poisoning) may retain malicious behaviours that are difficult to detect or reverse.
- LLM integration with MCP and A2A: Conducting implementation and code security reviews to ensure MCP and A2A components are securely configured and free from exploitable weaknesses.
For AI specific vulnerabilities, our consultants will test for:
- Prompt Injection: a vulnerability that occurs when prompts alter the LLM's behaviour, causing them to violate guidelines, generate harmful content, enable unauthorized access, or influence critical decisions.
- Sensitive Information Disclosure: determine if it is possible to trick the LLM into returning sensitive information, also in cases where vectors and embeddings for Retrieval Augmented Generation (RAG) are utilised by the solution.
- Improper Output Handling: verity if the solution suffers from insufficient validation, sanitisation, and handling of the outputs generated before they are passed downstream to other components and systems.
- Excessive Agency: determine if the model has been provided with excessive permissions for plugins (tools) that might be used by the solution, which could lead to unintended actions or access to sensitive data.
- System Prompt Leakage: verify it is not possible to retrieve the original system prompt, which could include sensitive information or could allow to construct specific prompts that might result in unintended actions.
At Tier Zero Security, we use Open Source Security Testing Methodology Manual (OSSTMM) and Open Web Application Security Project (OWASP) Testing Guide methodologies.
For a full list of Tier Zero Security's penetration testing services, please visit our Tier Zero Security penetration testing services page.
Discover all of our services at this link.