AI penetration testing services for Australian organisations
For over 25 years, dotSec has provided penetration testing services to a wide range of corporate and government organisations across Australia. With the rapid adoption of artificial intelligence (AI) across these sectors, we have evolved our methodologies to assess whether these new technologies are resilient against targeted attacks.
Our testers have deep experience building, using, and attacking AI applications and large language models (LLMs). By combining this domain-specific knowledge with our established penetration testing methodology, our team can identify the security weaknesses that are unique to AI systems, from prompt injection and data poisoning through to training data extraction and insecure output handling.
AI systems introduce a fundamentally different attack surface to traditional software. Where conventional applications follow deterministic logic, AI models are probabilistic, meaning the same input can produce different outputs. This non-deterministic behaviour creates vulnerability classes that standard penetration testing approaches are not designed to find. dotSec’s AI penetration testing addresses this gap directly.
What is an AI penetration test?
A traditional penetration test targets vulnerabilities in software, networks, and infrastructure. An AI penetration test applies the same rigorous methodology but focuses specifically on the components that make AI systems unique: the model itself, its training data, its integration points, and the guardrails designed to constrain its behaviour.
Because AI is non-deterministic (meaning different outputs may be generated even when the same input is provided), the nature of the vulnerabilities differs significantly from those found in traditional software. Where traditional pen testing is like trying to crack a safe’s lock, AI pen testing is often like trying to convince the person guarding the safe to hand over the contents. It requires a mindset closer to social engineering.
Our AI assessments cover attack vectors defined by the OWASP Top 10 for LLM Applications and the MITRE ATLAS framework, including prompt injection, data model poisoning, insecure output handling, and training data extraction. These vectors can lead to outcomes such as bypassing safety guardrails, exfiltrating sensitive data processed by the model, or manipulating downstream systems that consume AI-generated output.
How does AI pen testing improve your security?
AI is still relatively new to enterprise environments. While development teams are integrating AI tools rapidly, secure integration practices are not yet widely established. This creates gaps where governance failures and technical misconfigurations can expose your organisation to risks that conventional security testing will not detect.
An AI penetration test will provide you with:
An understanding of AI-specific risks
A list of vulnerabilities without context is not actionable. dotSec’s AI pen testing goes beyond checklist scanning to expose the risks that are specific to generative AI. Because AI is non-deterministic, vulnerabilities often exist in the model’s logic and behaviour rather than in the code. We identify these issues and map them to the OWASP Top 10 for LLM Applications and the MITRE ATLAS adversarial threat framework, so you understand your actual exposure.
A prioritised, practical remediation plan
Finding a flaw is only half the work; knowing how to fix it is what matters. AI security often requires remediation strategies that blend traditional code patches with controls specific to AI, such as prompt engineering, content filtering, and output sanitisation. We provide a prioritised, developer-friendly remediation plan with CVSS v4.0 severity ratings for every finding, so your team can focus limited resources on the most critical AI risks first.
Independent verification of AI governance
Governance policies are only effective if they work in practice. Your organisation may have directives stating that its AI must not generate harmful content or must not disclose sensitive internal information, but do these constraints hold up against a motivated attacker? dotSec provides independent, evidence-based verification of your AI’s actual behaviour. We validate that safety constraints, access controls, and content filtering are effective under adversarial conditions, aligning our findings to frameworks such as the NIST AI Risk Management Framework.
Coverage of AI-specific attack vectors
Standard vulnerability scanning tools are not designed to detect AI-specific weaknesses. Our testing covers the full range of attack vectors unique to AI systems: prompt injection to override safety guardrails, data poisoning to corrupt model behaviour, insecure output handling where AI responses can trigger vulnerabilities in downstream systems (such as XSS or command injection), and training data extraction to determine whether the model can be tricked into revealing sensitive data it was trained on. These are the vectors defined by the OWASP Top 10 for LLM Applications.
AI penetration testing FAQ
How is an AI pen test different from a web application pen test?
While they share similarities, AI pen testing targets the probabilistic nature of the model itself. In addition to standard web vulnerabilities like broken access controls and injection flaws, we test for logic manipulation, exploitable bias, and “hallucination” behaviours that an attacker can leverage. The attack methodology is closer to social engineering than to traditional technical exploitation.
Can you test our internal AI tools such as Copilot or custom GPTs?
Yes. For commercial AI products, the scope focuses on your organisation’s configuration, data handling, and governance controls rather than the vendor’s core model. We test whether your deployment prevents data leakage, enforces access controls, and complies with your internal policies. For custom-built AI applications and LLM integrations, we test the full stack including the model’s behaviour, its integration points, and its output handling.
Do you use AI to test AI?
We use a hybrid approach. Automated tooling helps us scale testing of common prompt patterns and known attack vectors, but the most critical findings, such as complex logic bypasses and chained exploitation paths, come from manual testing by our experienced assessors. Automated tools alone cannot replicate the creative, adversarial thinking required to find the vulnerabilities that matter most.
What standards do you test against?
We align our testing methodology with the OWASP Top 10 for LLM Applications, the MITRE ATLAS adversarial threat matrix for AI, and the NIST AI Risk Management Framework (AI RMF 1.0). These frameworks provide the current industry baseline for identifying and categorising security risks in AI and generative AI systems.
What types of AI systems can dotSec test?
We test any system that incorporates AI or LLM components, including customer-facing chatbots, internal knowledge assistants, AI-powered document processing and summarisation tools, code generation assistants, and custom LLM integrations via APIs. The methodology is adapted to the specific architecture and risk profile of each engagement.
What next?
If your organisation is developing or deploying AI solutions, a targeted AI penetration test is the most direct way to verify that your implementation is secure before an attacker finds out it is not.
Our team can help you scope a test that addresses your specific AI risks, whether you are running a customer-facing chatbot, an internal knowledge assistant, or a custom LLM integration. The findings feed directly into practical remediation, with our GRC specialists available to align your AI security posture with frameworks including the ACSC Essential Eight, ISO 27001, and the NIST AI Risk Management Framework.
dotSec has been testing systems for Australian organisations since 1999. AI changes the attack surface, but not our commitment to finding the vulnerabilities that matter and providing practical, prioritised guidance to fix them.
Premier australian cyber security specialists
