Application security scanning with LLMs 101: how we found 13 vulnerabilities in microG

1. the project’s main components (e.g., software and infrastructure)

2. the assets to protect (e.g., users’ personal data)

3. the trust boundaries (where external inputs can come from, and which should be considered trusted or untrusted)

4. potential threats

5. how threats will be dealt with

As you might expect, producing a thorough threat model requires collaboration between technical specialists (developers, system administrators, and security experts) and product experts. It also needs to be reviewed and updated over time, as the system evolves and the team gains a better understanding of the risks and security constraints.

The quality of the document directly affects the results of security scans performed by AI agents: wherever the model is lacking, the agent must try to infer information it needs but has not been given.

There was no ready-to-use threat model for microG. But even in cases like this, an agent can perform well with a sufficiently detailed prompt that covers as many of the points above as possible. The one we used for the experiment began like this:

The repo is composed by many submodules that make up the microG Services app, which is a FOSS framework to allow applications designed for Google Play Services to run on systems where Play Services is not available. The vending-app module contains the code for the microG Companion app, which provides Play Store API functionality.

I want you to perform a codebase scan for security vulnerabilities that can lead to arbitrary code execution, data leaks/exfiltration (including PII and Google account details), authorization bypass, injection, tapjacking or other types of severe vulnerabilities. As the attack vectors, consider malicious apps installed on the device targeting microG Services and microG Companion (e.g. via malicious intents) or malicious payloads coming from external services (like push notifications or other data received from the Internet). 
Always assume that microG is installed as a system app using the Play Services package name [...] 

Now that we have provided the context, we need to specify how we want the agent to work and how it should verify that it has done the job correctly. This is the prompt we used:

[...] For each vulnerability you think you have a found, you'll write a report and a POC that can be (1) a minimal app mimicking the behavior of a malicious app or (2) an instrumented test runnable on an Android device or emulator. 
The minimal POC app must be compilable and runnable standalone - using official Google Play client libraries whenever possible - and have an activity with a button to trigger the vulnerability and display the result (e.g. the exfiltrated data) to the screen.

In this case, we instructed the agent to immediately create a proof of concept (PoC) for each found vulnerability, so that we could easily validate it on a real device and try to reduce false positives and hallucinations.

Limitations

This strategy, however, proved less effective than we had hoped. This is mainly because, in our setup, the agent could not test a PoC on its own and adjust its approach accordingly. The PoCs required user interaction, displayed their results on screen, and had to be installed on a physical device with microG already configured. As a result, the agent sometimes produced non-functioning PoCs or overstated the impact of a vulnerability. Often, simply reporting back the error encountered when running a PoC was enough for the agent to fix it — a step that could have been avoided if it had been able to run the PoC autonomously and see the output.

Another risk that makes creating PoCs a double-edged sword is “self-censorship”: if the agent thinks a vulnerability is unlikely to be provable with a PoC, it might leave it out of the results. In one case involving microG, the LLM decided to exclude a potential finding for this exact reason. But when prompted to investigate further, it reconsidered its decision and was able to produce a PoC for the vulnerability, which turned out to be a high-severity one (GHSA-qqqw-h762-w2hw).

Precisely to avoid these situations, Anthropic suggests — in an excellent article on the topic that we discovered only after the experiment — using a separate agent to verify findings from the scanning phase. The goal of the latter is therefore to find as many potential vulnerabilities as possible, which will then be confirmed or ruled out in the verification phase.
Another point from that article which we consider fundamental is:

"failure to produce a working PoC is not proof of a false positive" 

Where LLMs falls short, human expertise comes into play.

Results and observations

With the help of LLMs, we have managed to find the following 13 vulnerabilities in microG, which we disclosed to the project maintainer and have since been fixed in versions 0.3.17 and 0.3.16.1.

It is worth noting that many of these vulnerabilities stem from small oversights or missing checks on parameters received from other apps (the so-called “untrusted inputs”). In the context of a system app with access to user data, vulnerabilities like these can have significant consequences.

It must be said that, in addition to the vulnerabilities listed above, LLMs reported a large number of false positives: around 40% for both models we used (Opus 4.8 and Grok 4.6). 
The lack of a detailed threat model was certainly the main culprit, because many false positives were intended behaviors that could have looked “insecure” in a different context. In other cases, they were bugs in sensitive areas of the code (such as authentication) that nevertheless had no impact on the application’s security. 
One clear example of a false positive in microG involved an API whose alleged vulnerability was that it allowed one app to revoke authentication tokens belonging to other apps. But how could an app possess another app’s token in the first place? If that were to happen, it would be a serious security problem in the app that owns the token — not something microG can be expected to handle.

Hallucinations, though rarer, did occur: in one case, both the report and the PoC for an alleged vulnerability were entirely based on an incorrect inference about the behavior of an Android API. 
Running multiple scans with LLMs from different model families can help mitigate this problem. Because their output is probabilistic and heavily influenced by their training data, two models working on the same part of a codebase may find different vulnerabilities. 
A prime example from our experiment involved vulnerabilities GHSA-29m7-f2vx-8cqq and GHSA-frxh-v6rf-xfg4, both of which originated in the same function of microG’s location service. Opus 4.8 found only the first, caused by a missing authorization check (line 36), and assumed that the other piece of code was secure because it had the check, even though it was implemented in a vulnerable way (line 30). Grok 4.6, on the other hand, detected the vulnerable check but was unable to determine that the missing check could be exploited to obtain the user’s location without the required permissions.

Another point to watch out for is the size of the codebase being scanned. 
When faced with a large codebase with many modules like microG’s, the agent is forced to decide for itself which areas to focus on, sometimes using arbitrary criteria such as the names of modules or classes that look more “sensitive” (e.g., those related to authorization, authentication, or cryptography). 
If, instead, we make it focus on one or a few small modules at a time, the model is more likely to find complex vulnerabilities, as it has more room in its context window to “reason” across multiple files in the same module.

An important clarification about LLM safety restrictions: although vulnerability PoCs are inherently dual-use (malicious actors can use them to create malware), their implementation is allowed for defensive purposes such as security analysis. After all, this is a practice recommended by Anthropic itself. 
On the few occasions when safety safeguards were triggered, it was because the language used when requesting corrections to non-functioning PoCs was overly explicit (e.g., “data is not getting leaked”). Rephrasing the request in a separate chat was enough to avoid triggering the safeguards again. 
At the time of writing, the only proprietary models that cannot be freely used for these activities are those from OpenAI, which require the user to be granted access to the Daybreak program.

Conclusions

Our experiment began without a threat model, using a relatively simple prompt, yet it uncovered 13 vulnerabilities — three of which were high severity — in a mature open-source project which several “de-Googled” Android systems are based upon. 
This is not a problem unique to microG, but rather a common issue with almost all software. Projects grow, become more complex, and are passed on to other people, while security concerns end up being neglected in favor of other priorities. What is different today is that AI models make it possible to conduct security analyses at a cost and speed that would have been unimaginable just a few years ago, enabling teams to take action before problems turn into incidents.

The limitations we encountered also point to where we can do much better. For example, leveraging a threat model to provide the context needed to reduce false positives, a harness that lets agents run and verify PoCs on their own, and a structured finding verification phase separate from the scan itself. 
And, at the end of the pipeline, experts who can assess the quality of the results, apply the necessary fixes to the software, and refine the process accordingly.

At Claranet, we are exploring this approach with our customers, combining the capabilities of AI agents with the expertise of our development and security teams. If an experiment using basic tools can produce tangible results, a process tailored to your product can provide you with a much more comprehensive and reliable picture of its security posture. Want to find out how to apply it to your software? Get in touch: we can start with an initial conversation.