IzzyOnDroid

• Intro
• App-Lists
• Articles
• Books
• Downloads
• App-Repo
• Imprint

How reliable are Malware Scans for Android Apps?

Many people and services trust VirusTotal to scan their APKs. This service integrates scanners from over 60 security vendors–which, among other things, enables the detection of so-called “false positives”: If one scanner flags an issue but all the others do not, this is usually an indication of a false alarm. VirusTotal seems to agree, going by the small green checkmark above the result in the screenshot.

Of course, even when these engines display green checkmarks, it simply means that nothing was detected. They are no guarantee that a file is harmless.

Usually, this works pretty well. But what if several scanners suddenly flag an APK as malicious (malicious/PUA¹/Trojan)? Is there any truth to that?

At IzzyOnDroid, we’ve noticed that this happens particularly often with SMS apps–especially when they also have the INTERNET permission (e.g., for SMS forwarding). These are then quickly flagged as “banking Trojans,” as they could intercept banking-related SMS messages². Since the apps are open source³, it is at least possible to determine whether the source code contains any indications of such behavior. This evidence would be even stronger with reproducible builds, since the specific APK could then be unambiguously linked to the verified source code⁴.

An app can only be clean when listed in the Google Play Store

In one case⁵, we therefore decided to get to the bottom of the matter. The app’s developer contacted all the affected vendors, provided them with detailed information, and asked them to correct their scanners accordingly. While some confirmed the corrections shortly thereafter, at least one vendor stubbornly refused–citing an interesting reason:

as per our internal protocol, we are unable to proceed with the whitelisting process unless the application is available on the Google Play Store.

And this is despite the fact that the evidence tells a different story: While I am not aware of any reliably documented case involving F-Droid or IzzyOnDroid that would be comparable to the recurring malware discoveries in the Play Store⁶, one repeatedly comes across articles about the Play Store reporting hundreds of apps had to be removed because they had infected millions of devices⁷. This makes such a statement particularly surprising.

Is there any truth to that? Let’s do a POC

My curiosity kicked in, now I wanted to know: What makes an APK file like this malware? So I decided to test this out with the app in question:

• scanning the original APK file yielded 7 matches
• renamed the file to test17.zip and removed AndroidManifest.xml, META-INF/, and kotlin-tooling-metadata.json from the archive⁸: 0 hits
• same as before, but this time leaving the AndroidManifest.xml file in the ZIP: the original 7 matches are back
• once more, but this time I only removed the AndroidManifest.xml file from the ZIP file: 0 hits

So is it just the AndroidManifest.xml that’s “evil”? Let’s put it to the test: Create a ZIP file that contains only the AndroidManifest.xml: 2 hits–including the vendor who insisted on a Google Play listing. Incidentally, this file contains only the app’s metadata (such as the app’s name, version, required permissions, and similar information); on its own, it cannot cause any harm.

It seems, then, that some scanners react strongly (or almost exclusively) to metadata and permissions, and less (or not at all) to actual malicious code.

After we drew some attention to the issue with a Fediverse post⁹ that received numerous boosts and likes, everything suddenly turned green: not a single hit anymore. The fact that the app is still not listed in the Google Play Store (and probably never will be) suddenly didn’t matter anymore.

So it’s all rubbish – or what?

One shouldn’t draw too many conclusions from a single example. We tried to replicate the POC¹⁰ above for another SMS app: The differences were less dramatic in that case. Nevertheless, there are several lessons to be learned from this case:

• you can’t blindly trust the scan results. They are indicators that you should take a closer look–but they are not definitive facts.
• some providers seem to determine their results on a rather questionable basis (in this case: K7GW); as long as only their scanners are flagging a threat, an actual infection is rather unlikely
• the case also illustrates how problematic it becomes when a Play Store listing is effectively treated as a trust anchor. This quickly puts independent app sources like F-Droid or IzzyOnDroid at a disadvantage, even though a Play Store listing in and of itself is no guarantee of security¹¹.

To sum it up briefly, then: Scanners provide clues, but not a definitive assessment.