False positives (FPs) have been an issue of extreme importance for anti-virus
(AV) systems for decades. As more security vendors turn to machine learning,
alert deluge has hit critical mass with over 20% of all alerts resulting in FPs
and, in some organizations, the number reaches half of all alerts. This
increase has resulted in fatigue, frustration, and, worst of all, neglect from
security workers on SOC teams. A foundational cause for FPs is that vendors
must build one global system to try and satisfy all customers, but have no
method to adjust to individual local environments. This leads to outrageous,
albeit technically correct, characterization of their platforms being 99.9%
effective. Once these systems are deployed the idiosyncrasies of individual,
local environments expose blind spots that lead to FPs and uncertainty.

We propose a strategy for fixing false positives in production after a model
has already been deployed. For too long the industry has tried to combat these
problems with inefficient, and at times, dangerous allowlist techniques and
excessive model retraining which is no longer enough. We propose using a
technique called passive-aggressive learning to alter a malware detection model
to an individual’s environment, eliminating false positives without sharing any
customer sensitive information. We will show how to use passive-aggressive
learning to solve a collection of notoriously difficult false positives from a
production environment without compromising the malware model’s accuracy,
reducing the total number of FP alerts by an average of 23x.

By admin