Malware Detection Rates Revealed for 28 AV Programs

The Drive for Awareness

In November of 2016, PC Pitstop, makers of PC Matic, commissioned a test with AV Comparatives focusing on ransomware and virus detection.  This test was commissioned in an effort to stress the importance of detection rates.  PC Matic chose to commission the test because several anti-virus (AV) companies choose not to participate in public testing.  However, these companies continue to be successful.  Therefore, suggesting buyers are not purchasing their security solutions based on malware detection rates.  This is concerning.  The primary purpose for a security program is to prevent a malware attack.  Yet, buyers do not know how well certain products can do this, because several companies choose to forego third-party detection testing.

It is time the security industry begins to drive an emphasis on detection rates.  By doing so, every security solution provider will be driven to do better.  Thus, the consumers will be better protected.  If products are not being tested for accuracy, how can buyers possibly know if it’s capable of protecting their data?  They cannot.

AV Comparatives Conducts Involuntary Malware Detection Test

In the test, PC Matic commissioned with AV Comparatives in November of 2016, 18 security solutions were tested.  Those results can be found here.   The results of the initial test brought in hundreds of comments from not only PC Matic users, but users of other security solutions as well.

Several users of other AV solutions requested PC Matic commission another test with AV Comparatives.  This time, to include additional security programs.  PC Matic took this feedback and did just that.

Another involuntary test was commissioned with AV Comparatives, this time including 28 AV solutions.  PC Matic also requested AV Comparatives test not only ransomware and virus samples, but polymorphic ransomware samples and false positives as well.  For those who may be unfamiliar, polymorphic ransomware is a ransomware variant that has the ability to change, or morph, by altering the code within the sample.  Consequently, this form of ransomware has the ability to alter itself every minute, potentially even every second.  False positives are unknown files.  Therefore, they have not been tested and deemed safe or malicious.

The Results

The test included 120 polymorphic ransomware samples, 1,000 recent ransomware samples, and 4,000 other malware variants.  As stated above, this test was involuntary, meaning none of the security solutions knew they were going to be tested, with the exception of PC Matic.  None of the AV programs, including PC Matic, knew when they would be tested.  During the testing process, the tests were run “On Execute” with the internet connection enabled.  Each sample was run through a script and executed, and then analyzed to see if the AV product in question was able to properly detect the sample.

Multiple security solutions tested well, including Microsoft.  Historically, Microsoft’s free AV program, Windows Defender was far from effective; testing dead last against several leading AV solutions in 2013.  Today, its detection is better than some purchased security solutions.

What PC users must understand is, if they are going to pay for a security solution, they must pay for the best.

PC Matic earned perfect scores in polymorphic ransomware, recent ransomware, and other malware detection.  Its false positive rate was 96.67%.  This is due to PC Matic’s whitelisting methodology.  Therefore, anyone who uses the traditional blacklisting methodology, which allows any unknown files to execute, will score 100%.

The chart below allows readers to hover over each category to see how their security solution scored in all four areas.

 22,755 total views,  3 views today

(Visited 1 times, 1 visits today)

21 thoughts on “Malware Detection Rates Revealed for 28 AV Programs”

  1. Is it true virus protection is not needed for Windows 10? My boss does not think my new computer at work needs protection.

  2. Is it true that Windows 10 doesn’t need anti-virus protection because it is already there inside Windows 10?

    1. Kayla Thrailkill

      Windows Defender is included within Windows 10, as well as other Windows platforms. They have been increasing their protection as the years have gone on. However, there are still better solutions available to increase your level of protection. So, to answer your question — technically you are protected with Windows Defender, but you could purchase better protection.

    2. @Kasanda Howard:

      Don’t be foolish, Windows Defender only covers a portion of the playing field. Get PC Matic to ensure total protection. My lord, in this day and age you cannot trust cyber space, there are so many dark figures lurking in the shadows. Your asking for trouble that could be potentially harmful to you financially, emotionally, etc.. etc..
      I don’t work for PC Matic or collect any money from them, I am no different than you. I have had many security programs, and have been down that road. Get the best on the market. Get PC Matic Super Shield. It will be the best decision you ever made. But make sure you run it two times in a row, each time you start it up, at least once a week.
      After you run it you will be shocked at the report on your computer. Its sicker than you think it is.

  3. In your linked article, you specifically criticize Malwarebytes and Webroot for non-participation in public testing. But as Neil J. Rubenking of PC Magazine has pointed out several times, their methods of protection aren’t entirely compatible with current testing approaches.

    In reviewing Malwarebytes 3.0 Premium just a few months ago, Rubenking wrote its “Advanced protection systems are extremely difficult to test”. Here’s a few of his comments:

    “There’s one small problem with these powerful, focused protection layers; they’re tough to test. Exploit attacks only work on a specific program version that contains the matching vulnerability. Malwarebytes kicks in only when such a matchup occurs, because without a match no actual damage is possible.

    “Many of the independent antivirus testing labs strive to create tests that emulate real-world situations, but this emulation isn’t perfect. And many of them still include simple file-recognition in their testing. My contact at Malwarebytes explained that the designers could bulk up the product with features aimed solely at passing tests, or keep it nimble and focus on actually protecting users. They chose the latter.

    “One problem with testing behavior-based detection is that the samples simply may not exhibit malicious behaviors. Some may refrain if they detect antivirus software, others may wait a while after installation.”

    Describing Webroot SecureAnywhere AntiVirus’s “journaling and rollback technique” last year, Rubenking pointed out a major problem with testing it:

    “This delayed-action response is a clever way to deal with never-before-seen malware, but it just doesn’t jibe with current antivirus tests. Researchers expect the antivirus products to take action right away, blocking installation of new malware samples and cleaning up any existing infestations. They may allow a few minutes to be sure the antivirus has finished. But Webroot’s analysis can take quite a bit longer. So what if it completely reverses the malware activity after an hour. At that point, it has already been marked as a failure. Due to this incompatibility, I have no results to report from my usual group of antivirus testing labs.”

    In his reviews, Rubenking hasn’t criticized Malwarebytes or Webroot for skipping out, and neither do I.

    1. @Brian: In the past, PC Magazine has been criticized for being paid for its reviews of software! I would be wary of what they say.

      1. @Wayne: Don’t forget that PC Matic’s own CEO has happily trumpeted select statements by Rubenking when they’ve worked in PC Matic’s favor. If you criticize me, then you have to criticize Rob, too, for quoting the exact same individual. Either you cite both the good and the bad from a given source, or nothing at all.

  4. PC Matic has saved me so many times, my life is hard enough with constantly being bombarded with viruses and malware. The little Asian guy on TV who doe’s the commercial is not lying. I’ve’ had MacAfee, Norton, AVG and on and on. But PC Matic doe’s exactly as it says.

    On top of that its 100% American Made. If you don’t have PC Matic, Don’t Wait another second! Get it now so you can sleep at night. There customer service is Awesome.

    I am a Believer, they have proved themselves time and time again. It will be the best $50 you have ever spent.

  5. “Michael says:
    April 15, 2017 at 3:59 am
    PC Matic get such a bad rating in the False Positive because it is bad at detecting valid software- it shouldn’t be up to the user to know what every single piece of installed software and other applications do.”

    I think you have it backwards. A lower false positive count (%) is better than 100% aka less false positives.
    But, the Whitelist approach is only really good for known problem software and or malware.
    Personally I prefer my AV suite(Vipre) asking me about files that are unknown to the blacklist and false positives can also be a good thing at times.

  6. PC Matic get such a bad rating in the False Positive because it is bad at detecting valid software- it shouldn’t be up to the user to know what every single piece of installed software and other applications do.

Leave a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.