Microsoft: Our AI can spot security flaws from just the titles of developers’ bug reports

Microsoft has revealed how it’s applying machine learning to the challenge of correctly identifying which bug reports are actually security-related.

Its goal is to correctly identify security bugs at scale using a machine-learning model to analyze just the label of bug reports.

According to Microsoft, its 47,000 developers generate about 30,000 bugs a month, but only some of the flaws have security implications that need to be addressed during the development cycle.

Microsoft says its machine-learning model correctly distinguishes between security and non-security bugs 99% of the time. It can also accurately identify critical security bugs 97% of the time.

The model allows Microsoft to label and prioritize bugs without necessarily throwing more human resources at the challenge. Fortunately for Microsoft, it has a trove of 13 million work items and bugs it’s collected since 2001 to train its machine-learning model on.

Microsoft used a supervised learning approach to teach a machine-learning model how to classify data from pre-labeled data and then used that model to label data that wasn’t already classified.

Importantly, the classifier is able to classify bug reports just from the title of the bug report, allowing it to get around the problem of handling sensitive information within bug reports such as passwords or personal information.

“We train classifiers for the identification of security bug reports (SBRs) based solely on the title of the reports,” explain Mayana Pereira, a Microsoft data scientist, and Scott Christiansen from Microsoft’s Customer Security and Trust division in a new paper titled Identifying Security Bug Reports Based Solely on Report Titles and Noisy Data.

“To the best of our knowledge this is the first work to do so. Previous works either used the complete bug report or enhanced the bug report with additional complementary features,” they write.

“Classifying bugs based solely on the tile is particularly relevant when the complete bug reports cannot be made available due to privacy concerns. For example, it is notorious the case of bug reports that contain passwords and other sensitive data.”

Microsoft still relies on security experts who are involved in training, retraining, and evaluating the model, as well as approving training data that its data scientists fed into the machine-learning model.

“By applying machine learning to our data, we accurately classify which work items are security bugs 99% of the time. The model is also 97% accurate at labeling critical and non-critical security bugs. This level of accuracy gives us confidence that we are catching more security vulnerabilities before they are exploited,” Pereira and Christiansen said in a blogpost.

Microsoft plans to share its methodology on GitHub in the coming months.

Hot this week

The Myth of Perfect Security

Perfect security is a myth, and focusing on resilience rather than prevention can better protect your organization from inevitable breaches.

Why Traditional Passwords Are Failing Us

Password fatigue from complex rules often causes more security breaches than weak passwords, requiring a shift toward user-friendly tools and behaviors.

Why Your Employees Are Your Best Security Defense

Empowering employees with security awareness training often provides better protection than stacking more technology, turning human factors from a weakness into your strongest defense.

Why Most Security Awareness Training Fails and What to Do About It

Security awareness training often fails because it focuses on knowledge rather than behavior, but shifting to a behavior-based approach can lead to better outcomes and fewer incidents.

The Myth of Multifactor Authentication Security

Multifactor authentication enhances security but is not foolproof, as it can be bypassed through social engineering and technical exploits. Understanding its limitations and adopting stronger methods is essential for effective protection.

Topics

The Myth of Perfect Security

Perfect security is a myth, and focusing on resilience rather than prevention can better protect your organization from inevitable breaches.

Why Traditional Passwords Are Failing Us

Password fatigue from complex rules often causes more security breaches than weak passwords, requiring a shift toward user-friendly tools and behaviors.

Why Your Employees Are Your Best Security Defense

Empowering employees with security awareness training often provides better protection than stacking more technology, turning human factors from a weakness into your strongest defense.

Why Most Security Awareness Training Fails and What to Do About It

Security awareness training often fails because it focuses on knowledge rather than behavior, but shifting to a behavior-based approach can lead to better outcomes and fewer incidents.

The Myth of Multifactor Authentication Security

Multifactor authentication enhances security but is not foolproof, as it can be bypassed through social engineering and technical exploits. Understanding its limitations and adopting stronger methods is essential for effective protection.

Why MFA Is Not Enough Anymore

Multi-factor authentication is no longer a silver bullet for security as attackers develop new bypass methods, requiring a layered defense approach with phishing-resistant tools and continuous monitoring.

Why Phishing Still Works and What to Do About It

Phishing remains a top threat because it exploits human psychology, not just technical gaps. Shifting focus to employee awareness and habits can build stronger defenses than relying solely on technology.

Rethinking Password Security

Complex password rules often increase risk by encouraging poor habits. Learn how password managers and multi-factor authentication offer more practical protection for organizations of all sizes.
spot_img

Related Articles

Popular Categories