By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
ToolAccuracy of FindingsDetects Non-Pattern-Based Issues?Coverage of SAST FindingsSpeed of ScanningUsability & Dev Experience
DryRun SecurityVery high – caught multiple critical issues missed by othersYes – context-based analysis, logic flaws & SSRFBroad coverage of standard vulns, logic flaws, and extendableNear real-time PR feedback
Snyk CodeHigh on well-known patterns (SQLi, XSS), but misses other categoriesLimited – AI-based, focuses on recognized vulnerabilitiesGood coverage of standard vulns; may miss SSRF or advanced auth logic issuesFast, often near PR speedDecent GitHub integration, but rules are a black box
GitHub Advanced Security (CodeQL)Very high precision for known queries, low false positivesPartial – strong dataflow for known issues, needs custom queriesGood for SQLi and XSS but logic flaws require advanced CodeQL experience.Moderate to slow (GitHub Action based)Requires CodeQL expertise for custom logic
SemgrepMedium, but there is a good community for adding rulesPrimarily pattern-based with limited dataflowDecent coverage with the right rules, can still miss advanced logic or SSRFFast scansHas custom rules, but dev teams must maintain them
SonarQubeLow – misses serious issues in our testingLimited – mostly pattern-based, code quality orientedBasic coverage for standard vulns, many hotspots require manual reviewModerate, usually in CIDashboard-based approach, can pass “quality gate” despite real vulns
Vulnerability ClassSnyk (partial)GitHub (CodeQL) (partial)SemgrepSonarQubeDryRun Security
SQL Injection
*
Cross-Site Scripting (XSS)
SSRF
Auth Flaw / IDOR
User Enumeration
Hardcoded Token
ToolAccuracy of FindingsDetects Non-Pattern-Based Issues?Coverage of C# VulnerabilitiesScan SpeedDeveloper Experience
DryRun Security
Very high – caught all critical flaws missed by others
Yes – context-based analysis finds logic errors, auth flaws, etc.
Broad coverage of OWASP Top 10 vulns plus business logic issuesNear real-time (PR comment within seconds)Clear single PR comment with detailed insights; no config or custom scripts needed
Snyk CodeHigh on known patterns (SQLi, XSS), but misses logic/flow bugsLimited – focuses on recognizable vulnerability patterns
Good for standard vulns; may miss SSRF or auth logic issues 
Fast (integrates into PR checks)Decent GitHub integration, but rules are a black box (no easy customization)
GitHub Advanced Security (CodeQL)Low - missed everything except SQL InjectionMostly pattern-basedLow – only discovered SQL InjectionSlowest of all but finished in 1 minuteConcise annotation with a suggested fix and optional auto-remedation
SemgrepMedium – finds common issues with community rules, some missesPrimarily pattern-based, limited data flow analysis
Decent coverage with the right rules; misses advanced logic flaws 
Very fast (runs as lightweight CI)Custom rules possible, but require maintenance and security expertise
SonarQube
Low – missed serious issues in our testing
Mostly pattern-based (code quality focus)Basic coverage for known vulns; many issues flagged as “hotspots” require manual review Moderate (runs in CI/CD pipeline)Results in dashboard; risk of false sense of security if quality gate passes despite vulnerabilities
Vulnerability ClassSnyk CodeGitHub Advanced Security (CodeQL)SemgrepSonarQubeDryRun Security
SQL Injection (SQLi)
Cross-Site Scripting (XSS)
Server-Side Request Forgery (SSRF)
Auth Logic/IDOR
User Enumeration
Hardcoded Credentials
VulnerabilityDryRun SecuritySemgrepGitHub CodeQLSonarQubeSnyk Code
1. Remote Code Execution via Unsafe Deserialization
2. Code Injection via eval() Usage
3. SQL Injection in a Raw Database Query
4. Weak Encryption (AES ECB Mode)
5. Broken Access Control / Logic Flaw in Authentication
Total Found5/53/51/51/50/5
Contextual Security Analysis
October 9, 2024

From BlackHat 2024: The AI Transformation in AppSec

As many of you know, DryRun Security was selected as one of the finalists in the Black Hat Startup Spotlight competition, and we had an incredible time showcasing how we're empowering application security teams and developers to detect risks that don't fit traditional patterns. 

Our AppSec Code Policies and code awareness view are helping teams sift through hundreds or even thousands of daily code changes to spot the most risky code merges. We're doing this by leveraging Large Language Models (LLMs) instead of relying on old-school pattern matching.

But, this isn’t a blog post about that. Instead this blog is a recap of the talk I gave at BlackHat and is focused on the industry at large and why I believe AI will transform AppSec specifically. 

The Big Idea: AI will do for Security what the Cloud did to Ops

Back in the early 2000s, the cloud came along and turned operations on its head. We went from managing a handful of servers to orchestrating thousands, thanks to virtualization and containerization. This seismic shift led to the rise of DevOps—a necessity for efficient operations in a cloud environment.

Tom Limoncelli summed up this view: "DevOps was the inevitable result of needing to do efficient operations in a distributed computing and cloud environment."

It was the macro-pressures of distributed computing where DevOps was born, it shifted the meaning of operations and scale an order of magnitude past where it had been. 

Once you agree to that premise, you wonder, why didn’t security make the shift along with Ops…

Why Security Was Largely Left out of DevOps

There was a desire by some in the industry (me included) to bring security to DevOps under the banners of DevSecOps or shift left. But, it was naive. Security teams have a very different problem than ops teams did at the rise of cloud. They have a data problem from too many documents, reports, audits, tool telemetry and more, along with a rapidly shifting development environment where code velocity is going through the roof.

So, then we tried to fix things with "shift left" and DevSecOps, but these often just added more gates and slowed everything down. Developers got bogged down with bloated backlogs and confusing security scans. 

Steve Bellovin, when opining about the state of the industry in his book Thinking Security, noted, "We're protecting the wrong things and we're hurting productivity in the process."

Ok, so How can AI Help?

As security engineers and practitioners it's easy for us to poke at AI and LLMs and show flaws and problems. It’s in our nature to do so, I get that. However, there are some tasks that LLMs are very well suited for and they are going to prove to be game changers for our industry. 

Namely, for tasks that require summarization, transformation, and information extraction, LLMs excel at these. For tasks that require generation, LLMs are getting better month-by-month. When you apply the list of things that LLMs are good at and overlap that with the data problem (documents, reports, audits, tool telemetry), you can see how our industry is ripe for leveraging AI to scale out AppSec efforts. 

Solving the Data Problem Security Faces

This isn't just theoretical. This is the approach we take at DryRun Security, and our customers are benefiting from it today.

Here's where AI and LLMs come into play. Instead of relying on pattern matching and rule-writing (which are time-consuming and often miss the mark), AI allows us to:

  • Ask Human-Like Questions About Code Changes: Who wrote this? What's the intent? Could this introduce a vulnerability?
  • Leverage Context in our Security Analysis: Understand not just the code, but the environment, the potential impact, and more.
  • Scale Our Efforts: In a sense, AI becomes the embedded security champion on each team and is able to scale our appsec program horizontally.

At DryRun Security, we've embraced this through the process we call Contextual Security Analysis, which classifies context in our SLIDE model:

1. Surface: Identify entry points and interfaces.

2. Language: Understand the programming language and framework.

3. Intent: Grasp what the developer intended the code to do.

4. Detection: Spot anomalies or potential issues.

5. Environment: Consider where and how the code will run in the real world.

Looking Ahead: A New Era of Security

In a few years, my bet is that we'll look back and wonder how we ever managed without AI in the field of application security. The old ways of pattern matching will seem as outdated as dial-up internet. We'll realize how much context we were missing and how AI opened the door to a more efficient, effective approach.

Join Us on This Journey

I'm genuinely excited about what the future holds, and I'd love for you to experience it firsthand. If you're curious about how AI can revolutionize your application security, check out a demo to see how we're bringing AI-powered security right to your fingertips.

Final Thoughts

The cloud forced operations to evolve, and now AI is doing the same for security. It's not just about keeping up—it's about staying ahead.

Thank you for being part of this incredible journey and being a friend of the company and reader of this blog.