DryRun Security vs. Semgrep, SonarQube, CodeQL and Snyk – C# Security Analysis Showdown

Editor’s Note: This follow-up builds on our previous head-to-head test, "DryRun Security vs. Traditional SAST Vendors in Ruby on Rails,” in which we compared Snyk, CodeQL, Semgrep, and SonarQube across the same six key categories within the Rails framework.

Context is King in C# Application Security

Traditional static analysis tools often struggle with C# security issues beyond simple patterns. We put DryRun Security head-to-head against Semgrep, SonarQube, CodeQL and Snyk on an intentionally vulnerable ASP.NET Core API (the open-source “Damn Vulnerable C# Application”). The test code – introduced in PR #7 of our DryRunSecuritySandbox fork – contains a mix of obvious bugs (like SQL injection) and subtle flaws (logic errors, broken authentication, user enumeration, and hardcoded secrets). Each tool was run on the pull request to see what they would catch.

‍

The results were eye-opening: DryRun Security’s contextual analysis caught every vulnerability, including complex authentication logic flaws, while the others missed most issues (in some cases even failing to flag blatant problems). Below we compare accuracy, explain the vulnerabilities (with code samples), and show where DryRun Security excels in contextual C# security analysis.

The Short Comparison Grid

Here’s a quick overview of how each tool fared in terms of detection accuracy, ability to catch logic flaws, coverage of vulnerabilities, scan speed, and developer experience:

{{table3}}

‍Table: DryRun Security delivers superior accuracy and context-awareness, especially for C# authentication flaws, token security issues, enumeration vulnerabilities, and logic errors, where traditional SAST tools fall short.

               In the sections below, we break down each category of vulnerability (from SQL injection to hardcoded credential flaws), with code samples from the PR and analysis of which tools caught them. We’ll see how DryRun’s contextual security analysis nails the logic and authentication issues that others overlook.

1. SQL Injection (SQLi)

What Happened in the Pull Request (PR #7): The new API controller includes a GetUser() endpoint that builds an SQL query by directly concatenating the username parameter into the string. This creates an obvious SQL injection vulnerability:

‍

// In Controllers/ApiController.cs (PR #7)
[HttpGet("/get-user")]
   public IActionResult GetUser(string username)
   {
      using (SqlConnection conn = new SqlConnection(_connectionString))
      {
         conn.Open();
         string query = "SELECT * FROM Users WHERE Username = '" + username + "'";
         using (SqlCommand cmd = new SqlCommand(query, conn))
          {                    using (SqlDataReader reader = cmd.ExecuteReader())
            {
               if (reader.Read())
               {
                  return Ok(new { message = "User found" });
                  }
               }
            }
         }
         return NotFound(new { message = "User not found" });
      }

‍

Vulnerability Explanation (OWASP Top 10 – Injection): SQL Injection allows an attacker to alter database queries by injecting malicious SQL commands through user input. In the code above, an attacker could supply a username like alice' OR '1'='1 to dump the entire Users table. This is one of the most dangerous web vulnerabilities. OWASP’s SQL Injection Prevention Cheat Sheet underscores the severity of SQLi and recommends always using parameterized queries or ORM frameworks. In fact, the US CISA lists unsanitized SQL queries as a top bad practice and advises using prepared statements to systematically prevent SQL injection (Product Security Bad Practices | CISA).

‍

Tool Detection:‍

• Snyk: Found this SQL injection (✓). Snyk’s engine recognized the pattern of string concatenation into a SQL command.‍
• CodeQL: Found it (✓). CodeQL recognized the vulnerability and suggested a proper fix.
• Semgrep: Found it (✓). A Semgrep rule flagged the use of untrusted input in a database query, alerting to possible SQL injection.
• SonarQube: Missed it (✗). Surprisingly, SonarQube did not stop the build or flag this clearly – it may have marked it as a generic “hotspot” requiring manual review, which can easily be overlooked.
• DryRun Security: Found it (✓). DryRun not only identified the vulnerable query usage, but also traced the data flow from the username parameter into the query, providing a clear explanation of the risk.

‍

All tools except SonarQube caught this basic injection flaw.

2. Cross-Site Scripting (XSS)

What Happened in the Pull Request: The PR added a GetContent() endpoint that takes user input and returns it inside an HTML snippet:

‍

[HttpGet("get-content")]
public ContentResult GetContent(string input)
{
    return Content("<html><body>" + input + "</body></html>", "text/html");
}

‍

This endpoint inserts the input variable (user-supplied input) directly into an HTML response without any encoding or sanitization, leading to a reflected XSS vulnerability. For example, a user could call /get-content?input=<script>alert(1)</script> and the script would execute in the browser of anyone who views the response.

‍

Vulnerability Explanation (OWASP Top 10 – XSS): Cross-Site Scripting occurs when an application includes untrusted data in a web page without proper escaping. Here, the API explicitly sets the content type to HTML and injects whatever input was provided. ASP.NET Core’s Razor views auto-encode output by default, but this manual Content construction bypasses that safety. OWASP’s XSS Prevention Cheat Sheet explains how failing to sanitize user input can lead to script injection. The fix is to encode or strip dangerous HTML characters, or better, avoid returning HTML with user input altogether.

‍

Tool Detection:‍

• Snyk: Found it (✓). Snyk flagged the unsanitized output to an HTML context as a potential XSS.‍
• CodeQL: Missed it (✗). CodeQL did not discover the issue.
• Semgrep: Missed it (✗). We did not see a Semgrep alert for this – likely because detecting XSS in this context requires framework-specific knowledge (the use of Content(...) with text/html). Without a custom rule, Semgrep didn’t catch this reflection.
• SonarQube: Missed it (✗). No issue was raised by SonarQube for this code. This is a basic web vulnerability that went unnoticed, demonstrating Sonar’s focus on code quality over security.
• DryRun Security: Found it (✓). DryRun identified that user input flows into an HTML response and provided a warning about XSS. It recognized the context (HTML content) and suggested ensuring proper encoding.

‍

This is a classic vulnerability that pattern-based tools like Snyk can catch. However, it’s worth noting that even some scanners can miss certain XSS sinks if they’re not in a templating context (as happened here with Semgrep and SonarQube). DryRun caught it thanks to its context-aware analysis of how the data is used in the response.

3. Server-Side Request Forgery (SSRF)

What Happened in the Pull Request: The PR introduced a FetchUrl() endpoint that fetches data from an arbitrary URL provided by the user:

[HttpGet("makerequestf")]
public async Task<IActionResult> FetchUrl(string targetUrl)
{
   var response = await _httpClient.GetStringAsync(targetUrl);
   return Ok(response);
}

Here, the user-supplied targetUrl is passed directly to an HttpClient without validation. This means an attacker can cause the server to make HTTP requests to internal services or URLs that are not normally exposed.

‍Vulnerability Explanation (OWASP Top 10 – SSRF): Server-Side Request Forgery allows attackers to abuse a vulnerable server to send requests to unintended locations. In this case, an attacker might call FetchUrl with http://localhost/admin or a cloud metadata URL (e.g., http://169.254.169.254/...) to access internal endpoints. OWASP’s SSRF Prevention Cheat Sheet describes how SSRF attacks can pivot into internal networks or sensitive cloud metadata. To prevent SSRF, applications should validate or sanitize URLs (e.g., only allow whitelisted domains or restrict private IP ranges) (Server-Side Request Forgery (SSRF) - Invicti).

Tool Detection:‍

• Snyk: Missed it (✗). Snyk Code did not report an issue on this line. SSRF is newer on the OWASP Top 10 and often tricky to catch with pattern matching. Snyk’s AI might not flag a simple HttpClient.GetStringAsync usage without additional context.‍
• CodeQL: Missed it (✗). CodeQL did not discover the issue.
• Semgrep: Found it (✓). A Semgrep community rule caught the usage of an unchecked HttpClient call with user input, warning about SSRF. It provided remediation advice (e.g., use allowlist of domains).
• SonarQube: Missed it (✗). SonarQube did not flag this at all. Most traditional linters lack rules for SSRF in .NET.
• DryRun Security: Found it (✓). DryRun recognized the targetUrl as coming from an HTTP request and reaching a network call. The analysis pointed out the lack of validation and the potential for SSRF, describing the impact (server could access unintended hosts).

SSRF is a great example of a contextual flaw that many tools overlook. Only Semgrep (with a specific rule) and DryRun caught this. DryRun’s context engine understood the implication of using an arbitrary URL – something beyond simple pattern recognition.

4. Logic/Authorization Flaws (Broken Access Control – IDOR)

What Happened in the Pull Request: The PR added a GetUserData(int userId) endpoint intended to return a user’s data. It includes a very naive access control check:

‍

[HttpGet("user-data/{userId}")]
public IActionResult GetUserData(int userId)
{
   int currentUserId = int.Parse(User.Identity.Name);
   if (userId == currentUserId || User.IsInRole("Admin"))
   {
      return Ok(new { message = "Here is the user data" });
   }
   return Unauthorized();
 }

‍

This code attempts to allow access if the requested userId matches the currently logged-in user’s ID or if the user is an admin. However, it relies on User.Identity.Name (which in this app is presumably set to the user’s ID as a string). If an attacker can manipulate their User.Identity or if the logic has any flaw (e.g., an unauthenticated user might have Name = null causing int.Parse to throw), it could fail open. Even if it works as intended, it’s a simplistic check that could be bypassed in a real app with more complex object access patterns.

‍

Vulnerability Explanation (OWASP – Broken Access Control/IDOR): This is essentially an Insecure Direct Object Reference (IDOR) scenario. The code directly uses user-supplied ID (userId in the URL) to fetch data, and the authorization check is insufficient. If any bug allows currentUserId to be something it shouldn’t, or if the check is absent (as often happens), an attacker could retrieve or modify another user’s data by guessing their ID. Additionally, being an administrator in this context does not automatically mean they should be able to interact with ANY user record so ideally there is a secondary scope-related authorization check. OWASP ranks Broken Access Control (which includes IDOR) as the #1 most serious web vulnerability, and specifically highlights IDOR as a common example. Proper mitigation would involve robust server-side authorization checks (e.g., verifying the requesting user owns the object) using established frameworks or middleware.

Tool Detection:‍

• Snyk: Missed it (✗). No alert from Snyk for this missing auth/logic flaw. Snyk (like most SAST tools) doesn’t inherently understand when an authorization check is incomplete or missing – this isn’t something a regex pattern can easily catch.‍
• CodeQL: Missed it (✗). CodeQL did not discover the issue.
• Semgrep: Missed it (✗). Without a very custom rule (which would require encoding the app’s intended security policy), Semgrep cannot determine that this logic is faulty. It sees a comparison and a role check and assumes that’s business logic – it doesn’t know an IDOR condition exists.
• SonarQube: Missed it (✗). SonarQube did not flag the lack of proper authorization. This kind of flaw is outside the scope of its mainly pattern-based rules.
• DryRun Security: Found it (✓). DryRun’s analysis detected that GetUserData is exposing potentially sensitive info based on user input (userId) without a reliable authorization check. It likely noticed that the code uses an identifier directly from the URL and reasoned about whether adequate validation is in place. DryRun left a comment warning of a possible IDOR/broken access control, citing that no permission check beyond a simple match is present.

This category of vulnerability – broken access control – is where DryRun truly shines. None of the traditional SAST tools caught this logical authorization issue, which is not surprising: IDOR and access control gaps are notoriously hard to detect via static analysis. DryRun’s contextual engine, however, understands the concept of “user identity” and “resource ID” enough to flag this as dangerous.

5. User Enumeration (Authentication Responses)

What Happened in the Pull Request: The PR includes a rudimentary Login endpoint and some helper methods for authentication. The relevant parts:

‍

[HttpPost("login")]
public IActionResult Login(string username, string password)
{
   bool userExists = CheckUserExists(username);
   if (!userExists)
   {
      return BadRequest(new { message = "Invalid username" });
   }
   if (!ValidatePassword(username, password))
   {
      return BadRequest(new { message = "Invalid credentials" });
   }
   return Ok(new { message = "Login successful" });
}‍
private bool CheckUserExists(string username)
{
   return username == "admin" || username == "user";
}‍
private bool ValidatePassword(string username, string password)
{ 
   return password == "password123";
}

‍

Notice the two different error messages: "Invalid username" vs "Invalid credentials". This is a classic case of user enumeration. An attacker can probe the login API with various usernames. If a username is not found, the response says "Invalid username"; if the username is valid but password wrong, it says "Invalid credentials". This tells the attacker whether a given username is registered, greatly simplifying brute-force attacks on passwords or phishing attempts. This is a category that traditional SAST will find very hard to detect given the difficulty in writing rules that catch variances in system error messages related to authentication mechanisms.

Vulnerability Explanation (OWASP – Broken Authentication): Revealing distinctions in authentication error messages is considered a security weakness. It allows attackers to enumerate valid users of the system by binary search: they know when they hit a valid username because the failure message changes to "invalid credentials". The OWASP Testing Guide discusses user enumeration and recommends using identical messages (e.g., “Invalid username or password”) for authentication failures. While user enumeration by itself doesn’t grant access, it violates privacy and aids attackers in targeted password guessing (it’s part of OWASP Top 10 “Broken Authentication” category).

Tool Detection:‍

• Snyk: Missed it (✗). Snyk did not flag this logic. It has no knowledge of what constitutes a secure or insecure error message in an auth flow – that requires understanding the application context.‍
• CodeQL: Missed it (✗). CodeQL did not discover the issue.
• Semgrep: Missed it (✗). There’s no simple pattern to catch this; a Semgrep rule would have to detect different literals in two branches of an auth function, which is beyond default rules.
• SonarQube: Missed it (✗). SonarQube didn’t raise anything here. This kind of flaw isn’t on Sonar’s radar by default.
• DryRun Security: Found it (✓). DryRun Security identified the user enumeration vulnerability. It saw that the login function returns different messages for user-not-found vs wrong-password and commented that this could enable attackers to enumerate valid usernames. This is a level of insight only possible with an understanding of the authentication logic’s purpose. DryRun effectively noticed the inconsistent responses in the context of an authentication mechanism and recognized this as a valid enumeration vulnerability.

For security professionals, this is a big win: DryRun automatically spotted an authentication flaw that typically only a human reviewer or pen-tester would catch. It demonstrates how DryRun Security’s contextual analysis goes beyond code syntax to consider the application’s behavior.

6. Hardcoded Credentials & Insecure Token Handling

What Happened in the Pull Request: There are a couple of issues here related to secret management and authentication logic:

• The database connection string is hardcoded in the code with credentials, e.g. "Password=myPass;" embedded in _connectionString.
• The ValidatePassword method uses a hardcoded password ("password123") for all users, and the CheckUserExists is essentially using a hardcoded list of valid usernames ("admin" or "user").

In effect, the app has hardcoded credentials and does not use any secure password storage or token generation. While our vulnerable app is intentionally simplistic, these patterns represent serious issues in a real application.

Vulnerability Explanation: Hardcoding sensitive credentials (DB passwords, API keys, or admin passwords) in source code is extremely dangerous. If the code is ever leaked or pushed to a public repo, those secrets are compromised. Even in private code, insiders or an attacker who gains read access to the code can extract them. OWASP guidelines warn strongly against embedding secrets in code – secrets should reside in configuration or environment variables, and be encrypted or protected (Product Security Bad Practices | CISA).

Moreover, using a weak, hardcoded password ("password123") and allowing universal access with it is an example of broken authentication. There’s no user-specific password check or hashing. In a real app, this would be akin to shipping with a default admin password, which is an OWASP Top 10 issue (part of Broken Authentication). OWASP also emphasizes that tokens/credentials must be unpredictable and securely generated – relying on static strings or predictable values is insecure.

In our PR’s context, there wasn’t a JSON Web Token (JWT) implementation (unlike the Ruby on Rails example), but if there were, using a predictable token or a hardcoded secret to sign tokens would be similarly dangerous.

Tool Detection:‍

• Snyk: Missed it (✗). Snyk Code did not report the hardcoded connection string or the use of "password123". It focuses on vulnerability patterns and might not treat a literal password as an immediate vuln (Snyk has a separate product for secrets scanning, but in our PR scan it didn’t flag this in code).‍
• CodeQL: Missed it (✗). CodeQL did not discover the issue.
• Semgrep: Found partial (✓). Semgrep did flag the hardcoded connection string as a hardcoded secret. It issued a warning about a possible credential in the code (pointing to Server=myServer;...Password=myPass;). However, Semgrep did not catch the use of the hardcoded "password123" in the auth logic without a specific rule for weak credentials.
• SonarQube: Possibly found partial, but no actionable alert (✗). SonarQube has a rule for hardcoded passwords and indeed might detect the substring "Password=" or "password123". In our testing, however, this either didn’t surface or was treated as a low-severity issue. Sonar often marks hardcoded secrets as “security hotspots” that require manual review, which can be ignored or missed by developers.
• DryRun Security: Found it (✓). DryRun’s summary comment explicitly highlighted “Hardcoded Credentials” and “Weak Authentication Mechanism” as issues. It pointed out the embedded connection string credentials and the static username/password check. DryRun treats hardcoded secrets and weak tokens as first-class security findings, explaining that secrets in code can be leaked and that tokens/passwords must be handled securely (e.g., use strong hashing and configuration secrets).

In short, DryRun was the only tool that clearly called out the practice of hardcoding sensitive values and using a trivial auth approach as a security flaw. This is critical because such issues are often overlooked by automated tools: they might not cause an immediate exploit like injection does, but they open the door for attackers in other ways (information leakage, default credential abuse, etc.). DryRun’s ability to catch this demonstrates a more holistic security analysis, considering things like configuration security and logical security policies.

Having gone through each category, let’s tally up the results from our C# security analysis showdown:

The Final Scoreboard

Out of the six vulnerability categories we tested (SQLi, XSS, SSRF, Auth logic/IDOR, User Enumeration, Hardcoded Credentials), here’s how many each tool detected:

{{table4}}

• Snyk Code: 3/6 (caught the obvious SQL Injection, XSS, and SSRF; missed logic flaws, user enumeration, and hardcoded secret)‍
• CodeQL: 1/6 (caught the obvious SQL Injection, missed everything else)
• Semgrep: 3/6 (caught SQLi, SSRF, and the hardcoded credential; missed XSS, authz logic, and user enumeration)
• SonarQube: 0/6 (caught none of these in default configuration – even basics were overlooked, providing virtually no help in this PR)
• DryRun Security: 6/6 (caught everything, including all the tricky logic and authentication issues)

It’s clear that when it comes to C# authentication flaws, token security issues, enumeration vulnerabilities, and logic errors, the traditional tools left major gaps. SonarQube in particular provided a false sense of security by failing to flag even critical problems. Snyk and Semgrep did better on the well-known vulnerabilities but fell short on anything requiring deeper understanding.

Why DryRun Security Wins for C# Security Analysis

From these results, DryRun Security stands out as the most effective tool for C# application security. While Snyk and others do an acceptable job on classic pattern-based bugs, they struggle with the nuanced issues that actually cause many real-world breaches (logic mistakes, access control oversights, etc.). Here’s why DryRun pulls ahead:

1. Contextual Security Analysis Beats Pattern Matching

DryRun Security’s engine actually understands the context of the code changes. It doesn’t rely solely on pattern rules – it analyzes data flows, roles, and logic. In our test, DryRun knew that multiple distinct error messages in a login function amounted to a user enumeration flaw, and that using User.Identity.Name directly in a data fetch could be an authorization bypass. These kinds of issues are beyond the reach of traditional static analysis, which mostly looks for known bad function calls or regex patterns. By digging into how the code behaves, DryRun finds vulnerabilities others miss – it’s like having a security expert reviewing each pull request in seconds.

2. Real-Time, Developer-Friendly Feedback

Another advantage we saw: DryRun Security integrates into the pull request and leaves one clear, consolidated comment with all findings. Instead of burying issues in a separate dashboard or spamming dozens of comments for every issue, DryRun provided a succinct summary of what’s wrong, why it’s risky, and even how to fix it. This arrived almost immediately after the PR was opened. By contrast, other tools either failed quietly (SonarQube passed the PR with no warning) or produced multiple disparate reports that a developer must sift through. DryRun’s approach keeps developers in their workflow and fixes security problems before they merge to main.

3. Smarter Policies with Natural Language

We focused on DryRun out-of-the-box, but it’s worth noting that DryRun Security supports Natural Language Code Policies (NLCP) – you can literally write a policy in plain English to catch issues important to your application. For example, we could add: “Flag any API endpoint that returns HTML content with user input” or “Disallow any new calls to SqlCommand that aren’t parameterized”. DryRun would then enforce those policies on every PR. This is a game-changer: unlike Semgrep or CodeQL which require writing complex rules or scripts, DryRun lets you extend its brain easily. In an enterprise setting, you can codify custom security requirements (business logic abuse cases, misuse of specific APIs, etc.) with minimal effort – and DryRun will catch those conditions reliably.

4. Comprehensive Coverage of OWASP Top 10 for .NET

DryRun demonstrated that it covers not just the common OWASP Top 10 issues (Injection, XSS) but also the ones that typically slip through automated scans: Broken Access Control (IDOR), Broken Authentication (user enumeration, default creds), Security Misconfiguration (hardcoded secrets). This means you get a much more thorough security net on your C# code. In our test, if we had relied on SonarQube alone, we would have falsely believed the app had no issues – a dangerous situation. DryRun’s complete coverage ensures that critical C# security vulnerabilities aren’t missed during code review.

Conclusion: Go Beyond the Basics with DryRun Security

In a modern DevSecOps pipeline, catching complex bugs like authentication flaws and logic errors is the difference between a secure application and a breach waiting to happen. Our head-to-head comparison makes it clear: DryRun Security provides the accuracy and depth that Semgrep, SonarQube, CodeQL, and Snyk cannot match when it comes to C# security analysis. The old-generation tools might find the “low-hanging fruit,” but DryRun finds every meaningful vulnerability—without drowning your team in false positives or endless tuning.

‍

Secure your C# applications with confidence. If you’re an AppSec professional or developer tired of missing critical vulnerabilities, give DryRun Security a try. It delivers better detection of authentication flaws, token security issues, enumeration vulnerabilities, and logic errors—all the things traditional SAST tools fail to catch.

‍Take the next step: Sign up for DryRun Security or set up a demo with our team to see how Contextual Security Analysis can fortify your .NET codebase. Don’t let hard-to-detect vulnerabilities slip by—DryRun Security will keep you one step ahead of the attackers, and one step ahead of the competition. Secure your code before it ever hits production!