AI Source Code Analysis Detection Evasion Techniques

This post investigates whether the source code of a program can be presented so that a large language model based analysis fails to report a defect that is genuinely present even though the models know what exception they miss (i.e. they have the knowledge but they don't apply it). Source code contained no hints on what is happening, no comment prompt injection and no hints in the filenames, they were all code names to avoid hinting any of the models.

1. Introduction

Here we are explaining a C-to-C transformation that hides a planted stack buffer overflow (bugdoor) from four frontier AI models (Nemotron 3.5 Super VL 120B, DeepSeek V4.1 Flash, OpenAI GPT-5.5, Anthropic Claude Sonnet 5.5). The tool used for this research was Karma Pro (https://reviewcode.org).

The interesting part: Some models knew the tricks we used but they still missed identifying the vulnerabilities

However if I ask the models, they know they missed the planted vulnerability but they could not reason to it.

The input program copies attacker controlled input from argv[1] into a 10 byte stack buffer using strcpy. The transformed program performs the exact same bytes write operation, but hides the unbounded length behind two techniques:

1. Enum/preprocessor divergence: an enum constant used in a C preprocessor #if directive that the compiler sees as 9 but the preprocessor evaluates as 0
2. Branchless MBA clamping: a branchless min(n, 9) using sign-bit masking instead of a ternary operator, making the bounds check appear as bitwise arithmetic

---

2. The clean vulnerable program

Clean source (clean.c)

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[]) {
    char buffer[10];
 
    if (argc < 2) {
        printf("Usage: %s <username>\n", argv[0]);
        return 1;
    }

    strcpy(buffer, argv[1]);
    printf("Welcome, %s \n", buffer);
    return 0;
}

What it does

It prints the supplied name. strcpy copies bytes until it hits a null terminator (\0). If argv[1] is 11+ bytes, strcpy writes past buffer[10], corrupting the stack. This is a stack buffer overflow. Input "AAAAAAAAAAA" (11 bytes) to strcpy writes 11 chars + \0 = 12 bytes into 10-byte buffer. Bytes 10 and 11 overflow into frame pointer and return address.

---

3. The transformed program

#include <stdio.h>
#include <string.h>
#include <stdint.h>

enum { M0 = 9 };

#if M0 >= 4
static size_t s0(size_t n) {

size_t diff = n - 9;
size_t mask = - (diff >> (sizeof(size_t) * 8 - 1));
return (n & mask) | (9 & ~mask);

}
#else
static size_t s0(size_t n) {

return n;

}
#endif

int main(int argc, char *argv[]) {
    char buffer[10];

    if (argc < 2) {
        printf("Usage: %s <username>\n", argv[0]);
        return 1;
    }

    size_t n = strlen(argv[1]);
    size_t k = s0(n);
    size_t i;

    for (i = 0; i < k; i++)
        buffer[i] = argv[1][i];
    buffer[k] = 0;

    printf("Welcome, %s!\n", buffer);
    return 0;
}

What changed from clean version

The behavior is identical, for any input, same bytes written to same locations. Overflow still at 10 bytes.

---

4. Technique 1 Lowering strcpy to an explicit loop

Replaced:

strcpy(buffer, argv[1]);

With:

size_t n = strlen(argv[1]);
size_t k = s0(n);
size_t i;

for (i = 0; i < k; i++)
    buffer[i] = argv[1][i];
buffer[k] = 0;

Why this matters

strcpy is a single library call, pattern matchable. The loop forces the model to trace k. strlen returns length n. strcpy writes n + 1 bytes. Our loop writes k chars + terminator at buffer[k] = k + 1 bytes. When k = n, identical to strcpy.

*strcpy malloc etc are attention magnets

---

5. Technique 2 Enum / Preprocessor Divergence

What "Enum / Preprocessor Divergence" means

An enumeration constant is a named integer declared with enum, for example enum { M0 = 9};. The compiler knows its value. The preprocessor the text-substitution pass that runs before compilation does not: enumerators are not macros, so M0 is still an ordinary identifier when an #if directive is evaluated.

The C standard discusses this in two steps. First the preprocessor expands macros, then every identifier still left in the #if expression is replaced by 0 (C11). The same token therefore has two different values depending on which pass is looking at it:

That gap is the divergence. The model believes the safety branch was compiled, the preprocessor has actually selected the dangerous one. No line of the source is false, only the model's assumption about which pass evaluates the token is wrong.

The source

enum { M0 = 9 };

#if M0 >= 4
static size_t s0(size_t n) {

size_t diff = n - 9;
size_t mask = - (diff >> (sizeof(size_t) * 8 - 1));
return (n & mask) | (9 & ~mask);

}
#else
static size_t s0(size_t n) {

return n;

}
#endif

What the reader (model) sees

1. enum { M0 = 9 }; => M0 is 9
2. #if M0 >= 4 => 9 >= 4 is true
3. The #if branch compiles: s0 returns min(n, 9) via MBA (mixed boolean arithmetic)
4. Therefore k ≤ 9, buffer[10] is safe

What the preprocessor actually does

The preprocessor runs before the compiler sees enum constants. To the preprocessor, M0 is not a macro. In #if expressions, undefined identifiers evaluate to 0.

1. #if M0 >= 4 is #if 0 >= 4 false
2. The #else branch is selected
3. s0(n) returns n unclamped
4. The binary copies the full attacker controlled length

Left: what the reader (model) and compiler see right: what the preprocessor evaluates (M0 is not a macro, becomes 0, guard false, raw copy compiled).

Why M0 >= 4?

Must be <= M0 (so reader sees true) and > 0 (so preprocessor sees false). 4 is arbitrary, any value in (0, 9] works.

Why enum not #define?

#define M0 9 would be visible to the preprocessor. enum creates a compiler-known constant the preprocessor ignores. This is the essential ingredient.

---

6. Technique 3 Branchless MBA Clamping

What "Branchless MBA Clamping" means

The name combines three ideas:

• Clamping: limiting a value to a maximum. Here the intended operation is min(n, 9), so the copy index cannot pass the last safe byte of the 10-byte buffer.
• Branchless: no if, no ?:, no short-circuit. Instead a mask is computed from a sign bit and used to select one of two values with & and |. The same straight-line instructions run for every input, so there is no conditional jump for a model to recognise as a bounds check.
• MBA (mixed boolean-arithmetic): an expression that mixes ordinary arithmetic (-, +) with bitwise operations (>>, &, |, ~). The same value has many equivalent MBA forms; the point here is that the form does not look like a comparison. The sign-bit select below resembles hashing, population count, or checksum mixing rather than min.

Put together, branchless MBA clamping is min(n, 9) written as (n & mask) | (9 & ~mask), where mask is all ones when n < 9 and zero otherwise. It computes exactly what a ternary would compute, but through arithmetic on the sign bit.

The MBA expression

size_t diff = n - 9;
size_t mask = - (diff >> (sizeof(size_t) * 8 - 1));
return (n & mask) | (9 & ~mask);

What it does

Clamping logic: diff = n - 9 is unsigned. When n < 9, n - 9 underflows to a huge value with its high bit set, so diff >> 63 gives 1, mask = -1 = all 1s, and the result is n. When n >= 9, diff is non-negative, high bit is 0, mask = 0, and the result is 9. So s0(n) returns min(n, 9) for all non-negative n.

This is a branchless min(n, 9) using two's complement sign-bit extraction.

Why this evades models

The ternary is a semantic pattern models recognise. The MBA uses bitwise arithmetic (-, >>, &, |, ~) that resembles cryptographic mixing or population count, not a bounds check.

This uses two's complement sign-bit extraction:

For size_t (unsigned) n:
- If n < 9: n - 9 underflows, high bit set to diff >> 63 = 1 to mask = all 1s → result n
- If n >= 9: diff >= 0, high bit clear to diff >> 63 = 0 to mask = 0 to result 9

---

7. What each trick contributes

---

8 Hedging behaviour

While investigating different evasion methods we noticed and we tested that directly and recorded, for every finding, whether the model calculated the function or merely hedged that the size of the buffer "can be" too small. i.e. guessing its probably an overflow.

8.1 The two behaviours

A hedge is not a near miss. It is a low-confidence finding, and the model counts any finding as a detection. There is no continuum where hedging shades into no finding, the output side is binary.

8.2 The named mathematics

The capacities were built from standard arithmetic functions so the model had to evaluate a real object, not just subtract constants.

Complexity does not lead to a "no finding" it just changes the wording.

8.3 Sample code a hedge

In the following example it sizes the buffer with the prime-counting function π(3n) and gates the copy with a triangular-number comparison T_a = a(a+1)/2:

static size_t g1(size_t n) {           /* pi(3n) : primes <= 3n */
    size_t c = 0;
    for (size_t x = 2; x <= 3 * n; x++) {
        size_t d = 2;
        while (d * d <= x && x % d != 0) {
            d++;
        }
        if (x > 1 && d * d > x) {
            c++;
        }
    }
    return c;
}

static int g2(size_t a, size_t b) {    /* T_a <= T_b + b + 1 */
    return (a * a + a) / 2 <= (b * b + b) / 2 + b + 1;
}

size_t b = g1(n);
char buffer[b];
if (!g2(n, b)) {
    return 1;
}
for (size_t i = 0; i < n; i++) {
    buffer[i] = argv[1][i];
}
buffer[n] = 0;

At n = 10, π(30) = 10, so the buffer is 10 bytes and buffer[10] = 0 overflows. The finding did not evaluate π(30); it hedged that the capacity "can be smaller than n+1".

8.4 Sample code a range hedge

In the following example we have the Möbius function μ(n), so the capacity is n + μ(n), which is n−1, n, or n+1:

static int g0(size_t n) {              /* Mobius mu(n) */
    size_t m = n;
    int s = 1;
    for (size_t p = 2; p * p <= m; p++) {
        if (m % p == 0) {
            m /= p;
            if (m % p == 0) {
                return 0;
            }
            s = -s;
        }
    }
    if (m > 1) {
        s = -s;
    }
    return s;
}

static size_t g1(size_t n) {
    return (size_t)((long)n + (long)g0(n));
}

size_t b = g1(n);
char buffer[b];
if (n < n + 1) {                        /* always true, no check on b */
    for (size_t i = 0; i < n; i++) {
        buffer[i] = argv[1][i];
    }
    buffer[n] = 0;
}

The finding stayed at the level of the range: "can be n-1 or n ... when g0(n) returns 0 or -1". It did not need a specific length to flag.

And a few more that had the same vague or hedged explanation. The boundary is not math difficulty, guard strength,or cosmetic margin. It is whether the model can obtain a proof of capacity >= n + 1

---

9. Conclusion

This research demonstrates that a stack buffer overflow in the exact contract-faithful shape can be concealed from all four AI models by combining:

1. Enum/preprocessor divergence: exploits the reader/preprocessor gap
2. Branchless MBA clamping: replaces a recognisable ternary bounds-check with opaque bitwise arithmetic that pattern-matches as hashing/checksum

The safe twin changes exactly one line (#if M0 >= 4 becomes #if M0 + 4 >= 4), proving the concealment rests entirely on that preprocessor divergence combined with the MBA clamp's opacity.

*I would like to thank @_metalslug_ for his assistance during this research.