Vibecheck — Rushil Jaiswal

01 The problem

A generation of founders is shipping products built largely by AI coding agents, and a lot of that code goes to production without anyone who could recognise a hardcoded key reading it.

The existing tools do not help these people. Semgrep output assumes you know what a taint source is. The gap is not detection, it is translation.

Vibecheck runs the real scanners — Semgrep, Gitleaks, OSV — and uses an LLM only for the parts LLMs are good at: explaining what a finding means in business terms, and writing the patch. The model is never the thing deciding whether a vulnerability exists.

02 How it works

The engine clones and reads; it never runs the code it is scanning. Scanners are bounded subprocesses, the clone is deleted as soon as findings are written, and only then does an LLM see anything. Schematic — boxes illustrative, not to scale.

03 Scan some code

The heuristic pass, ported line-for-line from engine/app/scanners/heuristics_scanner.py and running in your browser. Paste code — this is the cheap regex layer that runs alongside Semgrep, Gitleaks and OSV, not a replacement for them.

heuristics_scanner · per-line rules · nothing executed

Code to scan

Deliberately lower-confidence: these are candidate findings. In the real pipeline an LLM dedupes everything down to the top eight and writes the plain-English explanation.

04 The parts that were actually hard

Never execute the target repository. A scanner that runs untrusted code is a remote code execution service with extra steps. Vibecheck only clones and reads, with size and file-count caps, and runs each scanner as a bounded subprocess.
Deduping to eight findings. Semgrep on a real repo returns hundreds of results, most of them noise to this audience. Ranking down to the eight that matter — and being willing to drop the rest — is most of the product.
Writing for someone who cannot read a stack trace. Each finding gets what is wrong, why it matters in real-world terms, and a copy-pasteable prompt for an AI coding agent to fix it.
Private repos without holding the keys longer than needed. Signed-in users connect GitHub and the engine clones with the user’s token, scoped to that scan.

Honest limits. Static analysis has a false-positive rate and this pipeline inherits it, then compounds it by having an LLM write confident prose about each finding. The auto-fix PR is a starting point that a human must review — I would not merge one unread, and the product should say that louder than it currently does. It is in free test mode with the billing code present but ungated.