Threads
Loading user profile...
haha the irony would be peak. wrote the scanner manually tho along with my cofounder, did use ai for the test tho cuz needed quick results and decent repos available online lol. but yeah honestly your framing is the most accurate ive seen on this. its not vibe coding itself, it's the gap between what the output looks like and what the developer actually understands. a senior dev using Cursor who knows what SECURITY DEFINER does catches it in review. a non technical founder who's never touched postgres has no mental model to even know what to look for. thats the gap these numbers are showing. the test point is underrated too tbh none of these 48 apps had tests which forces you to think about edge cases which is half of what catches auth issues
>
you're right actually, security definer has totally legitimate uses, nothing wrong with the feature itself. the problem is specifically how ai tools reach for it, they get a permission error, they add SECURITY DEFINER because it makes the error go away, and that's it. no review, no guardrails, no checking the caller's identity at the top of the function. a properly written one is fine. an ai gen'd one that exists because claude got a 403 and "fixed" it? that one's running as superuser for anyone who calls it. so i mean not suggesting it but u could go for the spanking maybe haha
fair point, and i'm not saying it can't catch it like if you paste the right file and ask the right question it probably will. the issue is most founders don't know to ask. the model generated that SECURITY DEFINER function in the first place without flagging it, so why would someone think to specifically ask about it later. and even if they do, you get a different answer every time you run it which makes it pretty useless as an actual checklist. "can catch it when prompted correctly" and "reliably finds it across your whole codebase before you ship" i feel are pretty different things
I'm keeping the actual repo names private since most of these founders don't know about the issues yet and publicly naming them before they've had a chance to fix anything feels wrong. I'll be dming you an anonymized dataset tho. Anyone else curious about the dataset feel free to dm me!