Wait, you don't send the fixes into the IDE?

When I first tell people at Pixee that we send Pull Requests (or Merge Requests, for your GitLabers) to fix vulnerable code, they always look at us quizzically for a second -- whether they be partners, prospects, friends, or pets.

I get it. It seems natural to think about the IDE first. Other vendors do stuff that way. People have been chanting that we should "shift left" for many years, and what’s more left than the IDE? There's a few major inter-related problems with this approach, and there's no evidence to suggest that these solutions work. If this did work, and developers wanted to adopt these tools, and it led to mass remedation and fixed code -- wouldn't we have seen that by now?

It reminds me of how milk companies love to tell us how important Vitamin D is. But, after decades and countless studies, we still can't really tell you if Vitamin D is useful to supplement. We understand how it could be helpful. Observational studies show tantalizing results. But, when you actually run a randomized controlled trial, there's just not much there there, no matter how cool it would be if there were.

The IDE-based approach by vendors to fix your stuff is kind of how Big Milk wants you to think Vitamin D is super important, because it's already in what they give you today -- not because it's actually the best thing for the security outcomes you want.

Developers Want Their Flow State

Developers don't want security interruptions in their IDE. This doesn’t come from any opinion or value judgment on security – developers don’t want any interruptions or distractions. They want to maximize time in flow state.

I'm somewhat of an expert on this, given how much I interrupt my developers. I can see from the look in their eyes that they really wish I took more vacation.

Pull Requests Have Proof Points Built In

Modern SCM tools are where your CI lives too, so a new Pull Request often comes with built-in formatter checks, unit tests, integration tests, etc. The battery of tests new code goes through there form a standard of quality, consistency, etc., that we want already verified before we waste our precious time reviewing them.

Compare this to a random piece of advice, usually from out of context, that appears in your IDE. Not only are you probably not caring about that thing at that time (interrupting flow state), but you also don't have the ability to quickly reason about all those assurances the checks on your pull request give you out of the box.

Security Doesn't Get What They Want

Security also ends up not caring about these tools, too. At my last job, we made an IDE plugin, mostly for one customer who really wanted it. But, after a few months, it fizzled. They didn't know if their developers were using it, or if they were acting on the results. As an alternative, a Pull Request is much harder to ignore. This both raises the bar in that the Pull Request must be helpful and easy to review, to avoid adding to the noise.

But, this also puts the developer on the official record. It's easy to ignore some ephemeral suggestions in my IDE, but I don't want to be the guy that declines a vulnerability fix on the record in GitHub for no good reason.

Developers Have Already Voted With Their Feet

See for yourself -- go look at the IDE plugin marketplaces who publish plugin adoption numbers. Developers adopt IDE plugins that help them maintain focus, tighten feedback loops, and write more code, but they don't use security tools there.

And, Vitamin D probably won't hurt you. Keep taking it.