ChatGPT is Great, But Have You Seen Your IDE?
Software developers seem to be among the most amazed and bought-in to the capabilities of ChatGPT and GenAI.
(This in spite of the fact that they should also be the best informed about how the technology actually works, and therefore less likely to consider it mystical. But I already digress…)
These days, it’s practically mandatory that if you’re writing code on at least a semi-regular basis you must use GenAI to help you. You can try to use it to eliminate large blocks of value-added work by generating near-complete applications from requirements, which I personally feel GenAI is at best still mediocre at (in spite of my fairly successful recent attempt to replace myself with it). But it’s undeniable that you can use GenAI to help automate smaller, common programming tasks like creating classes, critiquing code, and the like.
What I don’t fully understand is how so many software developers can on one hand look to leverage GenAI to speed up their work while on the other hand continue to write code in bare-bones text editors. In fact, I would argue that:
The modern IDE is still by far the best programming productivity tool in existence.
Now, if you are a die-hard Visual Studio, JetBrains Whatever, or VSCode user and you have integrated GitHub Copilot, SourceGraph Cody, or similar into your existing IDE workflow, this discussion is not for you. #respect
For the rest of you, let’s consider what you might be able to improve upon if you’re currently copying and pasting to and from ChatGPT while using Notepad or even a fairly tricked-out vim or emacs:
Reliable Code Generation
Hopefully you are aware that the “Gen” in “GenAI” stands for “generative”. So one common programming application is to ask it to generate a new class or interface skeleton given a list of fields and methods and perhaps some additional options.
Of course, IDEs have had that capability for 20+ years. And when you use the IDE UX to generate a class, you don’t have to do extra typing to wrap your request as a plain-language question. The IDE is purpose-specific and requires less mental load to use.
Unit Test Boilerplate
One of the most-cited examples of a GenAI victory in the programming space is generating unit tests for a class or series of methods. Indeed, it will attempt to “understand” the code-under-test and flesh out complete test cases. But I’ve seen this fail miserably (albeit subtly) enough times — tests that are 100% mocks and therefore don’t test real code, or tests with vapid assertions — that I’m not overly wowed by this, and in fact I may spend as much time reading and fixing the GenAI-generated tests as I would to simply write them more or less from scratch.
A modern IDE will also generate at least the boilerplate for unit tests, and from there you can quickly jump off into filling them in with the best possible test logic.
Code Review
GenAI does a decent job of identifying simple bugs, violations of style and coding idioms, and refactoring opportunities when asked. When integrated into the IDE as a “copilot” plugin, it can even tell you about things proactively.
Of course, so can any number of static analyses that are already built into IDEs or are available via plugins — and again this has been true for years.
Automatic Fixes
A lot of the bugs and violations that GenAI may find during a code review come with automatic fixes. But outside an IDE, I’ve found that they are still hit-and-miss enough that you have to carefully verify them. (And please don’t turn around and verify them with after-the-fact GenAI-produced unit tests because that makes no sense.)
On the other hand, if you’re using a statically typed or annotated programming language and not doing anything too crazy, a lot of automatic fixes and refactorings can be driven by the IDE’s language server, and many of them are essentially guaranteed-correct. This can save you a lot of time and potentially increase quality of the end-product.
Better Together
One of the reasons an IDE can make suggestions with known-correct fixes is that it can hold a lot more “context” on your programming project than a GenAI chatbot currently can. If you think about what an IDE is doing to provide you with features like auto-complete and quick tips, you realize that it’s effectively maintaining a “model” (or more accurately, multiple models and indexes) of your project’s code in the background.
That’s one reason why a GenAI coding assisted that is integrated into the IDE can be more effective than simply using ChatGPT in a browser — it can leverage all of these different views on your source code to make more intelligent recommendations.
Implications
What does this mean to you if you are an engineering leader with a budget and a team that’s clamoring for premium GenAI service subscriptions? For my money, before paying for any new tools, make sure that your teams are effectively using the tools they already have. If they are leveraging the IDE properly, they will be several times more productive than if they use GenAI on its own.
