I just spent way too many hours installing local LLMs and asking them to write a super basic function:

“Write a PowerShell core function that will get all members of the Protected Users groups in all domains in a forest. It should use .NET classes instead of the ActiveDirectory module.”

This is something that would take me no time at all, but none of the local LLMs came close, even with coaching.

Maybe it’s the local LLM that’s the problem? So I tried ChatGPT through the web interface, and it was terrible.

What about a coding-trained assistant? GitHub’s Copilot was also a big miss.

Even the results that looked mostly correct were slightly flawed here and there. I’m still not opposed to LLMs writing code, but the things they are good at right now are not what I need from them.