How can we effectively program using AI?
With the recent development of the AI field, some software developers have started to use AI to generate code, and
crawshaw - How I program with LLMs - 2025-01-06
https://crawshaw.io/blog/programming-with-llms
There are particular conversational structures that work better than others when I 'chat' with LLMs. And a weirdly large amount of repeated mundane work, copying code to compile, pasting compiler errors back to the LLM.
— David Crawshaw (@davidcrawshaw) January 6, 2025
Crawshaw, who is interested in new technologies, focused on generative AI's 'technology that can generate sophisticated answers to difficult questions in a short time' and introduced generative AI into his programming work. Crawshaw uses generative AI in three ways: 'auto-fill', 'search', and 'chat-driven programming'.
・Automatic input
According to Crawshaw, AI can significantly improve productivity by taking over obvious typing tasks. 'I couldn't do a week's worth of work without AI,' Crawshaw said, praising the reduction in the burden of repetitive tasks.
·search
If you have a question like 'How can I make a button transparent with CSS?', you'll get a much better answer by asking a consumer AI like GPT-o1 or Claude 3.5 Sonnet than by using a traditional search engine. Crawshaw cautions against blindly trusting AI, saying, 'Just as humans can make mistakes, so can AI.'
Chat-driven programming
Crawshaw praised 'chat-driven programming,' which involves programming through dialogue with AI, for the following reasons: instructions can be given in natural language, there is no need to remember complex programming language syntax, and the AI handles most of the code generation, allowing developers to focus on solving more advanced problems. He said, 'This maximizes the value of AI.' However, he also said, 'AI can generate incorrect or inefficient code, it takes time to learn how to extract value from AI, and you also need to adjust your programming methods. This is the most difficult of the three methods.'
When programming using AI, the main job of a human is to 'read the generated code and judge whether it is appropriate or not.' In order to make it easier to judge the generated code, Mr. Crawshaw said that he pays attention to the following points when having AI program it.
Make the problem specific and pinpointed
Giving the AI all the information in the repository would distract it and make it harder to generate the right code, so Crawshaw uses the web version of the AI chat to narrow the context and create prompts that are focused on a single task.
Provide complete context
It's important to give the AI all the information it needs to understand the task, such as the relevant libraries and functions, and what result is required.
· Request work that is easy to verify
An ideal task for an AI is one that requires the writing of easy-to-read tests that use many common libraries that humans cannot handle. Since an AI can usually complete a task in a matter of tens of seconds and easily redo it, it is also good to give instructions such as 'Please introduce the concept ____ to make the test easier to read and redo it all' after executing the task once.
'The most effective way to use AI is to write prompts like exam questions in school,' Crawshaw said, in the sense of giving specific tasks and providing just the right amount of information.
As an example of how AI can be used, Crawshaw posted a blog post showing how he generated code and tests for a float quartile reservoir sampler program, then interacted with the AI to fix the code and improve the tests.
When Crawshaw gave the AI a task, it generated the following test along with what looked like good code: This is a form of testing that humans often write, where they generate expected output using other tools or manual calculations and check that it matches the code’s results. But this approach is suboptimal because AIs are bad at math and often just make up “expected output.”
So, Crawshaw prepared a different implementation that calculates the exact quartiles and compared it with the output of the test target to generate a more reliable test. The new test code also allows for fuzz testing, which generates a large amount of random input data to find unexpected bugs.
'AI should make it easier to write more comprehensive tests and fuzz test implementations that previously required developers to build them in a time-consuming manner,' Crawshaw said.
Crawshaw is also developing ' sketch.dev ' as a tool to support programming using AI. This tool provides an environment similar to Go Playground and features features such as a chat interface, integration with various tools, and automatic feedback to AI models. Crawshaw predicted that 'with the use of AI, in the future we will see more specialized code, fewer generalized packages, and more readable tests.'
Related Posts:
in Software, Posted by log1r_ut