AI programming capabilities evolve! Code quality can be improved through continuous PUA of large models - AI article

In recent years, artificial intelligence (AI) programming capabilities have continued to develop, but have not yet reached perfection. Recently, Max Woolf, a senior data scientist at BuzzFeed, discovered through experiments that if a large language model (LLM) is constantly provided with prompts to "write better code", AI can indeed generate better code. This discovery has attracted widespread attention, and well-known AI scientists in the industry have also expressed great interest in it, emphasizing the importance of iteration and prompt word design.

In Woolf's experiment, he used the Claude3.5Sonnet version of the AI ​​model to perform a series of programming tasks. At first, he posed a simple programming problem to the model: how to find the difference between the minimum and maximum values ​​among a million random integers whose digits sum to 30. After receiving this task, Claude generated code that met the requirements, but Woolf believed that the code still had room for optimization.

Then, Woolf decided to ask Claude to perform iterative optimization through the prompt "Write better code" every time after generating code. After the first iteration, Claude refactored the code into an object-oriented Python class and implemented two significant optimizations, increasing the running speed by 2.7 times. In the second iteration, Claude added multi-threading processing and vectorized calculations, which ultimately made the code run 5.1 times faster than the basic version.

However, as the number of iterations increases, the improvement in code quality begins to slow down. After several rounds of optimization, although the model tried to use some more sophisticated techniques, such as JIT compilation and asynchronous programming, some iterations resulted in performance degradation. Ultimately, Woolf's experiment revealed the potential and limitations of iterative prompts, giving people new thinking about the future of AI programming.

This research not only demonstrates the application potential of AI in the field of programming, but also reminds us that although AI can improve code quality through continuous iteration, in actual applications, how to properly design prompt words and balance performance and complexity is still a worthwhile issue. Topics discussed in depth.