What are the '8 misconceptions' about software engineering and generative AI?
In recent years, there has been an acceleration in the integration of generative AI tools into business operations, and the use of coding AI has become commonplace, especially in software development. In this context, the computer magazine ACM Queue has compiled a list of 'eight misconceptions' about software engineering and generative AI, arguing that claims about AI in software development are mixed with marketing appeals and misunderstandings.
Eight Myths on Software Engineering and GenAI - ACM Queue
According to ACM Queue, generative AI is rapidly transforming the software engineering landscape, but some of the widely publicized benefits of AI include marketing appeals, exceptional cases, and misinterpreted research findings. Based on recent large-scale research, interviews, and observations of development environments, ACM Queue has identified eight common misconceptions.
◆1: Developers spend most of their time writing code
Software engineering is a demanding job that requires creativity, long hours of concentration, and a lot of collaboration, but surprisingly, developers don't actually spend that much time writing code. Because large software projects are created through the collaboration of multiple teams and various groups, developers need to spend time on group activities such as meetings, planning, and code reviews in addition to writing code.
A 2025 survey of over 450 engineers working at Microsoft reported that developers spent approximately 14% of their working hours writing code, with the remainder spent on other tasks. Another study showed that even on days when engineers felt they were 'having a good day,' they spent only 18% of their time coding, compared to around 11% on days when they felt they were 'having a bad day.'
◆2: Writing code is the bottleneck.
People who aren't directly involved in software development tend to think that 'the biggest bottleneck in software development is code creation, and the introduction of coding AI should dramatically improve productivity.' However, since developers only spend about 15% of their total working hours writing code, even introducing coding AI to speed up coding will only result in an overall productivity improvement of less than 15%.
Furthermore, accelerating code creation with AI without considering peripheral tasks such as software design, understanding existing code, and environment setup can lead to unexpected results. For example, increased code generation capacity will also increase the burden on downstream processes such as code review, testing, and product integration. ACM Queue stated, 'While the overall development cycle is determined by the speed of the slowest phase, coding is not necessarily the slowest phase. Therefore, while using AI primarily as a code generation tool may be helpful for individuals, it is not necessarily the best way to use AI to release software more quickly.'
◆3: The number of lines of code written by AI is the best indicator to measure its impact.
In software development, more code isn't always better, yet many organizations still measure developer productivity by 'lines of code written per hour.' Therefore, similar metrics have been applied to coding AI, with the number of lines of code generated being used to measure AI performance.
However, using lines of code as a metric is dangerous in an unhealthy organizational culture. If developers are pressured to prioritize lines of code, they risk sacrificing collaboration and design quality, as well as increasing technical debt and security vulnerabilities. Coding AI that improves coding speed can exacerbate this situation, potentially increasing the burden of tasks such as review, testing, and maintenance.
◆4: AI will support all tasks and engineers equally.
Research on generative AI tools and productivity improvements has yielded mixed results. Many studies report significant productivity increases, while others suggest little change or even a decline in productivity. This is because whether generative AI is useful for coding tasks depends on various factors, including the nature of the task itself and the developer's skills.
A 2024 Microsoft research report showed that AI-driven productivity improvements were greater for tasks that developers were familiar with and understood compared to those they were not. The report also indicated that software development experience, experience using AI, and problem-solving approaches also played a role. ACM Queue stated, 'Generative AI has the potential to bring significant benefits to software engineering, but its effectiveness depends on the interaction of complex factors. Task characteristics, developer experience, codebase familiarity, confidence, and prompt creation skills all influence the outcome.'
◆5: AI will increase the productivity of individual developers tenfold.
One of the myths surrounding AI and software engineering is that 'AI tools will turn individual developers into 10 times more productive developers.' While productivity may indeed increase significantly in experimental environments simulating isolated tasks, this overlooks the complexities of real-world software development. ACM Queue points out that productivity improvements measured on single tasks do not take into account the coordination, collaboration, and knowledge sharing that are essential for successful software development.
◆6: Whether or not AI can be utilized depends on individual developers.
At the time of writing, most research on AI and software engineering focuses on 'how individual developers use generative AI tools.' This approach places the responsibility for productivity improvements through generative AI on individual developers, but historically, significant productivity improvements have been brought about not at the individual level, but through systematic changes at the organizational level.
For example, the automaker Ford was a pioneer in introducing
ACM Queue stated, 'The lack of the dramatic productivity improvements many people were expecting suggests that access rights alone are not enough. To fully unlock the potential of generative AI, organizations need to rethink their software engineering systems and processes and create an environment where developers can achieve greater results in a more productive ecosystem,' arguing that organizational reform is necessary for productivity improvements.
◆7: High-performance AI tools will be automatically implemented.
Some believe that the only criterion for developers deciding whether or not to adopt AI tools is whether or not it will improve performance. However, when developers actually decide whether or not to adopt new tools, they must consider a variety of issues, including social, organizational, and cognitive barriers. There are various cases to consider, such as developers who doubt the accuracy of coding AI in the first place, developers who do not use it because it cannot be integrated into existing workflows, developers who do not adopt it for fear of a decline in their skills as engineers, and developers who are negative towards AI for ethical or environmental concerns.
Furthermore, research has shown that using AI in the workplace can lead to harsher evaluations from others, and these factors may also hinder the adoption of AI tools.
Research suggests that using AI makes you more susceptible to harsh criticism from others - GIGAZINE
◆8: Generative AI enables companies to innovate at the speed of startups.
The emergence of generative AI has led to an increase in cases where small startups are releasing new products at an astonishing pace, raising the question: 'Why can't large corporations release new products at the same speed, even though they're also using generative AI tools?' ACM Queue answers this question by stating that there are structural differences between startups and large corporations.
Startups typically build their products on open-source components and widely documented frameworks, making them a good fit for coding AI that incorporates these as training data. Large corporations, on the other hand, rely on proprietary and legacy code that isn't used for AI training, and are constrained by compliance, security, privacy, and regulatory requirements that are less of a concern for startups.
Furthermore, generative AI performs best in new development environments free from the constraints of existing tools. In contrast, commercial software for large enterprises needs to maintain backward compatibility and seamlessly integrate with numerous internal systems and third-party tools, meaning the purpose of software development is fundamentally different. ACM Queue stated, 'AI can help accelerate the progress of both startups and large enterprises, but structural realities mean they will never operate under the same conditions. Speed is visible, but complexity is invisible.'
Related Posts:
