Open Collections

Basha, Manaal Ramadan

University of British Columbia

AI-based code generation tools have been revolutionizing the software development process and industry over the last couple of years. With many uncertain of what the industry will look like in the near future due to this automation, understanding user perceptions and experiences of Large Language Models (LLMs) and Code Generation Tools (CGTs) becomes imperative. In this thesis, I comprehensively investigate user sentiments and experiences with code generation tools from Twitter, Stack Overflow, and surveys. Our goals encompass identifying key themes and patterns of discussion on Twitter and Stack Overflow, determining common user questions and challenges, and providing recommendations for improving CGTs. Additionally, I explore the impact of CGTs on developers' programming abilities and their subjective experiences, including reasons for continued use through surveys. Leveraging NLP techniques such as BERTopic and RoBERTa for sentiment and topic modeling, I conduct thematic analysis through a multi-phased approach with a specific taxonomy developed using Stack Overflow. I use Twitter to understand general user opinions, Stack Overflow to extract developer experiences, and surveys to gain insights into intermediate programmers' perceptions. Through this comprehensive methodology, I aim to offer valuable insights into user perceptions and experiences with CGTs and inform future developments in this field. Our findings reveal overall positivity with geographic variations in sentiment and intermediate users intending to continue their use of CGTs. Primary sources of negativity revolve around difficulty interpreting tool output, prompting, integration issues, and concerns regarding legal and ethical implications. I propose actionable recommendations aimed at improving the usability of these tools and effectively mitigating potential risks associated with their usage. This thesis contributes to the field by offering a detailed analysis of user perceptions, identifying key challenges, and providing actionable insights for the future development of CGTs.

Text

2025-07-11

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/88725

Computer Science

University of British Columbia

UBCO

http://creativecommons.org/licenses/by-nc-nd/4.0/