مصطفي عبدالرزاق

The comprehensibility and sustainability of software significantly depend on the quality of source code comments, which connect a developer's intent with the code's functionality. Despite their essential function, consistent and high-quality documentation is frequently neglected, resulting in difficulties in collaborative software development. Recent breakthroughs in Large Language Models (LLMs), exemplified by ChatGPT, provide potential options for automating comment production through the utilization of extensive datasets comprising programming code and natural language. This thesis assesses the efficacy of ChatGPT in producing significant, contextually relevant, and precise source code annotations. A rubric-based eva‎luation framework was established to systematically eva‎luate the quality of generated comments across four principal dimensions: relevance, correctness, readability, and contextual awareness. Human assessors utilized this rubric to assess comments produced for a dataset of Python projects, juxtaposing them with human-authored comments as a standard. The research reveals substantial deficiencies in the existing capabilities of LLM-generated comments, especially in their ability to grasp contextual subtleties and uphold consistency. ChatGPT had merits in accuracy and readability; but, its effectiveness in delivering greater contextual relevance was inadequate. A statistical analysis of the results revealed these differences and emphasized the possibility of rubric-based testing as a more sophisticated eva‎luation tool than conventional automated metrics. This research enhances eva‎luation approaches for AI tools in software documentation and offers practical insights for improving the training and development of LLMs such as ChatGPT. This work seeks to resolve the issues of automated comment generation, thereby bridging the divide between human and machine-generated software documentation, and enhancing efficiency and collaboration in software engineering .

1- ChatGPT , 2- نسل نظرات , 3- مدل هاي زبان بزرگ (LLMs). , 4- ارزيابي كيفيت

1- ChatGPT , 2- Comment Generation , 3- Large Language Models (LLMs). , 4- Quality Assessment

Mostafa Abdolrazagh

Dr. Minaeei