Open Collections

Abstract

Large language models have emerged as promising tools for automated test generation, yet their effectiveness compared to systematic testing approaches remains empirically unvalidated for mathematical algorithms. This thesis investigates whether LLMs can generate unit tests as effective as systematic enumeration for piecewise function approximation algorithms — a fundamental class of algorithms requiring both tolerance satisfaction and segment minimization. We develop a comparative testing framework evaluating seven state-of-the-art language models against systematic exhaustive testing. The framework employs provably optimal algorithms as ground-truth oracles, enabling definitive identification of algorithmic failures across both tolerance violations and suboptimal solutions. Systematic generation produces 75 billion test cases through bounded parameter space enumeration, while LLM-based generation utilizes varied prompting strategies across multiple contemporary models. GPU acceleration and JIT compilation achieve computational feasibility, reducing ten-billion-scale evaluation from years to hours. We evaluate 14 candidate algorithms spanning diverse paradigms across piecewise constant and piecewise linear approximation problems. The comparative evaluation reveals substantial differences in failure detection effectiveness between systematic and LLM-based test generation. These findings establish empirical evidence regarding LLM capabilities and limitations for unit test generation in mathematical algorithm domains, informing practical decisions about appropriate deployment of AI-assisted testing methodologies.