UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Rounding errors in digital computer arithmetic subroutines. Lastman, Gary Joseph


In this thesis we investigate arithmetic subroutines and round-off procedures. An error analysis of single operations in normalized floating point arithmetic leads us to the construction of an improved form of addition-subtraction subroutine. In addition the properties of several types of round-off procedures are examined (adding ½; adding random digits; dropping digits). The experimental work (using the above mentioned subroutines) with the system x• = y, y• = -x shows that the systematic accumulation of round-off error observed by Huskey is due to the type of rounding-off procedure used. Furthermore, Hartree's explanation of this effect is found to be inadequate because carrying extra digits throughout the calculations does not eliminate the systematic round-off.

Item Media

Item Citations and Data


For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.