UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Rounding errors in digital computer arithmetic subroutines. Lastman, Gary Joseph

Abstract

In this thesis we investigate arithmetic subroutines and round-off procedures. An error analysis of single operations in normalized floating point arithmetic leads us to the construction of an improved form of addition-subtraction subroutine. In addition the properties of several types of round-off procedures are examined (adding ½; adding random digits; dropping digits). The experimental work (using the above mentioned subroutines) with the system x• = y, y• = -x shows that the systematic accumulation of round-off error observed by Huskey is due to the type of rounding-off procedure used. Furthermore, Hartree's explanation of this effect is found to be inadequate because carrying extra digits throughout the calculations does not eliminate the systematic round-off.

Item Media

Item Citations and Data

Rights

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

Usage Statistics