basic.precision

defines the number of fractional decimal places that a numerical value can hold.

The default precision is four. 'precision' may be set in the range of 0 to 9. Numbers with more decimal places than the precision are truncated on R83 and AP.

Only one 'precision' statement is allowed in a program and the 'precision' statement must precede the use of any numeric data. Programs calling subroutines or entering other programs must have the same precision. (See 'call' and 'enter'). If the precisions do not match, the program aborts into the debugger. This restriction also applies to main programs which share data in named common space. However, violations in this case are not reported, and values of named common variables will be incorrect.

For certain 'arithmetic functions', higher 'precision' limits the magnitude of the values returned. In R83, where the maximum precision is 6, exceeding the limits of 'precision' results in a negative number and invalidates the results of the calculation.

In AP, most functions have no limitation on their numeric range.

'precision' in the range of 1 to 9 handles numbers as 48-bit scaled binary numbers.

If the result of an 'arithmetic.expression' which divides (/), multiplies (*), takes a remainder (\), or produces an exponent (^) exceeds the maximum magnitude of the 48-bit representation, the system creates an internal variable type which uses a precision of 18 digits to the right of the decimal point, and unlimited to the left. This feature assures maximum accuracy.

Operations which exceed the limits of the current 'precision', round the results rather than truncate; (99999/100000 with 'precision 4' results in a 1).

A 'precision' of 0 forces 48-bit integer arithmetic for all operations. (99999/100000 results in a zero).