The PRECISION attribute applies to decimal and binary data as follows:
- The precision of a fixed-point data item is the total number of decimal or binary digits used to represent a value.
- The precision of a floating-point data item is the number of decimal or binary digits in the mantissa of the floating-point representation.
The scale of fixed-point data is the number of digits to the right of the decimal or binary point. Floating-point variables do not have a scale factor. In this manual, the letter p is used to indicate precision, and the letter q is used to indicate the scale factor.
You can specify both precision and scale in a declaration. For example:
DECLARE x FIXED DECIMAL(10,3) INITIAL(1.234);
|
This example indicates that the value of x has 10 decimal digits and that 3 of those are fractional. When a value is assigned to the variable, its internal representation does not include the decimal point; the previous value for x is stored as 1234, and the decimal point is inserted when the number is output. The scale factor has the effect of multiplying the internal representation of the decimal number by a factor of 10
-q (where q is the absolute value of the specified scale).
The ranges of values you can specify for the precision for each arithmetic data type, and the defaults applied if you do not specify a precision, are summarized as follows:
Data Type
Attributes |
Precision | Scale
Factor | Default Precision |
|---|
| BINARY FIXED | 1 <= p <= 31 | p >= q >= -31 | 31 |
| BINARY FLOAT (OpenVMS VAX systems) | 1 <= p <= 113 | - | 24 |
| BINARY FLOAT (OpenVMS Alpha systems) | 1 <= p <= 53 | - | 24 |
| DECIMAL FIXED | 1 <= p <= 31 | p >= q >= 0 | 10 |
| DECIMAL FLOAT (OpenVMS VAX systems) | 1 <= p <= 34 | - | 7 |
| DECIMAL FLOAT (OpenVMS Alpha systems) | 1 <= p <= 15 | - | 7 |
If no scale factor is specified for fixed-point data, the default is 0.
For fixed-point binary data, the scale factor must be within the range -31 through 31 and less than or equal to the specified precision. Positive scale factors for fixed binary numbers function according to the same principles as those for fixed decimal. That is, a positive scale factor is similar to multiplying the internal representation binary number by a factor of 2
-q.
A negative scale factor indicates that the number of fractional bits are shifted in the opposite direction. In effect, this is similar to multiplying the binary number by a factor of 2
q, where
q is the absolute value of the specified scale. For example:
DECLARE (A,B) FIXED BINARY(31,-3),
(C,D) FIXED BINARY(31,3);
A = 128; /* output = 128 */
B = 7; /* output = 0 */
C = 128; /* output = 128.0 */
D = 7; /* output = 7.0 */
PUT SKIP LIST (A,B,C,D);
END;
|
Internally, binary numbers undergo an implicit conversion and are represented as powers of 2. For instance, in the previous example variable A is first divided by 2
3 because it is declared with a scale factor of -3. The stored number is 16. On output, the number 16 is multiplied by 2
3 and the number is again 128. However, when variable B is first divided by 2
3, the result is 0, which is the value of the stored number. Therefore, on output, 0 is multiplied by 2
3 and the output is 0.
Integer variables declared in the previous example with a positive scale factor are output as they were input, but they are followed on the right with a decimal point and a 0.
Even though arithmetic operands can be of different arithmetic types, all operations must be performed on objects of the same type. Consequently, the compiler can convert operands to a derived type, as previously shown. Therefore, when you declare a fixed binary number with a scale factor and assign it a decimal value, the results may not be as you expect because the binary scale factor left-shifts the specified number of bits to the right of the decimal point. During conversion to a decimal representation, the difference between the resulting binary number and its decimal representation is not the equivalent of dividing or multiplying the decimal number by 10. Instead, the binary number is first converted to its internal representation and then this representation is converted to its decimal representation.
When excess fractional digits are truncated, no condition is signaled. If there is any resulting loss of precision, it may be difficult to detect because truncated fractional digits do not signal a condition.
For example:
A: PROCEDURE OPTIONS (MAIN);
DECLARE A FIXED BIN (31,3),
B DECIMAL (10,5),
C DECIMAL (10,5);
A = .3;
B = 34.8;
C = MULTIPLY(A,B,10,5);
PUT SKIP LIST (A,B,C);
END;
|
Before the multiplication is performed, the variables are converted to fixed binary so that the operands share a common data type. However, after conversion, variable A is output as 0.2 rather than 0.3. The output from the previous example is:
If variable A was declared with the attributes FIXED DECIMAL (10,5), the output will be:
0.30000 34.80000 10.44000
|
