Browse Source

fix the last known rounding bug in floating point printing

the observed symptom was that the code was incorrectly rounding up
1.0625 to 1.063 despite the rounding mode being round-to-nearest with
ties broken by rounding to even last place. however, the code was just
not right in many respects, and i'm surprised it worked as well as it
did. this time i tested the values that end up in the variables round,
small, and the expression round+small, and all look good.
Rich Felker 14 years ago
parent
commit
8628eff912
1 changed files with 4 additions and 3 deletions
  1. 4 3
      src/stdio/vfprintf.c

+ 4 - 3
src/stdio/vfprintf.c

@@ -326,9 +326,10 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
 		if (x || d+1!=z) {
 		if (x || d+1!=z) {
 			long double round = CONCAT(0x1p,LDBL_MANT_DIG);
 			long double round = CONCAT(0x1p,LDBL_MANT_DIG);
 			long double small;
 			long double small;
-			if (x<i/2) small=0x01p-1;
-			else if (i==i/2 && d+1==z) small=0x10p-1;
-			else small=0x11p-1;
+			if (*d/i & 1) round += 2;
+			if (x<i/2) small=0x0.8p0;
+			else if (x==i/2 && d+1==z) small=0x1.0p0;
+			else small=0x1.8p0;
 			if (pl && *prefix=='-') round*=-1, small*=-1;
 			if (pl && *prefix=='-') round*=-1, small*=-1;
 			*d -= x;
 			*d -= x;
 			/* Decide whether to round by probing round+small */
 			/* Decide whether to round by probing round+small */