Browse Source

MPILIB: Fix comparison of negative MPIs

If u and v both represent negative integers and their limb counts
happen to differ, mpi_cmp will always return a positive value - this
is obviously bogus. u is smaller than v if and only if it is larger in
absolute value.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dmitry Kasatkin <dmitry.kasatkin@gmail.com>
Rasmus Villemoes 10 years ago
parent
commit
b9f918a31d
1 changed files with 1 additions and 1 deletions
  1. 1 1
      lib/mpi/mpi-cmp.c

+ 1 - 1
lib/mpi/mpi-cmp.c

@@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v)
 	if (usize != vsize && !u->sign && !v->sign)
 		return usize - vsize;
 	if (usize != vsize && u->sign && v->sign)
-		return vsize + usize;
+		return vsize - usize;
 	if (!usize)
 		return 0;
 	cmp = mpihelp_cmp(u->d, v->d, usize);