Bug 1534156 - Get around int32_t to float rounding error. r=bryce
authorJean-Yves Avenard <jyavenard@mozilla.com>
Wed, 01 May 2019 23:11:56 +0000
changeset 531014 bcb48454bcae0d274f5b238dfb254d64897b5344
parent 531013 5acce9aaceb96cd5b8c0c9fde26286752e8cae29
child 531015 5abd778efd7aa8a03be09094bc5d0e852856552f
push id11265
push userffxbld-merge
push dateMon, 13 May 2019 10:53:39 +0000
treeherdermozilla-beta@77e0fe8dbdd3 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersbryce
bugs1534156
milestone68.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1534156 - Get around int32_t to float rounding error. r=bryce float(INT32_MAX); gets compiled into 2.14748365E+9 using clang, which is slightly bigger than INT32_MAX, as such 2.14748365E+9 <= INT32_MAX will return true (as INT32_MAX gets converted to a float) Differential Revision: https://phabricator.services.mozilla.com/D29464
dom/media/platforms/agnostic/bytestreams/H264.cpp
--- a/dom/media/platforms/agnostic/bytestreams/H264.cpp
+++ b/dom/media/platforms/agnostic/bytestreams/H264.cpp
@@ -529,17 +529,19 @@ class SPSNALIterator {
     }
     rbsp->AppendElement(val);
   }
   return rbsp.forget();
 }
 
 static int32_t ConditionDimension(float aValue) {
   // This will exclude NaNs and too-big values.
-  if (aValue > 1.0 && aValue <= INT32_MAX) return int32_t(aValue);
+  if (aValue > 1.0 && aValue <= INT32_MAX / 2) {
+    return int32_t(aValue);
+  }
   return 0;
 }
 
 /* static */
 bool H264::DecodeSPS(const mozilla::MediaByteBuffer* aSPS, SPSData& aDest) {
   if (!aSPS) {
     return false;
   }