Bug 947337 - Ensure the transform of input to Gecko space accounts for inflight paint requests. r=botond a=1.3+
☠☠ backed out by bfdb71aa8953 ☠ ☠
authorKartikaya Gupta <kgupta@mozilla.com>
Tue, 21 Jan 2014 16:27:20 -0500
changeset 175982 00a111a492398df64623807e1b8f6d4dda42e99f
parent 175981 d7cd7c0cedd36155e737d10ffa4c690672a1d12e
child 175983 31da64bcc970f21938054aa8c73f0bd341f6f743
push id445
push userffxbld
push dateMon, 10 Mar 2014 22:05:19 +0000
treeherdermozilla-release@dc38b741b04e [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersbotond, 1
bugs947337
milestone28.0a2
Bug 947337 - Ensure the transform of input to Gecko space accounts for inflight paint requests. r=botond a=1.3+
gfx/layers/composite/APZCTreeManager.cpp
gfx/layers/ipc/AsyncPanZoomController.cpp
gfx/layers/ipc/AsyncPanZoomController.h
--- a/gfx/layers/composite/APZCTreeManager.cpp
+++ b/gfx/layers/composite/APZCTreeManager.cpp
@@ -828,17 +828,17 @@ APZCTreeManager::GetAPZCAtPoint(AsyncPan
     return aApzc;
   }
   return nullptr;
 }
 
 /* This function sets the aTransformToApzcOut and aTransformToGeckoOut out-parameters
    to some useful transformations that input events may need applied. This is best
    illustrated with an example. Consider a chain of layers, L, M, N, O, P, Q, R. Layer L
-   is the layer that corresponds to the returned APZC instance, and layer R is the root
+   is the layer that corresponds to the argument |aApzc|, and layer R is the root
    of the layer tree. Layer M is the parent of L, N is the parent of M, and so on.
    When layer L is displayed to the screen by the compositor, the set of transforms that
    are applied to L are (in order from top to bottom):
 
         L's transient async transform       (hereafter referred to as transform matrix LT)
         L's nontransient async transform    (hereafter referred to as transform matrix LN)
         L's CSS transform                   (hereafter referred to as transform matrix LC)
         M's transient async transform       (hereafter referred to as transform matrix MT)
@@ -866,50 +866,65 @@ APZCTreeManager::GetAPZCAtPoint(AsyncPan
         MT.Inverse()
         LC.Inverse()
         LN.Inverse()
    This combined transformation is returned in the aTransformToApzcOut out-parameter.
 
    Next, if we want user inputs sent to gecko for event-dispatching, we will need to strip
    out all of the async transforms that are involved in this chain. This is because async
    transforms are stored only in the compositor and gecko does not account for them when
-   doing display-list-based hit-testing for event dispatching. Therefore, given a user input
-   in screen space, the following transforms need to be applied (in order from top to bottom):
+   doing display-list-based hit-testing for event dispatching.
+   Furthermore, because these input events are processed by Gecko in a FIFO queue that
+   includes other things (specifically paint requests), it is possible that by time the
+   input event reaches gecko, it will have painted something else. Therefore, we need to
+   apply another transform to the input events to account for the possible disparity between
+   what we know gecko last painted and the last paint request we sent to gecko. Let this
+   transform be represented by LD, MD, ... RD.
+   Therefore, given a user input in screen space, the following transforms need to be applied
+   (in order from top to bottom):
         RC.Inverse()
         RN.Inverse()
         RT.Inverse()
         ...
         MC.Inverse()
         MN.Inverse()
         MT.Inverse()
         LC.Inverse()
         LN.Inverse()
         LT.Inverse()
+        LD
         LC
+        MD
         MC
         ...
+        RD
         RC
    This sequence can be simplified and refactored to the following:
         aTransformToApzcOut
         LT.Inverse()
+        LD
         LC
+        MD
         MC
         ...
+        RD
         RC
    Since aTransformToApzcOut is already one of the out-parameters, we set aTransformToGeckoOut
-   to the remaining transforms (LT.Inverse() * LC * ... * RC), so that the caller code can
+   to the remaining transforms (LT.Inverse() * LD * ... * RC), so that the caller code can
    combine it with aTransformToApzcOut to get the final transform required in this case.
 
    Note that for many of these layers, there will be no AsyncPanZoomController attached, and
    so the async transform will be the identity transform. So, in the example above, if layers
-   L and P have APZC instances attached, MT, MN, NT, NN, OT, ON, QT, QN, RT and RN will be
-   identity transforms.
+   L and P have APZC instances attached, MT, MN, MD, NT, NN, ND, OT, ON, OD, QT, QN, QD, RT,
+   RN and RD will be identity transforms.
    Additionally, for space-saving purposes, each APZC instance stores its layer's individual
    CSS transform and the accumulation of CSS transforms to its parent APZC. So the APZC for
    layer L would store LC and (MC * NC * OC), and the layer P would store PC and (QC * RC).
+   The APZC instances track the last dispatched paint request and so are able to calculate LD and
+   PD using those internally stored values.
    The APZCs also obviously have LT, LN, PT, and PN, so all of the above transformation combinations
    required can be generated.
  */
 void
 APZCTreeManager::GetInputTransforms(AsyncPanZoomController *aApzc, gfx3DMatrix& aTransformToApzcOut,
                                     gfx3DMatrix& aTransformToGeckoOut)
 {
   MonitorAutoLock lock(mTreeLock);
@@ -926,35 +941,35 @@ APZCTreeManager::GetInputTransforms(Asyn
   gfx3DMatrix asyncUntransform = gfx3DMatrix(aApzc->GetCurrentAsyncTransform()).Inverse();
   // nontransientAsyncTransform is LN
   gfx3DMatrix nontransientAsyncTransform = aApzc->GetNontransientAsyncTransform();
   // transientAsyncUntransform is LT.Inverse()
   gfx3DMatrix transientAsyncUntransform = nontransientAsyncTransform * asyncUntransform;
 
   // aTransformToApzcOut is initialized to OC.Inverse() * NC.Inverse() * MC.Inverse() * LC.Inverse() * LN.Inverse()
   aTransformToApzcOut = ancestorUntransform * aApzc->GetCSSTransform().Inverse() * nontransientAsyncTransform.Inverse();
-  // aTransformToGeckoOut is initialized to LT.Inverse() * LC * MC * NC * OC
-  aTransformToGeckoOut = transientAsyncUntransform * aApzc->GetCSSTransform() * aApzc->GetAncestorTransform();
+  // aTransformToGeckoOut is initialized to LT.Inverse() * LD * LC * MC * NC * OC
+  aTransformToGeckoOut = transientAsyncUntransform * aApzc->GetTransformToLastDispatchedPaint() * aApzc->GetCSSTransform() * aApzc->GetAncestorTransform();
 
   for (AsyncPanZoomController* parent = aApzc->GetParent(); parent; parent = parent->GetParent()) {
     // ancestorUntransform is updated to RC.Inverse() * QC.Inverse() when parent == P
     ancestorUntransform = parent->GetAncestorTransform().Inverse();
     // asyncUntransform is updated to PA.Inverse() when parent == P
     asyncUntransform = gfx3DMatrix(parent->GetCurrentAsyncTransform()).Inverse();
     // untransformSinceLastApzc is RC.Inverse() * QC.Inverse() * PC.Inverse() * PA.Inverse()
     gfx3DMatrix untransformSinceLastApzc = ancestorUntransform * parent->GetCSSTransform().Inverse() * asyncUntransform;
 
     // aTransformToApzcOut is RC.Inverse() * QC.Inverse() * PC.Inverse() * PA.Inverse() * OC.Inverse() * NC.Inverse() * MC.Inverse() * LC.Inverse() * LN.Inverse()
     aTransformToApzcOut = untransformSinceLastApzc * aTransformToApzcOut;
-    // aTransformToGeckoOut is LT.Inverse() * LC * MC * NC * OC * PC * QC * RC
-    aTransformToGeckoOut = aTransformToGeckoOut * parent->GetCSSTransform() * parent->GetAncestorTransform();
+    // aTransformToGeckoOut is LT.Inverse() * LD * LC * MC * NC * OC * PD * PC * QC * RC
+    aTransformToGeckoOut = aTransformToGeckoOut * parent->GetTransformToLastDispatchedPaint() * parent->GetCSSTransform() * parent->GetAncestorTransform();
 
     // The above values for aTransformToApzcOut and aTransformToGeckoOut when parent == P match
-    // the required output as explained in the comment above GetTargetAPZC. Note that any missing terms
-    // are async transforms that are guaranteed to be identity transforms.
+    // the required output as explained in the comment above this method. Note that any missing
+    // terms are guaranteed to be identity transforms.
   }
 }
 
 already_AddRefed<AsyncPanZoomController>
 APZCTreeManager::CommonAncestor(AsyncPanZoomController* aApzc1, AsyncPanZoomController* aApzc2)
 {
   MonitorAutoLock lock(mTreeLock);
   nsRefPtr<AsyncPanZoomController> ancestor;
--- a/gfx/layers/ipc/AsyncPanZoomController.cpp
+++ b/gfx/layers/ipc/AsyncPanZoomController.cpp
@@ -1378,16 +1378,24 @@ ViewTransform AsyncPanZoomController::Ge
 
 gfx3DMatrix AsyncPanZoomController::GetNontransientAsyncTransform() {
   ReentrantMonitorAutoEnter lock(mMonitor);
   return gfx3DMatrix::ScalingMatrix(mLastContentPaintMetrics.mResolution.scale,
                                     mLastContentPaintMetrics.mResolution.scale,
                                     1.0f);
 }
 
+gfx3DMatrix AsyncPanZoomController::GetTransformToLastDispatchedPaint() {
+  ReentrantMonitorAutoEnter lock(mMonitor);
+  CSSPoint scrollChange = mLastContentPaintMetrics.mScrollOffset - mLastDispatchedPaintMetrics.mScrollOffset;
+  float zoomChange = mLastContentPaintMetrics.mZoom.scale / mLastDispatchedPaintMetrics.mZoom.scale;
+  return gfx3DMatrix::Translation(scrollChange.x, scrollChange.y, 0) *
+         gfx3DMatrix::ScalingMatrix(zoomChange, zoomChange, 1);
+}
+
 void AsyncPanZoomController::NotifyLayersUpdated(const FrameMetrics& aLayerMetrics, bool aIsFirstPaint) {
   ReentrantMonitorAutoEnter lock(mMonitor);
 
   mLastContentPaintMetrics = aLayerMetrics;
 
   bool isDefault = mFrameMetrics.IsDefault();
   mFrameMetrics.mMayHaveTouchListeners = aLayerMetrics.mMayHaveTouchListeners;
   APZC_LOG_FM(aLayerMetrics, "%p got a NotifyLayersUpdated with aIsFirstPaint=%d", this, aIsFirstPaint);
--- a/gfx/layers/ipc/AsyncPanZoomController.h
+++ b/gfx/layers/ipc/AsyncPanZoomController.h
@@ -200,16 +200,25 @@ public:
   /**
    * Returns the part of the async transform that will remain once Gecko does a
    * repaint at the desired metrics. That is, in the steady state:
    * gfx3DMatrix(GetCurrentAsyncTransform()) === GetNontransientAsyncTransform()
    */
   gfx3DMatrix GetNontransientAsyncTransform();
 
   /**
+   * Returns the transform to take something from the coordinate space of the
+   * last thing we know gecko painted, to the coordinate space of the last thing
+   * we asked gecko to paint. In cases where that last request has not yet been
+   * processed, this is needed to transform input events properly into a space
+   * gecko will understand.
+   */
+  gfx3DMatrix GetTransformToLastDispatchedPaint();
+
+  /**
    * Recalculates the displayport. Ideally, this should paint an area bigger
    * than the composite-to dimensions so that when you scroll down, you don't
    * checkerboard immediately. This includes a bunch of logic, including
    * algorithms to bias painting in the direction of the velocity.
    */
   static const CSSRect CalculatePendingDisplayPort(
     const FrameMetrics& aFrameMetrics,
     const ScreenPoint& aVelocity,