Bug 1465619 - Part 14. Add gtests for AnimationFrameRecyclingQueue. r=tnikkel Differential Revision: https://phabricator.services.mozilla.com/D7519

Bug 1465619 - Part 13. Add gtests for AnimationFrameDiscardingQueue. r=tnikkel Differential Revision: https://phabricator.services.mozilla.com/D7518

Bug 1465619 - Part 12. Add gtests for AnimationFrameRetainedBuffer. r=tnikkel Differential Revision: https://phabricator.services.mozilla.com/D7517

Bug 1465619 - Part 11. Add support for recycling animated image frames. r=tnikkel This is what we have been working towards in all of the previous parts in the series. This subclasses AnimationFrameDiscardingQueue to save the discarded frames for recycling by the decoder, if the frame is marked as supporting recycling. Differential Revision: https://phabricator.services.mozilla.com/D7516

Bug 1465619 - Part 10. Re-add support for discarding animated image frames. r=tnikkel AnimatedFrameDiscardingQueue subclasses AnimationFrameBuffer to allow a cleaner abstraction over the behaviour change when we cross the threshold of too high a memory footprint for an animated image. The next patch will build on top of this to provide an abstraction to reuse the discarded frames. Differential Revision: https://phabricator.services.mozilla.com/D7515

Bug 1465619 - Part 9. Use redesigned AnimationFrameBuffer interface and retaining buffer. r=tnikkel This patch makes AnimationSurfaceProvider use the new abstractions provided by AnimationFrameBuffer and AnimationFrameRetainedBuffer to provide storage and lifetime management of decoders and the produced frames. We initially start out with an implementation that will just keep every frame forever, like our historical behaviour. The next patch will add support for discarding. Differential Revision: https://phabricator.services.mozilla.com/D7514

Bug 1465619 - Part 8. Remove the old AnimationFrameBuffer implementation. r=tnikkel In the next few patches, AnimationFrameBuffer will be reworked to split the discarding behaviour from the retaining behaviour. Once implemented as separate classes, it will allow easier reuse of the discarding code for recycling. Differential Revision: https://phabricator.services.mozilla.com/D7513

Bug 1465619 - Part 7. Add support for recycling to image::Decoder. r=tnikkel The owner for the decoder may implement IDecoderFrameRecycler to allow the decoder to request a recycled frame instead of allocating a new one. If none are available, it will fallback to allocating a new frame. Not only may the IDecoderFrameRecycler not have any frames available for recycling, the recycled frame itself may still be in use by other entities outside of imagelib. Additionally it may still be required by BlendAnimationFilter to restore the previous frame's data. It may even be the same frame as to restore. In the worst case, we will simply choose to allocate an entirely new frame, just like before. When we allocate a new frame, that means the old frame we tried to recycle will be taken out of circulation and not reused again, regardless of why it failed. Differential Revision: https://phabricator.services.mozilla.com/D7512

Bug 1465619 - Part 6. Add support for recycling to imgFrame. r=tnikkel Beyond the necessary reinitialization methods, we need to protect ourselves from recycling a frame that some other entity in the browser is still using. Generally speaking the animated surface will only be used in imgFrame::Draw since we don't layerize animated images, which will be safe. However with OMTP or blob recordings, we could retain a reference to the surface outside the current stack context. Additional if something calls RasterImage::GetImageContainer(AtSize) or RasterImage::GetFrame(AtSize), it may also have a reference to the surface for an indetermine period of time. As such, if an imgFrame is a candidate for recycling, it will wrap imgFrame::mLockedSurface in a RecyclingSourceSurface. Its job is to track how many consumers there are still of the surface, so that after we advance the animation, the decoder will know if there are still outstanding consumers. If the surface is still in use, it will block for a finite period of time (the refresh interval) before giving up on reclaiming the surface, and will allocate a new surface. The old surface can then remain in circulation for as long as necessary without further blocking the animation progression, since we stop recycling that surface/imgFrame. Differential Revision: https://phabricator.services.mozilla.com/D7511

Bug 1465619 - Part 5. Move actual drawing in imgFrame::Draw outside the monitor. r=tnikkel Since imgFrame::Draw will limit the drawing to only look at pixels that we have written to and posted an invalidation for, there is no need to hold the monitor while doing so. By taking the most expensive operation outside the lock, we will minimize our chances of hitting contention with the decoder thread. A later part in this series will require that a surface be freed outside the lock because it may end up reacquiring it. In addition to the contention win, this change should facilitate that. Differential Revision: https://phabricator.services.mozilla.com/D7510

Bug 1465619 - Part 4. Move the first frame refresh area calculation to frame commit. r=tnikkel If we discard a frame during decoding, e.g. due to an error, then we don't want to take that frame into account for the first frame refresh area. We should also be handling partial frames here (the dirty rect needs to encompass the rows that did not get written with actual pixel data). The only place that can have the necessary information is at the end rather than at the beginning. Differential Revision: https://phabricator.services.mozilla.com/D7509

Bug 1465619 - Part 3. Improve the clear rect calculations to take into account recycling in BlendAnimationFilter. r=tnikkel The clear rect and the recycle rect can overlap, and depending on the size of the clear rect, it could be a significant amount of data that needs to be copied from the restore frame. This patch minimizes the copying for a row which contains both the recycle rect and the clear rect. Differential Revision: https://phabricator.services.mozilla.com/D7508

Bug 1465619 - Part 2. Add basic support for recycling a frame buffer to BlendAnimationFilter. r=tnikkel Given an invalidation rect, called the recycle rect, which represents the area which may have changed between the current frame and the frame we are recycling, we can not only reuse the buffer itself to avoid an allocation and free, we can also avoid copying pixel data from the restore frame which is already set. Differential Revision: https://phabricator.services.mozilla.com/D7507

Bug 1465619 - Part 1. Use imgFrame directly instead of RawAccessFrameRef in FrameAnimator. r=tnikkel When blending full frames off the main thread, FrameAnimator no longer requires access to the raw data of the frame to advance the animation. Now we only request a RawAccessFrameRef for the current/next frames when we have discovered that we need to do blending on the main thread. In addition to avoiding the mutex overhead of RawAccessFrameRef, this will also facilitate potentially optimizing the surfaces for the DrawTarget for individual animated image frames. Differential Revision: https://phabricator.services.mozilla.com/D7506

Bug 1466707 - Ensure that animated images always honour the animated image mode. r=tnikkel Animated images inside of SVGs and used in XUL frames did not get the configured animation image mode from the pres context. Differential Revision: https://phabricator.services.mozilla.com/D8586

Bug 1492930 - Part 6. Integrate extra shared surface memory reporting with imagelib. r=tnikkel

Bug 1492930 - Part 4. Add ImageMemoryReporter to support extra shared surfaces reporting. r=tnikkel By delegating responsibility for shared surfaces reporting to imagelib, we can cross reference the GPU shared surfaces cache with the local surface cache in a content process (or the main process). This will allow us to identify entries that are in the GPU cache but not in the content/main process cache, and aid in debugging memory leaks. This functionality is pref'd off by default behind image.mem.debug-reporting. Additionally, we want to report every entry that was mapped into the compositor process, in the compositor process memory report. This will give us a sense of how much of our resident memory is consumed by mapped images in absence of the more detailed cross referencing above.

Bug 1492930 - Part 3. Expose all frames to image memory reporting. r=tnikkel At present, surface providers roll up all of their individual surfaces into a single reporting unit. Specifically this means animated image frames are all reported as a block. This patch removes that consolidation and reports every frame as its own SurfaceMemoryReport. This is important because each frame may have its own external image ID, and we want to cross reference that with what we expect from the GPU shared surfaces cache.

Bug 1492930 - Part 6. Integrate extra shared surface memory reporting with imagelib. r=tnikkel

Bug 1492930 - Part 4. Add ImageMemoryReporter to support extra shared surfaces reporting. r=tnikkel By delegating responsibility for shared surfaces reporting to imagelib, we can cross reference the GPU shared surfaces cache with the local surface cache in a content process (or the main process). This will allow us to identify entries that are in the GPU cache but not in the content/main process cache, and aid in debugging memory leaks. This functionality is pref'd off by default behind image.mem.debug-reporting. Additionally, we want to report every entry that was mapped into the compositor process, in the compositor process memory report. This will give us a sense of how much of our resident memory is consumed by mapped images in absence of the more detailed cross referencing above.

Bug 1492930 - Part 3. Expose all frames to image memory reporting. r=tnikkel At present, surface providers roll up all of their individual surfaces into a single reporting unit. Specifically this means animated image frames are all reported as a block. This patch removes that consolidation and reports every frame as its own SurfaceMemoryReport. This is important because each frame may have its own external image ID, and we want to cross reference that with what we expect from the GPU shared surfaces cache.

Bug 1456558 - Part 3. Implement factor of 2 scaling support for SVGs in VectorImage. r=tnikkel If FLAG_HIGH_QUALITY_SCALING is used, we should use SurfaceCache::LookupBestMatch just like how it is done in RasterImage. This may provide an alternative size at which we should rasterize the SVG instead of the requested size. Since SurfaceCache imposes a maximum size for which it will permit rasterized SVGs, we should also bypass the cache entirely if we are well above that and simply draw directly to the draw target in such cases. With WebRender, it is somewhat more complicated. We will now return NOT_SUPPORTED if the size is too big, and this should trigger fallback to blob images. This should only produce drawing commands for the relevant region and save us the high cost of rasterized a very large surface on the main thread, which at the same time, looking as crisp as a user would expect.

Bug 1456558 - Part 2. Implement factor of 2 scaling support for SVGs in the surface cache. r=tnikkel There is one main difference between raster images and vector images with respect to factor of 2 scaling. Vector images may be scaled infinitely and so we need to extend factor of 2 scaling to permit growing instead of just shrinking. Also, we don't want to scale infinitely, so we should configure a maximum size limit. This size limit will apply even outside of factor of 2 scaling, and so the caller (VectorImage) will need to be careful to take this into account.

Bug 1456558 - Part 1. Move the allowed raster size calculation from imgFrame to SurfaceCache. r=tnikkel

Bug 1456558 - Part 3. Implement factor of 2 scaling support for SVGs in VectorImage. r=tnikkel If FLAG_HIGH_QUALITY_SCALING is used, we should use SurfaceCache::LookupBestMatch just like how it is done in RasterImage. This may provide an alternative size at which we should rasterize the SVG instead of the requested size. Since SurfaceCache imposes a maximum size for which it will permit rasterized SVGs, we should also bypass the cache entirely if we are well above that and simply draw directly to the draw target in such cases. With WebRender, it is somewhat more complicated. We will now return NOT_SUPPORTED if the size is too big, and this should trigger fallback to blob images. This should only produce drawing commands for the relevant region and save us the high cost of rasterized a very large surface on the main thread, which at the same time, looking as crisp as a user would expect.

Bug 1456558 - Part 2. Implement factor of 2 scaling support for SVGs in the surface cache. r=tnikkel There is one main difference between raster images and vector images with respect to factor of 2 scaling. Vector images may be scaled infinitely and so we need to extend factor of 2 scaling to permit growing instead of just shrinking. Also, we don't want to scale infinitely, so we should configure a maximum size limit. This size limit will apply even outside of factor of 2 scaling, and so the caller (VectorImage) will need to be careful to take this into account.

Bug 1456558 - Part 1. Move the allowed raster size calculation from imgFrame to SurfaceCache. r=tnikkel

Bug 1337111 - Part 6. Use shared surfaces for full animated frames for WebRender. r=tnikkel

Bug 1337111 - Part 5. Add pref to force decoding of full frames, disabled by default. r=tnikkel

Bug 1337111 - Part 4. Add gtests for BlendAnimationFilter. r=tnikkel

Bug 1337111 - Part 3. Land groundwork for new blended animation gtests. r=tnikkel

Bug 1337111 - Part 2. Add an image decoder flag to request complete frames. r=tnikkel DecoderFlags::BLEND_ANIMATION will cause the decoder to inject the BlendAnimationFilter from the previous patch into the SurfacePipe filter chain. All frames produced by this decoder will be complete, and should be equivalent to the result outputted by FrameAnimator.

Bug 1337111 - Part 1. Add BlendAnimationFilter to allow decoders to generate complete frames. r=tnikkel This new SurfaceFilter can be added to a SurfacePipe to perform the blending of a previous frame with the current partial frame, for an animated image. This functionality is currently provided by FrameAnimator and must be performed each time we want to advance the displayed frame, all on the main thread. Moving this to SurfacePipe allows us to do the same operation once per frame decode, and on a decoder thread. This should reduce the cost of a refresh tick since advancing animated images is reduced to merely checking if the frame is available. Also, if the image is below the discard frames threshold (to save memory), then we will also save CPU due to only blending once at decode.

Bug 1337111 - Part 6. Use shared surfaces for full animated frames for WebRender. r=tnikkel

Bug 1337111 - Part 5. Add pref to force decoding of full frames, disabled by default. r=tnikkel

Bug 1337111 - Part 4. Add gtests for BlendAnimationFilter. r=tnikkel

Bug 1337111 - Part 3. Land groundwork for new blended animation gtests. r=tnikkel

Bug 1337111 - Part 2. Add an image decoder flag to request complete frames. r=tnikkel DecoderFlags::BLEND_ANIMATION will cause the decoder to inject the BlendAnimationFilter from the previous patch into the SurfacePipe filter chain. All frames produced by this decoder will be complete, and should be equivalent to the result outputted by FrameAnimator.

Bug 1337111 - Part 1. Add BlendAnimationFilter to allow decoders to generate complete frames. r=tnikkel This new SurfaceFilter can be added to a SurfacePipe to perform the blending of a previous frame with the current partial frame, for an animated image. This functionality is currently provided by FrameAnimator and must be performed each time we want to advance the displayed frame, all on the main thread. Moving this to SurfacePipe allows us to do the same operation once per frame decode, and on a decoder thread. This should reduce the cost of a refresh tick since advancing animated images is reduced to merely checking if the frame is available. Also, if the image is below the discard frames threshold (to save memory), then we will also save CPU due to only blending once at decode.

Bug 1446309 - Part 3. Properly handle ImgDrawResult for WebRender display list generation. r=tnikkel When generating display lists for WebRender, we were not caching the draw result via nsDisplayItemGenericImageGeometry::UpdateDrawResult (or similar) after completing CreateWebRenderCommands. This is important because reftests use this to force sync decoding for images; it may be a reason for image-related intermittent failures on *-qr builds. Additionally, we may have been requesting fallback in cases where fallback could not do anything more than WebRender could. For example, if we can't get an image container yet, there is no point in requesting fallback because it might just be we haven't started decoding yet. We should just return the actual draw result in such cases.

Bug 1446309 - Part 2. Make nsDisplayImage fallback to the previous image to avoid flickering. r=tnikkel When the current image for an nsImageFrame/nsDisplayImage is not yet ready, we display the previous image, if any available, to avoid flickering while we wait for decoding to finish. On the WebRender path, this functionality was lost since we did not have the draw result information with image containers. With the API updated in part 1, we can now do this to avoid flickering.

Bug 1446309 - Part 1. Return draw result from imgIContainer::GetImageContainerAtSize. r=tnikkel In addition to the image container, the draw result can also be useful for callers to know whether or not the surface(s) in the container are fully decoded or not. This is used in subsequent parts to avoid flickering in some cases.

Bug 1486662 - Allow scroll without scrollbar for non-textcontrol elements on Android. r=tnikkel Differential Revision: https://phabricator.services.mozilla.com/D5115

Bug 1445570 - Remove EnsureEventualAfterPaint timer. r=tnikkel MozReview-Commit-ID: C7WICJ5Q0ES Differential Revision: https://phabricator.services.mozilla.com/D5005

Bug 1471583. Add a new ImgDrawResult variant to distinguish INCOMPLETE and 'fully drew an image which wasn't really complete'. r=tnikkel

Bug 1446309 - Part 2. Make nsDisplayImage fallback to the previous image to avoid flickering. r=tnikkel When the current image for an nsImageFrame/nsDisplayImage is not yet ready, we display the previous image, if any available, to avoid flickering while we wait for decoding to finish. On the WebRender path, this functionality was lost since we did not have the draw result information with image containers. With the API updated in part 1, we can now do this to avoid flickering.

Bug 1446309 - Part 1. Return draw result from imgIContainer::GetImageContainerAtSize. r=tnikkel In addition to the image container, the draw result can also be useful for callers to know whether or not the surface(s) in the container are fully decoded or not. This is used in subsequent parts to avoid flickering in some cases.

Bug 1416328 - Part 3. Enable img decoding attribute web-platform-tests. r=tnikkel

Bug 1416328 - Part 2. Expose decoding attribute for img elements. r=bz,tnikkel This adds support for HTMLImageElement's decoding attribute, as described by: https://github.com/whatwg/html/pull/3221 https://whatpr.org/html/3221/images.html#decoding-images It also exposes the same attribute on SVGImageElement, just as Blink has chosen to do so.

Bug 1416328 - Part 1. Add configurable synchronous decoding hint to nsImageLoadingContent. r=tnikkel The next part in this series depends on this to implement the HTMLImageElement's decoding attribute. This functionality allows the caller to force an nsImageLoadingContent and its associated nsImageFrame and nsSVGImageFrame objects to synchronously decode an image at draw time. It will only synchronously decode if it has completed metadata decoding (i.e. knows its size) and has received all of the encoded data; this mirrors the load event criteria for an image.