gfx/doc/MozSurface.md
author Ted Mielczarek <ted@mielczarek.org>
Fri, 06 Jan 2017 05:57:10 -0500
changeset 328603 dedf4f49b3756b09986e78dc5519a291b81f133f
parent 187201 f74d791a1904c1af513ed71d1d943333a07c6541
child 434376 21a1fadfbe5aa5f1e0f5004c071a5d1163e058ff
permissions -rw-r--r--
bug 1329320 - replace makecab with rust-makecab in symbolstore. r=gps It turns out that running makecab to compress PDB files takes a significant amount of time in the buildsymbols step. I wrote an implementation of makecab in Rust that implements only the subset of features we use and it's significantly faster: https://github.com/luser/rust-makecab This patch adds a makecab check to moz.configure, adds a release build of the makecab binary to the Windows tooltool manifests, points the build at it from mozconfig.win-common, and changes symbolstore.py to use MAKECAB from substs instead of calling `makecab.exe` directly. MozReview-Commit-ID: 76FHLIZFCXS

MozSurface {#mozsurface}
==========

**This document is work in progress.  Some information may be missing or incomplete.**

## Goals

We need to be able to safely and efficiently render web content into surfaces that may be shared accross processes.
MozSurface is a cross-process and backend-independent Surface API and not a stream API.

## Owner

Nicolas Silva

## Definitions

## Use cases

Drawing web content into a surface and share it with the compositor process to display it on the screen without copies.

## Requirement

* It must be possible to efficiently share a MozSurface with a separate thread or process through IPDL
* It must be possible to obtain read access a MozSurface on both the client and the host side at the same time.
* The creation, update and destrution of surfaces must be safe and race-free. In particular, the ownership of the shared data must be clearly defined.
* MozSurface must be a cross-backend/cross-platform abstraction that we will use on all of the supported platforms.
* It must be possible to efficiently draw into a MozSurface using Moz2D.
* While it should be possible to share MozSurfaces accross processes, it should not be limited to that. MozSurface should also be the preferred abstraction for use with surfaces that are not shared with the compositor process.

## TextureClient and TextureHost

TextureClient and TextureHost are the closest abstractions we currently have to MozSurface. The current plan is to evolve TextureClient into MozSurface. In its current state, TextureClient doesn't meet all the requirements and desisgn decisions of MozSurface yet.

In particular, TextureClient/TextureHost are designed around cross-process sharing specifically. See the SharedMozSurface design document for more information about TextureClient and TextureHost.

## Locking semantics

In order to access the shared surface data users of MozSurface must acquire and release a lock on the surface, specifying the open mode (read/write/read+write).

    bool Lock(OpenMode aMode);
    void Unlock();

This locking API has two purposes:

* Ensure that access to the shared data is race-free.
* Let the implemetation do whatever is necessary for the user to have access to the data. For example it can be mapping and unmapping the surface data in memory if the underlying backend requires it.

The lock is expected to behave as a cross-process blocking read/write lock that is not reentrant.

## Immutable surfaces

In some cases we know in advance that a surface will not be modified after it has been shared. This is for example true for video frames. In this case the surface can be marked as immutable and the underlying implementation doesn't need to hold an actual blocking lock on the shared data.
Trying to acquire a write lock on a MozSurface that is marked as immutable and already shared must fail (return false).
Note that it is still required to use the Lock/Unlock API to read the data, in order for the implementation to be able to properly map and unmap the memory. This is just an optimization and a safety check.

## Drawing into a surface

In most cases we want to be able to paint directly into a surface through the Moz2D API.

A surface lets you *borrow* a DrawTarget that is only valid between Lock and Unlock.

    DrawTarget* GetAsDrawTarget();

It is invalid to hold a reference to the DrawTarget after Unlock, and a different DrawTarget may be obtained during the next Lock/Unlock interval.

In some cases we want to use MozSurface without drawing into it. For instance to share video frames accross processes. Some surface types may also not be accessible through a DrawTarget (for example YCbCr surfaces).

    bool CanExposeDrawTarget();

helps with making sure that a Surface supports exposing a Moz2D DrawTarget.

## Using a MozSurface as a source for Compositing

To interface with the Compositor API, MozSurface gives access to TextureSource objects. TextureSource is the cross-backend representation of a texture that Compositor understands.
While MozSurface handles memory management of (potentially shared) texture data, TextureSource is only an abstraction for Compositing.

## Fence synchronization

TODO: We need to figure this out. Right now we have a Gonk specific implementation, but no cross-platform abstraction/design.

## Ownership of the shared data

MozSurface (TextureClient/TextureHost in its current form) defines ownership rules that depend on the configuration of the surface, in order to satisy efficiency and safety requirements.

These rules rely on the fact that the underlying shared data is strictly owned by the MozSurface. This means that keeping direct references to the shared data is illegal and unsafe.

## Internal buffers / direct texturing

Some MozSurface implementations use CPU-side shared memory to share the texture data accross processes, and require a GPU texture upload when interfacing with a TextureSource. In this case we say that the surface has an internal buffer (because it is implicitly equivalent to double buffering where the shared data is the back buffer and the GPU side texture is the front buffer). We also say that it doesn't do "direct texturing" meaning that we don't draw directly into the GPU-side texture.

Examples:

 * Shmem MozSurface + OpenGL TextureSource: Has an internal buffer (no direct texturing)
 * Gralloc MozSurface + Gralloc TextureSource: No internal buffer (direct texturing)

While direct texturing is usually the most efficient way, it is not always available depending on the platform and the required allocation size or format. Textures with internal buffers have less restrictions around locking since the host side will only need to read from the MozSurface once per update, meaning that we can often get away with single buffering where we would need double buffering with direct texturing.

## Alternative solutions

## Backends

We have MozSurface implementaions (classes inheriting from TextureClient/TextureHost) for OpenGL, Software, D3D9, and D3D11 backends.
Some implemtations can be used with any backend (ex. ShmemTextureClient/Host).

## Users of MozSurface

MozSurface is the mechanism used by layers to share surfaces with the compositor, but it is not limited to layers. It should be used by anything that draws into a surface that may be shared with the compositor thread.

## Testing

TODO - How can we make MozSurface more testable and what should we test?

## Future work

### Using a MozSurface as a source for Drawing

MozSurface should be able to expose a borrowed Moz2D SourceSurface that is valid between Lock and Unlock similarly to how it exposes a DrawTarget.

## Comparison with other APIs

MozSurface is somewhat equivalent to Gralloc on Android/Gonk: it is a reference counted cross-process surface with locking semantics. While Gralloc can interface itself with OpenGL textures for compositing, MozSurface can interface itself to TextureSource objects.

MozSurface should not be confused with higher level APIs such as EGLStream. A swap-chain API like EGLStream can be implemented on top of MozSurface, but MozSurface's purpose is to define and manage the memory and resources of shared texture data.