gecko-dev/gfx/webrender/doc/blob.md

Blob images

The blob image mechanism now has two traits:

• BlobImageHandler is roughly the equivalent of the previous BlobImageRenderer except that it doesn't do any rendering (it manages the state of the blob commands, and resources like fonts).
• AsyncBlobImageRasterizer is created by the handler and sent over to the scene builder thread. the async rasterizer is meant to be a snapshot of the state of blob image commands that can execute the commands if provided some requests.

When receiving a transaction, the render backend / resource cache look at the list of added and updated blob images in that transaction, collect the list of blob images and tiles that need to be rendered, create a rasterizer, and ship the two to the scene builder. After building the scene the rasterizer gets handed the list of blob requests and does all of the rasterization, blocking the scene builder thread until the work is done.

When the scene building and rasterization is done, the render backend receives the rasterized blobs and stores them so that they are available when frame building needs them.

Because blob images can be huge, we don't always want to rasterize them entirely during scene building. To decide what should be rasterized, we rely on gecko giving us a hint through the added set_image_visible_area API. When the render backend receives that message it decides which tiles are going to be rasterized. This information is also used to decide which tiles to evict, so that we don't keep thousands of tiles if we scroll through a massive blob image. The idea is for the visible area to correspond to the size of the display list.

Sometimes, however, Gecko gets this visible area "wrong", or at least gives webrender a certain visible area but eventually webrender requests tiles during frame building that weren't in that area. I think that this is inevitable because the culling logic in gecko and webrender works very differently, so relying on them to match exactly is fragile at best. So to work around this type of situation, keep around the async blob rasterizer that we sent to the scene builder, and store it in the resource cache when we swap the scene. This blob rasterizer represents the state of the blob commands at the time the transaction was built (and is potentially different from the state of the blob image handler). Frame building collects a list of blob images (or blob tiles) that are not already rasterized, and asks the current async blob rasterizer to rasterize them synchronously on the render backend. The hope is that this would happen rarely.

Another important detail is that for this to work, resources that are used by blob images (so currently only fonts), need to be in sync with the blobs. Fortunately, fonts are currently immutable so we mostly need to make sure they are added before the transaction is built and removed after the transaction is swapped. If blob images were to use images, then we'd have to either do the same for these images (and disallow updating them), or maintain the state of images before and after scene building like we effectively do for blobs.