94541 – [chromium] Refactor CCLayerTreeHostCommon: merge calculateVisibleRect post-walk into the main calcDrawTransforms recursion

Shawn Singh

Reported 2012-08-20 16:04:26 PDT

the "visibleLayerRect" of layers is currently computed in a post-walk through all layers that get scheduled to be drawn. In particular, the visibleLayerRect is computed as follows: // only makes a difference if the target surface actually clips the layer. (1) intersection of the layer bounds (in target surface space) with the target surface's contentRect. // further tighten the visible layer rect, which may have expanded due to enclosing the actual projection of the visible layer rect. (2) this rect is then projected back into layer's content space, and intersected with the layer's contentBounds again. It's worth noting that, even after these two steps, the visibleLayerRect is only an approximation, and not necessarily a nicely tight approximation. But still quite helpful, especially since most content with axis-aligned 2d transforms, it will be exact anyway. It should be possible to compute the same visible rect with the following refactorings: (1) remove CCLayerTreeHostCommon::calculateVisibleRects() functions (2) propagate the *approximate* clipRect across renderSurfaces (i.e. the enclosingIntRect of the inverse projection of the clipRect from parent targetSurface to the new surface's owning layer) (3) the drawableContentRectOfLayer, after being clipped, including the new clipRect that propagated from ancestor surfaces, will be the visibleLayerRect. And, because visibleLayerRect is technically (a) approximate and (b) in content space, not layer space, I would like to (4) re-name LayerChromium/CCLayerImpl visibleLayerRect property to "approximateVisibleRect" does this sound right? I'll give it a try and see what happens. To do this cleanly, I'll first add a bunch more visibleLayerRect tests first, in a separate patch.

Shawn Singh

Comment 1 2012-08-20 16:10:13 PDT

looks like my knowledge was a bit out-dated. It's already being called visibleContentRect =) would it still be OK with you guys if we re-name it to approximateVisibleContentRect ?

Dana Jansens

Comment 2 2012-08-20 16:12:34 PDT

Re naming: I was about to say it's already named visibleContentRect now. :) I'm not sure about adding approximate to it.. it's a rect that bounds what may be visible. Perhaps visibleContentBoundingRect if you think it's super important to say something but idk really. I like the rest :)

Shawn Singh

Comment 3 2012-08-20 16:14:48 PDT

Yeah - thanks for the input. Personally I was thinking it may become important to acknowledge it's approximate (and possibly coarsely approximate) in case it made a difference to be aware of that when trying to prepaint stuff that is not yet visible.

Dana Jansens

Comment 4 2012-08-20 16:16:19 PDT

I think I don't like approximate because it doesn't say how it may be wrong. If anything, bounding says it contains all visible stuff.

Dana Jansens

Comment 5 2012-08-20 16:17:03 PDT

That said, everything inside it will be considered "visible" because it's the best knowledge we do have :) So I don't think it is necessarily worth distinguishing in the variable name.

Adrienne Walker

Comment 6 2012-08-20 16:19:59 PDT

How about visibleContentRectValueMayBeApproximateButWeTriedOurBestReallyAndItMayGetBetterInTheFutureAfterSomeMoreRefactoring? visibleContentRect is fine. Leave a comment that says it's an approximation, but also that it will always enclose the "real" visibleContentRect.

Shawn Singh

Comment 7 2012-08-20 16:26:03 PDT

(In reply to comment #6) > How about visibleContentRectValueMayBeApproximateButWeTriedOurBestReallyAndItMayGetBetterInTheFutureAfterSomeMoreRefactoring? > > visibleContentRect is fine. Leave a comment that says it's an approximation, but also that it will always enclose the "real" visibleContentRect. alright alright... so be it =) I'll come back to this after 94542.

Shawn Singh

Comment 8 2012-08-20 16:53:48 PDT

(In reply to comment #7) > (In reply to comment #6) > > How about visibleContentRectValueMayBeApproximateButWeTriedOurBestReallyAndItMayGetBetterInTheFutureAfterSomeMoreRefactoring? > > > > visibleContentRect is fine. Leave a comment that says it's an approximation, but also that it will always enclose the "real" visibleContentRect. > > alright alright... so be it =) I'll come back to this after 94542. One last nit-picky naming point, for now... I think it might be appropriate to merge drawableContentRect and visibleContentRect after doing this refactor, since they are likely to end up being the same. And we did agree on re-naming drawableContentRect to clippedRectInTargetSpace... it might be appropriate to do that in this patch, anyway, so our naming discussion above was moot. Please let me know if you guys want to do something differently.

Shawn Singh

Comment 9 2012-08-20 16:56:14 PDT

Sorry for the noise; I just wanted to add one last thing for now - the reason I like this idea in comment #8 is because it avoids the word visible which could also be ambiguous for newcomers in the code. occluded things are technically not visible, but they would be part of this "visibleContentRect". so I think it's appropriate if it gets renamed to clippedRectInTargetSpace.

Dana Jansens

Comment 10 2012-08-20 17:00:25 PDT

Hm, if I project it back to the layer's content, i now have "clippedRect". Does the clipped in clippedRectInTargetSpace refer to non-visible parts being subtracted, or refer to being clipped by the camera and expanded (ie result of CCMathUtil::mapClippedRect)? I've always used clipped to mean the latter.

Shawn Singh

Comment 11 2012-08-20 17:05:45 PDT

(In reply to comment #10) > Hm, if I project it back to the layer's content, i now have "clippedRect". > > Does the clipped in clippedRectInTargetSpace refer to non-visible parts being subtracted, or refer to being clipped by the camera and expanded (ie result of CCMathUtil::mapClippedRect)? > > I've always used clipped to mean the latter. TLDR; I think the conclusion here is that let's just postpone re-naming confusion and NOT do it in this patch =) The way I use "clipped", CCMathUtil clipping is unrelated to "clippedRect". I feel like we don't need to indicate that rects are clipped by w < 0; but rather we need to clearly indicate to ourselves that they are not exact. There are other more common reasons that such transforms would not be exact, anyway. The way I had been thinking of "clippedRect" meant that it was intersected with ancestor bounds if masksToBounds() or hasMaskLayer(). and, you're right about the targetSpace vs contentSpace... my mistake totally. So the proposed rename should probably be: clippedRectInTargetSpace (new name for drawableContentRect) clippedRectInContentSpace (new name for visibleContentRect)

Adrienne Walker

Comment 12 2012-08-20 17:18:10 PDT

(In reply to comment #9) > Sorry for the noise; I just wanted to add one last thing for now - the reason I like this idea in comment #8 is because it avoids the word visible which could also be ambiguous for newcomers in the code. occluded things are technically not visible, but they would be part of this "visibleContentRect". so I think it's appropriate if it gets renamed to clippedRectInTargetSpace. There are plenty of variables in the compositor that we use that are loose in exactly one direction, especially when we are dealing with rects as a simplification for something more complex. Space outside the occlusion rect may actually occlude. Space inside the unocclusion rect may actually be occluded. Space outside the opaque rect may be opaque. Space inside the visible rect may actually not be visible. I appreciate that you want to make variables be more precise, but I don't think your proposed renamings meet the bar for increased clarity vs. increased churn.

Shawn Singh

Comment 13 2012-08-22 16:54:34 PDT

Created attachment 160036 [details] Finished patch, but should not land before unit tests--comment=passes manual testing and unit tests; layout tests are a bit noisy at the moment so I have to check more carefully, but they seem to be OK

Eric Penner

Comment 14 2012-08-22 19:13:15 PDT

Comment on attachment 160036 [details] Finished patch, but should not land before unit tests--comment=passes manual testing and unit tests; layout tests are a bit noisy at the moment so I have to check more carefully, but they seem to be OK View in context: https://bugs.webkit.org/attachment.cgi?id=160036&action=review > Source/WebCore/platform/graphics/chromium/cc/CCLayerTreeHostCommon.cpp:133 > + // There are three conditions where we can quickly return the full contentBounds of the layer: Does this depend on cases 1.) and 2.) to work properly? I like the idea of painting all animated layers, but we never came up with a solution to handle arbitrarily large layers that can't all fit into memory. We currently do some special logic to detect 'reasonable sized' layers and make this decision at paint time after we have already successfully allocated textures for the entire layer. Would it hurt to just remove the top two cases? Even if the exact transform isn't known, the visible rect from the approximate transform on the main thread is better than nothing.

Eric Penner

Comment 15 2012-08-22 20:19:08 PDT

Comment on attachment 160036 [details] Finished patch, but should not land before unit tests--comment=passes manual testing and unit tests; layout tests are a bit noisy at the moment so I have to check more carefully, but they seem to be OK View in context: https://bugs.webkit.org/attachment.cgi?id=160036&action=review >> Source/WebCore/platform/graphics/chromium/cc/CCLayerTreeHostCommon.cpp:133 >> + // There are three conditions where we can quickly return the full contentBounds of the layer: > > Does this depend on cases 1.) and 2.) to work properly? > > I like the idea of painting all animated layers, but we never came up with a solution to handle arbitrarily large layers that can't all fit into memory. We currently do some special logic to detect 'reasonable sized' layers and make this decision at paint time after we have already successfully allocated textures for the entire layer. Would it hurt to just remove the top two cases? Even if the exact transform isn't known, the visible rect from the approximate transform on the main thread is better than nothing. Now that I look again, case 3.) also seems like it could occur on an unbounded size layer which might OOM. This is where I don't see how we can do without propagated visibility info of some sort.

Shawn Singh

Comment 16 2012-08-22 21:34:11 PDT

(In reply to comment #15) > (From update of attachment 160036 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=160036&action=review > > >> Source/WebCore/platform/graphics/chromium/cc/CCLayerTreeHostCommon.cpp:133 > >> + // There are three conditions where we can quickly return the full contentBounds of the layer: > > > > Does this depend on cases 1.) and 2.) to work properly? > > > > I like the idea of painting all animated layers, but we never came up with a solution to handle arbitrarily large layers that can't all fit into memory. We currently do some special logic to detect 'reasonable sized' layers and make this decision at paint time after we have already successfully allocated textures for the entire layer. Would it hurt to just remove the top two cases? Even if the exact transform isn't known, the visible rect from the approximate transform on the main thread is better than nothing. > > Now that I look again, case 3.) also seems like it could occur on an unbounded size layer which might OOM. This is where I don't see how we can do without propagated visibility info of some sort. OK, sounds like we should discuss a little bit more offline with danakj and enne as well. In the meantime, personally I think this patch is a pretty good cleanup of the code anyway - do you think there's any reason we should actually not move forward with this?

Dana Jansens

Comment 17 2012-08-23 08:02:34 PDT

Comment on attachment 160036 [details] Finished patch, but should not land before unit tests--comment=passes manual testing and unit tests; layout tests are a bit noisy at the moment so I have to check more carefully, but they seem to be OK View in context: https://bugs.webkit.org/attachment.cgi?id=160036&action=review >>>> Source/WebCore/platform/graphics/chromium/cc/CCLayerTreeHostCommon.cpp:133 >>>> + // There are three conditions where we can quickly return the full contentBounds of the layer: >>> >>> Does this depend on cases 1.) and 2.) to work properly? >>> >>> I like the idea of painting all animated layers, but we never came up with a solution to handle arbitrarily large layers that can't all fit into memory. We currently do some special logic to detect 'reasonable sized' layers and make this decision at paint time after we have already successfully allocated textures for the entire layer. Would it hurt to just remove the top two cases? Even if the exact transform isn't known, the visible rect from the approximate transform on the main thread is better than nothing. >> >> Now that I look again, case 3.) also seems like it could occur on an unbounded size layer which might OOM. This is where I don't see how we can do without propagated visibility info of some sort. > > OK, sounds like we should discuss a little bit more offline with danakj and enne as well. > > In the meantime, personally I think this patch is a pretty good cleanup of the code anyway - do you think there's any reason we should actually not move forward with this? Is its behaviour the same? These sound different than what used to happen at least. I don't like the whole "cannot assume" stuff really. I know I was the first one to bring that terminology into the compositor, but I've tried to get away from that. In CCLTHC I think we should use "unknown transform" stuff *only* for making decisions about (not) dropping layers out of the RSLL. Other than that we should produce a valid and correct frame with exact values for the time at which the frame is generated. I don't see why animation should do anything to the calculation on the impl thread, which happens here for 1. If you want to make the render surface texture stay the same size, I think it should not be done by changing the visibleContentRect of its owning layer (and how would that guarantee it didnt change anyway?).

Shawn Singh

Comment 18 2012-08-23 09:18:51 PDT

(In reply to comment #17) > (From update of attachment 160036 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=160036&action=review > > >>>> Source/WebCore/platform/graphics/chromium/cc/CCLayerTreeHostCommon.cpp:133 > >>>> + // There are three conditions where we can quickly return the full contentBounds of the layer: > >>> > >>> Does this depend on cases 1.) and 2.) to work properly? > >>> > >>> I like the idea of painting all animated layers, but we never came up with a solution to handle arbitrarily large layers that can't all fit into memory. We currently do some special logic to detect 'reasonable sized' layers and make this decision at paint time after we have already successfully allocated textures for the entire layer. Would it hurt to just remove the top two cases? Even if the exact transform isn't known, the visible rect from the approximate transform on the main thread is better than nothing. > >> > >> Now that I look again, case 3.) also seems like it could occur on an unbounded size layer which might OOM. This is where I don't see how we can do without propagated visibility info of some sort. > > > > OK, sounds like we should discuss a little bit more offline with danakj and enne as well. > > > > In the meantime, personally I think this patch is a pretty good cleanup of the code anyway - do you think there's any reason we should actually not move forward with this? > > Is its behaviour the same? These sound different than what used to happen at least. > > I don't like the whole "cannot assume" stuff really. I know I was the first one to bring that terminology into the compositor, but I've tried to get away from that. In CCLTHC I think we should use "unknown transform" stuff *only* for making decisions about (not) dropping layers out of the RSLL. Other than that we should produce a valid and correct frame with exact values for the time at which the frame is generated. I don't see why animation should do anything to the calculation on the impl thread, which happens here for 1. > I'll look into this again. I think 1 & 2 were necessary because otherwise things don't get painted correctly when animating, because we think they are not visible. In the previous code, visibility was computed via the surface's contentRect, and that contentRect was not clipped (on the main thread) if the surface was animating. The difference, however, between these checks and the one dealing with surface's contentRect is that we are not necessarily at the exact point in recursion where all we need to do is check the layer's transformToParentIsKnown() helper. Instead, if we're further down in the recursion, the layer has to check if it's screen space transform is known. For the surface-is-animating check - yeah, it might need to be refined, or maybe it's already encompassed by the layer's transformToScreenIsKnown(). > If you want to make the render surface texture stay the same size, I think it should not be done by changing the visibleContentRect of its owning layer (and how would that guarantee it didnt change anyway?). Sorry I didn't understand this last part. The render surface size is determined by drawableContentRectOfSubtree, and if needed, the clipRect. So it has nothing to do with visibleContentRect?

Dana Jansens

Comment 19 2012-08-23 10:24:19 PDT

(In reply to comment #18) > > If you want to make the render surface texture stay the same size, I think it should not be done by changing the visibleContentRect of its owning layer (and how would that guarantee it didnt change anyway?). > > Sorry I didn't understand this last part. The render surface size is determined by drawableContentRectOfSubtree, and if needed, the clipRect. So it has nothing to do with visibleContentRect? Oh okay. I just read some conversation about preserving surfaces sizes during animation, and was wondering if that was part of the reasoning here for checking if animations are happening. If not, then I think/hope there's less reason to worry about if animation is happening. Animating things that have visibleContentRect.isEmpty() have been getting prepainted based on their size. I think it should be the prepainting code's job to figure out what to paint appropriately, and the visibleContentRect should describe what is actually visible - regardless of animation.

Adrienne Walker

Comment 20 2012-08-23 10:27:49 PDT

(In reply to comment #19) > > I think it should be the prepainting code's job to figure out what to paint appropriately, and the visibleContentRect should describe what is actually visible - regardless of animation. +1 The prepainting code definitely needs additional information to make better decisions, but visibleContentRect shouldn't lie to make that happen.

Shawn Singh

Comment 21 2012-08-23 11:25:09 PDT

(In reply to comment #20) > (In reply to comment #19) > > > > I think it should be the prepainting code's job to figure out what to paint appropriately, and the visibleContentRect should describe what is actually visible - regardless of animation. > > +1 > > The prepainting code definitely needs additional information to make better decisions, but visibleContentRect shouldn't lie to make that happen. Chalk it up to a brain fart on my part. It does work fine without those if statements, and I agree with both you guys' logic. So yes we should be able to just remove conditions #1 and #2. BUT... it looks like this refactor doesn't fit Eric's needs yet, since we still do need check #3, which seems to be problematic based on comment #15. So maybe we want the following middle-ground: (1) renderSurfaces and drawableContentRects compute their size the same way they do now, so they remain better cacheable. (2) visibleContentRects, however, always use clipRects the way that we were propagating them in https://bugs.webkit.org/show_bug.cgi?id=94663. This way they are always bounded. If we do this, there would be a nice relationship describing the meaning of drawableContentRect and visibleContentRect - and yes I might try to debate over renaming things some other day =) drawableContentRect --> the portion of the layer that contributes to the surface (not counting occlusion) visibleContentRect --> the portion of the layer that contributes to the screen (not counting occlusion)

Dana Jansens

Comment 22 2012-08-23 11:37:21 PDT

I like it :)

Adrienne Walker

Comment 23 2012-08-23 11:43:11 PDT

(In reply to comment #21) > If we do this, there would be a nice relationship describing the meaning of drawableContentRect and visibleContentRect - and yes I might try to debate over renaming things some other day =) > > drawableContentRect --> the portion of the layer that contributes to the surface (not counting occlusion) > visibleContentRect --> the portion of the layer that contributes to the screen (not counting occlusion) Wait, what? drawableContentRect --> the projection of a layer or contributing surface into its target space, bounded by the size of that target

Eric Penner

Comment 24 2012-08-23 12:05:03 PDT

> > drawableContentRect --> the portion of the layer that contributes to the surface (not counting occlusion) > > visibleContentRect --> the portion of the layer that contributes to the screen (not counting occlusion) > > Wait, what? > > drawableContentRect --> the projection of a layer or contributing surface into its target space, bounded by the size of that target Having these definitions as comments on LayerChromium would be ultra awesome! They might need maintenance as definitions change, but it's well worth it IMO. > BUT... it looks like this refactor doesn't fit Eric's needs yet, since we still do need check #3, which seems to be problematic based on comment #15. Regarding comment #15, is the layer size implicitly bounded in size because that clip is empty? If so then it's okay to use visible=bounds, but I couldn't tell if that was always the case.

Shawn Singh

Comment 25 2012-08-23 12:47:51 PDT

(In reply to comment #23) > Wait, what? > > drawableContentRect --> the projection of a layer or contributing surface into its target space, bounded by the size of that target Yes, I feel like that's the same thing I said, except you're wording acknowledges that the rect is described in target surface space (i.e. the projection part). I can be more careful about the wording in comments. (In reply to comment #24) > Regarding comment #15, is the layer size implicitly bounded in size because that clip is empty? If so then it's okay to use visible=bounds, but I couldn't tell if that was always the case. I'm a little unclear what you're asking... here's my understanding of the situation: In the current code, a huge unclipped layer: - has a huge drawableContentRect - causes its target renderSurface to grow to enclose the layer, so it becomes huge, too - has a huge visibleContentRect, which isn't helpful for intelligent prepainting. In future code, if we make the change, a huge unclipped layer: - (same) still has a huge drawableContentRect - (same) still causes the renderSurface to grow huge to enclose the layer - (new behavior) has a visibleContentRect that is well bounded, based on any ancestor clips, including the viewport. will that be useful for you? In this patch or a later patch, we could add a notion of "visibleContent" to renderSurfaces as well.

Adrienne Walker

Comment 26 2012-08-23 14:27:47 PDT

Given how many bugs have come out of CCLayerTreeHostCommon refactoring, how about doing this in two parts: (1) Do what the bug title says and remove the post-walk for visible layer rect calculation, but leave the results of that calculation identical to before the patch. (2) Tighten up the visibleLayerRect calculation by projecting clipRects down through intermediate render surfaces.

Eric Penner

Comment 27 2012-08-23 14:56:02 PDT

> > Regarding comment #15, is the layer size implicitly bounded in size because that clip is empty? If so then it's okay to use visible=bounds, but I couldn't tell if that was always the case. > > I'm a little unclear what you're asking... here's my understanding of the situation: > > In the current code, a huge unclipped layer: > - has a huge drawableContentRect > - causes its target renderSurface to grow to enclose the layer, so it becomes huge, too > - has a huge visibleContentRect, which isn't helpful for intelligent prepainting. > Could 'huge' mean say 100,000x100,000? Or just 2048x2048 (max surface size). If we ever set a visibleRect to the former then that's a problem. However, if it was already a problem before then no need to fix it in this CL.

Shawn Singh

Comment 28 2012-10-05 19:30:55 PDT

This is now being worked on on the chromium side. https://codereview.chromium.org/11079012/

Attachments
Finished patch, but should not land before unit tests--comment=passes manual testing and unit tests; layout tests are a bit noisy at the moment so I have to check more carefully, but they seem to be OK (27.12 KB, patch) 2012-08-22 16:54 PDT, Shawn Singh	no flags	Details Formatted Diff Diff
View All Add attachment proposed patch, testcase, etc.