Don't update layer positions on scrolling if we're in the middle of layout