Don't enable speculative tiles immediately after main load stops progressing