From f975ac57b05e1c365fec13b614b79b4597d115cd Mon Sep 17 00:00:00 2001 From: Brennan Shacklett Date: Wed, 3 Aug 2016 10:35:01 -0700 Subject: [PATCH] Temporary fix for 4X8 block intra prediction. Currently the RD loop traverses 4X8 blocks in inverted N order while the bitstream stores blocks smaller than 8x8 in Z order. This causes a discrepancy where the RD loop reads uninitialized data while performing intra prediction. As a temporary fix simply disable the use of the extended right edge for 4X8 blocks, until the bitstream can be changed to match the logical structure of the blocks. Change-Id: I44a9e4fc1a15cd551a7b38c3c1227bc5dac77e9a --- av1/common/reconintra.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/av1/common/reconintra.c b/av1/common/reconintra.c index f5733348f..668892a8d 100644 --- a/av1/common/reconintra.c +++ b/av1/common/reconintra.c @@ -83,6 +83,14 @@ static int av1_has_right(BLOCK_SIZE bsize, int mi_row, int mi_col, int ss_x) { if (!right_available) return 0; + // TODO(bshacklett, huisu): Currently the RD loop traverses 4X8 blocks in + // inverted N order while in the bitstream the subblocks are stored in Z + // order. This discrepancy makes this function incorrect when considering 4X8 + // blocks in the RD loop, so we disable the extended right edge for these + // blocks. The correct solution is to change the bitstream to store these + // blocks in inverted N order, and then update this function appropriately. + if (bsize == BLOCK_4X8 && y == 1) return 0; + if (y == 0) { int wl = mi_width_log2_lookup[bsize]; int hl = mi_height_log2_lookup[bsize];