Fix the transform size/type search condition

While encoding a key frame with quantizer = 0 and aq-mode = 1,
for some segment_ids, the quantizer got modified and could be
> 0, and lossless[segment_id] might be 0 or 1 depending on the
segment_id. Namely, blocks with lossless[segment_id] = 0 were
allowed to choose transform sizes other than 4x4. This conflicted
with tx_mode which was a frame-level decision. In this patch,
the transform search condition was modified so that the transform
choice was consistent with tx_mode of that frame.

BUG=aomedia:104

Change-Id: Ia39127b5dee129283a133cf5e4000da62d9e0f1c
This commit is contained in:
Yunqing Wang 2016-12-16 21:23:50 -08:00
Родитель feb925fe84
Коммит 2615d6eaac
1 изменённых файлов: 1 добавлений и 1 удалений

Просмотреть файл

@ -1849,7 +1849,7 @@ static void super_block_yrd(const AV1_COMP *const cpi, MACROBLOCK *x,
assert(bs == xd->mi[0]->mbmi.sb_type);
if (xd->lossless[xd->mi[0]->mbmi.segment_id]) {
if (xd->lossless[0]) {
choose_smallest_tx_size(cpi, x, rd_stats, ref_best_rd, bs);
} else if (cpi->sf.tx_size_search_method == USE_LARGESTALL) {
choose_largest_tx_size(cpi, x, rd_stats, ref_best_rd, bs);