(1) Make unit tests for masked sad/variance encoder-only
(2) Fix compile error with intrabc
(3) Fix warnings reported by static analysis
Change-Id: I0cd2176fcda0b81e1fc30283767678376ced4c42
Patch https://aomedia-review.googlesource.com/c/12602/ made the
variable 'sum' in the warp filter unsigned, to indicate that its
value should always be >= 0. But 'sum' is used to accumulate
signed values, and it is expected that some of those values
will be negative.
The issue is that, when running 'x += y', if x is a uint32_t
and y is an int (and is 32 bits), the C standard says to
convert y to a uint32_t before doing the addition. This causes
overflow, and so undefined behaviour, if y < 0.
This is fixed by making 'sum' signed, and by explicitly bounds
checking against zero at the end of the filter.
BUG=aomedia:572
Change-Id: I1d484b5f5698db0ec9761807610b3b2b35647983
Make the txfm_above and txfm_left be processed in the unit of
miniumum transform block size. Scale the transform block step
size with respect to the mode_info step size.
Change-Id: Iee4421e005db742cd4ff7899215560063e5f68e5
Resolve the enc/dec mismatch issue when filter intra is turned on,
in addition to chroma-sub8x8, ext-tx, and ext-inter experiments.
BUG=aomedia:570
Change-Id: Idf0875763ce24bd8d4022f7338fc86debf4dfd0c
The masked variance functions can read slightly off the
(nominal) end of their input array. This is not a problem,
since the extra data is never used, and when the functions
are used in the encoder proper, they will just read data
from the frame border.
However, the arrays in the test case had not been properly
padded, causing ASan to complain about reading off the end
of an array.
Also remove an accidental comment in the code.
BUG=aomedia:569
Change-Id: Icb5cf4d39ee50fc6e1b807f483940d7a7b12ce63
* Simplify the C version of the warp filter to make the intent
of the code clearer
* Replace saturate_uint() in the C warp filter with an assertion
that the intermediate values are in-range. This is because they
should (provably) *never* go out-of-range.
* Add a comment describing the intended hardware architecture
* Miscellaneous comment updates
Change-Id: I798736f923ece599f22d573d31c5dfccd18b2d0e
Parts of it are causing doxygen to emit warnings, and
AOM's doxygen configuration doesn't produce XML output.
Change-Id: I34271721b64f369dfc51b9be3f508db2bfcb5633
Implements the high precision Wiener filter with an offset
to reduce the error due to saturation without increasing
the number of bits needed for intermediate precision.
Also turns the high precision filter on.
Change-Id: I34037a5746a6a89c5fce67753c1b027749085edf
We would expect that these new functions would be slower than
the old masked SAD/SSE functions, as they do additional work
(blending two inputs and comparing to a third, rather than
just comparing two inputs).
This is true for the SAD functions, which are about 50% slower
(depending on block size and bit depth). However, the sub-pixel
SSE functions are comparable to the old speed for the accelerated
special cases (xoffset or yoffset = 0 or 4), and are
between 40-90% faster for the generic case.
Change-Id: I1a296ed8fc9e3edc313a6add516ff76b17cd3e9f
Only use compmode_interinter_cost and compmode_interintra_cost in
handle_inter_mode() since outside the function they can be reflected
by the total rate.
Plus adding config flags to separate wedge/compound_segment from
ext_inter
Change-Id: I56b402aab3cf2337bae8d7e682f93cd8557f69a4
- Add av1_build_quantize() function so that quantization
related unit test would be able to access quant/dequant
table without starting an encoder instance and directly
accessing cpi.
Change-Id: I8ba429e5deb7a4e7f967996aaec1d20bff0feb3c
Changes av1_scale_if_required to check if a resize is needed using the
width and height from cm directly rather than cm's mi_cols and mi_rows
times MI_SIZE.
This is simpler, and won't have issues with sizes that aren't divisible
by MI_SIZE.
Change-Id: Icbf34252a62436bb9bb0f83fa5fd688f7c5ae256
This fixes an issue with context clearing introduced by commit
5e81643d31.
In av1_zero_above_context width_y went from being assigned "2 *
aligned_width" to "width << 1". All this patch does is replace width
with the aligned_width.
This resolves an issue with random resizing being broken on the master
branch.
Change-Id: Ib60bf81e6091802053a5a26057c626d4efe6497c
Resolve the segmentation fault issue in YUV440 due to uneven
chroma block size scaling.
BUG=aomedia:563
Change-Id: Ib607fb663fe2d9f2da6eef5bd22b07da8569fdd1
This commit resolves a rare enc/dec mismatch issue due to the
intracacy of using rectangular transform block size for intra
coded blocks.
BUG=aomedia:562
Change-Id: I10c52339a995a89bafa3a3330469ee3f2b784768
- This change turns on low bit depth data path for
this function under default HBD build.
- Encoder user level encoding time reduces ~12%
on i7-6700.
Change-Id: I7ce21e8db1a379f972e51c3b4ab305ca10e41efb
Modify the function signatures of read_intra_mode_y() and
read_intra_mode_uv() to take a FRAME_CONTEXT instead of an AV1_COMMON
which pushes the CONFIG_EC_ADAPT ifdef into the calling function.
Change-Id: I89029273bcbdd200a5b928ad91ba9a6a22cf0a62
In both versions of av1_optimize_b, the code to calculate the
dequantized coefficient from a modified quantized coefficient
was not compatible with tx64x64.
Specifically, the new dequantized coefficient was calculated as an
offset applied to the original coefficient. This had a fixup block
for 32x32 transforms, but not for 64x64 transforms.
For simplicity, instead of extending the fixup block, we now
recalculate the dequantized coefficient directly.
BUG=aomedia:537
Change-Id: I86c1c8820f324ad3f6afd6a720ff59f754384a19
This coding tool is targeted to check redundant references available
to each frame at the frame level, and then try to save bits to avoid
coding redundant reference information at the block level.
Change-Id: I320178a9fe0cd806960318ac59659808675e5176
Debargha Mukherjee
Yue Chen
David Barker
Urvang Joshi
Jingning Han
Alex Converse
Thomas Daede
Yi Luo
Tom Finegan
Arild Fuldseth (arilfuld)
Yushin Cho
Sarah Parker
Fergus Simpson
Cheng Chen
Luc Trudeau
Nathan E. Egge
Zoe Liu