mozilla
/
gecko-dev
зеркало из https://github.com/mozilla/gecko-dev.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Bug 1848578 - Update libjxl to e6202f7181eff36c78bfdb79aa9bd45c3d1d614b r=saschanaz 

Differential Revision: https://phabricator.services.mozilla.com/D186097
This commit is contained in:
Updatebot 2023-08-16 10:25:21 +00:00
Родитель 3f5eefadfb
Коммит 78c3beaf49
43 изменённых файлов: 652 добавлений и 788 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
media/libjxl/moz.yaml
Просмотреть файл

@ -10,9 +10,9 @@ origin:
url: https://github.com/libjxl/libjxl
release: 69d06e17771830beae7fb62b76de5978b52546fc (2023-07-21T14:55:24Z).
release: e6202f7181eff36c78bfdb79aa9bd45c3d1d614b (2023-08-11T12:36:59Z).
revision: 69d06e17771830beae7fb62b76de5978b52546fc
revision: e6202f7181eff36c78bfdb79aa9bd45c3d1d614b
license: Apache-2.0

12
third_party/jpeg-xl/CHANGELOG.md поставляемый
Просмотреть файл

@ -15,6 +15,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `JxlEncoderChunkedImageFrameStart`,
- `JxlEncoderChunkedImageFrameAddPart` and new
- `JXL_ENC_FRAME_SETTING_BUFFERING` enum value.
- encoder API: new options for more fine-grained control over metadata
preservation when using `JxlEncoderAddJPEGFrame`:
- `JXL_ENC_FRAME_SETTING_JPEG_KEEP_EXIF`
- `JXL_ENC_FRAME_SETTING_JPEG_KEEP_XMP`
- `JXL_ENC_FRAME_SETTING_JPEG_KEEP_JUMBF`
- encoder API: new function `JxlEncoderSetUpsamplingMode` to change the upsampling
method, e.g. to use nearest-neighbor upsampling for pixel art
- cjxl can now be used to explicitly add/update/strip Exif/XMP/JUMBF metadata using
the decoder-hints syntax, e.g. `cjxl input.ppm -x exif=input.exif output.jxl`
- djxl can now be used to extract Exif/XMP/JUMBF metadata
### Removed
- API: the Butteraugli API (`jxl/butteraugli.h`) was removed.
- encoder and decoder API: all deprecated functions were removed:
@ -54,6 +65,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed / clarified
- encoder API: `JxlEncoderProcessOutput` requires at least 32 bytes of output
space to proceed and guarantees that at least one byte will be written
## [0.7] - 2022-07-21
### Added

10
third_party/jpeg-xl/experimental/fast_lossless/README.md поставляемый
Просмотреть файл

@ -1,10 +0,0 @@
# Fast-lossless
This is a script to compile a standalone version of a JXL encoder that supports
lossless compression, up to 16 bits, of 1- to 4-channel images and animations; it is
very fast and compression is slightly worse than PNG for 8-bit nonphoto content
and better or much better than PNG for all other situations.
The main encoder is made out of two files, `lib/jxl/enc_fast_lossless.{cc,h}`;
it automatically selects and runs a SIMD implementation supported by your CPU.
This folder contains an example build script and `main` file.

27
third_party/jpeg-xl/experimental/fast_lossless/build-android.sh поставляемый
Просмотреть файл

@ -1,27 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) the JPEG XL Project Authors. All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
set -e
DIR=$(realpath "$(dirname "$0")")
mkdir -p /tmp/build-android
cd /tmp/build-android
CXX="$ANDROID_NDK"/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang++
if ! command -v "$CXX" >/dev/null ; then
printf >&2 '%s: Android C++ compiler not found, is ANDROID_NDK set properly?\n' "${0##*/}"
exit 1
fi
[ -f lodepng.cpp ] || curl -o lodepng.cpp --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.cpp'
[ -f lodepng.h ] || curl -o lodepng.h --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.h'
[ -f lodepng.o ] || "$CXX" lodepng.cpp -O3 -o lodepng.o -c
"$CXX" -O3 \
-I. lodepng.o \
-I"${DIR}"/../../ \
"${DIR}"/../../lib/jxl/enc_fast_lossless.cc "${DIR}"/fast_lossless_main.cc \
-o fast_lossless

27
third_party/jpeg-xl/experimental/fast_lossless/build.sh поставляемый
Просмотреть файл

@ -1,27 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) the JPEG XL Project Authors. All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
set -e
DIR=$(realpath "$(dirname "$0")")
mkdir -p "$DIR"/build
cd "$DIR"/build
# set CXX to clang++ if not set in the environment
CXX="${CXX-clang++}"
if ! command -v "$CXX" >/dev/null ; then
printf >&2 '%s: C++ compiler not found\n' "${0##*/}"
exit 1
fi
[ -f lodepng.cpp ] || curl -o lodepng.cpp --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.cpp'
[ -f lodepng.h ] || curl -o lodepng.h --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.h'
[ -f lodepng.o ] || "$CXX" lodepng.cpp -O3 -o lodepng.o -c
"$CXX" -O3 \
-I. -g lodepng.o \
-I"$DIR"/../../ \
"$DIR"/../../lib/jxl/enc_fast_lossless.cc "$DIR"/fast_lossless_main.cc \
-o fast_lossless

26
third_party/jpeg-xl/experimental/fast_lossless/cross_compile_aarch64.sh поставляемый
Просмотреть файл

@ -1,26 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) the JPEG XL Project Authors. All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
set -e
DIR=$(realpath "$(dirname "$0")")
mkdir -p "$DIR"/build-aarch64
cd "$DIR"/build-aarch64
CXX="${CXX-aarch64-linux-gnu-c++}"
if ! command -v "$CXX" >/dev/null ; then
printf >&2 '%s: C++ compiler not found\n' "${0##*/}"
exit 1
fi
[ -f lodepng.cpp ] || curl -o lodepng.cpp --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.cpp'
[ -f lodepng.h ] || curl -o lodepng.h --url 'https://raw.githubusercontent.com/lvandeve/lodepng/8c6a9e30576f07bf470ad6f09458a2dcd7a6a84a/lodepng.h'
[ -f lodepng.o ] || "$CXX" lodepng.cpp -O3 -o lodepng.o -c
"$CXX" -O3 -static \
-I. lodepng.o \
-I"$DIR"/../../ \
"$DIR"/../../lib/jxl/enc_fast_lossless.cc "$DIR"/fast_lossless_main.cc \
-o fast_lossless

113
third_party/jpeg-xl/experimental/fast_lossless/fast_lossless_main.cc поставляемый
Просмотреть файл

@ -1,113 +0,0 @@
// Copyright (c) the JPEG XL Project Authors. All rights reserved.
//
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <atomic>
#include <chrono>
#include <thread>
#include <vector>
#include "lib/jxl/enc_fast_lossless.h"
#include "lodepng.h"
#include "pam-input.h"
int main(int argc, char** argv) {
if (argc < 3) {
fprintf(stderr,
"Usage: %s in.png out.jxl [effort] [num_reps] [num_threads]\n",
argv[0]);
return 1;
}
const char* in = argv[1];
const char* out = argv[2];
int effort = argc >= 4 ? atoi(argv[3]) : 2;
size_t num_reps = argc >= 5 ? atoi(argv[4]) : 1;
size_t num_threads = argc >= 6 ? atoi(argv[5]) : 0;
if (effort < 0 || effort > 127) {
fprintf(
stderr,
"Effort should be between 0 and 127 (default is 2, more is slower)\n");
return 1;
}
unsigned char* png;
unsigned w, h;
size_t nb_chans = 4, bitdepth = 8;
unsigned error = lodepng_decode32_file(&png, &w, &h, in);
size_t width = w, height = h;
if (error && !DecodePAM(in, &png, &width, &height, &nb_chans, &bitdepth)) {
fprintf(stderr, "lodepng error %u: %s\n", error, lodepng_error_text(error));
return 1;
}
auto parallel_runner = [](void* num_threads_ptr, void* opaque,
void fun(void*, size_t), size_t count) {
size_t num_threads = *(size_t*)num_threads_ptr;
if (num_threads == 0) {
num_threads = std::thread::hardware_concurrency();
}
if (num_threads > count) {
num_threads = count;
}
if (num_threads == 1) {
for (size_t i = 0; i < count; i++) {
fun(opaque, i);
}
} else {
std::atomic<int> task{0};
std::vector<std::thread> threads;
for (size_t i = 0; i < num_threads; i++) {
threads.push_back(std::thread([count, opaque, fun, &task]() {
while (true) {
int t = task++;
if (t >= count) break;
fun(opaque, t);
}
}));
}
for (auto& t : threads) t.join();
}
};
size_t encoded_size = 0;
unsigned char* encoded = nullptr;
size_t stride = width * nb_chans * (bitdepth > 8 ? 2 : 1);
auto start = std::chrono::high_resolution_clock::now();
for (size_t _ = 0; _ < num_reps; _++) {
free(encoded);
encoded_size = JxlFastLosslessEncode(
png, width, stride, height, nb_chans, bitdepth,
/*big_endian=*/true, effort, &encoded, &num_threads, +parallel_runner);
}
auto stop = std::chrono::high_resolution_clock::now();
if (num_reps > 1) {
float us =
std::chrono::duration_cast<std::chrono::microseconds>(stop - start)
.count();
size_t pixels = size_t{width} * size_t{height} * num_reps;
float mps = pixels / us;
fprintf(stderr, "%10.3f MP/s\n", mps);
fprintf(stderr, "%10.3f bits/pixel\n",
encoded_size * 8.0 / float(width) / float(height));
}
FILE* o = fopen(out, "wb");
if (!o) {
fprintf(stderr, "error opening %s: %s\n", out, strerror(errno));
return 1;
}
if (fwrite(encoded, 1, encoded_size, o) != encoded_size) {
fprintf(stderr, "error writing to %s: %s\n", out, strerror(errno));
}
fclose(o);
}

289
third_party/jpeg-xl/experimental/fast_lossless/pam-input.h поставляемый
Просмотреть файл

@ -1,289 +0,0 @@
// Copyright (c) the JPEG XL Project Authors. All rights reserved.
//
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
#include <limits.h>
#include <stdlib.h>
#include <string.h>
bool error_msg(const char* message) {
fprintf(stderr, "%s\n", message);
return false;
}
#define return_on_error(X) \
if (!X) return false;
size_t Log2(uint32_t value) { return 31 - __builtin_clz(value); }
struct HeaderPNM {
size_t xsize;
size_t ysize;
bool is_gray; // PGM
bool has_alpha; // PAM
size_t bits_per_sample;
};
class Parser {
public:
explicit Parser(uint8_t* data, size_t length)
: pos_(data), end_(data + length) {}
// Sets "pos" to the first non-header byte/pixel on success.
bool ParseHeader(HeaderPNM* header, const uint8_t** pos) {
// codec.cc ensures we have at least two bytes => no range check here.
if (pos_[0] != 'P') return false;
const uint8_t type = pos_[1];
pos_ += 2;
switch (type) {
case '5':
header->is_gray = true;
return ParseHeaderPNM(header, pos);
case '6':
header->is_gray = false;
return ParseHeaderPNM(header, pos);
case '7':
return ParseHeaderPAM(header, pos);
}
return false;
}
// Exposed for testing
bool ParseUnsigned(size_t* number) {
if (pos_ == end_) return error_msg("PNM: reached end before number");
if (!IsDigit(*pos_)) return error_msg("PNM: expected unsigned number");
*number = 0;
while (pos_ < end_ && *pos_ >= '0' && *pos_ <= '9') {
*number *= 10;
*number += *pos_ - '0';
++pos_;
}
return true;
}
bool ParseSigned(double* number) {
if (pos_ == end_) return error_msg("PNM: reached end before signed");
if (*pos_ != '-' && *pos_ != '+' && !IsDigit(*pos_)) {
return error_msg("PNM: expected signed number");
}
// Skip sign
const bool is_neg = *pos_ == '-';
if (is_neg || *pos_ == '+') {
++pos_;
if (pos_ == end_) return error_msg("PNM: reached end before digits");
}
// Leading digits
*number = 0.0;
while (pos_ < end_ && *pos_ >= '0' && *pos_ <= '9') {
*number *= 10;
*number += *pos_ - '0';
++pos_;
}
// Decimal places?
if (pos_ < end_ && *pos_ == '.') {
++pos_;
double place = 0.1;
while (pos_ < end_ && *pos_ >= '0' && *pos_ <= '9') {
*number += (*pos_ - '0') * place;
place *= 0.1;
++pos_;
}
}
if (is_neg) *number = -*number;
return true;
}
private:
static bool IsDigit(const uint8_t c) { return '0' <= c && c <= '9'; }
static bool IsLineBreak(const uint8_t c) { return c == '\r' || c == '\n'; }
static bool IsWhitespace(const uint8_t c) {
return IsLineBreak(c) || c == '\t' || c == ' ';
}
bool SkipBlank() {
if (pos_ == end_) return error_msg("PNM: reached end before blank");
const uint8_t c = *pos_;
if (c != ' ' && c != '\n') return error_msg("PNM: expected blank");
++pos_;
return true;
}
bool SkipSingleWhitespace() {
if (pos_ == end_) return error_msg("PNM: reached end before whitespace");
if (!IsWhitespace(*pos_)) return error_msg("PNM: expected whitespace");
++pos_;
return true;
}
bool SkipWhitespace() {
if (pos_ == end_) return error_msg("PNM: reached end before whitespace");
if (!IsWhitespace(*pos_) && *pos_ != '#') {
return error_msg("PNM: expected whitespace/comment");
}
while (pos_ < end_ && IsWhitespace(*pos_)) {
++pos_;
}
// Comment(s)
while (pos_ != end_ && *pos_ == '#') {
while (pos_ != end_ && !IsLineBreak(*pos_)) {
++pos_;
}
// Newline(s)
while (pos_ != end_ && IsLineBreak(*pos_)) pos_++;
}
while (pos_ < end_ && IsWhitespace(*pos_)) {
++pos_;
}
return true;
}
bool MatchString(const char* keyword) {
const uint8_t* ppos = pos_;
while (*keyword) {
if (ppos >= end_) return error_msg("PAM: unexpected end of input");
if (*keyword != *ppos) return false;
ppos++;
keyword++;
}
pos_ = ppos;
return_on_error(SkipWhitespace());
return true;
}
bool ParseHeaderPAM(HeaderPNM* header, const uint8_t** pos) {
size_t num_channels = 3;
size_t max_val = 255;
while (!MatchString("ENDHDR")) {
return_on_error(SkipWhitespace());
if (MatchString("WIDTH")) {
return_on_error(ParseUnsigned(&header->xsize));
} else if (MatchString("HEIGHT")) {
return_on_error(ParseUnsigned(&header->ysize));
} else if (MatchString("DEPTH")) {
return_on_error(ParseUnsigned(&num_channels));
} else if (MatchString("MAXVAL")) {
return_on_error(ParseUnsigned(&max_val));
} else if (MatchString("TUPLTYPE")) {
if (MatchString("RGB_ALPHA")) {
header->has_alpha = true;
} else if (MatchString("RGB")) {
} else if (MatchString("GRAYSCALE_ALPHA")) {
header->has_alpha = true;
header->is_gray = true;
} else if (MatchString("GRAYSCALE")) {
header->is_gray = true;
} else if (MatchString("BLACKANDWHITE_ALPHA")) {
header->has_alpha = true;
header->is_gray = true;
max_val = 1;
} else if (MatchString("BLACKANDWHITE")) {
header->is_gray = true;
max_val = 1;
} else {
return error_msg("PAM: unknown TUPLTYPE");
}
} else {
return error_msg("PAM: unknown header keyword");
}
}
if (num_channels !=
(header->has_alpha ? 1 : 0) + (header->is_gray ? 1 : 3)) {
return error_msg("PAM: bad DEPTH");
}
if (max_val == 0 || max_val >= 65536) {
return error_msg("PAM: bad MAXVAL");
}
header->bits_per_sample = Log2(max_val + 1);
*pos = pos_;
return true;
}
bool ParseHeaderPNM(HeaderPNM* header, const uint8_t** pos) {
return_on_error(SkipWhitespace());
return_on_error(ParseUnsigned(&header->xsize));
return_on_error(SkipWhitespace());
return_on_error(ParseUnsigned(&header->ysize));
return_on_error(SkipWhitespace());
size_t max_val;
return_on_error(ParseUnsigned(&max_val));
if (max_val == 0 || max_val >= 65536) {
return error_msg("PNM: bad MaxVal");
}
header->bits_per_sample = Log2(max_val + 1);
return_on_error(SkipSingleWhitespace());
*pos = pos_;
return true;
}
const uint8_t* pos_;
const uint8_t* const end_;
};
bool load_file(unsigned char** out, size_t* outsize, const char* filename) {
FILE* file;
file = fopen(filename, "rb");
if (!file) return false;
if (fseek(file, 0, SEEK_END) != 0) {
fclose(file);
return false;
}
*outsize = ftell(file);
if (*outsize == LONG_MAX || *outsize < 9 || fseek(file, 0, SEEK_SET)) {
fclose(file);
return false;
}
*out = (unsigned char*)malloc(*outsize);
if (!(*out)) return false;
size_t readsize;
readsize = fread(*out, 1, *outsize, file);
fclose(file);
if (readsize != *outsize) return false;
return true;
}
bool DecodePAM(const char* filename, uint8_t** buffer, size_t* w, size_t* h,
size_t* nb_chans, size_t* bitdepth) {
unsigned char* in_file;
size_t in_size;
if (!load_file(&in_file, &in_size, filename))
return error_msg("Could not read input file");
Parser parser(in_file, in_size);
HeaderPNM header = {};
const uint8_t* pos = nullptr;
if (!parser.ParseHeader(&header, &pos)) return false;
if (header.bits_per_sample == 0 || header.bits_per_sample > 16) {
return error_msg("PNM: bits_per_sample invalid (can do at most 16-bit)");
}
*w = header.xsize;
*h = header.ysize;
*bitdepth = header.bits_per_sample;
*nb_chans = (header.is_gray ? 1 : 3) + (header.has_alpha ? 1 : 0);
size_t pnm_remaining_size = in_file + in_size - pos;
size_t buffer_size = *w * *h * *nb_chans * (*bitdepth > 8 ? 2 : 1);
if (pnm_remaining_size < buffer_size) {
return error_msg("PNM file too small");
}
*buffer = (uint8_t*)malloc(buffer_size);
memcpy(*buffer, pos, buffer_size);
return true;
}

9
third_party/jpeg-xl/lib/extras/dec/apng.cc поставляемый
Просмотреть файл

@ -586,6 +586,7 @@ Status DecodeImageAPNG(const Span<const uint8_t> bytes,
std::vector<std::vector<uint8_t>> chunksInfo;
bool isAnimated = false;
bool hasInfo = false;
bool seenFctl = false;
APNGFrame frameRaw = {};
uint32_t num_channels;
JxlPixelFormat format;
@ -653,6 +654,7 @@ Status DecodeImageAPNG(const Span<const uint8_t> bytes,
while (!r.Eof()) {
id = read_chunk(&r, &chunk);
if (!id) break;
seenFctl |= (id == kId_fcTL);
if (id == kId_acTL && !hasInfo && !isAnimated) {
isAnimated = true;
@ -713,6 +715,10 @@ Status DecodeImageAPNG(const Span<const uint8_t> bytes,
}
} else if (id == kId_IDAT) {
// First IDAT chunk means we now have all header info
if (seenFctl) {
// `fcTL` chunk must appear after all `IDAT` chunks
return JXL_FAILURE("IDAT chunk after fcTL chunk");
}
hasInfo = true;
JXL_CHECK(w == png_get_image_width(png_ptr, info_ptr));
JXL_CHECK(h == png_get_image_height(png_ptr, info_ptr));
@ -780,6 +786,9 @@ Status DecodeImageAPNG(const Span<const uint8_t> bytes,
break;
}
} else if (id == kId_fdAT && isAnimated) {
if (!hasInfo) {
return JXL_FAILURE("fDAT chunk before iDAT");
}
png_save_uint_32(chunk.data() + 4, chunk.size() - 16);
memcpy(chunk.data() + 8, "IDAT", 4);
if (processing_data(png_ptr, info_ptr, chunk.data() + 4,

33
third_party/jpeg-xl/lib/extras/dec/color_hints.cc поставляемый
Просмотреть файл

@ -7,7 +7,10 @@
#include <jxl/encode.h>
#include <vector>
#include "lib/extras/dec/color_description.h"
#include "lib/jxl/base/status.h"
namespace jxl {
namespace extras {
@ -15,19 +18,15 @@ namespace extras {
Status ApplyColorHints(const ColorHints& color_hints,
const bool color_already_set, const bool is_gray,
PackedPixelFile* ppf) {
if (color_already_set) {
return color_hints.Foreach(
[](const std::string& key, const std::string& /*value*/) {
JXL_WARNING("Decoder ignoring %s hint", key.c_str());
return true;
});
}
bool got_color_space = false;
bool got_color_space = color_already_set;
JXL_RETURN_IF_ERROR(color_hints.Foreach(
[is_gray, ppf, &got_color_space](const std::string& key,
const std::string& value) -> Status {
[color_already_set, is_gray, ppf, &got_color_space](
const std::string& key, const std::string& value) -> Status {
if (color_already_set && (key == "color_space" || key == "icc")) {
JXL_WARNING("Decoder ignoring %s hint", key.c_str());
return true;
}
if (key == "color_space") {
JxlColorEncoding c_original_external;
if (!ParseDescription(value, &c_original_external)) {
@ -46,6 +45,18 @@ Status ApplyColorHints(const ColorHints& color_hints,
std::vector<uint8_t> icc(data, data + value.size());
ppf->icc.swap(icc);
got_color_space = true;
} else if (key == "exif") {
const uint8_t* data = reinterpret_cast<const uint8_t*>(value.data());
std::vector<uint8_t> blob(data, data + value.size());
ppf->metadata.exif.swap(blob);
} else if (key == "xmp") {
const uint8_t* data = reinterpret_cast<const uint8_t*>(value.data());
std::vector<uint8_t> blob(data, data + value.size());
ppf->metadata.xmp.swap(blob);
} else if (key == "jumbf") {
const uint8_t* data = reinterpret_cast<const uint8_t*>(value.data());
std::vector<uint8_t> blob(data, data + value.size());
ppf->metadata.jumbf.swap(blob);
} else {
JXL_WARNING("Ignoring %s hint", key.c_str());
}

2
third_party/jpeg-xl/lib/extras/dec/color_hints.h поставляемый
Просмотреть файл

@ -10,6 +10,8 @@
// information into the file, and those that support it may not have it.
// To allow attaching color information to those file formats the caller can
// define these color hints.
// Besides color space information, 'ColorHints' may also include other
// additional information such as Exif, XMP and JUMBF metadata.
#include <stddef.h>
#include <stdint.h>

8
third_party/jpeg-xl/lib/extras/dec/decode.cc поставляемый
Просмотреть файл

@ -120,9 +120,15 @@ Status DecodeBytes(const Span<const uint8_t> bytes,
return Codec::kPNM;
}
JXLDecompressParams dparams = {};
for (const uint32_t num_channels : {1, 2, 3, 4}) {
dparams.accepted_formats.push_back(
{num_channels, JXL_TYPE_FLOAT, JXL_LITTLE_ENDIAN, /*align=*/0});
}
size_t decoded_bytes;
if (DecodeImageJXL(bytes.data(), bytes.size(), dparams, &decoded_bytes,
ppf)) {
ppf) &&
ApplyColorHints(color_hints, true, ppf->info.num_color_channels == 1,
ppf)) {
return Codec::kJXL;
}
if (DecodeImageGIF(bytes, color_hints, ppf, constraints)) {

14
third_party/jpeg-xl/lib/extras/dec/jxl.cc поставляемый
Просмотреть файл

@ -125,12 +125,7 @@ bool DecodeImageJXL(const uint8_t* bytes, size_t bytes_size,
JxlPixelFormat format;
std::vector<JxlPixelFormat> accepted_formats = dparams.accepted_formats;
if (accepted_formats.empty()) {
for (const uint32_t num_channels : {1, 2, 3, 4}) {
accepted_formats.push_back(
{num_channels, JXL_TYPE_FLOAT, JXL_LITTLE_ENDIAN, /*align=*/0});
}
}
JxlColorEncoding color_encoding;
size_t num_color_channels = 0;
if (!dparams.color_space.empty()) {
@ -169,6 +164,10 @@ bool DecodeImageJXL(const uint8_t* bytes, size_t bytes_size,
} else {
events |= (JXL_DEC_COLOR_ENCODING | JXL_DEC_FRAME | JXL_DEC_PREVIEW_IMAGE |
JXL_DEC_BOX);
if (accepted_formats.empty()) {
// decoding just the metadata, not the pixel data
events ^= (JXL_DEC_FULL_IMAGE | JXL_DEC_PREVIEW_IMAGE);
}
}
if (JXL_DEC_SUCCESS != JxlDecoderSubscribeEvents(dec, events)) {
fprintf(stderr, "JxlDecoderSubscribeEvents failed\n");
@ -206,7 +205,7 @@ bool DecodeImageJXL(const uint8_t* bytes, size_t bytes_size,
return false;
}
uint32_t progression_index = 0;
bool codestream_done = false;
bool codestream_done = accepted_formats.empty();
BoxProcessor boxes(dec);
for (;;) {
JxlDecoderStatus status = JxlDecoderProcessInput(dec);
@ -285,6 +284,7 @@ bool DecodeImageJXL(const uint8_t* bytes, size_t bytes_size,
fprintf(stderr, "JxlDecoderGetBasicInfo failed\n");
return false;
}
if (accepted_formats.empty()) continue;
if (num_color_channels != 0) {
// Mark the change in number of color channels due to the requested
// color space.

28
third_party/jpeg-xl/lib/extras/enc/apng.cc поставляемый
Просмотреть файл

@ -191,9 +191,9 @@ void MaybeAddCICP(const JxlColorEncoding& c_enc, png_structp png_ptr,
cicp_data[3] = 1;
cicp_chunk.data = cicp_data;
cicp_chunk.size = sizeof(cicp_data);
cicp_chunk.location = PNG_HAVE_PLTE;
cicp_chunk.location = PNG_HAVE_IHDR;
memcpy(cicp_chunk.name, "cICP", 5);
png_set_keep_unknown_chunks(png_ptr, 3,
png_set_keep_unknown_chunks(png_ptr, PNG_HANDLE_CHUNK_ALWAYS,
reinterpret_cast<const png_byte*>("cICP"), 1);
png_set_unknown_chunks(png_ptr, info_ptr, &cicp_chunk, 1);
}
@ -243,6 +243,28 @@ void MaybeAddGAMA(const JxlColorEncoding& c_enc, png_structp png_ptr,
}
}
void MaybeAddCLLi(const JxlColorEncoding& c_enc, const float intensity_target,
png_structp png_ptr, png_infop info_ptr) {
if (c_enc.transfer_function != JXL_TRANSFER_FUNCTION_PQ) return;
const uint32_t max_cll =
static_cast<uint32_t>(10000.f * Clamp1(intensity_target, 0.f, 10000.f));
png_byte chunk_data[8] = {};
chunk_data[0] = (max_cll >> 24) & 0xFF;
chunk_data[1] = (max_cll >> 16) & 0xFF;
chunk_data[2] = (max_cll >> 8) & 0xFF;
chunk_data[3] = max_cll & 0xFF;
// Leave MaxFALL set to 0.
png_unknown_chunk chunk;
memcpy(chunk.name, "cLLi", 5);
chunk.data = chunk_data;
chunk.size = sizeof chunk_data;
chunk.location = PNG_HAVE_IHDR;
png_set_keep_unknown_chunks(png_ptr, PNG_HANDLE_CHUNK_ALWAYS,
reinterpret_cast<const png_byte*>("cLLi"), 1);
png_set_unknown_chunks(png_ptr, info_ptr, &chunk, 1);
}
Status APNGEncoder::EncodePackedPixelFileToAPNG(
const PackedPixelFile& ppf, ThreadPool* pool,
std::vector<uint8_t>* bytes) const {
@ -332,6 +354,8 @@ Status APNGEncoder::EncodePackedPixelFileToAPNG(
MaybeAddCHRM(ppf.color_encoding, png_ptr, info_ptr);
MaybeAddGAMA(ppf.color_encoding, png_ptr, info_ptr);
}
MaybeAddCLLi(ppf.color_encoding, ppf.info.intensity_target, png_ptr,
info_ptr);
std::vector<std::string> textstrings;
JXL_RETURN_IF_ERROR(BlobsWriterPNG::Encode(ppf.metadata, &textstrings));

27
third_party/jpeg-xl/lib/extras/enc/encode.cc поставляемый
Просмотреть файл

@ -129,6 +129,27 @@ Status SelectFormat(const std::vector<JxlPixelFormat>& accepted_formats,
return true;
}
template <int metadata>
class MetadataEncoder : public Encoder {
public:
std::vector<JxlPixelFormat> AcceptedFormats() const override {
std::vector<JxlPixelFormat> formats;
// empty, i.e. no need for actual pixel data
return formats;
}
Status Encode(const PackedPixelFile& ppf, EncodedImage* encoded,
ThreadPool* pool) const override {
JXL_RETURN_IF_ERROR(VerifyBasicInfo(ppf.info));
encoded->icc.clear();
encoded->bitstreams.resize(1);
if (metadata == 0) encoded->bitstreams.front() = ppf.metadata.exif;
if (metadata == 1) encoded->bitstreams.front() = ppf.metadata.xmp;
if (metadata == 2) encoded->bitstreams.front() = ppf.metadata.jumbf;
return true;
}
};
std::unique_ptr<Encoder> Encoder::FromExtension(std::string extension) {
std::transform(
extension.begin(), extension.end(), extension.begin(),
@ -143,6 +164,12 @@ std::unique_ptr<Encoder> Encoder::FromExtension(std::string extension) {
if (extension == ".ppm") return GetPPMEncoder();
if (extension == ".pfm") return GetPFMEncoder();
if (extension == ".exr") return GetEXREncoder();
if (extension == ".exif") return jxl::make_unique<MetadataEncoder<0>>();
if (extension == ".xmp") return jxl::make_unique<MetadataEncoder<1>>();
if (extension == ".xml") return jxl::make_unique<MetadataEncoder<1>>();
if (extension == ".jumbf") return jxl::make_unique<MetadataEncoder<2>>();
if (extension == ".jumb") return jxl::make_unique<MetadataEncoder<2>>();
return nullptr;
}

2
third_party/jpeg-xl/lib/extras/enc/encode.h поставляемый
Просмотреть файл

@ -43,6 +43,8 @@ class Encoder {
virtual ~Encoder() = default;
// Set of pixel formats that this encoder takes as input.
// If empty, the 'encoder' does not need any pixels (it's metadata-only).
virtual std::vector<JxlPixelFormat> AcceptedFormats() const = 0;
// Any existing data in encoded_image is discarded.

35
third_party/jpeg-xl/lib/extras/enc/jxl.cc поставляемый
Просмотреть файл

@ -5,6 +5,7 @@
#include "lib/extras/enc/jxl.h"
#include <jxl/encode.h>
#include <jxl/encode_cxx.h>
#include "lib/jxl/exif.h"
@ -87,9 +88,35 @@ bool EncodeImageJXL(const JXLCompressParams& params, const PackedPixelFile& ppf,
fprintf(stderr, "Storing JPEG metadata failed.\n");
return false;
}
if (!params.jpeg_store_metadata && params.jpeg_strip_exif) {
JxlEncoderFrameSettingsSetOption(settings,
JXL_ENC_FRAME_SETTING_JPEG_KEEP_EXIF, 0);
}
if (!params.jpeg_store_metadata && params.jpeg_strip_xmp) {
JxlEncoderFrameSettingsSetOption(settings,
JXL_ENC_FRAME_SETTING_JPEG_KEEP_XMP, 0);
}
if (params.jpeg_strip_jumbf) {
JxlEncoderFrameSettingsSetOption(
settings, JXL_ENC_FRAME_SETTING_JPEG_KEEP_JUMBF, 0);
}
if (JXL_ENC_SUCCESS != JxlEncoderAddJPEGFrame(settings, jpeg_bytes->data(),
jpeg_bytes->size())) {
fprintf(stderr, "JxlEncoderAddJPEGFrame() failed.\n");
JxlEncoderError error = JxlEncoderGetError(enc);
if (error == JXL_ENC_ERR_BAD_INPUT) {
fprintf(stderr,
"Error while decoding the JPEG image. It may be corrupt (e.g. "
"truncated) or of an unsupported type (e.g. CMYK).\n");
} else if (error == JXL_ENC_ERR_JBRD) {
fprintf(stderr,
"JPEG bitstream reconstruction data could not be created. "
"Possibly there is too much tail data.\n"
"Try using --jpeg_store_metadata 0, to losslessly "
"recompress the JPEG image data without bitstream "
"reconstruction data.\n");
} else {
fprintf(stderr, "JxlEncoderAddJPEGFrame() failed.\n");
}
return false;
}
} else {
@ -120,6 +147,12 @@ bool EncodeImageJXL(const JXLCompressParams& params, const PackedPixelFile& ppf,
fprintf(stderr, "JxlEncoderSetBasicInfo() failed.\n");
return false;
}
if (JXL_ENC_SUCCESS !=
JxlEncoderSetUpsamplingMode(enc, params.already_downsampled,
params.upsampling_mode)) {
fprintf(stderr, "JxlEncoderSetUpsamplingMode() failed.\n");
return false;
}
if (JXL_ENC_SUCCESS !=
JxlEncoderSetFrameBitDepth(settings, &params.input_bitdepth)) {
fprintf(stderr, "JxlEncoderSetFrameBitDepth() failed.\n");

4
third_party/jpeg-xl/lib/extras/enc/jxl.h поставляемый
Просмотреть файл

@ -43,12 +43,16 @@ struct JXLCompressParams {
bool use_container = false;
// Whether to enable/disable byte-exact jpeg reconstruction for jpeg inputs.
bool jpeg_store_metadata = true;
bool jpeg_strip_exif = false;
bool jpeg_strip_xmp = false;
bool jpeg_strip_jumbf = false;
// Whether to create brob boxes.
bool compress_boxes = true;
// Upper bound on the intensity level present in the image in nits (zero means
// that the library chooses a default).
float intensity_target = 0;
int already_downsampled = 1;
int upsampling_mode = -1;
// Overrides for bitdepth, codestream level and alpha premultiply.
size_t override_bitdepth = 0;
int32_t codestream_level = -1;

2
third_party/jpeg-xl/lib/extras/exif.cc поставляемый
Просмотреть файл

@ -23,7 +23,7 @@ void ResetExifOrientation(std::vector<uint8_t>& exif) {
return; // not a valid tiff header
}
t += 4;
uint32_t offset = (bigendian ? LoadBE32(t) : LoadLE32(t));
uint64_t offset = (bigendian ? LoadBE32(t) : LoadLE32(t));
if (exif.size() < 12 + offset + 2 || offset < 8) return;
t += offset - 4;
uint16_t nb_tags = (bigendian ? LoadBE16(t) : LoadLE16(t));

44
third_party/jpeg-xl/lib/include/jxl/encode.h поставляемый
Просмотреть файл

@ -349,6 +349,30 @@ typedef enum {
*/
JXL_ENC_FRAME_SETTING_BUFFERING = 34,
/** Keep or discard Exif metadata boxes derived from a JPEG frame when using
* JxlEncoderAddJPEGFrame. This has no effect on boxes added using
* JxlEncoderAddBox. When JxlEncoderStoreJPEGMetadata is set to 1, this option
* cannot be set to 0. Even when Exif metadata is discarded, the orientation
* will still be applied. 0 = discard Exif metadata, 1 = keep Exif metadata
* (default).
*/
JXL_ENC_FRAME_SETTING_JPEG_KEEP_EXIF = 35,
/** Keep or discard XMP metadata boxes derived from a JPEG frame when using
* JxlEncoderAddJPEGFrame. This has no effect on boxes added using
* JxlEncoderAddBox. When JxlEncoderStoreJPEGMetadata is set to 1, this option
* cannot be set to 0. 0 = discard XMP metadata, 1 = keep XMP metadata
* (default).
*/
JXL_ENC_FRAME_SETTING_JPEG_KEEP_XMP = 36,
/** Keep or discard JUMBF metadata boxes derived from a JPEG frame when using
* JxlEncoderAddJPEGFrame. This has no effect on boxes added using
* JxlEncoderAddBox. 0 = discard JUMBF metadata, 1 = keep JUMBF metadata
* (default).
*/
JXL_ENC_FRAME_SETTING_JPEG_KEEP_JUMBF = 37,
/** Enum value not to be used as an option. This value is added to force the
* C compiler to have the enum to take a known size.
*/
@ -932,6 +956,26 @@ JXL_EXPORT void JxlEncoderInitBlendInfo(JxlBlendInfo* blend_info);
JXL_EXPORT JxlEncoderStatus JxlEncoderSetBasicInfo(JxlEncoder* enc,
const JxlBasicInfo* info);
/**
* Sets the upsampling method the decoder will use in case there are frames
* with JXL_ENC_FRAME_SETTING_RESAMPLING set. This is useful in combination
* with the JXL_ENC_FRAME_SETTING_ALREADY_DOWNSAMPLED option, to control the
* type of upsampling that will be used.
*
* @param enc encoder object.
* @param factor upsampling factor to configure (1, 2, 4 or 8; for 1 this
* function has no effect at all)
* @param mode upsampling mode to use for this upsampling:
* -1: default (good for photographic images, no signaling overhead)
* 0: nearest neighbor (good for pixel art)
* 1: 'pixel dots' (same as NN for 2x, diamond-shaped 'pixel dots' for 4x/8x)
* @return JXL_ENC_SUCCESS if the operation was successful,
* JXL_ENC_ERROR or JXL_ENC_NOT_SUPPORTED otherwise
*/
JXL_EXPORT JxlEncoderStatus JxlEncoderSetUpsamplingMode(JxlEncoder* enc,
const int64_t factor,
const int64_t mode);
/**
* Initializes a JxlExtraChannelInfo struct to default values.
* For forwards-compatibility, this function has to be called before values

21
third_party/jpeg-xl/lib/jxl/dec_ans.h поставляемый
Просмотреть файл

@ -305,8 +305,8 @@ class ANSSymbolReader {
// Takes a *clustered* idx. Inlined, for use in hot paths.
template <bool uses_lz77>
JXL_INLINE size_t ReadHybridUintClustered(size_t ctx,
BitReader* JXL_RESTRICT br) {
JXL_INLINE size_t ReadHybridUintClusteredInlined(size_t ctx,
BitReader* JXL_RESTRICT br) {
if (uses_lz77) {
if (JXL_UNLIKELY(num_to_copy_ > 0)) {
size_t ret = lz77_window_[(copy_pos_++) & kWindowMask];
@ -363,6 +363,23 @@ class ANSSymbolReader {
return ret;
}
// same but not inlined
template <bool uses_lz77>
size_t ReadHybridUintClustered(size_t ctx, BitReader* JXL_RESTRICT br) {
return ReadHybridUintClusteredInlined<uses_lz77>(ctx, br);
}
// inlined only in the no-lz77 case
template <bool uses_lz77>
JXL_INLINE size_t
ReadHybridUintClusteredMaybeInlined(size_t ctx, BitReader* JXL_RESTRICT br) {
if (uses_lz77) {
return ReadHybridUintClustered<uses_lz77>(ctx, br);
} else {
return ReadHybridUintClusteredInlined<uses_lz77>(ctx, br);
}
}
// inlined, for use in hot paths
template <bool uses_lz77>
JXL_INLINE size_t

46
third_party/jpeg-xl/lib/jxl/dec_tone_mapping-inl.h поставляемый
Просмотреть файл

@ -63,15 +63,14 @@ class Rec2408ToneMapper {
Min(Set(df_, target_range_.second),
ZeroIfNegative(
Mul(Set(df_, 10000), TF_PQ().DisplayFromEncoded(df_, e4))));
const V ratio = Div(new_luminance, luminance);
const V inv_target_peak = Set(df_, inv_target_peak_);
const V min_luminance = Set(df_, 1e-6f);
const auto use_cap = Le(luminance, min_luminance);
const V ratio = Div(new_luminance, Max(luminance, min_luminance));
const V cap = Mul(new_luminance, Set(df_, inv_target_peak_));
const V normalizer = Set(df_, normalizer_);
const V multiplier = Mul(ratio, normalizer);
for (V* const val : {red, green, blue}) {
*val = IfThenElse(Le(luminance, Set(df_, 1e-6f)),
Mul(new_luminance, inv_target_peak),
Mul(*val, multiplier));
*val = IfThenElse(use_cap, cap, Mul(*val, multiplier));
}
}
@ -197,30 +196,37 @@ void GamutMap(V* red, V* green, V* blue, const float primaries_luminances[3],
// That will reduce its luminance.
// - For luminance preservation, getting all components below 1 is
// done by mixing in yet more gray. That will desaturate it further.
V gray_mix_saturation = Zero(df);
V gray_mix_luminance = Zero(df);
const V zero = Zero(df);
const V one = Set(df, 1);
V gray_mix_saturation = zero;
V gray_mix_luminance = zero;
for (const V* ch : {red, green, blue}) {
const V& val = *ch;
const V inv_val_minus_gray = Div(Set(df, 1), (Sub(val, luminance)));
const V val_minus_gray = Sub(val, luminance);
const V inv_val_minus_gray =
Div(one, IfThenElse(Eq(val_minus_gray, zero), one, val_minus_gray));
const V val_over_val_minus_gray = Mul(val, inv_val_minus_gray);
gray_mix_saturation =
IfThenElse(Ge(val, luminance), gray_mix_saturation,
Max(gray_mix_saturation, Mul(val, inv_val_minus_gray)));
IfThenElse(Ge(val_minus_gray, zero), gray_mix_saturation,
Max(gray_mix_saturation, val_over_val_minus_gray));
gray_mix_luminance =
Max(gray_mix_luminance,
IfThenElse(Le(val, luminance), gray_mix_saturation,
Mul(Sub(val, Set(df, 1)), inv_val_minus_gray)));
IfThenElse(Le(val_minus_gray, zero), gray_mix_saturation,
Sub(val_over_val_minus_gray, inv_val_minus_gray)));
}
const V gray_mix = Clamp(
MulAdd(Set(df, preserve_saturation),
Sub(gray_mix_saturation, gray_mix_luminance), gray_mix_luminance),
Zero(df), Set(df, 1));
for (V* const val : {red, green, blue}) {
*val = MulAdd(gray_mix, Sub(luminance, *val), *val);
zero, one);
for (V* const ch : {red, green, blue}) {
V& val = *ch;
val = MulAdd(gray_mix, Sub(luminance, val), val);
}
const V normalizer =
Div(Set(df, 1), Max(Set(df, 1), Max(*red, Max(*green, *blue))));
for (V* const val : {red, green, blue}) {
*val = Mul(*val, normalizer);
const V max_clr = Max(Max(one, *red), Max(*green, *blue));
const V normalizer = Div(one, max_clr);
for (V* const ch : {red, green, blue}) {
V& val = *ch;
val = Mul(val, normalizer);
}
}

8
third_party/jpeg-xl/lib/jxl/decode_test.cc поставляемый
Просмотреть файл

@ -1666,9 +1666,9 @@ TEST(DecodeTest, PixelTestWithICCProfileLossy) {
EXPECT_THAT(ButteraugliDistance(io0.frames, io1.frames, ba, jxl::GetJxlCms(),
/*distmap=*/nullptr, nullptr),
#if JXL_HIGH_PRECISION
IsSlightlyBelow(0.70f));
IsSlightlyBelow(0.9f));
#else
IsSlightlyBelow(0.78f));
IsSlightlyBelow(0.98f));
#endif
JxlDecoderDestroy(dec);
@ -1928,9 +1928,9 @@ TEST(DecodeTest, PixelTestOpaqueSrgbLossy) {
ButteraugliDistance(io0.frames, io1.frames, ba, jxl::GetJxlCms(),
/*distmap=*/nullptr, nullptr),
#if JXL_HIGH_PRECISION
IsSlightlyBelow(0.74f));
IsSlightlyBelow(0.93f));
#else
IsSlightlyBelow(0.75f));
IsSlightlyBelow(0.94f));
#endif
JxlDecoderDestroy(dec);

121
third_party/jpeg-xl/lib/jxl/enc_ac_strategy.cc поставляемый
Просмотреть файл

@ -378,6 +378,41 @@ static const float kChromaErrorWeight[AcStrategy::kNumValidStrategies] = {
2.0f, // DCT128X256 = 26,
};
// For DCT the maximum error is roughly a sum of the values.
// For some transforms, especially IDENTITY and DCT2X2, not all
// the coefficients affect the maximum error. Probably would
// be better to do transforms back and forth and look at the pixels
// but that would significantly slow down the computation.
static const float kMixLossTable[AcStrategy::kNumValidStrategies] = {
1.0f, // DCT = 0,
0.45f, // IDENTITY = 1,
0.45f, // DCT2X2 = 2,
0.7f, // DCT4X4 = 3,
1.0f, // DCT16X16 = 4,
1.0f, // DCT32X32 = 5,
1.0f, // DCT16X8 = 6,
1.0f, // DCT8X16 = 7,
1.0f, // DCT32X8 = 8,
1.0f, // DCT8X32 = 9,
1.0f, // DCT32X16 = 10,
1.0f, // DCT16X32 = 11,
0.96f, // DCT4X8 = 12,
0.96f, // DCT8X4 = 13,
0.94f, // AFV0 = 14,
0.94f, // AFV1 = 15,
0.94f, // AFV2 = 16,
0.94f, // AFV3 = 17,
1.0f, // DCT64X64 = 18,
1.0f, // DCT64X32 = 19,
1.0f, // DCT32X64 = 20,
1.0f, // DCT128X128 = 21,
1.0f, // DCT128X64 = 22,
1.0f, // DCT64X128 = 23,
1.0f, // DCT256X256 = 24,
1.0f, // DCT256X128 = 25,
1.0f, // DCT128X256 = 26,
};
float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
const ACSConfig& config,
const float* JXL_RESTRICT cmap_factors, float* block,
@ -390,7 +425,6 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
TransformFromPixels(acs.Strategy(), &config.Pixel(c, x, y),
config.src_stride, block_c, scratch_space);
}
HWY_FULL(float) df;
const size_t num_blocks = acs.covered_blocks_x() * acs.covered_blocks_y();
@ -401,7 +435,12 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
if (num_blocks == 1) {
// When it is only one 8x8, we don't need aggregation of values.
quant_norm8 = config.Quant(x / 8, y / 8);
masking = 2.0f * config.Masking(x / 8, y / 8);
masking = config.Masking(x / 8, y / 8);
// Make DCT2X2 more favored when area is exposed.
float kExposedMasking = 0.118f;
if (acs.RawStrategy() == 2 && masking >= kExposedMasking) {
masking = kExposedMasking + 0.56 * (masking - kExposedMasking);
}
} else if (num_blocks == 2) {
// Taking max instead of 8th norm seems to work
// better for smallest blocks up to 16x8. Jyrki couldn't get
@ -409,13 +448,13 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
if (acs.covered_blocks_y() == 2) {
quant_norm8 =
std::max(config.Quant(x / 8, y / 8), config.Quant(x / 8, y / 8 + 1));
masking = 2.0f * std::max(config.Masking(x / 8, y / 8),
config.Masking(x / 8, y / 8 + 1));
masking = std::max(config.Masking(x / 8, y / 8),
config.Masking(x / 8, y / 8 + 1));
} else {
quant_norm8 =
std::max(config.Quant(x / 8, y / 8), config.Quant(x / 8 + 1, y / 8));
masking = 2.0f * std::max(config.Masking(x / 8, y / 8),
config.Masking(x / 8 + 1, y / 8));
masking = std::max(config.Masking(x / 8, y / 8),
config.Masking(x / 8 + 1, y / 8));
}
} else {
float masking_norm2 = 0;
@ -438,12 +477,12 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
quant_norm8 = FastPowf(quant_norm8, 1.0f / 8.0f);
masking_norm2 = sqrt(masking_norm2 / num_blocks);
// This is a highly empirical formula.
masking = (masking_norm2 + masking_max);
masking = 0.5 * (masking_norm2 + masking_max);
}
const auto q = Set(df, quant_norm8);
// Compute entropy.
float entropy = config.base_entropy;
float entropy = 0.0f;
auto info_loss = Zero(df);
auto info_loss2 = Zero(df);
@ -453,9 +492,6 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
auto entropy_v = Zero(df);
auto nzeros_v = Zero(df);
auto cost1 = Set(df, config.cost1);
auto cost2 = Set(df, config.cost2);
auto cost_delta = Set(df, config.cost_delta);
for (size_t i = 0; i < num_blocks * kDCTBlockSize; i += Lanes(df)) {
const auto in = Load(df, block + c * size + i);
const auto in_y = Mul(Load(df, block + size + i), cmap_factor);
@ -467,18 +503,13 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
info_loss2 = MulAdd(diff, diff, info_loss2);
const auto q = Abs(rval);
const auto q_is_zero = Eq(q, Zero(df));
entropy_v = Add(entropy_v, IfThenElseZero(Ge(q, Set(df, 1.5f)), cost2));
// We used to have q * C here, but that cost model seems to
// be punishing large values more than necessary. Sqrt tries
// to avoid large values less aggressively. Having high accuracy
// around zero is most important at low qualities, and there
// we have directly specified costs for 0, 1, and 2.
entropy_v = MulAdd(Sqrt(q), cost_delta, entropy_v);
// to avoid large values less aggressively.
entropy_v = Add(Sqrt(q), entropy_v);
nzeros_v = Add(nzeros_v, IfThenZeroElse(q_is_zero, Set(df, 1.0f)));
}
entropy_v = MulAdd(nzeros_v, cost1, entropy_v);
entropy += cmul[c] * GetLane(SumOfLanes(df, entropy_v));
entropy += config.cost_delta * cmul[c] * GetLane(SumOfLanes(df, entropy_v));
size_t num_nzeros = GetLane(SumOfLanes(df, nzeros_v));
// Add #bit of num_nonzeros, as an estimate of the cost for encoding the
// number of non-zeros of the block.
@ -487,13 +518,16 @@ float EstimateEntropy(const AcStrategy& acs, size_t x, size_t y,
// bias.
entropy += config.zeros_mul * (CeilLog2Nonzero(nbits + 17) + nbits);
}
float ret =
entropy +
masking *
((config.info_loss_multiplier * GetLane(SumOfLanes(df, info_loss))) +
(config.info_loss_multiplier2 *
sqrt(num_blocks * GetLane(SumOfLanes(df, info_loss2)))));
return ret;
const float kMixLoss = kMixLossTable[acs.RawStrategy()];
const float loss1 = GetLane(SumOfLanes(df, info_loss));
const float loss2 =
sqrt(GetLane(SumOfLanes(df, info_loss2)) * (num_blocks * 64));
const float loss = kMixLoss * (config.info_loss_multiplier * loss1) +
(1.0 - kMixLoss) * (config.info_loss_multiplier2 * loss2);
const float kRegulateSurface = 11.5f;
float large_surface_error_mul =
(kRegulateSurface + sqrt(num_blocks)) * (1.0f / (kRegulateSurface + 1));
return entropy + large_surface_error_mul * masking * loss;
}
uint8_t FindBest8x8Transform(size_t x, size_t y, int encoding_speed_tier,
@ -513,7 +547,7 @@ uint8_t FindBest8x8Transform(size_t x, size_t y, int encoding_speed_tier,
AcStrategy::Type::DCT,
9,
3.0f,
0.745f,
0.785f,
},
{
AcStrategy::Type::DCT4X4,
@ -525,19 +559,19 @@ uint8_t FindBest8x8Transform(size_t x, size_t y, int encoding_speed_tier,
AcStrategy::Type::DCT2X2,
5,
0.0f,
0.66f,
0.685f,
},
{
AcStrategy::Type::DCT4X8,
4,
0.0f,
0.700754622182473063f,
3.0f,
0.745f,
},
{
AcStrategy::Type::DCT8X4,
4,
0.0f,
0.700754622182473063f,
3.0f,
0.745f,
},
{
AcStrategy::Type::IDENTITY,
@ -871,14 +905,14 @@ void ProcessRectACS(PassesEncoderState* JXL_RESTRICT enc_state,
float entropy_mul;
};
static const float k8X16mul1 = -0.55;
static const float k8X16mul2 = 0.865;
static const float k8X16mul2 = 0.885;
static const float k8X16base = 1.6;
const float entropy_mul16X8 =
k8X16mul2 + k8X16mul1 / (butteraugli_target + k8X16base);
// const float entropy_mul16X8 = mul8X16 * 0.91195782912371126f;
static const float k16X16mul1 = -0.35;
static const float k16X16mul2 = 0.798;
static const float k16X16mul2 = 0.808;
static const float k16X16base = 2.0;
const float entropy_mul16X16 =
k16X16mul2 + k16X16mul1 / (butteraugli_target + k16X16base);
@ -1067,7 +1101,6 @@ void AcStrategyHeuristics::Init(const Image3F& src,
this->enc_state = enc_state;
config.dequant = &enc_state->shared.matrices;
const CompressParams& cparams = enc_state->cparams;
const float butteraugli_target = cparams.butteraugli_distance;
if (cparams.speed_tier >= SpeedTier::kCheetah) {
JXL_CHECK(enc_state->shared.matrices.EnsureComputed(1)); // DCT8 only
@ -1098,20 +1131,10 @@ void AcStrategyHeuristics::Init(const Image3F& src,
// - estimate of the number of bits that will be used by the block
// - information loss due to quantization
// The following constant controls the relative weights of these components.
config.info_loss_multiplier = 138.0f;
config.info_loss_multiplier2 = 50.46839691767866;
// TODO(jyrki): explore base_entropy setting more.
// A small value (0?) works better at high distance, while a larger value
// may be more effective at low distance/high bpp.
config.base_entropy = 0.0;
config.zeros_mul = 7.565053364251793f;
// Lots of +1 and -1 coefficients at high quality, it is
// beneficial to favor them. At low qualities zeros matter more
// and +1 / -1 coefficients are already quite harmful.
float slope = std::min<float>(1.0f, butteraugli_target * (1.0f / 3));
config.cost1 = 1 + slope * 8.8703248061477744f;
config.cost2 = 4.4628149885273363f;
config.cost_delta = 5.3359184934516337f;
config.info_loss_multiplier = 58.67516723857484f;
config.info_loss_multiplier2 = 43.0f;
config.zeros_mul = 2.55f;
config.cost_delta = 4.9425062806007478f;
JXL_ASSERT(enc_state->shared.ac_strategy.xsize() ==
enc_state->shared.frame_dim.xsize_blocks);
JXL_ASSERT(enc_state->shared.ac_strategy.ysize() ==

5
third_party/jpeg-xl/lib/jxl/enc_ac_strategy.h поставляемый
Просмотреть файл

@ -37,12 +37,7 @@ struct ACSConfig {
size_t masking_field_stride;
const float* JXL_RESTRICT src_rows[3];
size_t src_stride;
// Cost for 1 (-1), 2 (-2) explicitly, cost for others computed with cost1 +
// cost2 + sqrt(q) * cost_delta.
float cost1;
float cost2;
float cost_delta;
float base_entropy;
float zeros_mul;
const float& Pixel(size_t c, size_t x, size_t y) const {
return src_rows[c][y * src_stride + x];

2
third_party/jpeg-xl/lib/jxl/enc_adaptive_quantization.cc поставляемый
Просмотреть файл

@ -736,7 +736,7 @@ ImageF TileDistMap(const ImageF& distmap, int tile_size, int margin,
static const float kDcQuantPow = 0.83;
static const float kDcQuant = 1.095924047623553f;
static const float kAcQuant = 0.7784;
static const float kAcQuant = 0.7635;
// Computes the decoded image for a given set of compression parameters.
ImageBundle RoundtripImage(const Image3F& opsin, PassesEncoderState* enc_state,

50
third_party/jpeg-xl/lib/jxl/enc_group.cc поставляемый
Просмотреть файл

@ -209,39 +209,39 @@ void AdjustQuantBlockAC(const Quantizer& quantizer, size_t c,
{
static const double kMul1[3][3] = {
{
0.30628347689416235,
0.19096514988140451,
0.10092267072278764,
0.13289977307244785,
0.13991489841351781,
0.083900681804010419,
},
{
0.68175730483344243,
0.19038660767376803,
0.14069887255219371,
0.69938583107168562,
0.19612117586770869,
0.15307492924107463,
},
{
0.74599469660659012,
0.10465705596003883,
0.075491104183520744,
0.099160801461836312,
0.16684944507307059,
0.16608517854968413,
},
};
static const double kMul2[3][3] = {
{
0.022707896753424779,
0.84465309720205983,
5.2275313293658812,
0.24773711435293466,
0.65189637683223112,
1.0,
},
{
0.17545973555482378,
0.97395015736868384,
1.9659234163151995,
0.46465181913392556,
0.3142440606068525,
0.30128806880068809,
},
{
0.75243833661051895,
1.7774383804879366,
0.3793181712352986,
0.45203398366713637,
0.15063329382779103,
0.067846407329923752,
},
};
const float kQuantNormalizer = 2.9037220690527175;
const float kQuantNormalizer = 2.8261379721245263;
sum_of_error *= kQuantNormalizer;
sum_of_vals *= kQuantNormalizer;
if (quant_kind >= AcStrategy::Type::DCT16X16) {
@ -252,9 +252,15 @@ void AdjustQuantBlockAC(const Quantizer& quantizer, size_t c,
} else if (quant_kind == AcStrategy::Type::DCT16X16) {
ix = 0;
}
if (sum_of_error > kMul1[ix][c] * xsize * ysize * kBlockDim * kBlockDim &&
sum_of_error > kMul2[ix][c] * sum_of_vals) {
*quant += 1;
int step =
sum_of_error / (kMul1[ix][c] * xsize * ysize * kBlockDim * kBlockDim +
kMul2[ix][c] * sum_of_vals);
if (step >= 2) {
step = 2;
}
if (sum_of_error > kMul1[ix][c] * xsize * ysize * kBlockDim * kBlockDim +
kMul2[ix][c] * sum_of_vals) {
*quant += step;
if (*quant >= Quantizer::kQuantMax) {
*quant = Quantizer::kQuantMax - 1;
}

5
third_party/jpeg-xl/lib/jxl/enc_params.h поставляемый
Просмотреть файл

@ -122,6 +122,11 @@ struct CompressParams {
// Use brotli compression for any boxes derived from a JPEG frame.
bool jpeg_compress_boxes = true;
// Preserve this metadata when doing JPEG recompression.
bool jpeg_keep_exif = true;
bool jpeg_keep_xmp = true;
bool jpeg_keep_jumbf = true;
// Set the noise to what it would approximately be if shooting at the nominal
// exposure for a given ISO setting on a 35mm camera.
float photon_noise_iso = 0;

76
third_party/jpeg-xl/lib/jxl/encode.cc поставляемый
Просмотреть файл

@ -802,6 +802,7 @@ JxlEncoderStatus JxlEncoderSetBasicInfo(JxlEncoder* enc,
info->exponent_bits_per_sample)) {
return JXL_API_ERROR(enc, JXL_ENC_ERR_API_USAGE, "Invalid bit depth");
}
enc->metadata.m.bit_depth.bits_per_sample = info->bits_per_sample;
enc->metadata.m.bit_depth.exponent_bits_per_sample =
info->exponent_bits_per_sample;
@ -919,6 +920,55 @@ void JxlEncoderInitExtraChannelInfo(JxlExtraChannelType type,
info->cfa_channel = 0;
}
JXL_EXPORT JxlEncoderStatus JxlEncoderSetUpsamplingMode(JxlEncoder* enc,
const int64_t factor,
const int64_t mode) {
// for convenience, allow calling this with factor 1 and just make it a no-op
if (factor == 1) return JXL_ENC_SUCCESS;
if (factor != 2 && factor != 4 && factor != 8)
return JXL_API_ERROR(enc, JXL_ENC_ERR_API_USAGE,
"Invalid upsampling factor");
if (mode < -1)
return JXL_API_ERROR(enc, JXL_ENC_ERR_API_USAGE, "Invalid upsampling mode");
if (mode > 1)
return JXL_API_ERROR(enc, JXL_ENC_ERR_NOT_SUPPORTED,
"Unsupported upsampling mode");
const size_t count = (factor == 2 ? 15 : (factor == 4 ? 55 : 210));
auto& td = enc->metadata.transform_data;
float* weights = (factor == 2 ? td.upsampling2_weights
: (factor == 4 ? td.upsampling4_weights
: td.upsampling8_weights));
if (mode == -1) {
// Default fancy upsampling: don't signal custom weights
enc->metadata.transform_data.custom_weights_mask &= ~(factor >> 1);
} else if (mode == 0) {
// Nearest neighbor upsampling
enc->metadata.transform_data.custom_weights_mask |= (factor >> 1);
memset(weights, 0, sizeof(float) * count);
if (factor == 2) {
weights[9] = 1.f;
} else if (factor == 4) {
for (int i : {19, 24, 49}) weights[i] = 1.f;
} else if (factor == 8) {
for (int i : {39, 44, 49, 54, 119, 124, 129, 174, 179, 204}) {
weights[i] = 1.f;
}
}
} else if (mode == 1) {
// 'Pixel dots' upsampling (nearest-neighbor with cut corners)
JxlEncoderSetUpsamplingMode(enc, factor, 0);
if (factor == 4) {
weights[19] = 0.f;
weights[24] = 0.5f;
} else if (factor == 8) {
for (int i : {39, 44, 49, 119}) weights[i] = 0.f;
for (int i : {54, 124}) weights[i] = 0.5f;
}
}
return JXL_ENC_SUCCESS;
}
JXL_EXPORT JxlEncoderStatus JxlEncoderSetExtraChannelInfo(
JxlEncoder* enc, size_t index, const JxlExtraChannelInfo* info) {
if (index >= enc->metadata.m.num_extra_channels) {
@ -1285,6 +1335,15 @@ JxlEncoderStatus JxlEncoderFrameSettingsSetOption(
"Buffering has to be in [0..3]");
}
return JXL_ENC_SUCCESS;
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_EXIF:
frame_settings->values.cparams.jpeg_keep_exif = value;
return JXL_ENC_SUCCESS;
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_XMP:
frame_settings->values.cparams.jpeg_keep_xmp = value;
return JXL_ENC_SUCCESS;
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_JUMBF:
frame_settings->values.cparams.jpeg_keep_jumbf = value;
return JXL_ENC_SUCCESS;
default:
return JXL_API_ERROR(frame_settings->enc, JXL_ENC_ERR_NOT_SUPPORTED,
@ -1376,6 +1435,9 @@ JxlEncoderStatus JxlEncoderFrameSettingsSetFloatOption(
case JXL_ENC_FRAME_SETTING_FILL_ENUM:
case JXL_ENC_FRAME_SETTING_JPEG_COMPRESS_BOXES:
case JXL_ENC_FRAME_SETTING_BUFFERING:
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_EXIF:
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_XMP:
case JXL_ENC_FRAME_SETTING_JPEG_KEEP_JUMBF:
return JXL_API_ERROR(frame_settings->enc, JXL_ENC_ERR_NOT_SUPPORTED,
"Int option, try setting it with "
"JxlEncoderFrameSettingsSetOption");
@ -1567,7 +1629,8 @@ JxlEncoderStatus JxlEncoderAddJPEGFrame(
frame_settings->enc->metadata.m.orientation);
jxl::InterpretExif(io.blobs.exif, &orientation);
frame_settings->enc->metadata.m.orientation = orientation;
}
if (!io.blobs.exif.empty() && frame_settings->values.cparams.jpeg_keep_exif) {
size_t exif_size = io.blobs.exif.size();
// Exif data in JPEG is limited to 64k
if (exif_size > 0xFFFF) {
@ -1581,19 +1644,26 @@ JxlEncoderStatus JxlEncoderAddJPEGFrame(
JxlEncoderAddBox(frame_settings->enc, "Exif", exif.data(), exif_size,
frame_settings->values.cparams.jpeg_compress_boxes);
}
if (!io.blobs.xmp.empty()) {
if (!io.blobs.xmp.empty() && frame_settings->values.cparams.jpeg_keep_xmp) {
JxlEncoderUseBoxes(frame_settings->enc);
JxlEncoderAddBox(frame_settings->enc, "xml ", io.blobs.xmp.data(),
io.blobs.xmp.size(),
frame_settings->values.cparams.jpeg_compress_boxes);
}
if (!io.blobs.jumbf.empty()) {
if (!io.blobs.jumbf.empty() &&
frame_settings->values.cparams.jpeg_keep_jumbf) {
JxlEncoderUseBoxes(frame_settings->enc);
JxlEncoderAddBox(frame_settings->enc, "jumb", io.blobs.jumbf.data(),
io.blobs.jumbf.size(),
frame_settings->values.cparams.jpeg_compress_boxes);
}
if (frame_settings->enc->store_jpeg_metadata) {
if (!frame_settings->values.cparams.jpeg_keep_exif ||
!frame_settings->values.cparams.jpeg_keep_xmp) {
return JXL_API_ERROR(frame_settings->enc, JXL_ENC_ERR_API_USAGE,
"Need to preserve EXIF and XMP to allow JPEG "
"bitstream reconstruction");
}
jxl::jpeg::JPEGData data_in = *io.Main().jpeg_data;
jxl::PaddedBytes jpeg_data;
if (!jxl::jpeg::EncodeJPEGData(data_in, &jpeg_data,

4
third_party/jpeg-xl/lib/jxl/encode_test.cc поставляемый
Просмотреть файл

@ -523,7 +523,7 @@ TEST(EncodeTest, LossyEncoderUseOriginalProfileTest) {
ASSERT_EQ(JXL_ENC_SUCCESS,
JxlEncoderFrameSettingsSetOption(
frame_settings, JXL_ENC_FRAME_SETTING_EFFORT, 8));
VerifyFrameEncoding(63, 129, enc.get(), frame_settings, 7173, true);
VerifyFrameEncoding(63, 129, enc.get(), frame_settings, 7228, true);
}
}
@ -842,6 +842,7 @@ TEST(EncodeTest, JXL_TRANSCODE_JPEG_TEST(JPEGReconstructionTest)) {
EXPECT_EQ(JXL_ENC_SUCCESS, process_result);
jxl::extras::JXLDecompressParams dparams;
jxl::test::DefaultAcceptedFormats(dparams);
std::vector<uint8_t> decoded_jpeg_bytes;
jxl::extras::PackedPixelFile ppf;
EXPECT_TRUE(DecodeImageJXL(compressed.data(), compressed.size(), dparams,
@ -883,6 +884,7 @@ TEST(EncodeTest, JXL_TRANSCODE_JPEG_TEST(ProgressiveJPEGReconstructionTest)) {
EXPECT_EQ(JXL_ENC_SUCCESS, process_result);
jxl::extras::JXLDecompressParams dparams;
jxl::test::DefaultAcceptedFormats(dparams);
std::vector<uint8_t> decoded_jpeg_bytes;
jxl::extras::PackedPixelFile ppf;
EXPECT_TRUE(DecodeImageJXL(compressed.data(), compressed.size(), dparams,

185
third_party/jpeg-xl/lib/jxl/jpeg/dec_jpeg_data_writer.cc поставляемый
Просмотреть файл

@ -111,9 +111,15 @@ static JXL_INLINE void DischargeBitBuffer(JpegBitWriter* bw, int nbits,
}
static JXL_INLINE void WriteBits(JpegBitWriter* bw, int nbits, uint64_t bits) {
JXL_DASSERT(nbits > 0);
bw->put_bits -= nbits;
if (JXL_UNLIKELY(bw->put_bits < 0)) {
DischargeBitBuffer(bw, nbits, bits);
if (JXL_UNLIKELY(nbits > 64)) {
bw->put_bits += nbits;
bw->healthy = false;
} else {
DischargeBitBuffer(bw, nbits, bits);
}
} else {
bw->put_buffer |= (bits << bw->put_bits);
}
@ -180,50 +186,25 @@ void DCTCodingStateInit(DCTCodingState* s) {
s->refinement_bits_.reserve(kJPEGMaxCorrectionBits);
}
enum OutputModes {
kModeHistogram,
kModeWrite,
};
template <int kOutputMode>
static JXL_INLINE void WriteSymbol(int symbol, HuffmanCodeTable* table,
JpegBitWriter* bw) {
if (kOutputMode == OutputModes::kModeHistogram) {
++table->depth[symbol];
} else {
// This is an optimization; if everything goes well,
// then |nbits| is positive; if non-existing Huffman symbol is going to be
// encoded, its length should be zero; later the encoder could check the
// "health" of JpegBitWriter.
// Not checking this can cause bad output without error on corrupt input,
// but that's OK. The check causes a noticeable slowdown.
#if false
bw->healthy &= table->depth[symbol] != 0;
#endif
WriteBits(bw, table->depth[symbol], table->code[symbol]);
}
WriteBits(bw, table->depth[symbol], table->code[symbol]);
}
template <int kOutputMode>
static JXL_INLINE void WriteSymbolBits(int symbol, HuffmanCodeTable* table,
JpegBitWriter* bw, int nbits,
uint64_t bits) {
if (kOutputMode == OutputModes::kModeHistogram) {
++table->depth[symbol];
} else {
WriteBits(bw, nbits + table->depth[symbol],
bits | (table->code[symbol] << nbits));
}
WriteBits(bw, nbits + table->depth[symbol],
bits | (table->code[symbol] << nbits));
}
// Emit all buffered data to the bit stream using the given Huffman code and
// bit writer.
template <int kOutputMode>
static JXL_INLINE void Flush(DCTCodingState* s, JpegBitWriter* bw) {
if (s->eob_run_ > 0) {
int nbits = FloorLog2Nonzero<uint32_t>(s->eob_run_);
int symbol = nbits << 4u;
WriteSymbol<kOutputMode>(symbol, s->cur_ac_huff_, bw);
WriteSymbol(symbol, s->cur_ac_huff_, bw);
if (nbits > 0) {
WriteBits(bw, nbits, s->eob_run_ & ((1 << nbits) - 1));
}
@ -237,7 +218,6 @@ static JXL_INLINE void Flush(DCTCodingState* s, JpegBitWriter* bw) {
// Buffer some more data at the end-of-band (the last non-zero or newly
// non-zero coefficient within the [Ss, Se] spectral band).
template <int kOutputMode>
static JXL_INLINE void BufferEndOfBand(DCTCodingState* s,
HuffmanCodeTable* ac_huff,
const std::vector<int>* new_bits,
@ -252,7 +232,7 @@ static JXL_INLINE void BufferEndOfBand(DCTCodingState* s,
}
if (s->eob_run_ == 0x7FFF ||
s->refinement_bits_.size() > kJPEGMaxCorrectionBits - kDCTBlockSize + 1) {
Flush<kOutputMode>(s, bw);
Flush(s, bw);
}
}
@ -395,10 +375,11 @@ bool EncodeDHT(const JPEGData& jpg, SerializationState* state) {
huff_table = &state->dc_huff_table[index];
}
// TODO(eustas): cache
// TODO(eustas): set up non-existing symbols
huff_table->InitDepths(127);
if (!BuildHuffmanCodeTable(huff, huff_table)) {
return false;
}
huff_table->initialized = true;
size_t total_count = 0;
size_t max_length = 0;
for (size_t i = 0; i < huff.counts.size(); ++i) {
@ -497,7 +478,6 @@ bool EncodeInterMarkerData(const JPEGData& jpg, SerializationState* state) {
return true;
}
template <int kOutputMode>
bool EncodeDCTBlockSequential(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
HuffmanCodeTable* ac_huff, int num_zero_runs,
coeff_t* last_dc_coeff, JpegBitWriter* bw) {
@ -511,7 +491,7 @@ bool EncodeDCTBlockSequential(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
temp2 ^= temp;
int dc_nbits = (temp2 == 0) ? 0 : (FloorLog2Nonzero<uint32_t>(temp2) + 1);
WriteSymbol<kOutputMode>(dc_nbits, dc_huff, bw);
WriteSymbol(dc_nbits, dc_huff, bw);
#if false
// If the input is corrupt, this could be triggered. Checking is
// costly though, so it makes more sense to avoid this branch.
@ -519,7 +499,9 @@ bool EncodeDCTBlockSequential(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
// of catching it and returning error)
if (dc_nbits >= 12) return false;
#endif
WriteBits(bw, dc_nbits, temp & ((1u << dc_nbits) - 1));
if (dc_nbits) {
WriteBits(bw, dc_nbits, temp & ((1u << dc_nbits) - 1));
}
int16_t r = 0;
for (size_t i = 1; i < 64; i++) {
@ -530,36 +512,35 @@ bool EncodeDCTBlockSequential(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
temp += temp2;
temp2 ^= temp;
if (JXL_UNLIKELY(r > 15)) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
r -= 16;
}
if (JXL_UNLIKELY(r > 15)) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
r -= 16;
}
if (JXL_UNLIKELY(r > 15)) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
if (r > 15) {
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
}
if (r > 15) {
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
}
}
int ac_nbits = FloorLog2Nonzero<uint32_t>(temp2) + 1;
int symbol = (r << 4u) + ac_nbits;
WriteSymbolBits<kOutputMode>(symbol, ac_huff, bw, ac_nbits,
temp & ((1 << ac_nbits) - 1));
WriteSymbolBits(symbol, ac_huff, bw, ac_nbits,
temp & ((1 << ac_nbits) - 1));
r = 0;
}
}
for (int i = 0; i < num_zero_runs; ++i) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
}
if (r > 0) {
WriteSymbol<kOutputMode>(0, ac_huff, bw);
WriteSymbol(0, ac_huff, bw);
}
return true;
}
template <int kOutputMode>
bool EncodeDCTBlockProgressive(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
HuffmanCodeTable* ac_huff, int Ss, int Se,
int Al, int num_zero_runs,
@ -579,8 +560,8 @@ bool EncodeDCTBlockProgressive(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
temp2--;
}
int nbits = (temp == 0) ? 0 : (FloorLog2Nonzero<uint32_t>(temp) + 1);
WriteSymbol<kOutputMode>(nbits, dc_huff, bw);
if (nbits > 0) {
WriteSymbol(nbits, dc_huff, bw);
if (nbits) {
WriteBits(bw, nbits, temp2 & ((1 << nbits) - 1));
}
++Ss;
@ -607,34 +588,33 @@ bool EncodeDCTBlockProgressive(const coeff_t* coeffs, HuffmanCodeTable* dc_huff,
r++;
continue;
}
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
while (r > 15) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
}
int nbits = FloorLog2Nonzero<uint32_t>(temp) + 1;
int symbol = (r << 4u) + nbits;
WriteSymbol<kOutputMode>(symbol, ac_huff, bw);
WriteSymbol(symbol, ac_huff, bw);
WriteBits(bw, nbits, temp2 & ((1 << nbits) - 1));
r = 0;
}
if (num_zero_runs > 0) {
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
for (int i = 0; i < num_zero_runs; ++i) {
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
}
}
if (r > 0) {
BufferEndOfBand<kOutputMode>(coding_state, ac_huff, nullptr, bw);
BufferEndOfBand(coding_state, ac_huff, nullptr, bw);
if (!eob_run_allowed) {
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
}
}
return true;
}
template <int kOutputMode>
bool EncodeRefinementBits(const coeff_t* coeffs, HuffmanCodeTable* ac_huff,
int Ss, int Se, int Al, DCTCodingState* coding_state,
JpegBitWriter* bw) {
@ -665,8 +645,8 @@ bool EncodeRefinementBits(const coeff_t* coeffs, HuffmanCodeTable* ac_huff,
continue;
}
while (r > 15 && k <= eob) {
Flush<kOutputMode>(coding_state, bw);
WriteSymbol<kOutputMode>(0xf0, ac_huff, bw);
Flush(coding_state, bw);
WriteSymbol(0xf0, ac_huff, bw);
r -= 16;
for (int bit : refinement_bits) {
WriteBits(bw, 1, bit);
@ -677,10 +657,10 @@ bool EncodeRefinementBits(const coeff_t* coeffs, HuffmanCodeTable* ac_huff,
refinement_bits.push_back(abs_values[k] & 1u);
continue;
}
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
int symbol = (r << 4u) + 1;
int new_non_zero_bit = (coeffs[kJPEGNaturalOrder[k]] < 0) ? 0 : 1;
WriteSymbol<kOutputMode>(symbol, ac_huff, bw);
WriteSymbol(symbol, ac_huff, bw);
WriteBits(bw, 1, new_non_zero_bit);
for (int bit : refinement_bits) {
WriteBits(bw, 1, bit);
@ -689,9 +669,9 @@ bool EncodeRefinementBits(const coeff_t* coeffs, HuffmanCodeTable* ac_huff,
r = 0;
}
if (r > 0 || !refinement_bits.empty()) {
BufferEndOfBand<kOutputMode>(coding_state, ac_huff, &refinement_bits, bw);
BufferEndOfBand(coding_state, ac_huff, &refinement_bits, bw);
if (!eob_run_allowed) {
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
}
}
return true;
@ -714,7 +694,7 @@ size_t HistogramIndex(const JPEGData& jpg, size_t scan_index,
return idx + component_index;
}
template <int kMode, int kOutputMode>
template <int kMode>
SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
SerializationState* state) {
const JPEGScanInfo& scan_info = jpg.scan_info[state->scan_index];
@ -772,6 +752,7 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
// DC-only is defined by [0..0] spectral range.
const bool want_ac = ((Ss != 0) || (Se != 0));
const bool want_dc = (Ss == 0);
// TODO: support streaming decoding again.
const bool complete_ac = true;
const bool has_ac = true;
@ -796,7 +777,7 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
for (int mcu_x = 0; mcu_x < MCUs_per_row; ++mcu_x) {
// Possibly emit a restart marker.
if (restart_interval > 0 && ss.restarts_to_go == 0) {
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
if (!JumpToByteBoundary(bw, &state->pad_bits, state->pad_bits_end)) {
return SerializationStatus::ERROR;
}
@ -811,14 +792,16 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
for (size_t i = 0; i < scan_info.num_components; ++i) {
const JPEGComponentScanInfo& si = scan_info.components[i];
const JPEGComponent& c = jpg.components[si.comp_idx];
size_t dc_tbl_idx = (kOutputMode == OutputModes::kModeHistogram
? HistogramIndex(jpg, state->scan_index, i)
: si.dc_tbl_idx);
size_t ac_tbl_idx = (kOutputMode == OutputModes::kModeHistogram
? HistogramIndex(jpg, state->scan_index, i)
: si.ac_tbl_idx);
size_t dc_tbl_idx = si.dc_tbl_idx;
size_t ac_tbl_idx = si.ac_tbl_idx;
HuffmanCodeTable* dc_huff = &state->dc_huff_table[dc_tbl_idx];
HuffmanCodeTable* ac_huff = &state->ac_huff_table[ac_tbl_idx];
if (want_dc && !dc_huff->initialized) {
return SerializationStatus::ERROR;
}
if (want_ac && !ac_huff->initialized) {
return SerializationStatus::ERROR;
}
int n_blocks_y = is_interleaved ? c.v_samp_factor : 1;
int n_blocks_x = is_interleaved ? c.h_samp_factor : 1;
// compressed size per block cannot be more than 512 bytes per component
@ -829,7 +812,7 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
int block_x = mcu_x * n_blocks_x + ix;
int block_idx = block_y * c.width_in_blocks + block_x;
if (ss.block_scan_index == ss.next_reset_point) {
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
ss.next_reset_point = get_next_reset_point();
}
int num_zero_runs = 0;
@ -842,16 +825,16 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
const coeff_t* coeffs = &c.coeffs[block_idx << 6];
bool ok;
if (kMode == 0) {
ok = EncodeDCTBlockSequential<kOutputMode>(
coeffs, dc_huff, ac_huff, num_zero_runs,
ss.last_dc_coeff + si.comp_idx, bw);
ok = EncodeDCTBlockSequential(coeffs, dc_huff, ac_huff,
num_zero_runs,
ss.last_dc_coeff + si.comp_idx, bw);
} else if (kMode == 1) {
ok = EncodeDCTBlockProgressive<kOutputMode>(
ok = EncodeDCTBlockProgressive(
coeffs, dc_huff, ac_huff, Ss, Se, Al, num_zero_runs,
coding_state, ss.last_dc_coeff + si.comp_idx, bw);
} else {
ok = EncodeRefinementBits<kOutputMode>(coeffs, ac_huff, Ss, Se,
Al, coding_state, bw);
ok = EncodeRefinementBits(coeffs, ac_huff, Ss, Se, Al,
coding_state, bw);
}
if (!ok) return SerializationStatus::ERROR;
++ss.block_scan_index;
@ -865,7 +848,7 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
if (!bw->healthy) return SerializationStatus::ERROR;
return SerializationStatus::NEEDS_MORE_INPUT;
}
Flush<kOutputMode>(coding_state, bw);
Flush(coding_state, bw);
if (!JumpToByteBoundary(bw, &state->pad_bits, state->pad_bits_end)) {
return SerializationStatus::ERROR;
}
@ -877,7 +860,6 @@ SerializationStatus JXL_NOINLINE DoEncodeScan(const JPEGData& jpg,
return SerializationStatus::DONE;
}
template <int kOutputMode>
static SerializationStatus JXL_INLINE EncodeScan(const JPEGData& jpg,
SerializationState* state) {
const JPEGScanInfo& scan_info = jpg.scan_info[state->scan_index];
@ -889,15 +871,14 @@ static SerializationStatus JXL_INLINE EncodeScan(const JPEGData& jpg,
const bool need_sequential =
!is_progressive || (Ah == 0 && Al == 0 && Ss == 0 && Se == 63);
if (need_sequential) {
return DoEncodeScan<0, kOutputMode>(jpg, state);
return DoEncodeScan<0>(jpg, state);
} else if (Ah == 0) {
return DoEncodeScan<1, kOutputMode>(jpg, state);
return DoEncodeScan<1>(jpg, state);
} else {
return DoEncodeScan<2, kOutputMode>(jpg, state);
return DoEncodeScan<2>(jpg, state);
}
}
template <int kOutputMode>
SerializationStatus SerializeSection(uint8_t marker, SerializationState* state,
const JPEGData& jpg) {
const auto to_status = [](bool result) {
@ -913,8 +894,7 @@ SerializationStatus SerializeSection(uint8_t marker, SerializationState* state,
return to_status(EncodeSOF(jpg, marker, state));
case 0xC4:
return to_status((kOutputMode == OutputModes::kModeHistogram) ||
EncodeDHT(jpg, state));
return to_status(EncodeDHT(jpg, state));
case 0xD0:
case 0xD1:
@ -930,7 +910,7 @@ SerializationStatus SerializeSection(uint8_t marker, SerializationState* state,
return to_status(EncodeEOI(jpg, state));
case 0xDA:
return EncodeScan<kOutputMode>(jpg, state);
return EncodeScan(jpg, state);
case 0xDB:
return to_status(EncodeDQT(jpg, state));
@ -968,7 +948,6 @@ SerializationStatus SerializeSection(uint8_t marker, SerializationState* state,
}
// TODO(veluca): add streaming support again.
template <int kOutputMode>
Status WriteJpegInternal(const JPEGData& jpg, const JPEGOutput& out,
SerializationState* ss) {
const auto maybe_push_output = [&]() -> Status {
@ -999,18 +978,8 @@ Status WriteJpegInternal(const JPEGData& jpg, const JPEGOutput& out,
ss->stage = SerializationState::STAGE_ERROR;
break;
}
if (kOutputMode == OutputModes::kModeHistogram) {
size_t num_histo = NumHistograms(jpg);
ss->dc_huff_table.resize(num_histo);
ss->ac_huff_table.resize(num_histo);
for (size_t i = 0; i < num_histo; ++i) {
ss->dc_huff_table[i].InitDepths();
ss->ac_huff_table[i].InitDepths();
}
} else {
ss->dc_huff_table.resize(kMaxHuffmanTables);
ss->ac_huff_table.resize(kMaxHuffmanTables);
}
ss->dc_huff_table.resize(kMaxHuffmanTables);
ss->ac_huff_table.resize(kMaxHuffmanTables);
if (jpg.has_zero_padding_bit) {
ss->pad_bits = jpg.padding_bits.data();
ss->pad_bits_end = ss->pad_bits + jpg.padding_bits.size();
@ -1028,8 +997,7 @@ Status WriteJpegInternal(const JPEGData& jpg, const JPEGOutput& out,
break;
}
uint8_t marker = jpg.marker_order[ss->section_index];
SerializationStatus status =
SerializeSection<kOutputMode>(marker, ss, jpg);
SerializationStatus status = SerializeSection(marker, ss, jpg);
if (status == SerializationStatus::ERROR) {
JXL_WARNING("Failed to encode marker 0x%.2x", marker);
ss->stage = SerializationState::STAGE_ERROR;
@ -1064,12 +1032,7 @@ Status WriteJpegInternal(const JPEGData& jpg, const JPEGOutput& out,
Status WriteJpeg(const JPEGData& jpg, const JPEGOutput& out) {
SerializationState ss;
return WriteJpegInternal<OutputModes::kModeWrite>(jpg, out, &ss);
}
Status ProcessJpeg(const JPEGData& jpg, SerializationState* ss) {
auto nullout = [](const uint8_t* buf, size_t len) { return len; };
return WriteJpegInternal<OutputModes::kModeHistogram>(jpg, nullout, ss);
return WriteJpegInternal(jpg, out, &ss);
}
} // namespace jpeg

4
third_party/jpeg-xl/lib/jxl/jpeg/dec_jpeg_data_writer.h поставляемый
Просмотреть файл

@ -25,10 +25,6 @@ using JPEGOutput = std::function<size_t(const uint8_t* buf, size_t len)>;
Status WriteJpeg(const JPEGData& jpg, const JPEGOutput& out);
// Same as WriteJpeg, but instead of writing to the output, collects statistics
// about the bit-stream into `ss`.
Status ProcessJpeg(const JPEGData& jpg, SerializationState* ss);
} // namespace jpeg
} // namespace jxl

9
third_party/jpeg-xl/lib/jxl/jpeg/dec_jpeg_serialization_state.h поставляемый
Просмотреть файл

@ -16,9 +16,12 @@ namespace jxl {
namespace jpeg {
struct HuffmanCodeTable {
int depth[256];
int code[256];
void InitDepths() { std::fill(std::begin(depth), std::end(depth), 0); }
int8_t depth[256];
uint16_t code[256];
bool initialized = false;
void InitDepths(int value = 0) {
std::fill(std::begin(depth), std::end(depth), value);
}
};
// Handles the packing of bits into output bytes.

42
third_party/jpeg-xl/lib/jxl/jpeg/jpeg_data.cc поставляемый
Просмотреть файл

@ -343,7 +343,7 @@ Status JPEGData::VisitFields(Visitor* visitor) {
}
uint32_t tail_data_len = tail_data.size();
if (!visitor->IsReading() && tail_data_len > 4260096) {
return JXL_FAILURE("Tail data too large (max size = 4260096, size = %u).",
return JXL_FAILURE("Tail data too large (max size = 4260096, size = %u)",
tail_data_len);
}
JXL_RETURN_IF_ERROR(visitor->U32(Val(0), BitsOffset(8, 1),
@ -370,6 +370,46 @@ Status JPEGData::VisitFields(Visitor* visitor) {
}
}
{
size_t dht_index = 0;
size_t scan_index = 0;
bool is_progressive = false;
bool ac_ok[kMaxHuffmanTables] = {false};
bool dc_ok[kMaxHuffmanTables] = {false};
for (uint8_t marker : marker_order) {
if (marker == 0xC2) {
is_progressive = true;
} else if (marker == 0xC4) {
for (; dht_index < huffman_code.size();) {
const JPEGHuffmanCode& huff = huffman_code[dht_index++];
size_t index = huff.slot_id;
if (index & 0x10) {
index -= 0x10;
ac_ok[index] = true;
} else {
dc_ok[index] = true;
}
if (huff.is_last) break;
}
} else if (marker == 0xDA) {
const JPEGScanInfo& si = scan_info[scan_index++];
for (size_t i = 0; i < si.num_components; ++i) {
const JPEGComponentScanInfo& csi = si.components[i];
size_t dc_tbl_idx = csi.dc_tbl_idx;
size_t ac_tbl_idx = csi.ac_tbl_idx;
bool want_dc = !is_progressive || (si.Ss == 0);
if (want_dc && !dc_ok[dc_tbl_idx]) {
return JXL_FAILURE("DC Huffman table used before defined");
}
bool want_ac = !is_progressive || (si.Ss != 0) || (si.Se != 0);
if (want_ac && !ac_ok[ac_tbl_idx]) {
return JXL_FAILURE("AC Huffman table used before defined");
}
}
}
}
}
// Apply postponed actions.
if (visitor->IsReading()) {
tail_data.resize(tail_data_len);

42
third_party/jpeg-xl/lib/jxl/jxl_test.cc поставляемый
Просмотреть файл

@ -122,8 +122,8 @@ TEST(JxlTest, RoundtripSmallD1) {
{
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), {}, {}, pool, &ppf_out), 766, 40);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.2));
EXPECT_NEAR(Roundtrip(t.ppf(), {}, {}, pool, &ppf_out), 816, 40);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.888));
}
// With a lower intensity target than the default, the bitrate should be
@ -203,7 +203,7 @@ TEST(JxlTest, RoundtripOutOfOrderProcessing) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_EPF, 3);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 23553, 400);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 22999, 400);
EXPECT_LE(ButteraugliDistance(t.ppf(), ppf_out), 1.35);
}
@ -223,7 +223,7 @@ TEST(JxlTest, RoundtripOutOfOrderProcessingBorder) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_RESAMPLING, 2);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 10907, 200);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 11015, 200);
EXPECT_LE(ButteraugliDistance(t.ppf(), ppf_out), 2.9);
}
@ -296,9 +296,9 @@ TEST(JxlTest, RoundtripMultiGroup) {
};
auto run_kitten = std::async(std::launch::async, test, SpeedTier::kKitten,
1.0f, 56570u, 11.7);
1.0f, 55602u, 11.7);
auto run_wombat = std::async(std::launch::async, test, SpeedTier::kWombat,
2.0f, 35274u, 20.0);
2.0f, 33624u, 20.0);
}
TEST(JxlTest, RoundtripRGBToGrayscale) {
@ -357,7 +357,7 @@ TEST(JxlTest, RoundtripLargeFast) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_EFFORT, 7); // kSquirrel
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 453713, 5000);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 445555, 5000);
EXPECT_THAT(ComputeDistance2(t.ppf(), ppf_out), IsSlightlyBelow(100));
}
@ -374,7 +374,7 @@ TEST(JxlTest, RoundtripDotsForceEpf) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_DOTS, 1);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 41087, 300);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 41777, 300);
EXPECT_THAT(ComputeDistance2(t.ppf(), ppf_out), IsSlightlyBelow(18));
}
@ -454,7 +454,7 @@ TEST(JxlTest, RoundtripSmallNL) {
t.SetDimensions(xsize, ysize);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), {}, {}, pool, &ppf_out), 753, 45);
EXPECT_NEAR(Roundtrip(t.ppf(), {}, {}, pool, &ppf_out), 801, 45);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.1));
}
@ -470,7 +470,7 @@ TEST(JxlTest, RoundtripNoGaborishNoAR) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_GABORISH, 0);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 38009, 200);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 38900, 200);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.8));
}
@ -515,7 +515,7 @@ TEST(JxlTest, RoundtripSmallPatchesAlpha) {
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 597, 100);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.015f));
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.018f));
}
TEST(JxlTest, RoundtripSmallPatches) {
@ -540,7 +540,7 @@ TEST(JxlTest, RoundtripSmallPatches) {
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 486, 100);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.015f));
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.018f));
}
// TODO(szabadka) Add encoder and decoder API functions that accept frame
@ -846,14 +846,14 @@ TEST(JxlTest, RoundtripAlphaPremultiplied) {
EXPECT_THAT(ButteraugliDistance(io.frames, io2.frames,
ButteraugliParams(), GetJxlCms(),
/*distmap=*/nullptr),
IsSlightlyBelow(1.2));
IsSlightlyBelow(1.111));
EXPECT_TRUE(UnpremultiplyAlpha(io2));
EXPECT_FALSE(io2.Main().AlphaIsPremultiplied());
}
EXPECT_THAT(ButteraugliDistance(io_nopremul.frames, io2.frames,
ButteraugliParams(), GetJxlCms(),
/*distmap=*/nullptr),
IsSlightlyBelow(1.47));
IsSlightlyBelow(1.55));
}
}
}
@ -874,7 +874,7 @@ TEST(JxlTest, RoundtripAlphaResampling) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_EXTRA_CHANNEL_RESAMPLING, 2);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 12745, 130);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 13155, 130);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(5.2));
}
@ -1129,7 +1129,7 @@ TEST(JxlTest, RoundtripDots) {
cparams.distance = 0.04;
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 277976, 4000);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 273333, 4000);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(0.35));
}
@ -1149,8 +1149,8 @@ TEST(JxlTest, RoundtripNoise) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_NOISE, 1);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 38244, 750);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.3));
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, pool, &ppf_out), 39261, 750);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.35));
}
TEST(JxlTest, RoundtripLossless8Gray) {
@ -1262,6 +1262,7 @@ size_t RoundtripJpeg(const PaddedBytes& jpeg_in, ThreadPool* pool) {
&compressed));
jxl::JXLDecompressParams dparams;
test::DefaultAcceptedFormats(dparams);
test::SetThreadParallelRunner(dparams, pool);
std::vector<uint8_t> out;
jxl::PackedPixelFile ppf;
@ -1290,6 +1291,7 @@ void RoundtripJpegToPixels(const PaddedBytes& jpeg_in,
EXPECT_TRUE(extras::EncodeImageJXL({}, extras::PackedPixelFile(), &jpeg_bytes,
&compressed));
test::DefaultAcceptedFormats(dparams);
test::SetThreadParallelRunner(dparams, pool);
EXPECT_TRUE(DecodeImageJXL(compressed.data(), compressed.size(), dparams,
nullptr, ppf_out, nullptr));
@ -1473,7 +1475,7 @@ TEST(JxlTest, RoundtripProgressive) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_RESPONSIVE, 1);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 63570, 750);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 62160, 750);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.4));
}
@ -1490,7 +1492,7 @@ TEST(JxlTest, RoundtripProgressiveLevel2Slow) {
cparams.AddOption(JXL_ENC_FRAME_SETTING_RESPONSIVE, 1);
PackedPixelFile ppf_out;
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 74621, 1000);
EXPECT_NEAR(Roundtrip(t.ppf(), cparams, {}, &pool, &ppf_out), 71111, 1000);
EXPECT_THAT(ButteraugliDistance(t.ppf(), ppf_out), IsSlightlyBelow(1.17));
}

80
third_party/jpeg-xl/lib/jxl/modular/encoding/encoding.cc поставляемый
Просмотреть файл

@ -194,7 +194,7 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
pixel_type *JXL_RESTRICT r = channel.Row(y);
for (size_t x = 0; x < channel.w; x++) {
uint32_t v =
reader->ReadHybridUintClustered<uses_lz77>(ctx_id, br);
reader->ReadHybridUintClusteredInlined<uses_lz77>(ctx_id, br);
r[x] = UnpackSigned(v);
}
}
@ -203,7 +203,8 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
pixel_type *JXL_RESTRICT r = channel.Row(y);
for (size_t x = 0; x < channel.w; x++) {
uint32_t v =
reader->ReadHybridUintClustered<uses_lz77>(ctx_id, br);
reader->ReadHybridUintClusteredMaybeInlined<uses_lz77>(ctx_id,
br);
r[x] = make_pixel(v, multiplier, offset);
}
}
@ -255,7 +256,8 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
pixel_type top = (y ? *(r + x - onerow) : left);
pixel_type topleft = (x && y ? *(r + x - 1 - onerow) : left);
pixel_type guess = ClampedGradient(top, left, topleft);
uint64_t v = reader->ReadHybridUintClustered<uses_lz77>(ctx_id, br);
uint64_t v = reader->ReadHybridUintClusteredMaybeInlined<uses_lz77>(
ctx_id, br);
r[x] = make_pixel(v, 1, guess);
}
}
@ -293,34 +295,70 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
std::max<pixel_type_w>(-kPropRangeFast, top + left - topleft),
kPropRangeFast - 1);
uint32_t ctx_id = context_lookup[pos];
uint64_t v = reader->ReadHybridUintClustered<uses_lz77>(ctx_id, br);
uint64_t v =
reader->ReadHybridUintClusteredMaybeInlined<uses_lz77>(ctx_id, br);
r[x] = make_pixel(v, multipliers[pos],
static_cast<pixel_type_w>(offsets[pos]) + guess);
}
}
} else if (!uses_lz77 && is_wp_only) {
} else if (!uses_lz77 && is_wp_only && channel.w > 8) {
JXL_DEBUG_V(8, "WP fast track.");
const intptr_t onerow = channel.plane.PixelsPerRow();
weighted::State wp_state(wp_header, channel.w, channel.h);
Properties properties(1);
for (size_t y = 0; y < channel.h; y++) {
pixel_type *JXL_RESTRICT r = channel.Row(y);
for (size_t x = 0; x < channel.w; x++) {
const pixel_type *JXL_RESTRICT rtop = (y ? channel.Row(y - 1) : r - 1);
const pixel_type *JXL_RESTRICT rtoptop =
(y > 1 ? channel.Row(y - 2) : rtop);
const pixel_type *JXL_RESTRICT rtopleft =
(y ? channel.Row(y - 1) - 1 : r - 1);
const pixel_type *JXL_RESTRICT rtopright =
(y ? channel.Row(y - 1) + 1 : r - 1);
size_t x = 0;
{
size_t offset = 0;
pixel_type_w left = (x ? r[x - 1] : y ? *(r + x - onerow) : 0);
pixel_type_w top = (y ? *(r + x - onerow) : left);
pixel_type_w topleft = (x && y ? *(r + x - 1 - onerow) : left);
pixel_type_w topright =
(x + 1 < channel.w && y ? *(r + x + 1 - onerow) : top);
pixel_type_w toptop = (y > 1 ? *(r + x - onerow - onerow) : top);
pixel_type_w left = y ? rtop[x] : 0;
pixel_type_w toptop = y ? rtoptop[x] : 0;
pixel_type_w topright = (x + 1 < channel.w && y ? rtop[x + 1] : left);
int32_t guess = wp_state.Predict</*compute_properties=*/true>(
x, y, channel.w, top, left, topright, topleft, toptop, &properties,
x, y, channel.w, left, left, topright, left, toptop, &properties,
offset);
uint32_t pos =
kPropRangeFast + std::min(std::max(-kPropRangeFast, properties[0]),
kPropRangeFast - 1);
uint32_t ctx_id = context_lookup[pos];
uint64_t v = reader->ReadHybridUintClustered<uses_lz77>(ctx_id, br);
uint64_t v =
reader->ReadHybridUintClusteredInlined<uses_lz77>(ctx_id, br);
r[x] = make_pixel(v, multipliers[pos],
static_cast<pixel_type_w>(offsets[pos]) + guess);
wp_state.UpdateErrors(r[x], x, y, channel.w);
}
for (x = 1; x + 1 < channel.w; x++) {
size_t offset = 0;
int32_t guess = wp_state.Predict</*compute_properties=*/true>(
x, y, channel.w, rtop[x], r[x - 1], rtopright[x], rtopleft[x],
rtoptop[x], &properties, offset);
uint32_t pos =
kPropRangeFast + std::min(std::max(-kPropRangeFast, properties[0]),
kPropRangeFast - 1);
uint32_t ctx_id = context_lookup[pos];
uint64_t v =
reader->ReadHybridUintClusteredInlined<uses_lz77>(ctx_id, br);
r[x] = make_pixel(v, multipliers[pos],
static_cast<pixel_type_w>(offsets[pos]) + guess);
wp_state.UpdateErrors(r[x], x, y, channel.w);
}
{
size_t offset = 0;
int32_t guess = wp_state.Predict</*compute_properties=*/true>(
x, y, channel.w, rtop[x], r[x - 1], rtop[x], rtopleft[x],
rtoptop[x], &properties, offset);
uint32_t pos =
kPropRangeFast + std::min(std::max(-kPropRangeFast, properties[0]),
kPropRangeFast - 1);
uint32_t ctx_id = context_lookup[pos];
uint64_t v =
reader->ReadHybridUintClusteredInlined<uses_lz77>(ctx_id, br);
r[x] = make_pixel(v, multipliers[pos],
static_cast<pixel_type_w>(offsets[pos]) + guess);
wp_state.UpdateErrors(r[x], x, y, channel.w);
@ -351,8 +389,8 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
PredictionResult res =
PredictTreeNoWPNEC(&properties, channel.w, p + x, onerow, x, y,
tree_lookup, references);
uint64_t v =
reader->ReadHybridUintClustered<uses_lz77>(res.context, br);
uint64_t v = reader->ReadHybridUintClusteredInlined<uses_lz77>(
res.context, br);
p[x] = make_pixel(v, res.multiplier, res.guess);
}
for (size_t x = channel.w - 2; x < channel.w; x++) {
@ -368,8 +406,8 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
PredictionResult res =
PredictTreeNoWP(&properties, channel.w, p + x, onerow, x, y,
tree_lookup, references);
uint64_t v =
reader->ReadHybridUintClustered<uses_lz77>(res.context, br);
uint64_t v = reader->ReadHybridUintClusteredMaybeInlined<uses_lz77>(
res.context, br);
p[x] = make_pixel(v, res.multiplier, res.guess);
}
}
@ -399,8 +437,8 @@ Status DecodeModularChannelMAANS(BitReader *br, ANSSymbolReader *reader,
PredictionResult res =
PredictTreeWPNEC(&properties, channel.w, p + x, onerow, x, y,
tree_lookup, references, &wp_state);
uint64_t v =
reader->ReadHybridUintClustered<uses_lz77>(res.context, br);
uint64_t v = reader->ReadHybridUintClusteredInlined<uses_lz77>(
res.context, br);
p[x] = make_pixel(v, res.multiplier, res.guess);
wp_state.UpdateErrors(p[x], x, y, channel.w);
}

2
third_party/jpeg-xl/lib/jxl/render_pipeline/render_pipeline_test.cc поставляемый
Просмотреть файл

@ -232,7 +232,7 @@ TEST_P(RenderPipelineTestParam, PipelineTest) {
#if JXL_HIGH_PRECISION
constexpr float kMaxError = 1e-5;
#else
constexpr float kMaxError = 1e-4;
constexpr float kMaxError = 5e-4;
#endif
Image3F def = std::move(*io_default.frames[i].color());
Image3F pip = std::move(*io_slow_pipeline.frames[i].color());

3
third_party/jpeg-xl/lib/jxl/splines.cc поставляемый
Просмотреть файл

@ -641,6 +641,9 @@ Status Splines::InitializeDrawCache(const size_t image_xsize,
JXL_WARNING(
"Large total_estimated_area_reached, expect slower decoding: %" PRIu64,
total_estimated_area_reached);
#ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
return JXL_FAILURE("Total spline area is too large");
#endif
}
for (Spline& spline : splines) {

12
third_party/jpeg-xl/lib/jxl/test_utils.cc поставляемый
Просмотреть файл

@ -56,9 +56,19 @@ PaddedBytes ReadTestData(const std::string& filename) {
return result;
}
void DefaultAcceptedFormats(extras::JXLDecompressParams& dparams) {
if (dparams.accepted_formats.empty()) {
for (const uint32_t num_channels : {1, 2, 3, 4}) {
dparams.accepted_formats.push_back(
{num_channels, JXL_TYPE_FLOAT, JXL_LITTLE_ENDIAN, /*align=*/0});
}
}
}
Status DecodeFile(extras::JXLDecompressParams dparams,
const Span<const uint8_t> file, CodecInOut* JXL_RESTRICT io,
ThreadPool* pool) {
DefaultAcceptedFormats(dparams);
SetThreadParallelRunner(dparams, pool);
extras::PackedPixelFile ppf;
JXL_RETURN_IF_ERROR(DecodeImageJXL(file.data(), file.size(), dparams,
@ -139,6 +149,7 @@ bool Roundtrip(const CodecInOut* io, const CompressParams& cparams,
extras::JXLDecompressParams dparams,
CodecInOut* JXL_RESTRICT io2, std::stringstream& failures,
size_t* compressed_size, ThreadPool* pool, AuxOut* aux_out) {
DefaultAcceptedFormats(dparams);
if (compressed_size) {
*compressed_size = static_cast<size_t>(-1);
}
@ -203,6 +214,7 @@ size_t Roundtrip(const extras::PackedPixelFile& ppf_in,
extras::JXLCompressParams cparams,
extras::JXLDecompressParams dparams, ThreadPool* pool,
extras::PackedPixelFile* ppf_out) {
DefaultAcceptedFormats(dparams);
SetThreadParallelRunner(cparams, pool);
SetThreadParallelRunner(dparams, pool);
std::vector<uint8_t> compressed;

2
third_party/jpeg-xl/lib/jxl/test_utils.h поставляемый
Просмотреть файл

@ -50,6 +50,8 @@ PaddedBytes ReadTestData(const std::string& filename);
void JxlBasicInfoSetFromPixelFormat(JxlBasicInfo* basic_info,
const JxlPixelFormat* pixel_format);
void DefaultAcceptedFormats(extras::JXLDecompressParams& dparams);
template <typename Params>
void SetThreadParallelRunner(Params params, ThreadPool* pool) {
if (pool && !params.runner_opaque) {

2
third_party/jpeg-xl/plugins/gimp/file-jxl-save.cc поставляемый
Просмотреть файл

@ -358,8 +358,6 @@ bool JpegXlSaveGui::SaveDialog() {
gtk_widget_show(separator);
// Advanced Settings Frame
std::vector<GtkWidget*> advanced_opts;
frame_advanced = gtk_frame_new("Advanced Settings");
gimp_help_set_help_data(frame_advanced,
"Some advanced settings may produce malformed files.",

2
third_party/jpeg-xl/third_party/CMakeLists.txt поставляемый
Просмотреть файл

@ -31,7 +31,7 @@ if (EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/highway/CMakeLists.txt" AND
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/highway/LICENSE"
${PROJECT_BINARY_DIR}/LICENSE.highway COPYONLY)
else()
find_package(HWY 1.0.0)
find_package(HWY 1.0.4)
if (NOT HWY_FOUND)
message(FATAL_ERROR
"Highway library (hwy) not found. Install libhwy-dev or download it "

1
third_party/jpeg-xl/tools/fast_lossless/.gitignore поставляемый Normal file
Просмотреть файл

@ -0,0 +1 @@
build/