Explictly depend on the internal thread functions that we need, and
allow unused ones to be DCE'd.
The biggest win here was removing the forced dependency on vsnprintf
which was being pulled in via
`emscripten_sync_run_in_main_thread_xprintf_varargs`. Interestingly
this function doesn't seem to have any callers at all but I assume
it has some use so leaving it in the code for now.
We had a single browser test set OFFSCREENCANVAS_SUPPORT=1 but
wasn't decorated with @requires_offscreen_canvas.
This test was relying on the bahaviour emscripten_proxy_main where
it would end up running main directly if pthread_create failed.
In thie case pthread_create fails because its tries to setup the
offscreen canvas.
This change avoids running that test when it offscree canvas is
not available and removes the fallback from emscripten_proxy_main.
Calling realpath with SAFE_HEAP enabled was causing a read of
address zero because realpath calls sys_open with a third argument
and therefor without any varargs at all
Fixes: #12999
This test needs to be disaled so that a recent llvm change
can roll: https://reviews.llvm.org/D93066
It causes the following diff/failure:
```
AssertionError: Unexpected difference:
--- /b/s/w/ir/k/install/emscripten/tests/other/metadce/hello_world_O3_MAIN_MODULE_2.funcs
+++ /b/s/w/ir/k/install/emscripten/tests/other/metadce/hello_world_O3_MAIN_MODULE_2.funcs.new
@@ -4,7 +4,8 @@
$__overflow
$__stdio_write
$__towrite
-$__wasm_apply_relocs
+$__wasm_apply_data_relocs
+$__wasm_apply_global_relocs
$__wasm_call_ctors
$dlmalloc
$fwrite
```
For simple browser tests we can just rely on the process
exit code being reported. I've added a new btest_exit
method so that tests can opt into this new behavior.
This allows test code to be identical whether running
in the browser or outside.
Modifying this global state seems wrong. Its probably safe because when
we run tests in parallel we use seperate processes, but it wouldn't work
if we switched to using threads.
Instead maintain test-local list of JS engines, and also a seperate list
of extra arguments for node and/v8.
In order to support the combined use of GLFW and WebGPU, GLFW
should not assign anything to the Emscripten modules context variable.
GLFW supports not creating any OpenGL context using the windows
attribute GLFW_CLIENT_API.
This prevents the user from seeing backtraces like:
```
Traceback (most recent call last):
File "./emcc.py", line 3310, in <module>
sys.exit(main(sys.argv))
File "./emcc.py", line 3303, in main
ret = run(args)
File "./emcc.py", line 916, in run
options, settings_changes, user_js_defines, newargs = parse_args(newargs)
File "./emcc.py", line 2446, in parse_args
options.pre_js += open(consume_arg_file()).read() + '\n'
FileNotFoundError: [Errno 2] No such file or directory: 'foo.js'
```
V8 has begun work supporting the updated EH proposal, so it's no longer
compatible with LLVM. We can re-enable these tests on CI when V8 and
LLVM converge again.
Just like emcc and other tools we don't need to know internally
that this is python tool. Plus using the launcher script means
we are testing whata our users will run.
This option uses the new wasm-split tool in binaryen to generate profile
data and then split a module into a primary and secondary component
modules that can be linked together at runtime.
This change adds posixtestsuite as a git submodule (in tests/third_party).
It also adds a new test suite that auto-populates based on the files
in `posixtestsuite/conformance/interfaces/pthread_*`
The submodule is optional in that the testsuite simply remains empty
if it is not found.
This change doesn't do much other than compile in the relevant parts of
musl.
For all functions exception thrd_detach we simply sompile the musl code.
For thrd_detach musl normally aliases this to pthread_detech in the
pthread_detach.c but we implement pthread_detech so add the alias there.
Fixes#9242
This means that tests don't necessarily need to call REPORT_RESULT.
I'm going to be taking advantage of this in #12923.
Also, don't initialize EXIT_STATUS to zero, but leave it as undefined.
It seems useful to be able to tell the diference between a program that
exits with a zero status and one that has not in fact exited yet.
The proxying of JS methods has no type info. It used to just use a JS
Number (a double) for everything, which works, except with BigInts
that's no longer enough.
We have to serialize type info in this case. This does a simple thing of
prefixing each argument with a non-zero value if it is a bigint. This is
not very compact, but is maybe good enough for now.
Also add a HEAP64 global array which is useful here (and will be
with wasm64 too).
Fixes#12079
Because this header is force-included into all browser tests it should
be as un-intrusive as possible.
This also involved adding includes to a few tests that were indirectly
relying on these includes. Better to IWYN.
Split out from #12928
Rather than use `with_report_result` to create modified versions
of the original sources we can instead just compile them with
`compile_btest` which takes care of adding `report_result.c` and
including `report_result.h`.
LLVM now emits larger code in unoptimized builds, with some extra local
assignments, instead of the simple pattern the test expected, which is to
set the local to the constant value and then do the call. Instead, it sets
the constants earlier on and then does some copying of those locals. I
can't see a simple way to parse that pattern in a useful way, so skip part
of the test in unoptimized builds. (If we think it's useful to test this kind of
thing, we'd need a different way entirely, probably, that isn't sensitive to
LLVM changes like this.)
Fixes#12903
This test is failing intermittently but is currently going
unnoticed since we were not checking the thread exit code.
This change should make it easier for somebody to pick up
this bug and fix it.
See: #12885
Also rename to test_html5_core so that running test_html5_core*
can be used to run all the variants without matching other tests
that exist with that prefix.
This means we don't need to pass -nostdsysteminc and also means we can
use shorter sysroot-relative paths when specifying include directories.
This is a step toawrds a real emscripten sysroot which is alos needed
for goma integration (IIUC).
See: #9353
This change make emscripten more like other platorms in that
`__pthread_self` is implementd in the header file with inline asm and
the public `pthread_self` function comes from pthread_self.c
Previously we had it kind of backwards where the low level
`__pthread_self` was calling into the higher level `pthread_self`.
This is also the first use of inline asm we have in emscripten
and its good to know that it works.
This doesn't actaully work right now because musl's syslog
depends on all kinds of things like sockets and being
able to open "/dev/console".
So this is really just a compile and link test.
The indirect dependency on malloc comes from that fact
that it calls gmtime_r which can allocate in JS.
When minified JS tests fail the resulting error messages from node
normally contain the entire program (since its all on the same line).
This didn't used to be the case with older versions of node IIRC.
This change used the exiting `limit_size` function when displaying JS
output and also limits the size of any give line to avoid flodding the
terminal with a million chars on a single line.
If folks find this annoying, or were relying on this output when running
tests we add options. For example we could limit this behaviour to only
apply run running in a terminal.
For the smallest hello world test, don't just check the output size
but check the full JS output matches the expectations.
That means that any change that touches those core lines of output
will need to rebaseline this test. However:
a) such changes deserve extra scrutiny
b) such changes should be few and far between
c) rebaselining is trivial (just run with EMTEST_REBASELINE=1)
Note that we do not compare the full wasm output since that is even
more fragile and can change with LLVM updates.
There are some cases where we want to create the memory in JS but for
most programs exporting the memory from wasm makes more sense and is
more compact and reliable.
The new EXTERNAL_MEMORY setting can be used to override this behaviour.
These two files come directly the libpng distribution and
so belong in third_party. I removed the local mod we had
done in favor of checking the existing output of the
upstream test code.
Fixes: #12795
This has several advantages:
1. Its easy to re-run just one of them when debugging.
2. Running all of them is way faster because the run in parallel
3. When they fail the name of the failing test gives more information
I believe this tests is always supposed to run with
MINIMAL_RUNTIME. Without this change the half of the
innter test runs just run with pthreads which is
redundant.
* Fix regression that broke calls to makeDynCall in JS library code. Old syntax is {{{ makeDynCall('sig') }}}(funcPtr, arg1, arg2);. New syntax is {{{ makeDynCall('sig', 'funcPtr') }}}(arg1, arg2). With this change, both old and new syntax work, with a compile time warning issued for old syntax to migrate to new one. Add ChangeLog entry about broken static dynCall invocation outside JS library files.
* Address review
When building direclty to wasm (e.g. with `-o <out>.wasm`
or with `--oformat=wasm`) we were not reporting undefined
symbols since we were not running the js compiler at all.
Fixed: #12725
Move config file handling into its own file. Also, move a few small
utilities into `utils.py` to avoid circular dependencies.
For a while now shared.py has been way too big and this is step towards
breaking it up and breaking the existing circular dependencies between
python modules that use and also also used by shared.py (cache.py for
example).
* Fix Closure to print clear readable error messages, instead of a single wall of minified text.
* Fix --cpuprofiler and --memoryprofiler to work with --closure 1, and add testing for --cpuprofiler, --memoryprofiler and --threadprofiler with Closure.
* Flake
* Address review.
* Address review
In compile-only mode we can write all object files directly to their
final locations completely avoid the need for temp files or for
renaming. Also avoid the `.d` file modification that we had in there.
This is a really nice simplification that has become possible as we have
made the emscripten code more precise about exactly which mode it is
running in.
Fixes: #12709
This is needed to allow a binaryen change to roll in
which gives a nice code size reduction:
https://github.com/WebAssembly/binaryen/pull/3325
Also relax the pattern matching used in another test
to as not to be sensitive the this change.
Because we were only checking for `#if` at the start of the
line we were accidenatlly also accepting `#ifdef` and also
`#ifanything`.
For backwards compatibility continue to accept `#ifdef` in
case people are using it in the wild but issue warning.
As a side effect of this refactor we not allow support
preprocessor directives that are not in column 0.
Its seems we painted with too broad a brush in #7116.
This change was generated programmatically using this type of thing:
for f in `git grep --name-only "//.*The Emscripten Authors" tests/third_party`; do tail -n +6 $f > tmp && /bin/mv tmp $f; done
Fixes: #12711
This is an initial PR that only supports the most basic of
wasi-sdk-produced programs. Most ones that just use stdout and argv.
Adds a simple test which uses the official wasi-sdk sysroot
to build the wasm binary and then uses emscripten to post-process
this to turn this into a runnable JS file.
See #12341
This change introduces a new output format (--oformat) called
`bare`. In `bare` mode emscripten will stop after running wasm-ld
and output just the bare wasm module.
It also introduces a new command line argument `--post-link` which
take as input a raw wasm module and run all the post link phases
on it (e.g. wasm-emscripten-finalize, jsifier, etc).
The precise command line UI for this feature is still in the
design phase so I've added a warning so that any users of this
command line UI know that it is experiment.
For libary config element such as `foo__sig` error out if
the corresponding library member is not found (in this
case if `foo` is not in the library.
This found `memcpy__inline` which can be removed since
memcpy is no longer part of the JS library.
Also remove `__inline` completely since this was the last
use of it, and its asmjs specific.
The filesystem library generally reports errors by throwing
`FS.ErrnoError` exceptions so it should not need ever need
to set errno directly.
Errno gets set in library_syscalls were each syscall is
is wrapped in a try/catch that returns errno back to libs.
In order to remove this dependency I had to remove the three
different uses of `setErrNo` in library_fs.js:
1. handleFSError. This only had two caller, both in
library.js so moved this there and added a seperate
dependency on setErrNo.
2. findObject: Only has a single caller in src/library_dylink.js
which only cared about a binary result and certainly
doesn't check the errno location.
3. forceLoadFile: Had exactly two internal callers that both
turned the error into an exception anyway, so just throw
the exception directly.
MODULARIZE + WASM_ASYNC_COMPILATION=0, that is, modularize mode but with
async compilation turned off, so that startup is synchronous, should return the
Module object from the factory function (as it would not make sense to return
a Promise without async startup).
Fixes#12647
This did not just work because we build libSDL2_mixer_ogg etc., with a suffix
for the codecs we build in. This PR adds an explicit mapping to handle such
cases.
Also improve the test to check this, and add a lot more checks there, like that
we don't use SDL1 somehow (which can happen with files like this where the
SDL1 and SDL2 APIs agree, and so SDL2 does not need to be linked in to
get a running executable).
Fixes#12589
Without this, the subprocess output (which will be flushed
on sub-process exit) may appear in the output before message
that we have written.
This particular effects python3 which changed the default
buffering of std streams when in non-interative mode:
https://stackoverflow.com/questions/22471023/python-sys-stderr-flush-frequencyFixes: #12654
When strings are converted without using TextDecoder, an off-by-on error
causes an out-of-bounds memory read. Fix this by turning the while loop
into a for loop with an appropriate bounds check.
Test the marshalling with null-terminated and non-null-terminated
strings. The code is based on the test for UTF32.
This change replaces the current PIC ABI (where function and data
addresses are resolved via `fp$` and `g$` function calls with the PIC
ABI use by llvm which involved importing `GOT.func` and `GOT.mem`
mutable wasm globals.
This allows us to pass `--new-pic-abi` to wasm-emscripten-finalize
which disabled the PIC ABI transformation normally done by binaryen.
As part of this change we now store the `__sig` of each JavaScript
function on the function so it is available at runtime. This
runtime signature is needed when a dynamically linked module
imports a JavaScript function by address. In this case the function
must be added to the table, and for that to succeed we need to know
its signature.
Unlike the other browser tests they don't use the test
runner harness and instead launch their own browser.
I was finding that running the test locally in isolation
that browser test harness would interfere with the
browser that emrun was starting.. since they both honor
EMTEST_BROWSER.
With this change, we no longer need any JavaScript variables to track the
state of the stack. The base and max as well as the current stack
pointer are all managed in native code.
The following JavaScript variable a no longer present in the library
code:
STACK_BASE -> emscripten_stack_get_base()
STACK_MAX -> emscripten_stack_get_end()
STACKTOP -> emscripten_stack_get_current()
TOTAL_STACK -> emscripten_stack_get_base() - emscripten_stack_get_end()
We have been error'ing out python3 usage for a while now.
This change finally removes the python2 support so there
is no way back.
And to show we mean it, include an f-string in there
for good measure.
Fixes: #7198
In this mode dlsym is supposed to be able to looking all symbols
in default library search path which should include JS symbols.
Unfortunately if the symbol being looked up is not already in the
table then its signature needs to be known at this point. To
handle this case we keep the signatures around at runtime by
attaching them to the JS library functions. This a small extra
overhead that only applies to dynamic linking.
This change is needed for and split out from #12461
This change includes a bunch of cleanups but the main change
here is to force the use of the Ninja generator. Without
this this test will try to VS or nmake on windows which fails
in various ways.
This change is needed since we stopped forcing the default
generator on windows in #12595 landed.
emcmake should be a tools that simply wraps cmake. We should not be
opinionated about the generator that a user might want to use of
try to work around MinGW issues on behalf of the user.
Without this flag the output format is determined by the name of the
output file.
With this flag the name of the output file is ignored and the format is
determined solely by this flag.
This can be useful in various situation but the motivation for doing
it now that it provides a place to specify a new output format which
will be the raw linker output, which can then be fed into that new
post-link-only mode that we are planning.
This workaround was inadvertantly disabled completely back in #9177. As
part of this change it became impossible to set the global variable
EM_POPEN_WORKAROUND which enables this workaround. After this change
the environment variable `EM_POPEN_WORKAROUND` and the config setting
`POPEN_WORKAROUND` both control the global variable calles
`POPEN_WORKAROUND` but its the global variable called
`EM_POPEN_WORKAROUND` that continues to enable this workaround.
Since that change landed over a year ago and nobody has complained
about this workaround not working I can only assume that we don't
have any Windows 7 users who I believe are the only ones effected
by this python bug.
* emrun: use argparse in more conventional way
This is not quite NFC since passing command line arguments
to the page that start with `-` now requires a separator.
This used to work:
./emrun filename.html --foo --bar
Where as now one needs to add `--` to signal the end of emrun's
argument parsing.
So either:
./emrun filename.html -- --foo --bar
or:
./emrun -- filename.html --foo --bar
* error_check
This basically involved looking the `orig$` functions whenever we
want a native WebAssembly function.
Regarding test_dlfcn_i64: It seems that this test wasn't actually
testing functions that have i64 in thier signature but some kind of i64
handling from back in the asm.js days. See #2060. This change
repurposes it to and makes it more useful.
This implicitly-open handle looks up symbols in the same way that they
are looked up when loading shared libraries. i.e. it searches all
symbols in the main module + any shared libraries loaded with
RTLD_GLOBAL.
In https://github.com/emscripten-core/emscripten/blob/master/src/library_exceptions.js,
the uncaught exception count is incremented and decremented exactly once during
__cxa_throw and __cxa_begin_catch respectively (as the ABI says it should be). This
however doesn't get re-incremented in the event of a rethrow, so anything that happens
between the first catch and the second catch (eg. destructors) receive the decremented
value.
In LLVM the count is incremented again in __cxa_rethrow, which curiously the ABI does
not ask us to do, but seems to be the behaviour we expect. Perhaps the ABI assumes
that rethrow calls throw under the hood. This PR brings us closer to that, and in alignment
with LLVM.
The testcase added here now emits the same correct output as a native build, which
it did not before this PR.
We can already use EXPORT_FUNCTIONS to export data symbols but
the symbol check was only looking at function exports.
This was leading to erroneous error message for users who were
doing `-s EXPORTED_FUNCTIONS=[data_symbol]`.
See: #12558
For example, if a global constructor performs an invalid memory access,
an exception will be thrown that bubbles up to receiveInstantiatedSource.
That causes a rejection of the promise returned from instantiateAsync,
and we should forward that to the module consumer via the module ready promise.
By handling that rejection we also avoid an unhandled-promise-rejection
error in browsers.
Fixes#12396
Third-party SDL modules query require a recent-ish version of SDL2 (e.g.
standard SDL2_image requires SDL2 >= 2.0.8). Emscripten currently
ships SDL2 tag version_22, that is 2.0.10.
Without this change we were using a normal Number and then
`wasm-emscripten-finalize` would use that value to set an
internal mutable global.
This change means that users emscripten's dynamic linking
now depends on the mutable globals proposal. However this
is already available in all major browsers:
https://webassembly.org/roadmap/
This is small step towards #12461 which can be landing
in isolation without otherwise changing the ABI.
Once this lands we can remove `internalizeStackPointerGlobal`
from `wasm-emscripten-finalize`.
- Use C over C++ in test_dlfcn_longjmp since the test
doesn't use C++ and it makes debugging simpler.
- Add/improve a couple of assertions
- Remove duplicate `var` in declaration wasmTable.
We had a redundant check there.
Also improve the test to handle the case where malloc failed in the constructor. This
might help debug the asan failure on the bot on this test (which does not fail locally
for me).
We used to require Binaryen postprocessing for EH and have a `fail`
requirement for it in `other.test_immutable_after_link`. I deleted the
requirement in #12399, but it could have been better if I didn't just
delete it but changed it to an `ok` requirement. This PR adds that.
Now that we are renaming invoke wrappers and `emscripten_longjmp_jmpbuf`
in the wasm backend, using Emscripten EH or SjLj does not need
Binaryen's postprocessing. This makes Emscripten not enable Binaryen
postprocessing using wasm-emscripten-finalize when we are using them,
and deletes all special handling for `emscripten_longjmp_jmpbuf`, given
that Emscripten will not see it.
Addresses:
WebAssembly/binaryen#3043WebAssembly/binaryen#3081
Companions:
https://reviews.llvm.org/D88697WebAssembly/binaryen#3191
When displaying a callstack from a wasm module names will either
not appear at all or they will be populated from the wasm names
section which never contains managled names.
As far as I can tell mangled names no longer appear be backtraces
so demanagling is no longer a useful features.
Sam Clegg
Aleksey Kliger (λgeek)
Milek7
juj
Basil Fierz
Derek Schuff
Alon Zakai
Wouter van Oortmerssen
Kai Ninomiya
sam142
jkae
Riccardo Magliocchetti
algestam
Sam Gao
Stephan Reiter
Beuc
Heejin Ahn
Doug Conmy