Граф коммитов

970 Коммитов

Автор SHA1 Сообщение Дата
Kleis Auke Wolthuizen 9cfcefb54d
Remove _pthread_isduecanceled in favor of ->cancel. NFC. (#15625) 2021-11-29 17:18:39 -08:00
Ethan Lee 5ec1962f49
[WASMFS] Remove EM_ASM from wasmfs.cpp (#15646)
Relevant Issue: #15041

Move EM_ASM block code to library_wasmfs.js.
2021-11-29 15:29:53 -08:00
Ethan Lee ae5c256cac
[WASMFS] Enforce file name char limit (#15641)
Relevant Issue: #15041
- Enforce file name character limit
2021-11-29 14:19:46 -08:00
Ethan Lee 0b10027619
[WASMFS] Generalize File Preloading (#15643)
Relevant Issue: #15041

Remove preloadFromJS from MemoryFile.
Provide a general preloading function on all DataFiles.
2021-11-29 13:51:47 -08:00
Ethan Lee c288dbb67d
[WASMFS] Wasmfs fix ENOENT (#15642)
Relevant Issue: #15041
- Improve tests and return ENOENT when path is empty
2021-11-29 16:20:20 -05:00
Ethan Lee d1195cb3a1
[WASMFS] Include file type in file mode (#15640)
* include file type in file mode

* fix open test
2021-11-29 16:19:42 -05:00
Ethan Lee 9297133fd7
update comment with node issue (#15631)
Updated comment to link ongoing issue with Node: nodejs/node#40961
2021-11-24 17:42:31 -08:00
Ethan Lee 1e57083618
[WASMFS] Switch to using `out` and `err` (#15629)
Add `emscripten_out` and `emscripten_err` along with `f` versions.
Replaced `emscripten_console_log`/`error` with those two functions respectively in `streams.cpp`.
2021-11-24 20:55:28 +00:00
Sam Clegg d7367b340d
Set emscripten_is_main_browser_thread based on ENVIRONMENT_IS_WEB (#15630)
The this is more accurate that !ENVIRONMENT_IS_WORKER, and holds
true, for example, under node where there is no main browser thread,
only a main runtime thread.
2021-11-24 20:08:08 +00:00
Ethan Lee e228888222
fix off by off by one in memory_file.h read and write (#15622)
Relevant Issue: #15041

Fix small unnecessary off-by-one size checks.
2021-11-24 10:09:33 -05:00
Ethan Lee 7e6d511d91
[WASMFS] JS File Backend Structure (#15562)
Relevant Issue: #15041

Tentative JS File Backend Structure

Introduced user-visible header file.
Users can create a new JS Backend and create JS Backed Files under both In-Memory and JS Backend directories.
2021-11-24 06:38:48 +01:00
Sam Clegg 362adbb791
Fix for unlink() of empty path (#15612)
See: #15599
Fixes: #15600
2021-11-23 16:09:49 -08:00
Kleis Auke Wolthuizen d9a4cf95b3
Check against thread id instead. NFC. (#15606) 2021-11-23 11:25:28 -08:00
Ethan Lee e4a54136d8
[WASMFS] Modular backend - MemoryFile (#15543)
Relevant Issue: #15041

- Introduce new modular backend structure.
- Convert existing file operations to use the MemoryFile backend.
2021-11-19 12:44:51 -08:00
Sam Clegg 9e97b0463c
Allocate stack for threads in native code. NFC (#15564)
Followup to #15533
2021-11-18 12:20:29 -08:00
Ethan Lee f3106922f6
[WASMFS] rename syscall (#15527)
Relevant Issue: #15041

Implement rename
Add to existing test_rename.c
Gave write permissions to root directory.
Introduce recursive_mutex and try locking mechanism.
2021-11-18 12:06:50 -08:00
Sam Clegg bb7b2e12b4
emmalloc: use SMALLEST_ALLOCATION_SIZE. NFC (#15567)
This macro was clearly written to be used in this function but has no
current users.  I could alternatively remove SMALLEST_ALLOCATION_SIZE
completely but that seems worse.
2021-11-18 11:46:23 -08:00
Sam Clegg d2b7c6f1d2
Move free'ing of thread data to native code. NFC (#15551)
I have a followup which also move the allocation of the
stack region to native code but that can land separately.
2021-11-18 07:33:53 -08:00
Sam Clegg 774c71bddd
Fix for code size regression in log/log2/pow due to recent musl upgrade (#15544)
Restore the older versions of these files and used them in place of the
new ones when optimizing for size.

These regressions were intended and deemed acceptable in upsteeam
musl:
e4dd65305a
2a3210cf4a
236cd056e8

Fixes: #15483
2021-11-16 17:41:13 -08:00
Sam Clegg 9914b83647
Move setting of initial thread state to native code. NFC (#15533) 2021-11-16 17:24:06 -08:00
Sam Clegg dc7acaa784
Fix for calling `pthread_getattr_np` on the main thread (#15540) 2021-11-16 11:48:44 -08:00
Sam Clegg b347ced682
Fix misues of `is_main_runtime_thread` over `is_main_browser_thread` (#15503)
In all these places the code is really attempting to figure out if it is
the main runtime thread.  i.e. the first place the program is loaded and
the place that runs the callback and async events send from secondary
threads.

The reason this mistake often goes unnoticed is that in almost all cases
the main runtime thread is also running on the main browser thread.

One easy way to see that `is_main_browser_thread` is the wrong question
to be asking in many of these cases is to remember that when emscripten
is started in a worker there is no main browser involved and so this
function will return false on *all* threads.

In the cast of `__timedwait.c` and `pthread_barrier_wait.c` the desire
is to avoid blocking the main runtime threads so that calls from other
threads can be processed by `emscripten_main_thread_process_queued_calls`.
`emscripten_main_thread_process_queued_calls` is expected to always run
on the main runtime thread, and not necessarily on the main browser
thread.  Indeed its first line is:

 `assert(emscripten_is_main_runtime_thread());`
2021-11-16 09:10:49 -08:00
Wouter van Oortmerssen 72fd2b4706
Fix pthread_attr_t having 9 elems instead of 10 (#15530)
Regression introduced here: https://github.com/emscripten-core/emscripten/pull/13006
We are relying on them having 10 elems thru the `_a_prio` field (in wasm64) and also offsets in library_pthread.js
2021-11-15 15:00:37 -08:00
Sam Clegg b9f61d527b
Move most of the pthread_join to native. NFC (#15502)
The only part of pthread_join that needs to remain in JS
is the code for calling `cleanupThread`.
2021-11-15 14:47:59 -08:00
Wouter van Oortmerssen e5c241cc7e
[Memory64] Made clock_t 64-bit compatible (#15500) 2021-11-15 14:41:36 -08:00
Sam Clegg d8c78d653a
Fix asan.test_emmalloc_memalign_corruption (#15499)
This test started failing after this line was added as part of #13006.
However it seems this change was unneeded.

See: #15498
2021-11-11 17:00:04 -08:00
Ethan Lee 73f2fc0e6d
[WASMFS] FS.readFile() (#15450)
Relevant Issue: #15041

Implement FS.readfile() which copies an in-memory file into JS.
Parameterize `test_sdl_audio_mix_channels` and `test_sdl_audio_mix` to work with `WASMFS`.
2021-11-11 16:07:49 -08:00
Ethan Lee 2d0ffdf265
[WASMFS] rmdir and unlink syscalls (#15455)
Relevant Issue: #15041

Implement rmdir and unlink syscalls.
Added to existing test unistd/unlink.c.
2021-11-10 18:14:59 -08:00
Sam Clegg 0522fe6614
Move pthread_tryjoin_np to native code (#15458)
This matches more closely the musl version.
2021-11-10 18:12:28 -08:00
Ethan Lee 0773510eb5
[WASMFS] lstat and stat syscalls (#15437)
Relevant Issue #15041 

- Implement `lstat` and `stat` syscalls.
- Edit relevant test in `wasmfs_stat.c`
2021-11-10 12:37:24 -08:00
Ethan Lee cc6a9cdc1c
Added missing case for lstat in musl (#15465)
Relevant Issue: #15041 

- Added missing case for lstat in musl.
2021-11-10 11:18:17 -08:00
Ethan Lee 6621c7b27f [WASMFS] getdents syscall
Relevant Issue: #15041

Implement getdents syscall and add required tests.
2021-11-09 17:18:29 -08:00
Ethan Lee b9c78a4834 [WASMFS] Fix mode on file and dir creation (#15445)
Relevant Issue: #15041
* fix mode on file and dir creation, update tests to verify
* file type is now derived from kind
* Add optional mode to open and refactor tests
2021-11-09 17:18:29 -08:00
Sam Clegg e05e72d9c4
Update musl to v1.2.2 (#13006)
The changes here mostly come from running the update_musl.py script
against the following commit:

https://github.com/emscripten-core/musl/tree/merge_v1.2.2

Notes:

- `weak` is defined in musl-internal headers so some of our usage of
  `__attribute__((weak))` was switched to `__weak__` (which is a
  synonym).

- Quite a few changes needed in library_pthread.js due to changes to
  the internal representation of __pthread struct in C.
  - removal of emscripten-specific threadStatus field.  This can
    can be fully replaced with the updated `detach_state` that
    was added to musl (along with enum values for states!) along
    with the existing `cancel` bit.
  - `attr` is no longer stored on `__pthread`; it was only duplicating
    other fields anyway.  We now include `pthread_getattr_np.c` which
    can pull those field out on demand into at attr structure.
2021-11-09 15:56:19 -08:00
Ethan Lee 7712529643
[Minor] Add wasmfs constructor to readme (#15464)
Relevant Issue: #15041

Updated list of static constructors in system/lib/README.md.
2021-11-09 13:21:01 -08:00
Sam Clegg f370d3a8bc
Fix test_proxy_to_pthread_stack test (#15463)
The type of the local used in this test was 4-bytes wide rather
than 1-byte wide.
2021-11-09 13:13:06 -08:00
Sam Clegg f11647ff10
Add scripts for copying changes out back into upstream llvm and musl (#15430)
We are now maintaining our llvm and musl patches in external
repositories so that it is easier to update from upstream
and rebase onto new releases.

I already created scripts for copying changes in one direction
(i.e. into emscripten).  This change adds push_llvm_changes.py
and push_musl_changes.py which can be used to push changes
in the other direction.
2021-11-05 10:09:48 -07:00
Ethan Lee 37e8f99ada
[WASMFS] Current Working Directory and Relative Paths (#15410)
Relevant Issue: #15041

Implement cwd and locking for the global wasmFS state object.
Add relative paths to general path parsing.
Add parent weak_ptr to all File nodes.
Add chdir and getcwd syscalls.
Add test for chdir and getcwd.
2021-11-05 00:59:16 +00:00
Sam Clegg 9e26babac0
Use musl's weak symbol trick to initialize main thread TSD block. NFC (#15441)
TSD == Thread Specific Data.

Basically, if there are any calls to to `pthread_key_create` we get a
full sized `__pthread_tsd_main` otherwise we get dummy one.

Split out from #13006.
2021-11-04 17:55:40 -07:00
Sam Clegg 6cb88b83c2
Remove SYS_dup2 syscall. NFC (#15418)
The musl implemenation of dup2 works fine without this syscall as it can
be implemented in terms of dup3.

As part of this change I was forced to implement __wasi_fd_fdstat_get
for wasmfs since dup2 not depends on this wasi syscall.

Split out from #15411
2021-11-03 16:13:58 -07:00
Ethan Lee 01b42bffbc
[WASMFS] Enable preloading files (#15368)
Relevant issue: #15041

Enable loading of files when --preload-file is specified.
FS.createDataFile and FS.createPath are emulated in the new file system.
Edited browser.test_preload_file() to check preloading functionality.
2021-11-03 12:31:11 -07:00
Sam Clegg ba71da1686
Compile and test faccessat musl function (#15408)
We already implemented `__syscall_faccessat` but we were not
using or testing it.
2021-11-02 12:43:29 -07:00
Sam Clegg e05a26f85b
Remove declarations of unused syscalls. NFC (#15409)
I found these programatically using:

```
$ cat system/lib/libc/musl/arch/emscripten/bits/syscall.h | awk '{print $3}' | sort | uniq > delcared.txt
$ llvm-nm cache/sysroot/lib/wasm32-emscripten/libc.a  cache/sysroot/lib/wasm32-emscripten/libsockets.a | grep "U __syscall_" | awk '{print $2}' | sort | uniq > used_syscalls.txt
$ diff -u used_syscalls.txt delcared.txt
```
2021-11-02 18:53:25 +00:00
Ethan Lee bad04ceac1
[WASMFS] Restructure source files to include global file state. (#15388)
Relevant Issue: #15041

- Refactored file structure to include a global wasmfs state object that one has to go through to gain access to the FileTable or getRootDirectory().
- Simplified streams.h header by moving function implementations into streams.cpp.
- Removed static member entries from FileTable due to new init_priority(100) on wasmFS.
2021-11-02 01:06:54 +01:00
Heejin Ahn 9ecfa19d67
Add Wasm SjLj support (#14976)
This adds SjLj handling support using Wasm EH instructions:
https://github.com/WebAssembly/exception-handling/blob/master/proposals/exception-handling/Exceptions.md

This does not yet support mixing of EH and SjLj.

These are SjLj tests that I couldn't attach `@with_both_sjlj_handling`
and the reasons:
- `test_dlfcn_longjmp`: Uses NodeRAWFS
- `test_longjmp_throw`: Mixes EH and SjLj
- `test_exceptions_longjmp1`: Mixes EH and SjLj
- `test_exceptions_longjmp2`: Mixes EH and SjLj
- `test_exceptions_longjmp3`: Mixes EH and SjLj
- `test_longjmp`: This currently uses `@also_with_standalone_wasm`, and
  it looks a test can't take two decorators when both of them take
  arguments, which is the case for `@also_with_standalone_wasm` and
  `@with_both_sjlj_handling`.
2021-11-01 16:33:21 -07:00
Sam Clegg bbc208cfdf
Initial support runtime dlopen in multi-threaded applications (#15317)
This change maintains a shared global list of DSOs loaded and adds a
`emscripten_thread_sync_code` helper function which can be used to bring
the current thread up to date by loading all the modules in the this
list.

The static table region allocated for a particular module is allocated
by the first loading thread.  All other threads will use the same
pre-allocated region.

This change does not yet deal with sychronizing pointers loaded by
`dlsym`.
2021-11-01 10:47:06 -07:00
Ethan Lee 093fd65f37
[Minor] Move around code and rename wasmfs.cpp -> syscalls.cpp (#15391)
- Rename wasmfs.cpp -> syscalls.cpp
- New streams.h file
- Moved some functions to file.cpp and file.h
2021-10-28 18:49:53 -07:00
Ethan Lee 5364b11763
[WASMFS] Move getSize member into File parent class (#15383)
Relevant Issue: #15041

- Move getSize() method into File parent class.
- Addresses comment in #15368.
2021-10-28 21:33:12 +01:00
Sam Clegg 6c1fa3015a
Move dlopen flags from args to memory. NFC (#15360)
This NFC change was split out from #15317.  These flags need to be
persisted and in shared memory so that each thread/worker can load with
the libraries with the same flags.
2021-10-26 16:24:31 -07:00
Ethan Lee c273f250d7
[WASMFS] Mkdir Syscall (#15323)
Relevant Issue: #15041
- Implement mkdir and create relevant tests for the new file system.
2021-10-25 17:10:07 -07:00