Граф коммитов

925 Коммитов

Автор SHA1 Сообщение Дата
David Chisnall b908e6e12c Fix the process sandbox.
libfmt changed a bunch of things, snmalloc is no longer in the tree.
2023-02-08 16:55:50 +00:00
Matthew Parkinson e434183362
Linking to new repo for runtime. (#627) 2022-10-24 16:17:12 +01:00
Matthew Parkinson 6d2ca07b6a
Add cown_ptr::weak to the API (#625) 2022-10-14 15:24:41 +01:00
Matthew Parkinson 301a538bd4
Small improvement to epoch rejoin code (#621) 2022-10-13 14:25:16 +01:00
Erik Mavrinac 70d70fc89e
Pass through additional ADO env vars to AnyBuild (#622)
To improve joins of AnyBuild telemetry with ADO telemetry.
2022-10-13 14:23:19 +01:00
Matthew Parkinson c067ace3a8
Fix bug in weak reference counting (#624)
* Fix region_rc to use correct method

* Fix bug in weak reference count

The existing implementation has a bug in promoting a weak reference
count to strong.  This address this bug, and
documents the fix, and provides a proof of the properties using
starling.
2022-10-13 14:11:03 +01:00
Matthew Parkinson 36734fd5d0
Some refactoring of reference count manipulation (#623)
* GC old code.

* Fix harness for deterministic epoch order

* Add collection of simple tests for debugging.

* Reduce reference count

* Remove a guard variable.
2022-10-11 15:18:59 +01:00
Matthew Parkinson 7edea0394e
Epoch fix (#619)
* Enable preventing pre-emption in systematic testing.

It can be useful to force a collection of operations to occur at a
sensitive point in an algorithm.  This block systematic testing from
breaking them into smaller operations.

* Disable FPWB in SysTesting

In systematic testing, there is no need to perform
FlushProcessWriteBuffers.  The program is executing sequentially
already, so this is not required.

* Fix bug in epoch mechanism

The logic was incorrectly determining if all the threads were in the
correct epoch.

The next commit will document this file.

* Make Asym lock return if it is the rentrant case.

* Add a few methods for checking the asym lock state

* Tidying and documentation of epoch mechanism

* Improve test coverage for noticeboard epoch usage

* Fixup changes to epoch mechanism

* Remove broken assert

* Increase frequency of epoch change

* White space

* Alter teardown to handle external threads.

* Fixup flushing.

* Fix atomic annotation

* Clangformat

* Minor

* CR with Sylvan

* Clangformat

* Improve rejoin protocol.

* Fix teardown leak.
2022-10-02 13:13:36 +01:00
Matthew Parkinson 1e9e46e0df
Update snmalloc (#620) 2022-09-30 14:25:02 +01:00
Matthew Parkinson 0bf233de8c
Remove leak detector (#617) 2022-09-29 09:57:14 +01:00
Luke Cheeseman e064cf576a
adding == and != comparison operators for cown_ptr and nullptr (#618)
* adding == and != comparison operators for cown_ptr and nullptr

* moving operator with member access inside struct

* adding nullptr assignment to cowns

* adding std to assignment operator
2022-09-28 18:42:00 +01:00
cui fliter 5bf2f75b7b
all: fix some typos (#616)
Signed-off-by: cui fliter <imcusg@gmail.com>
2022-09-20 09:14:28 +01:00
Matthew Parkinson 815741d2ce
Minor updates (#614)
* Win build fix.

* Update snmalloc.
2022-09-15 09:58:44 +01:00
ghosn adrien 4a41848d3e
Decoupling SchedulerThreads & Cown queues (#612) 2022-09-13 11:31:17 +01:00
Luke Cheeseman 61b55bc27c
Read-only Cowns (#595)
Co-authored-by: Matthew Parkinson <mattpark@microsoft.com>
2022-09-09 15:44:10 +01:00
Matthew Parkinson 49ee515333
Move header under conditional flag. (#613) 2022-09-05 17:54:32 +01:00
Matthew Parkinson 8f667eeea3
Remove assertion that is not true. (#610) 2022-09-01 18:25:44 +01:00
Matthew Parkinson 3e10870229
Consolidate allocation of message body (#609)
* Remove index as no longer needed.

* GC old code

* Optimise the multimessage body allocation

Prior to this commit the multimessage was composed of three allocations,
the body, the cowns array and the behaviour/closure to execute.  This
commit merges the three structures into a single
allocation.
2022-09-01 14:36:16 +01:00
Luke Cheeseman 84e0a1bd00
Add manual lock ordering to dining phils benchmark (#589)
* adding manual lock ordering for pthread dining philosophers benchmark

* do the index calculation only once in dining phils
2022-08-31 14:33:32 +01:00
Marios Kogias f9970e3921
Add Smallbank experiment (#601) 2022-08-31 13:43:35 +01:00
Matthew Parkinson 40862942a2
Optimise the RC for cown_ptr (#607) 2022-08-31 13:30:40 +01:00
Matthew Parkinson 956833494d
Fix #552 (#578)
Fixes #552. Their was a sporadic memory leak only observed on Windows. This PR adds some logging that helped diganose the issue, and fixes the issue.

The poller cown is decrefed by the external source, however this decref can occur
after the runtime has executed the teardown, and thus is leads to a memory
leak.

This commit changes the order so the last thing the external event source does
is decrease the event source count, and thus the runtime cannot begin teardown
until the external thread has completed all its actions.
2022-08-31 12:54:14 +01:00
Matthew Parkinson 2226b1c0cd
Fix `total_cowns` invariant (#608)
The variable `total_cowns` should be the length of the per-thread cown list.
This is not the case, this commit checks that this is the case, and correctly
updates it.
2022-08-30 17:47:10 +01:00
Matthew Parkinson cde8bcf647
Fix bug in C++ cown api (#603) 2022-07-12 18:02:27 +01:00
David Chisnall e2fe309320 Fix process sandbox for double faults.
The `snmalloc::message` function calls `fsync` on `stderr`.  With
Capsicum, this is an explicit permission and so tests that hit a code
path that did some logging (specifically, the curl test) in debug builds
during system call emulation would receive the second trap.

The first symptom of this is fixed by adding fsync permission to the
standard out / error file descriptors.  With that change, all of the
tests pass but the underlying problem remains: the signal is delivered
*after* return from the first system call, at which point it does not
have a sensible `ucontext` and things break in exciting ways
(sufficiently exciting that a debugger can't tell you anything useful).

The root cause is addressed by adding `SA_NODEFER` when registering the
signal handler.  This allows system calls that are used to emulate
other signal handlers to deliver their signals on top of the stack.  If
a system call is used in emulating itself, this still goes horribly
wrong, but in a way that's much easier to debug.

At the same time, use the SIGCAP support that is in process of being
upstreamed to FreeBSD.  This makes debugging Capsicum failures easier
because the debugger does not try to hide the signal from the
application whose correctness depends on handling it.
2022-07-12 10:22:38 +01:00
Matthew Parkinson 301482fe1a
Improved API for CPP (#598)
* Improved API for CPP

This introduces a namespace for the verona C++ api:
  verona::cpp

It also fixes a couple of move/copy constructor/operators.

* Fix assert, now assignment operator is correct.
2022-07-08 10:05:23 +01:00
Matthew Parkinson 95b3315efb
Updating snmalloc (#596) 2022-06-08 09:38:17 +01:00
David Chisnall 4d23e93ea6 [process-sandbox] Defence in depth
Add some deterministic initialisation to a buffer that is populated with
a variable-length attacker-controlled structure.

Fixes #561
2022-05-25 10:29:55 +01:00
David Chisnall 08cabb1a62 Try to fix the snmalloc update checker. 2022-05-20 13:48:13 +01:00
David Chisnall 5f4b094c95 Make sandbox allocation functions return an option type.
This forces the caller to check for allocation failure and not blindly
dereference the pointer and hit a null pointer.

Fixes #588
2022-05-20 13:48:13 +01:00
David Chisnall 23a0eeea3b Don't do blocking operations in the parent.
The sandbox can leave the sockets' buffers full and so in the parent we
should always do non-blocking operations.

This includes some refactoring to avoid code duplication.  The new
functions are now marked nodiscard and the resulting errors are fixed.

Fixes #583
Fixes #584
Fixes #586
2022-05-20 13:48:13 +01:00
Matthew Parkinson 67fc1bb53b
Update snmalloc (#591)
* Update snmalloc

* Updated sandbox.
2022-05-10 19:06:31 +01:00
Luke Cheeseman 37ed8e7491
Add copy assignment operator for cown_ptr (#590) 2022-05-08 20:43:02 +01:00
David Chisnall b75b96bd2b Add more validation for DeallocChunk messages
Fixes #574
Fixes #575
Fixes #576

Also set timeout on tests.
They all currently complete in <1s, so a 30s timeout is plenty.
2022-04-29 14:07:38 +01:00
Renato Golin dbd6da4c11 Organise people in docs 2022-04-27 18:08:48 +01:00
Renato Golin 51f286437e Update my page/photo on docs 2022-04-27 15:07:23 +01:00
Matthew Parkinson 1cdcc8273b
C++ When support (#572)
* WIP: Add some perf benchmarking

* Try different experiment configs

- Implement optimal scheduling order to break the dining philosophers
  causality. Schedule first the even philosophers and then the odd ones.
- Break causality through scheduling an intermediate empty cown. The
  results of this configuration are highly variable but always better than
  the one with causal order.
- Implement a busy loop for the use of each fork. This removes the OS
  scheduling overheads.

* Add pthread experiment

* Break circular dependency

* Add automatic deadlock prevention

* Tidied example. (#5)

* Unified the pthread and verona benchmarks.

* Another benchmark

* Change params

* More changes.

* Add missing file.

* Rename

* Add bank and logger pthread bench

* Unifying next example.

* Updated the bank example.

* WIP

* Added a static assert to help with errors.

* Rearrange

* Clang format

* Copyright

* Fix assert.

* Code review

* Clangformat

* CR

* Remove ref count optimisation for now.

* Clangformat

Co-authored-by: Marios Kogias <marioskogias@gmail.com>
2022-04-25 20:17:57 +01:00
Matthew Parkinson e7e3e5d0f4 Event source lifetime with SharedPtr
This makes the lifetime of the event source allocation a shared_ptr to
make the lifetime clearer.  This was giving memory leaks previously
#552.
2022-04-25 19:27:59 +01:00
David Chisnall 26a0b4ee45
Report an error if getaddrinfo is called with an invalid argument. (#570)
Fixes #568
2022-04-21 10:29:22 +01:00
Matthew Parkinson 9692e28dde Remove original acquire one
The semantics of AcquireAll appears superior in almost every way to the
original chain of acquiring cowns.  This
commit removes the original semantics.
2022-04-21 09:14:25 +01:00
David Chisnall cb3c14d62c
Fix DoS from the sandbox. (#567)
If the sandbox tried writing incomplete messages to the parent, or
not reading responses, it was possible to end up with a blocking read or
write call and either an infinite loop or the parent crashing.

Fixes #566
2022-04-19 17:06:21 +01:00
David Chisnall 525156101e
Add some missing checks. (#565)
Includes a test case derived from @saaramar's example.

Also fix another corner case:

The child can set the remote pointer for any allocation, to anywhere in
the shared memory region.  This region also includes the memory that
contains the parent's message queue for the sandbox.  Not checking this
meant that the sandbox was able to persuade the parent that it owned
a memory chunk, with the metadata pointer completely under the control
of the attacker.

Fixes #562
2022-04-19 14:27:05 +01:00
David Chisnall 982bb7e4d6 [NFC] Add a comment about metadata ownership.
Addresses the confusion in #563
2022-04-19 10:04:00 +01:00
David Chisnall 77dde60984 The prior version of snmalloc was a PR branch, not main. 2022-04-19 09:50:00 +01:00
David Chisnall cd77fd469a Add some defence in depth for the getaddrinfo host call.
This function takes a `hint` parameter that uses the same structure as
the return type, but ignores all except for four fields.  The others are
required to be null.

It is possible that a buggy implementation of the checker could
try to dereference the pointer fields.  By zeroing them at the boundary,
we add some defence in depth.

@saaramar concluded that this was not currently a security issue but the
fact that it showed up on his review at all suggests that the security
properties of the code are insufficiently clear, since the existing code
was safe only in the presence of the specific behaviour of the callee
implementation.
2022-04-14 12:15:30 +01:00
David Chisnall a312df1d64 Fix missing check for path canonicalisation.
bug found by @saaramar.

This is not exploitable, because the `/..` doesn't exist, but it may
hide other bugs.
2022-04-14 11:46:53 +01:00
David Chisnall 7892507cc1 Remove accidental debugging code. 2022-04-13 17:10:26 +01:00
David Chisnall 086575ab36
Fix a bug in bounds checking that @saaramar found. (#555)
Also add a test to make sure that it stays fixed.
2022-04-13 15:15:46 +01:00
David Chisnall 950cd20d08
Update for new snmalloc. (#554) 2022-04-11 17:06:45 +01:00
Matthew Parkinson 48d3ac28bf Alter mark_sleeping notify
Only return a notification if the queue only contains the notification.
2022-03-24 11:55:46 +00:00