microsoft
/
snmalloc
зеркало из https://github.com/microsoft/snmalloc.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
snmalloc/CMakeLists.txt

362 строки
14 KiB
CMake
Исходник Обычный вид История

Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
cmake_minimum_required(VERSION 3.8)
project(snmalloc C CXX)
Default build to Release. (#177)
2020-05-06 20:24:45 +03:00
if (NOT CMAKE_BUILD_TYPE)
message(STATUS "No build type selected, default to: Release")
set(CMAKE_BUILD_TYPE "Release")
endif()
CI for both arms (#151) * Add another ARM
2020-03-20 12:17:38 +03:00
include(CheckCXXCompilerFlag)
add android support (#171) * adjust for android * update docs * add const qualifier to `alloc_size` * check const qualifier in cmake
2020-04-18 09:58:13 +03:00
include(CheckCSourceCompiles)
CI for both arms (#151) * Add another ARM
2020-03-20 12:17:38 +03:00
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
option(USE_SNMALLOC_STATS "Track allocation stats" OFF)
Changed abort behaviour for Windows CI.
2019-08-13 17:37:54 +03:00
option(SNMALLOC_CI_BUILD "Disable features not sensible for CI" OFF)
Splitting EXPOSE_EXTERNAL_ALLOCATOR_API to EXPOSE_EXTERNAL_PAGEMAP and EXPOSE_EXTERNAL_RESERVE. Only the shim can be affected by passing the new pre-processor definitions to cmake.
2019-04-12 14:32:37 +03:00
option(EXPOSE_EXTERNAL_PAGEMAP "Expose the global pagemap" OFF)
option(EXPOSE_EXTERNAL_RESERVE "Expose an interface to reserve memory using the default memory provider" OFF)
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
option(SNMALLOC_RUST_SUPPORT "Build static library for rust" OFF)
add static library (#189) * add static library * leave static library on by default * fix 1mib layout * code reviews
2020-05-19 08:46:40 +03:00
option(SNMALLOC_STATIC_LIBRARY "Build static libraries" ON)
Workaround for QEMU behaviour. (#147) * Fixes for ARM * Workaround for QEMU behaviour.
2020-03-19 15:37:44 +03:00
option(SNMALLOC_QEMU_WORKAROUND "Disable using madvise(DONT_NEED) to zero memory on Linux" Off)
Make binaries more compatible by default (#206) * Make binaries more compatible by default Turn `-march=native` off by default. This makes binaries more portable, but may harm performance. However, fast paths look unaltered * Change setting to on if specified.
2020-05-28 18:56:48 +03:00
option(SNMALLOC_OPTIMISE_FOR_CURRENT_MACHINE "Compile for current machine architecture" Off)
add static library (#189) * add static library * leave static library on by default * fix 1mib layout * code reviews
2020-05-19 08:46:40 +03:00
set(SNMALLOC_STATIC_LIBRARY_PREFIX "sn_" CACHE STRING "Static library function prefix")
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
option(SNMALLOC_USE_CXX17 "Build as C++17 for legacy support." OFF)
Place the next pointer at a different place on every object. This limits the collision in cache buckets, especially for larger objects.
2019-02-12 22:08:31 +03:00
Fix the sandbox use case and add a test. (#269) Summary of changes: - Add a new PAL that doesn't allocate memory, which can be used with a memory provider that is pre-initialised with a range of memory. - Add a `NoAllocation` PAL property so that the methods on a PAL that doesn't support dynamically reserving address space will never be called and therefore don't need to be implemented. - Slightly refactor the memory provider class so that it has a narrower interface with LargeAlloc and is easier to proxy. - Allow the address space manager and the memory provider to be initialised with a range of memory. This may eventually also remove the need for (or, at least, simplify) the Open Enclave PAL. This commit also ends up with a few other cleanups: - The `malloc_useable_size` CMake test that checks whether the parameter is const qualified was failing on FreeBSD where this function is declared in `malloc_np.h` but where including `malloc.h` raises an error. This should now be more robust. - The BSD aligned PAL inherited from the BSD PAL, which does not expose aligned allocation. This meant that it exposed both the aligned and non-aligned allocation interfaces and so happily accepted incorrect `constexpr` if blocks that expected one or the other but accidentally required both to exist. The unaligned function is now deleted so the same failures that appear in CI should appear locally for anyone using this PAL.
2021-01-11 17:06:51 +03:00
# malloc.h will error if you include it on FreeBSD, so this test must not
# unconditionally include it.
add android support (#171) * adjust for android * update docs * add const qualifier to `alloc_size` * check const qualifier in cmake
2020-04-18 09:58:13 +03:00
CHECK_C_SOURCE_COMPILES("
Fix the sandbox use case and add a test. (#269) Summary of changes: - Add a new PAL that doesn't allocate memory, which can be used with a memory provider that is pre-initialised with a range of memory. - Add a `NoAllocation` PAL property so that the methods on a PAL that doesn't support dynamically reserving address space will never be called and therefore don't need to be implemented. - Slightly refactor the memory provider class so that it has a narrower interface with LargeAlloc and is easier to proxy. - Allow the address space manager and the memory provider to be initialised with a range of memory. This may eventually also remove the need for (or, at least, simplify) the Open Enclave PAL. This commit also ends up with a few other cleanups: - The `malloc_useable_size` CMake test that checks whether the parameter is const qualified was failing on FreeBSD where this function is declared in `malloc_np.h` but where including `malloc.h` raises an error. This should now be more robust. - The BSD aligned PAL inherited from the BSD PAL, which does not expose aligned allocation. This meant that it exposed both the aligned and non-aligned allocation interfaces and so happily accepted incorrect `constexpr` if blocks that expected one or the other but accidentally required both to exist. The unaligned function is now deleted so the same failures that appear in CI should appear locally for anyone using this PAL.
2021-01-11 17:06:51 +03:00
#if __has_include(<malloc_np.h>)
#include <malloc_np.h>
#if __has_include(<malloc/malloc.h>)
#include <malloc/malloc.h>
#else
add android support (#171) * adjust for android * update docs * add const qualifier to `alloc_size` * check const qualifier in cmake
2020-04-18 09:58:13 +03:00
#include <malloc.h>
Fix the sandbox use case and add a test. (#269) Summary of changes: - Add a new PAL that doesn't allocate memory, which can be used with a memory provider that is pre-initialised with a range of memory. - Add a `NoAllocation` PAL property so that the methods on a PAL that doesn't support dynamically reserving address space will never be called and therefore don't need to be implemented. - Slightly refactor the memory provider class so that it has a narrower interface with LargeAlloc and is easier to proxy. - Allow the address space manager and the memory provider to be initialised with a range of memory. This may eventually also remove the need for (or, at least, simplify) the Open Enclave PAL. This commit also ends up with a few other cleanups: - The `malloc_useable_size` CMake test that checks whether the parameter is const qualified was failing on FreeBSD where this function is declared in `malloc_np.h` but where including `malloc.h` raises an error. This should now be more robust. - The BSD aligned PAL inherited from the BSD PAL, which does not expose aligned allocation. This meant that it exposed both the aligned and non-aligned allocation interfaces and so happily accepted incorrect `constexpr` if blocks that expected one or the other but accidentally required both to exist. The unaligned function is now deleted so the same failures that appear in CI should appear locally for anyone using this PAL.
2021-01-11 17:06:51 +03:00
#endif
add android support (#171) * adjust for android * update docs * add const qualifier to `alloc_size` * check const qualifier in cmake
2020-04-18 09:58:13 +03:00
size_t malloc_usable_size(const void* ptr) { return 0; }
int main() { return 0; }
" CONST_QUALIFIED_MALLOC_USABLE_SIZE)
Fix building on old libc systems without `getentropy` (#329) Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
2021-05-25 18:46:06 +03:00
# older libcs might not have getentropy, e.g. it appeared in gliobc 2.25
# so we need to fallback if we cannot compile this
CHECK_C_SOURCE_COMPILES("
#if __has_include(<unistd.h>)
# include <unistd.h>
#endif
#if __has_include(<sys/random.h>)
#include <sys/random.h>
#endif
int main() {
int entropy = 0;
int res = getentropy(&entropy, sizeof(entropy));
return res;
}
" SNMALLOC_PLATFORM_HAS_GETENTROPY)
Change behaviour of USE_POSIX_COMMIT_CHECKS The previous setting applied USE_POSIX_COMMIT_CHECKS to snmalloc if it was a non-release build. This caused issues in CCF virtual mode, as it was being built in RelWithDebInfo. This commit changes the flag to be applied less, but for tests to always apply the setting independent of build type. This means that when snmalloc is being used as a library, it will be off, unless explicitly requested.
2021-02-19 12:07:42 +03:00
if (NOT SNMALLOC_CI_BUILD)
Code review feedback.
2020-02-05 15:47:24 +03:00
option(USE_POSIX_COMMIT_CHECKS "Instrument Posix PAL to check for access to unused blocks of memory." Off)
else ()
Change behaviour of USE_POSIX_COMMIT_CHECKS The previous setting applied USE_POSIX_COMMIT_CHECKS to snmalloc if it was a non-release build. This caused issues in CCF virtual mode, as it was being built in RelWithDebInfo. This commit changes the flag to be applied less, but for tests to always apply the setting independent of build type. This means that when snmalloc is being used as a library, it will be off, unless explicitly requested.
2021-02-19 12:07:42 +03:00
# This is enabled in every bit of CI to detect errors.
Code review feedback.
2020-02-05 15:47:24 +03:00
option(USE_POSIX_COMMIT_CHECKS "Instrument Posix PAL to check for access to unused blocks of memory." On)
endif ()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
# Provide as macro so other projects can reuse
macro(warnings_high)
if(MSVC)
# Force to always compile with W4
if(CMAKE_CXX_FLAGS MATCHES "/W[0-4]")
string(REGEX REPLACE "/W[0-4]" "/W4" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /W4")
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
endif()
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
# /Wv18 is required for the annotation to force inline a lambda.
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
add_compile_options(/WX /wd4127 /wd4324 /wd4201)
Use VirtualAlloc2 on Windows. The newer API (Windows 10 and newer) allows the allocator to ask for strongly aligned memory. This is enabled only if the `WINVER` macro is set to target Windows 10 or newer. There is now a CMake option to target older versions of Windows, so we can test both code paths. The Azure Pipelines config now includes a test of the compatibility version. This runs only the release build, because it's mainly there as a sanity check - 99% of the code is the same as the default Windows config.
2019-02-19 15:34:53 +03:00
else()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
Add -Wconversion to clang builds MSVC has strong opinions on implicit conversions as used in CI, while Clang both locally and in CI has weaker opinions. In an effort to avoid subsequent roundtrips through CI, make clang more strict. Adding -Wconversion definitely increases the strength of clang's opinions, apparently to include frowning on some that even MSVC considers OK, so go make explicit the current implicit behavior.
2021-02-24 15:54:14 +03:00
add_compile_options(-Wsign-conversion -Wconversion)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
endif ()
-Wundef
2019-07-25 22:06:03 +03:00
add_compile_options(-Wall -Wextra -Werror -Wundef)
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
endif()
endmacro()
Make clangformat a macro to allow sharing.
2019-05-22 15:20:43 +03:00
macro(clangformat_targets)
# The clang-format tool is installed under a variety of different names. Try
Update to use clangformat9 With clangformat9 the AfterCaseLabel is introduced. And this defaults to false, but our code is formatted implicitly with this set to true. PRs to Verona and Snmalloc are being formatted with clangformat9, and this is causing complexity. Let's move forward to clangformat9 in CI.
2020-02-06 12:09:32 +03:00
# to find a sensible one. Only look for versions 9 explicitly - we don't
Don't use our own loop to find clang-format find_program can do that for us. Additionally that loop was resetting the CLANG_FORMAT every time ninja was run, making it impossible to pass a specific -DCLANG_FORMAT=... to cmake. I've tried version 6 to 8 and formatting remains stable. clang 9 is when it breaks down because of AfterCaseLabel.
2020-01-30 23:31:20 +03:00
# know whether our clang-format file will work with newer versions of the
Update to use clangformat9 With clangformat9 the AfterCaseLabel is introduced. And this defaults to false, but our code is formatted implicitly with this set to true. PRs to Verona and Snmalloc are being formatted with clangformat9, and this is causing complexity. Let's move forward to clangformat9 in CI.
2020-02-06 12:09:32 +03:00
# tool. It does not work with older versions as AfterCaseLabel is not supported
# in earlier versions.
Don't use our own loop to find clang-format find_program can do that for us. Additionally that loop was resetting the CLANG_FORMAT every time ninja was run, making it impossible to pass a specific -DCLANG_FORMAT=... to cmake. I've tried version 6 to 8 and formatting remains stable. clang 9 is when it breaks down because of AfterCaseLabel.
2020-01-30 23:31:20 +03:00
find_program(CLANG_FORMAT NAMES
Add missing clang-format name.
2021-01-07 13:43:14 +03:00
clang-format90 clang-format-9)
Make clangformat a macro to allow sharing.
2019-05-22 15:20:43 +03:00
# If we've found a clang-format tool, generate a target for it, otherwise emit
# a warning.
if (${CLANG_FORMAT} STREQUAL "CLANG_FORMAT-NOTFOUND")
message(WARNING "Not generating clangformat target, no clang-format tool found")
else ()
message(STATUS "Generating clangformat target using ${CLANG_FORMAT}")
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
file(GLOB_RECURSE ALL_SOURCE_FILES CONFIGURE_DEPENDS src/*.cc src/*.h src/*.hh)
Add C++ concept for PAL This will not be used unless the C++ standard version is raised to 20. As concepts and C++20 more generally are quite new, this does not do so. Nevertheless, the use of concepts can improve the local development experience as type mismatches are discovered earlier (at template invocation rather than only during expansion).
2020-09-02 14:41:09 +03:00
# clangformat does not yet understand concepts well; for the moment, don't
# ask it to format them. See https://reviews.llvm.org/D79773
NFC: Add an AAL Concept, too While here, pull out some constants to their own header. Eventually we'll want to match on AalFeatures in the AAL Concept.
2020-11-20 15:56:03 +03:00
list(FILTER ALL_SOURCE_FILES EXCLUDE REGEX "src/[^/]*/[^/]*_concept\.h$")
Make clangformat a macro to allow sharing.
2019-05-22 15:20:43 +03:00
add_custom_target(
clangformat
COMMAND ${CLANG_FORMAT}
-i
${ALL_SOURCE_FILES})
endif()
endmacro()
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
# Have to set this globally, as can't be set on an interface target.
if(SNMALLOC_USE_CXX17)
set(CMAKE_CXX_STANDARD 17)
else()
set(CMAKE_CXX_STANDARD 20)
endif()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
# The main target for snmalloc
add_library(snmalloc_lib INTERFACE)
target_include_directories(snmalloc_lib INTERFACE src/)
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
if(NOT MSVC)
find_package(Threads REQUIRED COMPONENTS snmalloc_lib)
target_link_libraries(snmalloc_lib INTERFACE ${CMAKE_THREAD_LIBS_INIT})
if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
target_link_libraries(snmalloc_lib INTERFACE atomic)
endif()
portability: platform compiler flags -mcx16 and -march=native are platform specific. Leave them out if we aren't targeting chips that need it.
2019-04-23 15:42:10 +03:00
if(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86_64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
atomic swap 16 flag missing fix. In some platforms, it is rather amd64 than the more common x86_64, thus missing to rightfully set this flag.
2019-11-28 12:30:43 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "amd64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
fix mingw
2020-03-08 17:13:49 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "AMD64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
portability: platform compiler flags -mcx16 and -march=native are platform specific. Leave them out if we aren't targeting chips that need it.
2019-04-23 15:42:10 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
portability: platform compiler flags -mcx16 and -march=native are platform specific. Leave them out if we aren't targeting chips that need it.
2019-04-23 15:42:10 +03:00
# XXX elseif ARM?
endif()
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
endif()
if (WIN32)
Use VirtualAlloc2 on Windows. The newer API (Windows 10 and newer) allows the allocator to ask for strongly aligned memory. This is enabled only if the `WINVER` macro is set to target Windows 10 or newer. There is now a CMake option to target older versions of Windows, so we can test both code paths. The Azure Pipelines config now includes a test of the compatibility version. This runs only the release build, because it's mainly there as a sanity check - 99% of the code is the same as the default Windows config.
2019-02-19 15:34:53 +03:00
set(WIN8COMPAT FALSE CACHE BOOL "Avoid Windows 10 APIs")
if (WIN8COMPAT)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
target_compile_definitions(snmalloc_lib INTERFACE -DWINVER=0x0603)
message(STATUS "snmalloc: Avoiding Windows 10 APIs")
Use VirtualAlloc2 on Windows. The newer API (Windows 10 and newer) allows the allocator to ask for strongly aligned memory. This is enabled only if the `WINVER` macro is set to target Windows 10 or newer. There is now a CMake option to target older versions of Windows, so we can test both code paths. The Azure Pipelines config now includes a test of the compatibility version. This runs only the release build, because it's mainly there as a sanity check - 99% of the code is the same as the default Windows config.
2019-02-19 15:34:53 +03:00
else()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
message(STATUS "snmalloc: Using Windows 10 APIs")
# VirtualAlloc2 is exposed by mincore.lib, not Kernel32.lib (as the
# documentation says)
target_link_libraries(snmalloc_lib INTERFACE mincore)
Removing warnings from Windows Ninja configuration Windows Ninja build was complaining about /W4 and /W3 being specified, and C++ latest and C++ 17. This fixs those complaints.
2019-01-17 21:23:06 +03:00
endif()
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
endif()
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
Few build tweaks.
2020-03-17 14:18:24 +03:00
if(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86_64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
Few build tweaks.
2020-03-17 14:18:24 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "amd64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
Few build tweaks.
2020-03-17 14:18:24 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "AMD64")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
Few build tweaks.
2020-03-17 14:18:24 +03:00
elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86")
Gated the mcx16 flag to C++ compilation (#332) * Restricted mcx16 option to C++ files as only those can include the snmalloc header
2021-05-28 19:13:49 +03:00
target_compile_options(snmalloc_lib INTERFACE $<$<COMPILE_LANGUAGE:CXX>:-mcx16>)
implementations moved on the composer class.
2020-03-13 18:39:49 +03:00
endif()
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
endif ()
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
if(USE_SNMALLOC_STATS)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
target_compile_definitions(snmalloc_lib INTERFACE -DUSE_SNMALLOC_STATS)
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
endif()
Workaround for QEMU behaviour. (#147) * Fixes for ARM * Workaround for QEMU behaviour.
2020-03-19 15:37:44 +03:00
if(SNMALLOC_QEMU_WORKAROUND)
target_compile_definitions(snmalloc_lib INTERFACE -DSNMALLOC_QEMU_WORKAROUND)
endif()
Changed abort behaviour for Windows CI.
2019-08-13 17:37:54 +03:00
if(SNMALLOC_CI_BUILD)
target_compile_definitions(snmalloc_lib INTERFACE -DSNMALLOC_CI_BUILD)
endif()
Code review feedback.
2020-02-05 15:47:24 +03:00
if(USE_POSIX_COMMIT_CHECKS)
target_compile_definitions(snmalloc_lib INTERFACE -DUSE_POSIX_COMMIT_CHECKS)
endif()
Fix building on old libc systems without `getentropy` (#329) Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
2021-05-25 18:46:06 +03:00
if(SNMALLOC_PLATFORM_HAS_GETENTROPY)
Fix still non getentropy platform builds
2021-05-30 02:12:47 +03:00
target_compile_definitions(snmalloc_lib INTERFACE -DSNMALLOC_PLATFORM_HAS_GETENTROPY)
Fix building on old libc systems without `getentropy` (#329) Signed-off-by: Matthias Wahl <mwahl@wayfair.com>
2021-05-25 18:46:06 +03:00
endif()
add android support (#171) * adjust for android * update docs * add const qualifier to `alloc_size` * check const qualifier in cmake
2020-04-18 09:58:13 +03:00
if(CONST_QUALIFIED_MALLOC_USABLE_SIZE)
target_compile_definitions(snmalloc_lib INTERFACE -DMALLOC_USABLE_SIZE_QUALIFIER=const)
endif()
Code review feedback.
2020-02-05 15:47:24 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
# To build with just the header library target define SNMALLOC_ONLY_HEADER_LIBRARY
# in containing Cmake file.
if(NOT DEFINED SNMALLOC_ONLY_HEADER_LIBRARY)
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
warnings_high()
Address review comments.
2019-02-19 00:29:32 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
if(MSVC)
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Zi")
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} /DEBUG")
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
target_compile_definitions(snmalloc_lib INTERFACE -D_HAS_EXCEPTIONS=0)
add_compile_options(/std:c++latest)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
else()
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
add_compile_options(-fno-exceptions -fno-rtti -fomit-frame-pointer -ffunction-sections)
Haiku debug build, rdynamic flag unsupported.
2020-09-11 19:52:22 +03:00
# Static TLS model is unsupported on Haiku.
# All symbols are always dynamic on haiku and -rdynamic is redundant (and unsupported).
Haiku support proposal. (#218) * Haiku support proposal. Basic PAL implementation. This platform does not support TLS modes. * MAP_NORESERVE usage
2020-06-30 13:01:49 +03:00
if (NOT CMAKE_SYSTEM_NAME MATCHES "Haiku")
add_compile_options(-ftls-model=initial-exec)
Haiku debug build, rdynamic flag unsupported.
2020-09-11 19:52:22 +03:00
if(SNMALLOC_CI_BUILD OR (${CMAKE_BUILD_TYPE} MATCHES "Debug"))
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
# Get better stack traces in CI and Debug.
target_link_libraries(snmalloc_lib INTERFACE "-rdynamic")
add_compile_options(-g)
Haiku debug build, rdynamic flag unsupported.
2020-09-11 19:52:22 +03:00
endif()
Defensive code for alloc/dealloc during TLS teardown (#161) * Defensive code for alloc/dealloc during TLS teardown If an allocation or deallocation occurs during TLS teardown, then it is possible for a new allocator to be created and then this is leaked. On the mimalloc-bench mstressN benchmark this was observed leading to a large memory leak. This fix, detects if we are in the TLS teardown phase, and if so, the calls to alloc or dealloc must return the allocator once they have perform the specific operation. Uses a separate variable to represent if a thread_local's destructor has run already. This is used to detect thread teardown to put the allocator into a special slow path to avoid leaks. * Added some printing first operation to track progress * Improve error messages on posix Flush errors, print assert details, and present stack traces. * Detect incorrect use of pool. * Clang format. * Replace broken LL/SC implementation LL/SC implementation was broken, this replaces it with a locking implementation. Changes the API to support LL/SC for future implementation on ARM. * Improve TLS teardown. * Make std::function fully inlined. * Factor out PALLinux stack trace. * Add checks for leaking allocators. * Add release build of Windows Clang
2020-04-07 17:37:26 +03:00
endif()
CI for both arms (#151) * Add another ARM
2020-03-20 12:17:38 +03:00
Make binaries more compatible by default (#206) * Make binaries more compatible by default Turn `-march=native` off by default. This makes binaries more portable, but may harm performance. However, fast paths look unaltered * Change setting to on if specified.
2020-05-28 18:56:48 +03:00
if(SNMALLOC_OPTIMISE_FOR_CURRENT_MACHINE)
CI for both arms (#151) * Add another ARM
2020-03-20 12:17:38 +03:00
check_cxx_compiler_flag(-march=native SUPPORT_MARCH_NATIVE)
if (SUPPORT_MARCH_NATIVE)
add_compile_options(-march=native)
Make binaries more compatible by default (#206) * Make binaries more compatible by default Turn `-march=native` off by default. This makes binaries more portable, but may harm performance. However, fast paths look unaltered * Change setting to on if specified.
2020-05-28 18:56:48 +03:00
else()
message(WARNING "Compiler does not support `-march=native` required by SNMALLOC_OPTIMISE_FOR_CURRENT_MACHINE")
CI for both arms (#151) * Add another ARM
2020-03-20 12:17:38 +03:00
endif()
portability: platform compiler flags -mcx16 and -march=native are platform specific. Leave them out if we aren't targeting chips that need it.
2019-04-23 15:42:10 +03:00
endif()
Use cmake to find backtrace() (#187) On FreeBSD, this notably requires the use of -lexecinfo, as backtrace() is not available in -lc. Rather than testing in C, test in cmake.
2020-05-16 14:45:51 +03:00
find_package(Backtrace)
if(${Backtrace_FOUND})
target_compile_definitions(snmalloc_lib INTERFACE -DBACKTRACE_HEADER="${Backtrace_HEADER}")
target_link_libraries(snmalloc_lib INTERFACE ${Backtrace_LIBRARIES})
backtrace cmake vars build fix, got undetected if the system package include was already in the flags.
2021-05-06 10:47:01 +03:00
target_include_directories(snmalloc_lib INTERFACE ${Backtrace_INCLUDE_DIRS})
Use cmake to find backtrace() (#187) On FreeBSD, this notably requires the use of -lexecinfo, as backtrace() is not available in -lc. Rather than testing in C, test in cmake.
2020-05-16 14:45:51 +03:00
endif()
Address review comments.
2019-02-19 00:29:32 +03:00
endif()
Splitting EXPOSE_EXTERNAL_ALLOCATOR_API to EXPOSE_EXTERNAL_PAGEMAP and EXPOSE_EXTERNAL_RESERVE. Only the shim can be affected by passing the new pre-processor definitions to cmake.
2019-04-12 14:32:37 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
macro(subdirlist result curdir)
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
file(GLOB children CONFIGURE_DEPENDS LIST_DIRECTORIES true RELATIVE ${curdir} ${curdir}/* )
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
set(dirlist "")
foreach(child ${children})
if(IS_DIRECTORY ${curdir}/${child})
list(APPEND dirlist ${child})
endif()
endforeach()
set(${result} ${dirlist})
endmacro()
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
if(CMAKE_VERSION VERSION_LESS 3.14)
set(CMAKE_REQUIRED_LIBRARIES -fuse-ld=lld)
else()
set(CMAKE_REQUIRED_LINK_OPTIONS -fuse-ld=lld)
endif()
check_cxx_source_compiles("int main() { return 1; }" LLD_WORKS)
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
macro(add_shim name type)
add_library(${name} ${type} ${ARGN})
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
target_link_libraries(${name} snmalloc_lib)
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
if(NOT MSVC)
target_compile_definitions(${name} PRIVATE "SNMALLOC_EXPORT=__attribute__((visibility(\"default\")))")
endif()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
set_target_properties(${name} PROPERTIES CXX_VISIBILITY_PRESET hidden)
Make internal symbols hidden. Only the malloc/free/... interface is exported. This allows all of C++'s weak ODR symbols to be resolved at compile-time, rather than staying weak and resolved at load-time. Since internal calls don't need to go through the PLT anymore, we get a nice speedup, at least on small objects.
2019-02-12 01:16:47 +03:00
Set linker-language C; prevents linking c++ stdlib Thanks to Alex Richardson for the suggestion
2019-11-14 18:40:05 +03:00
# Ensure that we do not link against C++ stdlib when compiling shims.
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
if(NOT MSVC)
set_target_properties(${name} PROPERTIES LINKER_LANGUAGE C)
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
if (LLD_WORKS)
message("Using LLD.")
# Only include the dependencies that actually are touched.
# Required so C++ library is not included with direct pthread access.
target_link_libraries(${name} -Wl,--as-needed)
# Remove all the duplicate new/malloc and free/delete definitions
target_link_libraries(${name} -Wl,--icf=all -fuse-ld=lld)
endif()
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
endif()
Set linker-language C; prevents linking c++ stdlib Thanks to Alex Richardson for the suggestion
2019-11-14 18:40:05 +03:00
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
endmacro()
add static library (#189) * add static library * leave static library on by default * fix 1mib layout * code reviews
2020-05-19 08:46:40 +03:00
if (SNMALLOC_STATIC_LIBRARY)
add_shim(snmallocshim-static STATIC src/override/malloc.cc)
target_compile_definitions(snmallocshim-static PRIVATE
SNMALLOC_STATIC_LIBRARY_PREFIX=${SNMALLOC_STATIC_LIBRARY_PREFIX})
endif ()
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
if(NOT WIN32)
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
set(SHARED_FILES src/override/new.cc)
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
add_shim(snmallocshim SHARED ${SHARED_FILES})
Protect free list pointers stored in object space Free list pointers can be exploited by attackers. This commit implements a simple encoding scheme to detect corruption of the pointers. This can be used to detect UAF and double free. This does not currently address anything for Medium or Large allocations. It also does not address cross thread deallocations. Co-authored-by: Nathaniel Wesley Filardo <nfilardo@microsoft.com>
2021-03-13 23:46:49 +03:00
add_shim(snmallocshim-checks SHARED ${SHARED_FILES})
target_compile_definitions(snmallocshim-checks PRIVATE CHECK_CLIENT)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
endif()
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
if(SNMALLOC_RUST_SUPPORT)
add_shim(snmallocshim-rust STATIC src/override/rust.cc)
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
add_shim(snmallocshim-checks-rust STATIC src/override/rust.cc)
target_compile_definitions(snmallocshim-checks-rust PRIVATE CHECK_CLIENT)
add rust support (#113) * add rust support * move aligned_size to sizeclass.h * add static qualifier * adjust CMakeLists.txt, may broke CI tests * fix msvc's complaining on c++17 * use SNMALLOC_FAST_PATH as the decorator of aligned_size * adapt new alignment algorithm and add related test Co-authored-by: mjp41 <mattpark@microsoft.com> * fix test cases for msvc * add extra test for size == 0 * treat memory block of same sizeclass as the same * fix formatting problem * remove extra declarations Co-authored-by: Matthew Parkinson <mjp41@users.noreply.github.com>
2020-01-23 10:08:18 +03:00
endif()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
enable_testing()
set(TESTDIR ${CMAKE_CURRENT_SOURCE_DIR}/src/test)
subdirlist(TEST_CATEGORIES ${TESTDIR})
list(REVERSE TEST_CATEGORIES)
foreach(TEST_CATEGORY ${TEST_CATEGORIES})
subdirlist(TESTS ${TESTDIR}/${TEST_CATEGORY})
foreach(TEST ${TESTS})
Natural alignment for USE_MALLOC (#248) * Add concept of natural alignment to tests. snmalloc naturally aligns blocks very heavily, so that the largest power-of-two in the rounded size is the alignment. This checks that in the test, and provides a method for finding the natural alignment of a block. * Improve USE_MALLOC to provide alignment snmalloc provides a lot of alginment guarantees. This ensures that when we pass through to the system allocator we still get those alignment guarantees. The commit also fixes the tests to work with USE_MALLOC, and builds a set of unit tests for ctest to check behaviour.
2020-09-28 12:08:19 +03:00
if (WIN32
OR (CMAKE_SYSTEM_NAME STREQUAL NetBSD)
DragonFly support (userland). - Close to OpenBSD as there is no malloc*size api nor arbritrary alignment support. - Like FreeBSD, MAP_NORESERVE never had been implemented even tough still present in the header but not mentioned in the man page, FreeBSD has reserved the value for another later usage seems DragonFly has just out of sync header.
2020-10-02 20:15:28 +03:00
OR (CMAKE_SYSTEM_NAME STREQUAL OpenBSD)
Solaris based system build fix. malloc_usable_size or similar unsupported.
2020-11-18 20:13:21 +03:00
OR (CMAKE_SYSTEM_NAME STREQUAL DragonFly)
OR (CMAKE_SYSTEM_NAME STREQUAL SunOS))
Natural alignment for USE_MALLOC (#248) * Add concept of natural alignment to tests. snmalloc naturally aligns blocks very heavily, so that the largest power-of-two in the rounded size is the alignment. This checks that in the test, and provides a method for finding the natural alignment of a block. * Improve USE_MALLOC to provide alignment snmalloc provides a lot of alginment guarantees. This ensures that when we pass through to the system allocator we still get those alignment guarantees. The commit also fixes the tests to work with USE_MALLOC, and builds a set of unit tests for ctest to check behaviour.
2020-09-28 12:08:19 +03:00
# Windows does not support aligned allocation well enough
# for pass through.
DragonFly support (userland). - Close to OpenBSD as there is no malloc*size api nor arbritrary alignment support. - Like FreeBSD, MAP_NORESERVE never had been implemented even tough still present in the header but not mentioned in the man page, FreeBSD has reserved the value for another later usage seems DragonFly has just out of sync header.
2020-10-02 20:15:28 +03:00
# NetBSD, OpenBSD and DragonFlyBSD do not support malloc*size calls.
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
set(FLAVOURS fast;check)
Natural alignment for USE_MALLOC (#248) * Add concept of natural alignment to tests. snmalloc naturally aligns blocks very heavily, so that the largest power-of-two in the rounded size is the alignment. This checks that in the test, and provides a method for finding the natural alignment of a block. * Improve USE_MALLOC to provide alignment snmalloc provides a lot of alginment guarantees. This ensures that when we pass through to the system allocator we still get those alignment guarantees. The commit also fixes the tests to work with USE_MALLOC, and builds a set of unit tests for ctest to check behaviour.
2020-09-28 12:08:19 +03:00
else()
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
set(FLAVOURS fast;check) #malloc - TODO-need to add pass through back
Natural alignment for USE_MALLOC (#248) * Add concept of natural alignment to tests. snmalloc naturally aligns blocks very heavily, so that the largest power-of-two in the rounded size is the alignment. This checks that in the test, and provides a method for finding the natural alignment of a block. * Improve USE_MALLOC to provide alignment snmalloc provides a lot of alginment guarantees. This ensures that when we pass through to the system allocator we still get those alignment guarantees. The commit also fixes the tests to work with USE_MALLOC, and builds a set of unit tests for ctest to check behaviour.
2020-09-28 12:08:19 +03:00
endif()
Randomise slab allocation pattern (#304) The slab allocation pattern is randomised based on the deallocation pattern. This achieved by using two queues to enqueue free elements onto. We pick "randomly", which queue to add to, and then when we take the free_queue to use, we splice the two queues together.
2021-03-24 19:12:22 +03:00
foreach(FLAVOUR ${FLAVOURS})
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
unset(SRC)
aux_source_directory(${TESTDIR}/${TEST_CATEGORY}/${TEST} SRC)
Randomise slab allocation pattern (#304) The slab allocation pattern is randomised based on the deallocation pattern. This achieved by using two queues to enqueue free elements onto. We pick "randomly", which queue to add to, and then when we take the free_queue to use, we splice the two queues together.
2021-03-24 19:12:22 +03:00
set(TESTNAME "${TEST_CATEGORY}-${TEST}-${FLAVOUR}")
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
Remove new from examples as they don't use it.
2019-08-13 12:56:54 +03:00
add_executable(${TESTNAME} ${SRC})
Change behaviour of USE_POSIX_COMMIT_CHECKS The previous setting applied USE_POSIX_COMMIT_CHECKS to snmalloc if it was a non-release build. This caused issues in CCF virtual mode, as it was being built in RelWithDebInfo. This commit changes the flag to be applied less, but for tests to always apply the setting independent of build type. This means that when snmalloc is being used as a library, it will be off, unless explicitly requested.
2021-02-19 12:07:42 +03:00
# For all tests enable commit checking.
target_compile_definitions(${TESTNAME} PRIVATE -DUSE_POSIX_COMMIT_CHECKS)
Randomise slab allocation pattern (#304) The slab allocation pattern is randomised based on the deallocation pattern. This achieved by using two queues to enqueue free elements onto. We pick "randomly", which queue to add to, and then when we take the free_queue to use, we splice the two queues together.
2021-03-24 19:12:22 +03:00
if (${FLAVOUR} STREQUAL "malloc")
Natural alignment for USE_MALLOC (#248) * Add concept of natural alignment to tests. snmalloc naturally aligns blocks very heavily, so that the largest power-of-two in the rounded size is the alignment. This checks that in the test, and provides a method for finding the natural alignment of a block. * Improve USE_MALLOC to provide alignment snmalloc provides a lot of alginment guarantees. This ensures that when we pass through to the system allocator we still get those alignment guarantees. The commit also fixes the tests to work with USE_MALLOC, and builds a set of unit tests for ctest to check behaviour.
2020-09-28 12:08:19 +03:00
target_compile_definitions(${TESTNAME} PRIVATE SNMALLOC_PASS_THROUGH)
endif()
Randomise slab allocation pattern (#304) The slab allocation pattern is randomised based on the deallocation pattern. This achieved by using two queues to enqueue free elements onto. We pick "randomly", which queue to add to, and then when we take the free_queue to use, we splice the two queues together.
2021-03-24 19:12:22 +03:00
if (${FLAVOUR} STREQUAL "check")
target_compile_definitions(${TESTNAME} PRIVATE CHECK_CLIENT)
endif()
Reverting separate compilation of new.cc.
2019-06-12 17:04:25 +03:00
target_link_libraries(${TESTNAME} snmalloc_lib)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
if (${TEST} MATCHES "release-.*")
message(STATUS "Adding test: ${TESTNAME} only for release configs")
add_test(NAME ${TESTNAME} COMMAND ${TESTNAME} CONFIGURATIONS "Release")
else()
message(STATUS "Adding test: ${TESTNAME}")
add_test(${TESTNAME} ${TESTNAME})
endif()
if (${TEST_CATEGORY} MATCHES "perf")
Reduce parallism in tests If some tests on Windows are co-scheduled, then they run out of commit space and crash. For example, in func-memory, test_external_pointer_large can cause the small CI machines to run out of commit space on Windows.
2019-08-14 16:21:35 +03:00
message(STATUS "Single threaded test: ${TESTNAME}")
Bump the perf tests up to counting as 4 tests. They're failing on macOS and 32-bit Windows CI, but not locally on 32-bit Windows, so probably running out of memory...
2019-07-05 15:03:58 +03:00
set_tests_properties(${TESTNAME} PROPERTIES PROCESSORS 4)
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
endif()
Make snmalloc build on Windows with Clang Fixes a few places where Clang complains about Windows specific code, and also uses macros supported by Clang on Windows. A few places separating platform and compiler specific code, as MSVC and WIN32 were used interchangably previously.
2020-01-24 16:17:45 +03:00
if(WIN32)
Reduce parallism in tests If some tests on Windows are co-scheduled, then they run out of commit space and crash. For example, in func-memory, test_external_pointer_large can cause the small CI machines to run out of commit space on Windows.
2019-08-14 16:21:35 +03:00
# On Windows these tests use a lot of memory as it doesn't support
# lazy commit.
if (${TEST} MATCHES "two_alloc_types")
message(STATUS "Single threaded test: ${TESTNAME}")
set_tests_properties(${TESTNAME} PROPERTIES PROCESSORS 4)
endif()
if (${TEST} MATCHES "fixed_region")
message(STATUS "Single threaded test: ${TESTNAME}")
set_tests_properties(${TESTNAME} PROPERTIES PROCESSORS 4)
endif()
if (${TEST} MATCHES "memory")
message(STATUS "Single threaded test: ${TESTNAME}")
set_tests_properties(${TESTNAME} PROPERTIES PROCESSORS 4)
endif()
endif()
Major refactor of snmalloc (#343) # Pagemap The Pagemap now stores all the meta-data for the object allocation. The meta-data in the pagemap is effectively a triple of the sizeclass, the remote allocator, and a pointer to a 64 byte block of meta-data for this chunk of memory. By storing the pointer to a block, it allows the pagemap to handle multiple slab sizes without branching on the fast path. There is one entry in the pagemap per 16KiB of address space, but by using the same entry in the pagemap for 4 adjacent entries, then we can treat a 64KiB range can be treated as a single slab of allocations. This change also means there is almost no capability amplification required by the implementation on CHERI for finding meta-data. The only amplification is required, when we change the way a chunk is used to a size of object allocation. # Backend There is a second major aspect of the refactor that there is now a narrow API that abstracts the Pagemap, PAL and address space management. This should better enable the compartmentalisation and makes it easier to produce alternative backends for various research directions. This is a template parameter that can be used to specialised by the front-end in different ways. # Thread local state The thread local state has been refactored into two components, one (called 'localalloc') that is stored directly in the TLS and is constant initialised, and one that is allocated in the address space (called 'coreallloc') which is lazily created and pooled. # Difference This removes Superslabs/Medium slabs as there meta-data is now part of the pagemap.
2021-07-12 17:53:36 +03:00
# if (${TEST_CATEGORY} MATCHES "func")
# target_compile_definitions(${TESTNAME} PRIVATE -DUSE_SNMALLOC_STATS)
# endif ()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
endforeach()
Build both 1MiB and 16MiB tests
2019-01-21 20:53:52 +03:00
endforeach()
Initial commit of snmalloc History squashed from internal development. Internal history has commit hash: e27a0e485c44a5003a802de2661ce3b21e120316
2019-01-15 17:17:55 +03:00
endforeach()
CMake Header-Only Target (#46) Make header only library target. Use the CMake INTERFACE target type to include the build settings for snmalloc in other projects using headers only. Made warnings setting a macro for reuse.
2019-05-09 14:32:32 +03:00
Make clangformat a macro to allow sharing.
2019-05-22 15:20:43 +03:00
clangformat_targets()
use clangformat cmake target
2019-05-18 13:54:35 +03:00
endif()