Typo was not caught before because my checkedc tests build got corrupted and I had to rebuild clang and the checkedc tests from scratch. The tests caught the typo. (#312)
Change the snprintf bounds-safe interface.
By default, the destination of snprintf is null-terminated and therefore has bounds one-less than the second argument.
Add a second bounds-safe interface for when snprintf destination is an array_ptr only; this one is in checkedc_extensions.
Changed the signature of malloc, calloc and free to make them polymorphic bounds safe interface functions and fixed the tests that failed due to the change in signature.
By spec, memmove allows the dest and src arguments to point to overlapping areas of memory. Restrict indicates that the programmer has enforced that the arguments do not point to overlapping areas of memory. Thus, memmove should not have restrict.
* Add header guards.
* Also add guarding to make sure this is not included in C++ compilation. If it is C++, still want to make sure the unchecked version of the header is included.
Related Issue: https://github.com/Microsoft/checkedc-clang/issues/397
1. Added test cases for possible ways through which _Nt_initializers arrays can be initialized
2. Added negative test cases (e.g. array initializers without null terminators at the end) for which checkedc clang compiler is expected to report errors
* Added new extensions header file with bounds safe interfaces for functions whose interfaces differ between array_ptr and nt_array_ptr. Developers can include this checkedc_extensions header and then choose the appropriate interface for each invocation. Since C does not support function overloading, the functions in the checkedc_extensions header have different names, so the developer will have to rename their function calls.
* Added comments in the existing header files to point to the alternates in the new files. We currently have alternates for free() and strncmp().
* Added the new header to the test that includes all the headers.
Addresses #290 and #291
Checked C restricts taking the addresses of:
1. Members with member bounds declarations.
2. Members used in member bounds declarations.
3. Variables with bounds declarations.
4. Variables/variable members used in bounds declarations.
This add tests of restrictions 1-3, as part of implementing https://github.com/Microsoft/checkedc-clang/issues/213 and https://github.com/Microsoft/checkedc-clang/issues/212:
- Taking the address of non-array members with or used in bounds declarations is now an error.
- Taking the address of non-array members with or used in bounds-safe interfaces is allowed in unchecked scopes. It is an error in checked scopes.
- Taking the address of non-array variables with bounds declaration is now an error.
It is OK to take the address of an array variable or member because you can't use the resulting pointer to modify the pointer that the array converts to.
The trickier cases to test involve nested members. Given
```
struct NestedLen {
int len;
};
struct S {
struct NestedLen n;
_Array_ptr<int> p : count(n.len);
}
```
we don't allow the addresses of `n` or `n.len` to be taken. However, if `NestedLen` is not embedded in `S`, we allow the address of a struct of type NestedLen to be taken.
I plan to release an updated version of the spec because the BOUNDS_CHECKED pragma has been renamed to CHECKED_SCOPE. Address some other issues where the specification doesn't match the implementation.
- Address issue #100 (update motivation and definitions of explicit dynamic check). Add a statement that for dynamic_check(e), e must be nonmodifying.
- Address issue #269. Update specification to allow an interop type and a bounds expression as part of a bounds-safe interface.
- Change the wording to be clear that bounds-safe interface can be applied to unchecked arrays too. This addresses some feedback in issue #272.
- Remove the requirement that every parameter for a function or every member of a struct/member must be annotated with a bounds-safe interface, if one of them is annotated. The checking as implemented actually works on a per parameter/per member basis. Requiring that everything be converted could get in the way of incremental conversion.
The compiler crashed when `signal` with a bounds-safe interface was called in a checked scope. PR https://github.com/Microsoft/checkedc-clang/pull/483 fixes this. This change adds more tests of type checking of functions that return function pointers, including testing uses in and outside of checked scopes.
A programmer reported that a function call involving a const member with checked function pointer type resulted in an unexpected compiler error (https://github.com/Microsoft/checkedc-clang/issues/481). This is due to a compiler bug. This change improves the testing of function pointer casts.
- Add tests of assignments to members with function pointer types.
- Add tests of calls via members with function pointer types.
- Add tests of uses of const members with function pointer types. These
uses include member reads, member writes, and indirect calls via members.
- Add case from programmer bug report.
- Spot check uses of members with bounds-safe interfaces that are checked
function pointer types.
C provides no way to ensure that function pointer casts are correct. It allows casts between function pointers with different numbers of arguments, for example, which can result in crashes involving corrupted stack pointers.
Checked C restricts casts to checked function pointer to ensure that values are actually valid function pointers. For example, casts from a variable with an unchecked function pointer type to a checked function pointer type is not allowed. The variable with unchecked type could have been
corrupted via an incorrect function pointer cast. Checked C allows casts from declared functions, null pointers, and values with checked pointer types. It also allows casts via value-preserving casts and cast-like operations applied to these expressions. (address-of and dereference of function pointer types are value-preserving operations).
The current specification rules for inferring bounds for an expression can lead to inferred bounds that contain modifying expression. These aren't allowed. Add tests that check that the compiler is flagging them (the Checked C version of clang suggests that the programmer use a temporary variable when this happens).
The particular problematic case that is likely to arise in practice is for member access: `e1.f` and `e1->f`. If e1 contains a side-effect, we can't expression that bounds for `f` when other members are involved:
```
struct S {
array_ptr<int> p : count(len);
int len;
};
```
`e1 ->p` `has `bounds(e1->p, e1->p + e1->len)`. This is problematic if `e1` has side-effects. Note that the case where only `e1->p` is involved isn't a problem because we can use `current_expr_value`.
This adds several small programs to the Checked C samples directory that show some common tasks. It includes the canonical "hello, world" program, how to catch a runtime check failure and exit gracefully, and how to avoid a compile-time warning by using a dynamic check. It also includes an intentionally buggy program. The README.md file explains the programs.
Several of those sample programs are for code examples in the new Checked C clang users manual. The change also moves samples taken from the specification to a subdirectory.
This matches a compiler change that turns on checking of bounds declarations everywhere and teaches the compiler about simple equality facts.
Checking of bounds declarations is a static analysis that checks that declared bounds are valid (follow from existing bounds and other information in the environment). It was only on by default for checked scopes. It is now on by default for unchecked scopes too. The checking of bounds declarations is still simple, so it may produce unnecessary warnings. It can be turned off using `-Wno-check-bounds-decls` (everywhere), `-Wno-check-bounds-decl-unchecked-scope` (for unchecked scopes), and `-Wno-check-bounds-decls-checked-scope` (for checked scopes),
- Added more tests of expression equivalence (lexical_equality.c should be renamed because it now tests for expression equivalence using a few additional facts about C expressions).
- Updated tests to handle the fact that checking of bounds declaration is on everywhere by default. Replace a bunch of TODO's with expected warning messages. Also deleted some warnings that are no longer issued because the compiler can prove the bounds are valid.
- Kept checking of bounds declarations on for most tests and only disabled it for some parsing and typechecking tests.
- In some cases, made simple changes to avoid warnings about bounds declarations (the original cases will be captured in the new Checked C clang repo file test/CheckedC/static-checking/bounds-decl-challenges.c)
We recently added compiler support for declaring bounds-safe interfaces that have interop types and bounds expressions. Add tests of runtime bounds checking involving parameters declared this way. The variables are declared in unchecked scopes and used in checked scopes. Add tests for these combinations:
- checked array interop types. This implies bounds that are the count of the first dimension of the array.
- array_ptr interop types with bounds expressions.
- nt_array_ptr interop types with bounds expressions.
We already had a test of nt_array_ptr with no bounds expression, which implies bounds of count(0).
Update tests and examples to the new syntax for bounds cast expressions.
We changed bounds cast expressions to take a bounds expression as the
second argument, instead of implicitly inferring the type of bounds
expression from the number of arguments. This is more verbose but
clearer. Bounds cast expressions now have two forms:
Op<T>(e1)
Op<T>(e1, bounds-expression)
where Op is one of _Assume_bounds_cast or _Dynamic_bounds_cast
and T must be a pointer type.
A user reported an internal compiler crash (https://github.com/Microsoft/checkedc-clang/issues/458). This change adds additional testing for the compiler fix (https://github.com/Microsoft/checkedc-clang/pull/466):
- Test failing dynamic_bounds_cast operations that convert to ptr<int>.
- Test passing and failing dynamic_bounds_cast operations that
convert to ptr<void> and void *. These casts can fail if the
source range does not have at least one byte in it.
Remove inferred return bounds workarounds. Add inferred parameter bounds tests:
- Update tests now that the compiler properly infers return bounds of count(0) for nt_array_ptr return types.
- Remove workaround for strtok from checked header files.
- Add more tests of inference of bounds/itypes for parameters. This includes tests that mix parameter types that are pointers and arrays, as well as tests of inferred bounds of nt_checked arrays parameters.
Added paragraph for "Stack Bounds Protection with Low Fat Pointers" (NDSS 2017) as well as a reference to the open source implementation of the research prototype.
- Test parsing and typechecking of bounds-safe interfaces with interface types and bounds expressions.
- Test redeclarations of functions and variables with bounds-safe interfaces with interface typese and bounds expressions.
- Add tests of bounds-safe interface in checked scopes that cover the different forms of bounds expressions. This includes a test case that covers https://github.com/Microsoft/checkedc-clang/issues/443.
- Add tests of redeclarations involving `nt_array_ptr`.
- Fix bounds-safe interface for `strtok`: the compiler isn't properly inferring that return types of the form `nt_array_ptr<char>` have bounds of `count(0)`. We can just declare the bounds of `count(0)` for now.
Add runtime testing of reads and writes through pointers with null-terminated interface types. A pointer with a null-terminated type with no bounds information has a bounds of count(0). A program can read the element at the pointer and write null. Other reads/writes will cause bounds checking failures.
I extended bounds-safe-interfaces.c, which runs a sequence of read/tests, expecting conditions to be set properly for the nth test in the sequence to fail. For bounds-safe interfaces for null-terminated types, we currently have no way to specify a length. In this case, the bounds are fixed. Instead, we vary the position where the read/write occurs to force a failure.
The existing syntax of the *_bound_cast operators is confusing. The kind
of bounds being specified is not explicit. It depends on the number of
arguments to the operator. It is clearer to just use a bounds expression
to describe the bounds. That involves a little more typing, but it is
easier to understand and allows programmers to use all the different variants
of bounds expressions.
Add examples of using dynamic_bounds_cast and assume_bounds_cast for
clarity.
This change improves the formatting of code examples in the Checked C specification. We now use the LaTeX listings package for source code examples and snippets of source code. This should make the source code easier to read. We use a similar color scheme to the one used by Visual Studio for C: blue for keywords, light blue for variables, and black for symbolic characters.
The listings package can import examples from source files. This will let us move larger examples to separate source code files and make sure they compile during automated testing.
Variables with checked pointer type that may be used to access memory must be initialized when they are declared. This also applies to variables with members or elements with checked pointer type that may be used to access memory. Update the specification to make this clear. Include some examples. This addresses issue #98.
The compiler already checks these requirements for scalar variables. I opened https://github.com/Microsoft/checkedc-clang/issues/445 to track checking these requirements for struct and array variables.
Add tests for the struct/array cases to the Checked C tests.
This updates the type checking rules for implicit casts. This addresses issues
#150 and #206 n the specification. It removes the subsections for each kind
of cast between pointer kinds and presents a unified framework instead.
- Use a diagram to describe what kinds of pointers can be converted to other
kinds of pointers. It is easier to visualize this than to explain in text.
The conversions form a lattice. We can use this fact to more simply
describe conversion rules.
- Allow conversions from checked void pointers. We allow this by necessity
already in the implementation. It is needed so that malloc
can be used in checked code.
- Mention that we plan to replace casts from checked void pointers.
- Move the description of assignment compatibility earlier.
In the description of C-style casts, remove the special case for ptr<void>.
It would be confusing to expand it now that we allow conversions from
array_ptr<void too. State in text instead that the conversions between
checked void pointers are allowed.
Add a description of the type checking rules for conditional expressions.
This was only partly covered, and that description was incomplete.
It is easier to explain this more generally. Note that the compiler doesn't
quite match the spec. It doen't allow casts between ptr<T> and array_ptr<T>,
which are now allowed. I opened Checked C clang issue 444 to cover this.
Update some tests for type checking of conditional expressions to more
precisely check for arm types of the form 'T *' and "checked pointer to void"
that we're not merging them to T *. This could then be converted to checked
pointer type. The existing tests would pass and things would look OK, even
though from a typing perspective, it wouldn't be.
Describe extensions to C typing rules that allow literals to be used in checked scopes or where expressions with checked pointers are expected. This completes the work for issue #207. For checked scopes, string literals are given the type "null-terminated checked array of character", where "character" is the character type for the string literal. For contexts where an expression with a checked pointer type is expected but not found, convert string literals and compound array literals to checked pointer types.
Also describe the rules for initializers. The types of initializer values are determined by the type of the variable being declared, so there isn't any change needed for Checked C.
Add examples.
Move the section describing checked scopes earlier in Chapter 2, so that it precedes the discussion of literals (this is most of the change diff shown on GitHub).
Anna Kornfeld Simpson
Prabhuk
David Tarditi
Shen Liu
Lorenzo Cavallaro