C++ Safe Buffers¶
Introduction¶
Clang can be used to harden your C++ code against buffer overflows, an otherwise common security issue with C-based languages.
The solution described in this document is an integrated programming model as it combines:
a family of opt-in Clang warnings (
-Wunsafe-buffer-usage) emitted at during compilation to help you update your code to encapsulate and propagate the bounds information associated with pointers;runtime assertions implemented as part of (libc++ hardening modes) that eliminate undefined behavior as long as the coding convention is followed and the bounds information is therefore available and correct.
The goal of this work is to enable development of bounds-safe C++ code. It is not a “push-button” solution; depending on your codebase’s existing coding style, significant (even if largely mechanical) changes to your code may be necessary. However, it allows you to achieve valuable safety guarantees on security-critical parts of your codebase.
This solution is under active development. It is already useful for its purpose but more work is being done to improve ergonomics and safety guarantees and reduce adoption costs.
The solution aligns in spirit with the “Ranges” safety profile that was proposed by Bjarne Stroustrup for standardization alongside other C++ safety features.
Pre-Requisites¶
In order to achieve bounds safety, your codebase needs to have access to
well-encapsulated bounds-safe container, view, and iterator types.
If your project uses libc++, standard container and view types such as
std::vector and std::span can be made bounds-safe by enabling
the “fast” hardening mode
(passing -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST) to your
compiler) or any of the stricter hardening modes.
In order to harden iterators, you’ll need to also obtain a libc++ binary
built with _LIBCPP_ABI_BOUNDED_ITERATORS – which is a libc++ ABI setting
that needs to be set for your entire target platform if you need to maintain
binary compatibility with the rest of the platform.
A relatively fresh version of C++ is recommended. In particular, the very useful
standard view class std::span requires C++20.
Other implementations of the C++ standard library may provide different flags to enable such hardening hardening.
If you’re using custom containers and views, they will need to be hardened this way as well, but you don’t necessarily need to do this ahead of time.
This approach can theoretically be applied to plain C codebases, assuming that safe primitives are developed to encapsulate all buffer accesses, acting as “hardened custom containers” to replace raw pointers. However, such approach would be very unergonomic in C, and safety guarantees will be lower due to lack of good encapsulation technology. A better approach to bounds safety for non-C++ programs, -fbounds-safety, is currently in development.
Technically, safety guarantees cannot be provided without hardening the entire technology stack, including all of your dependencies. However, applying such hardening technology to even a small portion of your code may be significantly better than nothing.
The Programming Model for C++¶
Assuming that hardened container, view, and iterator classes are available,
what remains is to make sure they are used consistently in your code.
Below we define the specific coding convention that needs to be followed
in order to guarantee safety and how the compiler technology
around -Wunsafe-buffer-usage assists with that.
Buffer operations should never be performed over raw pointers¶
Every time a memory access is made, a bounds-safe program must guarantee that the range of accessed memory addresses falls into the boundaries of the memory allocated for the object that’s being accessed. In order to establish such a guarantee, the information about such valid range of addresses – the bounds information associated with the accessed address – must be formally available every time a memory access is performed.
A raw pointer does not naturally carry any bounds information. The bounds information for the pointer may be available somewhere, but it is not associated with the pointer in a formal manner, so a memory access performed through a raw pointer cannot be automatically verified to be bounds-safe by the compiler.
That said, the Safe Buffers programming model does not try to eliminate all pointer usage. Instead it assumes that most pointers point to individual objects, not buffers, and therefore they typically aren’t associated with buffer overflow risks. For that reason, in order to identify the code that requires manual intervention, it is desirable to initially shift the focus away from the pointers themselves, and instead focus on their usage patterns.
The compiler warning -Wunsafe-buffer-usage is built to assist you
with this step of the process. A -Wunsafe-buffer-usage warning is
emitted whenever one of the following buffer operations are performed
on a raw pointer:
array indexing with
[],pointer arithmetic,
bounds-unsafe standard C functions such as
std::memcpy(),C++ smart pointer operations such as
std::unique_ptr<T[N]>::operator[](), which unfortunately cannot be made fully safe within the rules of the C++ standard (as of C++23).
This is sufficient for identifying each raw buffer pointer in the program at at least one point during its lifetime across your software stack.
For example, both of the following functions are flagged by
-Wunsafe-buffer-usage because pointer gets identified as an unsafe
buffer pointer. Even though the second function does not directly access
the buffer, the pointer arithmetic operation inside it may easily be
the only formal “hint” in the program that the pointer does indeed point
to a buffer of multiple objects:
int get_last_element(int *pointer, size_t size) {
return ptr[sz - 1]; // warning: unsafe buffer access
}
int *get_last_element_ptr(int *pointer, size_t size) {
return ptr + (size - 1); // warning: unsafe pointer arithmetic
}
All buffers need to be encapsulated into safe container and view types¶
It immediately follows from the previous requirement that once an unsafe pointer is identified at any point during its lifetime, it should be immediately wrapped into a safe container type (if the allocation site is “nearby”) or a safe view type (if the allocation site is “far away”). Not only memory accesses, but also non-access operations such as pointer arithmetic need to be covered this way in order to benefit from the respective runtime bounds checks.
If a container type (std::array, std::vector, std::string)
is used for allocating the buffer, this is the best-case scenario because
the container naturally has access to the correct bounds information for the
buffer, and the runtime bounds checks immediately kick in. Additionally,
the container type may provide automatic lifetime management for the buffer
(which may or may not be desirable).
If a view type is used (std::span, std::string_view), this typically
means that the bounds information for the “adopted” pointer needs to be passed
to the view’s constructor manually. This makes runtime checks immediately
kick in with respect to the provided bounds information, which is an immediate
improvement over the raw pointer. However, this situation is still fundamentally
insufficient for security purposes, because bounds information provided
this way cannot be guaranteed to be correct.
For example, the function get_last_element() we’ve seen in the previous
section can be made slightly safer this way:
int get_last_element(int *pointer, size_t size) {
std::span<int> sp(pointer, size);
return sp[size - 1]; // warning addressed
}
Here std::span eliminates the potential concern that the operation
size - 1 may overflow when sz is equal to 0, leading to a buffer
“underrun”. However, such program does not provide a guarantee that
the variable sz correctly represents the actual size fo the buffer
pointed to by ptr. The std::span constructed this way may be ill-formed.
It may fail to protect you from overrunning the original buffer.
The following example demonstrates one of the most dangerous anti-patterns of this nature:
void convert_data(int *source_buf, size_t source_size,
int *target_buf, size_t target_size) {
// Terrible: mismatched pointer / size.
std::span<int> target_span(target_buf, source_size);
// ...
}
The second parameter of std::span should never be the desired size
of the buffer. It should always be the actual size of the buffer.
Such code often indicates that the original code has already contained
a vulnerability – and the use of a safe view class failed to prevent it.
If target_span actually needs to be of size source_size, a significantly
safer way to produce such a span would be to build it with the correct size
first, and then resize it to the desired size by calling .first():
void convert_data(int *source_buf, size_t source_size,
int *target_buf, size_t target_size) {
// Safer.
std::span<int> target_span(target_buf, target_size).first(source_size);
// ...
}
However, these are still half-measures. This code still accepts the bounds information from the caller in an informal manner, and such bounds information cannot be guaranteed to be correct.
In order to mitigate problems of this nature in their entirety, the third guideline is imposed.
Encapsulation of bounds information must be respected continuously¶
The allocation site of the object is the only reliable source of bounds information for that object. For objects with long lifespans across multiple functions or even libraries in the software stack, it is essential to formally preserve the original bounds information as it’s being passed from one piece of code to another.
Standard container and view classes are designed to preserve bounds information correctly by construction. However, they offer a number of ways to “break” encapsulation, which may cause you to temporarily lose track of the correct bounds information:
The two-parameter constructor
std::span(ptr, size)allows you to assemble an ill-formedstd::span;Conversely, you can unwrap a container or a view object into a raw pointer and a raw size by calling its
.data()and.size()methods.The overloaded
operator&()found on container and iterator classes acts similarly to.data()in this regard; operations such as&span[0]and&*span.begin()are effectively unsafe.
Additional -Wunsafe-buffer-usage warnings are emitted when encapsulation
of standard containers is broken in this manner. If you’re using
non-standard containers, you can achieve a similar effect with facilities
described in the next section: Backwards Compatibility, Interoperation with Unsafe Code, Customization.
For example, our previous attempt to address the warning in
get_last_element() has actually introduced a new warning along the way,
that notifies you about the potentially incorrect bounds information
passed into the two-parameter constructor of std::span:
int get_last_element(int *pointer, size_t size) {
std::span<int> sp(pointer, size); // warning: unsafe constructor
return sp[size - 1];
}
In order to address this warning, you need to make the function receive
the bounds information from the allocation site in a formal manner.
The function doesn’t necessarily need to know where the allocation site is;
it simply needs to be able to accept bounds information when it’s available.
You can achieve this by refactoring the function to accept a std::span
as a parameter:
int get_last_element(std::span<int> sp) {
return sp[size - 1];
}
This solution puts the responsibility for making sure the span is well-formed on the caller. They should do the same, so that eventually the responsibility is placed on the allocation site!
Such definition is also very ergonomic as it naturally accepts arbitrary standard containers without any additional code at the call site:
void use_last_element() {
std::vector<int> vec { 1, 2, 3 };
int x = get_last_element(vec); // x = 3
}
Such code is naturally bounds-safe because bounds-information is passed down from the allocation site to the buffer access site. Only safe operations are performed on container types. The containers are never “unforged” into raw pointer-size pairs and never “reforged” again. This is what ideal bounds-safe C++ code looks like.
Backwards Compatibility, Interoperation with Unsafe Code, Customization¶
Some of the code changes described above can be somewhat intrusive.
For example, changing a function that previously accepted a pointer and a size
separately, to accept a std::span instead, may require you to update
every call site of the function. This is often undesirable and sometimes
completely unacceptable when backwards compatibility is required.
In order to facilitate incremental adoption of the coding convention
described above, as well as to handle various unusual situations, the compiler
provides two additional facilities to give the user more control over
-Wunsafe-buffer-usage diagnostics:
#pragma clang unsafe_buffer_usageto mark code as unsafe and suppress-Wunsafe-buffer-usagewarnings in that code.[[clang::unsafe_buffer_usage]]to annotate potential sources of discontinuity of bounds information – thus introducing additional-Wunsafe-buffer-usagewarnings.
In this section we describe these facilities in detail and show how they can help you with various unusual situations.
Suppress unwanted warnings with #pragma clang unsafe_buffer_usage¶
If you really need to write unsafe code, you can always suppress all
-Wunsafe-buffer-usage warnings in a section of code by surrounding
that code with the unsafe_buffer_usage pragma. For example, if you don’t
want to address the warning in our example function get_last_element(),
here is how you can suppress it:
int get_last_element(int *pointer, size_t size) {
#pragma clang unsafe_buffer_usage begin
return ptr[sz - 1]; // warning suppressed
#pragma clang unsafe_buffer_usage end
}
This behavior is analogous to #pragma clang diagnostic (documentation)
However, #pragma clang unsafe_buffer_usage is specialized and recommended
over #pragma clang diagnostic for a number of technical and non-technical
reasons. Most importantly, #pragma clang unsafe_buffer_usage is more
suitable for security audits because it is significantly simpler and
describes unsafe code in a more formal manner. On the contrary,
#pragma clang diagnostic comes with a push/pop syntax (as opposed to
the begin/end syntax) and it offers ways to suppress warnings without
mentioning them by name (such as -Weverything), which can make it
difficult to determine at a glance whether the warning is suppressed
on any given line of code.
There are a few natural reasons to use this pragma:
In implementations of safe custom containers. You need this because ultimately
-Wunsafe-buffer-usagecannot help you verify that your custom container is safe. It will naturally remind you to audit your container’s implementation to make sure it has all the necessary runtime checks, but ultimately you’ll need to suppress it once the audit is complete.In performance-critical code where bounds-safety-related runtime checks cause an unacceptable performance regression. The compiler can theoretically optimize them away (eg. replace a repeated bounds check in a loop with a single check before the loop) but it is not guaranteed to do that.
For incremental adoption purposes. If you want to adopt the coding convention gradually, you can always surround an entire file with the
unsafe_buffer_usagepragma and then “make holes” in it whenever you address warnings on specific portions of the code.In the code that interoperates with unsafe code. This may be code that will never follow the programming model (such as plain C code that will never be converted to C++) or with the code that simply haven’t been converted yet.
Interoperation with unsafe code may require a lot of suppressions. You are encouraged to introduce “unsafe wrapper functions” for various unsafe operations that you need to perform regularly.
For example, if you regularly receive pointer/size pairs from unsafe code, you may want to introduce a wrapper function for the unsafe span constructor:
#pragma clang unsafe_buffer_usage begin
template <typename T>
std::span<T> unsafe_forge_span(T *pointer, size_t size) {
return std::span(pointer, size);
}
#pragma clang unsafe_buffer_usage end
Such wrapper function can be used to suppress warnings about unsafe span constructor usage in a more ergonomic manner:
void use_unsafe_c_struct(unsafe_c_struct *s) {
// No warning here.
std::span<int> sp = unsafe_forge_span(s->pointer, s->size);
// ...
}
The code remains unsafe but it also continues to be nicely readable, and it
proves that -Wunsafe-buffer-usage has done it best to notify you about
the potential unsafety. A security auditor will need to keep an eye on such
unsafe wrappers. It is still up to you to confirm that the bounds information
passed into the wrapper is correct.
Flag bounds information discontinuities with [[clang::unsafe_buffer_usage]]¶
The clang attribute [[clang::unsafe_buffer_usage]]
(attribute documentation)
allows the user to annotate various objects, such as functions or member
variables, as incompatible with the Safe Buffers programming model.
You are encouraged to do that for arbitrary reasons, but typically the main
reason to do that is when an unsafe function needs to be provided for
backwards compatibility.
For example, in the previous section we’ve seen how the example function
get_last_element() needed to have its parameter types changed in order
to preserve the continuity of bounds information when receiving a buffer pointer
from the caller. However, such a change breaks both API and ABI compatibility.
The code that previously used this function will no longer compile, nor link,
until every call site of that function is updated. You can reclaim the
backwards compatibility – in terms of both API and ABI – by adding
a “compatibility overload”:
int get_last_element(std::span<int> sp) {
return sp[size - 1];
}
[[clang::unsafe_buffer_usage]] // Please use the new function.
int get_last_element(int *pointer, size_t size) {
// Avoid code duplication - simply invoke the safe function!
// The pragma suppresses the unsafe constructor warning.
#pragma clang unsafe_buffer_usage begin
return get_last_element(std::span(pointer, size));
#pragma clang unsafe_buffer_usage end
}
Such an overload allows the surrounding code to continue to work.
It is both source-compatible and binary-compatible. It is also strictly safer
than the original function because the unsafe buffer access through raw pointer
is replaced with a safe std::span access no matter how it’s called. However,
because it requires the caller to pass the pointer and the size separately,
it violates our “bounds information continuity” principle. This means that
the callers who care about bounds safety needs to be encouraged to use the
std::span-based overload instead. Luckily, the attribute
[[clang::unsafe_buffer_usage]] causes a -Wunsafe-buffer-usage warning
to be displayed at every call site of the compatibility overload in order to
remind the callers to update their code:
void use_last_element() {
std::vector<int> vec { 1, 2, 3 };
// no warning
int x = get_last_element(vec);
// warning: this overload introduces unsafe buffer manipulation
int x = get_last_element(vec.data(), vec.size());
}
The compatibility overload can be further simplified with the help of the
unsafe_forge_span() wrapper as described in the previous section –
and it even makes the pragmas unnecessary:
[[clang::unsafe_buffer_usage]] // Please use the new function.
int get_last_element(int *pointer, size_t size) {
// Avoid code duplication - simply invoke the safe function!
return get_last_element(unsafe_forge_span(pointer, size));
}
Notice how the attribute [[clang::unsafe_buffer_usage]] does not
suppress the warnings within the function on its own. Similarly, functions whose
entire definitions are covered by #pragma clang unsafe_buffer_usage do
not become automatically annotated with the attribute
[[clang::unsafe_buffer_usage]]. They serve two different purposes:
The pragma says that the function isn’t safely written;
The attribute says that the function isn’t safe to use.
Also notice how we’ve made an unsafe wrapper for a safe function. This is significantly better than making a safe wrapper for an unsafe function. In other words, the following solution is significantly more unsafe and undesirable than the previous solution:
int get_last_element(std::span<int> sp) {
// You've just added that attribute, and now you need to
// immediately suppress the warning that comes with it?
#pragma clang unsafe_buffer_usage begin
return get_last_element(sp.data(), sp.size());
#pragma clang unsafe_buffer_usage end
}
[[clang::unsafe_buffer_usage]]
int get_last_element(int *pointer, size_t size) {
// This access is still completely unchecked. What's the point of having
// perfect bounds information if you aren't performing runtime checks?
#pragma clang unsafe_buffer_usage begin
return ptr[sz - 1];
#pragma clang unsafe_buffer_usage end
}
Structs and classes, unlike functions, cannot be overloaded. If a struct
contains an unsafe buffer (in the form of a nested array or a pointer/size pair)
then it is typically impossible to replace them with a safe container (such as
std::array or std::span respectively) without breaking the layout
of the struct and introducing both source and binary incompatibilities with
the surrounding client code.
Additionally, member variables of a class cannot be naturally “hidden” from
client code. If a class needs to be used by clients who haven’t updated to
C++20 yet, you cannot use the C++20-specific std::span as a member variable
type. If the definition of a struct is shared with plain C code that manipulates
member variables directly, you cannot use any C++-specific types for these
member variables.
In such cases there’s usually no backwards-compatible way to use safe types
directly. The best option is usually to discourage the clients from using
member variables directly by annotating the member variables with the attribute
[[clang::unsafe_buffer_usage]], and then to change the interface
of the class to provide safe “accessors” to the unsafe data.
For example, let’s assume the worst-case scenario: struct foo is an unsafe
struct type fully defined in a header shared between plain C code and C++ code:
struct foo {
int *pointer;
size_t size;
};
In this case you can achieve safety in C++ code by annotating the member variables as unsafe and encapsulating them into safe accessor methods:
struct foo {
[[clang::unsafe_buffer_usage]]
int *pointer;
[[clang::unsafe_buffer_usage]]
size_t size;
// Avoid showing this code to clients who are unable to digest it.
#if __cplusplus >= 202002L
std::span<int> get_pointer_as_span() {
#pragma clang unsafe_buffer_usage begin
return std::span(pointer, size);
#pragma clang unsafe_buffer_usage end
}
void set_pointer_from_span(std::span<int> sp) {
#pragma clang unsafe_buffer_usage begin
pointer = sp.data();
size = sp.size();
#pragma clang unsafe_buffer_usage end
}
// Potentially more utility functions.
#endif
};
Future Work¶
The -Wunsafe-buffer-usage technology is in active development. The warning
is largely ready for everyday use but it is continuously improved to reduce
unnecessary noise as well as cover some of the trickier unsafe operations.
Fix-It Hints for -Wunsafe-buffer-usage¶
A code transformation tool is in development that can semi-automatically
transform large bodies of code to follow the C++ Safe Buffers programming model.
It can currently be accessed by passing the experimental flag
-fsafe-buffer-usage-suggestions in addition to -Wunsafe-buffer-usage.
Fixits produced this way currently assume the default approach described
in this document as they suggest standard containers and views (most notably
std::span and std::array) as replacements for raw buffer pointers.
This also additionally requires libc++ hardening in order to make the runtime
bounds checks actually happen.
Static Analysis to Identify Suspicious Sources of Bounds Information¶
The unsafe constructor span(pointer, size) is often a necessary evil
when it comes to interoperation with unsafe code. However, passing the
correct bounds information to such constructor is often difficult.
In order to detect those span(target_pointer, source_size) anti-patterns,
path-sensitive analysis performed by the clang static analyzer can be taught to identify situations
when the pointer and the size are coming from “suspiciously different” sources.
Such analysis will be able to identify the source of information with
significantly higher precision than that of the compiler, making it much better
at identifying incorrect bounds information in your code while producing
significantly fewer warnings. It will also need to bypass
#pragma clang unsafe_buffer_usage suppressions and “see through”
unsafe wrappers such as unsafe_forge_span – something that
the static analyzer is naturally capable of doing.