8000 rust: add `CacheAligned` for easy cache line alignment of values · Rust-for-Linux/linux@d78c295 · GitHub
[go: up one dir, main page]

Skip to content

Commit d78c295

Browse files
committed
rust: add CacheAligned for easy cache line alignment of values
`CacheAligned` allows to easily align values to a 64 byte boundary. An example use case is the kernel `struct spinlock`. This struct is 4 bytes on x86 when lockdep is not enabled. The structure is not padded to fit a cache line. The effect of this for `SpinLock` is that the lock variable and the value protected by the lock might share a cache line, depending on the alignment requirements of the protected value. Wrapping the value in `CacheAligned` to get a `SpinLock<CacheAligned<T>>` solves this problem. Aligning the lock variable and the protected value to a cache line yields a 20% performance increase for the Rust null block driver for sequential reads to memory backed devices at 6 concurrent readers. Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
1 parent 5f1db1a commit d78c295

File tree

2 files changed

+66
-0
lines changed

2 files changed

+66
-0
lines changed

rust/kernel/cache_aligned.rs

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
// SPDX-License-Identifier: GPL-2.0
2+
3+
use macros::pin_data;
4+
5+
use crate::{init::PinInit, pin_init, try_pin_init};
6+
7+
/// Wrapper type that alings content to a cache line.
8+
#[repr(align(64))]
9+
#[pin_data]
10+
pub struct CacheAligned<T: ?Sized> {
11+
#[pin]
12+
value: T,
13+
}
14+
15+
impl<T> CacheAligned<T> {
16+
/// Pads and aligns a value to 64 bytes.
17+
pub const fn new(t: T) -> CacheAligned<T> {
18+
CacheAligned::<T> { value: t }
19+
}
20+
21+
/// Creates an initializer for `CacheAligned<T>` form an initalizer for `T`
22+
pub fn new_initializer(
23+
t: impl PinInit<T>,
24+
) -> impl PinInit<CacheAligned<T>> {
25+
pin_init!( CacheAligned {
26+
value <- t
27+
})
28+
}
29+
30+
/// Creates a fallible initializer for `CacheAligned<T>` form a fallible
31+
/// initalizer for `T`
32+
pub fn try_new_initializer(
33+
t: impl PinInit<T, crate::error::Error>,
34+
) -> impl PinInit<CacheAligned<T>, crate::error::Error> {
35+
try_pin_init!( CacheAligned {
36+
value <- t
37+
})
38+
}
39+
40+
/// Get a pointer to the contained value without creating a reference.
41+
///
42+
/// # Safety
43+
///
44+
/// `ptr` must point to a valid and initialized instance of `Self` and it
45+
/// must be valid for read and write.
46+
pub const unsafe fn raw_get(ptr: *mut Self) -> *mut T {
47+
// SAFETY: by function safety requirements `ptr` is valid for read
48+
unsafe { core::ptr::addr_of_mut!((*ptr).value) }
49+
}
50+
}
51+
52+
impl<T: ?Sized> core::ops::Deref for CacheAligned<T> {
53+
type Target = T;
54+
55+
fn deref(&self) -> &T {
56+
&self.value
57+
}
58+
}
59+
60+
impl<T: ?Sized> core::ops::DerefMut for CacheAligned<T> {
61+
fn deref_mut(&mut self) -> &mut T {
62+
&mut self.value
63+
}
64+
}

rust/kernel/lib.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ extern crate self as kernel;
3535
#[cfg(not(testlib))]
3636
mod allocator;
3737
mod build_assert;
38+
mod cache_aligned;
3839
pub mod error;
3940
pub mod folio;
4041
pub mod hrtimer;
@@ -61,6 +62,7 @@ pub mod xarray;
6162

6263
#[doc(hidden)]
6364
pub use bindings;
65+
pub use cache_aligned::CacheAligned;
6466
pub use macros;
6567
pub use uapi;
6668

0 commit comments

Comments
 (0)
0