8000 fix(compiler): replace createInBoundsGEP1 with createGEP1 to prevent UB by blackdragoon26 · Pull Request #4563 · WasmEdge/WasmEdge · GitHub
[go: up one dir, main page]

Skip to content

Conversation

@blackdragoon26
Copy link
Contributor

Summary:
Replaces 'createInBoundsGEP1' with 'createGEP1' for memory access operations in the AOT compiler. This fixes a raw Segmentation Fault issue where the WasmEdge signal handler failed to trap out-of-bounds accesses.

Root Cause:
WasmEdge's zero-cost bounds checking relies on pointers legally hitting the OS Guard Page to trigger a SIGSEGV.
The LLVM 'inbounds' keyword contract states that pointers must stay within allocated objects. When a pointer hits the Guard Page, this contract is violated, resulting in Undefined Behavior (Poison). This UB allowed LLVM to optimize away the necessary trap mechanics.

Fix:
Using standard 'createGEP1' (without 'inbounds') forces the compiler to generate the exact pointer arithmetic required to hit the Guard Page and trigger the trap deterministically.

Verification:

  • Checked LLVM IR: Verified 'inbounds' keyword is removed from generated 'getelementptr' instructions.
  • Runtime: Confirmed raw Segfault is replaced by graceful '[error] execution failed: out of bounds memory access'.

Fixes #3063

@github-project-automation github-project-automation bot moved this to Triage-required in WasmEdge Roadmap Jan 27, 2026
@github-actions github-actions bot added c-Test An issue/PR to enhance the test suite and removed c-Test An issue/PR to enhance the test suite labels Jan 27, 2026
@blackdragoon26
Copy link
Contributor Author

before

root@fa71c617d832:/app/build# ninja
./tools/wasmedge/wasmedgec --generic-binary --enable-tail-call ../filea29176_1.wasm filea29176_1.so
[6/6] Linking CXX executable tools/wasmedge/wasmedgec
[2026-01-27 12:17:30.800] [info] compile start
  %35 = getelementptr inbounds i8, i8* %34, i64 7431  %13 = getelementptr inbounds i8, i8* %12, i64 3505  %22 = getelementptr inbounds i8, i8* %21, i64 5068  %126 = getelementptr inbounds i8, i8* %125, i64 9339  %13 = getelementptr inbounds i8, i8* %12, i64 5864  %152 = getelementptr inbounds i8, i8* %151, i64 8695  %260 = getelementptr inbounds i8, i8* %259, i64 2503  %67 = getelementptr inbounds i8, i8* %66, i64 %61  %106 = getelementptr inbounds i8, i8* %105, i64 11400  %80 = getelementptr inbounds i8, i8* %79, i64 2552  %11 = getelementptr inbounds i8, i8* %10, i64 8416  %27 = getelementptr inbounds i8, i8* %26, i64 4495  %45 = getelementptr inbounds i8, i8* %44, i64 %39  %96 = getelementptr inbounds i8, i8* %95, i64 %90  %14 = getelementptr inbounds i8, i8* %13, i64 7688  %23 = getelementptr inbounds i8, i8* %22, i64 3997  %88 = getelementptr inbounds i8, i8* %87, i64 %82  %218 = getelementptr inbounds i8, i8* %217, i64 3274  %49 = getelementptr inbounds i8, i8* %48, i64 4511  %91 = getelementptr inbounds i8, i8* %90, i64 %85  %108 = getelementptr inbounds i8, i8* %107, i64 13102  %38 = getelementptr inbounds i8, i8* %37, i64 %32  %49 = getelementptr inbounds i8, i8* %48, i64 5263  %290 = getelementptr inbounds i8, i8* %289, i64 %284  %82 = getelementptr inbounds i8, i8* %81, i64 %76  %91 = getelementptr inbounds i8, i8* %90, i64 7348  %11 = getelementptr inbounds i8, i8* %10, i64 14554  %112 = getelementptr inbounds i8, i8* %111, i64 372  %119 = getelementptr inbounds i8, i8* %118, i64 9555  %213 = getelementptr inbounds i8, i8* %212, i64 %207  %13 = getelementptr inbounds i8, i8* %12, i64 1667  %8 = getelementptr inbounds i8, i8* %7, i64 11214  %262 = getelementptr inbounds i8, i8* %261, i64 8514  %364 = getelementptr inbounds i8, i8* %363, i64 19114  %470 = getelementptr inbounds i8, i8* %469, i64 7303  %32 = getelementptr inbounds i8, i8* %31, i64 7181  %42 = getelementptr inbounds i8, i8* %41, i64 6498  %54 = getelementptr inbounds i8, i8* %53, i64 2233  %71 = getelementptr inbounds i8, i8* %70, i64 %65  %64 = getelementptr inbounds i8, i8* %63, i64 1782  %74 = getelementptr inbounds i8, i8* %73, i64 %68  %559 = getelementptr inbounds i8, i8* %558, i64 2847  %587 = getelementptr inbounds i8, i8* %586, i64 1791  %688 = getelementptr inbounds i8, i8* %687, i64 10706  %16 = getelementptr inbounds i8, i8* %15, i64 11612  %70 = getelementptr inbounds i8, i8* %69, i64 11904  %138 = getelementptr inbounds i8, i8* %137, i64 14017  %354 = getelementptr inbounds i8, i8* %353, i64 %348[2026-01-27 12:17:30.865] [info] verify start
[2026-01-27 12:17:30.875] [info] optimize start
[2026-01-27 12:17:31.234] [info] optimize done
[2026-01-27 12:17:31.235] [info] codegen start
[2026-01-27 12:17:31.663] [info] output start
[2026-01-27 12:17:31.755] [info] codegen done
root@fa71c617d832:/app/build# ./tools/wasmedge/wasmedge --reactor filea29176_1.so main
[2026-01-27 12:48:53.041] [error] execution failed: out of bounds memory access, Code: 0x408
[2026-01-27 12:48:53.044] [error] calling stack:37, 38
[2026-01-27 12:48:53.045] [error]     When executing function name: "main"
root@fa71c617d832:/app/build# 

after

root@a0187074d3ef:/app/build_x86# ninja
./tools/wasmedge/wasmedgec --generic-binary --enable-tail-call ../filea29176_1.wasm filea29176_1.so
[6/6] Linking CXX executable tools/wasmedge/wasmedgec
[2026-01-27 12:29:24.880] [info] compile start
  %35 = getelementptr i8, i8* %34, i64 7431  %13 = getelementptr i8, i8* %12, i64 3505  %22 = getelementptr i8, i8* %21, i64 5068  %126 = getelementptr i8, i8* %125, i64 9339  %13 = getelementptr i8, i8* %12, i64 5864  %152 = getelementptr i8, i8* %151, i64 8695  %260 = getelementptr i8, i8* %259, i64 2503  %67 = getelementptr i8, i8* %66, i64 %61  %106 = getelementptr i8, i8* %105, i64 11400  %80 = getelementptr i8, i8* %79, i64 2552  %11 = getelementptr i8, i8* %10, i64 8416  %27 = getelementptr i8, i8* %26, i64 4495  %45 = getelementptr i8, i8* %44, i64 %39  %96 = getelementptr i8, i8* %95, i64 %90  %14 = getelementptr i8, i8* %13, i64 7688  %23 = getelementptr i8, i8* %22, i64 3997  %88 = getelementptr i8, i8* %87, i64 %82  %218 = getelementptr i8, i8* %217, i64 3274  %49 = getelementptr i8, i8* %48, i64 4511  %91 = getelementptr i8, i8* %90, i64 %85  %108 = getelementptr i8, i8* %107, i64 13102  %38 = getelementptr i8, i8* %37, i64 %32  %49 = getelementptr i8, i8* %48, i64 5263  %290 = getelementptr i8, i8* %289, i64 %284  %82 = getelementptr i8, i8* %81, i64 %76  %91 = getelementptr i8, i8* %90, i64 7348  %11 = getelementptr i8, i8* %10, i64 14554  %112 = getelementptr i8, i8* %111, i64 372  %119 = getelementptr i8, i8* %118, i64 9555  %213 = getelementptr i8, i8* %212, i64 %207  %13 = getelementptr i8, i8* %12, i64 1667  %8 = getelementptr i8, i8* %7, i64 11214  %262 = getelementptr i8, i8* %261, i64 8514  %364 = getelementptr i8, i8* %363, i64 19114  %470 = getelementptr i8, i8* %469, i64 7303  %32 = getelementptr i8, i8* %31, i64 7181  %42 = getelementptr i8, i8* %41, i64 6498  %54 = getelementptr i8, i8* %53, i64 2233  %71 = getelementptr i8, i8* %70, i64 %65  %64 = getelementptr i8, i8* %63, i64 1782  %74 = getelementptr i8, i8* %73, i64 %68  %559 = getelementptr i8, i8* %558, i64 2847  %587 = getelementptr i8, i8* %586, i64 1791  %688 = getelementptr i8, i8* %687, i64 10706  %16 = getelementptr i8, i8* %15, i64 11612  %70 = getelementptr i8, i8* %69, i64 11904  %138 = getelementptr i8, i8* %137, i64 14017  %354 = getelementptr i8, i8* %353, i64 %348[2026-01-27 12:29:24.946] [info] verify start
[2026-01-27 12:29:24.957] [info] optimize start
[2026-01-27 12:29:25.367] [info] optimize done
[2026-01-27 12:29:25.368] [info] codegen start
[2026-01-27 12:29:25.851] [info] output start
[2026-01-27 12:29:25.946] [info] codegen done
root@a0187074d3ef:/app/build_x86# ./tools/wasmedge/wasmedge --reactor filea29176_1.so main
[2026-01-27 12:48:30.072] [error] execution failed: out of bounds memory access, Code: 0x408
[2026-01-27 12:48:30.075] [error] calling stack:37, 38
[2026-01-27 12:48:30.075] [error]     When executing function name: "main"
root@a0187074d3ef:/app/build_x86# 

@blackdragoon26
Copy link
Contributor Author

@hydai @q82419 , kindly review this PR

Copy link
Member
@hydai hydai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code is fine. The comments are really redundant.

@github-project-automation github-project-automation bot moved this from Triage-required to In progress in WasmEdge Roadmap Jan 28, 2026
@dannypsnl
Copy link
Member

Also, no merging master to branch, use rebase instead in the future. I think you can:

  1. build a clean branch
  2. apply patches from the current branch
  3. delete the current branch locally
  4. rename clean branch to replace it
  5. force push

This gains you a single commit for your changes here.

@blackdragoon26
Copy link
Contributor Author

The code is fine. The comments are really redundant.

Thanks, will reduce it to single line comment.

@blackdragoon26
Copy link
Contributor Author

Also, no merging master to branch, use rebase instead in the future. I think you can:

  1. build a clean branch
  2. apply patches from the current branch
  3. delete the current branch locally
  4. rename clean branch to replace it
  5. force push

This gains you a single commit for your changes here.

Thank You, I really needed those tips since I did realise that unknowingly i pushed 5 commits into this PR, even though even doing pushing with no addition msg

Copy link
Member
@hydai hydai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
@ibmibmibm PTAL.

Summary:
Replaces 'createInBoundsGEP1' with 'createGEP1' for memory access operations in the AOT compiler. This fixes a raw Segmentation Fault issue where the WasmEdge signal handler failed to trap out-of-bounds accesses.

Root Cause:
WasmEdge's zero-cost bounds checking relies on pointers legally hitting the OS Guard Page to trigger a SIGSEGV.
The LLVM 'inbounds' keyword contract states that pointers must stay within allocated objects. When a pointer hits the Guard Page, this contract is violated, resulting in Undefined Behavior (Poison). This UB allowed LLVM to optimize away the necessary trap mechanics.

Fix:
Using standard 'createGEP1' (without 'inbounds') forces the compiler to generate the exact pointer arithmetic required to hit the Guard Page and trigger the trap deterministically.

Verification:
- Checked LLVM IR: Verified 'inbounds' keyword is removed from generated 'getelementptr' instructions.
- Runtime: Confirmed raw Segfault is replaced by graceful '[error] execution failed: out of bounds memory access'.

Fixes WasmEdge#3063

Signed-off-by: blackdragoon26 <sankalp.jha9643@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

bug: Different output when running a same binary

3 participants

0