Skip to content

Conversation

zig-robotics
Copy link

This required some additional flags to make UBSan happy. For python 3.12.11 building without additional flags first led to:

$ zig build -Doptimize=ReleaseFast -Dversion=3.12.11 
install
└─ install generated/
  └─ WriteFile lib/python3.12
     └─ compile exe python ReleaseFast native
        └─ run _freeze_module.py for 'Lib/site.py' (frozen_modules/site.h) failure
thread 3227794 panic: applying non-zero offset to non-null pointer 0x360a4f20 produced null pointer
/home/user/.cache/zig/p/N-V-__8AAPXFFgaXQejAFSUHUzijh3iPdy1ps-UZiLaGwczp/Parser/tokenizer.c:376:37: 0x1506f5b in restore_fstring_buffers (/home/user/.cache/zig/p/N-V-__8AAPXFFgaXQejAFSUHUzijh3iPdy1ps-UZiLaGwczp/Parser/tokenizer.c)
   mode->f_string_start = tok->buf + mode->f_string_start_offset;

Which can be fixed "properly" (as in not working around the ubsan checks) with the compiler flag -fno-delete-null-pointer-checks. I'm not entirely sold this is actually fixing things generically because after adding this I then got a null pointer overflow in the same location in tokenizer.c, but for a different python file deepfreeze.py:

install
└─ install generated/
   └─ WriteFile lib/python3.12
      └─ compile exe python ReleaseFast native
         └─ run deepfreeze.py (deepfreeze.c)
            └─ add modules to run deepfreeze.py
               └─ freeze
                  └─ run _freeze_module.py for 'Lib/_sitebuiltins.py' (frozen_modules/_sitebuiltins.h) failure
thread 3535199 panic: pointer index expression with base 0x389e69d0 overflowed to 0xfffffffffffc0e10
/home/user/.cache/zig/p/N-V-__8AAPXFFgaXQejAFSUHUzijh3iPdy1ps-UZiLaGwczp/Parser/tokenizer.c:376:37: 0x15034d3 in restore_fstring_buffers (/home/user/.cache/zig/p/N-V-__8AAPXFFgaXQejAFSUHUzijh3iPdy1ps-UZiLaGwczp/Parser/tokenizer.c)
    mode->f_string_start = tok->buf + mode->f_string_start_offset;
                                    ^

adding the flag to disable null pointer overflow checks in ubsan can work around this -fno-sanitize=pointer-overflow. Needing to disable this check seems like a red flag to me but I did not investigate the root cause as I assume diagnosing python issues is out of scope for this repo.

Additionally since I was adding specific ubsan flags I re enabled ubsan for both python versions and disable only the alignment checks for python version 3.11.13. This allows other ubsan checks to continue to work. The specific ubsan flags ideally should be only applied to the offending c file. This seemed like it would upset the overall structure of the build so I add them as common flags.

For both versions I was able to run the produced python executable and run basic commands on linux. I haven't tried cross compiling or operation on other platforms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant