Skip to content

Experimental support for podman. #13973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Experimental support for podman. #13973

Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
213baf1
Experimental support for podman.
oliverchang Sep 10, 2025
afd8854
formatt
oliverchang Sep 10, 2025
a5e214a
docstrings
oliverchang Sep 10, 2025
21ef97d
remove --pull=never
oliverchang Sep 15, 2025
fdadba1
fix check
oliverchang Sep 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions infra/helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@
INDEXER_PREBUILT_URL = ('https://clusterfuzz-builds.storage.googleapis.com/'
'oss-fuzz-artifacts/indexer')

CONTAINER_TOOL = os.getenv('OSS_FUZZ_CONTAINER_TOOL', 'docker')
Copy link
Contributor

@evverx evverx Sep 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it can be confusing in the sense that podman is often installed with podman-docker pointing docker to podman (https://packages.fedoraproject.org/pkgs/podman/podman-docker/) so CONTAINER_TOOL is docker even though podman is used under the hood. It would probably be safer to call CONTAINER_TOOL to figure out what it is.

$ docker --version
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
podman version 5.6.1

$ podman --version
podman version 5.6.1

The result can then be used to get around differences like #9439.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should avoid handling these differences as much as we can.

For #9439, is this still a problem? Can we just add the makedirs as a default behaviour for docker also?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For #9439, is this still a problem?

I don't know. I have been applying that patch since then. I'll double-check to see if it's still needed a bit later today.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It failed without that patch with

INFO:__main__:Running: docker run --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=centipede -e SANITIZER=address -e ARCHITECTURE=x86_64 -e PROJECT_NAME=util-linux -e HELPER=True -e FUZZING_LANGUAGE=c -v /home/vagrant/oss-fuzz/build/out/util-linux/__centipede_address:/out -v /home/vagrant/oss-fuzz/build/work/util-linux:/work -t gcr.io/oss-fuzz/util-linux.
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Error: statfs /home/vagrant/oss-fuzz/build/out/util-linux/__centipede_address: no such file or directory
ERROR:__main__:Building fuzzers failed.

so it looks like makedirs is still required.

Can we just add the makedirs as a default behaviour for docker also?

I think it should be fine (I'm not 100% sure though)

IMO we should avoid handling these differences as much as we can.

Agreed. I think it should probably be possible to avoid that to cover most use cases but given that for example #4774 was supposed to get it to work with rootless podman containers with SELinux enabled without --privileged I guess at some point it should be necessary to tell docker and podman apart.


logger = logging.getLogger(__name__)

if sys.version_info[0] >= 3:
Expand Down Expand Up @@ -222,6 +224,10 @@
else:
args.sanitizer = constants.DEFAULT_SANITIZER

if (hasattr(args, 'architecture') and
args.architecture != constants.DEFAULT_ARCHITECTURE):
raise RuntimeError('Non-default architectures require Docker.')

Check failure on line 229 in infra/helper.py

View workflow job for this annotation

GitHub Actions / build (libfuzzer, address, i386) 

Non-default architectures require Docker.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be OK to pass i386 on x86_64 with podman. In theory with qemu(-user) installed it should be able to accept aarch64 but I don't think I have ever built fuzz targets like that using helper.py (as far as I can remember it downloaded various images, launched a bunch qemu-aarch64-static processes and went sideways somewhere along the way)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! This is entirely un-tested territory, so I'd prefer not to support this for now.


if args.command == 'generate':
result = generate(args)
elif args.command == 'build_image':
Expand Down Expand Up @@ -585,7 +591,14 @@
def _check_fuzzer_exists(project, fuzzer_name, architecture='x86_64'):
"""Checks if a fuzzer exists."""
platform = 'linux/arm64' if architecture == 'aarch64' else 'linux/amd64'
command = ['docker', 'run', '--rm', '--platform', platform]
command = [
CONTAINER_TOOL,
'run',
'--rm',
'--platform',
platform,
"--pull=never",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to admit I can't remember where --pull=never came from. I'll try to decipher my comments from #4774 (comment).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I built systemd but check_build failed with

OSS_FUZZ_CONTAINER_TOOL=podman ./infra/helper.py check_build systemd
INFO:__main__:Running: podman run --pull=never --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=x86_64 -e FUZZING_LANGUAGE=c++ -e HELPER=True -v /home/vagrant/oss-fuzz/build/out/systemd:/out -t gcr.io/oss-fuzz-base/base-runner test_all.py.
Error: gcr.io/oss-fuzz-base/base-runner: image not known
ERROR:__main__:Check build failed.

As far as I can see it ran

/usr/bin/podman run --pull=never --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=x86_64 -e FUZZING_LANGUAGE=c++ -e HELPER=True -v

With my kludges (#13966 (comment)) it doesn't add --pull=never there and runs

./docker run --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=x86_64 -e FUZZING_LANGUAGE=c++ -e HELPER=True -v /home/vagrant/oss-fuzz/build/out/systemd:/out

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for testing this. I've removed the --pull=never.

is it still required at all?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has to be used to build projects integrated into OSS-Fuzz a while ago (when those gcr.io/oss-fuzz/ images were still built). For example systemd and zlib fail to build with podman without --pull=never.

I guess it should be possible to remove the images. I don't think they are needed anymore. The latest systemd image was created in 2019 to judge from https://console.cloud.google.com/artifacts/docker/oss-fuzz/us/gcr.io/systemd.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm still a bit unclear on why it was needed at all. Isn't the default behaviour to only pull if the image doesn't exist?

https://docs.podman.io/en/stable/markdown/podman-run.1.html#pull-policy

Docker's documentation says the same thing:

https://docs.docker.com/reference/cli/docker/container/run/#pull

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand the issue is helper.py ends up with that out-of-date image and tries to build systemd from 2019. It fails with

+ wget -O /out/fuzz-json_seed_corpus.zip https://storage.googleapis.com/skia-fuzzer/oss-fuzz/skjson_seed_corpus.zip
--2025-09-15 10:33:44--  https://storage.googleapis.com/skia-fuzzer/oss-fuzz/skjson_seed_corpus.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 64.233.164.207, 108.177.14.207, 74.125.131.207, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|64.233.164.207|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2025-09-15 10:33:44 ERROR 403: Forbidden.
ERROR:__main__:Building fuzzers failed.

That thing was removed in 2019 in systemd/systemd@2dd9b65 (so helper.py clearly does the wrong thing here).

Based on my comment from #4774 (comment) it looks like it's containers/podman#17063.

]
command.extend(['-v', '%s:/out' % project.out])
command.append(BASE_RUNNER_IMAGE)

Expand Down Expand Up @@ -750,10 +763,11 @@


def docker_run(run_args, print_output=True, architecture='x86_64'):
"""Calls `docker run`."""
"""Calls `CONTAINER_TOOL run`."""
platform = 'linux/arm64' if architecture == 'aarch64' else 'linux/amd64'
command = [
'docker', 'run', '--privileged', '--shm-size=2g', '--platform', platform
CONTAINER_TOOL, 'run', '--pull=never', '--privileged', '--shm-size=2g',
'--platform', platform
]
if os.getenv('OSS_FUZZ_SAVE_CONTAINERS_NAME'):
command.append('--name')
Expand Down Expand Up @@ -781,8 +795,8 @@


def docker_build(build_args):
"""Calls `docker build`."""
command = ['docker', 'build']
"""Calls `CONTAINER_TOOL build`."""
command = [CONTAINER_TOOL, 'build']
command.extend(build_args)
logger.info('Running: %s.', _get_command_string(command))

Expand All @@ -796,8 +810,8 @@


def docker_pull(image):
"""Call `docker pull`."""
command = ['docker', 'pull', image]
"""Call `CONTAINER_TOOL pull`."""
command = [CONTAINER_TOOL, 'pull', image]
logger.info('Running: %s', _get_command_string(command))

try:
Expand Down Expand Up @@ -1032,7 +1046,7 @@
]
tag = f'gcr.io/oss-fuzz/{args.project.name}'
subprocess.run([
'docker', 'tag', 'gcr.io/oss-fuzz-base/base-builder-fuzzbench',
CONTAINER_TOOL, 'tag', 'gcr.io/oss-fuzz-base/base-builder-fuzzbench',
'gcr.io/oss-fuzz-base/base-builder'
],
check=True)
Expand Down
Loading