Add resident device change call #1517

lslusarczyk · 2025-09-11T12:30:55Z

UMF part of https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/supported/sycl_ext_oneapi_peer_access.asciidoc feature.

UR/llvm part is in intel/llvm#19257

include/umf/memory_pool_ops.h

src/libumf.map

bratpiorka · 2025-09-11T12:39:25Z

src/memory_provider.c

+        hProvider->provider_priv, deviceIndex, isAdding);
+    checkErrorAndSetLastProvider(res, hProvider);
+    return res;
+}


add empty line

src/provider/provider_level_zero.c

bratpiorka · 2025-09-11T12:48:49Z

src/provider/provider_cuda.c

    return UMF_RESULT_ERROR_INVALID_ARGUMENT;
 }

+static umf_result_t


add a "default" handler in memory_provider.c (see umfDefaultCtlHandle) and do not modify CUDA provider

Where exactly in the code you think "default" is missing?

do not modify CUDA provid

Rejected. umf_memory_provider_ops_t type received a new field and it should be somehow initialized in UMF_CUDA_MEMORY_PROVIDER_OPS as all other fields. Please explain if you still want me to remove this change.

Please initialize cuda ops .ext_resident_device_change to NULL.
Next, define the umfDefaultResidentDeviceChange in src/memory_provider.c that returns UMF_RESULT_ERROR_NOT_SUPPORTED and set this handler for each created provider if the ext_resident_device_change is set to NULL (see how this is done for other default handlers like "umfDefaultCloseIPCHandle")

ok, applied

bratpiorka · 2025-09-11T12:51:31Z

src/provider/provider_tracking.c

        p->hUpstream, memory_property_id, size);
 }

+static umf_result_t trackingResidentDeviceChange(void *provider,


Rejected. umf_memory_provider_ops_t type received a new field and it should be somehow initialized in UMF_TRACKING_MEMORY_PROVIDER_OPS as all other fields. Please explain if you still want me to remove this change.

applied in the same way as in cuda provider

bratpiorka · 2025-09-11T12:51:53Z

test/common/provider_null.c

    return UMF_RESULT_SUCCESS;
 }

+static umf_result_t nullResidentDeviceChange(void *provider,


rejected, same reason as above

bratpiorka · 2025-09-12T08:15:51Z

src/provider/provider_level_zero.c


-    if (deviceCount && !hDevices) {
-        LOG_ERR("Resident devices array is NULL, but deviceCount is not zero");
+    if (residentDevicesCount && !residentDevicesIndices) {


please also check if residentDevicesCount == 0 but residentDevicesIndices != NULL

additionally, should indices be unique?

ok, added checking for uniqueness

bratpiorka · 2025-09-12T08:18:39Z

src/provider/provider_level_zero.c

 }

+struct ze_memory_provider_resident_device_change_data {
+    bool isAdding;


bratpiorka · 2025-09-12T08:19:39Z

src/provider/provider_level_zero.c

+        change_data->source_memory_provider
+            ->device_handles[change_data->peer_device_index];
+
+    // TODO: add assertions to UMF and change it to be an assertion


use ASSERT() macros from src/utils/utils_assert.h

Postponed to some future PR. I need here a permanent ASSERT, not a one that compiles on in debug config like ASSERT does.

you assert only things which are can be an artefact of some internal umf error - if it is true debug only assert is fine. If an error can be a result of an user input you just handle the issue and return a error to the user

This case it is artifact of some internal umf error.

I believe, what Łukasz was saying, since it is artifact of some internal umf error a simple debug assert is ok - pls add assert() and remove the TODO

bratpiorka · 2025-09-12T08:21:28Z

src/provider/provider_level_zero.c

+            info->props.base, info->props.base_size);
+    } else {
+        result = ZE_RESULT_SUCCESS;
+        // TODO: currently not implemented call evict here


do you plan to add the missing code here?

Not in this PR.

bratpiorka · 2025-09-12T08:24:18Z

src/provider/provider_level_zero.c

+    if (ze_provider->resident_device_count == 0 ||
+        existing_peer_index == ze_provider->resident_device_count) {
+        // not found
+        if (!isAdding) { // impossible for UR, should be an assertion


if this should be an assertion, please use the ASSERT() macro from src/utils/utils_assert.h
Also, move the comment to the next line

To be changed in future PR when umf will have permanent assertions. Comments moved.

bratpiorka · 2025-09-12T08:24:27Z

src/provider/provider_level_zero.c

+
+static umf_result_t
+ze_memory_provider_resident_device_change(void *provider, uint32_t device_index,
+                                          bool isAdding) {


bratpiorka · 2025-09-12T08:24:46Z

src/provider/provider_level_zero.c

+        // adding case
+        if (ze_provider->device_count <=
+            ze_provider
+                ->resident_device_count) { // impossible for UR, should be an assertion


if this should be an assertion, please use the ASSERT() macro from src/utils/utils_assert.h
Also, move the comment to the next line

postponed when UMF have permanent assertions, comments moved

bratpiorka · 2025-09-12T08:24:54Z

src/provider/provider_level_zero.c

+        }
+
+        if (ze_provider->device_count <=
+            device_index) { // impossible for UR, should be an assertion


applied as above

bratpiorka · 2025-09-12T08:25:32Z

src/provider/provider_level_zero.c

+
+    } else {
+        // found
+        if (isAdding) { // impossible for UR, should be an assertion


applied as above

bratpiorka · 2025-09-12T08:27:28Z

src/provider/provider_level_zero.c

+        LOG_ERR("umfMemoryTrackerIterateAll did not manage to do some change "
+                "numFailed:%d, numSuccess:%d",
+                privData.success_changes, privData.failed_changes);
+        return UMF_RESULT_ERROR_INVALID_ARGUMENT; // probably some other result is better, best just change into assertion


you could use UMF_RESULT_ERROR_MEMORY_PROVIDER_SPECIFIC
or change to assert

src/critnib/critnib.c

src/libumf.map

lukaszstolarczuk · 2025-09-15T09:02:58Z

src/libumf.def

    umfMemoryProviderPurgeForce
    umfMemoryProviderPurgeLazy
    umfMemoryProviderPutIPCHandle
+    umfMemoryProviderResidentDeviceChange


pls move down in the file (after a comment ; Added in UMF_1.0)

first of all, wrong function was moved - umfMemoryProviderResidentDeviceChange is the new one

mea culpa, it should be already in ; Added in UMF_1.1 section

Moved down in 01d74ba

umfLevelZeroMemoryProviderResidentDeviceChange still not in the 1.1 section (it should be right after umfLevelZeroMemoryProviderParamsSetName)

src/memory_provider.c

lplewa · 2025-09-15T10:18:31Z

include/umf/memory_provider_ops.h

+    /// @brief Adds or removes devices on which allocations should be made
+    ///        resident.
+    /// @param provider handle to the memory provider
+    /// @param device_index identifier of device
+    /// @param is_adding Boolean indicating if peer is to be removed or added
+    /// @return UMF_RESULT_SUCCESS on success or appropriate error code on
+    ///         failure.
+    umf_result_t (*ext_resident_device_change)(void *provider,
+                                               uint32_t device_index,
+                                               bool is_adding);
+


This function should not be in OPS. This is something very specific to L0 provider. Resident devices are passed to L0 provider thru provider specific params, so control of them should be olso be done through provider specific API, this is why i think with should be implemented through CTL

Rejected. We discussed this in teams (4th Aug, 25) we do it as ops (Piotr, sorry in Polish): co do tego API co Ty robisz (w stylu daj mi wszystkie zaalokowane page'e) ja bym chyba sugerował zrobić API a nie robić przez CTLa, Rafał: no to faktycznie chyba dedykowane API lepiej pasuje niż CTL.

CTL should be for statistics, not a universal, hard-to-read tool to implement any API, Łukasz (me): Być może będziecie chcieli uprościć ext_ctl by nie był maszynką, którą mozna zaimplementować wszystko, a jedynie służył do statystyk. Ale to już ja się na tym nie znam - zostawiam do przemyśleń.

We talked about API to iterate over all allocations, not about API to modify some internal settings of the L0

I can see it both ways. My comment during that discussion was about a generic functionality implementable for all providers. This isn't that.
I also remember making a point about CTL being useful for provider/pool-specific functionality.

On the other hand, adding an API function is simpler.

On the other hand, adding an API function is simpler.

Thank you for the comment. Chosen simpler way.

lplewa · 2025-09-15T10:18:33Z

include/umf/providers/provider_level_zero.h

 /// @brief  Set the resident devices in the parameters struct.
 /// @param  hParams handle to the parameters of the Level Zero Memory Provider.
-/// @param  hDevices array of devices for which the memory should be made resident.
-/// @param  deviceCount number of devices for which the memory should be made resident.
+/// @param  hDevices array of all devices for which the memory can be made resident.
+/// @param  deviceCount number of devices for which the memory can be made resident.
+/// @param  residentDevicesIndices array of indices in all devices array to devices for which the memory should be made resident.
+/// @param  residentDevicesCount number of items in indices array.
 /// @return UMF_RESULT_SUCCESS on success or appropriate error code on failure.
 umf_result_t umfLevelZeroMemoryProviderParamsSetResidentDevices(
    umf_level_zero_memory_provider_params_handle_t hParams,
-    ze_device_handle_t *hDevices, uint32_t deviceCount);
+    ze_device_handle_t *hDevices, uint32_t deviceCount,
+    uint32_t *residentDevicesIndices, uint32_t residentDevicesCount);


This is an API(and ABI) break - we are post 1.0 release so you cannot do changes like this.

Shall I bump some number? Only UR uses this API and I am changing UR right now as well.

Even if UR uses only this API, old UR should work with new umf. And we will not do 2.0 release just after 1.0.

You cannot brake API. If changes are needed we have to keep old function working correctly and add new one.

We can't break backwards compatibility with SYCL / UR. Changing a major version is a significant undertaking that involves updating the components across all the different layers (UMF is nearly the lowest-most component in the stack). We've been burned on this in the past, and it's very disruptive.

In this case, my suggestion would be to simply create a new function that sets the indices.

OK, applied, I rewrote code in 53ec753 to be backward compatible

ldorau

14 / 16 files reviewed (will be continued)

pbalcer · 2025-09-16T08:54:51Z

include/umf/memory_provider_ops.h

+    ///         failure.
+    umf_result_t (*ext_resident_device_change)(void *provider,
+                                               uint32_t device_index,
+                                               bool is_adding);


I'd rather we have two functions - add/remove.

I will not apply this, please, because this will introduce too much copy-pasted code.

lslusarczyk · 2025-09-22T13:45:24Z

I still oppose against adding provider specific code to ops interface. If you don't want to do it thru CTL just implement it as umfLevelZeroMemoryProviderResidentDeviceChange() in provider_level_zero.h which do not go throught OPS. This function will act as an "level zero specific api extension".

note: remember that you will have to call umfMemoryProviderGetPriv() in this function to retrieve ze_memory_provider_t from provider handle

Great! Using umfLevelZeroMemoryProviderResidentDeviceChange simplified code even more than using ops. Applied with pleasure.

lukaszstolarczuk · 2025-09-22T14:46:32Z

test/common/ze_loopback.cpp

+
+// defined in both libze_ops from src/utils/utils_level_zero.cpp and
+// ze_ops_t operations from src/provider/provider_level_zero.c
+// zeDeviceGetProperties, zeMemFree, zeMemGetAllocProperties, zeMemFree, zeMemAllocDevice, find them in the former ones


missing EOF

lukaszstolarczuk · 2025-09-22T14:47:55Z

test/common/level_zero_mocks.h

+ *
+ */
+
+#ifndef UMF_PROVIDER_LEVEL_ZERO_MOCKS_H


UMF_PROVIDER_LEVEL_ZERO_MOCKS_H -> UMF_TEST_PROVIDER_LEVEL_ZERO_MOCKS_H

lukaszstolarczuk · 2025-09-22T14:48:26Z

test/common/level_zero_mocks.h

+    MOCK_METHOD2(zeMemFree,
+                 ze_result_t(ze_context_handle_t hContext, void *ptr));
+
+    // helper setting all expects related to successful l0 provider creation & initialization and calling its creation & initialization


pls break a long line

lukaszstolarczuk · 2025-09-22T14:48:50Z

test/common/ze_loopback.h

+// Under the Apache License v2.0 with LLVM Exceptions. See LICENSE.TXT.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+#ifndef UMF_ZE_LOOPBACK_H


UMF_ZE_LOOPBACK_H -> UMF_TEST_ZE_LOOPBACK_H

lukaszstolarczuk · 2025-09-22T14:49:26Z

test/common/level_zero_mocks.cpp

@@ -0,0 +1,96 @@
+/*
+ *
+ * Copyright (C) 2023-2025 Intel Corporation


simply 2025, as it's a new file

// apply to all new files

lukaszstolarczuk · 2025-09-22T14:49:50Z

test/pools/pool_residency.cpp

+    InitGoogleTest(&argc, argv);
+    AddGlobalTestEnvironment(new MockedLevelZeroTestEnvironment);
+    return RUN_ALL_TESTS();
+}


missing EOF

lukaszstolarczuk · 2025-09-22T14:53:37Z

src/provider/provider_level_zero.c

+        change_data->source_memory_provider
+            ->device_handles[change_data->peer_device_index];
+
+    // TODO: add assertions to UMF and change it to be an assertion


I believe, what Łukasz was saying, since it is artifact of some internal umf error a simple debug assert is ok - pls add assert() and remove the TODO

lukaszstolarczuk · 2025-09-22T15:04:09Z

src/provider/provider_level_zero.c


 static void init_ze_global_state(void) {
+
+    const char *lib_name = getenv("UMF_ZE_LOADER_LIB_NAME");


should we document this new env var? is there a use case to set it outside of testing...?

There is no use case to use it outside tests. And Łukasz P. wants to change it into CTL. Therefore I do not document it.
If I am allowed to leave it as env var I will document.

I believe, what Łukasz was saying, since it is artifact of some internal umf error a simple debug assert is ok - pls add assert() and remove the TODO

(strange I can not answer in above comment)

applied
disclaimer: I think this is worse solution, but this is not so important and I don't want to spend time arguing now. Let's discuss how assertions should be implemented later if there will be time for it.

…OINTER_TESTING_PROPS

lslusarczyk requested a review from a team as a code owner September 11, 2025 12:30

bratpiorka requested changes Sep 11, 2025

View reviewed changes

bratpiorka requested review from lplewa and ldorau September 11, 2025 12:53

bratpiorka reviewed Sep 12, 2025

View reviewed changes

lslusarczyk force-pushed the memresident branch from d53a23f to 9d7e12c Compare September 15, 2025 08:16

lukaszstolarczuk reviewed Sep 15, 2025

View reviewed changes

lplewa requested changes Sep 15, 2025

View reviewed changes

lslusarczyk mentioned this pull request Sep 15, 2025

memory resident limit to enabled peers intel/llvm#19257

Draft

ldorau reviewed Sep 15, 2025

View reviewed changes

pbalcer reviewed Sep 16, 2025

View reviewed changes

lslusarczyk added 10 commits September 19, 2025 13:56

Add resident device change call

e5d8739

added missing doxygen documentation

8c54695

fix documentation spelling

064d1ac

applied Rafal's comments

c4a7499

fixed compilation error

588d1af

applied second round of Rafal's comments

b24574c

applied Lukasz's comments

5983cfa

one more missed Lukasz comment applied

5cbbad2

added some tests for resident device feature

c27407a

add unittests and refactor to be backward compatible

53ec753

lslusarczyk added 3 commits September 22, 2025 15:20

reverted log change as it was moved to different PR

50db5c5

removed accidental changes

780794c

fix doc

4f75bca

lukaszstolarczuk reviewed Sep 22, 2025

View reviewed changes

lslusarczyk added 25 commits September 23, 2025 11:40

applied Lukasz S. comments

9f1c588

change ze_api include due to windows compile error

3f4db4f

fix link in windows

cf4254a

not using utils_log in ze_loopback

7bd5c24

fix includes for level_zero_mocks.cpp

dbb8298

windows use _putenv_s instead of setenv

58fae70

TestCreatePointer fix

53bfcb0

more windows fixes

822f278

windows next fix attempt

340bfb1

fixed sharedlib compilation

fad510c

removed pkg_check_modules ZE_LOADER

636477a

change ZE_LOADER_INCLUDE_DIRS into LEVEL_ZERO_INCLUDE_DIRS

a6e456e

fix includes for umf_test_mocks and umf_ze_loopback

877a36e

maybe UMF_UTIL_OPEN_LIBRARY_NO_LOAD was wrong?

6b0e070

ze_dll or ze_api export missing?

0498da7

dll path fix

e9ccf45

back with UMF_UTIL_OPEN_LIBRARY_NO_LOAD?

d30da26

add_library changed to add_umf_library

29a706d

fix dllexport used more than once

5713a96

see logs in case of residency tests failure

134e16b

utils_rwlock_init fix

5adb37c

some logs upon free and add closing library

058bf8d

typo fix

4803c47

fix LevelZeroMock::initializeMemoryProviderWithResidentDevices with P…

a3cc32e

…OINTER_TESTING_PROPS

remove not needed free logs

5b7f61a


		static void init_ze_global_state(void) {

		const char *lib_name = getenv("UMF_ZE_LOADER_LIB_NAME");

Add resident device change call #1517

Are you sure you want to change the base?

Add resident device change call #1517

Conversation

lslusarczyk commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lslusarczyk Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lslusarczyk Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lslusarczyk commented Sep 11, 2025 •

edited

Loading

lslusarczyk Sep 12, 2025 •

edited

Loading

lslusarczyk Sep 15, 2025 •

edited

Loading

lplewa Sep 15, 2025 •

edited

Loading