Proposing a useOverlay hook for automatic cleanup

[yoonminsang]

I've been enjoying using overlay-kit, and I appreciate the library's functionality!

I'd like to propose a small improvement based on my experience. The current implementation takes snapshots to create modals, which allows opening or closing them from anywhere in the application, even outside of JSX. This is a powerful feature.

However, I've noticed that in most real-world scenarios, overlays are typically associated with specific components, and ideally should be unmounted when those components are unmounted (e.g., during navigation or when using browser back buttons). Currently, this doesn't happen automatically.

To address this, I've created a simple useOverlay hook that maintains all the original functionality while adding automatic cleanup:

export const useOverlay = () => {
  const [defaultOverlayId] = useState<string>(randomId());
  const [overlayId, setOverlayId] = useState<string>(defaultOverlayId);

  const open = useCallback(
    (controller: Parameters<typeof overlay.open>[0], options?: Parameters<typeof overlay.open>[1]) => {
      const overlayId = options?.overlayId ? options.overlayId : defaultOverlayId;
      setOverlayId(overlayId);
      return overlay.open(controller, { ...options, overlayId });
    },
    [defaultOverlayId]
  );
  const close = useCallback(() => overlay.close(overlayId), [overlayId]);
  const unmount = useCallback(() => overlay.unmount(overlayId), [overlayId]);
  const openAsync = useCallback(
    <T>(
      controller: OverlayAsyncControllerComponent<T>,
      options?: { overlayId?: string | undefined } | undefined
    ): Promise<T> => {
      const overlayId = options?.overlayId ? options.overlayId : defaultOverlayId;
      setOverlayId(overlayId);
      return overlay.openAsync<T>(controller, { ...options, overlayId });
    },
    [defaultOverlayId]
  );

  useEffect(() => {
    return () => {
      unmount();
    };
  }, [unmount]);

  return { open, close, unmount, openAsync };
};

This hook unmounts the overlay when the component using it is unmounted, while maintaining the same API.

I've been using this hook in my own projects and found it very useful. I'm sharing it here in case other users might benefit from it as well. Would this be something you'd consider adding to the library, perhaps as an optional utility?
What are your thoughts on this approach?

[jgjgill]

Oh thanks for sharing your great thoughts. 🙇‍♂️
I was thinking that using that hook would be hard to handle the following requirement in reverse, but what do you think? 🤔

• Optimize performance by leaving the state in memory via close on page A.
• Go to page B, from page B back to page A
• Open open an ovelay left by close on page A

Maybe defining overlay.unmount(overlayId), overlay.close(overlayId) to suit your needs every time you use them was a bad idea?

e.g.

overlay.open(
  ({ isOpen, close, unmount }) => {
    return <ConfirmDialog isOpen={isOpen} close={close} onExit={unmount} />;
  },
  { overlayId: 'temp' }
);

...

useEffect(() => {
  return () => {
    overlay.unmount('temp'); // or overlay.close('temp')
  };
}, []);

[yoonminsang]

You are absolutely right, but I added this hook for a slightly different reason.
It's quite annoying to run useEffect every time and unmount in cleanup.
And I think most cases don't need optimization using close.
If you need memory optimization, you can write it like you did.

[jgjgill]

Oh, I understand. 🙇‍♂️
Thanks for the quick response. 😊

[jungpaeng]

Hello, thank you for sharing the useOverlay hook – I had a chance to take a look, and I completely agree that it’s a very convenient feature, especially since it removes the need to manually handle unmounting when triggering an overlay.

That said, I do have one concern. I see overlay-kit as a relatively low-level utility library that emphasizes flexibility. With that in mind, I think we need to carefully consider whether including a higher-level abstraction like useOverlay directly in the core of overlay-kit is appropriate.

If we decide not to include useOverlay directly in overlay-kit, I think we could consider the following alternatives:

1. Document the usage pattern of the hook so users can easily adopt it themselves if needed
2. Create a separate third-party library based on overlay-kit that provides this hook

The first approach is similar to the strategy used by shadcn/ui, while the second one is reminiscent of how libraries like jotai offer extended functionality through separate packages. Both approaches are fairly familiar and make sense to me.

Additionally, we’re currently working on a third-party library based on overlay-kit, which will include utilities like devtools and accessibility helpers to support overlay development. So the second option feels especially viable in our case.

For now, I’d suggest introducing the hook pattern in the documentation, and then bundling useOverlay into the upcoming third-party library when it’s ready. What are your thoughts on this approach?

[yoonminsang]

You suggested two options, and I think they are both good choices. Your opinion made me think a little bit.

Some users may want to close in cleanup without unmounting for optimization purposes, so they need code like this:

export const useOverlay = ({ overlayId }: { overlayId: string }) => {
  const open = useCallback(
    (controller: Parameters<typeof overlay.open>[0]) => {
      return overlay.open(controller, { overlayId });
    },
    [overlayId],
  );
  const close = useCallback(() => overlay.close(overlayId), [overlayId]);
  const unmount = useCallback(() => overlay.unmount(overlayId), [overlayId]);
  const openAsync = useCallback(
    <T,>(controller: OverlayAsyncControllerComponent<T>): Promise<T> => {
      return overlay.openAsync<T>(controller, { overlayId });
    },
    [overlayId],
  );

  useEffect(() => {
    return () => {
      close();
    };
  }, [unmount]);

  return { open, close, unmount, openAsync };
};

Also, some users might want close animation and unmount in cleanup.

  useEffect(() => {
    return () => {
      close();
      setTimeout(() => {
        unmount();
      }, CLOSE_ANIMATION_TIME);
    };
  }, [unmount]);

It may be difficult to provide all of these diverse requirements as an extension.

I think it would be good to provide only the ones that you think are really generally necessary as extensions, and to let people know that other things can be extended in various ways through docs.

[jungpaeng]

I agree that each user may have different use cases.

I’ll do my best to provide as many examples as possible tailored to each case when writing the documentation.

Thank you for your valuable feedback.