Windows Longpath support does not fully work? Error: "fatal: '$GIT_DIR' too big"

[skdsp]

• [ x] I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.31.1.windows.1
cpu: x86_64
built from commit: c5f0be26a7e3846e3b6268d1c6c4800d838c6bbb
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

 - Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.18363.1679]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

** insert your machine's response here **
Editor Option: Notepad++
Custom Editor Path: 
Default Branch Option:  
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: ConHost
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled


 - Any other interesting things about your environment that might be related
   to the issue you're seeing?

Longpaths activated (C:\Program Files\Git\etc\gitconfig): 	
[core] 
  longpaths = true 

### Details

 - Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

git bash (MINGW64)

 - What commands did you run to trigger this issue? If you can provide a
   [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve)
   this will help us understand the issue.

** insert your commands here **

1. Cloning a Git Repo into a folder with an already long path
   git clone -c core.longpaths=true https://github.com/git/git.git --recurse-submodules //?/D:/eval/Git_Test/looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
2. Cloning a Git Repo with multiple levels of submodules (nested submodules). Here we are really running into issues with levels of e.g. 6 nested submodules, when trying to clone into already very short pathes. Shortening paths within the submodules is only solving temporally the issue. I'd like to have something that is really future proof.
   git clone -c core.longpaths=true --recurse-submodules //?/

• What did you expect to occur after running these commands?

All Repos with submodules are fetched recursively.

• What actually happened instead?

Error message: fatal: '$GIT_DIR' too big.
The error seems to point to a define (PATH_MAX) in setup.c code (https://github.com/git/git/blob/54e85e7af1ac9e9a92888060d6811ae767fea1bc/setup.c#L812)
Thus I am afraid the PATH_MAX is set to 260 for the installation already?

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

e.g. https://github.com/git/git.git

[dscho]

Cloning a Git Repo into a folder with an already long path

That sounds like a catch-22: now, when Git wants to read the config, it has to open a long path. But it does not yet know whether it is supposed to support long paths. So it doesn't (which is the default). And therefore it can never read the config that tells it that it should turn on the long path support.

[skdsp]

Hmm, I think this is a general issue with paths > 260. Or am I doing something wrong?

In the first example, the path is 230 chars long. Regardless of the passed parameters to the clone command, it is not possible to clone the repo:

1. git clone -c core.longpaths=true https://github.com/git/git.git --recurse-submodules D:/eval/Git_Test/looooooooooooooo
   oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong
   Cloning into 'D:/eval/Git_Test/looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong'...
   fatal: '$GIT_DIR' too big
2. git clone https://github.com/git/git.git --recurse-submodules D:/eval/Git_Test/loooooooooooooooooooooooooooooooooooooo
   oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
   ooooooooooooooooooooooooooooooooooooooooooooooooooong
   Cloning into 'D:/eval/Git_Test/looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong'...
   fatal: '$GIT_DIR' too big

Hint: The longpath setting is also set in my system git core settings.

I think the define PATH_MAX is set for windows by limit.h to 260 and thus the longpath feature is prohibited
(https://github.com/git/git/blob/54e85e7af1ac9e9a92888060d6811ae767fea1bc/git-compat-util.h#L320-L325).

As mentioned, the first example is something to show the general issue, the second example (nested submodules) is the real issue at our side.

[skdsp]

Okay, maybe now I got your point. I have removed some of the chars in the path of the first example and the clone is fine.

I thought by setting the system git core settings, the default behavior for longpaths should have changed? How can I change the behavior otherwise?

Anyhow, I still don't get how this is connected to the mentioned error message and the define? Is the function "setup_explicit_git_dir" not called when longpath is activated?

[skdsp]

Maybe my issue is a duplication of: #2075
Yet it seems there was no solution?

[dscho]

The solution is probably a bit more involved than a casual contributor is prepared for.

Currently, the way things are handled look like this: 5eb8d3a

The strategy chosen is to have a platform-specific mingw_core_config() function that interprets the value of core.longPaths, and then to make sure that each and every command reads the config.

Before the config is read, the core.longPaths setting is not respected, and that includes the time when we actually try to read the config (or for that matter, to detect whether we are in a Git worktree or repository).

The way things should be done is via a function similar to git_config_get_bool().

I say similar to because we cannot use it. You see, if you call git_config_get_bool() when trying to determine whether to use long path support or not, you might be in the call that tries to determine whether we're inside a Git worktree or not, and if called at that stage, it will read the config without heeding any worktree config (and cache the results!).

So what we would need is something akin to git_config_get_bool(), but a variant that emulates the read_very_early_config() function in that it does read the system and global config, and does not cache anything.

However, calling such a function would be wasteful because Git already calls read_very_early_config() as part of tr2_sysenv_load(), which is called from trace2_initialize_fl(), which in turn is called from main().

So: the most appropriate solution to this would be to:

• Refactor the part of trace2_initialize() that reads the config,
• Probably by making tr2_sysenv_cb() a public function,
• No longer calling it from trace2_initialize(), but from a static callback function in common-main.c,
• Calling read_very_early_config() explicitly in main(), with that static callback function that calls into tr2_sysenv_cb().
• Then extend that static callback function to allow for platform-specific callback functions,
• Implement such a thing for Windows, and parse core.longPaths in that function.

One particular complication with this entire plan is that read_very_early_config() ignores the command-line config passed via git -c <key>=<value> ....

As you probably suspect by now, this requires a lot more care and effort than you might originally have thought.

Having said that, if you are interested, I can help you implement it.

[skdsp]

Thank you for your reply.
I am sorry to say that I won't have the time for implementing this.

[dscho]

Closing as stale.