Add `include` and `exclude` information to metadata

[GuillaumeGomez]

Problem

I'm currently working on rust-lang/rust-clippy#11677. However, to finish this lint, I need to have the include and exclude information in the metadata (to be used with cargo_metadata).

Proposed Solution

I think that to prevent having to recompute include and exclude when using it from metadata and duplicate what cargo does, it should contain the equivalent of cargo package --list. An initial implementation was done in

Notes

No response

[epage]

FYI the title and problem are still focused on one specific solution, rather than the problem at hand.

Going to copy oer from #13330 @weihanglo's comment which has some good context when considering solutions:

While it might be useful to have include and exclude in cargo metadata output, it doesn't seem to be a proper way to fix rust-lang/rust-clippy#11677. The solution might end up reimplement the logic Cargo find and package files, which includes .gitignore, shell glob syntax, symlinks handling, git dirty status, and other details (see #11405). cargo package --list might be more reliable, but not 100% accurate (see #8407 and #11666).

I would suggest starting the discussion from an issue, and jump back to implementation when the design is settled.

For the most part, cargo package --list should be fine for your use case since this involves the source code looking up paths the same way. For other use cases,
we'd likely need to output more than cargo package --list but the mapping cargo makes from source to destination (file moves, resolving of symlinks, whether the Cargo.lock is generated or pulled from the root, etc). We'd need to be confident enough in this new data structure to meet our compatibility guarantees.

Besides those limitations in cargo package --list, it can be slow. I currently use it in cargo release to identify which packages are affected by which commits (making me a candidate for that extended use case). I worry that if we include it within cargo metadata, we'll negatively impact all other users.

We could have clippy run cargo package --list only if that lint is enabled but then that will forever keep this as a niche lint.

[GuillaumeGomez]

I was pretty sure that I was missing on the overall complexity on cargo side hence why I implemented it with just the fields values and why I was planning to retrieve the information from clippy by emulating how cargo was computing it. ^^'

[weihanglo]

The Cargo team has discussed this today, and we favor extending cargo package --list with a experimental JSON format over this proposal. The JSON format of cargo package --list could contain richer information like whether a file is generated or where the original path is for a symlink-and-copied file. The information is also more precise than reimplementing the include/exclude rules again in tools calling cargo metadata.

If you have any idea of what information should be included, feel free to add comments in #11666. Thank you!