build_rag.sh: install mistral-common for convert_hf_to_gguf.py #1925
Conversation
Updating the version of llama.cpp also updated convert_hf_to_gguf.py, which introduced a new dependency on mistral-common. Update the minimum version of sentencepiece to 0.2.1 to fix compilation with gcc 15. Signed-off-by: Mike Bonnet <[email protected]>
mikebonnet
requested review from
rhatdan,
bmahabirbu,
maxamillion,
swarajpande5,
jhjaggars,
cgruver and
engelmi
as code owners
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThe build_rag.sh script’s to_gguf function was updated to remove the temporary GCC 15 workaround, bump sentencepiece to ≥0.2.1, and add mistral-common as a new dependency, with pip install arguments reformatted for readability. Flow diagram for updated dependency installation in to_gguf()flowchart TD
A["Start to_gguf()"] --> B["Install numpy~=1.26.4"]
B --> C["Install sentencepiece~=0.2.1"]
C --> D["Install transformers>=4.45.1,<5.0.0"]
D --> E["Install gguf-py from llama.cpp repo"]
E --> F["Install protobuf>=5.27.2,<6.0.0"]
F --> G["Install mistral-common>=1.8.3"]
G --> H["End to_gguf()"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @mikebonnet, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request updates the build script for RAG components to ensure compatibility with recent changes in llama.cpp. It addresses new dependency requirements and resolves a compilation issue related to sentencepiece, streamlining the build process.
Highlights
- New Dependency for convert_hf_to_gguf.py: The
mistral-commonpackage (version1.8.3or higher) is now installed as a new dependency. This is required by the updatedconvert_hf_to_gguf.pyscript fromllama.cpp. - sentencepiece Version Update: The minimum version of
sentencepiecehas been updated from0.2.0to0.2.1. This change addresses a compilation issue with GCC 15. - Removed GCC 15 Workaround: The
export CXXFLAGS="-include cstdint"line, which was a temporary workaround for GCC 15 compilation issues withsentencepiece, has been removed as it's no longer needed after thesentencepieceversion update.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `container-images/scripts/build_rag.sh:67` </location>
<code_context>
- ${python} -m pip install "numpy~=1.26.4" "sentencepiece~=0.2.0" "transformers>=4.45.1,<5.0.0" git+https://github.com/ggml-org/llama.cpp#subdirectory=gguf-py "protobuf>=5.27.2,<6.0.0"
+ ${python} -m pip install "numpy~=1.26.4" "sentencepiece~=0.2.1" "transformers>=4.45.1,<5.0.0" \
+ "git+https://github.com/ggml-org/llama.cpp#subdirectory=gguf-py" \
+ "protobuf>=5.27.2,<6.0.0" "mistral-common>=1.8.3"
}
</code_context>
<issue_to_address>
Consider pinning mistral-common to a specific version for reproducibility.
Pinning the version or specifying a tighter range will help prevent future incompatibilities and ensure consistent builds.
</issue_to_address>
<suggested_fix>
<<<<<<< SEARCH
${python} -m pip install "numpy~=1.26.4" "sentencepiece~=0.2.1" "transformers>=4.45.1,<5.0.0" \
"git+https://github.com/ggml-org/llama.cpp#subdirectory=gguf-py" \
"protobuf>=5.27.2,<6.0.0" "mistral-common>=1.8.3"
=======
${python} -m pip install "numpy~=1.26.4" "sentencepiece~=0.2.1" "transformers>=4.45.1,<5.0.0" \
"git+https://github.com/ggml-org/llama.cpp#subdirectory=gguf-py" \
"protobuf>=5.27.2,<6.0.0" "mistral-common==1.8.3"
>>>>>>> REPLACE
</suggested_fix>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Code Review
This pull request updates Python dependencies for the GGUF conversion process. It correctly bumps sentencepiece to version 0.2.1 to resolve a compilation issue with GCC 15, removing a previous workaround. It also adds mistral-common as a new dependency required by an updated llama.cpp. My main feedback is to pin the llama.cpp dependency to a specific commit hash to ensure build reproducibility and stability, as installing directly from the main branch can lead to unexpected issues. This is a general best practice for managing dependencies from version control systems.
|
LGTM |
Updating the version of
llama.cppalso updatedconvert_hf_to_gguf.py, which introduced a new dependency onmistral-common.Update the minimum version of
sentencepieceto0.2.1to fix compilation with gcc 15.Summary by Sourcery
Update build_rag.sh to install the new mistral-common dependency for convert_hf_to_gguf.py and to bump sentencepiece to 0.2.1, removing the GCC 15 workaround.
Enhancements: