Skip to content

Conversation

ksivaman
Copy link
Member

@ksivaman ksivaman commented Apr 4, 2024

A follow-up to #740

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>
@ksivaman ksivaman added the 1.5.0 label Apr 4, 2024
@ksivaman ksivaman requested a review from denera April 4, 2024 00:54
@ksivaman ksivaman self-assigned this Apr 4, 2024
@ksivaman
Copy link
Member Author

ksivaman commented Apr 4, 2024

/te-ci pytorch

@erhoo82
Copy link
Collaborator

erhoo82 commented Apr 4, 2024

I tested this with both pretraining, LoRA, and SFT.
In the case of LoRA and SFT, this is one of the input types passed as arg, which is not captured.
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/packed_seq_params.py#L7

@ksivaman
Copy link
Member Author

ksivaman commented Apr 4, 2024

/te-ci pytorch

@ksivaman ksivaman merged commit ce328fa into NVIDIA:main Apr 4, 2024
ksivaman added a commit that referenced this pull request Apr 4, 2024
* Args can be None

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

* Fix other arg types

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

---------

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>
pggPL pushed a commit to pggPL/TransformerEngine that referenced this pull request May 15, 2024
* Args can be None

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

* Fix other arg types

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

---------

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>
Signed-off-by: Pawel Gadzinski <[email protected]>
pggPL pushed a commit to pggPL/TransformerEngine that referenced this pull request May 23, 2024
* Args can be None

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

* Fix other arg types

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>

---------

Signed-off-by: Kirthi Shankar Sivamani <[email protected]>
Signed-off-by: Pawel Gadzinski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants