Skip to content

Jenkinsfile Fixes Part 2 #627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 70 commits into from
Mar 2, 2022
Merged

Jenkinsfile Fixes Part 2 #627

merged 70 commits into from
Mar 2, 2022

Conversation

ravi-mosaicml
Copy link
Contributor

@ravi-mosaicml ravi-mosaicml commented Mar 1, 2022 •
edited
Loading

  1. Trigger a conda build by updating setup.py
  2. Trigger a docker build by updating the Dockerfile
  3. Excluding the Jenkinsfile repo from the changesets
  4. Updated the build matrix to also tag the latest image.
  5. Fix a race condition with the Junit merge. Each rank now has its own junitxml file to which it writes. Modified the composer launcher to use a shell, so the dist env variables are available when specifying the junitxml path.
  6. Disable deepspeed tests. They are too flaky.

When this PR is merged, the dockerfiles will be updated on Dockerhub.

@ravi-mosaicml
Jenkins + Docker Improvements
ebf6052 
1. Set the ephemeral storage limit; switch to useing command v2 to run tests.
2. Add the build discarder config to the Jenkinsfile
3. Re-enable the conda builds. It's slow -- I know -- but it only runs whenever the dependencies (setup.py or meta.yaml) changes.
4. Not installing node globally in the dockerfile.
@ravi-mosaicml
Fixed typo
f2cb6a1
@ravi-mosaicml
Echoing job urls
6870eb0
@ravi-mosaicml
Fix typo
987ae95
@ravi-mosaicml
Testing...
3940a6f
@ravi-mosaicml
Testing
3f8ec23
@ravi-mosaicml
Added storage limit; moved the build matrix locally
3984964
@ravi-mosaicml
Fixes
cf27afc
@ravi-mosaicml
testing
8d2fb3d
@ravi-mosaicml
testing
816c08e
@ravi-mosaicml
Fixing for merge commits
2385489
@ravi-mosaicml
testing
6da83a6
@ravi-mosaicml
Merge branch 'dev' into ravi/jenkinsfile_improvements
48f4d7a
@ravi-mosaicml
testing
1373409
@ravi-mosaicml
Merge branch 'ravi/jenkinsfile_improvements' of github.com:mosaicml/c…
4f2808a 
…omposer into ravi/jenkinsfile_improvements
@ravi-mosaicml
testing
48bf896
@ravi-mosaicml
testing
1a417e5
@ravi-mosaicml
testing
057ae81
@ravi-mosaicml
testing
80d12a0
@ravi-mosaicml
Jenkinsfile cleanup
1975611
@ravi-mosaicml
Removed runWithChecks; fixed echoing of URL on subjob failures
203df72
@ravi-mosaicml
Reconfigured docker builds
e646a56
@ravi-mosaicml
testing
9fd831b
@ravi-mosaicml
Fixed typo
a083a4d
@ravi-mosaicml
Parallelize the dockerbuilds
0cd68d6
@ravi-mosaicml
Testing
737f810
@ravi-mosaicml
Fixed pytorchDockerBuildMatrix
8dae40e
@ravi-mosaicml
Bugfixes
1c6d3d3
@ravi-mosaicml
Added missing def
075e228
@ravi-mosaicml
testing
0f6d896
Base automatically changed from ravi/jenkinsfile_improvements to dev March 1, 2022 16:13
@ravi-mosaicml ravi-mosaicml changed the title Updated the description in setup.py to match the readme Jenkinsfile Fixes Part 2 Mar 1, 2022
ravi-mosaicml
ravi-mosaicml commented Mar 1, 2022
View reviewed changes
@ravi-mosaicml ravi-mosaicml mentioned this pull request Mar 2, 2022
Closed
@ravi-mosaicml ravi-mosaicml merged commit 4ce703c into dev Mar 2, 2022
@ravi-mosaicml ravi-mosaicml deleted the ravi/jenkinsfile_improvments_test branch March 2, 2022 07:30
@ravi-mosaicml ravi-mosaicml mentioned this pull request Mar 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hanlint hanlint hanlint approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ravi-mosaicml @hanlint