Skip to content
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
7782f92
chore: Add GitHub Actions workflow for PR validation
Artur- Sep 22, 2025
8fd17d6
Update validation.yml
Artur- Sep 22, 2025
aad8744
Merge branch 'main' into github-pr-validation
Artur- Sep 22, 2025
5ce460e
fix: Update Sauce Connect to v5.x and configure PRO_KEY environment v…
Artur- Sep 22, 2025
2fe6888
fix: Correct Sauce Connect 5.x download URL format
Artur- Sep 22, 2025
4eea994
refactor: Merge format-check workflow into validation workflow
Artur- Sep 22, 2025
f064420
feat: Add Maven cache to format-check job
Artur- Sep 22, 2025
944296c
fix: Run format-check and build-and-test jobs sequentially
Artur- Sep 22, 2025
3647990
chore: set TB License
ZheSun88 Sep 23, 2025
d615238
Merge branch 'main' into github-pr-validation
mcollovati Sep 24, 2025
281467a
Merge branch 'main' into github-pr-validation
mshabarov Sep 26, 2025
968565b
fix: Correct Sauce Connect binary path for version 5.x
Artur- Sep 28, 2025
884c1dc
fix: Improve Sauce Connect setup and monitoring
Artur- Sep 28, 2025
243275b
Fix version
Artur- Sep 28, 2025
351054d
refactor: Use official sauce-connect-action v3 instead of manual setup
Artur- Sep 28, 2025
b7ceb44
use existing version, see https://github.com/saucelabs/sauce-connect-…
Artur- Sep 28, 2025
bd9f42d
Secrets
Artur- Sep 28, 2025
25ba241
.
Artur- Sep 28, 2025
d179e57
region
Artur- Sep 28, 2025
67c2c54
fix: Separate error screenshots into their own artifact
Artur- Sep 29, 2025
98085a5
fix: Enable localhost proxying for Sauce Connect tunnel
Artur- Sep 29, 2025
13c1d1f
fix: Use correct proxyLocalhost parameter for Sauce Connect
Artur- Sep 29, 2025
0af2f9e
no phantomjs
Artur- Sep 29, 2025
1232207
fix: Exclude failsafe-summary.xml from test result parsing
Artur- Sep 29, 2025
3208e7e
upgrade java version to 21 and remove master branch
ZheSun88 Sep 30, 2025
03dd64b
Merge branch 'main' into github-pr-validation
ZheSun88 Sep 30, 2025
f60ac1a
Merge branch 'main' into github-pr-validation
Artur- Sep 30, 2025
110bc21
Merge branch 'main' into github-pr-validation
ZheSun88 Oct 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions .github/workflows/format-check.yml

This file was deleted.

154 changes: 154 additions & 0 deletions .github/workflows/validation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
name: TestBench Validation

on:
push:
branches:
- main
- master
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:

env:
JAVA_VERSION: 17

jobs:
format-check:
name: Format Check
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up JDK ${{ env.JAVA_VERSION }}
uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: "temurin"

- name: Cache Maven dependencies
uses: actions/cache@v4
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-

- name: Run formatter validation
run: mvn formatter:validate --batch-mode --no-transfer-progress

build-and-test:
name: Build and Test
needs: format-check
runs-on: ubuntu-latest
timeout-minutes: 45

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up JDK ${{ env.JAVA_VERSION }}
uses: actions/setup-java@v4
with:
java-version: ${{ env.JAVA_VERSION }}
distribution: "temurin"

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "20"

- name: Cache Maven dependencies
uses: actions/cache@v4
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-

- name: Clean Maven cache (if needed)
if: github.event_name == 'workflow_dispatch'
run: |
echo "---> Clean maven cache"
rm -rf ~/.m2/repository ~/.npm* ~/.pnpm*

- name: Set up Sauce Labs tunnel
uses: saucelabs/[email protected]
with:
username: ${{ secrets.SAUCE_USERNAME }}
accessKey: ${{ secrets.SAUCE_ACCESS_KEY }}
tunnelName: ${{ github.run_id }}-${{ github.run_number }}
region: us-west-1
retryTimeout: 300
proxyLocalhost: allow

- name: Set TB License
run: |
TB_LICENSE=${{secrets.TB_LICENSE}}
mkdir -p ~/.vaadin/
echo '{"username":"'`echo $TB_LICENSE | cut -d / -f1`'","proKey":"'`echo $TB_LICENSE | cut -d / -f2`'"}' > ~/.vaadin/proKey

- name: Build with Maven
run: |
mvn clean install -DskipTests -B

- name: Run Tests and Generate Javadoc
env:
SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
SAUCE_ACCESS_KEY: ${{ secrets.SAUCE_ACCESS_KEY }}
SAUCE_TUNNEL_ID: ${{ github.run_id }}-${{ github.run_number }}
run: |
mvn javadoc:javadoc verify \
-P validation \
-Dtestbench.javadocs \
-Dsystem.com.vaadin.testbench.Parameters.testsInParallel=5 \
-Dsystem.com.vaadin.testbench.Parameters.maxAttempts=2 \
-Dcom.vaadin.testbench.Parameters.hubHostname=localhost \
-Dsauce.tunnelId=${SAUCE_TUNNEL_ID} \
-Dfailsafe.forkCount=5 \
-Dsystem.sauce.user=${SAUCE_USERNAME} \
-Dsystem.sauce.sauceAccessKey=${SAUCE_ACCESS_KEY} \
-B

- name: Upload error screenshots
if: failure()
uses: actions/upload-artifact@v4
with:
name: error-screenshots-${{ github.run_id }}
path: |
**/error-screenshots/**
retention-days: 7

- name: Upload test reports on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ github.run_id }}
path: |
**/target/surefire-reports/
**/target/failsafe-reports/
retention-days: 7

- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ github.run_id }}
path: |
**/target/surefire-reports/TEST-*.xml
**/target/failsafe-reports/TEST-*.xml
retention-days: 7

- name: Publish test results
if: always()
uses: EnricoMi/publish-unit-test-result-action@v2
with:
files: |
**/target/surefire-reports/TEST-*.xml
**/target/failsafe-reports/TEST-*.xml
check_name: Test Results
comment_mode: failures