No output in Test Results tab, and no coverage when coverage enabled

[navalex]

Environment

1. vscode-jest version: 5.2.3
2. node -v: 18.17.0
3. npm -v or yarn --version: 9.6.7
4. npm ls jest or npm ls react-scripts (if you haven’t ejected): jest@29.6.1
5. your vscode-jest settings if customized: default settings, just enabled coverage using vscode command
6. Operating system: Ubuntu 22.04 (KDE)

Prerequisite

• are you able to run jest test from the command line? yes
• how do you run your tests from the command line? (for example: npm run test or node_modules/.bin/jest) using npm run test, which looks like: jest --coverage --watch

Steps to Reproduce

I'm making a little project using Next.js 13.4.4 and Capacitor.JS ^5.0.4 (to make mobile apps). I recently installed jest following the next.js doc and jest doc. If i use my npm command, in a terminal, it is working well, but when i'm using the vscode extension, it runs tests, but I only see output in terminal view, nothing in the test results tab, and even in terminal output, there is no coverage, and in the command executed by the extension, I can see the --coverage twice. If I disable the coverage, it will still have a --coverage + a --no-coverage in the same command.

Expected Behavior

Showing test outputs in the dedicated tab, and show coverage if enabled

Actual Behavior

No output in test results tab, only in terminal tab, and there is no coverage, even if it's written in the command args.

[connectdotz]

but I only see output in terminal view, nothing in the test results tab

Not sure what you mean by "results tab", the "OUTPUT" tab? Anyway, the jest test run output will only appear in the terminal now. And it will not show test coverage in the file unless you enable the test coverage: please see How to use code coverage?. If you always want to see test coverage, you can also consider the setting jest.showCoverageOnLoad

Let me know if you still have any problem seeing them.

[navalex]

Hm i'm talking about the "Test Results" tabs on bottom of my VSCode. I guess it would be easier if i can get the output of a specific test when i select it in the "Tests" tab for example. That is at least the feature that i would have to make me stop use the terminal jest command manually with watch.
And for the coverage, even if I enable it in the extension settings, it doesn't change anything.
Thanks for your reactivity.

[Sammyjroberts]

Hey, I am having the same issue, it seems to create 2 Test Runs, one has the output and one has the successful test.

this is the output of the test with the green check, the one with the gray circle has the actual test output (also the terminal for jest)

[connectdotz]

I see. Yes, the output you expect to see in the "TEST RESULTS" tab is actually in the terminals, but I agree we should look into adding better support for "TEST RESULTS" tab.

Right now, the test and coverage results UI are displayed in the editors; and the detailed test run output (exclude coverage) is shown in the test terminals (see here). I will mark this as a feature request. Meanwhile, you should be able to use the terminals for the test output.

[hindicator]

@connectdotz

I have the same problem,
I actually want the output to be in the terminal,
and the output is in the terminal.
but the "TEST RESULTS" tab Popup every time I'm running a test instead of just remaining in the terminal window.

I'm using

{
"jest.showCoverageOnLoad": false,
 "jest.virtualFolders": [
        {
            "name": "package1",
            "rootPath": "packages/package1",
            "showCoverageOnLoad": false,
        },
        {
            "name": "package2",
            "rootPath": "packages/package2",
            "showCoverageOnLoad": false,
        }
    ],
}

[connectdotz]

For those who landed here because of the new "TEST RESULTS" tab: you can disable it so it won't automatically switch to this tab when you run tests via the gutter or test explorer UI:

"testing.openTesting": "neverOpen"

A bit of history:

A while back, we decided to pipe the test output to our own terminals (Jest terminal for each workspace/virtual-folder) instead of the single TestExplorer's output (now shown in the "TEST RESULTS" tab), which displays the output from the current running tests regardless of workspace/virtual-folder. We found a single output UI could be confusing and hard to find the actual output you are interested in, especially for multiroot or virtual-folder projects, therefore decided to adopt the multi-terminal approach...

A quick survey:

I would like to hear the use cases that make this single output interface more useful than the multi-terminals and therefore we should support both...

[connor4312]

@connectdotz I have some time this month to focus on test and improvements, and I actually came here to inquire about why Jest opted to use other APIs like diagnostics rather than relying solely on the testing API, and see if there were any gaps we could improve to make the experience better 🙂

The latest Insiders has lots of refinement this iteration, including features you don't get running solely in a terminal, such as some new contribution points which may be useful for snapshot tests and the ability to show and filter to test-specific appendOutput in an inline terminal.

[connectdotz]

Hi @connor4312, thank you for continuously improving the VS Code testing API; it has indeed come a long way. Your heads-up regarding the upcoming changes is appreciated, and I'll take a closer look and discuss them separately later.

why Jest opted to use other APIs like diagnostics rather than relying solely on the testing API

You might be aware that we do support both. The strategy started as a backward compatibility consideration - the extension used error diagnostics long before the testing API was available. Since the testing API errors introduced quite a different UI, we decided to use a switc (testExplorer) for users to transition and choose their preferred user experience.

I have not heard any user requests to switch the default (which currently defaults to diagnostics), so I never went back to revisit it. While UI is always subjective, I think the diagnostics UI offers the following distinct experiences:

1. It is less intrusive - it only appears when requested (hovering over the red underline, for example), with the bonus of auto-populating the PROBLEMS tab.
2. The UI is consistent with other tooling UIs, like eslint, tsc, for flagging errors.

I hope this helps.

[connor4312]

To paraphrase, it sounds like the gaps that make you avoid the native testing experience are:

• By 'less intrusive', you refer to the "Test Results" view automatically opening when a user (manually) runs tests?
• You'd like test failures to show up as diagnostics in the Problems view

Is that accurate?

[connectdotz]

@connor4312, it looks like we might be mixing two things here: errors and output.

Errors
We use the diagnostics API for errors. My reply earlier (regarding intrusiveness, etc.) referred to how the errors are presented in the editor itself, not the "Test Results" tab. The errors from the testing API seem to inject a large window that splits the code by default. In contrast, other error UIs, adopted by tools like eslint, tsc, and this extension, use a less intrusive mechanism: using color-underline indicators in the source code and only showing details when users request them, such as by mouse hovering.

The population of the PROBLEMS tab follows the same error UI standard that other tools are already using. I think the gap seems to be whether to adopt the existing error reporting UI pioneered by the diagnostics API or create a new one as the testing API did. Hard to say which is better thus the customization setting for users to choose. Would be good if we can take the best of both worlds, perhaps a hybrid approach? Maybe to enhance the diagnostics popup to host richer features like those provided by the testing API?

Output
The "TEST RESULTS" tab you referred to above is what we call the "output" of the jest process run. This is merely a pass-through from the jest process; we neither parse nor extract errors or status from it. The test errors, coverages, and status are produced by jest separately (as a JSON file), specifically to spare tooling systems the pain of parsing process output.

You mentioned test-correlated console output earlier, if I understand it correctly, it requires test providers to submit test-block-specific output. While this feature is indeed valuable, supporting it would be challenging for us because of output parsing mentioned above. It will work when the user runs a specific leaf test block via the testItem UI, where all output relates to that specific block. But if the user runs a describe block, a test file, or a directory containing multiple test blocks, without parsing, we won't know which output belongs to which tests.

That said, I suppose we could contribute all the non-parsed output to the workspace testItem. But then we'd need to guide our users to look for the output in that specific place, possibly leading to a discovery issue. Also, if most testItems show empty output, I'm not sure if that aligns with the ideal user experience you or our users envisioned.

Perhaps I'm missing something? I'd love to hear your thoughts.

[mbeerta-factset]

@connectdotz: We were actually looking into the same thing today as we are not seeing the test output in the test results tab. Your explanation provides valuable feedback here.

But then we'd need to guide our users to look for the output in that specific place

I don't know how it happens, but whenever I press the "run test" action, my vscode automatically puts me into the test output window, so there seems to be at least some setting I actually set, which would guide me to this.

[abm0]

@connectdotz, thanks for the suggestion on fixing the "Test Results" panel opening automatically, that was exactly what I was looking for.
Regarding your survey: although auto-opening of the panel felt annoying, the output to this panel may be useful, especially when I need to see the history of previous test runs - I simply need to press the "Show output" button in the Testing panel, it feels native and intuitive.
I also could not get the "Auto Reveal Output" option to work - it just doesn't open the terminal as it should. Could you please suggest anything to solve that issue?