Skip to content

Conversation

fundakol
Copy link
Contributor

@fundakol fundakol commented Aug 27, 2025

Twister blackbox tests are not executed on CI.
This change fixes the problem.

It is broken since https://github.com/zephyrproject-rtos/zephyr/pull/88754/files

Twister blackbox tests are not executed on CI.
This change fixes the problem.

Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
@fundakol fundakol force-pushed the fix-github-actions-for-twister-blackbox branch from 0af20cd to 4b34651 Compare August 27, 2025 11:33
Copy link

@fabiobaltieri fabiobaltieri requested a review from pdgendt August 29, 2025 10:25
@pdgendt
Copy link
Contributor

pdgendt commented Aug 29, 2025

CI doesn't appear to be happy?

@fabiobaltieri
Copy link
Member

uh oh you are right, guess it was masking a failure after all, @fundakol have you looked into it already by any chance?

@fundakol
Copy link
Contributor Author

uh oh you are right, guess it was masking a failure after all, @fundakol have you looked into it already by any chance?

I reviewed the failed tests, but they are too complicated for me to fix. You might want to find the person who broke those tests and ask him to provide a fix. Alternatively, you can mark all those tests with pytest.mark.xfail and merge the fix for the broken CI. The tests can be fixed later.

@fundakol
Copy link
Contributor Author

fundakol commented Aug 29, 2025

I found one commit which breaks one test:
9219c81

You can bisect other tests with this solution (I assume you have installed all required packages)

Install nox:

pip install nox

Create ~/zephyrproject/zephyr/noxfile.py file with following code:

import os
from pathlib import Path
import nox

ROOT_DIR = Path.home().joinpath("zephyrproject")

@nox.session(python=False)
def west_update(session):
    with session.chdir(ROOT_DIR):
        session.run('west', 'update', silent=True)

@nox.session(python=False)
def test(session):
    env = os.environ.copy()
    env["PYTHONPATH"] = f'{ROOT_DIR / "zephyr/scripts"}:{ROOT_DIR / "zephyr/scripts/pylib"}:{ROOT_DIR / "zephyr/scripts/pylib/twister"}'
    
    session.run(
        'pytest', "scripts/tests/twister_blackbox/test_testplan.py",
        '-k', "test_subtest[valid]",
        env=env
    )

Start bisect:

git bisect start
git bisect good d2ebe9c6de2ca96ca7370a82589ce6f2cd0c8385
git bisect bad b68058787e992ca199ff287336e349d1c1ba22b1
git bisect run nox

@nashif
Copy link
Member

nashif commented Aug 30, 2025

merged in another PR

@nashif nashif closed this Aug 30, 2025
@fundakol fundakol deleted the fix-github-actions-for-twister-blackbox branch August 30, 2025 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Continuous Integration size: XS A PR changing only a single line of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants