Skip to content

Conversation

rugvedS07
Copy link
Member

@rugvedS07 rugvedS07 commented Aug 15, 2025

Overview

Adds the ability to run full tests in CI for linux using the docker container.

Fixes #140 for linux tests

@github-actions github-actions bot added the CLA signed Indicates that all contributors have signed label Aug 15, 2025
docker stop llmster || true
docker rm llmster || true

# Coverage check for Docker tests (higher target since they include LM Studio tests)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one Python version per-platform actually collects coverage data, so if we're not combining across platforms, there's no need for the coverage analysis to be a separate job.

I'm not sure of the rationale for skipping the combined coverage report, though. Checking the per-platform coverage in addition to the combined coverage does seem reasonable.

@ncoghlan
Copy link
Collaborator

@rugvedS07 One of your remaining failures is an existing test ordering issue: the intended usage of smollm2-135m-instruct is solely as the model used to test the model catalog search-and-download functionality, so it isn't on the list of models to download and load prior to running the tests.

test_jit_unloading_[a]sync isn't using it that way: it is incorrectly assuming that the model is already downloaded and available for loading.

Probably the simplest interim fix is to pre-download (but not load) smollm2-135m-instruct in the Docker testing setup - the catalog tests still run with that model pre-downloaded, and it avoids having to unload and reload one of the bigger models twice in that test case (that one is not only quicker to load, but none of the other functional tests use it, so it can be left unloaded at the end of the test case).

The other failure I saw is due to one of the plugin test cases not seeing the expected graceful shutdown message. It's plausible that one is just outright flaky and/or distro sensitive (it's previously only been tested by me locally in Fedora WSL and Windows Powershell)

@ncoghlan
Copy link
Collaborator

I just added Python 3.14 to the CI test matrix in main, so that's the origin of the reported merge conflict.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA signed Indicates that all contributors have signed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run LM Studio in headless mode in CI
2 participants