Is there an easy way to build a test suite from all tests in a folder containing multiple packages?

I have a non-package directory, containing several packages. Each package may contain subpackages. The packages have class-based unit tests. I can run all the tests for a given package by calling

runtests('mypackage1', "IncludeSubpackages", true)

Is there an easy way to run all tests for all packages?

If I call

runtests(pwd, "IncludeSubfolders", true, "IncludeSubpackages", true)

No tests are run.

Answer 1

[Andy Campbell]

Hi Brian,

I definitely endorse Sean's suggestion to use projects. That said, this is good feedback since it seems you expected IncludeSubfolders to just work in this case. There are reasons for the current behavior, but we can take this feedback to the development team.

In the meantime, if you don't use Sean;s suggestion of putting it into a project, you should be able to create your suite as follows:

import matlab.unittest.TestSuite;
w = what(pwd);
suiteCell = cellfun(@(p) TestSuite.fromPackage(p, 'IncludingSubpackages',true), w.packages, 'UniformOutput', false)
suite = [suiteCell{:}];
run(suite);

Hope that helps!

[TallBrian]

Thanks, this solution works for me, and I will also look into projects.

Answer 2

[Sean de Wolski]

Have you considered adopting projects? I'd recommend it (for most things and especially for this use case) then you can just create the TestSuite from all files in the project.

https://www.mathworks.com/help/releases/R2019b/matlab/ref/matlab.unittest.testsuite.fromproject.html

Otherwise, shy of manually appending multiple TestSuites I don't know a way. My team usually keeps all tests as classes optionally in subppackages of a parent +test package so it's obvious it's a test.

[TallBrian]

Thanks, I will look into projects.

[TallBrian]

@seandewolski I looked into projects. I have a couple of questions.

1) Is it possible to separate functions into namespaces similar to package functionality in projects?

2) Can you have projects of projects?

3) Is it possible to generate a test suite from both the root level of a project (yes, you have linked the instructions) as well a subfolder of the project?

I would like to have some structure where I can run all tests for everything easily and then also just run tests for one subgroup easily. Is there a best practice for this?

thank you.