How to dynamically set method/class setup parameters

hi,

I'm trying to write a parameterized test where one class setup parameter depends on another one. I'm running it from a script.

myScript.m:

suite = matlab.unittest.TestSuite.fromClass(?myTest)
runner = matlab.unittest.TestRunner.withTextOutput();
results = runner.run(suite);

The test class is defined in a seperate file:

 classdef myTest < matlab.unittest.TestCase
   properties(ClassSetupParameter)
     prop1 = foo();
     prop2 = {1,2,3};
     prop3 = bar(prop1);
   end
 end
 function foo_result = foo()
   foo_result = {3,4,5};
 end
 function bar_result = bar(prop1)
   bar_result = prop1 -1;
 end

I'm getting an error:

Error using myScript (line 1)
Invalid default value for property 'prop3' in class 'myTest':
Undefined function or variable 'prop1'.

I have also tried seperating prop1 and prop3 to Class and Method setup parameters:

   classdef myTest < matlab.unittest.TestCase
     properties(ClassSetupParameter)
       prop1 = foo();
       prop2 = {1,2,3};
     end
     properties(MethodSetupParameter)
       prop3 = bar(prop1);
     end
   end
   function foo_result = foo()
     foo_result = {3,4,5};
   end
   function bar_result = bar(prop1)
    bar_result = prop1 -1;
   end

And I'm getting the same error

Any ideas how to overcome this? I really don't want to have another loop inside any test method.

I'm on Matlab 2017a

Thank you! Jack

Answer 1

[Steven Lord]

Try this:

classdef myTest < matlab.unittest.TestCase
    properties(ClassSetupParameter)
        prop1 = foo();
    end

      properties
          prop3;
      end

      methods(TestClassSetup)
          function setup_prop3(testcase, prop1)
              testcase.prop3 = computeProp3FromProp1(prop1);
          end
      end

      methods(Test)
          function displayProp3(testcase)
              testcase.log(sprintf('The value of prop3 is %d.\n', testcase.prop3));
          end
      end
  end

function y = foo()
y = {3, 4, 5};
end

function y = computeProp3FromProp1(p)
y = p-1;
end

To see that the prop3 property takes on the values 2, 3, and 4 run this with concise logging.

suite = matlab.unittest.TestSuite.fromClass(?myTest)
runner = matlab.unittest.TestRunner.withTextOutput();
L = matlab.unittest.plugins.LoggingPlugin.withVerbosity(matlab.unittest.Verbosity.Concise);
runner.addPlugin(L);
results = runner.run(suite)

[Jack German]

I need prop3 to be an iterable parameter for the test methods. If I understand correctly prop3 is now a regular class property, so unittest will not iterate through it?

I'll try this tomorrow first thing and report back. Thanks!

[Jack German]

ok. Saving prop3 into a regular property worked. but now I can't have prop3 as an iterable parameter in my class setup. I guess I should have been more clear. I apologize.

classdef myTest < matlab.unittest.TestCase
   properties(ClassSetupParameter)
     prop1 = foo();
     prop2 = {1,2,3};
     prop3 = bar(prop1);
   end
   properties
     data;
   end
   methods(TestClassSetup)
     testCase.data = do_somehting_for_setup(prop1,prop2,prop3)
   end
   methods(Test)
     function test1(testCase)
       verify_something_about_data
     end
     function test2(testCase)
        verify_something_else_about_data
     end
 end
 function foo_result = foo()
   foo_result = {3,4,5};
 end
 function bar_result = bar(prop1)
   bar_result = prop1 -1;
 end

I have the class setup each time with a fresh permutation of prop1, prop2 and prop3. I do is this way because this setup is computationally heavy and I don't want to do it before each test method.

[Steven Lord]

So you want to run your TestClassSetup method 27 (3^3) times, each with a different combination of prop1, prop2, and prop3 values? In that case, I'd make a Constant property from which prop1 and prop3 can be computed. As stated in the documentation (this page hasn't change much if at all between release R2017a and release R2018a):

"MATLAB evaluates the expressions used in the class definition without any workspace. Therefore, these expressions cannot reference variables of any kind.

MATLAB evaluates expressions in the context of the class file, so these expressions can access any functions, static methods, and constant properties of other classes that are on your path at the time MATLAB initializes the class. Expressions defining property default values can access constant properties defined in their own class."

So try running this test as before.

classdef myTest < matlab.unittest.TestCase
    properties(Constant)
        propertyFoo = foo();
    end
    properties(ClassSetupParameter)
        prop1 = myTest.propertyFoo;
        prop2 = {pi, exp(1), 42};
        prop3 = computeProp3FromProp1(myTest.propertyFoo);
    end
    methods(TestClassSetup)
        function displayPropertyValues(testcase, prop1, prop2, prop3)
            testcase.log(sprintf('The value of prop1 is %d.\n', prop1));
            testcase.log(sprintf('The value of prop2 is %f.\n', prop2));
            testcase.log(sprintf('The value of prop3 is %d.\n', prop3));
        end
    end
    methods(Test)
        function onePlusOne(testcase)
            testcase.verifyEqual(1+1, 2);
        end
    end
end

function y = foo()
y = {3, 4, 5};
end
function y = computeProp3FromProp1(p)
y = cellfun(@(x) x-1, p, 'UniformOutput', false);
end

When you run this you should see the onePlusOne test run 27 times, with each combination of values for prop1, prop2, and prop3. FYI, if running each of your tests for each of the 27 combinations is a bit much, consider specifying the ParameterCombination attribute as 'pairwise'.

methods(TestClassSetup, ParameterCombination = 'pairwise')

[Jack German]

This looks great. I am truly sorry for being a bother, but I have another twist which I just realized...(thanks to your answers :) )

prop3 is actually just sequential integers running from 1 to an unknown number which is determined by running a function on prop1.

Thus, my test parameter matrix (might) look like this (exclude prop2 - doesn't change here).

 prop1   prop3
---------------
   v1      1
   v1      2
---------------
   v2      1
---------------
   v3      1
   v3      2
---------------
   v4      1
   v4      2
   v4      3
   v4      4

Is it possible?

[Steven Lord]

I'm not completely sure what you're asking. Can you compute the "unknown number" for prop3 given the value of prop1 (say by changing the computeProp3FromProp1 function in my example test?)

Or do you want a different set/number of values for prop3 based on the particular value of the prop1 property that's being used in this call to your TestClassSetup method? So when prop1 is 1, prop3 can be 2 or 4; when prop1 is 2, prop3 can be 3, 5, or 7; that sort of thing?

[Jack German]

I'm trying to accomplish the second option. While prop1 is (say) 1 prop3 will change from 1 to a number that can be determined from prop1.

[Steven Lord]

Rather than using two separate properties in that case I'd probably assemble a cell array of vectors, the first element of each vector being the value of prop1 and the second prop3, and use that cell array as the property.

prop13 = {[1 1], [1 2], [2 1], [2 2], [2 3]};