First, we need to understand why MathWorks has made the Test Suite visible to students. Students are encouraged to use MATLAB on their computer to solve problems. To make this possible, students must be able to copy and paste the Test Suite code into a MATLAB script which, in turn, calls their solution function. Once satisfied, they then copy their solution function into Cody Coursework for assessment.
With this in mind, consider how a student might "game the system" presented with the following Test Suite from the Times2 problem on Cody:
Here, the Test Suite simply calls the students' function, times2, using explicit data values to confirm the students' code returns the expected result. This approach is morally hazardous, as a student can simply write a case or series of if statements to compromise the approach.
Each Test Suite is a MATLAB script. Here we draw on this power to more effectively assess students' code in response to the Times2 problem,
Y_answer = x*2;
This still poses two issues, 1. it reveals the answer the students' code needs to implement (i.e. return x * 2), and 2. it does not apply well to complex programming problems.
Now that we've examined what not to do, let's discuss a more effective approach for assessing students' MATLAB code. Faculty already have a reference solution to each programming problem assigned, traditionally offered to teaching assistant staff for manual grading. Protected MATLAB code offers the key of how we use these reference solutions as part of a generalized recipe to write an effective Test Suite.
First, compile your reference solution as Protected MATLAB code.
Second, post this .p file somewhere online accessible by Cody Coursework. This must not be behind the school's firewall, it must be publicly accessible. One option is to use the public folder of a free dropbox account. If using DropBox, it will help to " copy the public link " using their web interface.
Third, implement a test case of your test suite to use your protected reference solution code, for example:
%%Test Functional Correctness of Students’ Code
prof_answer = ProfsFunction(data_vector);
student_answer = StudentsFunction(data_vector);
assert(isequal(prof_answer,student_answer), 'student code gave an incorrect answer');
Adapting this for more complex problems will require you adapt the data_vector randomization to something appropriate for your subject matter. Alternatively, you might wish to incorporate a data file from which your Test Suite randomly selects input/output values.
Fourth, add a test case to prevent student attempts to call on the .p file themselves, for example by parsing the students' solution file directly,
%%Test for attempts to cheat
found = 0;
fid = fopen('StudentsFunction.m');
s = fgetl(fid);
k = regexp(s,'urlwrite\(|ftp\(|mget\(|web\(|fprintf\(|fwrite\(|sprintf\(');
found = 1;
Fifth, use the solution map as a "line of last defense." Typically, honest solutions which are correct follow similar patterns. Horizontal bars of these patterns form on the solution map. The few stray solutions below these bars which Cody Coursework has marked correct merit manual code inspection.
There are many possibilities the power of the MATLAB language offers for implementing test suites. We will add more information and examples in future, and I encourage you to share your own insights as answers or comments below.