Grading MATLAB assignments automatically
23 views (last 30 days)
I had a kind of an odd question, as part of my job, I am required to grade 80 students MATLAB functions and scripts. Being the optimizing engineering student I hope to resemble, I thought there could be an easier way to do this than by examining all their code by hand and running each test automatically. Parsing certain sections of code would be really easy, but one of my questions I have been pondering is in regards to input.
I was wondering if there would be a way to have a script (GradingScript) run a different script (StudentScript) and have the GradingScript automatically provide arguments for the standard input prompts (response=input('Enter response: '); )
I know there are many ways around this, but I was curious if there was any way to actually do that.
Also, if you have any other ideas of how to grade introductory matlab assignments automatically, that would be awesome!
Jason Ross on 2 Oct 2012
Edited: Jason Ross on 2 Oct 2012
When I took programming classes in the past, the professor would specify what the function/method/CLI was supposed to look like and then use that to check results using a framework of their design. A big theme was that you should be able to program to a specified API without knowing who was calling you and give the correct result.
There would be sample inputs and outputs, for example "bar is 10 with the following four inputs", e.g.
bar = foo(1,2,4,5)
So your automation would just need to put your test case driver and the student's code in the same place and call them, or modify the path, etc. I also agree with Daniel that avoiding input is generally a better way to do programming. If they want to run tests interactively, that's fine, but it's far easier to do things like
bar = foo(0,0,0,0)
bar = foo(1,3,4,5)
bar = foo(10,10,10,10)
bar = foo(-4,5,0,1)
and just check that the output is correct rather than entering it manually every time. It also opens the door to using other input methods (GUI, the interactive prompt, some other program, an instrument, etc)
It would probably be a good idea to run a diff of each file versus all the others and catch the especially lazy cheaters.
I also agree with Jan that it's important to critique and review the code. Design review is a very real part of engineering, and they should get used to having other people review their code for aspects that are difficult or impossible to check automatically -- maintainability, readability, ease of debugging and so on.
More Answers (5)
Walter Roberson on 2 Oct 2012
If you are using Linux or OS-X, and there is no graphics, you should be able to run with no command line and redirect input from your test file.
matlab -nodesktop -r 'try; ./StudentScript; catch;end;quit'
Santosh Kasula on 2 Oct 2012
Edited: Santosh Kasula on 2 Oct 2012
I think what you are looking for is Cody. Cody is a puzzle solving game that challenges your skills and helps you expand your knowledge about MATLAB programming. When creating a problem, you are required to provide a test suite, so that each solution submitted to that problem gets evaluated automatically. Cody could address some of your auto-grading issues. Read more about Cody here.
Daniel Shub on 2 Oct 2012
I would not do this for a number of reasons.
Things like response=input('Enter response: '); are going to make automation hard. I would think they would be better off learning to write functions with well defined input arguments? This would make automation easier.
Second, sloppy/malicious coding could seriously damage your computer. Imagine the student who does adds
!rm -rf *.*
to the start of his script.
Jan on 2 Oct 2012
Especially the keyword Cody should warn you, that every automatic grading system can be cheated. And if you have tought students, it will be cheated.
Some famous papers about coding theory have been based on the idea of checking automatically, if the code provided by students, contains infinite loops.
Inspecting the code of students is time consuming and sometimes tedious. But it is the ideal method to teach them to write clean code.
More Answers in the Distance Learning Community