masoud - triu returns the upper triangular part of a matrix so it is should be straightforward on how to implement this function. If X is your input matrix, then the output matrix U has to be the same size as X, so
Now if you are only concerned with those elements above and including the main diagonal, then you need to copy these elements from X into U. But which ones are they? Look at the diagram drawn in the description for triu. If we start with the first row, then we copy all column elements i.e. those elements from columns 1,2,3,...,*m* where m is the number of columns. When we move to the second row, we copy all those element from the second column onwards. So what is the pattern?
Since this is homework, I will leave it to you to determine the pattern and what c should begin iterating over.