Alysia,
[Edit: correct typos.]
This sounds like a great project!
Tracking the x-y coordinates of the paintbrush will be the hard part. Once you have that data, the rest will be relatively easy, including computing the velocity. As you know, a theramin has two antennas: one for volume and one for pitch. Your proposal to use x,y in a similar way makes sense. You suggest using velocity to control the "tonality of the instrument" or "harshness". No doubt you know more about music than I. Your use of "tonality" to refer to a single instrument is different than what I'm used to. I think of tonality as referring to the use, or not, of a diatonic scale or mode (major, minor, phrygian, locrian), or to the greater use of tones that are not part of a diatonic scale (chromaticism). But I think I know what you mean by harshness. Some people consider the oboe to be more harsh than the clarinet. You could try to achieve different "harshnesses" by using velocity to adjust the overtones associated with each pitch.
To track the x,y coordinates of the brush using a cell phone video, you'll want a marker on the brush that stands out so it is trackable. Maybe a white dot painted on the brush, or a small white sphere glued onto the brush. But if the canvas is white, it will still be hard to track. I suggest you have the painter wear black, darken their painting hand, and paint on a black canvas with a dark tarp hung behind the easel, so that the only bright thing in the video is the white dot on the brush. This will make it easier to track the brush marker in the video. If you or a friend is facile with electronics, you could attach a bright LED (such as this) to the brush to make it stand out in a more normal painting environment. Power the LED with 4 AA batteries in series (6V), two wires running to the LED, and a 150 ohm resistor to get ~20 mA through the LED {(6V-3.2V forward drop)/150 ohm = 0.019 amps.} 
