See my attached tracking demo. It uses color segmentation. You might be able to use optical flow but then you'd also have to deal with the other things that move, like the paddles and players.
I doubt whether your algorithm would be fast enough to track a ball during a live game. Regardless, you'd need a very expensive high speed camera because I don't think the frame rate of a standard camera is high enough. I mean, the ball may travel across the whole table in just one frame time.
