use matlab to interact with external program?

Hi

Is it possible to have MATLAB interact with an external application by not only receiving input from it but running some calculations on that input and then sending output back to the program to control it in real time as though MATLAB is the user?

I want to make a robot that controls a game, but the game must be external to MATLAB.

Thanks!

Answer 1

[Walter Roberson]

It sounds to me as if what you need is for MATLAB to talk to the robot . The robot would have sensors (video almost certainly, possibly audio as well) that would have to be sent back to MATLAB, and would have control surfaces of some kind that would be activated by MATLAB -- but MATLAB would not be connecting to the game, MATLAB would be connecting to the robot.

There are a number of packages are appropriate, including Raspberry Pi, Arduino, and a couple of robotics devices.

[Lewis]

Well when I said robot I was thinking more along the lines of software written in MATLAB that controls the game, rather than actual hardware. I would open up the game and MATLAB on the same computer and then run the program for MATLAB to start playing the game. Something like that.

[Walter Roberson]

Java Robot Class can be used to click buttons and press keys.

Getting information back from a graphics executable is tricky. You would need to find the executable's window handle (HWND) and ask to get the graphics buffer associated, and analyze that buffer.

I do not know the restrictions on one program fetching another's graphics buffers in MS Windows. In the general case it is not possible because of anti-copying measures for DVD and CD (if you could get the graphics then you could take a perfect digital copy of a DVD at that resolution, which hardware and copyright protection is designed to prevent.) But that is the general case, and there could potentially be exceptions, such as if the second process was started under the debugger.