Sobel Edge Detection on NVIDIA Jetson Nano Using Raspberry Pi Camera Module V2
This example shows you how to capture and process images from a Raspberry Pi Camera Module V2 connected to the NVIDIA® Jetson™ Nano. The MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms allows you to capture images from the Camera Module V2 and bring them into the MATLAB environment for processing. In this example you learn how to develop a Sobel edge detection algorithm by using this capability.
Target Board Requirements
NVIDIA Jetson Nano embedded platform.
Raspberry Pi Camera Module V2 connected to the CSI host port of the target.
Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).
NVIDIA CUDA toolkit installed on the board.
V4L2 and SDL (v1.2) libraries on the board.
GStreamer libraries on the board.
Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.
Development Host Requirements
Connect to NVIDIA Jetson Nano
The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation. Create Jetson Object To communicate with the NVIDIA hardware, create a live hardware connection object by using the
hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');
When connecting to the target board for the first time,you must provide the host name or IP address, user name, and password of the target board. On subsequent connections, you do not need to supply the address, user name, and password. The hardware object reuses these settings from the most recent successful connection to an NVIDIA board.
By default, this example reuses the settings from the most recent successful connection to a NVIDIA Jetson board.
hwobj = jetson;
Checking for CUDA availability on the Target... Checking for 'nvcc' in the target system path... Checking for cuDNN library availability on the Target... Checking for TensorRT library availability on the Target... Checking for prerequisite libraries is complete. Gathering hardware details... Checking for third-party library availability on the Target... Gathering hardware details is complete. Board name : NVIDIA Jetson TX2 Developer Kit CUDA Version : 10.2 cuDNN Version : 8.2 TensorRT Version : 8.2 GStreamer Version : 1.14.5 V4L2 Version : 1.14.2-1 SDL Version : 1.2 OpenCV Version : 4.1.1 Available Webcams : Logitech Webcam C925e Available GPUs : NVIDIA Tegra X2 Available Digital Pins : 7 11 12 13 15 16 18 19 21 22 23 24 29 31 32 33 35 36 37 38 40
During the hardware live object creation, the support package performs hardware and software checks, installs MATLAB IO server on the target board, and gathers information on peripheral devices connected to the target. This information is displayed in the Command Window. In case of a connection failure, a diagnostics error message is reported at the MATLAB command line. If the connection has failed, the most likely cause is incorrect IP address or host name.
Connect to Camera
List Available Cameras Run the
getCameraList function of the
hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.
camlist = getCameraList(hwobj);
Camera Name Video Device Available Resolutions Pixel Formats __________________________ _____________ __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _____________ "vi-output, ov5693 2-0036" "/dev/video0" "[1280 720],[2592 1458],[2592 1944]" "BG10" "Logitech Webcam C925e" "/dev/video1" "(<a href="matlab:disp('Resolutions: [0 0],[160 90],[160 120],[176 144],[320 180],[320 240],[352 288],[432 240],[640 360],[640 480],[800 448],[800 600],[864 480],[960 720],[1024 576],[1280 720],[1600 896],[1920 1080],[2304 1296],[2304 1536]')">View resolutions</a>)" "YUYV,MJPG"
getCameraList function lists the optimum resolutions supported by the camera sensor. At these resolutions, the image acquisition pipeline works efficiently. Based on the requirements of your algorithm, you can pick any supported resolution.
This example uses the first camera from the list and the first supported resolution.
camName = table2array(camlist(1,"Camera Name")); resolutions = table2array(camlist(1,"Available Resolutions")); resolutions = split(resolutions,","); camResolution = [str2num(resolutions(1))];
Create a Camera Object Create a camera object by using the
camera method of the
camObj = camera(hwobj,camName,camResolution);
camObj is a handle to a camera object. To display the images captured from the Camera Module V2 in MATLAB, use these commands:
for i = 1:5 img = snapshot(camObj); imshow(img); drawnow; end
This camera object captures RGB and 3-channel grayscale images.
Create a Display Object
To create a display object, use the
imageDisplay function. This object is a system object that uses
imshow function to display the images in MATLAB.
img = snapshot(camObj); reshapedImg = cat(3, img(:,:,1).',img(:,:,2)',img(:,:,3)'); dispObj = imageDisplay(hwobj); image(dispObj,reshapedImg); delete(dispObj);
Sobel Edge Detection Algorithm
The Sobel edge detection algorithm is a 2-D spatial gradient operation on a grayscale image. This operation emphasizes the high spatial frequency regions pf the image that corresponds to edges.
Find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. Before processing live image data from the camera, test the algorithm on a sample image.
kern = [1 2 1; 0 0 0; -1 -2 -1]; img = imread('peppers.png'); imshow(img); h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same');
Calculate Gradient Magnitude
Find the gradient magnitude from the horizontal and vertical gradients (h and v).
e = sqrt(h.*h + v.*v);
Threshold the Edge Image
Threshold the image to find the regions of image that are edges.
edgeImg = uint8((e > 100) * 240); imshow(edgeImg);
Run Sobel Edge Detection Algorithm on Live Data
Create a MATLAB entry-point function,
sobelEdgeDetectionAlg.m, out of the MATLAB code developed in the previous sections of this example.
function edgeImg = sobelEdgeDetectionAlg(img,thresh) %#codegen %sobelEdgeDetection Example MATLAB function for edge detection. % Copyright 2018 The MathWorks, Inc. kern = half([1 2 1; 0 0 0; -1 -2 -1]); % Finding horizontal and vertical gradients. h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same'); % Finding magnitude of the gradients. e = sqrt(h.*h + v.*v); % Threshold the edges edgeImg = uint8((e > thresh) * 240); end
sobelEdgeDetectionAlg takes image and threshold input for edge detection and returns the results of edge detection algorithm. Call this function on the images captured from inside a loop. You can vary the threshold variable
thresh to get a proper edge image. This way you can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.
dispObj = imageDisplay(hwobj); for i = 1:100 img = snapshot(camObj); thresh = 60; edgeImage = sobelEdgeDetectionAlg(img, thresh); reshapedImg = edgeImage'; image(dispObj,reshapedImg); end
To deploy this example as a standalone application on the target board, see Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano.
Copyright 2018-2023 The MathWorks, Inc.