OBJECT DETECTION BY USING ESP32-CAM
Department of Electronics and Communication
Engineering
Submitted by
NAME REGISTER
NUMBER
DHANEESH A 963521106027
CHANCHU S U 963521106026
DIXON J 963521106031
RAHUL CHURCHIL R E 963521106047
NAFIL A 963521106046
Guided by
Mr. G. Sivaramalingam - M.
Tech Mr. K. Gopal Ram AP/ECE
Stella Mary’s College of Engineering
BONAFIDE CERTIFICATE
Certified that this project report on “OBJECT DETECTION
BY USING ESP32 CAM” is the bonafide record of work
done by Dhaneesh A (963521106027), Chanchu. S. U
(963521106026), Dixon. J (963521106031), Rahul
Churchil. R. E (963521106047)
and Nafil. A (963521106046) from the Department of
Electronics and Communication Engineering by Anna
University, Chennai.
Internal Guide Head of the Department
Internal Examiner External Examiner
TABLE OF CONTENTS:
1. ABSTRACT
2. INTRODUCTION
3. PROBLEM STATEMENT
4. PROPOSED SYSTEM
5. COMPONENTS REQUIRED
6. BLOCK DIAGRAM
7. PIN DIAGRAM
8. WORKING
9. CODES
10. CONCLUSION
1. ABSRACT
The ESP32-CAM, a low-cost microcontroller with a built-
in camera, offers a powerful platform for implementing on-
device object detection. This approach eliminates the need
for internet connectivity and reduces processing latency. It
Capture images of the target objects from various angles
and lighting conditions. And it manually drawing bounding
boxes around the objects in the images and assigning them
labels. It Utilizing a platform like Edge Impulse to train a
machine learning model on the label data. The model learns
to identify the objects in new images and it Converting the
trained model into a format compatible with the ESP32-CAM
for on-device execution. And it Uploading the model to the
ESP32-CAM and running inference to detect objects in real-
time captured frames.
2. INTRODUCTION
The ESP32-CAM, a budget-friendly microcontroller with a
built-in camera, brings this vision to life. This powerful
combination allows you to perform object detection directly
on the device, eliminating the need for constant internet
connection and reducing processing delays. Object
detection, a branch of Artificial Intelligence (AI), empowers
devices to identify and classify objects within an image or
video stream. Traditionally, this required powerful
computers. However, the ESP32-CAM, with its processing
capabilities and compact size, makes object detection
accessible for a wider range of projects. This introduction
dives into the exciting world of object detection with ESP32-
CAM. We'll explore how to leverage this technology to
create intelligent applications for various purposes, from
security systems to smart toys and beyond. Stay tuned as
we unveil the steps involved in training a model, deploying
it on the ESP32-CAM, and unlocking the potential for
seeing and understanding the world around us with AI.
3. PROBLEM STATEMENT:
While cloud-based object detection offers impressive
accuracy, it suffers from several limitations for resource-
constrained devices like the ESP32-CAM and it Uploading
the video streams to the cloud for processing consumes
significant bandwidth and power on the ESP32-CAM.and it
have latency Issues for Real-time applications require low
latency. Sending data to the cloud and receiving results
introduces delays that can be detrimental. and its
Uploading video data to the cloud raises privacy concerns,
especially in sensitive applications. And it have Internet
Dependency for Consistent internet connectivity is a
requirement, limiting the functionality in areas with
unreliable internet access.
4. PROPOSED SYSTEM:
In The this proposed system we tackles the limitations of
cloud- based object detection by performing object
identification directly on the ESP32-CAM.It captures images
or video frames and processes them locally using a pre-
trained machine learning model. This eliminates the need
for constant internet connection and reduces power
consumption. And its Object detection happens directly on
the ESP32-CAM, enabling the real-time response and
immediate actions based on the detected objects. And its
Enhanced Privacy is no video data is sent to the cloud,
ensuring privacy for sensitive applications. and it have
Offline Functionality for system functions even without
an internet connection, making it ideal for remote locations
or applications requiring uninterrupted operation.
5. COMPONENTS REQUIRED
5.1 HARD WARE COMPONENTS
For object detection, we need ESP32CAM and TTL to USB
converter
5.1.1. ESP32-CAM AND TTL USB CONNECTER
The ESP32-CAM is a small-sized camera module with the
ESP32-S chip at its core. It allows for Wi-Fi and Bluetooth
connectivity, making it suitable for IoT (Internet of Things)
projects involving image and video processing. The camera
module usually comes with a built-in OV2640 camera and a
microSD card slot for storing images and videos. A TTL
USB connector is typically used for serial communication
with devices like microcontrollers, GPS modules, or even
the ESP32-CAM. It converts signals between USB and UART
(serial) interfaces, allowing you to connect your device to a
computer for programming, debugging, or data transfer. If
you're working with an ESP32-CAM, you might not need a
TTL USB connector unless you're interfacing it with other
devices or platforms that use serial communication.
However, if you want to program or debug the ESP32-CAM
via, serial connection, you can use a TTL USB connecter to
connect it to your computers USB port for communication.
5.2. S0FTWARE COMPONENT
5.2.1 EDGE IMPULSE
Edge Impulse provides tools to gather sensor data and
other inputs from the ESP32-CAM, including camera feeds. It
also allows you to label this data to train your machine
learning model effectively. and This platform offers a web-
based interface for building and training machine learning
models directly on Edge Impulse. These models are
specifically optimized for running on resource-constrained
devices like the ESP32-CAM, ensuring efficient
performance. If Once trained, Edge Impulse helps convert
the model into a format compatible with the ESP32-CAM.
This ensures seamless deployment and on-device execution
of the object detection model. and Edge Impulse boasts
integrations with various hardware platforms, including
sensors, microcontrollers (MCUs), and microprocessors
(MPUs) commonly used in edge devices. Edge Impulse
emphasizes a user-friendly experience with low-code
functionalities. This makes it accessible to developers of all
levels, allowing them to build and deploy machine learning
models for edge devices without extensive coding
expertise.
6. BLOCK DIAGRAM
Fig : Object detection block diagram
7. PIN DIAGRAM
FIG: Pin diagram for ESP32-CAM to TTL
USB Connecter
8. WORKING
8.1 OBJECT DETECTION BY USING ESP32-CAM
Object detection with an ESP32-CAM involves leveraging
machine learning models to analyze images captured by
the ESP32-CAM's camera module and identify objects within
those images. The ESP32-CAM captures images using its
built-in camera module. These images are typically in the
form of JPEG or BMP files. And before feeding the image into
the object detection model, pre- processing might be
necessary. This could involve resizing the image, adjusting
its colour channels, or applying other transformations to
optimize it for the model. A pre-trained object detection
model is deployed on the ESP32. This model could be based
on Convolutional Neural Networks (CNNs), such as YOLO
(You Only Look Once), SSD (Single Shot Multibox Detector),
or others. These models are trained on large datasets
containing images labeled with bounding boxes around
objects of interest. The ESP32 runs inference on the
captured and pre-processed image using the object
detection model. During inference, the model processes
the image and outputs bounding boxes along with the
class labels of the detected objects. Finally, the detected
objects can be visualized on the image (bounding boxes
drawn around them) or used to trigger actions based on the
application's requirements. For example, if the ESP32-CAM
is part of a security system, it could raise an alarm when a
person or intruder is detected.
9. code
9.1. CODE FOR OBJECT DETECTION BY
USING ESP32-CAM
/**
* Run Edge Impulse FOMO model.
* It works on both PSRAM and non-PSRAM boards.
*
* The difference from the PSRAM version
* is that this sketch only runs on 96x96 frames,
* while PSRAM version runs on higher resolutions too.
*
* The PSRAM version can be found in my
* "ESP32S3 Camera Mastery" course
* at https://dub.sh/ufsDj93
*
* BE SURE TO SET "TOOLS > CORE DEBUG LEVEL = INFO"
* to turn on debug messages
*/
#include <Object_Detection_inferencing.h>
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/edgeimpulse/fomo.h>
using eloq::camera;
using eloq::ei::fomo;
/**
*
*/
void setup() {
delay(3000);
Serial.begin(115200);
Serial.println("_EDGE IMPULSE FOMO (NO-PSRAM)_");
// camera settings
// replace with your own model!
camera.pinout.aithinker();
camera.brownout.disable();
// NON-PSRAM FOMO only works on 96x96 (yolo) RGB565 images
camera.resolution.yolo();
camera.pixformat.rgb565();
// init camera
while (!camera.begin().isOk())
Serial.println(camera.exception.toString());
Serial.println("Camera OK");
Serial.println("Put object in front of camera");
}
void loop() {
// capture picture
if (!camera.capture().isOk()) {
Serial.println(camera.exception.toString());
return;
}
// run FOMO
if (!fomo.run().isOk()) {
Serial.println(fomo.exception.toString());
return;
}
// how many objects were found?
Serial.printf(
"Found %d object(s) in %dms\n",
fomo.count(),
fomo.benchmark.millis()
);
// if no object is detected, return
if (!fomo.foundAnyObject())
return;
// if you expect to find a single object, use fomo.first
Serial.printf(
"Found %s at (x = %d, y = %d) (size %d x %d). "
"Proba is %.2f\n",
fomo.first.label,
fomo.first.x,
fomo.first.y,
fomo.first.width,
fomo.first.height,
fomo.first.proba
);
// if you expect to find many objects, use fomo.forEach
if (fomo.count() > 1) {
fomo.forEach([](int i, bbox_t bbox)
{ Serial.printf(
"#%d) Found %s at (x = %d, y = %d) (size %d x %d). "
"Proba is %.2f\n",
i + 1,
bbox.label,
bbox.x,
bbox.y,
bbox.width,
bbox.height,
bbox.proba
);
});
}
}
10. CONCLUSION
In this project, object detection using ESP32-CAM is a
fascinating application that brings together hardware and
software to enable intelligent vision capabilities in a
compact and affordable device. This technology finds
applications in various fields such as security, surveillance,
robotics, and IoT, where real-time object identification is
crucial for decision-making and automation. As the
capabilities of both hardware and software continue to
evolve, object detection on microcontrollers like the ESP32-
CAM holds great promise for enabling smarter and more
responsive embedded systems in the future.
