Page 183 - Kaleidoscope Academic Conference Proceedings 2021

Connecting physical and virtual worlds

scope of this research. The test set data is data that the model display. The Arduino and OLED are connected via I2C pins.
has never seen before and may be used to evaluate whether Arduino uses these pins to display the binary image of the
the model works on the unseen data. The following figure mosquito on the OLED display as illustrated schematically
illustrates the model test results, i.e., model’s performance on below (Figure 11).
the testing data (Figure 9).

Figure 9 – Model testing results

2.4 Model deployment
The model may be deployed to Arduino Nano 33 BLE Figure 11 – Circuit schematic
Sense once the impulse has been created, trained, and
evaluated. This allows the model to function without
The flow of the code and ML model on the Arduino board is
an Internet connection, with little latency and battery
described in Figure 12. The first block, or input, is provided
consumption. Edge Impulse can compile the whole impulse,
by the Arduino board’s on–board microphone. When the
including the MFE algorithm, neural network weights, and
board is turned on, it is configured to record the sound
classification code, into a single Arduino library[11]. Edge
constantly. The spectrogram is represented by the second
Impulse has previously calculated on–device performance
block. The spectrogram, or equivalent pictorial data, is
for a low–power micro–controller (Cortex–M4F @ 80MHz),
transformed from the input audio stream and given to the
with inferencing time of 337 milliseconds, peak RAM
model through the feature extraction block. The feature
consumption of 9.2 kb, and flash usage of 43.4 kb to evaluate
extractor extracts features from spectrograms or pictures that
one second of data, as shown in the figure 10.
have been given to it. It can currently extract 680 features
from the spectrograms of all audio recordings.
Figure 10 – On–device performance
Because the MFCC/MFE/spectrogram computation takes a
long time (500ms for a 1000ms window on some targets),
only one classification per second is achievable (some more
time is required to call the classifier).That is not possible since
there is a risk of missing events that happen half in one second
and half in the next. To counteract this, the spectrogram might
be computed in smaller slices (for example, 250ms slices),
stitched together, and then classified. Because the time spent
on the spectrogram slice is about 125ms + the time to classify
(say 50ms), each 250ms window can now be categorized, and
the chance of missing an event is now nil. Figure 12 – Diagram of the model’s flow on Arduino

3. HARDWARE IMPLEMENTATION
These features are then input into the pre–trained model,
which processes them via six layers: a 40–column reshape
The hardware implementation is done using the Arduino
layer, an 8–filter 2D convolution layer, a dropout layer with a
Nano 33 BLE Sense development board and a 0.9 inch OLED

– 121 –