8000 [PC-1828] Speech recognition tutorial check and revamp (#2151) · CoolCoderBhavya/docs-content@115cdf7 · GitHub
[go: up one dir, main page]
Skip to content

Commit 115cdf7

Browse files
mcmchrismcmchris
authored
[PC-1828] Speech recognition tutorial check and revamp (arduino#2151)
* Nicla Vision Added to requirements * chris modification * finished
1 parent b3793b5 commit 115cdf7

File tree

1 file changed

+132
-9
lines changed
  • content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine

1 file changed

+132
-9
lines changed

‎content/hardware/04.pro/shields/portenta-vision-shield/tutorials/speech-recognition-engine/content.md

Lines changed: 132 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ In case you are interested in unlocking the full potential of the tool, the tuto
5757
To use the Arduino Speech Recognition Engine, you will need one of the following boards:
5858

5959
- [Arduino Portenta H7 (any variant)](https://store.arduino.cc/portenta-h7) + Portenta Vision Shield ([LoRa](https://store.arduino.cc/products/arduino-portenta-vision-shield-lora%C2%AE) or [Ethernet](https://store.arduino.cc/products/arduino-portenta-vision-shield-ethernet))
60+
- [Arduino Nicla Vision](https://store.arduino.cc/products/nicla-vision)
6061
- [Arduino Nano 33 BLE Sense Rev 1](https://store.arduino.cc/products/arduino-nano-33-ble-sense)
6162
- [Arduino Nano 33 BLE Sense Rev 2](https://store.arduino.cc/products/nano-33-ble-sense-rev2)
6263
- [Arduino Nano RP2040](https://store.arduino.cc/products/arduino-nano-rp2040-connect)
@@ -133,7 +134,7 @@ Once everything is ready, click on the **submit** button to get your license, it
133134
};
134135
```
135136
136-
**Remember to replace the g_lpdwLicense values with your license ones.**
137+
***Remember to replace the `g_lpdwLicense` values with your license ones.***
137138
138139
Now switch back to the `VoiceRecognition` tab and upload the sketch.
139140
@@ -149,7 +150,7 @@ Please note that the sketch and license that you are using are a "demo" version
149150
150151
### Customized Commands
151152
152-
To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the _Cyberon Model Configuration_ webpage to create a new project with custom voice commands.
153+
To expand the features and human interaction of your projects, you can create your own voice commands with the Arduino Speech Recognition Engine. In this section, you will learn about how to use the [Cyberon Model Configuration](https://tool.cyberon.com.tw/ArduinoDSpotterAuth/CTMain.php) webpage to create a new project with custom voice commands.
153154
154155
***These steps are compatible with the trial license, used in this tutorial, and the paid license***
155156
@@ -169,18 +170,18 @@ Click next, you will see a new page to:
169170
170171
***Warning: each project is bound to a single Arduino board with the declared Serial Number. If you would like to use the Arduino Speech Recognition Engine on another Arduino board, you need to create a new project from scratch and assign it to the new Serial Number.***
171172
172-
To create a new project first you need to select the desired language for the speech recognition. Once is set, click **create**.
173+
To create a new project first you need to select the desired language for the speech recognition. Once is set, click **Create**.
173174
174175
![Cyberon, New Project](assets/newProject.png)
175176
176177
Now you need to configure the following steps to create a new trigger:
177178
178179
* Create the **Input Trigger word**, for example "Hey Arduino".
179-
The **Input Trigger word** will trigger the device, to let the board know that you are going to say a command after that.
180-
![Cyberon Adding the Input trigger](assets/newProjectTrigger.png)
180+
The **Input Trigger word** will trigger the device to let the board know that you are going to say a command after that.
181+
![Cyberon Adding the Input trigger](assets/newProjectTrigger.png)
181182
* Add the **Command** list.
182-
These commands will be used to do tasks on your sketch. For example, if you have a command which is "Play", later you will be able to get that command and proceed with some job inside the sketch easily.
183-
![Cyberon Adding the Command list](assets/newProjectCommands.png)
183+
These commands will be used to do tasks on your sketch. For example, if you have a command which is `share`, later you will be able to get that command and proceed with some job inside the sketch easily.
184+
![Cyberon Adding the Command list](assets/newProjectCommands.png)
184185
* The next step is just to confirm that the data written is correct. Click on **Next** to finish.
185186
186187
On the next page you will see all the configurations already set. Check it out to confirm that everything is correct. In case something is wrong, click on **Back** to fix it.
@@ -201,9 +202,9 @@ On your sketch directory, paste the files you have got in your e-mail inbox befo
201202
* `CybLicense_<id>.h`
202203
* `Model_<id>.h`
203204
204-
Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following `#include`
205+
Now to implement the **Input Trigger Command** and the **Command List** open the sketch and change the following headers with the downloaded ones:
205206
206-
```cpp
207+
```arduino
207208
...
208209
209210
#include "CybLicense.h" -> #include "CybLicense_<id>.h"
@@ -220,6 +221,128 @@ Now to implement the **Input Trigger Command** and the **Command List** open the
220221

221222
At this point, the project is set to be used. Upload the sketch and open the Serial Monitor. Now you can test your new **Input Trigger Word** and the **Command** list that you have created. Pronounce the new trigger words out loud, you will see the recognized words on the **Serial Monitor**.
222223

224+
#### Modify the Example Codes
225+
226+
If you want to execute custom actions with the **trigger** and **commands** you defined, you can do it inside the `VRCallback` function.
227+
228+
First you need to know your trigger and commands IDs. You can find them listed in your Serial Monitor, in this case are the following:
229+
230+
| **Keyword** | **ID** |
231+
| :---------: | :----: |
232+
| Hey Arduino | 100 |
233+
| read | 10000 |
234+
| upload | 10001 |
235+
| share | 10002 |
236+
237+
Here is the `VRCallback` function modified so you can easily give functionality to the commands detection:
238+
239+
```arduino
240+
// Callback function for VR engine
241+
void VRCallback(int nFlag, int nID, int nScore, int nSG, int nEnergy)
242+
{
243+
if (nFlag==DSpotterSDKHL::InitSuccess)
244+
{
245+
//ToDo
246+
}
247+
else if (nFlag==DSpotterSDKHL::GetResult)
248+
{
249+
/*
250+
When getting an recognition result,
251+
the following index and scores are also return to the VRCallback function:
252+
nID The result command id
253+
nScore nScore is used to evaluate how good or bad the result is.
254+
The higher the score, the more similar the voice and the result command are.
255+
nSG nSG is the gap between the voice and non-command (Silence/Garbage) models.
256+
The higher the score, the less similar the voice and non-command (Silence/Garbage) models are.
257+
nEnergy nEnergy is the voice energy level.
258+
The higher the score, the louder the voice.
259+
*/
260+
//ToDo
261+
switch(nID)
262+
{
263+
case 100:
264+
Serial.println("The Trigger was heard");
265+
// Add your own code here
266+
break;
267+
case 10000:
268+
Serial.println("The Command -read- was heard");
269+
// Add your own code here
270+
break;
271+
case 10001:
272+
Serial.println("The Command -upload- was heard");
273+
// Add your own code here
274+
break;
275+
case 10002:
276+
Serial.println("The Command -share- was heard");
277+
// Add your own code here
278+
break;
279+
default:
280+
break;
281+
}
282+
283+
}
284+
else if (nFlag==DSpotterSDKHL::ChangeStage)
285+
{
286+
switch(nID)
287+
{
288+
case DSpotterSDKHL::TriggerStage:
289+
LED_RGB_Off();
290+
LED_BUILTIN_Off();
291+
break;
292+
case DSpotterSDKHL::CommandStage:
293+
LED_BUILTIN_On();
294+
break;
295+
default:
296+
break;
297+
}
298+
}
299+
else if (nFlag==DSpotterSDKHL::GetError)
300+
{
301+
if (nID == DSpotterSDKHL::LicenseFailed)
302+
{
303+
//Serial.print("DSpotter license failed! The serial number of your device is ");
304+
//Serial.println(DSpotterSDKHL::GetSerialNumber());
305+
}
306+
g_oDSpotterSDKHL.Release();
307+
while(1);//hang loop
308+
}
309+
else if (nFlag == DSpotterSDKHL::LostRecordFrame)
310+
{
311+
//ToDo
312+
}
313+
}
314+
```
315+
We added a `switch...case` that evaluates the `nID` variable and compares it with the different IDs.
316+
Inside each case add your custom code to be executed with the trigger or commands.
317+
318+
```arduino
319+
...
320+
switch(nID)
321+
{
322+
case 100:
323+
Serial.println("The Trigger was heard");
324+
// Add your own code here
325+
break;
326+
case 10000:
327+
Serial.println("The Command -read- was heard");
328+
// Add your own code here
329+
break;
330+
case 10001:
331+
Serial.println("The Command -upload- was heard");
332+
// Add your own code here
333+
break;
334+
case 10002:
335+
Serial.println("The Command -share- was heard");
336+
// Add your own code here
337+
break;
338+
default:
339+
break;
340+
}
341+
...
342+
343+
```
344+
345+
223346
#### Unlock Limitations (License)
224347

225348
In case you want to further customize your model or add additional commands or trigger words, we have the right licenses for you:

0 commit comments

Comments
 (0)
0