Quality of audio pick-up with NDP120

Hello. We are testing the Nicla Voice to validate if the NDP120 is suitable for our designs. After some struggle (4 days) have the Impulse Edge RC demo working. Our fixes have been posted on their user forum.

The demo works but the single microphone solution is weak and unlike the impressive demo in the NDP120 videos. How can we achieve the same quality of support like barge-in, noise cancellation so the KWS can be detected while there is background noise from a distance? We are ok to deploy with 2 microphones.

Is there a reference design we can follow with 2 microphones? Is the audio chain IP included with the purchase of the silicon or is an option?

Can the lower cost NDP115 be used with the same results?

We wish to build our own custom KWS with Edge Impulse but with higher audio pick-up than the RC demo.

Also, where is the code that contains the trigger for the RC demo? Can we customize the triggers so that the output string and port action for the KWS led is to our configuration? There are many complex and canned steps to this solution. Hoping you can help. Thanks.

Hi. Good to see you have had your demo running on Nicla voice.
Please share the the link you mentioned about porting fixes in the Arduino Nicla forum, that will help us understand the issue.

Nicla voice schematics allow external microphone to be connected, One can design 2 mic support using this mechanism. However there will be some software changes that needs to be carried out specific to the design. We would like to know more about your requirements for 2 mics on Nicla and business volumes to be able to provide guidance. Please share the details to info@syntiant.com, so we can understand and respond appropriately.

NDP115 uses the Syntiant Core2 design, which is the same as NDP120, but with reduced feature & size, With your detailed requirement, we would be able to review and provide recommendations about the choice of chip that will meet your requirements.

Nicla voice code can be tweaked to customize the GPIO configurations, triggers and handlers. Nicla Voice examples sources are available in - https://github.com/arduino/ArduinoCore-mbed/tree/main/libraries/NDP/examples
Thanks.

Sample code to trigger GPIO

#include “NDP.h”

//const bool lowestPower = true;
const bool lowestPower = false;

void checkmatch(char* label)
{
if(strcmp((char *)label,“NN0:go”) == 0)
{
ledGreenOn();
digitalWrite(4,HIGH);
}

else if(strcmp((char *)label,“NN0:stop”) == 0)
{
ledBlueOn();
digitalWrite(4,LOW);
}

if (!lowestPower) {
Serial.println(label);

}

}

void ledMagentaOn() {

nicla::leds.begin();
nicla::leds.setColor(magenta);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledYellowOn() {

nicla::leds.begin();
nicla::leds.setColor(yellow);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledCyanOn() {

nicla::leds.begin();
nicla::leds.setColor(cyan);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledWhiteOn() {

nicla::leds.begin();
nicla::leds.setColor(white);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledBlueOn() {

nicla::leds.begin();
nicla::leds.setColor(blue);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledGreenOn() {
nicla::leds.begin();
nicla::leds.setColor(green);
delay(200);
nicla::leds.setColor(off);
nicla::leds.end();
}

void ledRedBlink() {
while (1) {
nicla::leds.begin();
nicla::leds.setColor(red);
delay(200);
nicla::leds.setColor(off);
delay(200);
nicla::leds.end();
}
}

void setup() {
pinMode(4,OUTPUT);
Serial.begin(115200);
nicla::begin();
nicla::disableLDO();
nicla::leds.begin();

NDP.onError(ledRedBlink);
NDP.onMatch(checkmatch);
NDP.onEvent(ledGreenOn);
Serial.println(“Loading synpackages”);
NDP.begin(“mcu_fw_120_v91.synpkg”);
NDP.load(“dsp_firmware_v91.synpkg”);
NDP.load(“ei_model.synpkg”);
Serial.println(“packages loaded”);
NDP.getInfo();
Serial.println(“Configure mic”);
NDP.turnOnMicrophone();
NDP.interrupts();

// For maximum low power; please note that it’s impossible to print after calling these functions
nicla::leds.end();
if (lowestPower) {
NRF_UART0->ENABLE = 0;
}
//NDP.turnOffMicrophone();
}

void loop() {
uint8_t command = 0xFF;

while (Serial.available()) {
command = Serial.read();
if (command == ‘f’) {
Serial.println(“Interrupts disabled”);
NDP.noInterrupts();
} else if (command == ‘o’) {
Serial.println(“Interrupts enabled”);
NDP.interrupts();
}
}
//Serial.println(“in the loop”);
delay(100);
}

Thanks for the follow up. Openly a bit disappointed with the results as the quality of the demo does not follow the numerous videos that are public. Is there not a factory software demo to build the advertised KWS with background noise cancellation? The demo was impressive that shows the speaker’s lips moving from a distance and the radio is blasting nearby yet the KWS works. We wish to deploy the same but have not seen that operation to date with the purchased tools. Our development is frozen on using this device. The ESP32 module claims the same and offers GitHub examples to test. We are about to validate if their claim is true. Interest is in an offline battery powered KWS module for consumer space. To date, have not seen a vendor that can supply an out of the box working solution as noted. Each sells a rather costly demo board that demands extra extra fees for code licenses from Sensory or Edge or xx audio IP. That has been our experience to date. To us, we do not agree that we should have to justify the volumes of a business model to buy your hardware. If the product worked as advertised with available IP without further licenses, we would have been building our Syntiant based widgets by now. Factory should release a reference design that is proven with 2-4 microphones with barge in and background noise cancellation with KWS demo. Offer a clear tutorial on how to expand the demo for customized KWS.

Hello @Mon2
Thank you for reaching out to us.
NDP120 is capable of providing Far Field solutions with Noise cancellations, Barge in support using 2 microphones.
This solution is available with our Eval board as reference and there are existing customer designs with NDP120 chip using these features.
The demo video for the same is available in https://www.youtube.com/watch?v=w7e3zrG7_jA. I presume this is what you have viewed and mentioned in your message.
The out of box Nicla voice does not have 2 microphone and the right firmwares to demonstrate these features. However this is something that can be custom build for specific use cases. For any such custom modifications, we will have to assess the business fitment. I would request you to reach out help@syntiant.com with the necessary details like Organization name and the intended use case. We would then be able to suggest the best way forward. I will be happy to assist you in the evaluation and creating product.

Thank you. Very likely interested to proceed but the shared email is bouncing back. Would you please confirm the details? Would like to request the details of the demo board and the IP required to run like the YT video. We have a mutual NDA in place.

@Mon2 Kindly try the given email: info@syntiant.com

Just resent. Wish to further ask if the demo is dependent on the power hungry rpi platform or can we deploy with the ndp120 and a local SPI bus master or local firmware flash to load your device? Need a low power solution.

No reply from factory from numerous emails. Lots of broken links on the website.

Is the NDP120 kit in the witness protection program? Unable to locate any details on this EVB,

Hello @Mon2 , Sorry for the inconvenience. May I know on which email id you are trying to reach out?

Thanks agiri.

  1. we have had a mutual NDA in place with Syntiant for a while.

  2. Sent an email to Chris Alger (FAE @ Syntiant; we have known him since the many years we have supported XMOS on their forum when Chris worked for XMOS)

  3. Sent an email to James Fife @ Syntiant

Not sure if either is still employed at Syntiant. It is like pulling teeth to get support from the factory yet they want to sell container loads of this silicon that has yet to be proven. Unfortunately a common trend in the silicon field.

  1. The secret for the far field IP is inside the binary uploaded into the HIFI 3 DSP (which is now considered to be very old) by Cadence. We have pinged others that claim to know the HIFI 3 DSP but no reply as of this writing. Guessing that they will outsource the help to answer our raised questions.

  2. Based on the feedback from other Syntiant developers, they have yet to see the far field audio pickup function as shown in the YT videos.

  3. We are in chats with numerous NN chip vendors in China for a resolution to our concept.

  4. If there is indeed a binary for the DSP to allow for quality audio pickup with dual microphones (far field with background noise) like the YT video - do share asap. Otherwise we have other options we are able to explore at a fraction of the cost of the NDP120.