Azure Bot Framework with Cognitive Speech not working - javascript

I'm following this link where we can use the Speech recognition in the bot framework.
The default code is working with Option 2,
// // Option 2: Native browser speech (not supported by all browsers, no speech recognition priming support)
//
// Note that Chrome automatically blocks speech if the HTML file is loaded from disk. You can run a server locally
// or launch Chrome (close all the existing Chrome browsers) with the following option:
// chrome.exe --allow-file-access-from-files <sampleHtmlFile>
//
const speechOptions = {
speechRecognizer: new BotChat.Speech.BrowserSpeechRecognizer(),
speechSynthesizer: new BotChat.Speech.BrowserSpeechSynthesizer()
};
But when I tried to use cognitive services it's not working, meaning the mic is not going to listening mode.
This is the change I made,
// // Option 3: Cognitive Services speech recognition using API key (cross browser, speech priming support)
const speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: 'YOUR_COGNITIVE_SPEECH_API_KEY' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: 'YOUR_COGNITIVE_SPEECH_API_KEY',
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
})
};
Apart from the commenting and uncommenting i didn't do any thing. But still the code is working only with Option 2
Pls help me solve this

After some deep digging from my colleague, we found the issue.
The original code is using the javascript from https://cdn.botframework.com/botframework-webchat/latest/CognitiveServices.js
<div id="BotChatGoesHere"></div>
<!-- If you do not want to use Cognitive Services library, comment out the following line -->
<script src="https://cdn.botframework.com/botframework-webchat/latest/CognitiveServices.js"></script>
If we open that JS file, you can find a line like below where it's using bing speech url
Storage.Local.GetOrAdd("Host","wss://speech.platform.bing.com")}
Since bing speech is depricted we have to update this line into our own subsciption
Storage.Local.GetOrAdd("Host","wss://<region>.stt.speech.microsoft.com")}
Once we updated it's working fine now

Related

Google Auth OAuth 2.0 SvelteKit wierd behavior

I am using Google Auth OAuth 2.0 One Tap Sign and Sveltekit,
and I got some really weird behavior,
I followed this doc https://developers.google.com/identity/gsi/web/guides/overview with javascript
onMount(async () => {
const handleCredentialResponse = async (response) => {
console.log('Encoded JWT ID token: ' + response.credential);
};
google.accounts.id.initialize({
client_id: clientId,
callback: handleCredentialResponse,
});
google.accounts.id.renderButton(
document.getElementById('buttonDiv'),
{ theme: 'outline', size: 'large' } // customization attributes
);
google.accounts.id.prompt();
});
the code from the doc.
Sometimes it works everything goes well,
Sometimes I got
Uncaught (in promise) ReferenceError: google is not defined
And some mobile / mobile browsers I get
[GSI_LOGGER]: The given origin is not allowed for the given client ID.
but works on laptop
Hope someone can understand and help.
Do we have any way to check logs for investigation?
I got your code to successfully run without modification: https://google-signin-kit.vercel.app/ (This app is in "dev" mode so signin may only succeed with my Google account. If you clone my repo and set your Google client id, signin will work for you.)
I would check how my code is different from your code outside onMount(). For example, how did you include the Google javascript? I describe one major change below.
You also need to check your Google app settings. [GSI_LOGGER]: The given origin is not allowed... is fixed by adding the HTTPS app domain to your Google app "Authorized JavaScript origins." Non-HTTPS domains are not allowed. For example, these domains would not work:
https://google-signin-kit-leftium.vercel.app/ (Not added as Authorized JavaScript origin)
http://google-signin-kit.vercel.app/ (Not HTTPS, if Vercel did not automatically redirect to HTTPS)
Of course, raw IP addresses will not work, either.
localhost is a special exception, but not easy (impossible?) to access from mobile.
ReferenceError: google is not defined (sometimes) happens because onMount() runs before the Google script is loaded.
To get a consistent reproduction, USB debug/inspect Android Chrome and set "disable caching" and throttling to "Slow 3G." (I could not reproduce on desktop Chrome).
Fixed by removing defer async when including Google's script: <script src="https://accounts.google.com/gsi/client"></script>
It should also be possible to call a function after the script is loaded, but I got inconsistent results in SvelteKit. For some reason Kit doesn't always call the load function. Possibly a bug?
Ideally, the Google script should be imported, but I couldn't find a version of the Google script that was built for importing. You may be able to construct a library that you can import? Importing would compile the Google script code into your app, tree-shaking the unused portions. Thus saving a network request and bandwidth.
Another workaround is to have your onMount function check for the google variable. If google is still undefined, call your function again after a certain timeout.

How to connect to a SK1 sanei thermal printer using star webPRNT js SDK connected via USB?

How can i send print data to a Sanei SK1 printer which is connected to my computer via USB, using star-m webPRNT javascript SDK?
I could see this method,
var manager = new StarWebPrintExtManager();
function onConnect(url) {
manager.connect({url:url});
}
But what should be the url? how to form it?
Any help would be appreciated.
I've not used the ExtManager, but for the StarWebPrintTrader, we have
/**
* #return {StarWebPrintTrader}
*/
createStarPrintTrader() {
return new StarWebPrintTrader({ url: '//localhost:8001/StarWebPrnt/SendMessage' });
},
Our use case is to be able to print from a web app on a mobile device to a locally recognised bluetooth printer.
We generate a canvas and send an image of the canvas to the printer as we our print template is HTML. It works well for us.
Just in case you were not aware, the WebPRNT SDK is for use of web content within the Star WebPRNT browser, not a standard browser (Chrome, Firefox, etc.). Their browser presents access to a printer via localhost:8001 (that may be configurable - so check that - it's been a while since I've done any work with this SDK).
Whilst we don't support the driver in any way, https://github.com/digitickets/star-web-print may be of use to you at least to be able to include the package in a more convenient manner as well as get upgrades.
V1.7.0 is currently (2021-02-27) outstanding, but a PR exists.

Chrome extensions for silent print?

I have made a silent print web application that prints a PDF file. The key was to add JavaScript to the PDF file that silently print itself.
To do this I open the PDF with acrobat reader in chrome, that allow me to execute the script (with the proper permissions).
But as it was announced this solution won't work after chrome 45 because the npapi issue.
I guess a possible solution could be to use the recently release printProvider of chrome extensions.
Nevertheless I can't imagine how to fire any of the printProvider events.
So the question is: Is ok to think in chrome extensions to make a silent print web application, and how can I fire and handle a print job for an embedded PDF of a HTML Page.
Finally I reached an acceptable solution for this problem, as I couldn't find it out there, but read to many post with the same issue I will leave my solution here.
So first you need to add your printer to the Google Cloud Print and then you will need to add a proyect to the Google Developers Console
Then add this script and any time you need to print something execute the print() function. This method will print the document indicated in the content
The application will ask for your permission once to manage your printers.
function auth() {
gapi.auth.authorize({
'client_id': 'YOUR_GOOGLE_API_CLIENT_ID',
'scope': 'https://www.googleapis.com/auth/cloudprint',
'immediate': true
});
}
function print() {
var xhr = new XMLHttpRequest();
var q = new FormData()
q.append('xsrf', gapi.auth.getToken().access_token);
q.append('printerid', 'YOUR_GOOGLE_CLOUD_PRINTER_ID');
q.append('jobid', '');
q.append('title', 'silentPrintTest');
q.append('contentType', 'url');
q.append('content',"http://www.pdf995.com/samples/pdf.pdf");
q.append('ticket', '{ "version": "1.0", "print": {}}');
xhr.open('POST', 'https://www.google.com/cloudprint/submit');
xhr.setRequestHeader('Authorization', 'Bearer ' + gapi.auth.getToken().access_token);
xhr.onload = function () {
try {
var r = JSON.parse(xhr.responseText);
console.log(r.message)
} catch (e) {
console.log(xhr.responseText)
}
}
xhr.send(q)
}
window.addEventListener('load', auth);
<script src="https://apis.google.com/js/client.js"></script>
Anyway this script throw a 'Access-Control-Allow-Origin' error, even though this appears in the documentation... I couldn't make it work :(
Google APIs support requests and responses using Cross-origin Resource Sharing (CORS). You do not need to load the complete JavaScript client library to use CORS. If you want your application to access a user's personal information, however, it must still work with Google's OAuth 2.0 mechanism. To make this possible, Google provides the standalone auth client — a subset of the JavaScript client.
So to go throw this I had to install this chrome extension CORS. I'm sure that some one will improve this script to avoid this chrome extension.
You can register an Application to a URI Scheme to trigger the local application to print silently. The setting is pretty easy and straightforward. It's a seamless experience. I have posted the solution here with full example:
https://stackoverflow.com/a/37601807/409319
After the removal of npapi, I don't believe this is possible solely programmatically. The only current way I know to get chrome to print silently is using chrome kiosk mode, which is a flag (mode) you have to set when starting chrome.
Take a look at these SO posts:
Silent printing (direct) using KIOSK mode in Google Chrome
Running Chrome with extension in kiosk mode
This used to be possible using browser plugins (e.g. Java + NPAPI, ActiveX) but has been blacklisted by most browsers for several years.
If interested in modern solutions that use similar techniques, the architecture usually requires the following:
WebSocket, HTTP or Custom URI connection back to localhost
API that talks through web transport (JavaScript or custom URI scheme) to an app running locally.
A detail of projects (several of them are open source) that leverage these technologies are available here:
https://stackoverflow.com/a/28783269/3196753
Since the source code of these projects can vary (hundreds of lines to tens-of-thousands of lines), a code snippet would be too large unless a inquiring about a specific project's API.
Side note: Some technologies offer dedicated cloud resources, which add convenience at the expense of potential latency and privacy. At the time of writing this, the most popular "free" cloud solution -- Google Cloud Print -- is slated to be retired in December 2020.

Synthesize speech from text in web app

How can one synthesize speech in a web app?
Is there a way using the HTML5 Web Speech API?
For example, if I wanted to synthesize the sentence 'A quick brown fox jumps over the lazy dog', how could I do that without playing a pre-recorded file of someone reading exactly that sentence
As it is right now, the SpeechSynthesis features which are part of the Web Speech API Specification have not been implemented in any browser yet.
However, you can have a look at this Chrome extension.
EDIT:
It seems that the lastest Chrome Canary build might include the feature, however it only specifies that the feature has been started (http://www.chromestatus.com/features) and I was unable to find any more substantive information about it.
EDIT2:
As mentionned in the comments by #cdf, it seems that you can now play around with that feature by launching chrome with the --enable-speech-synthesis flag. Please see this post.
EDIT3:
This appears to be in Webkit now, but not on iOS at the moment. Not even in Chrome on iOS. Demo #BrandonAaskov
That code works both on Chrome and on Safari (taken from my app ttsreader):
if (!('speechSynthesis' in window)) {
// Synthesis not supported.
alert('Speech Synthesis is not supported by your browser. Switch to Chrome or Safari');
}
var msg = new SpeechSynthesisUtterance('hello world');
msg.volume = 1; // 0 to 1
msg.rate = 0.9; // 0.1 to 10
msg.pitch = 1; //0 to 2
msg.lang = "en-GB";
msg.text = "I will speak this out";
speechSynthesis.speak(msg);

Using the input 'speech' tag via javascript

Doing this in chrome:
<input id='speech-this' type='text' speech />
Creates an input tag with a little mic. Clicking on the mic does voice recognition, like android phones search.
My question is: Is it possible to do this without the <input> field? I mean, the ideal thing would be a javascript object that does something like:
var what_i_said = chrome.Speech.listen();
Or something like that.
Thanks!
Opera supports http://www.w3.org/TR/xhtml+voice/ (see http://dev.opera.com/articles/voice/).
You could look at the WAMI toolkit. WAMI toolkit is an interesting project from MIT - http://wami.csail.mit.edu/. In their own words "WAMI: Web-Accessible Multimodal Applications. WAMI is a simple way to add speech recognition capabilities to any web page." WAMI gives you a java applet that can run in your web page to perform audio capture for speech recognition.
There is actually way to do this with JavaScript and it's done with the Web Speech API. This allows you to quickly do voice recognition as well as speech synthesis.
Simplest example of Speech Synthesis:
var utterance = new SpeechSynthesisUtterance('Hello World');
window.speechSynthesis.speak(utterance);
Simplest example of Voice Recognition:
var recognition = new webkitSpeechRecognition();
recognition.onresult = function(event) {
console.log(event);
}
recognition.start();

Categories

Resources