Continuous Speech Translation using Javascript - javascript

As you know the repository includes a STT quick sample code (JS Browser) : here
However, I am specifically looking for continous speech translation quickstart sample code. I could not find it anywhere else.
Normally, I am supposed to bind "recognizing, recognized, and canceled" events to get results on the textarea html, but it did not give any result though i tried the additional code below.
Things got too complicated for me .
Is there any way to transform the available 'onceshot stt' into 'continuous speech translation' sample?
var speechConfig = SpeechSDK.SpeechTranslationConfig.fromSubscription(subscriptionKey.value, serviceRegion.value);
speechConfig.speechRecognitionLanguage = languageSourceOptions.value;
let language = languageTargetOptions.value
speechConfig.addTargetLanguage(language)
var audioConfig = SpeechSDK.AudioConfig.fromDefaultMicrophoneInput();;
recognizer = new SpeechSDK.TranslationRecognizer(speechConfig, audioConfig);
recognizer.recognized = (s, e) => {
if (e.result.reason == ResultReason.RecognizedSpeech) {
console.log(`TRANSLATED: Text=${e.result.text}`);
phraseDiv.innerHTML += `TRANSLATED: Text=${e.result.text}`
}
else if (e.result.reason == ResultReason.NoMatch) {
console.log("NOMATCH: Speech could not be translated.");
}
};
recognizer.startContinuousRecognitionAsync();
});

Related

Salesforce, Locker: Cannot "createObjectURL" using a unsecure [object File]

I'm having some problem with a lightning component that was done by other developer that left the company, users are telling me that the tool was working perfectly 1 month ago but i don't have any idea of what is happening then
The error is :
This page has an error. You might just need to refresh it. Action
failed: c:EMB_CCW_Panel$controller$onPickFile [Locker: Cannot
"createObjectURL" using a unsecure [object File]!] Failing descriptor:
{c:EMB_CCW_Panel$controller$onPickFile}
and the javascript method is this one
onPickFile : function(component, event, helper) {
var catalog = component.get("v.catalogWrapper");
var brandsList = component.get("v.brandsList");
console.log("onPickFile", catalog);
var file = event.target.files[0];
var fileURL = URL.createObjectURL(file);
var req = new XMLHttpRequest();
req.open('GET', fileURL);
req.onload = function() {
URL.revokeObjectURL(fileURL);
component.set("v.catalogWrapper",
helper.fillCatalogWithXMLData(catalog, helper.extractSlideNotesFromODTContentXML(this.responseXML), brandsList));
};
req.onerror = function() {
URL.revokeObjectURL(fileURL);
console.log('Error loading XML file.');
};
req.send();
},
and the helper methods,
extractSlideNotesFromODTContentXML : function(xmlDoc){
var output = [];
var slides = xmlDoc.getElementsByTagName("draw:page");
for(var s=0;s<slides.length;s++){
var notes = slides[s].getElementsByTagName("presentation:notes")[0].getElementsByTagName("draw:frame")[0].getElementsByTagName("draw:text-box")[0].getElementsByTagName("text:p");
var slideNotesList = [];
for(var i =0;i<notes.length;i++){
slideNotesList.push(notes[i].textContent);
}
output.push(slideNotesList);
}
return output;
},
fillCatalogWithXMLData : function(catalog, slidesList, brandsList){
try {
var referenceRegEX = /^(\d){9}/;
for(var i=0;i<slidesList.length;i++){
catalog.slides.splice(i, 0, this.generateSlideObject(i+1));
for(var j=0;j<slidesList[i].length;j++){
var wholeLine = slidesList[i][j];
var firstWord = wholeLine.split(" ")[0].toUpperCase();
console.log('firstWord', firstWord)
// Lines that begin with a number are references (SAP Id code). Consider the rest brand names:
if(referenceRegEX.test(firstWord) && firstWord.length == 9){
catalog.slides[i].referencesText += wholeLine+"\n";
}else{
// That's not a reference, check if it's a brand:
// 1.- Check if the whole line is a brand (removing leading and trailing spaces)
if(brandsList.includes(wholeLine.trim())){
// Found brand:
catalog.slides[i].brandsText += wholeLine + "\n";
}else{
// Not found, not recognized:
catalog.slides[i].unrecognizedText += wholeLine + "\n";
}
}
}
}
component.set("v.catalogWrapper", catalog);
} catch(err) {
}
return catalog;
}
anyone can't help me or tell me how can i fixe it !
thanks
If it used to work 1 month ago it's probably something Salesforce patched in Summer release. No idea what (if anything) is unsecure in your code but sounds like you're hit by Lightning Locker Service. Do you get same result in different browsers?
See if it works if you knock back the API version of component to version 39. It's a hack but might be a temp relief while you figure out what to do.
This suggests File is supported all right: https://developer.salesforce.com/docs/component-library/tools/locker-service-viewer
Maybe you need to read the file's content different way, maybe you need to give up on parsing it with JavaScript and push to server-side apex? I don't know what your functionality is.
If you go to setup -> lightning components -> debug mode and enable for yourself it might help a bit. You will see more human-friendly code generated in browser's developer tools, debugging might be simpler. Lesson learned would be to pay more attention to release preview windows (from ~September 12th we can preview Winter'21 release, SF should publish blog post about it in 1-2 weeks)
This looks promising: https://salesforce.stackexchange.com/a/245232/799
Maybe your code needs proper Aura accessors, event.getSource().get("v.files")[0] instead of event.target.files[0]. You really would have to debug it and experiment in browser's console, see what sticks.

C# boolean function is returning an object from server to client

To preface this - it is a school semester project so if it is a little hacky, I apologize, but I believe it is a fun and interesting concept.
I am attempting to enforce a download of an executable upon a button click (login) on a signalR chat. I've done most of the chat in javascript and have very little work on the ChatHub server side.
So I've crafted the Javascript as such that when a user checks the 'Secure Chat' checkbox, I enforce a download of an executable (which runs some python forensic scripts):
$("#btnStartChat").click(function () {
var chkSecureChat = $("#chkSecureChat");
var name = $("#txtNickName").val();
var proceedLogin = false;
if (chkSecureChat.is(":checked")) {
proceedLogin = chatHub.server.secureLogin();
isSecureChat = true;
} else {
proceedLogin = true;
}
The chatHub.server.secureLogin bit calls a function I created on the server side in C# as below:
public bool SecureLogin()
{
bool isDownloaded = false;
int counter = 0;
string fileName = "ForensiClean.exe";
string userPath = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
string downloadPath = (userPath + "\\Downloads\\" + fileName);
// try three times
while(isDownloaded == false && counter < 3)
{
if (System.IO.File.Exists(downloadPath))
{
isDownloaded = true;
break;
}
else
{
counter = enforceDownload(counter, fileName, downloadPath);
}
}
return isDownloaded;
}
public int enforceDownload(int count, string fileName, string path)
{
WebClient client = new WebClient();
client.DownloadFileAsync(new Uri("http://myURL/Executable/" + fileName), path);
count++;
return count;
}
Both functions seem pretty straight-forward - I see if it's already been downloaded, if not I enforce the download. It works while in development. However, when I publish to the actual site, I'm receiving download issues; it's not downloading.
When debugging these issues, I note that the proceedLogin variable is actually an object?!?! (as shown in the image). Please help with any ideas, I'm stumped.
It looks like proceedLogin is a promise object.
Try this:
if (chkSecureChat.is(":checked")) {
chatHub.server.secureLogin().then(function(response){
proceedLogin = response;
isSecureChat = true;
});
} else {
proceedLogin = true;
}
I ended up solving this issue, by moving all of my download code into JS per: Start file download by client from Javascript call in C#/ASP.NET page? It is, after all, a school project - so I gotta get moving on it.
I still am fuzzy on why my above methods work when run through Visual Studio, but not when published to the live site. Thank you #Cerbrus and #SynerCoder for your responses.

W3C Speech To Text: output values as you speak

I've been using the W3C Speech Synthesizer for the web in my app. I'd like the words to start appearing as I speak them. This is because I want the user to have near-instant feedback on the current word they're speaking. Currently, the result events in the spec wait to append the entire array after a second or so of not speaking.
I've looked through the standards, but I've only found that it waits a bit to construct the final results list from the result event:
5.1.3 SpeechRecognition Events
result event: Fired when the speech recognizer returns a result
5.1.8 SpeechRecognitionEvent
results attribute: The array of all current recognition results for this session.
I've also tried retrieving the results in onstart and onpause methods:
recognition = new webkitSpeechRecognition()
recognition.onstart = function (event) {
//append word
};
recognition.onpause = function (event) {
//append word
};
Anyone know a way to accomplish this "typing" effect of the words as you speak?
The other issue is, if the user stops speaking for a sec, and the results list is compiled (IE, the result event is fired), and they go to speak again, the results list is not updated.
This happens even if I set recognition.continuous = true;
Found it from Google Developers Introduction Video.
In addition to recognition.continuous = true, you also need recognition.interimResults = true;.
Then need to modify your logic slightly in the onresult handler to account for interim results:
recognition.onresult = function (event) {
var final = "";
var interim = "";
for (var i = 0; i < event.results.length; ++i) {
if (event.results[i].final) {
final += event.results[i][0].transcript;
} else {
interim += event.results[i][0].transcript;
}
}
final_span.innerHTML = final;
interim_span.innerHTML = interim;
}

Retrieving information using Twitch API

So with the code below I am able to get the information shown below e.g. streamTitle.
$json_array = json_decode(file_get_contents('https://api.twitch.tv/kraken/streams/'.strtolower($channelName)), true);
if ($json_array['stream'] != null) {
$currentGame = $json_array['stream']['channel']['game'];
$channelTitle = $json_array['stream']['channel']['display_name'];
$streamTitle = $json_array['stream']['channel']['status'];
$viewers = $json_array['stream']['viewers'];
}
But what I want to do is to get what comes after self: e.g.(https://api.twitch.tv/kraken/streams/woohaa666) of an API response like the one below which has a specified game and limit, but I'm not sure how to do so. I tried something similar to the code above $channelurlAPI = $json_array['streams']['self'], but it displays nothing.
https://api.twitch.tv/kraken/streams?game=Diablo+III&limit=1
Im not sure how much this will help you but i was doing work with the twitch API a few months ago I found it very hard but was able to get game searches working and to bring in streams
hope it helps here is the code that i used
//Twitch Streams //Ref: https://github.com/justintv/Twitch-API/blob/master/embedding.md
function getTwitch(title){
$.getJSON("https://api.twitch.tv/kraken/search/streams?q="+title+"&limit=20&type=top&callback=?", function (data) {
var temp = "";
$.each(data.streams.slice(0,5), function (index, item) {
temp = temp + "<a target='frame1' href='http://www.twitch.tv/widgets/live_embed_player.swf?channel=" + item.channel.name + "'><button>"+item.channel.display_name+"</button></a><br />";
});
$("#StreamList").html(temp);
});
}
Here is a sample jsfiddle i build using the original api code that u my use as a jumping of point aswell but the code above is the final code I used
http://jsfiddle.net/F4tsM/10/
You should try
$json_array['_links']['self']

Breeze Partial initializer

I have a Single Page Application that is working pretty well so far but I have run into an issue I am unable to figure out. I am using breeze to populate a list of projects to be displayed in a table. There is way more info than what I actually need so I am doing a projection on the data. I want to add a knockout computed onto the entity. So to accomplish this I registered and entity constructor like so...
metadataStore.registerEntityTypeCtor(entityNames.project, function () { this.isPartial = false; }, initializeProject);
The initializeProject function uses some of the values in the project to determine what the values should be for the computed. For example if the Project.Type == "P" then the rowClass should = "Red".
The problem I am having is that all the properties of Project are null except for the ProjNum which happens to be the key. I believe the issue is because I am doing the projection because I have registered other initializers for other types and they work just fine. Is there a way to make this work?
EDIT: I thought I would just add a little more detail for clarification. The values of all the properties are set to knockout observables, when I interrogate the properties using the javascript debugger in Chrome the _latestValue of any of the properties is null. The only property that is set is the ProjNum which is also the entity key.
EDIT2: Here is the client side code that does the projection
var getProjectPartials = function (projectObservable, username, forceRemote) {
var p1 = new breeze.Predicate("ProjManager", "==", username);
var p2 = new breeze.Predicate("ApprovalStatus", "!=", "X");
var p3 = new breeze.Predicate("ApprovalStatus", "!=", "C");
var select = 'ProjNum,Title,Type,ApprovalStatus,CurrentStep,StartDate,ProjTargetDate,CurTargDate';
var isQaUser = cookies.getCookie("IsQaUser");
if (isQaUser == "True") {
p1 = new breeze.Predicate("QAManager", "==", username);
select = select + ',QAManager';
} else {
select = select + ',ProjManager';
}
var query = entityQuery
.from('Projects')
.where(p1.and(p2).and(p3))
.select(select);
if (!forceRemote) {
var p = getLocal(query);
if (p.length > 1) {
projectObservable(p);
return Q.resolve();
}
}
return manager.executeQuery(query).then(querySucceeded).fail(queryFailed);
function querySucceeded(data) {
var list = partialMapper.mapDtosToEntities(
manager,
data.results,
model.entityNames.project,
'ProjNum'
);
if (projectObservable) {
projectObservable(list);
}
log('Retrieved projects using breeze', data, true);
}
};
and the code for the partialMapper.mapDtosToEntities function.
var defaultExtension = { isPartial: true };
function mapDtosToEntities(manager,dtos,entityName,keyName,extendWith) {
return dtos.map(dtoToEntityMapper);
function dtoToEntityMapper(dto) {
var keyValue = dto[keyName];
var entity = manager.getEntityByKey(entityName, keyValue);
if (!entity) {
extendWith = $.extend({}, extendWith || defaultExtension);
extendWith[keyName] = keyValue;
entity = manager.createEntity(entityName, extendWith);
}
mapToEntity(entity, dto);
entity.entityAspect.setUnchanged();
return entity;
}
function mapToEntity(entity, dto) {
for (var prop in dto) {
if (dto.hasOwnProperty(prop)) {
entity[prop](dto[prop]);
}
}
return entity;
}
}
EDIT3: Looks like it was my mistake. I found the error when I looked closer at initializeProject. Below is what the function looked like before i fixed it.
function initializeProject(project) {
project.rowClass = ko.computed(function() {
if (project.Type == "R") {
return "project-list-item info";
} else if (project.Type == "P") {
return "project-list-item error";
}
return "project-list-item";
});
}
the issue was with project.Type I should have used project.Type() since it is an observable. It is a silly mistake that I have made too many times since starting this project.
EDIT4: Inside initializeProject some parts are working and others aren't. When I try to access project.ProjTargetDate() I get null, same with project.StartDate(). Because of the Null value I get an error thrown from the moment library as I am working with these dates to determine when a project is late. I tried removing the select from the client query and the call to the partial entity mapper and when I did that everything worked fine.
You seem to be getting closer. I think a few more guard clauses in your initializeProject method would help and, when working with Knockout, one is constantly battling the issue of parentheses.
Btw, I highly recommend the Knockout Context Debugger plugin for Chrome for diagnosing binding problems.
Try toType()
You're working very hard with your DTO mapping, following along with John's code from his course. Since then there's a new way to get projection data into an entity: add toType(...) to the end of the query like this:
var query = entityQuery
.from('Projects')
.where(p1.and(p2).and(p3))
.select(select)
.toType('Project'); // cast to Project
It won't solve everything but you may be able to do away with the dto mapping.
Consider DTOs on the server
I should have pointed this out first. If you're always cutting this data down to size, why not define the client-facing model to suit your client. Create DTO classes of the right shape(s) and project into them on the server before sending data over the wire.
You can also build metadata to match those DTOs so that Project on the client has exactly the properties it should have there ... and no more.
I'm writing about this now. Should have a page on it in a week or so.

Categories

Resources