CSV Parser adding columns on empty values - javascript

So I having problems with my csv-parser that is reading values where it adds a column on empty cells from a csv file. It gives an error of
column header mismatch expected: 17 columns got: 18
For now I have to go in the csv file and backspace a comma to match the columns. I know is a parse csv issue, has anyone encounter this? below is my csv code.
function readStream () {
let stream = fs.createReadStream("accounts.csv");
fast
.fromStream(stream, {
headers: true
})
.on("data" , fetchYelp, fetchWhitePages, fetchGooglePlace, writeStream
)
.on("end", function () {
console.log("Done Reading");
});
}
readStream();

Could you try using the discardUnmappedColumns option, e.g. ? That works for me!
function readStream () {
let stream = fs.createReadStream("accounts.csv");
fast
.fromStream(stream, {
headers: true,
discardUnmappedColumns: true
})
.on("data" , fetchYelp, fetchWhitePages, fetchGooglePlace, writeStream ) {
})
.on("end", function () {
console.log("Done Reading");
});
}
readStream();

Related

Excel file contains {"sharedString":0} instead of actual value

I am trying to read from a excel file, manipulate and create another excel file from it, i am using stream support for this. Most of it is working fine but i see resultant excel file containing {"sharedString":0} instead of actual values.
Below is my relevant code
let ws = fs.createWriteStream(fpath);
const workbook = new ExcelJS.stream.xlsx.WorkbookWriter({stream: ws, useStyles: true, useSharedStrings: true});
const myworksheet = workbook.addWorksheet('sheet1');
const workbookReader = new ExcelJS.stream.xlsx.WorkbookReader(sheet.path, options);
workbookReader.read();
workbookReader.on('worksheet', worksheet => {
worksheet.on('row', row => {
myworksheet.addRow(row.values);
});
});
workbookReader.on('shared-strings', sharedString => {
console.log('not coming here');
});
workbookReader.on('end', async () => {
console.log('processing done...');
await workbook.commit();
});
Please see the attached file for your reference.
Any help on how to fix this will be really great, Thanks.
Once i created the WorkbookReader with below options
const options = {
sharedStrings: 'cache',
hyperlinks: 'cache',
worksheets: 'emit',
styles: 'cache',
};
it worked!

Glue42 - Register for updates in excel

I want to link an application to validate updates made in Excel. I am trying to use Glue42 interop methods. Does anyone have a code sample?
I have successfully linked two apps, but not excel
I change a cell in excel and my external app validates this can be changed.
I believe you missed the Glue for Office documentation. There are two sections you can go to from the home page: one for general programming and one for the Office connectors.
Anyways, up to the question, you need first to obtain a reference to a Glue4Office object with code, similar to the one below:
const config = {
// ...,
excel: true // enable Excel integration
}
Glue4Office(config)
.then(g4o => {
const excel = g4o.excel
// interact with Excel
})
.catch(console.error)
Then, once you open a sheet and get a reference to it, you can subscribe to its onChanged event, where you can call the errorCallback argument. Here is an example:
excel.openSheet(config)
.then(sheet => {
sheet.onChanged((data, errorCallback, doneCallback) => {
// process changes here
// errorCallback(...) - call if there are validation errors
// doneCallback() - call if not doing validations or everything's OK
})
})
sheet.onChanged((data, errorCallback, doneCallback) => {
// ...
const errors = []
data.reduce(
(errors, rowData, rowIndex) => {
if (!rowData['firstName']) {
errors.push({
row: rowIndex + 1,
column: 'firstName',
description: 'First name is mandatory'
})
}
if (Number(rowData['subscriptionMonths']) < 6) {
errors.push({
row: rowIndex + 1,
column: 1, // <- note column index this time
description: 'Subscription period must be at least 6 months',
text: '6' // <- replacing what the user typed
})
}
},
[])
// if during the validation pass we've accumulated any errors
// we need to call the errorCallback, otherwise the doneCallback
if (errors.length > 0) {
errorCallback(errors)
}
else {
doneCallback()
}
})
There are convenient declarative approaches as well.

Get contents of a zip file

This URL below points to a zip file which contains a file called bundlesizes.json. I am trying to read the contents of that json file within my React application (no node server/backend involved)
https://dev.azure.com/uifabric/cd9e4e13-b8db-429a-9c21-499bf1c98639/_apis/build/builds/8838/artifacts?artifactName=drop&api-version=4.1&%24format=zip
I was able to get the contents of the zip file by doing the following
const url =
'https://dev.azure.com/uifabric/cd9e4e13-b8db-429a-9c21-499bf1c98639/_apis/build/builds/8838/artifacts?artifactName=drop&api-version=4.1&%24format=zip';
const response = await Axios({
url,
method: 'GET',
responseType: 'stream'
});
console.log(response.data);
This emits the zip file (non-ascii characters). However, I am looking to read the contents of the bundlesizes.json file within it.
For that I looked up jszip and tried the following,
var zip = new JSZip();
zip.createReader(
new zip.BlobReader(response.data),
function(reader: any) {
// get all entries from the zip
reader.getEntries(function(entries: any) {
if (entries.length) {
// get first entry content as text
entries[0].getData(
new zip.TextWriter(),
function(text: any) {
// text contains the entry data as a String
console.log(text);
// close the zip reader
reader.close(function() {
// onclose callback
});
},
function(current: any, total: any) {
// onprogress callback
console.log(current);
console.log(total);
}
);
}
});
},
function(error: any) {
// onerror callback
console.log(error);
}
);
However, this does not work for me, and errors out.
This is the error I receive
How can I read the contents of the file within the zip within my React application by using Javascript/Typescript?

How can we send messages from the main process to renderer process in Electron

I'm playing with electron for the first time. Trying to create a text editor
In render I'm sending a message to indicated the content has changed and needs saving:
document.getElementById('content').onkeyup = e => {
ipcRenderer.send('SAVE_NEEDED', {
content: e.target.innerHTML,
fileDir
})
}
Then ipcMain receives it no problem. On the menu I have this:
{
label: 'Save',
click: _ => {
saveFile(message)
// trying:
// ipcMain.send('SAVED', 'File Saved')
},
accelerator: 'cmd+S', // shortcut
}
So that the user knows the files has have. But that doesn't seem to work. Is there any other way to do this? I would have thought "save" would be a pre-created role (sort of)
To send a message back to the renderer you would use:
win.webContents.send('asynchronous-message', {'SAVED': 'File Saved'});
And receive it like this:
ipcRenderer.on('asynchronous-message', function (evt, message) {
console.log(message); // Returns: {'SAVED': 'File Saved'}
});
Where asynchronous-message is simply the channel you're sending it to. It can literally be anything.
webContents.send Docs
alternatively - when you want to respond to an event received from renderer process you can do something like this:
ipcMain.on("eventFromRenderer", (event) => {
event.sender.send("eventFromMain", someReply);
}
Source: https://electronjs.org/docs/api/ipc-main
Here is what I tinkered with (using the new way of doing with contextBridge), my use was simply to have a menuItem call a navigation event in my React code:
// preload.js
const exposedAPI = {
// `(customData: string) => void` is just the typing here
onMenuNav: (cb: (customData: string) => void) => {
// Deliberately strip event as it includes `sender` (note: Not sure about that, I partly pasted it from somewhere)
// Note: The first argument is always event, but you can have as many arguments as you like, one is enough for me.
ipcRenderer.on('your-event', (event, customData) => cb(customData));
}
};
contextBridge.exposeInMainWorld("electron", exposedAPI);
// typing for curious peoples
onMenuNav(cb: ((customData: string) => void)): void;
// renderer.tsx
// Call it in the renderer process, in my case it is called once at the end of renderer.tsx.
window.electron.onMenuNav(customData => {
console.log(customData); // 'something'
});
// in main process
const customData = 'something';
mainWindow.webContents.send('your-event', customData);
Old question but I found new good solution;
// on main process index.js
ipcMain.on('event-name', (event, data) => {
const value = 'return value';
event.reply(data.waitingEventName, value);
});
// on render frame index.html
const send =(callback)=>{
const waitingEventName = 'event-name-reply';
ipcRenderer.once(waitingEventName, (event, data) => {
callback(data);
});
ipcRenderer.send('event-name', {waitingEventName});
};
send((value)=>{
console.log(value);
});
From #raksa answer
You can send a request to main and get a response, rather than sending a response that might not be delivered.
Use this sample
//render process
ipcRenderer.send('hello', ['one', 'two', 'three']);
ipcRenderer.once('nice', (e, data) => {
console.log(data); //['one','two','three']
})
//main process
ipcMain.on('hello', (e, data) => {
e.reply('nice', data)
})

Google Speech API - Server Unavailable Error for long audio files

I'm using Google's nodejs-speech package to use the longRunningRecognize endpoint/function in Google's Speech API.
I've used both v1 and v1p1beta, and run into an error with longer files. (48 mins is as long as I've tried, and 15 mins causes the same problem, though 3 mins does not). I've tried both the promise pattern and separating the request into two parts -- one to start the longRunningRecognize process, and the other to check on results after waiting. The error is shown below the code samples for both.
Example promise version of request:
import speech from '#google-cloud/speech';
const client = new speech.v1p1beta1.SpeechClient();
const audio = {
uri: 'gs://my-bucket/file.m4a'
};
const config = {
encoding: 'AMR_WB',
sampleRateHertz: 16000,
languageCode: 'en-US',
enableWordTimeOffsets: true,
enableSpeakerDiarization: true
};
const request = {
audio,
config
};
client.longRunningRecognize(request)
.then(data => {
const operation = data[0];
return operation.promise();
})
.then(data => {
const response = data[0];
const results = response.results;
const transcription = results
.filter(result => result.alternatives)
.map(result => result.alternatives[0].transcript)
.join('\n');
console.log(transcription);
})
.catch(error => {
console.error(error);
});
(I've since closed the tab with the results, but I think this returned an error object that just said { error: { code: 13 } }, which matches the below, more descriptive error).
Separately, I've tried a version where instead of chaining promises to get the final transcription result, I collect the name from the operation, and make a separate request to get the result.
Here's that request code:
... // Skipping setup
client.longRunningRecognize(request)
.then(data => {
const operation = data[0];
console.log(operation.latestResponse.name);
})
.catch(error => {
console.error(error);
});
When I hit the relevant endpoint (https://speech.googleapis.com/v1p1beta1/operations/81703347042341321989?key=ABCD12345) before it's had time to process, I get this:
{
"name": "81703347042341321989",
"metadata": {
"#type": "type.googleapis.com/google.cloud.speech.v1p1beta1.LongRunningRecognizeMetadata",
"startTime": "2018-08-16T19:33:26.166942Z",
"lastUpdateTime": "2018-08-16T19:41:31.456861Z"
}
}
Once it's fully processed, though, I've been running into this:
{
"name": "81703347042341321989",
"metadata": {
"#type": "type.googleapis.com/google.cloud.speech.v1p1beta1.LongRunningRecognizeMetadata",
"progressPercent": 100,
"startTime": "2018-08-16T17:20:28.772208Z",
"lastUpdateTime": "2018-08-16T17:44:40.868144Z"
},
"done": true,
"error": {
"code": 13,
"message": "Server unavailable, please try again later."
}
}
I've tried with shorter audio files (3 mins, same format and encoding), and the above processes both worked.
Any idea what's going on?
A possible workaround is changing the audio format to FLAC, which is the recommended encoding type for Cloud Speech-to-text API due to its lossless compression.
For reference, this can be done using sox, through the following command:
sox file.m4a --rate 16k --bits 16 --channels 1 file.flac
Additionally, this error may also happen when there is a long period of silence at the beginning. In this case, the audio files can be trimmed by specifying after trim the amount of seconds the audio should skip at the beginning and at the end of the file:
sox input.m4a --rate 16k --bits 16 --channels 1 output.flac trim 20 5

Categories

Resources