open a file as base 64 in nodejs - javascript

quick question here, I'm pretty confident this is not complicated and a lot of chance that is a duplicate but I still cannot find a way to do it.
I'm in the backend and the front end return send me a cdv file in the body. I cannot change that.
What I need to do is simply parse it but this is the trick. I can't "open" the given data to have something to work with cdv-parse.
the format of the data is (when I do a console.log): data:text/csv;name=toto.csv;base64,VHlwZSBk (and so on)
console.log(myfile); // data:text/csv;name=toto.csv;base64,VHlwZSBk (and so on)
console.log(fs.readFileSync(myfile, "base64"), "base64")); // error: ENOENT: no such file or directory, open 'u�Z��m�
I also tried with ```Buffer.from(myfile, "base64").toString()`` and again the format isn't the data expected.
EDIT: it seem that using myfile = myfile.replace("data:text/csv;name=toto.csv;base64,", "");does the trick with buffer.from().toString();
but I want something more générique, I guess something exist no ?
Thanks in advance

Related

Fetching ical url

I'm trying to retrieve a content from the specific ical url using ical npm.
I'm testing this code on the client side. There are a few icals that I've managed to fetch. But this one and several others I didn't.
It works great on POSTMAN.
My code is:
const ical = require('ical');
ical.fromURL('https://admin.vrbo.com/icalendar/5d19afbfbf144218a3c76eacf76267c6.ics', {mode: 'cors'}, (err, data) => {
console.log(err)
});
The error I get:
Error: Invalid value for opts.mode
at new push../node_modules/stream-http/lib/request.js.module.exports (request.js:58)
at Object.push../node_modules/stream-http/index.js.http.request (index.js:30)
at Object.push../node_modules/https-browserify/index.js.https.request (index.js:13)
at Request.push../node_modules/request/request.js.Request.start (request.js:829)
at Request.push../node_modules/request/request.js.Request.end (request.js:1639)
at end (request.js:628)
at request.js:644
at run (setImmediate.js:48)
at runIfPresent (setImmediate.js:83)
at onGlobalMessage (setImmediate.js:125)
Pleas tell me what I'm doing wrong and how to make this code work.
I think the issue here is, that you are trying to pass an invalid option ({mode: 'cors'}). ical.fromURL(...) will pass the options argument as it is to request (See Line 8). I've checked the source of request, but did not find any option with the name mode (Maybe you are mixing it up with the Fetch API, because there is an option called mode), so in my opinion, that is also the reason, why you are getting the error (But I'm not sure why it worked out for you, to fetch some other calendars). You can check the full list of possible options by yourself.

Loading JSON data from server into knockout.js observable

I am trying to load initial data (from a local JSON string) using knockouts.js in order to displayed it in the UI right after loading the page.
I have tried a LOT of options but all failed or produce errors.
My latest try is http://jsfiddle.net/z22m1798/17/ (base on the 'Cart' example) which produce the error:
TypeError: cartLine.filterValue is not a function.
The relevant code is:
//Load initial data from server
var JSONdataFromServer = $("#JSONdataFromServer").val();
console.log(JSONdataFromServer);
var dataFromServer = ko.utils.parseJson(JSONdataFromServer);
self.lines.push(new CartLine(dataFromServer));
Someone maybe know what am I doing wrong here?
It's look like a difficult issue.
Thanks
I found that the way to solve it (don't sure it is the most accurate way) is by receiving from the server the entire JSON so the first cells will contain the filters I wish to display. In addition, the server will send a parameter "filtersToDisplay" that indicate how many filters need to display.
By default, filtersToDisplay=1 to display only the first line.
http://jsfiddle.net/z22m1798/25/
for (i = 0; i < $("#filtersToDisplay").val(); i++) {
self.lines.push(new CartLine(self.lines))// Put one line in by default
}
Thanks

Node.JS req.body String Too Long?

I am once again asking a probably beginners question, but I have looked around quite a bit, and was unable to find the answer I need. I have created a Node.JS server pulling data for a Game State Integration project I am working on, and I need to request certain data from the game.
Basically, what is happening in the code is I am setting the variable newKillStatus to a certain part of the player_state file from which I am reading. Before I talk a bit further, here is the code I am using: (If you need more, please ask)
fs.readFile('player_state', 'utf8', function(err, killStatus) {
var player = 'player' in req.body ? req.body.player : null;
if (player && player.state !== killStatus) {
var newKillStatus = req.body.player.state.round_kills;
if (!newKillStatus) {
newKillStatus = '';
}
fs.writeFile('player_state', newKillStatus);
console.log(newKillStatus);
}
});
So as you can see I am trying to set newKillStatus to req.body.state.round_kills; which, from what I can understand is too long, because when I try to start the server, it gives me this error:
cannot read property 'round_kills' of undefined
But If I get rid of the round_kills part it gives me round_kills, but it also gives me a whole bunch of other statistics that I don't want to set the variable to.
So basically my only question is how can I set var newKillStatus to req.body.player.state.round_kills without an error.
Thank you for helping!
EDIT:
I have tried all the solutions and multiple combinations of strings, is it possible that the underscore is creating an error?
After several attempts to find the answer myself, I have found it. I am actually quite surprised it took me so long to spot this. The probably was actually above what I though was causing it, and what the error said was causing it. The problem was actually in var player = 'player' in req.body ? req.body.player : null; I needed to change req.body.player to req.body.player.state do the code below would know what I referencing.

Parsing a large JSON array in Javascript

I'm supposed to parse a very large JSON array in Javascipt. It looks like:
mydata = [
{'a':5, 'b':7, ... },
{'a':2, 'b':3, ... },
.
.
.
]
Now the thing is, if I pass this entire object to my parsing function parseJSON(), then of course it works, but it blocks the tab's process for 30-40 seconds (in case of an array with 160000 objects).
During this entire process of requesting this JSON from a server and parsing it, I'm displaying a 'loading' gif to the user. Of course, after I call the parse function, the gif freezes too, leading to bad user experience. I guess there's no way to get around this time, is there a way to somehow (at least) keep the loading gif from freezing?
Something like calling parseJSON() on chunks of my JSON every few milliseconds? I'm unable to implement that though being a noob in javascript.
Thanks a lot, I'd really appreciate if you could help me out here.
You might want to check this link. It's about multithreading.
Basically :
var url = 'http://bigcontentprovider.com/hugejsonfile';
var f = '(function() {
send = function(e) {
postMessage(e);
self.close();
};
importScripts("' + url + '?format=json&callback=send");
})();';
var _blob = new Blob([f], { type: 'text/javascript' });
_worker = new Worker(window.URL.createObjectURL(_blob));
_worker.onmessage = function(e) {
//Do what you want with your JSON
}
_worker.postMessage();
Haven't tried it myself to be honest...
EDIT about portability: Sebastien D. posted a comment with a link to mdn. I just added a ref to the compatibility section id.
I have never encountered a complete page lock down of 30-40 seconds, I'm almost impressed! Restructuring your data to be much smaller or splitting it into many files on the server side is the real answer. Do you actually need every little byte of the data?
Alternatively if you can't change the file #Cyrill_DD's answer of a worker thread will be able to able parse data for you and send it to your primary JS. This is not a perfect fix as you would guess though. Passing data between the 2 threads requires the information to be serialised and reinterpreted, so you could find a significant slow down when the data is passed between the threads and be back to square one again if you try to pass all the data across at once. Building a query system into your worker thread for requesting chunks of the data when you need them and using the message callback will prevent slow down from parsing on the main thread and allow you complete access to the data without loading it all into your main context.
I should add that worker threads are relatively new, main browser support is good but mobile is terrible... just a heads up!

Drupal.t doesn't show up among translations

I'm trying to make a translation in JavaScript on a Drupal site. I'm using the js function Drupal.t(). Everything seems to work; Drupal is loaded, the function gets called, placeholders get replaced, but the translation doesn't happen. The words stay in English, and the words aren't added to the translations database. Does anybody know why this happens and how to get solve it?
I know your post is rather old, but my answer might help others.
I've noticed that Drupal.locale.strings doesn't get populated with the JavaScript calls.
So what I usually do is simply create a portion of PHP code that does this work on the server side. You can do it anywhere in your PHP code. But the best is to do it in the module you are coding. It will then be easier to export it with the Potx module so that you can have your strings in some *.po files for a later use of your module.
If you are in a hurry, you can simply do it in the body of a dummy node (just do a "preview") with the PHP input format:
<?php
print t('Example : %variable', array('%variable' => 'test'));
?>
Once this has been done, you should be able to find your strings and translate them in the admin page.
To regenerate the JavaScript file, you'll have to clear all your cache (with Devel or by visiting the modules page).
Not sure about the javascript part but I've used the t funciton a lot. If you want the strings to show up in the translation table you have to load the corresponding page in two different languages before it will allow you to translate them. Hope that helps.
The Drupal.t() function is very small, so I bet we can just dissect it here.
function (str, args) {
// Fetch the localized version of the string.
if (Drupal.locale.strings && Drupal.locale.strings[str]) {
str = Drupal.locale.strings[str];
}
if (args) {
// Transform arguments before inserting them
for (var key in args) {
switch (key.charAt(0)) {
// Escaped only
case '#':
args[key] = Drupal.checkPlain(args[key]);
break;
// Pass-through
case '!':
break;
// Escaped and placeholder
case '%':
default:
args[key] = Drupal.theme('placeholder', args[key]);
break;
}
str = str.replace(key, args[key]);
}
}
return str;
}
So you can see that it checks Drupal.locale.strings to find the translatable strings for the current locale. Can you pop open a console on your Drupal site and type Drupal.locale and see what it spits out?
In Drupal, to add a new language, you must first enable the Locale module. Have you done that?
Next, you have to import your language through the Translate Interface at admin/build/translate. Have you done that?
Here you can see I've imported French and German: http://grimhappy.com/i/ce75ca.png
¿Are you compressing out the js? Check at the performance section of your site.
Clearing the cache should regenerate the js with the right translations now.

Categories

Resources