Reading local BSON file in Javascript - javascript

I have an HTML 5 game. I'm using webpack/typescript for development.
There is some data I have which I was including by using require like the following
const dataJson = require('scripts/data/data.json');
I would like to do the equivalent, except with bson. I tried the naive approach of doing something like this
const dataJson = require('scripts/data/data.bson');
but this of course fails since there is no loader (won't compile with currently no loaders are configured to process this file.).
I'd like to then include the file locally, load the file and then deserialize the bson. Or I'd like to embed the bson like when using require. This is some tool generated data, so it will be in a data file.
I haven't been able to figure it out. I've tried the following. But this results in the result containing either the bits of File or what looks like the content type (if done as readAsDataURL).
What I have tried
const file = new File(['data'], 'assets/data.bson', { type: 'application/bson' });
const reader = new FileReader();
reader.onload=(theFile) => {
if (theFile.target) {
console.log(theFile.target.result);
}
} ;
reader.readAsDataURL(file);
//reader.readAsBinaryString(file);
What is the correct method to load a local binary file? Presumably, once I have the data, I can just call deserialize from the bson package.
Okay, I'm adding some corrections here.
My method to read the files is wrong. I know understand File will actually create a file. So when this is passed to FileReader it gets the value of the both bits passed in.
I have since discovered I can get the local files 2 ways. I can use XMLHttpRequest as well as the raw-loader loader.
However once I do this. I cannot seem to convert the contents into JSON using bson. Any variant of deserialize, serialize, parse, or stringify has some issue.
Does anyone happen to have the correct method to convert the BSON contents into either a Javascript Object?
Note that the BSON is generated from python using pymongo. My code to generate the BSON is the following.
with open(args.output_bson, 'wb') as fp:
encoded = bson.encode(data_meta.to_dict())
fp.write(encoded)
to_dict is a dictionary. I output both the JSON (using json) and BSON.
I also tested the file with bsondump and it does indeed convert to JSON. So it does appear the file I've loaded is valid.

you can use bson
then call:
BSON.deserialize(theFile.target.result);

Related

PDFMake Base64 generates malformed document

I'm using PDFMake to generate some PDF reports on client side.
When I specify the definition for document and pass it to createPdf(DEFINITION_HERE) I get my PDF and it looks nice and shiny. But I'm saving it in the DB as Base64 encoded string and when I try to reopen it with data URI (ie: data:application/pdf;base64,JVBER...) the document is completely malformed.
My DB call looks like this:
await pdfFile.getDataUrl(async (result: string) => {
await myServiceLayer.generatePdfReport(result, fileName, creator);
});
Anyone else had this issue with PDFMake library? Any ideas on what could be the cause of this?
OK, so after trying different things and debugging left and right, I stumbled upon this:
https://github.com/bpampuch/pdfmake/issues/318
So I removed call to .download() on my PDF file and checked for Base64 again, now it had correct value and file looks as it should.
So in a nutshell, if we call multiple methods on PDFMake, usually something will get messed up.
I had my .download() call just before my .getBase64() as I wanted to first download the file for the user and then also store it in DB as Base64, as .download() happened .getBase64() kept on giving me Base64 output which resulted in malformed PDF file.

Cordova fileWriter.write() doesn't work with blobs

In the official documentation and many other solutions posted online, the following code is supposed to be the way of writing data to a file in Cordova
function writeFile(fileEntry, dataObj) {
fileEntry.createWriter(function (fileWriter) {
fileWriter.write(dataObj);
});
}
writeFile(someFileEntry, blob);
In the documentation, they explicitly shown that a blob can be passed as the "dataObj", but whenever I passed a blob as "dataObj", the resulting file only has 2 bytes of data. After inspecting the file, I found that the contents of the file only contain a single string
{}
hence the 2 bytes.
I've tried passing a string as the "dataObj", and the contents of the resulting file was the exact same string, so string works I guess? But the data I'm wishing to write to a file is a blob that contains video data recorded from a canvas, so either I'll have to
somehow convert the the video blob into a string and write the string into the file
somehow fix the "fileWriter.write" function
But I've gone nowhere with these solutions. I've tried "blob.text()" or using a "fileReader" to get the contents of the blob as a string, but the resulting file is always broken. And fixing the "fileWriter.write" function is way out of my depth. Can someone help me out on this?
I am having the same issue. I think it might have something to do with the following as stated on the cordova-plugin-file docs:
But I find that odd since the examples all use FileWriter.write(blob) but it says that the platforms do not support that function.

How to read an image file (pixel data) into a JavaScript array?

I have a very basic question: how can I read an image file in javascript and get access to its pixel data as arrays? I am writing a local script to be run as node myscript.js, so no need for any web-stuff.
Basically I need a javascript equivalent of the following python 2-liner, preferably with as few external dependencies as possible:
import skimage.io
image = skimage.io.imread('someimage.file',as_gray=False).astype('float64')
# do stuff to image
You can read a file/image using a FileReader() object in javascript.
From the documentation:
The FileReader object lets web applications asynchronously read the contents of files (or raw data buffers) stored on the user's computer, using File or Blob objects to specify the file or data to read.
And after reading an image, you can use a third-party library to manipulate it. Here are a few good libraries:
Caman JS
glfx.js
Grafi.js
Jimp
A basic example of reading file:
var fs = require('fs');
fs.readFile('image.jpg', function(err, data){
console.log(data);
});

Displaying a generated JSON file on a simple website

I have a Java tool, which schedules a task every 24 hours and writes the result in a result.json file. Now I want to display this result.json file on a simple website, but I know that it's not natively possible with JavaScript to access local files. But what other, simple ways exist for this problem? I try to avoid a webservice to keep the scheduling-program and the website on the same server.
Thanks!
You can use the object FileReader for read the file stream from your system and gets the JSON file in a string, later use JSON.parse() to get the JSON in JS object and iterate over it for pretty representing in document DOM, or you can print the string in the HTML without parse to JS object.
Here there are a very completely example of FileReader.

Best way to store data to be read by JS

I am new to Javascript and HTML5.
What is the best way to store text data to be read in JS.
I want to load all the characters of a file into an array, the file is similar to a map file and is stored amongst my resource file.
In Java I would create a text file and read each byte into an array, what is the equivalency of doing that in JS? Does JS have a better method of doing this?
Best way - create .js file with JSON encoded data. Almost all languages have functions for this, json_encode in PHP for example.
You can create a file like:
window.data = {something: true, fromfile: ['f','d','s']};
and include it as javascript file on html page:
<script src="data.js"></script>
So you'll have that array in window.data.
Or create a file:
{something: true, fromfile: ['f','d','s']}
And load it with jQuery's AJAX:
$.post('fileURL',{}, function(data) { console.log(data); }, 'json');
and get it at any moment you need it. You can also generate such data at any time with server-side scripts to get actual data, just change fileURL.js to server-side script's URL.
You can also change last parameter (json) to 'text' and get file content as string. But I'd not trust it for binary files, better to convert it on server side to JSON.

Categories

Resources