Develop Reference

How do I open and display a base64 pdf from inside my Cordova App?

I am creating an App for Android using Cordova, and I would like to open and display a file (PDF or image) that is served from the server as Base64-encoded binary data.
Of course I have read the multiple other posts on the subject that already exist on this website, but none of the proposed solutions have worked for me, more details below.
To be more precise, the server sends a JSON-file to the app, which among many other things contains a string consisting of the base64-encoded contents of a PDF file. I want to convert this data back into the represented PDF and display it to the user.
If this were a pure browser page, I would simply package my base64 data into a data-URL, attach this as the href of some anchor, and add a download-attribute. Optionally I could wrap all of my data into a blob and create an object url for that first.
In Cordova, this does not work. Clicking the <a> does nothing. Here is what I have attempted so far:
Using the file plugin, I can write the binary data to a file on the device. This works, and using a terminal I can see that the file was downloaded correctly, but into an app-private directory which I cannot access normally (e.g. through the file explorer).
Accessing the user's "downloads" folder is blocked by the file system
Using window.open with the file path as the first argument and "_system" as the target does nothing. There is no error but also nothing happens. Setting the target to "_blank" instead, I get an error saying ACCESS_DENIED.
Using cordova.InAppBrowser behaves the same was as window.open
With the plugin file-opener2 installed, the app will not compile, because the plugin is looking for an android4 toolchain, and I am building for android 9 and up
The plugin document-viewer (restricting to PDFs for the time being) suffers the same problem and does not compile.
Passing the data-URI to window.open (or cordova.InAppBrowser) directly loads for a very long time and eventually tells me that the desired page could not be loaded.
The PDF file I am using for testing is roughly 17kb after converting to base64. I know this is technically above the spec for how long data-URIs can be, but Chrome in the browser has no trouble with it whatsoever, and using a much shorter URI (only a few dozen bytes) produces the same behavior.
Ideally, what I would like to do, is download the file and then trigger the user's standard browser to open the file itself. That was, I would not have to deal with MIME types and also it would look exactly how the user expected from their own device.
Alternatively, if that doesn't work, I would be ok with downloading the file into a system-wide directory and prompting the user to open it themselves. This is not optimal, but I would be able to swallow that pill.
And lastly, if there is a plugin or some other solution that solves the problem amazingly, but for PDFs only, then I can also work out something else for images (e.g. embedding a new into my app and assigning the URI to that).
I would be thankful for any suggestion you might have on how to solve this problem. The code I use to download the file currently is shown below.
Thank you for your time.
var filePath = cordova.file.externalDataDirectory; // Note: documentsDirectory is set to "" by Cordova, so I cannot use that
var fileName = "someFileName.pdf";
var mime = "application/pdf";
var dataBlob = /* some blob containing the binary data for a PDF */
function writeFile(fileEntry, dataBlob) {
// Create a FileWriter object for our FileEntry.
// This code is taken directly from the cordova-plugin-file documentation
fileEntry.createWriter(function (fileWriter) {
fileWriter.onwriteend = function() {
console.log("Successful file write...");
readFile(fileEntry);
};
fileWriter.onerror = function (e) {
console.log("Failed file write: " + e.toString());
};
fileWriter.write(dataBlob);
});
}
window.resolveLocalFileSystemURL(
filePath,
function onResolveSuccess (dirEntry) {
dirEntry.getFile(
fileName,
{ create: true },
function onGetFileSuccess (file) (
writeFile(file, dataBlob);
// At this point, the file has been downloaded successfully
window.open(file.toURL(), "_system"); // This line does nothing, and I don't understand why.
}
);
}
);

I managed to solve the problem.
As per the documentation of the file-opener2 plugin, you need to also add the androidx-adapter plugin to correct for the outdated (android 4) packages. With the plugins file, file-opener2 and androidx-adapter installed, the complete code is the following:
var filePath = cordova.file.externalDataDirectory; // Note: documentsDirectory is set to "" by Cordova, so I cannot use that
var fileName = "someFileName.pdf";
var mime = "application/pdf";
var dataBlob = /* some blob containing the binary data for a PDF */
function writeFile(fileEntry, dataBlob) {
// Create a FileWriter object for our FileEntry.
// This code is taken directly from the cordova-plugin-file documentation
fileEntry.createWriter(function (fileWriter) {
fileWriter.onwriteend = function() {
console.log("Successful file write...");
readFile(fileEntry);
};
fileWriter.onerror = function (e) {
console.log("Failed file write: " + e.toString());
};
fileWriter.write(dataBlob);
});
}
window.resolveLocalFileSystemURL(
filePath,
function onResolveSuccess (dirEntry) {
dirEntry.getFile(
fileName,
{ create: true },
function onGetFileSuccess (file) (
writeFile(file, dataBlob);
// At this point, the file has been downloaded successfully
cordova.plugins.fileOpener2.open(
    filepath + filename,
    mime,
    {
     error : function(){ },
success : function(){ }
    }
);
}
);
}
);

node.js - How to chain download many images from a url

Basically, I want to download a large amount of images from an image service. I have a very large JSON object with all of the URLs (~500 or so) in that JSON object. I tried a few npm image downlader packages as well as some other code that did each image downloading all at the same time; however, about 50% of the downloaded images had data loss while downloaded (a large portion of the image was transparent when viewed). How can I download each image, one after another (waiting until the last one is complete before starting the next) to avoid the data loss?
Edit: here is the relevant code, using request:
var download = function(url, dest, callback){
request.get(url)
.on('error', function(err) {console.log(err)} )
.pipe(fs.createWriteStream(dest))
.on('close', callback);
};
links.forEach( function(str) {
var filename = str[0].split('/').pop() + '.jpeg';
console.log(filename);
console.log('Downloading ' + filename);
download(str[0], filename, function(){console.log('Finished Downloading ' + filename)});
});
My links JSON looks like this:
[["link.one.com/image-jpeg"], ["link.two.com/image-jpeg"]]

Okay, so first thing first :
I really do not believe that downloading those 500+ images will all start at once. The V8 engine (kind of the nodejs code executor) actually manages a reasonable number of threads and reuse them to do the stuff. So, it wont create "lots of" new threads, but will wait for other threads to get done.
Now, even if it was all starting at once, I don't think the files would get damaged. I the files were getting corrupt, you wouldn't have been able to open those files.
So, I am pretty sure the problem with the images is not what you think.
Now, for the original question, and to test if I am wrong, you can try to download those files in a sequence like this :
var recursiveDowload = function (urlArray, nameArray, i) {
if (i < urlArray.length) {
request.get(urlArray[i])
.on('error', function(err) {console.log(err)} )
.pipe(fs.createWriteStream(nameArray[i]))
.on('close', function () { recursiveDownload (urlArray, nameArrya, i+1); });
}
}
recursiveDownload(allUrlArrya, allNameArray, 0);

Since you are doing large number of downloads, try Aria2c. Use Aria2 documentations for further details.

How to check if the file is having the same type as of its extension?

I want to build Javascript code which checks for the file type.In the web application which I am creating allows user to upload document files viz, doc, xls, ppt, docx, xlsx, pptx, txt, rar, zip, pdf, jpg, png, gif, jpeg, odt, but it should not allow other files. I can't check just extension name in file name. As user may change it.
I tried checking content-type but it is also getting changed everytime. Suggestions are appreciated.

In "modern" browsers (IE10+, Firefox 4+, Chrome 7+, Safari 6.0.2+ etc.), you could use the File/FileReader API to read the contents of the file and parse it client-side. E.g. (example, not production code):
var fileInput = /* Your <input type="file"> element */
fileInput.addEventListener("change", function(e) {
var file = e.currentTarget.files[0];
var reader = new FileReader();
reader.onload = fileLoaded;
reader.readAsArrayBuffer(file);
});
function fileLoaded(e)
{
var arrayBuffer = e.currentTarget.result;
// 32 indicates that we just want a look at the first 32 bytes of the buffer.
// If we don't specify a length, we get the entire buffer.
var bytes = new Uint8Array(arrayBuffer, 0, 32);
// Now we can check the content, comparing to whatever file signatures we
// have, e.g.:
if (bytes[0] == 0x50 &&
bytes[1] == 0x4b &&
bytes[2] == 0x03 &&
bytes[3] == 0x04)
{
// This is most likely docx, xlsx, pptx or other zip file.
}
}
http://jsfiddle.net/35XfG/
Note, however, that e.g. a .zip doesn't have to start with 50 4b 03 04. So, unless you spend quite a bit of time looking into different file signatures (or find some library that already did this), you're likely to be rejecting files that might actually be valid. Of course, it's also possible that it will give false positives.
False positives don't matter that much in this case, though - because this is only useful as a user friendly measure to check that the user isn't uploading files that will be rejected by the server anyway. The server should always validate what it ends up getting sent.
Of course, reading the entire file to look at the first few bytes isn't all that efficient either. :-) See Ray Nicholus' comment about that.

Unless you can actually parse the content and by the results tell whether the file is of a certain type, I don't see a good way of doing that with pure JS. You might want to consider to upload the file to the sever temporarily, and then perform the check on the server. The unix file command is a very useful tool for that. It does not rely on file extensions, but uses the file content to analyze the file type.

Huge JavaScript HTML5 blob (from large ArrayBuffers) to build a giant file in client side

I'm writing a web browser app (client-side) that downloads a huge amount of chunks from many locations and joins them to build a blob. Then that blob is saved to local filesystem as a common file. The way I'm doing this is by mean of ArrayBuffer objects and a blob.
var blob = new Blob([ArrayBuffer1, ArrayBuffer2, ArrayBuffer3, ...], {type: mimetype})
This works ok for small and medium-sized files (until 700 MB aprox), but browser crashes with larger files. I understand that RAM memory has its limits. The case is that I need to build the blob in order to generate a file, but I wanna allow users to download files much larger than that size (imagine, for instance, files about 8GB).
¿How can I build the blob avoiding size limits? LocalStorage is more limited than RAM, so I do not know what to use or how to do it.

It looks like you are just concatenating arrays of data together? Why not go about appending the array-buffers together in a giant blob. You'd have to iterate and append each arrayBuffer one at a time. You would seek to the end of the filewriter to append arrays. And for reading only portions of your giant blob back you get a slice of the blob to avoid the browser crashing.
Appending Function
function appendToFile(fPath,data,callback){
fs.root.getFile(fPath, {
create: false
}, function(fileEntry) {
fileEntry.createWriter(function(writer) {
writer.onwriteend = function(e) {
callback();
};
writer.seek(writer.length);
var blob = new Blob([data]);
writer.write(blob);
}, errorHandler);
}, errorHandler);
}
Again to avoid reading the entire blob back, only read portions/chunks of your giant blob when generating the file you mention.
Partial Read Function
function getPartialBlobFromFile(fPath,start,stop,callback){
fs.root.getFile(fPath, {
creation:false
}, function(fileEntry){
fileEntry.file(function(file){
var reader = new FileReader();
reader.onloadend = function(evt){
if(evt.target.readyState == FileReader.DONE){
callback(evt.target.result);
}
};
stop = Math.min(stop,file.size);
reader.readAsArrayBuffer(file.slice(start,stop));
}, errorHandler)
}, errorHandler);
}
You may have to keep indexes, perhaps in a header section of your giant BLOB - I would need to know more before I could give more precise feedback.
Update - avoiding quota limits, Temporary vs Persistent
in response to your comments below
It appears that you are running into issues with storage quota because you are using temporary storage. The following is a snippet borrowed from google found here
Temporary storage is shared among all web apps running in the browser. The shared pool can be up to half of the of available disk space. Storage already used by apps is included in the calculation of the shared pool; that is to say, the calculation is based on (available storage space + storage being used by apps) * .5 .
Each app can have up to 20% of the shared pool. As an example, if the total available disk space is 50 GB, the shared pool is 25 GB, and the app can have up to 5 GB. This is calculated from 20% (up to 5 GB) of half (up to 25 GB) of the available disk space (50 GB).
To avoid this limit you'll have to switch to persistent, it will allow you to quota up to the available free space on the disk. To do this use the following to initialize the File-system instead of the temporary storage request.
navigator.webkitPersistentStorage.requestQuota(1024*1024*5,
function(gB){
window.requestFileSystem(PERSISTENT, gB, onInitFs, errorHandler);
}, function(e){
console.log('Error', e);
})

javascript - zip a Blob with zip.js

I need to store a lot of text in WebSQl, so I decided to compress the text with zip.js
and store the compressed Blobs.
From the documentation you can compress a blob as follows:
function zipBlob(filename, blob, callback) {
// use a zip.BlobWriter object to write zipped data into a Blob object
zip.createWriter(new zip.BlobWriter("application/zip"), function(zipWriter) {
// use a BlobReader object to read the data stored into blob variable
zipWriter.add(filename, new zip.BlobReader(blob), function() {
// close the writer and calls callback function
zipWriter.close(callback);
});
}, onerror);
}
Although this works, I don't understand why you need to specify a filename. Is this really necessary? And, is this file always removed after compression?

Check this answer here - it does not require a filename, and I'll bet its much easier to use. I've tried quite a few javascript compression/decompression implementations and have been stung by problems such as limits on size of original data, overall speed, efficiency, and so forth. It is oddly difficult to find a good compression/decompression implementation in javascript, but thankfully this one hasn't failed me yet (and I've used it quite a bit):
Compressing a blob in javascript
The implementation you have currently requires the filename because it is trying to be consistent with the zip, so that you can save it for example to a desktop and open it with your favorite zip utility. It sounds like your challenge is very similar to mine, I needed to save and restore compressed items out of local storage in the browser as well as on the server.

The filename is necessary according to this implementation. It wouldn't be necessary if you were only compressing the data, but zip.js builds zip files, which store files, which must have filenames.
In your original example, zipWriter.add() effectively converts your blob into a new file and adds it to a zip – and the "filename" parameter is the name you want that new file to have.
Here's an example that uses zip.js to add multiple blobs to a zip and then downloads it with FileSaver.js:
function zipBlob() {
zip.createWriter(new zip.BlobWriter("application/zip"), function(writer) {
files = ["abcd", "123"];
var f = 0;
function nextFile(f) {
fblob = new Blob([files[f]], { type: "text/plain" });
writer.add("file"+f, new zip.BlobReader(fblob), function() {
// callback
f++;
if (f < files.length) {
nextFile(f);
} else close();
});
}
function close() {
// close the writer
writer.close(function(blob) {
// save with FileSaver.js
saveAs(blob, "example.zip");
});
}
nextFile(f);
}, onerror);
}