Develop Reference

Taking an image and converting it into BLOB for MySQL

I am trying to make a JavaScript that would take an image file and covert it into BLOB (by converting the file into Base64 first and then into BLOB), my project doesn't have a support for toBlob() so I have found different convering steps and put them together and they work to a point where I have to pass the BLOB from the function where its made out for the Mysql part of code that takes care of communicating with the database. (I have that fully working). Now I only need to find a way how to connect them through a variable that saves the results of the imageforQuery function.
My code so far is this:
let base64String = "";
function imageforQuery(imageid) {
//takes file and converts to Base64
var file = document.getElementById(imageid).files[0];
var reader = new FileReader();
console.log("next");
imgFileFrontBlob = "";
reader.onload = function () {
base64String = reader.result.replace("data:", "")
.replace(/^.+,/, "");
// console.log(base64String);
base64String = 'data:image/jpeg;base64,'+ base64String;
console.log(base64String);
//converts Base64 into BLOB
var binary = atob(base64String.split(',')[1]);
console.log(binary);
var array = [];
for(var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var imgFileFrontBlob = new Blob([new Uint8Array(array)], {type: 'image/png'});
console.log(imgFileFrontBlob);
return imgFileFrontBlob
}
reader.readAsDataURL(file);
};
by experimenting with console.log() at different stages and return I have found out that I can't pass the converted BLOB result out, as the function imageforQuery() only returns what is after reader.readAsDataURL(file); and I don't know of a way of getting that result out.
––––––––––––––ADDITIONAL PROBLEMS I HAVE ENCOUNTERED––––––––––––––
okay so thanks to Emiel Zuurbier (Thank you!) I have managed to rewrite my code with the help of his solution. However as much as it helped one part of the problem, it didn't help with the JavaScript Blob object as we found out it is not the exact same thing as SQL BLOB.
Now the problem is that upon trying to send the Blob object in a SQL query, this resulted in just sending pure text "[Blob object]".
But I am using JavaScript successfully to pull the data from a BLOB field from my database and convert it into Base64 images from that data that was stored in the BLOB in a different part of my application. The code for that is below:
var converterEngine = function (input) {
// fn BLOB => Binary => Base64 ?
var uInt8Array = new Uint8Array(input),
i = uInt8Array.length;
var biStr = []; //new Array(I);
while (i--) { biStr[i] = String.fromCharCode(uInt8Array[i]); }
var base64 = window.btoa(biStr.join(''));
return base64;
};
What I need to do is just reverse this and in theory, it should get me the same data that I receive from the database.
My reversal code is below:
// this is the inside bit of code from the first problem that is solved and the
// typeOfData variable is parsed into the function in imageforQuery() as a second input
// variable (in other words its not to be of concern)
reader.onload = function () {
let base64String = reader.result.replace("data:", "").replace(/^.+,/, "");
base64String = "data:" + typeOfData + ";base64," + base64String;
var binary = atob(base64String.split(",")[1]);
// console.log(binary);
var array = [];
for (var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var ourArray = new Uint8Array(array);
resolve(ourArray);
};
However, as I mentioned the data that comes out (ourArray) isn't actually identical to the original file from the BLOB in the database so my code doesn't function correctly and I don't know why. Any ideas where I've made a mistake?

Base64 is simply ascii text. So MySQL's datatype BLOB or TEXT would work. That is, after converting to Base64, don't worry about "convert to blob"; it is not necessary.
That is, you can probably replace the code from //converts ... through return ... by simply
return base64String;

You can wrap the FileReader instance and calls inside of a Promise. Return the Promise immediately. In the reader.onload function call resolve() to exit the Promise with a value.
function imageforQuery(imageid) {
return new Promise(resolve => {
var file = document.getElementById(imageid).files[0];
var reader = new FileReader();
reader.onload = function () {
let base64String = reader.result.replace("data:", "").replace(/^.+,/, "");
base64String = "data:image/jpeg;base64," + base64String;
var binary = atob(base64String.split(",")[1]);
var array = [];
for (var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var imgFileFrontBlob = new Blob([new Uint8Array(array)], {
type: "image/png",
});
resolve(imgFileFrontBlob);
};
reader.readAsDataURL(file);
});
}
This results in being able to use your function like here below. imageforQuery is called, returns a Promise. When the promise is finished (meaning resolve is called) the function in the then method will run.
imageforQuery(imageId).then(imgFileFrontBlob => {
// Use your blob here.
saveToDB(imgFileFrontBlob); // Example of how you would use it.
});
Or use it with async / await.
(async () => {
function imageforQuery(imageid) {
...
}
// Here we can wait for imageforQuery to finish and save the variable.
const imgFileFrontBlob = await imageforQuery(imageId);
saveToDB(imgFileFrontBlob); // Example of how you would use it.
})()

Create ReadStream from Base64 encoded string by file

I feel I got lack of understanding about Buffer and File Stream, but I can't find any specific idea from other answers which is treating base64 string as a actual file.
I used 'request' package from Npm, to send a file to other server by http, multipart protocol.
The code below is working well, read a file from actual file by 'fs' package and send it by ReadStream object by createReadStream method.
(The codes is coffeescript)
#working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: fs.createReadStream('/path/' + 'myfile.doc')
What I want to do is creating a same ReadStream object by fs module from a file Based64 encoded String.
I tested something like this, but it's not working properly.
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: new Buffer(base64EncodedString, 'base64')
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: _base64ToArrayBuffer(base64EncodedString)
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: convertDataURIToBinary(base64EncodedString)
##(used function)
_base64ToArrayBuffer = (base64) ->
binary_string = require('atob')(base64)
len = binary_string.length
bytes = new Uint8Array(len)
i = 0
while i < len
bytes[i] = binary_string.charCodeAt(i)
i++
bytes.buffer
convertDataURIToBinary = (dataURI) ->
BASE64_MARKER = ';base64,'
base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length
base64 = dataURI.substring(base64Index)
raw = require('atob')(base64)
rawLength = raw.length
array = new Uint8Array(new ArrayBuffer(rawLength))
i = 0
while i < rawLength
array[i] = raw.charCodeAt(i)
i++
array
question conclusion
The 'base64EncodedString' is validated by decode & creating file, so i don't doubt it is the matter, so I think I could have this achievement with write file from base64 and read it again with fs module, but I believe that's not the proper way.
The point of question is,
1. How could I send the base64encoded string as a ReadStream object properly in this case
2. How could I figure out relationship clearly between buffer and dataview(uint8array ...) something like that
Thanks in advance.
appendix
From #Alex Nikulin 's comment, I've tested stream-buffers package.
streamBuffers = require('stream-buffers')
myReadableStreamBuffer = new (streamBuffers.ReadableStreamBuffer)(
frequency: 10
chunkSize: 2048)
myReadableStreamBuffer.put base64.decode example
myReadableStreamBuffer.stop()
And when I tried bellow it failed again, and the 'AnotherUrl' returns Error message like this, 'unexpected end of part'
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: myReadableStreamBuffer
I figured out the myReadableStreamBuffer object is Readable object, so it might be different with ReadStream object. Can I get it as a ReadStream from myReadableStreamBuffer?

Try this
//if you need just a buffer
var base64ToBuffer = function(base64) {
var byteString = new Buffer(base64, 'base64').toString('binary');
var ab = new Buffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
return ab;
}
//if you need a stream, not a buffer
var stream = require('stream');
// Initiate the source
var bufferStream = new stream.PassThrough();
// Write your buffer
bufferStream.end(base64ToBuffer(base64));
bufferStream.pipe( process.stdout );

Client download of a server generated zip file

Before somebody says, "duplicate", I just want to make sure, that folks know, that I have already reviewed these questions:
1) Uses angular and php, not sure what is happening here (I don't know PHP): Download zip file and trigger "save file" dialog from angular method
2) Can't get this answer to do anything: how to download a zip file using angular
3) This person can already download, which is past the point I'm trying to figure out:
Download external zip file from angular triggered on a button action
4) No answer for this one:
download .zip file from server in nodejs
5) I don't know what language this even is:
https://stackoverflow.com/questions/35596764/zip-file-download-using-angularjs-directive
Given those questions, if this is still a duplicate, I apologize. Here is, yet, another version of this question.
My angular 1.5.X client gives me a list of titles, of which each have an associated file. My Node 4.X/Express 4.X server takes that list, gets the file locations, creates a zip file, using express-zip from npm, and then streams that file back in the response. I then want my client to initiate the browser's "download a file" option.
Here's my client code (Angular 1.5.X):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
$http.get('/query/bulkdownload',{
params:{titles:titles},
responseType:'arraybuffer'
})
.then(successCb,errorCb)
.catch(exceptionCb);
}
function successCb(response){
// This is the part I believe I cannot get to work, my code snippet is below
};
function errorCb(error){
alert('Error: ' + JSON.stringify(error));
};
function exceptionCb(ex){
alert('Exception: ' + JSON.stringify(ex));
};
};
Node (4.X) code with express-zip, https://www.npmjs.com/package/express-zip:
router.get('/bulkdownload',function(req,resp){
var titles = req.query.titles || [];
if ( titles.length > 0 ){
utils.getFileLocations(titles).
then(function(files){
let filename = 'zipfile.zip';
// .zip sets Content-Type and Content-disposition
resp.zip(files,filename,console.log);
},
_errorCb)
}
});
Here's my successCb in my client code (Angular 1.5.X):
function successCb(response){
var URL = $window.URL || $window.webkitURL || $window.mozURL || $window.msURL;
if ( URL ) {
var blob = new Blob([response.data],{type:'application/zip'});
var url = URL.createObjectURL(blob);
$window.open(url);
}
};
The "blob" part seems to work fine. Checking it in IE's debugger, it does look like a file stream of octet information. Now, I believe I need to get that blob into the some HTML5 directive, to initiate the "Save File As" from the browser. Maybe? Maybe not?
Since 90%+ of our users are using IE11, I test all of my angular in PhantomJS (Karma) and IE. When I run the code, I get the old "Access is denied" error in an alert window:
Exception: {"description":"Access is denied...<stack trace>}
Suggestions, clarifications, answers, etc. are welcome!

Use this one:
var url="YOUR ZIP URL HERE";
window.open(url, '_blank');

var zip_file_path = "" //put inside "" your path with file.zip
var zip_file_name = "" //put inside "" file name or something
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = zip_file_path;
a.download = zip_file_name;
a.click();
document.body.removeChild(a);

As indicated in this answer, I have used the below Javascript function and now I am able to download the byte[] array content successfully.
Function to convert byte array stream (type of string) to blob object:
var b64toBlob = function(b64Data, contentType, sliceSize) {
contentType = contentType || '';
sliceSize = sliceSize || 512;
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
};
An this is how I call this function and save the blob object with FileSaver.js (getting data via Angular.js $http.get):
$http.get("your/api/uri").success(function(data, status, headers, config) {
//Here, data is type of string
var blob = b64toBlob(data, 'application/zip');
var fileName = "download.zip";
saveAs(blob, fileName);
});
Note: I am sending the byte[] array (Java-Server-Side) like this:
byte[] myByteArray = /*generate your zip file and convert into byte array*/ new byte[]();
return new ResponseEntity<byte[]>(myByteArray , headers, HttpStatus.OK);

I updated my bulkdownload method to use $window.open(...) instead of $http.get(...):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
var url = '/query/bulkdownload?';
var len = titles.length;
for ( var ii = 0; ii < len; ii++ ) {
url = url + 'titles=' + titles[ii];
if ( ii < len-1 ) {
url = url + '&';
}
}
$window.open(url);
}
};
I have only tested this in IE11.

Illegal Constructor error: convert Base64 to Blob fails on cordova

I want to convert my Base64 image to a blob in my cordova app project using AngularJS but i keep getting Illegal constructor error. I have tried a lot of the solutions given online but none seems to be working. Any help is appreciated.
var imageElement = angular.element(document.querySelector('#profileImg'));
var imageURI = dataURIToBlobURI(imageElement.attr('src'));
function dataURIToBlobURI(dataURI) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
var bb = new Blob([ab], {type: 'image/png'});
return bb;
}
I keep getting an error over here new Blob([ab], {type: 'image/png'}) and dont seem to knw how to make it work. Only happens when the app is in Android or iOS not when viewed in Chrome.
I have tried the following but all to no avail.
var bb = new Blob(ab);
var bb = new Blob([ab]);
var bb = new Blob(dataURI);
Thanks

Kingsley! Possible, device where you could reproduce the error doesn't support Blob actually. Actually you could use two ways:
Firstly, check
polyfill or smth similar to fix your problem. It will allow you to use Blob as a constructor.
Secondly, you could use BlobBuilder except of Blob. Small exmaple below,
var bb = new BlobBuilder();
bb.append('blob content');
var blob = bb.getBlob('text/plain');

I used this to solve my problem. Just incase anyone runs into this problem. All solutions didnt work for me on my device. Just follow instructions and add the javascript file and you shud be fine. https://github.com/blueimp/JavaScript-Canvas-to-Blob
var b64Data = 'R0lGODdhUAA8AIABAAAAAP///ywAAAAAUAA8AAACS4SPqcvtD6' +
'OctNqLs968+w+G4kiW5omm6sq27gvH8kzX9o3n+s73/g8MCofE' +
'ovGITCqXzKbzCY1Kp9Sq9YrNarfcrvcLDovH5PKsAAA7',
imageUrl = 'data:image/gif;base64,' + b64Data,
blob = window.dataURLtoBlob && window.dataURLtoBlob(imageUrl);

reading a binary file in javascript using browserify

I am trying to use browserify to access a local binary file (that is, the binary file is in the same directory as the javascript file, which is in the user's computer).
I haven't succeeded. Here's what I tried and what I know:
~) I know fs won't work...
0) I tried using the require('html') but it says 'ajax not supported in this browser' [I am using chromium... but I'd assume it's roughly the same thing as chrome].
1) I tried using 'browser-request'. This reads the binary file... as a string.
It is based on 'request' so I should be able to configure the options, including encoding: null, which would solve all my problems but...looking at the source code, you'll see that no support for the encoding option is present. Not even a warning.
2) I used xmlhttprequest, which required the 'html' module... so again, I get the same error as in 0) Strangely enough, 'browser-request' uses this module and it works... and I have absolutely no idea why.
3) At this point, I looked into html5 file system support. It would work but I don't want the user to specify a file... seeing as I really ONLY want to get the buffer to memory. Is there any other way to access the file? Perhaps using --allow-file-access when starting chromium?
4) If all else fails, I just want a way to get the Buffer into my code. I guess I could just use node on shell and copy paste the result of reading the file into memory...
Is there any hope at all?

Here's what somewhat works:
function toArrayBuffer(buffer) {
var ab = new ArrayBuffer(buffer.length);
var view = new Uint8Array(ab);
for (var i = 0; i < buffer.length; ++i) {
view[i] = buffer[i];
}
return ab;
}
// node: readFileSync + toArrayBuffer
// browser: ajax http request
function readFile(filename, doneCb) {
var isNode =
typeof global !== "undefined" &&
global.toString() == '[object global]';
if (isNode) {
var fs = require('fs');
var buffer = fs.readFileSync(filename);
buffer = toArrayBuffer(buffer);
doneCb(buffer);
} else {
var http = require('http');
var buf;
var req = http.get({ path : '/'+ filename }, function (res) {
res.on('data', function (chunk) {
buf = chunk;
});
res.on('end', function () {
doneCb(buf);
});
});
req.xhr.responseType = 'arraybuffer';
}
}
It requires a server and I'm strugging with on how to make it work in testling.
Another approach I can think of is to use brfs with base64 encoding:
var base64 = fs.readFileSync('file.bin', enc='base64');
var buf = new Buffer(base64, 'base64');
var ab = toArrayBuffer(buf);
It is simpler, but it is not dynamic and cannot be refactored to self-contained function.

If it's not dynamic use brfs transform.