I want to read a slice of a file using a FileReader, and then send it to a server. This is what I have so far:
const createReader = onRead => {
const reader = new FileReader();
reader.onloadend = function(evt) {
if (evt.target.readyState === FileReader.DONE) {
const arrayBuffer = evt.target.result;
const bytes = new Uint8Array(arrayBuffer);
console.log("BYTES: ", bytes);
onRead(evt.target.result);
}
};
return reader;
};
const reader = createReader(fileSlice => {
console.log("BYTES: ", fileSlice);
// send to server
});
reader.readAsArrayBuffer(blob);
Here's what it printed when I uploaded a simple .txt file:
However, it returns an array, and I'm not sure how to convert this into a format that I can send over to my server through HTTP.
Does anyone know what I need to do in order to convert that byte array into a format that I can eventually turn back into the original file?
You can POST the Uint8Array to the server. You can convert a Uint8Array to a string using TextDecoder
Read the file chunk, make sure you use readAsArrayBuffer
var readChunck = function (offset, length, file) {
var r = new FileReader();
var chunck = file.slice(offset, length + offset);
r.onload = readEventHandler;
r.readAsArrayBuffer(chunck)
}
In the vent handler (readEventHandler), convert the ArrayBuffer to Base64String and post to the server:
var readEventHandler = function (evt) {
var data = {
data: self.arrayBufferToBase64String(evt.target.result)
}
self.httpClient.post(url, data)
.subscribe(data=> {
// do stuff
});
// ...
}
And here is the conversion function:
private arrayBufferToBase64String(buffer: ArrayBuffer) {
let binaryString = ''
var bytes = new Uint8Array(buffer);
for (var i = 0; i < bytes.byteLength; i++) {
binaryString += String.fromCharCode(bytes[i]);
}
return window.btoa(binaryString);
}
On the server side (ASP Core Web API in this case):
[HttpPost, DisableRequestSizeLimit]
[Route("{blobId}/uploads/{uploadId}/parts/{partNumber}")]
public async Task<IActionResult> UploadPart(
[FromRoute] string blobId,
[FromRoute] string uploadId,
[FromRoute] int partNumber,
[FromBody] FilePartModel part)
{
if (!GuidValidator.IsValidGuid(blobId)) throw new ArgumentException("Invalid BLOB Id");
if (!GuidValidator.IsValidGuid(uploadId)) throw new ArgumentException("Invalid Upload Id");
var bytes = Convert.FromBase64String(part.Data);
using (var stream = new MemoryStream(bytes))
{
var etag = await _blobsService.UploadPartAsync(Guid.Parse(blobId), Guid.Parse(uploadId), partNumber, stream);
return Ok(new FilePart { ETag = etag, PartNumber = partNumber });
}
}
FilePartModel is just a simple model class:
public class FilePartModel
{
public string Data { get; set; }
}
Related
I am working on an application that [is trying to] optimize storing images and videos in the AWS while also considering retrieval of the data.
I do this in 2 steps: A. Store the data into AWS & B. Retrieve the data from AWS.
The current problem I am trying to solve exists in B: Step 3. I cannot correctly decode Base64 into a Zip File.
A. I store the data in a 6 step process to ensure that the data is compressed as much as possible (cost-savings) and encrypted so only the user can access it (encryption).
Attain images / videos from user
Reason: User Data Selection
import { launchImageLibrary } from 'react-native-image-picker';
launchImageLibrary(options, (response) => {
if(response.assets) {
const assets = response.assets;
completeCallback(assets);
}
else if(response.errorCode) {
const message = response.errorMessage;
console.log(message);
}
else {
cancelCallback();
}
});
Store images from original location into Document Directory
Reason: Consolidated Location to Zip
static async copyFilesToDocumentDirectory(urls) {
var path = `${RNFS.DocumentDirectoryPath}/media`;
let existDir = await RNFS.exists(RNFS.DocumentDirectoryPath + '/media').then(boolean => boolean);
if(!existDir) { RNFS.mkdir(path); };
return Promise.all(urls.map(url => {
return RNFS.exists(url)
.then((status) => {
if(status) {
var filename = url.substring(url.lastIndexOf('/')+1);
var newPath = path + "/" + filename;
return RNFS.copyFile(url, newPath);
};
});
}));
};
Zip up documents
Reason: Should reduce file size but isn't necessarily in every case
static zipDocumentDirectory = async () => {
var sourcePath = `${RNFS.DocumentDirectoryPath}/media`;
var targetPath = `${RNFS.DocumentDirectoryPath}/myFile.zip`;
return await zip(sourcePath, targetPath);
}
Encode Zip as base64
Encode data so I can encrypt it
AWS.getPresignedURLPut(DBKey, passCode,
async (url) => {
await RNFS.readFile(zipURI, 'base64')
.then(res => {
// Next Section
...
}, (err) => {
console.log(`Err: ${err}`)
}
);
Encrypt base64
Reason: E2E Encryption
const encryptedData = CryptoJS.AES.encrypt(res, ENKey).toString();
const json =
{
'DBKey': DBKey,
'PassCode': passCode,
'base64data': encryptedData
};
// Store data in Next Section
Store in Database
Reason: Database Storage
AWS.storeJSONToAWSS3PresignedURL(json, url, onComplete);
B. I retrieve the data in the same manner, just the reverse order.
Retrieve Encrypted Base64 from Database
Reason: Get Original Data
AWS.getJSONFromAWSS3PresignedURL(url); //returns Base64 String
Decrypt to Original Base64
Reason: Need to Attain Original Base64
/* Decryption */
const bytes = CryptoJS.AES.decrypt(data, endToEndEncryptionKey)
originalBase64 = bytes.toString(CryptoJS.enc.Utf8);
Decode Base64 to Zip File PROBLEM
Reason: Need the original Zip File Containing Videos/Photos
static writeBase64ToDocumentDirectory = async (base64) => {
var path = `${RNFS.DocumentDirectoryPath}/myFile.zip`;
const blob = b64toBlob(base64);
const file = new File([blob], path, { type: 'application/x-zip-compressed' });
console.log(`F:`);
console.log(file);
console.log(`B:`);
console.log(blob);
};
const b64toBlob = (b64Data, contentType='', sliceSize=512) => {
const byteCharacters = global.atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, {type: contentType});
return blob;
}
/*
F:
{"_data": {"__collector": {}, "blobId": "9341bc78-11e7-428c-96c4-e761dc47f75a", "lastModified": undefined, "name": "/var/mobile/Containers/Data/Application/986B087C-A19A-4DDA-9180-831B4EDE89B9/Documents/myFile.zip", "offset": 0, "size": 1948698, "type": "application/x-zip-compressed"}}
B:
{"_data": {"__collector": {}, "blobId": "66b4e283-5a26-4684-a6f7-49f3615c8509", "lastModified": undefined, "offset": 0, "size": 1948698, "type": ""}}
*/
Last step is to unzip the file but it doesn't exist.
await RNFS.readDir(`${RNFS.DocumentDirectoryPath}`)
.then((result) => {
console.log(`Zip Path: ${result} -- ${result.length}`);
console.log(result);
});
/*
Zip Path: -- 0
[]
*/
PROBLEM:
I cannot seem to find a way to convert the base64 to a decoded zip file.
Working Validation:
I know this can work as I can decode the base64 using this website. But, I cannot reproduce the code to do the same local to the device.
const b64toBlob = (b64Data, contentType='', sliceSize=512) => {
const byteCharacters = global.atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, {type: contentType});
return blob;
};
After you have converted the Base64 string to a Blob, you can write the Blob to a file using the File constructor and the writeFile method from the react-native-fs library.
I have an encoded base64 data from an API response and stored in a variable encodedBase64.
let encodedBase64 = 'some base64 encoded long data';
function base64ToArrayBuffer(base64) {
var binaryString = window.atob(base64);
console.log('binaryString ', binaryString);
var binaryLen = binaryString.length;
var bytes = new Uint8Array(binaryLen);
for (var i = 0; i < binaryLen; i++) {
var ascii = binaryString.charCodeAt(i);
bytes[i] = ascii;
}
return bytes;
}
function saveByteArray(reportName, byte) {
var blob = new Blob([byte], {type: "application/pdf"});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
var fileName = reportName;
link.download = fileName;
link.click();
};
var sampleArr = base64ToArrayBuffer(encodedBase64);
saveByteArray("Sample Report", sampleArr);
after executing this code i am able to download pdf file names SampleReport.pdf but when i open this it is showing Failed to load PDF document. error
what is the wrong in my code ?
difficult to get it done using front end side
but it can be done easily using Nodejs using the following code.
fs.writeFile('pdfFileName.pdf', base64DataString, {encoding: 'base64'}, error => {
if (error) {
throw error;
} else {
console.log('buffer saved!');
}
});
I've got an array of URLs [URL1, URL2, URL3,...] : each element is a link to one of the chunks of the same file. Each chunk is separately encrypted, with the same key as all the other chunks.
I download each chunk (in a forEach function) with a XMLHttpRequest. onload :
each chunk is first decrypted
then each chunk is converted to an ArrayBuffer (source)
each ArrayBuffer is pushed to an array (source)
when the three first steps are done for each chunk (callback by a var incremented on step#1 === the array.length), a blob is constructed with the array
the blob is saved as file with FileReader API & filesaver.js
If it's a one chunk's file, everything works fine.
But with multiple chunks, steps #1 & #2 are ok, but only the last ArrayBuffer seems to be pushed to the array. What am I missing?
Below my code
// var for incrementation in forEach funtion
var chunkdownloaded = 0;
// 'clearfileurl' is the array of url's chunks :[URL1, URL2, URL3,...]
clearfileurl.forEach(function(entry) {
var xhr = new XMLHttpRequest();
var started_at = new Date();
xhr.open('GET', entry, true);
xhr.responseType = 'text';
// request progress
xhr.onprogress = function(pe) {
if (pe.lengthComputable) {
downloaderval.set((pe.loaded / pe.total) * 100);
}
};
// on request's success
xhr.onload = function(e) {
if (this.status == 200) {
chunkdownloaded+=1;
var todecrypt = this.response;
// decrypt request's response: get a dataURI
try {
var bytesfile = CryptoJS.AES.decrypt(todecrypt.toString(), userKey);
var decryptedfile = bytesfile.toString(CryptoJS.enc.Utf8);
} catch(err) {
console.log (err);
return false;
}
//convert a dataURI to a Blob
var MyBlobBuilder = function() {
this.parts = [];
}
MyBlobBuilder.prototype.append = function(dataURI) {
//function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
// var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
this.parts.push(ab);
console.log('parts', this.parts)
this.blob = undefined; // Invalidate the blob
}
MyBlobBuilder.prototype.getBlob = function() {
if (!this.blob) {
console.log (this.parts);
this.blob = new Blob(this.parts);
}
return this.blob;
};
var myBlobBuilder = new MyBlobBuilder();
myBlobBuilder.append(decryptedfile);
// if all chunks are downloaded
if (chunkdownloaded === clearfileurl.length) {
// get the blob
var FinalFile = myBlobBuilder.getBlob();
// launch consturction of a file with'FinalFile' inside FileReader API
var reader = new FileReader();
reader.onload = function(e){
// build & save on client the final file with 'file-saver' library
var FileSaver = require('file-saver');
var file = new File([FinalFile], clearfilename, {type: clearfiletype});
FileSaver.saveAs(file);
};
reader.readAsText(FinalFile);
} else {
console.log('not yet');
}
}
};
// sending XMLHttpRequest
xhr.send();
});
You need to take out the declaration of MyBlobBuilder, try this:
// var for incrementation in forEach funtion
var chunkdownloaded = 0;
//convert a dataURI to a Blob
var MyBlobBuilder = function() {
this.parts = [];
}
MyBlobBuilder.prototype.append = function(dataURI, index) {
//function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
// var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
this.parts[index] = ab;
console.log('parts', this.parts)
this.blob = undefined; // Invalidate the blob
}
MyBlobBuilder.prototype.getBlob = function() {
if (!this.blob) {
console.log (this.parts);
this.blob = new Blob(this.parts);
}
return this.blob;
};
var myBlobBuilder = new MyBlobBuilder();
// 'clearfileurl' is the array of url's chunks :[URL1, URL2, URL3,...]
clearfileurl.forEach(function(entry, index) {
var xhr = new XMLHttpRequest();
var started_at = new Date();
xhr.open('GET', entry, true);
xhr.responseType = 'text';
// request progress
xhr.onprogress = function(pe) {
if (pe.lengthComputable) {
downloaderval.set((pe.loaded / pe.total) * 100);
}
};
// on request's success
xhr.onload = function(e) {
if (this.status == 200) {
chunkdownloaded+=1;
var todecrypt = this.response;
// decrypt request's response: get a dataURI
try {
var bytesfile = CryptoJS.AES.decrypt(todecrypt.toString(), userKey);
var decryptedfile = bytesfile.toString(CryptoJS.enc.Utf8);
} catch(err) {
console.log (err);
return false;
}
myBlobBuilder.append(decryptedfile, index);
// if all chunks are downloaded
if (chunkdownloaded === clearfileurl.length) {
// get the blob
var FinalFile = myBlobBuilder.getBlob();
// launch consturction of a file with'FinalFile' inside FileReader API
var reader = new FileReader();
reader.onload = function(e){
// build & save on client the final file with 'file-saver' library
var FileSaver = require('file-saver');
var file = new File([FinalFile], clearfilename, {type: clearfiletype});
FileSaver.saveAs(file);
};
reader.readAsText(FinalFile);
} else {
console.log('not yet');
}
}
};
// sending XMLHttpRequest
xhr.send();
});
*edit I also updated the append function to ensure that the files are in the correct order
I try to convert a file that i get through an input file into a byte[].
I tried with a FileReader, but i must miss something :
var bytes = [];
var reader = new FileReader();
reader.onload = function () {
bytes = reader.result;
};
reader.readAsArrayBuffer(myFile);
But in the end, my bytes var doesn't content a byte array.
I saw this post : Getting byte array through input type = file but it doesn't ends with a byte[], and readAsBinaryString() is deprecated
What do i miss?
Faced a similar problem and its true the 'reader.result' doesn't end up as 'byte[]'. So I have cast it to Uint8Array object. This too is not a perfect 'byte[]' ,so I had to create a 'byte[]' from it. Here is my solution to this problem and it worked well for me.
var reader = new FileReader();
var fileByteArray = [];
reader.readAsArrayBuffer(myFile);
reader.onloadend = function (evt) {
if (evt.target.readyState == FileReader.DONE) {
var arrayBuffer = evt.target.result,
array = new Uint8Array(arrayBuffer);
for (var i = 0; i < array.length; i++) {
fileByteArray.push(array[i]);
}
}
}
'fileByteArray' is what you are looking for. Saw the comments and seems you did the same, still wanted to share the approach.
Seems to me you just want to get files into an array? How about these functions - one where you can read it as text, another as a base64 byte string, and if you really want the readAsArrayBuffer array buffer output, I've included that, too:
document.getElementById("myBtn").addEventListener("click", function() {
uploadFile3();
});
var fileByteArray = [];
function uploadFile1(){
var files = myInput.files[0];
var reader = new FileReader();
reader.onload = processFile(files);
reader.readAsText(files);
}
function uploadFile2(){
var files = document.querySelector('input').files[0];
var reader = new FileReader();
reader.onload = processFile(files);
reader.readAsDataURL(files);
}
function uploadFile3(){
var files = myInput.files[0];
var reader = new FileReader();
reader.onload = processFile(files);
reader.readAsArrayBuffer(files);
}
function processFile(theFile){
return function(e) {
var theBytes = e.target.result; //.split('base64,')[1]; // use with uploadFile2
fileByteArray.push(theBytes);
document.getElementById('file').innerText = '';
for (var i=0; i<fileByteArray.length; i++) {
document.getElementById('file').innerText += fileByteArray[i];
}
}
}
<input id="myInput" type="file">
<button id="myBtn">Try it</button>
<span id="file"></span>
this works very well for me in React JS:
const handleUpload = async (e) => {
let image = e.currentTarget.files[0];
const buffer = await image.arrayBuffer();
let byteArray = new Int8Array(buffer);
console.log(byteArray)
formik.setFieldValue(name, byteArray);
}
Here is a modified, and in my opinion easier version of the accepted answer. This function returns a Promise with a value of the byte[].
function fileToByteArray(file) {
return new Promise((resolve, reject) => {
try {
let reader = new FileReader();
let fileByteArray = [];
reader.readAsArrayBuffer(file);
reader.onloadend = (evt) => {
if (evt.target.readyState == FileReader.DONE) {
let arrayBuffer = evt.target.result,
array = new Uint8Array(arrayBuffer);
for (byte of array) {
fileByteArray.push(byte);
}
}
resolve(fileByteArray);
}
}
catch (e) {
reject(e);
}
})
}
This way you can simply call this function in an async function like this
async function getByteArray() {
//Get file from your input element
let myFile = document.getElementById('myFileInput').files[0];
//Wait for the file to be converted to a byteArray
let byteArray = await fileToByteArray(myFile);
//Do something with the byteArray
console.log(byteArray);
}
I'm trying to draw photo with correct orientation in canvas after capture photo by using input[type='file'] in mobile web browser for that I'm using:
fileReader.onloadend = function() {
var exif = EXIF.readFromBinaryFile(new BinaryFile(this.result));
switch(exif.Orientation){
case 8:
ctx.rotate(90*Math.PI/180);
break;
case 3:
ctx.rotate(180*Math.PI/180);
break;
case 6:
ctx.rotate(-90*Math.PI/180);
break;
}
};
But I get: TypeError: First argument to DataView constructor must be an ArrayBuffer?
How can I get this array buffer?
I'm using EXIF.js and BinaryFile.js
You need to convert the base64 string to an ArrayBuffer for ExifJs:
function base64ToArrayBuffer (base64) {
base64 = base64.replace(/^data\:([^\;]+)\;base64,/gmi, '');
var binaryString = atob(base64);
var len = binaryString.length;
var bytes = new Uint8Array(len);
for (var i = 0; i < len; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return bytes.buffer;
}
You don't need BinaryFile:
var exif = EXIF.readFromBinaryFile(base64ToArrayBuffer(this.result));
This assumes you are using FileReader with readAsDataURL to get this.result.
A better approach would be to read the file as an array buffer to begin with and not convert it to base64 and then back again using FileReader.readAsArrayBuffer(). Something along the lines of this (pseudocode):
// `file` = files[0] from input change event
function getFileArrayBuffer(file) {
return new Promise(function (resolve, reject) {
var reader = new FileReader();
reader.onload = function() {
resolve(new Uint8Array(reader.result));
}
reader.readAsArrayBuffer(file);
});
}
#chings228 you must pass base64 data to base64ToArrayBuffer, not blob data.
function base64ToArrayBuffer (base64) {
base64 = base64.replace(/^data\:([^\;]+)\;base64,/gmi, '');
var binaryString = atob(base64);
var len = binaryString.length;
var bytes = new Uint8Array(len);
for (var i = 0; i < len; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return bytes.buffer;
}
var b64 = "data:image/jpeg;base64,"+$parameters.image;
var exif = EXIF.readFromBinaryFile(base64ToArrayBuffer(b64));
alert(exif.Orientation);