I feel I got lack of understanding about Buffer and File Stream, but I can't find any specific idea from other answers which is treating base64 string as a actual file.
I used 'request' package from Npm, to send a file to other server by http, multipart protocol.
The code below is working well, read a file from actual file by 'fs' package and send it by ReadStream object by createReadStream method.
(The codes is coffeescript)
#working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: fs.createReadStream('/path/' + 'myfile.doc')
What I want to do is creating a same ReadStream object by fs module from a file Based64 encoded String.
I tested something like this, but it's not working properly.
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: new Buffer(base64EncodedString, 'base64')
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: _base64ToArrayBuffer(base64EncodedString)
#not working
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: convertDataURIToBinary(base64EncodedString)
##(used function)
_base64ToArrayBuffer = (base64) ->
binary_string = require('atob')(base64)
len = binary_string.length
bytes = new Uint8Array(len)
i = 0
while i < len
bytes[i] = binary_string.charCodeAt(i)
i++
bytes.buffer
convertDataURIToBinary = (dataURI) ->
BASE64_MARKER = ';base64,'
base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length
base64 = dataURI.substring(base64Index)
raw = require('atob')(base64)
rawLength = raw.length
array = new Uint8Array(new ArrayBuffer(rawLength))
i = 0
while i < rawLength
array[i] = raw.charCodeAt(i)
i++
array
question conclusion
The 'base64EncodedString' is validated by decode & creating file, so i don't doubt it is the matter, so I think I could have this achievement with write file from base64 and read it again with fs module, but I believe that's not the proper way.
The point of question is,
1. How could I send the base64encoded string as a ReadStream object properly in this case
2. How could I figure out relationship clearly between buffer and dataview(uint8array ...) something like that
Thanks in advance.
appendix
From #Alex Nikulin 's comment, I've tested stream-buffers package.
streamBuffers = require('stream-buffers')
myReadableStreamBuffer = new (streamBuffers.ReadableStreamBuffer)(
frequency: 10
chunkSize: 2048)
myReadableStreamBuffer.put base64.decode example
myReadableStreamBuffer.stop()
And when I tried bellow it failed again, and the 'AnotherUrl' returns Error message like this, 'unexpected end of part'
res = request.postSync 'http://anotherUrl/uploadDocument',
formData: file: myReadableStreamBuffer
I figured out the myReadableStreamBuffer object is Readable object, so it might be different with ReadStream object. Can I get it as a ReadStream from myReadableStreamBuffer?
Try this
//if you need just a buffer
var base64ToBuffer = function(base64) {
var byteString = new Buffer(base64, 'base64').toString('binary');
var ab = new Buffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
return ab;
}
//if you need a stream, not a buffer
var stream = require('stream');
// Initiate the source
var bufferStream = new stream.PassThrough();
// Write your buffer
bufferStream.end(base64ToBuffer(base64));
bufferStream.pipe( process.stdout );
Related
So here in C# code i am sending corgi to client which has corgiBabies. Using ClosedXml here.
var wbCorgiBabiesTemplate = new XLWorkbook();
var wsCoriBabiesAmendementTemplate = wbCorgiBabiesTemplate.Worksheets.Add(" Work Sheet Corgi baby Template");
wsCoriBabiesAmendementTemplate.Cell("A1").Value = "Corgi Parent";
wsCoriBabiesAmendementTemplate.Cell("B1").Value = "Corgi Child";
wsCoriBabiesAmendementTemplate.Cell("A2").Value = "Petunia";
wsCoriBabiesAmendementTemplate.Cell("B2").Value = "Khaleesi";
using (var ms = new MemoryStream())
{
wbCorgiBabiesTemplate.SaveAs(ms);
byte[] Corgibabies = ms.ToArray();
}
corgi.Corgibabies = System.Text.Encoding.UTF8.GetString(Corgibabies);
return corgi;
After that in Client i want to open corgibabies in excel sheet but the conversion here is wrong somewhere i think that excel sheet doesn't open correctly.
var fileName = 'CorgiBabies.xlsx';
dataAccessService.get('corgi')
.then(function(response) {
let utf8Encode = new TextEncoder();
var strBytes = utf8Encode.encode(response.corgiBabies);
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
var file = new Blob([strBytes], {type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'});
var fileURL = window.URL.createObjectURL(file);
a.href = fileURL;
a.download = fileName;
a.click();
})
Below what excel sheet gives me error in image
Assuming you're on .Net Core+ (otherwise you can find the System.Buffers Nuget package for .Net standard or framework), on server side try
using System.Buffers;
using System.Buffers.Text;
and insert
var outputBuffer = new Span<byte>();
var status = Base64.EncodeToUtf8(Corgibabies, outputBuffer, out var consumed, out var written);
// sanity check
// if (status != OperationStatus.Done) throw new Exception();`
// do the above just before replacing
// System.Text.Encoding.UTF8.GetString(Corgibabies);
// with
System.Text.Encoding.UTF8.GetString(outputBuffer);
Now I'm pretty certain that will ensure that the server responds with what the client should expect but I'm not set up to test the Javascript side of things (yet). In the meantime let me know if this helps you make progress.
PS1: the error in your original code was the implicit assumption that Corgibabies is an array containing the bytes of a UTF8 encoded string. It actually contains the raw bytes of what would normally be an .xlsx file on disk. What is needed is to make that into text (Base64 encoding) and ensure that text is UTF8. Obviously in the Javascript you need to do the reverse - UTF8 Base64 to binary, save to disk, open in Excel...
Instead of returning string as the Content, you can make it work with File.
public ActionResult Get()
{
var wbCorgiBabiesTemplate = new XLWorkbook();
var wsCoriBabiesAmendementTemplate = wbCorgiBabiesTemplate.Worksheets.Add(" Work Sheet Corgi baby Template");
wsCoriBabiesAmendementTemplate.Cell("A1").Value = "Corgi Parent";
wsCoriBabiesAmendementTemplate.Cell("B1").Value = "Corgi Child";
wsCoriBabiesAmendementTemplate.Cell("A2").Value = "Petunia";
wsCoriBabiesAmendementTemplate.Cell("B2").Value = "Khaleesi";
wbCorgiBabiesTemplate.SaveAs("new.xlsx");
var ms = new MemoryStream();
wbCorgiBabiesTemplate.SaveAs(ms);
ms.Position = 0;
var fileName = "CorgiBabies.xlsx";
return File(ms, "application/octet-stream", fileName);
}
Api call:
or
fetch('https://localhost:7135/api/downloadExcel')
.then(resp => resp.blob())
.then(blob => {
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.style.display = 'none';
a.href = url;
// the filename you want
a.download = 'CorgiBabies.xlsx';
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
})
.catch(() => alert('oh no!'));
Ref: git
ClosedXML has several extensions that will help you acheive what you need :
ClosedXML.Extensions.AspNet
ClosedXML.Extensions.Mvc
ClosedXML.Extensions.WebApi
You can install the appropriate extension for your project, to help give you a fast access to download the workbook. You can also save the file on disk, and pass the file link (path) to JavaScript, and continue your work on the file from JavaScript.
if you need to know how you would let the user download the file from ASP.NET,
then you can do this :
Simple workbook :
C#: ASP.NET MVC
[HttpGet]
public ActionResult Download(string fileName)
{
// create workbook
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Worksheet 1");
sheet.Cell("A1").Value = "A1";
sheet.Cell("B1").Value = "B1";
sheet.Cell("A2").Value = "A2";
sheet.Cell("B2").Value = "B2";
// get workbook bytes
byte[] workbookBytes;
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
workbookBytes = memoryStream.ToArray();
}
return File(workbookBytes, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", fileName);
}
C#: ASP.NET Web Forms
public void Export(HttpResponse response, string fileName)
{
// create workbook
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Worksheet 1");
sheet.Cell("A1").Value = "A1";
sheet.Cell("B1").Value = "B1";
sheet.Cell("A2").Value = "A2";
sheet.Cell("B2").Value = "B2";
HttpResponse httpResponse = response;
httpResponse.Clear();
httpResponse.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
httpResponse.AddHeader("content-disposition", $"attachment;filename=\"{fileName}.xlsx\"");
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
memoryStream.WriteTo(httpResponse.OutputStream);
}
httpResponse.End();
}
the above examples will directly download the file into the client device. However, if you want to pass the workbook bytes to the JavaScript, you will need to convert it to base64 string and pass it to the JavaScript like so :
var base64String = Convert.ToBase64String(workbookBytes);
Then from JavaScript decode it to Uint8Array :
/*
JavaScript
*/
// get base64 string array and decoded it
var data = atob(serverSideResult);
var array = new Array(data.length);
for (var i = 0; i < data.length; i++) {
array[i] = data.charCodeAt(i);
}
// final result
var dataUint8Array = new Uint8Array(array);
now you can work with dataUint8Array as normal Uint8Array.
if you want to pass it back to the server-side, you can convert the array to base64 string, and pass it to the server-side like so :
/*
JavaScript
*/
let binaryString = ''
for (var i = 0; i < dataUint8Array.byteLength; i++) {
binaryString += String.fromCharCode(dataUint8Array[i]);
}
//pass base64Result to the server-side (C#)
var base64Result = window.btoa(binaryString);
then from C# you just need to convert it back to array from base64 string like so :
var bytes = Convert.FromBase64String(dataReceivedFromJavaScript);
where bytes would be byte[].
Hi I'm trying to move the bytes of my video which is in c# to javascript to make the bytes into URL.createObjectURL on Blazor server-side
I moved the bytes using Js Invoke
.cs
if (!string.IsNullOrEmpty(item.PathFile))
{
//Byte Video
byte[] result = GetFile(item.PathFile);
if (result != null)
{
var url = await Js.InvokeAsync<string>("videoUrl", result);
data.ImageString = url;
}
}
.js
function videoUrl(value) {
var byteCharacters = atob(value);
var byteNumbers = new Array(byteCharacters.length);
for (var i = 0; i < byteCharacters.length; i++) {
byteNumbers[i] = byteCharacters.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
//Byte Array -> Blob
var file = new Blob([byteArray], { type: 'data:video/mp4;base64' });
//Blob -> Object URL
var fileURL = URL.createObjectURL(file);
return fileURL;
}
My problem is, I tried this script for a video with a size of 3 Mb it runs fine, but when I try for a 133Mb video I get an error:
Error: System.ArgumentException: The JSON value of length 139569235 is too large and not supported.
I've tried to fix it, but it still fails, it makes me a little frustrated
So is there a solution for my error ? or what should I do?
I thank you for any suggestions or feedback
So, reading on the AspNetDocs github, there is a startup option that can change the max message size, but I think it only applies to calls from JS to .Net (https://github.com/dotnet/AspNetCore.Docs/issues/21208). Worth a check though.
services.AddServerSideBlazor()
.AddHubOptions(options => options.MaximumReceiveMessageSize = 32000);
Personally though, I would do as Mister Magoo said in the comment and either use an API or chunk the data and reassemble at the other end.
I need to modify existing frontend (angular) code that involves uploading files to a server. Now the files need to be encrypted before being uploaded.
The current approach uses FormData to append a number of files and send them in a single request as shown below:
function uploadFiles(wrappers){
var data = new FormData();
// Add each file
for(var i = 0; i < wrappers.length; i++){
var wrapper = wrappers[i];
var file = wrapper.file;
data.append('file_' + i, file);
}
$http.post(uri, data, requestCfg).then(
/*...*
I have been using Forge in other projects, but never in this sort of context and don't really see how to encrypt files on the fly and still append them as FormData contents.
Forge provides an easy API:
var key = forge.random.getBytesSync(16);
var iv = forge.random.getBytesSync(8);
// encrypt some bytes
var cipher = forge.rc2.createEncryptionCipher(key);
cipher.start(iv);
cipher.update(forge.util.createBuffer(someBytes));
cipher.finish();
var encrypted = cipher.output;
The backend recieves files using Formidable and all the file hanlding is already wired. I would thus like to stick to using the existing front-end logic but simply insert the encryption logic. In that, it's not the entire formdata that must be encrypted... I haven't found a good lead yet to approach this.
Suggestions are very welcome!
Ok, found a solution and added the decrypt code as well. This adds a layer of async code.
function appendFile(aFile, idx){
// Encrypt if a key was provided for this protocol test
if(!key){
data.append('dicomfile_' + idx, file);
appendedCount++;
onFileAppended();
}
else{
var reader = new FileReader();
reader.onload = function(){
// 1. Read bytes
var arrayBuffer = reader.result;
var bytes = new Uint8Array(arrayBuffer); // byte array aka uint8
// 2. Encrypt
var cipher = forge.cipher.createCipher('AES-CBC', key);
cipher.start({iv: iv});
cipher.update(forge.util.createBuffer(bytes));
cipher.finish();
// 3. To blob (file extends blob)
var encryptedByteCharacters = cipher.output.getBytes(); // encryptedByteCharacters is similar to an ATOB(b64) output
// var asB64 = forge.util.encode64(encryptedBytes);
// var encryptedByteCharacters = atob(asB64);
// Convert to Blob object
var blob = byteCharsToBlob(encryptedByteCharacters, "application/octet-stream", 512);
// 4. Append blob
data.append('dicomfile_' + idx, blob, file.name);
// Decrypt for the sake of testing
if(true){
var fileReader = new FileReader();
fileReader.onload = function() {
arrayBuffer = this.result;
var bytez = new Uint8Array(arrayBuffer);
var decipher = forge.cipher.createDecipher('AES-CBC', key);
decipher.start({iv: iv});
decipher.update(forge.util.createBuffer(bytez));
decipher.finish();
var decryptedByteCharacters = decipher.output.getBytes();
var truz = bytes === decryptedByteCharacters;
var blob = byteCharsToBlob(decryptedByteCharacters, "application/octet-stream", 512);
data.append('decrypted_' + idx, blob, file.name + '.decrypted');
appendedCount++;
onFileAppended();
};
fileReader.readAsArrayBuffer(blob);
}
else{
// z. Resume processing
appendedCount++;
onFileAppended();
}
}
// Read file
reader.readAsArrayBuffer(aFile);
}
}
function onFileAppended(){
// Only proceed when all files were appended and optionally encrypted (async)
if(appendedCount !== wrappers.length) return;
/* resume processing, upload or do whathever */
Before somebody says, "duplicate", I just want to make sure, that folks know, that I have already reviewed these questions:
1) Uses angular and php, not sure what is happening here (I don't know PHP): Download zip file and trigger "save file" dialog from angular method
2) Can't get this answer to do anything: how to download a zip file using angular
3) This person can already download, which is past the point I'm trying to figure out:
Download external zip file from angular triggered on a button action
4) No answer for this one:
download .zip file from server in nodejs
5) I don't know what language this even is:
https://stackoverflow.com/questions/35596764/zip-file-download-using-angularjs-directive
Given those questions, if this is still a duplicate, I apologize. Here is, yet, another version of this question.
My angular 1.5.X client gives me a list of titles, of which each have an associated file. My Node 4.X/Express 4.X server takes that list, gets the file locations, creates a zip file, using express-zip from npm, and then streams that file back in the response. I then want my client to initiate the browser's "download a file" option.
Here's my client code (Angular 1.5.X):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
$http.get('/query/bulkdownload',{
params:{titles:titles},
responseType:'arraybuffer'
})
.then(successCb,errorCb)
.catch(exceptionCb);
}
function successCb(response){
// This is the part I believe I cannot get to work, my code snippet is below
};
function errorCb(error){
alert('Error: ' + JSON.stringify(error));
};
function exceptionCb(ex){
alert('Exception: ' + JSON.stringify(ex));
};
};
Node (4.X) code with express-zip, https://www.npmjs.com/package/express-zip:
router.get('/bulkdownload',function(req,resp){
var titles = req.query.titles || [];
if ( titles.length > 0 ){
utils.getFileLocations(titles).
then(function(files){
let filename = 'zipfile.zip';
// .zip sets Content-Type and Content-disposition
resp.zip(files,filename,console.log);
},
_errorCb)
}
});
Here's my successCb in my client code (Angular 1.5.X):
function successCb(response){
var URL = $window.URL || $window.webkitURL || $window.mozURL || $window.msURL;
if ( URL ) {
var blob = new Blob([response.data],{type:'application/zip'});
var url = URL.createObjectURL(blob);
$window.open(url);
}
};
The "blob" part seems to work fine. Checking it in IE's debugger, it does look like a file stream of octet information. Now, I believe I need to get that blob into the some HTML5 directive, to initiate the "Save File As" from the browser. Maybe? Maybe not?
Since 90%+ of our users are using IE11, I test all of my angular in PhantomJS (Karma) and IE. When I run the code, I get the old "Access is denied" error in an alert window:
Exception: {"description":"Access is denied...<stack trace>}
Suggestions, clarifications, answers, etc. are welcome!
Use this one:
var url="YOUR ZIP URL HERE";
window.open(url, '_blank');
var zip_file_path = "" //put inside "" your path with file.zip
var zip_file_name = "" //put inside "" file name or something
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = zip_file_path;
a.download = zip_file_name;
a.click();
document.body.removeChild(a);
As indicated in this answer, I have used the below Javascript function and now I am able to download the byte[] array content successfully.
Function to convert byte array stream (type of string) to blob object:
var b64toBlob = function(b64Data, contentType, sliceSize) {
contentType = contentType || '';
sliceSize = sliceSize || 512;
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
};
An this is how I call this function and save the blob object with FileSaver.js (getting data via Angular.js $http.get):
$http.get("your/api/uri").success(function(data, status, headers, config) {
//Here, data is type of string
var blob = b64toBlob(data, 'application/zip');
var fileName = "download.zip";
saveAs(blob, fileName);
});
Note: I am sending the byte[] array (Java-Server-Side) like this:
byte[] myByteArray = /*generate your zip file and convert into byte array*/ new byte[]();
return new ResponseEntity<byte[]>(myByteArray , headers, HttpStatus.OK);
I updated my bulkdownload method to use $window.open(...) instead of $http.get(...):
function bulkdownload(titles){
titles = titles || [];
if ( titles.length > 0 ) {
var url = '/query/bulkdownload?';
var len = titles.length;
for ( var ii = 0; ii < len; ii++ ) {
url = url + 'titles=' + titles[ii];
if ( ii < len-1 ) {
url = url + '&';
}
}
$window.open(url);
}
};
I have only tested this in IE11.
I want to convert my Base64 image to a blob in my cordova app project using AngularJS but i keep getting Illegal constructor error. I have tried a lot of the solutions given online but none seems to be working. Any help is appreciated.
var imageElement = angular.element(document.querySelector('#profileImg'));
var imageURI = dataURIToBlobURI(imageElement.attr('src'));
function dataURIToBlobURI(dataURI) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
var bb = new Blob([ab], {type: 'image/png'});
return bb;
}
I keep getting an error over here new Blob([ab], {type: 'image/png'}) and dont seem to knw how to make it work. Only happens when the app is in Android or iOS not when viewed in Chrome.
I have tried the following but all to no avail.
var bb = new Blob(ab);
var bb = new Blob([ab]);
var bb = new Blob(dataURI);
Thanks
Kingsley! Possible, device where you could reproduce the error doesn't support Blob actually. Actually you could use two ways:
Firstly, check
polyfill or smth similar to fix your problem. It will allow you to use Blob as a constructor.
Secondly, you could use BlobBuilder except of Blob. Small exmaple below,
var bb = new BlobBuilder();
bb.append('blob content');
var blob = bb.getBlob('text/plain');
I used this to solve my problem. Just incase anyone runs into this problem. All solutions didnt work for me on my device. Just follow instructions and add the javascript file and you shud be fine. https://github.com/blueimp/JavaScript-Canvas-to-Blob
var b64Data = 'R0lGODdhUAA8AIABAAAAAP///ywAAAAAUAA8AAACS4SPqcvtD6' +
'OctNqLs968+w+G4kiW5omm6sq27gvH8kzX9o3n+s73/g8MCofE' +
'ovGITCqXzKbzCY1Kp9Sq9YrNarfcrvcLDovH5PKsAAA7',
imageUrl = 'data:image/gif;base64,' + b64Data,
blob = window.dataURLtoBlob && window.dataURLtoBlob(imageUrl);