Develop Reference

sending string from C# to client and converting into Uint8Array type byte array and then into blob to open excel file. Corgi Involved

So here in C# code i am sending corgi to client which has corgiBabies. Using ClosedXml here.
var wbCorgiBabiesTemplate = new XLWorkbook();
var wsCoriBabiesAmendementTemplate = wbCorgiBabiesTemplate.Worksheets.Add(" Work Sheet Corgi baby Template");
wsCoriBabiesAmendementTemplate.Cell("A1").Value = "Corgi Parent";
wsCoriBabiesAmendementTemplate.Cell("B1").Value = "Corgi Child";
wsCoriBabiesAmendementTemplate.Cell("A2").Value = "Petunia";
wsCoriBabiesAmendementTemplate.Cell("B2").Value = "Khaleesi";
using (var ms = new MemoryStream())
{
wbCorgiBabiesTemplate.SaveAs(ms);
byte[] Corgibabies = ms.ToArray();
}
corgi.Corgibabies = System.Text.Encoding.UTF8.GetString(Corgibabies);
return corgi;
After that in Client i want to open corgibabies in excel sheet but the conversion here is wrong somewhere i think that excel sheet doesn't open correctly.
var fileName = 'CorgiBabies.xlsx';
dataAccessService.get('corgi')
.then(function(response) {
let utf8Encode = new TextEncoder();
var strBytes = utf8Encode.encode(response.corgiBabies);
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
var file = new Blob([strBytes], {type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'});
var fileURL = window.URL.createObjectURL(file);
a.href = fileURL;
a.download = fileName;
a.click();
})
Below what excel sheet gives me error in image

Assuming you're on .Net Core+ (otherwise you can find the System.Buffers Nuget package for .Net standard or framework), on server side try
using System.Buffers;
using System.Buffers.Text;
and insert
var outputBuffer = new Span<byte>();
var status = Base64.EncodeToUtf8(Corgibabies, outputBuffer, out var consumed, out var written);
// sanity check
// if (status != OperationStatus.Done) throw new Exception();`
// do the above just before replacing
// System.Text.Encoding.UTF8.GetString(Corgibabies);
// with
System.Text.Encoding.UTF8.GetString(outputBuffer);
Now I'm pretty certain that will ensure that the server responds with what the client should expect but I'm not set up to test the Javascript side of things (yet). In the meantime let me know if this helps you make progress.
PS1: the error in your original code was the implicit assumption that Corgibabies is an array containing the bytes of a UTF8 encoded string. It actually contains the raw bytes of what would normally be an .xlsx file on disk. What is needed is to make that into text (Base64 encoding) and ensure that text is UTF8. Obviously in the Javascript you need to do the reverse - UTF8 Base64 to binary, save to disk, open in Excel...

Instead of returning string as the Content, you can make it work with File.
public ActionResult Get()
{
var wbCorgiBabiesTemplate = new XLWorkbook();
var wsCoriBabiesAmendementTemplate = wbCorgiBabiesTemplate.Worksheets.Add(" Work Sheet Corgi baby Template");
wsCoriBabiesAmendementTemplate.Cell("A1").Value = "Corgi Parent";
wsCoriBabiesAmendementTemplate.Cell("B1").Value = "Corgi Child";
wsCoriBabiesAmendementTemplate.Cell("A2").Value = "Petunia";
wsCoriBabiesAmendementTemplate.Cell("B2").Value = "Khaleesi";
wbCorgiBabiesTemplate.SaveAs("new.xlsx");
var ms = new MemoryStream();
wbCorgiBabiesTemplate.SaveAs(ms);
ms.Position = 0;
var fileName = "CorgiBabies.xlsx";
return File(ms, "application/octet-stream", fileName);
}
Api call:
or
fetch('https://localhost:7135/api/downloadExcel')
.then(resp => resp.blob())
.then(blob => {
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.style.display = 'none';
a.href = url;
// the filename you want
a.download = 'CorgiBabies.xlsx';
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
})
.catch(() => alert('oh no!'));
Ref: git

ClosedXML has several extensions that will help you acheive what you need :
ClosedXML.Extensions.AspNet
ClosedXML.Extensions.Mvc
ClosedXML.Extensions.WebApi
You can install the appropriate extension for your project, to help give you a fast access to download the workbook. You can also save the file on disk, and pass the file link (path) to JavaScript, and continue your work on the file from JavaScript.
if you need to know how you would let the user download the file from ASP.NET,
then you can do this :
Simple workbook :
C#: ASP.NET MVC
[HttpGet]
public ActionResult Download(string fileName)
{
// create workbook
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Worksheet 1");
sheet.Cell("A1").Value = "A1";
sheet.Cell("B1").Value = "B1";
sheet.Cell("A2").Value = "A2";
sheet.Cell("B2").Value = "B2";
// get workbook bytes
byte[] workbookBytes;
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
workbookBytes = memoryStream.ToArray();
}
return File(workbookBytes, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", fileName);
}
C#: ASP.NET Web Forms
public void Export(HttpResponse response, string fileName)
{
// create workbook
var workbook = new XLWorkbook();
var sheet = workbook.Worksheets.Add("Worksheet 1");
sheet.Cell("A1").Value = "A1";
sheet.Cell("B1").Value = "B1";
sheet.Cell("A2").Value = "A2";
sheet.Cell("B2").Value = "B2";
HttpResponse httpResponse = response;
httpResponse.Clear();
httpResponse.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
httpResponse.AddHeader("content-disposition", $"attachment;filename=\"{fileName}.xlsx\"");
using (var memoryStream = new MemoryStream())
{
workbook.SaveAs(memoryStream);
memoryStream.WriteTo(httpResponse.OutputStream);
}
httpResponse.End();
}
the above examples will directly download the file into the client device. However, if you want to pass the workbook bytes to the JavaScript, you will need to convert it to base64 string and pass it to the JavaScript like so :
var base64String = Convert.ToBase64String(workbookBytes);
Then from JavaScript decode it to Uint8Array :
/*
JavaScript
*/
// get base64 string array and decoded it
var data = atob(serverSideResult);
var array = new Array(data.length);
for (var i = 0; i < data.length; i++) {
array[i] = data.charCodeAt(i);
}
// final result
var dataUint8Array = new Uint8Array(array);
now you can work with dataUint8Array as normal Uint8Array.
if you want to pass it back to the server-side, you can convert the array to base64 string, and pass it to the server-side like so :
/*
JavaScript
*/
let binaryString = ''
for (var i = 0; i < dataUint8Array.byteLength; i++) {
binaryString += String.fromCharCode(dataUint8Array[i]);
}
//pass base64Result to the server-side (C#)
var base64Result = window.btoa(binaryString);
then from C# you just need to convert it back to array from base64 string like so :
var bytes = Convert.FromBase64String(dataReceivedFromJavaScript);
where bytes would be byte[].

Taking an image and converting it into BLOB for MySQL

I am trying to make a JavaScript that would take an image file and covert it into BLOB (by converting the file into Base64 first and then into BLOB), my project doesn't have a support for toBlob() so I have found different convering steps and put them together and they work to a point where I have to pass the BLOB from the function where its made out for the Mysql part of code that takes care of communicating with the database. (I have that fully working). Now I only need to find a way how to connect them through a variable that saves the results of the imageforQuery function.
My code so far is this:
let base64String = "";
function imageforQuery(imageid) {
//takes file and converts to Base64
var file = document.getElementById(imageid).files[0];
var reader = new FileReader();
console.log("next");
imgFileFrontBlob = "";
reader.onload = function () {
base64String = reader.result.replace("data:", "")
.replace(/^.+,/, "");
// console.log(base64String);
base64String = 'data:image/jpeg;base64,'+ base64String;
console.log(base64String);
//converts Base64 into BLOB
var binary = atob(base64String.split(',')[1]);
console.log(binary);
var array = [];
for(var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var imgFileFrontBlob = new Blob([new Uint8Array(array)], {type: 'image/png'});
console.log(imgFileFrontBlob);
return imgFileFrontBlob
}
reader.readAsDataURL(file);
};
by experimenting with console.log() at different stages and return I have found out that I can't pass the converted BLOB result out, as the function imageforQuery() only returns what is after reader.readAsDataURL(file); and I don't know of a way of getting that result out.
––––––––––––––ADDITIONAL PROBLEMS I HAVE ENCOUNTERED––––––––––––––
okay so thanks to Emiel Zuurbier (Thank you!) I have managed to rewrite my code with the help of his solution. However as much as it helped one part of the problem, it didn't help with the JavaScript Blob object as we found out it is not the exact same thing as SQL BLOB.
Now the problem is that upon trying to send the Blob object in a SQL query, this resulted in just sending pure text "[Blob object]".
But I am using JavaScript successfully to pull the data from a BLOB field from my database and convert it into Base64 images from that data that was stored in the BLOB in a different part of my application. The code for that is below:
var converterEngine = function (input) {
// fn BLOB => Binary => Base64 ?
var uInt8Array = new Uint8Array(input),
i = uInt8Array.length;
var biStr = []; //new Array(I);
while (i--) { biStr[i] = String.fromCharCode(uInt8Array[i]); }
var base64 = window.btoa(biStr.join(''));
return base64;
};
What I need to do is just reverse this and in theory, it should get me the same data that I receive from the database.
My reversal code is below:
// this is the inside bit of code from the first problem that is solved and the
// typeOfData variable is parsed into the function in imageforQuery() as a second input
// variable (in other words its not to be of concern)
reader.onload = function () {
let base64String = reader.result.replace("data:", "").replace(/^.+,/, "");
base64String = "data:" + typeOfData + ";base64," + base64String;
var binary = atob(base64String.split(",")[1]);
// console.log(binary);
var array = [];
for (var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var ourArray = new Uint8Array(array);
resolve(ourArray);
};
However, as I mentioned the data that comes out (ourArray) isn't actually identical to the original file from the BLOB in the database so my code doesn't function correctly and I don't know why. Any ideas where I've made a mistake?

Base64 is simply ascii text. So MySQL's datatype BLOB or TEXT would work. That is, after converting to Base64, don't worry about "convert to blob"; it is not necessary.
That is, you can probably replace the code from //converts ... through return ... by simply
return base64String;

You can wrap the FileReader instance and calls inside of a Promise. Return the Promise immediately. In the reader.onload function call resolve() to exit the Promise with a value.
function imageforQuery(imageid) {
return new Promise(resolve => {
var file = document.getElementById(imageid).files[0];
var reader = new FileReader();
reader.onload = function () {
let base64String = reader.result.replace("data:", "").replace(/^.+,/, "");
base64String = "data:image/jpeg;base64," + base64String;
var binary = atob(base64String.split(",")[1]);
var array = [];
for (var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
var imgFileFrontBlob = new Blob([new Uint8Array(array)], {
type: "image/png",
});
resolve(imgFileFrontBlob);
};
reader.readAsDataURL(file);
});
}
This results in being able to use your function like here below. imageforQuery is called, returns a Promise. When the promise is finished (meaning resolve is called) the function in the then method will run.
imageforQuery(imageId).then(imgFileFrontBlob => {
// Use your blob here.
saveToDB(imgFileFrontBlob); // Example of how you would use it.
});
Or use it with async / await.
(async () => {
function imageforQuery(imageid) {
...
}
// Here we can wait for imageforQuery to finish and save the variable.
const imgFileFrontBlob = await imageforQuery(imageId);
saveToDB(imgFileFrontBlob); // Example of how you would use it.
})()

Sending (JavaScript / Ajax) and receiving (PHP) files without utf8 issues [duplicate]

I'm looking for a way to split up any text/data file on the front end in the browser before being uploaded as multiple files. My limit is 40KB per upload. So if a user uploads a 400KB file, it would split this file into 10 separate chunks or 10 separate files on the front end before uploading it to the server.
Currently, I'm doing it by converting this file into a base64 formatted string, then split this string by 40KB which comes out to 10 separate chunks. From there I upload each chunk as with a filename of chunk-1-of-10, chunk-2-of-10...
When pulling down these files, I just concat all these chunks back and deconvert it from base64 into its file format.
Is there a better way of doing this? Is there a library that handles all of this instead of writing it from scratch? I'm not sure if the base64 route is the best way to do this.

There is no need for reading the content into ram with the FileReader
using base64 will only increase the size of what you need to upload, base64 takes up ~33% more in size
Use Blob.slice to get chunks
blob slices (chunks) will not increase the memory, it will just create a new reference to the old blob with a changed offset and a new size to where it should start reading from.
when fetch sends the data it will be piped directly from the disk to the network without even touching the main thread.
// simulate a file from a input
const file = new File(['a'.repeat(1000000)], 'test.txt')
const chunkSize = 40000
const url = 'https://httpbin.org/post'
for (let start = 0; start < file.size; start += chunkSize) {
const chunk = file.slice(start, start + chunkSize + 1)
const fd = new FormData()
fd.set('data', chunk)
await fetch(url, { method: 'post', body: fd }).then(res => res.text())
}

You could avoid having to base64 encode by using a FileReader and then sending as binary:
const url = 'http://www.example.com/upload';
document.getElementById('file-uploader').addEventListener('change', function(e) {
const size = 40000;
var reader = new FileReader();
var buf;
var file = document.getElementById('file-uploader').files[0];
reader.onload = function(e) {
buf = new Uint8Array(e.target.result);
for (var i = 0; i < buf.length; i += size) {
var fd = new FormData();
fd.append('fname', [file.name, i+1, 'of', buf.length].join('-'));
fd.append('data', new Blob([buf.subarray(i, i + size)]));
var oReq = new XMLHttpRequest();
oReq.open("POST", url, true);
oReq.onload = function (oEvent) {
// Uploaded.
};
oReq.send(fd);
}
}
reader.readAsArrayBuffer(file);
});
<input type="file" id="file-uploader"/>

Split an uploaded file into multiple chunks using javascript

I'm looking for a way to split up any text/data file on the front end in the browser before being uploaded as multiple files. My limit is 40KB per upload. So if a user uploads a 400KB file, it would split this file into 10 separate chunks or 10 separate files on the front end before uploading it to the server.
Currently, I'm doing it by converting this file into a base64 formatted string, then split this string by 40KB which comes out to 10 separate chunks. From there I upload each chunk as with a filename of chunk-1-of-10, chunk-2-of-10...
When pulling down these files, I just concat all these chunks back and deconvert it from base64 into its file format.
Is there a better way of doing this? Is there a library that handles all of this instead of writing it from scratch? I'm not sure if the base64 route is the best way to do this.

There is no need for reading the content into ram with the FileReader
using base64 will only increase the size of what you need to upload, base64 takes up ~33% more in size
Use Blob.slice to get chunks
blob slices (chunks) will not increase the memory, it will just create a new reference to the old blob with a changed offset and a new size to where it should start reading from.
when fetch sends the data it will be piped directly from the disk to the network without even touching the main thread.
// simulate a file from a input
const file = new File(['a'.repeat(1000000)], 'test.txt')
const chunkSize = 40000
const url = 'https://httpbin.org/post'
for (let start = 0; start < file.size; start += chunkSize) {
const chunk = file.slice(start, start + chunkSize + 1)
const fd = new FormData()
fd.set('data', chunk)
await fetch(url, { method: 'post', body: fd }).then(res => res.text())
}

You could avoid having to base64 encode by using a FileReader and then sending as binary:
const url = 'http://www.example.com/upload';
document.getElementById('file-uploader').addEventListener('change', function(e) {
const size = 40000;
var reader = new FileReader();
var buf;
var file = document.getElementById('file-uploader').files[0];
reader.onload = function(e) {
buf = new Uint8Array(e.target.result);
for (var i = 0; i < buf.length; i += size) {
var fd = new FormData();
fd.append('fname', [file.name, i+1, 'of', buf.length].join('-'));
fd.append('data', new Blob([buf.subarray(i, i + size)]));
var oReq = new XMLHttpRequest();
oReq.open("POST", url, true);
oReq.onload = function (oEvent) {
// Uploaded.
};
oReq.send(fd);
}
}
reader.readAsArrayBuffer(file);
});
<input type="file" id="file-uploader"/>

Convert binary file to JavaScript string and then to Uint8Array

I'm trying to create a web application that can be used via a file:// URI. This means that I can't use AJAX to load binary files (without turning off security features in the browser, which I don't want to do as a matter of principle).
The application uses a SQLite database. I want to provide the database to a sql.js constructor, which requires it in Uint8Array format.
Since I can't use AJAX to load the database file, I could instead load it with <input type="file"> and FileReader.prototype.readAsArrayBuffer and convert the ArrayBuffer to a Uint8Array. And that's working with the following code:
input.addEventListener('change', function (changeEvent) {
var file = changeEvent.currentTarget.files[0];
var reader = new FileReader();
reader.addEventListener('load', function (loadEvent) {
var buffer = loadEvent.target.result;
var uint8Array = new Uint8Array(buffer);
var db = new sql.Database(uint8Array);
});
reader.readAsArrayBuffer(file);
});
However, <input type="file"> requires user interaction, which is tedious.
I thought I might be able to work around the no-AJAX limitation by using a build tool to convert my database file to a JavaScript object / string and generate a ".js" file providing the file contents, and then convert the file contents to a Uint8Array, somehow.
Psuedocode:
// In Node.js:
var fs = require('fs');
var sqliteDb = fs.readFileSync('path/to/sqlite.db');
var string = convertBufferToJsStringSomehow(sqliteDb);
fs.writeFileSync('build/db.js', 'var dbString = "' + string + '";');
// In the browser (assume "build/db.js" has been loaded via a <script> tag):
var uint8Array = convertStringToUint8ArraySomehow(dbString);
var db = new sql.Database(uint8Array);
In Node.js, I've tried the following:
var TextEncoder = require('text-encoding').TextEncoder;
var TextDecoder = require('text-encoding').TextEncoder;
var sql = require('sql.js');
var string = new TextDecoder('utf-8').decode(fs.readFileSync('path/to/sqlite.db'));
// At this point, I would write `string` to a ".js" file, but for
// the sake of determining if converting back to a Uint8Array
// would work, I'll continue in Node.js...
var uint8array = new TextEncoder().encode(string);
var db = new sql.Database(uint8array);
db.exec('SELECT * FROM tablename');
But when I do that, I get the error "Error: database disk image is malformed".
What am I doing wrong? Is this even possible? The database disk image isn't "malformed" when I load the same file via FileReader.

Using the following code, I was able to transfer the database file's contents to the browser:
// In Node.js:
var fs = require('fs');
var base64 = fs.readFileSync('path/to/sqlite.db', 'base64');
fs.writeFileSync('build/db.js', 'var dbString = "' + base64 + '";');
// In the browser (assume "build/db.js" has been loaded via a <script> tag):
function base64ToUint8Array (string) {
var raw = atob(string);
var rawLength = raw.length;
var array = new Uint8Array(new ArrayBuffer(rawLength));
for (var i = 0; i < rawLength; i += 1) {
array[i] = raw.charCodeAt(i);
}
return array;
}
var db = new sql.Database(base64ToUint8Array(dbString));
console.log(db.exec('SELECT * FROM tablename'));

And that's working with the following code:
input.addEventListener('change', function (changeEvent) {
var file = changeEvent.currentTarget.files[0];
var reader = new FileReader();
reader.addEventListener('load', function (loadEvent) {
var buffer = loadEvent.target.result;
var uint8Array = new Uint8Array(buffer);
var db = new sql.Database(uint8Array);
});
reader.readAsArrayBuffer(file);
});
However, <input type="file"> requires user interaction, which is
tedious.
Using current working approach would be less tedious than attempting to create workarounds. If user intends to use application, user can select file from their filesystem to run application.