I'm using the ASP.NET WebApi with ReactJs application in the front, I'm creating a Get method to download file from the server, and I trying to set both Content-Type and Content-Length in the response headers :
var result = new HttpResponseMessage(HttpStatusCode.OK);
result.Content = new StreamContent(new MemoryStream(bytes));
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentLength = bytes.Length;
When I'm calling this method from the ReactJs Application using fetch method like below:
await fetch(`someservice/${clientid}/download/${fileName}`, { responseType: "arraybuffer" })
.then((response) => {
const reader = response.body.getReader();
//get total length
const contentLength = response.headers.get('Content-Length');
console.log(response.headers);
//read the data
let receivedLength = 0; // received that many bytes at the moment
let chunks = []; // array of received binary chunks (comprises the body)
while (true) {
const { done, value } = reader.read();
if (done) {
break;
}
console.log(value);
chunks.push(value);
receivedLength += value.length;
console.log(`Received ${receivedLength} of ${contentLength}`)
}
//concatenate chunks into single Uint8Array
let chunksAll = new Uint8Array(receivedLength);
let position = 0;
for (let chunk of chunks) {
chunksAll.set(chunk, position);
position += chunk.length;
}
});
I got a response without Content-Type and Content-Length headers:
But isn't Content-Type and Content-Length a valid headers ?
response headers in fetch api JavaScript has been implemented as an iterator rather than an object , you could try below to access the headers :-
for (var header of respone.headers.entries()) {
console.log(header);
}
each header item in the loop will be an array of string where the first element is the key and 2nd element is value.
Related
I am trying to upload a pdf from the frontend to my node server. The PDF successfully uploads on the node server but when I go to open it, I am unable to. Instead, I see a message that says "File cant be opened. Something went wrong." Why is this happening?
Also please dont suggest third party pdf uploaders like multer, etc. I am aware of these third party libraries but I just want pure node. Thank you so much.
Frontend code:
const uploadFile = document.getElementById("uploadFile");
uploadFile.addEventListener("change", (event) => {
readFile(event.target.files[0]);
});
function readFile(file) {
const uploadDesignPDF = `http://localhost:7000/api/upload/design`;
let fileReader = new FileReader();
fileReader.readAsDataURL(file);
fileReader.addEventListener("load", async (event) => {
let pdfStrChunk = event.target.result.replace(
/^data:application\/[a-z]+;base64,/,
""
);
let fileSize = file.size;
const chunk = 85000;
let numOfChunkSet = Math.ceil(fileSize / chunk);
let remainingChunk = fileSize;
let currentChunk = 0;
let chunkSet = [];
let range = {};
let data = {};
for (let i = 0; i < numOfChunkSet; i++) {
remainingChunk -= chunk;
if (remainingChunk < 0) {
remainingChunk += chunk;
chunkSet.push(remainingChunk);
range.start = currentChunk;
range.end = currentChunk + chunk;
currentChunk += remainingChunk;
} else {
chunkSet.push(chunk);
range.start = currentChunk;
range.end = (i + 1) * chunkSet[i];
currentChunk += chunk;
}
const chunkRead = pdfStrChunk.slice(range.start, range.end);
data.dataPDF = chunkRead;
let response = await fetch(uploadDesignPDF, {
method: "POST",
body: JSON.stringify(data),
headers: {
"Content-Type": "application/json",
},
responseType: "arrayBuffer",
responseEncoding: "binary",
});
let results = await response.json();
console.log(results);
}
});
}
Backend route:
const { uploadDesigns } = require("./upload.designs.controller.js");
const router = require("express").Router();
router.post("/upload/design", uploadDesigns);
Backend:
uploadDesigns: async (req, res) => {
try {
fs.writeFileSync(`./designs/testingPDF6.pdf`, req.body.dataPDF, "base64");
res.status(200).json({
message: "done with chunk",
});
} catch (error) {
res.status(500).json({
message: "Something went wrong. Please refresh page.",
});
}
}
You are working with base64-URL in vain. It is much more effective to use ArrayBuffer. The main advantage of ArrayBuffer is the 1-byte unit, while base64 breaks the byte representation three out of four times.
Instead of sending the file in chunks, I would suggest tracking progress through XMLHttpRequest.upload.onprogress(). I would only use chunks if the upload is through a WebSocket.
If the PDF file is the only information sent to the server, I'd prefer to send the file directly without any field names or other FormData information provided. In that case, it would be appropriate to change the POST method to PUT.
If you prefer to send the file directly, it would be ideal to use fs.createWriteStream() instead of fs.writeFileSync().
Then this approach will work
const ws = fs.createWriteStream(tmpFilePath);
request.pipe(ws);
To control the integrity of the data, you can add md5 or sha hash to the request headers and, on the server, duplicate the data stream into the object created by crypto.createHash(). In case of a hash mismatch, the file can be uploaded again.
I am using worker to do my caching task based on device type because the theme I am using doesn't have a separate mobile cache and also Cloudflare doesn't have device type caching at the server's edge, Therefor I used a worker to do this task. With the normal GET request at first, it shows cache status dynamic after a couple of reloads it shows to hit.
There is also a POST method in the worker code so when the POST request is made then it should make also a GET request so the cache for that product is saved. The problem I am facing with the POST method whenever I publish a product the cache is not re-validated and therefore I am getting an old cache. Even if I purge cache it doesn't re-validate cache or show expire cache, It shows to hit.
I know I am not doing it wrong, I would be very thankful if you can help me, below is the code snippet I am using.
Thank you so much.
async function run(event) {
const { request } = event;
const cache = caches.default;
// Read the user agent of the request
const ua = request.headers.get('user-agent');
let uaValue;
if (ua.match(/mobile/i)) {
uaValue = 'mobile';
} else {
uaValue = 'desktop';
}
// Construct a new response object which distinguishes the cache key by device
// type.
const url = new URL(request.url);
url.searchParams.set('ua', uaValue);
const newRequest = new Request(url, request);
let response = await cache.match(newRequest);
if (!response) {
// Use the original request object when fetching the response from the
// server to avoid passing on the query parameters to our backend.
response = await fetch(request);
response = new Response(response.body, response)
response.headers.append("Cache-Control", "max-age=31536000")
// Store the cached response with our extended query parameters.
event.waitUntil(cache.put(newRequest, response.clone()));
}
return response;
}
async function sha256(message) {
// encode as UTF-8
const msgBuffer = new TextEncoder().encode(message)
// hash the message
const hashBuffer = await crypto.subtle.digest("SHA-256", msgBuffer)
// convert ArrayBuffer to Array
const hashArray = Array.from(new Uint8Array(hashBuffer))
// convert bytes to hex string
const hashHex = hashArray.map(b => ("00" + b.toString(16)).slice(-2)).join("")
return hashHex
}
async function handlePostRequest(event) {
const request = event.request
const body = await request.clone().text()
const hash = await sha256(body)
const cacheUrl = new URL(request.url)
// Store the URL in cache by prepending the body's hash
cacheUrl.pathname = "/posts" + cacheUrl.pathname + hash
// Convert to a GET to be able to cache
const cacheKey = new Request(cacheUrl.toString(), {
headers: request.headers,
method: "GET",
})
const cache = caches.default
// Find the cache key in the cache
let response = await cache.match(cacheKey)
// Otherwise, fetch response to POST request from origin
if (!response) {
response = await fetch(request)
event.waitUntil(cache.put(cacheKey, response.clone()))
}
return response
}
addEventListener("fetch", event => {
try {
const request = event.request
if (request.method.toUpperCase() === "POST")
return event.respondWith(handlePostRequest(event))
return event.respondWith(run(event))
} catch (e) {
return event.respondWith(new Response("Error thrown " + e.message))
}
})
I created an Express Backend which sends JSON data as a chunk of text using res.write(JSON.stringify(JSONChunk)).
I want to handle and process each chunck of res.write in react front end and am using the following method:
My Backend pseudo code
for(let i = 0; i < slices; i ++) {
var JSONChunck = await getData(i); // getData operation can take some time
res.write(JSON.stringify(JSONChunck));
}
res.end();
FE pseudocode:
fetch(API, OPTS).then(async (response) => {
const reader = response.body.getReader();
while (true) {
const { done, value } = await reader.read();
if (done) {
break;
}
var text = new TextDecoder("utf-8").decode(value);
var result = JSON.parse(text);
var processedResult = process(result);
}
})
However, when I try the above code in some systems, I receive an error when trying to do JSON.parse(text) and see that the value of 'text' does not fetch full JSON string and only a partial string.
I was wondering if I am doing anything wrong or if there is any better way to do the above.
Any help/suggestions appreciated.
Thank you!
i'm working with angular 4, I'm using an api that return an image with content-type : img/png
The http method :
return this.http.get('URL', this.options)
.map((res: Response) => res.text());
// can be also : res.arrayBuffer() // res.blob()
The http get response (in text and in ARC ) is like that :
�PNG IHDR��"�W�W��zؽ�|+q%� ��Y������M缥{��U��H�ݏ)L�L�~�6/'6Q�}���:��l'���� �R�L�&�~Lw?�
I tried different methods to convert it and display it :
getting response as blob and convert it using :
new Uint8Array(response)
Getting the image as arrayBuffer and then convert it using :
arrayBufferToBase64(buffer) {
let binary = '';
let bytes = new Uint8Array(buffer);
let len = bytes.byteLength;
for (let i = 0; i < len; i++) {
binary += String.fromCharCode(bytes[i]);
}
return window.btoa(binary);
}
Both of them didnt worked for me and the image is not displaying.
My question so is , what is the real format of the response (blob, arraybuffer or text ) and how to display it ?
You can achieve this using the fetch API.
Let's first return the response as a blob.
And then you can use URL.createObjectURL() to create an file object.
fetch(URL)
.then(res=>{return res.blob()})
.then(blob=>{
var img = URL.createObjectURL(blob);
// Do whatever with the img
document.getElementById('img').setAttribute('src', img);
})
Demo
I am using the Office Javascript API to write an Add-in for Word using Angular.
I want to retrieve the Word document through the API, then convert it to a file and upload it via POST to a server.
The code I am using is nearly identical to the documentation code that Microsoft provides for this use case: https://dev.office.com/reference/add-ins/shared/document.getfileasync#example---get-a-document-in-office-open-xml-compressed-format
The server endpoint requires uploads to be POSTed through a multipart form, so I create a FormData object on which I append the file (a blob) as well as some metadata, when creating the $http call.
The file is being transmitted to the server, but when I open it, it has become corrupted and it can no longer be opened by Word.
According to the documentation, the Office.context.document.getFileAsync function returns a byte array. However, the resulting fileContent variable is a string. When I console.log this string it seems to be compressed data, like it should be.
My guess is I need to do some preprocessing before turning the string into a Blob. But which preprocessing? Base64 encoding through atob doesn't seem to be doing anything.
let sendFile = ( fileContent ) => {
let blob = new Blob([fileContent], { type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' }),
fd = new FormData();
blob.lastModifiedDate = new Date();
fd.append('file', blob, 'uploaded_file_test403.docx');
fd.append('case_id', caseIdReducer.data());
$http.post('/file/create', fd, {
transformRequest: angular.identity,
headers: { 'Content-Type': undefined }
})
.success( ( ) => {
console.log('upload succeeded');
})
.error(( ) => {
console.log('upload failed');
});
};
function onGotAllSlices(docdataSlices) {
let docdata = [];
for (let i = 0; i < docdataSlices.length; i++) {
docdata = docdata.concat(docdataSlices[i]);
}
let fileContent = new String();
for (let j = 0; j < docdata.length; j++) {
fileContent += String.fromCharCode(docdata[j]);
}
// Now all the file content is stored in 'fileContent' variable,
// you can do something with it, such as print, fax...
sendFile(fileContent);
}
function getSliceAsync(file, nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived) {
file.getSliceAsync(nextSlice, (sliceResult) => {
if (sliceResult.status === 'succeeded') {
if (!gotAllSlices) { // Failed to get all slices, no need to continue.
return;
}
// Got one slice, store it in a temporary array.
// (Or you can do something else, such as
// send it to a third-party server.)
docdataSlices[sliceResult.value.index] = sliceResult.value.data;
if (++slicesReceived === sliceCount) {
// All slices have been received.
file.closeAsync();
onGotAllSlices(docdataSlices);
} else {
getSliceAsync(file, ++nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
}
} else {
gotAllSlices = false;
file.closeAsync();
console.log(`getSliceAsync Error: ${sliceResult.error.message}`);
}
});
}
// User clicks button to start document retrieval from Word and uploading to server process
ctrl.handleClick = ( ) => {
Office.context.document.getFileAsync(Office.FileType.Compressed, { sliceSize: 65536 /*64 KB*/ },
(result) => {
if (result.status === 'succeeded') {
// If the getFileAsync call succeeded, then
// result.value will return a valid File Object.
let myFile = result.value,
sliceCount = myFile.sliceCount,
slicesReceived = 0, gotAllSlices = true, docdataSlices = [];
// Get the file slices.
getSliceAsync(myFile, 0, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
} else {
console.log(`Error: ${result.error.message}`);
}
}
);
};
I ended up doing this with the fileContent string:
let bytes = new Uint8Array(fileContent.length);
for (let i = 0; i < bytes.length; i++) {
bytes[i] = fileContent.charCodeAt(i);
}
I then proceed to build the Blob with these bytes:
let blob = new Blob([bytes], { type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' });
If I then send this via a POST request, the file isn't mangled and can be opened correctly by Word.
I still get the feeling this can be achieved with less hassle / less steps. If anyone has a better solution, I'd be very interested to learn.
thx for your answer, Uint8Array was the solution. Just a little improvement, to avoid creating the string:
let bytes = new Uint8Array(docdata.length);
for (var i = 0; i < docdata.length; i++) {
bytes[i] = docdata[i];
}
Pff! what is wrong with a getting a instance of File and not using FileReader api? c'mon Microsoft!
You should take the byte array and throw it into the blob constructor, turning a binary blob to string in javascript is a bad idea that can lead to "out of range" error or incorrect encoding
just do something along with this
var byteArray = new Uint8Array(3)
byteArray[0] = 97
byteArray[1] = 98
byteArray[2] = 99
new Blob([byteArray])
if the chunk is an instance of a typed arrays or a instance of blob/file. in that case you can just do:
blob = new Blob([blob, chunk])
And please... don't base64 encode it (~3x larger + slower)