Unable to save binary data into AWS S3 from jQuery - javascript

I'm trying to upload a Blob object into S3, which does get uploaded but in a corrupted way. The application is about recording audio on a web page and saving it to S3.
HTML + Javascript Code:
<p>
<button id=startRecord>START</button>
<button id=stopRecord disabled>Submit</button>
</p>
<p id="recording"></p>
<p>
<a id=audioDownload></a>
</p>
<script
src="https://code.jquery.com/jquery-3.4.1.min.js"
integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo="
crossorigin="anonymous"></script>
<script type="text/javascript">
var audioContent;
navigator.mediaDevices.getUserMedia({audio:true})
.then(stream => {
rec = new MediaRecorder(stream);
rec.ondataavailable = e => {
audioChunks.push(e.data);
if (rec.state == "inactive"){
let blob = new Blob(audioChunks);
// audioContent = blob
// audioContent = URL.createObjectURL(new Blob(audioChunks));
// console.log(audioContent);
$.ajax({
type: "POST",
url: 'https://aws-api-url/prod/audio',
data: new Blob(audioChunks),
crossDomain: true,
processData: false,
headers: {"x-api-key": 'someKey'},
contentType: false
});
// audioDownload.href = audioContent;
// audioDownload.download = 'test';
// audioDownload.innerHTML = 'download';
}
}
})
.catch(e=>console.log(e));
startRecord.onclick = e => {
startRecord.disabled = true;
stopRecord.disabled=false;
document.getElementById("recording").innerHTML = "Listening...";
audioChunks = [];
rec.start();
}
stopRecord.onclick = e => {
document.getElementById("recording").innerHTML = "";
startRecord.disabled = false;
stopRecord.disabled=true;
rec.stop();
}
</script>
AWS Lambda that dumps into S3
import json
import boto3
def lambda_handler(event, context):
# TODO implement
s3_client = boto3.client('s3', aws_access_key_id='',aws_secret_access_key='')
s3_client.put_object(Body=event['body'], Bucket='bucket', Key='incoming/test.wav')
return {
'statusCode': 200,
'headers': {
'Access-Control-Allow-Origin': '*'
},
'body': json.dumps(event)
}
What changes can I possibly make into my Javascript to send this data safely

MediaRecorder does not generate the .wav data type. By default it probably generates data of the MIME type audio/webm; codecs=opus. or audio/ogg; codec=vorbis. Your lambda function looks like it faithfully stores the incoming data. But it's not .wav data, it's something else.
Your sample code lets MediaRecorder choose its own MIME type. In this case you should ask it what it used. For example
rec = new MediaRecorder(stream);
console.log (rec.mimeType);
Or, you can (try to) tell it the MIME type you want. In this case you should still ask it what it actually used. (Browsers vary in the MIME types they generate, and in the ways they respond when they can't deliver the exact type you want.) If your browser can do it, this code will probably generate mp3 (aka MPEG Layer III) audio.
rec = new MediaRecorder(stream, {mimeType: "audio/mpeg"});
console.log (rec.mimeType);
Or, you can try the audio/mp4 MIME type, and see what audio codec you get. It may vary from browser to browser.
You can generally use ffmpeg to convert any MIME type to another once you've recorded it. This is handy if require .wav output, or some other particular format. It takes some hacking, but you can do it in your lambda function.

Related

Convert int8 opus-encoded array into raw waveform (Python)

I have a js script that records audio from microphone and sends it via a websocket to a python REST API.
Part of js script (works in console):
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
var options = {mimeType: "audio/webm;codecs=opus", audioBitsPerSecond:16000};
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.addEventListener("dataavailable", function(event) {
var reader = new FileReader();
reader.addEventListener("loadend", function() {
var int8View = new Int8Array(reader.result);
console.log(int8View);
});
reader.readAsArrayBuffer(event.data);
});
mediaRecorder.start(200);
If I understand correctly the int8array that this code outputs is opus-encoded audio.
How can I decode it into a raw waveform (something that soundfile.read would return, for example) using python?
I can use ffmpeg to decode opus, but it works on files, how can I turn this array into a proper .opus or .ogg file on python side (since I don't want to send files via a websocket)?
how can I turn this array into a proper .opus or .ogg file?
You're capturing a Webm file stream via audio/webm;codecs=opus, so that would need to be converted from Webm to Ogg Opus. Instead, you could simply download or send the Webm Opus stream as a file as-is:
const recordedBuffers = []
reader.addEventListener('loadend', () => {
recordedBuffers.push(reader.result)
})
function download() {
const a = document.createElement('a')
a.href = URL.createObjectURL(new Blob(recordedBuffers))
a.download = 'my-recorded-audio.webm'
a.click()
}
function saveToServer(url) {
return fetch(url, {
method: 'post',
body: new Blob(recordedBuffers),
headers: {
'content-type': 'application/octet-stream', // optional
'content-type': 'audio/webm', // optional
},
})
}

JavaScript Blob to FormData with SAPUI5 mobile app

I know there are several threads about this topic, but I was not able to identify the problem in my case.
I have an application, where I upload an image to an endpoint-URL and after processing I'll receive a response. Works fine so far. The file is contained within a formdata object when using FileUploader-Control from SAPUI5.
When switching from file upload to "taking a picture with smartphone-camera", I dont have a file, I have an base64 dataurl (XString) image object.
var oImage = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABQAA…8ryQAbwUjsV5VUaAX/y+YSPJii2Z9GAAAAABJRU5ErkJggg=="} // some lines are missing > 1 million lines
I thought converting it to blob and appending it to FormData might be the solution, but it does not work at all.
var blob = this.toBlob(oImage)
console.log("Blob", blob); // --> Blob(857809) {size: 857809, type: "image/png"} size: 857809 type: "image/png" __proto__: Blob
var formData = new window.FormData();
formData.append("files", blob, "test.png");
console.log("FormData", formData); // seems empty --> FormData {}__proto__: FormData
Functions (works fine from my perspective)
toBlob: function dataURItoBlob(dataURI) {
var byteString = atob(dataURI.split(',')[1]);
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
var bb = new Blob([ab], {
"type": mimeString
});
return bb;
},
This is my problem, FormData is empty and my POST-request throws an undefined error (Loading of data failed: TypeError: Cannot read property 'status' of undefined at constructor.eval (...m/resources/sap/ui/core/library-preload.js?eval:2183:566))
//Create JSON Model with URL
var oModel = new sap.ui.model.json.JSONModel();
var sHeaders = {
"content-type": "multipart/form-data; boundary=---011000010111000001101001",
"APIKey": "<<myKey>>"
};
var oData = {
formData
};
oModel.loadData("/my-destination/service", oData, true, "POST", null, false, sHeaders);
oModel.attachRequestCompleted(function (oEvent) {
var oData = oEvent.getSource().oData;
console.log("Final Response XHR: ", oData);
});
Thanks for any hint
The upload collection is a complex standard control that can be used for attachment management. On desktop it opens a file dialog, on mobile it opens the ios or android photo options, which means picking a photo from the camera roll, or taking a new photo.
Fairly basic example, including the upload URL's and other handlers you'll need. More options are available, adjust to suit your needs. In your XML:
<UploadCollection
uploadUrl="{path:'Key',formatter:'.headerUrl'}/Attachments"
items="{Attachments}"
change="onAttachUploadChange"
fileDeleted="onAttachDelete"
uploadEnabled="true"
uploadComplete="onAttachUploadComplete">
<UploadCollectionItem
documentId="{DocID}"
contributor="{CreatedBy}"
fileName="{ComponentName}"
fileSize="{path:'ComponentSize',formatter:'.formatter.parseFloat'}"
mimeType="{MIMEType}"
thumbnailUrl="{parts:[{path:'MIMEType'},{path:'DocID'}],formatter:'.thumbnailURL'}"
uploadedDate="{path:'CreatedAt', formatter:'.formatter.Date'}" url="{path:'DocID',formatter:'.attachmentURL'}" visibleEdit="false"
visibleDelete="true" />
</UploadCollection>
Here's the handlers. Especially the onAttachUploadChange is important. I should mention there's no explicit post. If the uploadUrl is set correctly a post is triggered anyway.
onAttachUploadChange: function(oEvent) {
var csrf = this.getModel().getSecurityToken();
var oUploader = oEvent.getSource();
var fileName = oEvent.getParameter('files')[0].name;
oUploader.removeAllHeaderParameters();
oUploader.insertHeaderParameter(new UploadCollectionParameter({
name: 'x-csrf-token',
value: csrf
}));
oUploader.insertHeaderParameter(new UploadCollectionParameter({
name: 'Slug',
value: fileName
}));
},
onAttachDelete: function(oEvent) {
var id = oEvent.getParameter('documentId');
var oModel = this.getModel();
//set busy indicator maybe?
oModel.remove(`/Attachments('${encodeURIComponent(id)}')`, {
success: (odata, response) => {
//successful removal
//oModel.refresh();
},
error: err => console.log(err)
});
},
onAttachUploadComplete: function(oEvent) {
var mParams = oEvent.getParameter('mParameters');
//handle errors an success in here. Check `mParams`.
}
as for the formatters to determine URLs, that depends on your setup. In the case below, the stream is set up on the current binding contect, in which case this is one way to do it. You'll need the whole uri so including the /sap/opu/... etc bits.
headerUrl: function() {
return this.getModel().sServiceUrl + this.getView().getBindingContext().getPath()
},
URL for attachments is similar, but generally points to an entity of the attachment service itself.
attachmentURL: function(docid) {
return this.getModel().sServiceUrl + "/Attachments('" + docid + "')/$value";
},
You could fancy it up to check if it's an image, in which case you could include the mime type to show a thumbnail.
There might be better ways of doing this, but I've found this fairly flexible...

Save texts from textarea as file and force browser to download [duplicate]

I have a javascript app that sends ajax POST requests to a certain URL. Response might be a JSON string or it might be a file (as an attachment). I can easily detect Content-Type and Content-Disposition in my ajax call, but once I detect that the response contains a file, how do I offer the client to download it? I've read a number of similar threads here but none of them provide the answer I'm looking for.
Please, please, please do not post answers suggesting that I shouldn't use ajax for this or that I should redirect the browser, because none of this is an option. Using a plain HTML form is also not an option. What I do need is to show a download dialog to the client. Can this be done and how?
Don't give up so quickly, because this can be done (in modern browsers) using parts of the FileAPI:
var xhr = new XMLHttpRequest();
xhr.open('POST', url, true);
xhr.responseType = 'blob';
xhr.onload = function () {
if (this.status === 200) {
var blob = this.response;
var filename = "";
var disposition = xhr.getResponseHeader('Content-Disposition');
if (disposition && disposition.indexOf('attachment') !== -1) {
var filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/;
var matches = filenameRegex.exec(disposition);
if (matches != null && matches[1]) filename = matches[1].replace(/['"]/g, '');
}
if (typeof window.navigator.msSaveBlob !== 'undefined') {
// IE workaround for "HTML7007: One or more blob URLs were revoked by closing the blob for which they were created. These URLs will no longer resolve as the data backing the URL has been freed."
window.navigator.msSaveBlob(blob, filename);
} else {
var URL = window.URL || window.webkitURL;
var downloadUrl = URL.createObjectURL(blob);
if (filename) {
// use HTML5 a[download] attribute to specify filename
var a = document.createElement("a");
// safari doesn't support this yet
if (typeof a.download === 'undefined') {
window.location.href = downloadUrl;
} else {
a.href = downloadUrl;
a.download = filename;
document.body.appendChild(a);
a.click();
}
} else {
window.location.href = downloadUrl;
}
setTimeout(function () { URL.revokeObjectURL(downloadUrl); }, 100); // cleanup
}
}
};
xhr.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
xhr.send($.param(params, true));
Or if using jQuery.ajax:
$.ajax({
type: "POST",
url: url,
data: params,
xhrFields: {
responseType: 'blob' // to avoid binary data being mangled on charset conversion
},
success: function(blob, status, xhr) {
// check for a filename
var filename = "";
var disposition = xhr.getResponseHeader('Content-Disposition');
if (disposition && disposition.indexOf('attachment') !== -1) {
var filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/;
var matches = filenameRegex.exec(disposition);
if (matches != null && matches[1]) filename = matches[1].replace(/['"]/g, '');
}
if (typeof window.navigator.msSaveBlob !== 'undefined') {
// IE workaround for "HTML7007: One or more blob URLs were revoked by closing the blob for which they were created. These URLs will no longer resolve as the data backing the URL has been freed."
window.navigator.msSaveBlob(blob, filename);
} else {
var URL = window.URL || window.webkitURL;
var downloadUrl = URL.createObjectURL(blob);
if (filename) {
// use HTML5 a[download] attribute to specify filename
var a = document.createElement("a");
// safari doesn't support this yet
if (typeof a.download === 'undefined') {
window.location.href = downloadUrl;
} else {
a.href = downloadUrl;
a.download = filename;
document.body.appendChild(a);
a.click();
}
} else {
window.location.href = downloadUrl;
}
setTimeout(function () { URL.revokeObjectURL(downloadUrl); }, 100); // cleanup
}
}
});
Create a form, use the POST method, submit the form - there's no need for an iframe. When the server page responds to the request, write a response header for the mime type of the file, and it will present a download dialog - I've done this a number of times.
You want content-type of application/download - just search for how to provide a download for whatever language you're using.
I faced the same issue and successfully solved it. My use-case is this.
"Post JSON data to the server and receive an excel file.
That excel file is created by the server and returned as a response to the client. Download that response as a file with custom name in browser"
$("#my-button").on("click", function(){
// Data to post
data = {
ids: [1, 2, 3, 4, 5]
};
// Use XMLHttpRequest instead of Jquery $ajax
xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
var a;
if (xhttp.readyState === 4 && xhttp.status === 200) {
// Trick for making downloadable link
a = document.createElement('a');
a.href = window.URL.createObjectURL(xhttp.response);
// Give filename you wish to download
a.download = "test-file.xls";
a.style.display = 'none';
document.body.appendChild(a);
a.click();
}
};
// Post data to URL which handles post request
xhttp.open("POST", excelDownloadUrl);
xhttp.setRequestHeader("Content-Type", "application/json");
// You should set responseType as blob for binary responses
xhttp.responseType = 'blob';
xhttp.send(JSON.stringify(data));
});
The above snippet is just doing following
Posting an array as JSON to the server using XMLHttpRequest.
After fetching content as a blob(binary), we are creating a downloadable URL and attaching it to invisible "a" link then clicking it.
Here we need to carefully set few things at the server side. I set few headers in Python Django HttpResponse. You need to set them accordingly if you use other programming languages.
# In python django code
response = HttpResponse(file_content, content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
Since I download xls(excel) here, I adjusted contentType to above one. You need to set it according to your file type. You can use this technique to download any kind of files.
What server-side language are you using? In my app I can easily download a file from an AJAX call by setting the correct headers in PHP's response:
Setting headers server-side
header("HTTP/1.1 200 OK");
header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
// The optional second 'replace' parameter indicates whether the header
// should replace a previous similar header, or add a second header of
// the same type. By default it will replace, but if you pass in FALSE
// as the second argument you can force multiple headers of the same type.
header("Cache-Control: private", false);
header("Content-type: " . $mimeType);
// $strFileName is, of course, the filename of the file being downloaded.
// This won't have to be the same name as the actual file.
header("Content-Disposition: attachment; filename=\"{$strFileName}\"");
header("Content-Transfer-Encoding: binary");
header("Content-Length: " . mb_strlen($strFile));
// $strFile is a binary representation of the file that is being downloaded.
echo $strFile;
This will in fact 'redirect' the browser to this download page, but as #ahren alread said in his comment, it won't navigate away from the current page.
It's all about setting the correct headers so I'm sure you'll find a suitable solution for the server-side language you're using if it's not PHP.
Handling the response client side
Assuming you already know how to make an AJAX call, on the client side you execute an AJAX request to the server. The server then generates a link from where this file can be downloaded, e.g. the 'forward' URL where you want to point to.
For example, the server responds with:
{
status: 1, // ok
// unique one-time download token, not required of course
message: 'http://yourwebsite.com/getdownload/ska08912dsa'
}
When processing the response, you inject an iframe in your body and set the iframe's SRC to the URL you just received like this (using jQuery for the ease of this example):
$("body").append("<iframe src='" + data.message +
"' style='display: none;' ></iframe>");
If you've set the correct headers as shown above, the iframe will force a download dialog without navigating the browser away from the current page.
Note
Extra addition in relation to your question; I think it's best to always return JSON when requesting stuff with AJAX technology. After you've received the JSON response, you can then decide client-side what to do with it. Maybe, for example, later on you want the user to click a download link to the URL instead of forcing the download directly, in your current setup you would have to update both client and server-side to do so.
Here is how I got this working
https://stackoverflow.com/a/27563953/2845977
$.ajax({
url: '<URL_TO_FILE>',
success: function(data) {
var blob=new Blob([data]);
var link=document.createElement('a');
link.href=window.URL.createObjectURL(blob);
link.download="<FILENAME_TO_SAVE_WITH_EXTENSION>";
link.click();
}
});
Updated answer using download.js
$.ajax({
url: '<URL_TO_FILE>',
success: download.bind(true, "<FILENAME_TO_SAVE_WITH_EXTENSION>", "<FILE_MIME_TYPE>")
});
For those looking for a solution from an Angular perspective, this worked for me:
$http.post(
'url',
{},
{responseType: 'arraybuffer'}
).then(function (response) {
var headers = response.headers();
var blob = new Blob([response.data],{type:headers['content-type']});
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
link.download = "Filename";
link.click();
});
For those looking for a more modern approach, you can use the fetch API. The following code shows how to download a spreadsheet file.
fetch(url, {
body: JSON.stringify(data),
method: 'POST',
headers: {
'Content-Type': 'application/json; charset=utf-8'
},
})
.then(response => response.blob())
.then(response => {
const blob = new Blob([response], {type: 'application/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'});
const downloadUrl = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = downloadUrl;
a.download = "file.xlsx";
document.body.appendChild(a);
a.click();
})
I believe this approach to be much easier to understand than other XMLHttpRequest solutions. Also, it has a similar syntax to the jQuery approach, without the need to add any additional libraries.
Of course, I would advise checking to which browser you are developing, since this new approach won't work on IE. You can find the full browser compatibility list on the following [link][1].
Important: In this example I am sending a JSON request to a server listening on the given url. This url must be set, on my example I am assuming you know this part. Also, consider the headers needed for your request to work. Since I am sending a JSON, I must add the Content-Type header and set it to application/json; charset=utf-8, as to let the server know the type of request it will receive.
I see you've already found out a solution, however I just wanted to add some information which may help someone trying to achieve the same thing with big POST requests.
I had the same issue a couple of weeks ago, indeed it isn't possible to achieve a "clean" download through AJAX, the Filament Group created a jQuery plugin which works exactly how you've already found out, it is called jQuery File Download however there is a downside to this technique.
If you're sending big requests through AJAX (say files +1MB) it will negatively impact responsiveness. In slow Internet connections you'll have to wait a lot until the request is sent and also wait for the file to download. It isn't like an instant "click" => "popup" => "download start". It's more like "click" => "wait until data is sent" => "wait for response" => "download start" which makes it appear the file double its size because you'll have to wait for the request to be sent through AJAX and get it back as a downloadable file.
If you're working with small file sizes <1MB you won't notice this. But as I discovered in my own app, for bigger file sizes it is almost unbearable.
My app allow users to export images dynamically generated, these images are sent through POST requests in base64 format to the server (it is the only possible way), then processed and sent back to users in form of .png, .jpg files, base64 strings for images +1MB are huge, this force users to wait more than necessary for the file to start downloading. In slow Internet connections it can be really annoying.
My solution for this was to temporary write the file to the server, once it is ready, dynamically generate a link to the file in form of a button which changes between "Please wait..." and "Download" states and at the same time, print the base64 image in a preview popup window so users can "right-click" and save it. This makes all the waiting time more bearable for users, and also speed things up.
Update Sep 30, 2014:
Months have passed since I posted this, finally I've found a better approach to speed things up when working with big base64 strings. I now store base64 strings into the database (using longtext or longblog fields), then I pass its record ID through the jQuery File Download, finally on the download script file I query the database using this ID to pull the base64 string and pass it through the download function.
Download Script Example:
<?php
// Record ID
$downloadID = (int)$_POST['id'];
// Query Data (this example uses CodeIgniter)
$data = $CI->MyQueries->GetDownload( $downloadID );
// base64 tags are replaced by [removed], so we strip them out
$base64 = base64_decode( preg_replace('#\[removed\]#', '', $data[0]->image) );
// This example is for base64 images
$imgsize = getimagesize( $base64 );
// Set content headers
header('Content-Disposition: attachment; filename="my-file.png"');
header('Content-type: '.$imgsize['mime']);
// Force download
echo $base64;
?>
I know this is way beyond what the OP asked, however I felt it would be good to update my answer with my findings. When I was searching for solutions to my problem, I read lots of "Download from AJAX POST data" threads which didn't give me the answer I was looking for, I hope this information helps someone looking to achieve something like this.
Here is my solution using a temporary hidden form.
//Create an hidden form
var form = $('<form>', {'method': 'POST', 'action': this.href}).hide();
//Add params
var params = { ...your params... };
$.each(params, function (k, v) {
form.append($('<input>', {'type': 'hidden', 'name': k, 'value': v}));
});
//Make it part of the document and submit
$('body').append(form);
form.submit();
//Clean up
form.remove();
Note that I massively use JQuery but you can do the same with native JS.
I want to point out some difficulties that arise when using the technique in the accepted answer, i.e. using a form post:
You can't set headers on the request. If your authentication schema involves headers, a Json-Web-Token passed in the Authorization header, you'll have to find other way to send it, for example as a query parameter.
You can't really tell when the request has finished. Well, you can use a cookie that gets set on response, as done by jquery.fileDownload, but it's FAR from perfect. It won't work for concurrent requests and it will break if a response never arrives.
If the server responds with a error, the user will be redirected to the error page.
You can only use the content types supported by a form. Which means you can't use JSON.
I ended up using the method of saving the file on S3 and sending a pre-signed URL to get the file.
As others have stated, you can create and submit a form to download via a POST request. However, you don't have to do this manually.
One really simple library for doing exactly this is jquery.redirect. It provides an API similar to the standard jQuery.post method:
$.redirect(url, [values, [method, [target]]])
This is a 3 years old question but I had the same problem today. I looked your edited solution but I think that it can sacrifice the performance because it has to make a double request. So if anyone needs another solution that doesn't imply to call the service twice then this is the way I did it:
<form id="export-csv-form" method="POST" action="/the/path/to/file">
<input type="hidden" name="anyValueToPassTheServer" value="">
</form>
This form is just used to call the service and avoid to use a window.location(). After that you just simply have to make a form submit from jquery in order to call the service and get the file. It's pretty simple but this way you can make a download using a POST. I now that this could be easier if the service you're calling is a GET, but that's not my case.
I used this FileSaver.js. In my case with csv files, i did this (in coffescript):
$.ajax
url: "url-to-server"
data: "data-to-send"
success: (csvData)->
blob = new Blob([csvData], { type: 'text/csv' })
saveAs(blob, "filename.csv")
I think for most complicated case, the data must be processed properly. Under the hood FileSaver.js implement the same approach of the answer of Jonathan Amend.
see: http://www.henryalgus.com/reading-binary-files-using-jquery-ajax/
it'll return a blob as a response, which can then be put into filesaver
Here is my solution, gathered from different sources:
Server side implementation :
String contentType = MediaType.APPLICATION_OCTET_STREAM_VALUE;
// Set headers
response.setHeader("content-disposition", "attachment; filename =" + fileName);
response.setContentType(contentType);
// Copy file to output stream
ServletOutputStream servletOutputStream = response.getOutputStream();
try (InputStream inputStream = new FileInputStream(file)) {
IOUtils.copy(inputStream, servletOutputStream);
} finally {
servletOutputStream.flush();
Utils.closeQuitely(servletOutputStream);
fileToDownload = null;
}
Client side implementation (using jquery):
$.ajax({
type: 'POST',
contentType: 'application/json',
url: <download file url>,
data: JSON.stringify(postObject),
error: function(XMLHttpRequest, textStatus, errorThrown) {
alert(errorThrown);
},
success: function(message, textStatus, response) {
var header = response.getResponseHeader('Content-Disposition');
var fileName = header.split("=")[1];
var blob = new Blob([message]);
var link = document.createElement('a');
link.href = window.URL.createObjectURL(blob);
link.download = fileName;
link.click();
}
});
Below is my solution for downloading multiple files depending on some list which consists of some ids and looking up in database, files will be determined and ready for download - if those exist.
I am calling C# MVC action for each file using Ajax.
And Yes, like others said, it is possible to do it in jQuery Ajax.
I did it with Ajax success and I am always sending response 200.
So, this is the key:
success: function (data, textStatus, xhr) {
And this is my code:
var i = 0;
var max = 0;
function DownloadMultipleFiles() {
if ($(".dataTables_scrollBody>tr.selected").length > 0) {
var list = [];
showPreloader();
$(".dataTables_scrollBody>tr.selected").each(function (e) {
var element = $(this);
var orderid = element.data("orderid");
var iscustom = element.data("iscustom");
var orderlineid = element.data("orderlineid");
var folderPath = "";
var fileName = "";
list.push({ orderId: orderid, isCustomOrderLine: iscustom, orderLineId: orderlineid, folderPath: folderPath, fileName: fileName });
});
i = 0;
max = list.length;
DownloadFile(list);
}
}
Then calling:
function DownloadFile(list) {
$.ajax({
url: '#Url.Action("OpenFile","OrderLines")',
type: "post",
data: list[i],
xhrFields: {
responseType: 'blob'
},
beforeSend: function (xhr) {
xhr.setRequestHeader("RequestVerificationToken",
$('input:hidden[name="__RequestVerificationToken"]').val());
},
success: function (data, textStatus, xhr) {
// check for a filename
var filename = "";
var disposition = xhr.getResponseHeader('Content-Disposition');
if (disposition && disposition.indexOf('attachment') !== -1) {
var filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/;
var matches = filenameRegex.exec(disposition);
if (matches != null && matches[1]) filename = matches[1].replace(/['"]/g, '');
var a = document.createElement('a');
var url = window.URL.createObjectURL(data);
a.href = url;
a.download = filename;
document.body.append(a);
a.click();
a.remove();
window.URL.revokeObjectURL(url);
}
else {
getErrorToastMessage("Production file for order line " + list[i].orderLineId + " does not exist");
}
i = i + 1;
if (i < max) {
DownloadFile(list);
}
},
error: function (XMLHttpRequest, textStatus, errorThrown) {
},
complete: function () {
if(i===max)
hidePreloader();
}
});
}
C# MVC:
[HttpPost]
[ValidateAntiForgeryToken]
public IActionResult OpenFile(OrderLineSimpleModel model)
{
byte[] file = null;
try
{
if (model != null)
{
//code for getting file from api - part is missing here as not important for this example
file = apiHandler.Get<byte[]>(downloadApiUrl, token);
var contentDispositionHeader = new System.Net.Mime.ContentDisposition
{
Inline = true,
FileName = fileName
};
// Response.Headers.Add("Content-Disposition", contentDispositionHeader.ToString() + "; attachment");
Response.Headers.Add("Content-Type", "application/pdf");
Response.Headers.Add("Content-Disposition", "attachment; filename=" + fileName);
Response.Headers.Add("Content-Transfer-Encoding", "binary");
Response.Headers.Add("Content-Length", file.Length.ToString());
}
}
catch (Exception ex)
{
this.logger.LogError(ex, "Error getting pdf", null);
return Ok();
}
return File(file, System.Net.Mime.MediaTypeNames.Application.Pdf);
}
As long as you return response 200, success in Ajax can work with it, you can check if file actually exist or not as the line below in this case would be false and you can inform user about that:
if (disposition && disposition.indexOf('attachment') !== -1) {
To get Jonathan Amends answer to work in Edge I made the following changes:
var blob = typeof File === 'function'
? new File([this.response], filename, { type: type })
: new Blob([this.response], { type: type });
to this
var f = typeof File+"";
var blob = f === 'function' && Modernizr.fileapi
? new File([this.response], filename, { type: type })
: new Blob([this.response], { type: type });
I would rather have posted this as a comment but I don't have enough reputation for that
there is another solution to download a web page in ajax. But I am referring to a page that must first be processed and then downloaded.
First you need to separate the page processing from the results download.
1) Only the page calculations are made in the ajax call.
$.post("CalculusPage.php", { calculusFunction: true, ID: 29, data1: "a", data2: "b" },
function(data, status)
{
if (status == "success")
{
/* 2) In the answer the page that uses the previous calculations is downloaded. For example, this can be a page that prints the results of a table calculated in the ajax call. */
window.location.href = DownloadPage.php+"?ID="+29;
}
}
);
// For example: in the CalculusPage.php
if ( !empty($_POST["calculusFunction"]) )
{
$ID = $_POST["ID"];
$query = "INSERT INTO ExamplePage (data1, data2) VALUES ('".$_POST["data1"]."', '".$_POST["data2"]."') WHERE id = ".$ID;
...
}
// For example: in the DownloadPage.php
$ID = $_GET["ID"];
$sede = "SELECT * FROM ExamplePage WHERE id = ".$ID;
...
$filename="Export_Data.xls";
header("Content-Type: application/vnd.ms-excel");
header("Content-Disposition: inline; filename=$filename");
...
I hope this solution can be useful for many, as it was for me.
If response is an Array Buffer, try this under onsuccess event in Ajax:
if (event.data instanceof ArrayBuffer) {
var binary = '';
var bytes = new Uint8Array(event.data);
for (var i = 0; i < bytes.byteLength; i++) {
binary += String.fromCharCode(bytes[i])
}
$("#some_id").append("<li><img src=\"data:image/png;base64," + window.btoa(binary) + "\"/></span></li>");
return;
}
where event.data is response received in success function of xhr event.
I needed a similar solution to #alain-cruz's one, but in nuxt/vue with multiple downloads. I know browsers block multiple file downloads, and I also have API which returns a set of csv formatted data.I was going to use JSZip at first but I needed IE support so here is my solution. If anyone can help me improve this that would be great, but it's working for me so far.
API returns:
data : {
body: {
fileOne: ""col1", "col2", "datarow1.1", "datarow1.2"...so on",
fileTwo: ""col1", "col2"..."
}
}
page.vue:
<template>
<b-link #click.prevent="handleFileExport">Export<b-link>
</template>
export default = {
data() {
return {
fileNames: ['fileOne', 'fileTwo'],
}
},
computed: {
...mapState({
fileOne: (state) => state.exportFile.fileOne,
fileTwo: (state) => state.exportFile.fileTwo,
}),
},
method: {
handleExport() {
//exportFileAction in store/exportFile needs to return promise
this.$store.dispatch('exportFile/exportFileAction', paramsToSend)
.then(async (response) => {
const downloadPrep = this.fileNames.map(async (fileName) => {
// using lodash to get computed data by the file name
const currentData = await _.get(this, `${fileName}`);
const currentFileName = fileName;
return { currentData, currentFileName };
});
const response = await Promise.all(downloadPrep);
return response;
})
.then(async (data) => {
data.forEach(({ currentData, currentFileName }) => {
this.forceFileDownload(currentData, currentFileName);
});
})
.catch(console.error);
},
forceFileDownload(data, fileName) {
const url = window.URL
.createObjectURL(new Blob([data], { type: 'text/csv;charset=utf-8;' }));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', `${fileName}.csv`);
document.body.appendChild(link);
link.click();
},
}
I used Naren Yellavula's solution and got it working with few changes to the script, after trying several other solutions using jquery. But, jquery will not download a zip file properly. I can't unzip the file after download.
In my use case, I have to upload a zip file, which is unzipped in the Servlet, files are processed and zipped again before the zip file is downloaded to the client. This is what you need to do on client side.
$('#fileUpBtn').click(function (e){
e.preventDefault();
var file = $('#fileUpload')[0].files[0];
var formdata = new FormData();
formdata.append('file', file);
// Use XMLHttpRequest instead of Jquery $ajax to download zip files
xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (xhttp.readyState === 4 && xhttp.status === 200) {
var a = document.createElement('a');
a.href = window.URL.createObjectURL(xhttp.response);
a.download = "modified_" + file.name;
a.style.display = 'none';
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
window.URL.revokeObjectURL(a.href);
}
};
xhttp.open("POST", "<URL to Servlet>", true);
xhttp.responseType = 'blob';
xhttp.send(formdata);
});
<div class="form-group">
<label id="fileUpLabel" for="fileUpload"></label>
<input type="file" class="form-control" id="fileUpload" name="file" accept="" required/>
</div>
<button class="btn" type="submit" id="fileUpBtn"></button>

How to play AudioStream response in AWS Polly using JavaScript SDK?

This is my script:
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.7.13.min.js"></script>
<script>
AWS.config.region = 'eu-west-1';
AWS.config.accessKeyId = 'FOO';
AWS.config.secretAccessKey = 'BAR';
var polly = new AWS.Polly({apiVersion: '2016-06-10'});
var params = {
OutputFormat: 'mp3', /* required */
Text: 'Hello world', /* required */
VoiceId: 'Joanna', /* required */
SampleRate: '22050',
TextType: 'text'
};
polly.synthesizeSpeech(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
</script>
The request succeeds, and I get this kind of response:
How do I use this kind of response? I understand that the response is deserialized audio, but how do I actually play it, say, inside a HTML5 audio element?
Furthermore, this answer on SO explains why is this type of array suitable for audio data: https://stackoverflow.com/a/26320913/1325575
var uInt8Array = new Uint8Array(audioStream);
var arrayBuffer = uInt8Array.buffer;
var blob = new Blob([arrayBuffer]);
var url = URL.createObjectURL(blob);
audioElement.src = url;
audioElement.play();
I created a Javascript library called ChattyKathy that will handle the entire process for you if you want to take the easy way out.
Just pass it an AWS Credentials object and then tell her what to say. She'll call AWS, transform the response, and play the audio.
var settings = {
awsCredentials: awsCredentials,
awsRegion: "us-west-2",
pollyVoiceId: "Justin",
cacheSpeech: true
}
var kathy = ChattyKathy(settings);
kathy.Speak("Hello world, my name is Kathy!");
kathy.Speak("I can be used for an amazing user experience!");
Elliott's Chatty Kathy code worked beautifully for me, but there are two separate issues with Safari and mobile.
Safari: When creating the blob, the content type MUST be specified:
var blob = new Blob([arrayBuffer], {type: 'audio/mpeg'});
url = webkitURL.createObjectURL(blob);
Mobile: The above must be true, plus playback needs to be initiated by a user touch event. Note: Older iOS versions seem to require that playback be initiated in the same thread as the touch event, so a touch event that initiates a promise chain that eventually calls audio.play() will fail. Later iOS versions seem to be smarter about this.
Using the Web Audio API:
const result = await polly.synthesizeSpeech(params).promise();
const aContext = new AudioContext();
const source = aContext.createBufferSource();
source.buffer = await aContext.decodeAudioData(result.AudioStream.buffer);
source.connect(aContext.destination);
source.start();
Docs:
AudioContext
Decode ArrayBuffer

Chrome extension: how to pass ArrayBuffer or Blob from content script to the background without losing its type?

I have this content script that downloads some binary data using XHR, which is sent later to the background script:
var self = this;
var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.responseType = 'arraybuffer';
xhr.onload = function(e) {
if (this.status == 200) {
self.data = {
data: xhr.response,
contentType: xhr.getResponseHeader('Content-Type')
};
}
};
xhr.send();
... later ...
sendResponse({data: self.data});
After receiving this data in background script, I'd like to form another XHR request that uploads this binary data to my server, so I do:
var formData = new FormData();
var bb = new WebKitBlobBuilder();
bb.append(data.data);
formData.append("data", bb.getBlob(data.contentType));
var req = new XMLHttpRequest();
req.open("POST", serverUrl);
req.send(formData);
The problem is that the file uploaded to the server contains just this string: "[object Object]". I guess this happens because ArrayBuffer type is lost somehow while transferring it from content process to the background? How can I solve that?
Messages passed between a Content Script and a background page are JSON-serialized.
If you want to transfer an ArrayBuffer object through a JSON-serialized channel, wrap the buffer in a view, before and after transferring.
I show an isolated example, so that the solution is generally applicable, and not just in your case. The example shows how to pass around ArrayBuffers and typed arrays, but the method can also be applied to File and Blob objects, by using the FileReader API.
// In your case: self.data = { data: new Uint8Array(xhr.response), ...
// Generic example:
var example = new ArrayBuffer(10);
var data = {
// Create a view
data: Array.apply(null, new Uint8Array(example)),
contentType: 'x-an-example'
};
// Transport over a JSON-serialized channel. In your case: sendResponse
var transportData = JSON.stringify(data);
//"{"data":[0,0,0,0,0,0,0,0,0,0],"contentType":"x-an-example"}"
// At the receivers end. In your case: chrome.extension.onRequest
var receivedData = JSON.parse(transportData);
// data.data is an Object, NOT an ArrayBuffer or Uint8Array
receivedData.data = new Uint8Array(receivedData.data).buffer;
// Now, receivedData is the expected ArrayBuffer object
This solution has been tested successfully in Chrome 18 and Firefox.
new Uint8Array(xhr.response) is used to create a view of the ArrayBuffer, so that the individual bytes can be read.
Array.apply(null, <Uint8Array>) is used to create a plain array, using the keys from the Uint8Array view. This step reduces the size of the serialized message. WARNING: This method only works for small amounts of data. When the size of the typed array exceeds 125836, a RangeError will be thrown. If you need to handle large pieces of data, use other methods to do the conversion between typed arrays and plain arrays.
At the receivers end, the original buffer can be obtained by creating a new Uint8Array, and reading the buffer attribute.
Implementation in your Google Chrome extension:
// Part of the Content script
self.data = {
data: Array.apply(null, new Uint8Array(xhr.response)),
contentType: xhr.getResponseHeader('Content-Type')
};
...
sendResponse({data: self.data});
// Part of the background page
chrome.runtime.onMessage.addListener(function(data, sender, callback) {
...
data.data = new Uint8Array(data.data).buffer;
Documentation
MDN: Typed Arrays
MDN: ArrayBuffer
MDN: Uint8Array
MDN: <Function> .apply
Google Chrome Extension docs: Messaging > Simple one-time requests
"This lets you send a one-time JSON-serializable message from a content script to extension, or vice versa, respectively"
SO bonus: Upload a File in a Google Chrome Extension - Using a Web worker to request, validate, process and submit binary data.
For Chromium Extensions manifest v3, URL.createObjectURL() approach doesn't work anymore because it is prohibited in the service workers.
The best (easiest) way to pass data from a service worker to a content script (and vice-versa), is to convert the blob into a base64 representation.
const fetchBlob = async url => {
const response = await fetch(url);
const blob = await response.blob();
const base64 = await convertBlobToBase64(blob);
return base64;
};
const convertBlobToBase64 = blob => new Promise(resolve => {
const reader = new FileReader();
reader.readAsDataURL(blob);
reader.onloadend = () => {
const base64data = reader.result;
resolve(base64data);
};
});
Then send the base64 to the content script.
Service worker:
chrome.tabs.sendMessage(sender.tab.id, { type: "LOADED_FILE", base64: base64 });
Content script:
chrome.runtime.onMessage.addListener(async (request, sender) => {
if (request.type == "LOADED_FILE" && sender.id == '<your_extension_id>') {
// do anything you want with the data from the service worker.
// e.g. convert it back to a blob
const response = await fetch(request.base64);
const blob = await response.blob();
}
});

Categories

Resources