I am unable to create a blob from the response stream of a request (the response comes from a node js server), here is the code:
fetch('/api/load', {
method: 'POST',
headers: { 'Content-Type': 'text/plain' },
body: src
}).then((response) => {
const reader = response.body.getReader()
return new ReadableStream({
start(controller) {
return pump()
function pump() {
return reader.read().then(({ done, value }) => {
if (done) {
controller.close();
return;
}
controller.enqueue(value);
return pump();
})
}
}
})
}).then((stream) => {
// What to do next with the stream???
})
I already tried new Blob([new Uint8Array(stream)], {type: 'image/jpeg'}), but it doesn't work :(
What is the correct way to create a blob from the stream?
Related
Here is my frontend code:
let audioFile = require("assets/hello.wav");
let blob = new Blob([audioFile], { type: "audio/wav" });
try {
await customFetch(`${API_URL}/new-audio-message`, {
method: "POST",
body: JSON.stringify({
audio: blob,
cloneId: cloneId,
}),
});
} catch (error) {
console.log(error);
}
Here is is how I upload the file to s3:
const { audio } = JSON.parse(event.body);
const fileKey = `${sub}/${cloneId}/audio/${uuidv4()}.wav`;
const buffer = Buffer.from(JSON.stringify(audio));
try {
await s3
.putObject({
Bucket: PUBLIC_BUCKET,
Key: fileKey,
Body: buffer,
})
.promise();
} catch (err) {
console.error(err);
}
The file uploads to s3 but the file size for every audio file is 155 B irrespective of the length of the audio file.
The issue seems to be that the audio file is not being properly converted to a buffer before being sent to S3. The line const buffer = Buffer.from(JSON.stringify(audio)) is attempting to convert the audio object to a string and then create a buffer from that string. However, this is not the correct way to convert a Blob object to a buffer.
Updated frontend code
let audioFile = require("assets/hello.wav");
let blob = new Blob([audioFile], { type: "audio/wav" });
const reader = new FileReader();
reader.readAsArrayBuffer(blob);
reader.onloadend = async () => {
const buffer = Buffer.from(reader.result);
try {
await customFetch(`${API_URL}/new-audio-message`, {
method: "POST",
body: JSON.stringify({
audio: buffer,
cloneId: cloneId,
}),
});
} catch (error) {
console.log(error);
}
};
Updated backend code
const { audio } = JSON.parse(event.body);
const fileKey = `${sub}/${cloneId}/audio/${uuidv4()}.wav`;
try {
await s3
.putObject({
Bucket: PUBLIC_BUCKET,
Key: fileKey,
Body: audio,
})
.promise();
} catch (err) {
console.error(err);
}
I got problem how to display image send by API from backend it not display. And when I console.log, I got this error.
This is my code as your reference.
HTML
<img [src]="imageToShow" style="width:100%;margin-left: -14px;">
Component
ngOnInit() {
this.getBanner()
}
getBanner() {
this.bannerId = {
confId: 1,
type: "Banner",
};
this.httpService.getBanner(this.bannerId).subscribe(
(baseImage: any) => {
let objectURL = "data:image/jpeg;base64," + baseImage.image;
this.imageToShow = this.sanitizer.bypassSecurityTrustUrl(objectURL);
},
(error) => {
// this.isImageLoading = false;
console.log(error);
}
);
}
Service
public getBanner(data){
console.log(data)
const httpOptions = {
headers: new HttpHeaders({
'Content-Type': 'application/json',
responseType: 'blob',
Authorization: 'Bearer '+this.getToken()
})
};
return this.httpClient.post((this.serverUrl + this.basePath + '/landing/conferenceitem'),data,httpOptions);
}
edit
when I check up Network Response I got this image
Try this
Step #1
Remove Content-Type header and set responseType to blob in httpOptions, but not in the header part like you did. Now, you should get a blob as a response. Before, angular was trying to parse your response as JSON, hence the error
public getBanner(data){
console.log(data)
const httpOptions = {
headers: new HttpHeaders({
Authorization: 'Bearer '+this.getToken()
}),
responseType: 'blob'
};
return this.httpClient.post((this.serverUrl + this.basePath + '/landing/conferenceitem'),data,httpOptions);
}
Step #2 Use baseImage instead of baseImage.image (the response is a blob, it does not have an image property), and then use createObjectURL to get an image url from the blob. Sanitize that URL like your did
this.httpService.getBanner(this.bannerId).subscribe(
(baseImage: Blob) => {
let objectURL = URL.createObjectURL(baseImage);
this.imageToShow = this.sanitizer.bypassSecurityTrustUrl(objectURL);
},
(error) => {
// this.isImageLoading = false;
console.log(error);
}
);
One way to fix this is by Setting the response type to blob
const requestOptions: Object = {
/* other options here */
responseType: 'blob'
}
return this.httpClient.post((this.serverUrl + this.basePath + '/landing/conferenceitem'),data,requestOptions);
and you have to convert your image data to a dataURL:
this.httpService.getBanner(this.bannerId).subscribe(
(baseImage: any) => {
this.imageToShow = baseImage;
},
(error) => {
// this.isImageLoading = false;
console.log(error);
}
);
Change Your getBannerMethod as below :-
getBanner() {
this.bannerId = {
confId: 1,
type: "Banner",
};
this.httpService.getBanner(this.bannerId).subscribe(
(baseImage: any) => {
const reader = new FileReader();
const url = reader.readAsDataURL(baseImage.image);
reader.onloadend = () => this.imageToShow = reader.result;
},
(error) => {
// this.isImageLoading = false;
console.log(error);
}
);
}
Working Stackblitz :- https://stackblitz.com/edit/angular-yvicvq
I am quite new to Redux Thunk and have an issue that I want to update a contract with a 'FileList' (file appendix), but if I use JSON.stringify the file will have a 0 value. If I convert the file to Base64 this problem is solved, but the PUT request is performed before the file is converted.
I searched a lot about Redux Thunk and think it might be some issue with Dispatch, I tried quite a lot and didn't become much wiser. Most of the things that I tried returned: "Actions must be plain objects. Use custom middleware for async actions."
Would appreciate some help or some search suggestions..
ps. contract.answers[0].answer[0] is the file. This acquires some refactoring, but first it needs to work.
const toBase64 = (file) => new Promise((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result);
reader.onerror = (error) => reject(error);
});
export function updateContract(contract) {
const base64File = toBase64(contract.answers[0].answer[0]);
base64File.then((value) => {
contract.answers[0].answer[0] = value; //Set file as base64
});
return {
type: SAVE,
fetchConfig: {
uri: contract._links.self,
method: 'PUT',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(contract), // Does not handle files
failureHandler(error) {
const {
details,
status,
} = error;
// If the contract was invalid, throw form errors:
if (status.code === 400 && details) {
// Map the question ids to fields:
throw new SubmissionError(Object.keys(details).reduce(
(acc, questionId) => {
acc[`question${questionId}`] = details[questionId];
return acc;
},
{},
));
}
return {
type: SAVE_FAILURE,
error,
};
},
successHandler(json) {
return {
type: SAVE_SUCCESS,
data: json,
};
},
},
};
}
Kind regards,
Gust de Backer
This happen because toBase64 return a Promise and itself is async, so in your case is necessary encapsule inside a new then.
export function updateContract(contract) {
const base64File = toBase64(contract.answers[0].answer[0]);
base64File.then((value) => {
contract.answers[0].answer[0] = value; //Set file as base64
});
return (dispatch) => {
base64File.then(() => dispatch({
type: SAVE,
fetchConfig: {
uri: contract._links.self,
method: 'PUT',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(contract), // Does not handle files
failureHandler(error) {
const {
details,
status,
} = error;
// If the contract was invalid, throw form errors:
if (status.code === 400 && details) {
// Map the question ids to fields:
throw new SubmissionError(Object.keys(details).reduce(
(acc, questionId) => {
acc[`question${questionId}`] = details[questionId];
return acc;
}, {},
));
}
return {
type: SAVE_FAILURE,
error,
};
},
successHandler(json) {
return {
type: SAVE_SUCCESS,
data: json,
};
},
},
}))
};
}
Yes, the redux accept a function as return, that function receive a dispatch on params, you can use it to dispatch the request after convert is ready :)
I am trying to write a function that would:
Take a remote URL as a parameter,
Get the file using axios
Upload the stream to amazon s3
And finally, return the uploaded url
I found help here on stackoverflow. So far, I have this:
/*
* Method to pipe the stream
*/
const uploadFromStream = (file_name, content_type) => {
const pass = new stream.PassThrough();
const obj_key = generateObjKey(file_name);
const params = { Bucket: config.bucket, ACL: config.acl, Key: obj_key, ContentType: content_type, Body: pass };
s3.upload(params, function(err, data) {
if(!err){
return data.Location;
} else {
console.log(err, data);
}
});
return pass;
}
/*
* Method to upload remote file to s3
*/
const uploadRemoteFileToS3 = async (remoteAddr) => {
axios({
method: 'get',
url: remoteAddr,
responseType: 'stream'
}).then( (response) => {
if(response.status===200){
const file_name = remoteAddr.substring(remoteAddr.lastIndexOf('/')+1);
const content_type = response.headers['content-type'];
response.data.pipe(uploadFromStream(file_name, content_type));
}
});
}
But uploadRemoteFileToS3 does not return anything (because it's a asynchronous function). How can I get the uploaded url?
UPDATE
I have further improved upon the code and wrote a class. Here is what I have now:
const config = require('../config.json');
const stream = require('stream');
const axios = require('axios');
const AWS = require('aws-sdk');
class S3RemoteUploader {
constructor(remoteAddr){
this.remoteAddr = remoteAddr;
this.stream = stream;
this.axios = axios;
this.config = config;
this.AWS = AWS;
this.AWS.config.update({
accessKeyId: this.config.api_key,
secretAccessKey: this.config.api_secret
});
this.spacesEndpoint = new this.AWS.Endpoint(this.config.endpoint);
this.s3 = new this.AWS.S3({endpoint: this.spacesEndpoint});
this.file_name = this.remoteAddr.substring(this.remoteAddr.lastIndexOf('/')+1);
this.obj_key = this.config.subfolder+'/'+this.file_name;
this.content_type = 'application/octet-stream';
this.uploadStream();
}
uploadStream(){
const pass = new this.stream.PassThrough();
this.promise = this.s3.upload({
Bucket: this.config.bucket,
Key: this.obj_key,
ACL: this.config.acl,
Body: pass,
ContentType: this.content_type
}).promise();
return pass;
}
initiateAxiosCall() {
axios({
method: 'get',
url: this.remoteAddr,
responseType: 'stream'
}).then( (response) => {
if(response.status===200){
this.content_type = response.headers['content-type'];
response.data.pipe(this.uploadStream());
}
});
}
dispatch() {
this.initiateAxiosCall();
}
async finish(){
//console.log(this.promise); /* return Promise { Pending } */
return this.promise.then( (r) => {
console.log(r.Location);
return r.Location;
}).catch( (e)=>{
console.log(e);
});
}
run() {
this.dispatch();
this.finish();
}
}
But still have no clue how to catch the result when the promise is resolved. So far, I tried these:
testUpload = new S3RemoteUploader('https://avatars2.githubusercontent.com/u/41177');
testUpload.run();
//console.log(testUpload.promise); /* Returns Promise { Pending } */
testUpload.promise.then(r => console.log); // does nothing
But none of the above works. I have a feeling I am missing something very subtle. Any clue, anyone?
After an upload you can call the getsignedurl function in s3 sdk to get the url where you can also specify the expiry of the url as well. You need to pass the key for that function. Now travelling will update with example later.
To generate a simple pre-signed URL that allows any user to view the
contents of a private object in a bucket you own, you can use the
following call to getSignedUrl():
var s3 = new AWS.S3();
var params = {Bucket: 'myBucket', Key: 'myKey'};
s3.getSignedUrl('getObject', params, function (err, url) {
console.log("The URL is", url);
});
Official documentation link
http://docs.amazonaws.cn/en_us/AWSJavaScriptSDK/guide/node-examples.html
Code must be something like this
function uploadFileToS3AndGenerateUrl(cb) {
const pass = new stream.PassThrough();//I have generated streams from file. Using this since this is what you have used. Must be a valid one.
var params = {
Bucket: "your-bucket", // required
Key: key , // required
Body: pass,
ContentType: 'your content type',
};
s3.upload(params, function(s3Err, data) {
if (s3Err) {
cb(s3Err)
}
console.log(`File uploaded successfully at ${data.Location}`)
const params = {
Bucket: 'your-bucket',
Key: data.key,
Expires: 180
};
s3.getSignedUrl('getObject', params, (urlErr, urlData) => {
if (urlErr) {
console.log('There was an error getting your files: ' + urlErr);
cb(urlErr);
} else {
console.log(`url: ${urlData}`);
cb(null, urlData);
}
})
})
}
Please check i have update your code might its help you.
/*
* Method to upload remote file to s3
*/
const uploadRemoteFileToS3 = async (remoteAddr) => {
const response = await axios({
method: 'get',
url: remoteAddr,
responseType: 'stream'
})
if(response.status===200){
const file_name = remoteAddr.substring(remoteAddr.lastIndexOf('/')+1);
const content_type = response.headers['content-type'];
response.data.pipe(uploadFromStream(file_name, content_type));
}
return new Promise((resolve, reject) => {
response.data.on('end', (response) => {
console.log(response)
resolve(response)
})
response.data.on('error', () => {
console.log(response);
reject(response)
})
})
};
*
* Method to pipe the stream
*/
const uploadFromStream = (file_name, content_type) => {
return new Promise((resolve, reject) => {
const pass = new stream.PassThrough();
const obj_key = generateObjKey(file_name);
const params = { Bucket: config.bucket, ACL: config.acl, Key: obj_key, ContentType: content_type, Body: pass };
s3.upload(params, function(err, data) {
if(!err){
console.log(data)
return resolve(data.Location);
} else {
console.log(err)
return reject(err);
}
});
});
}
//call uploadRemoteFileToS3
uploadRemoteFileToS3(remoteAddr)
.then((finalResponse) => {
console.log(finalResponse)
})
.catch((err) => {
console.log(err);
});
How send canvas to server?
I'm using react-cropper
Output is base64 but i need send it like form data with fetch
Is it possible to do like that?
static upload(files) {
let data = new FormData();
data.append(files[0].name, files[0]);
const request = new Request("/api/upload-photo", {
method: 'POST',
credentials: 'same-origin',
body: JSON.stringify(files)
});
return fetch(request).then(response => {
return response.json();
}).catch(error => {
return error;
});
}