How to test image upload (stream) with supertest and jest? - javascript

I have an image upload endpoint in my API that accepts application/octet-stream requests and handles these streams. I'd like to write test coverage for this endpoint but cannot figure out how to use supertest to stream an image.
Here's my code so far:
import request from 'supertest'
const testImage = `${__dirname}/../../../assets/test_image.jpg`
describe('Upload endpoint', () => {
test('Successfully uploads jpg image', async () =>
request(app)
.post(`${ROOT_URL}${endpoints.add_image.route}`)
.set('Authorization', `Bearer ${process.env.testUserJWT}`)
.set('content-type', 'application/octet-stream')
.pipe(fs.createReadStream(testImage))
.on('finish', (something) => {
console.log(something)
}))
})
This code produces nothing, the finish event is never called, nothing is console logged, and this test suite actually passes as nothing is expected. I cannot chain a .expect onto this request, otherwise I get this runtime error:
TypeError: (0 , _supertest2.default)(...).post(...).set(...).set(...).pipe(...).expect is not a function
How is such a thing accomplished?

This should work. To pipe data to a request you have to tell the readable stream to pipe to the request. The other way is for receiving data from the server. This also uses done instead of async as pipes do not work with async/await.
Also worth nothing is that by default the pipe will call end and then superagent will call end, resulting in an error about end being called twice. To solve this you have to tell the pipe call not to do that and then call end in the on end event of the stream.
import request from 'supertest'
const testImage = `${__dirname}/../../../assets/test_image.jpg`
describe('Upload endpoint', () => {
test('Successfully uploads jpg image', (done) => {
const req = request(app)
.post(`${ROOT_URL}${endpoints.add_image.route}`)
.set('Authorization', `Bearer ${process.env.testUserJWT}`)
.set('content-type', 'application/octet-stream')
const imgStream = fs.createReadStream(testImage);
imgStream.on('end', () => req.end(done));
imgStream.pipe(req, {end: false})
})
})
Edited to add: this has worked for me with small files. If I try testing it with a large test_image.jpg the request times out.

const testImage = `${__dirname}/../../../assets/test_image.jpg`
describe('Upload endpoint', () => {
test('Successfully uploads jpg image', async () =>
request(app)
.post(`${ROOT_URL}${endpoints.add_image.route}`)
.set('Authorization', `Bearer ${process.env.testUserJWT}`)
.attach("name",testImage,{ contentType: 'application/octet-stream' })
.expect(200)
.then(response => {
console.log("response",response);
})
);
});

I had to make assumptions about your upload method taking the body as input instead of multipart form-data. So below is an example where the raw body is passed for upload
const request = require('supertest');
const express = require('express');
const fs = require('fs')
const app = express();
var bodyParser = require('body-parser')
app.use(bodyParser.raw({type: 'application/octet-stream'}))
app.post('/user', function(req, res) {
res.status(200).json({ name: 'tobi' });
});
testImage = './package.json'
resp = request(app)
.post('/user')
resp.set('Authorization', `Bearer Test`).set('Content-Type', 'application/octet-stream')
resp.send(fs.readFileSync(testImage, 'utf-8'))
resp.expect(200)
.then(response => {
console.log("response",response);
}).catch((err) => {
console.log(err)
})
If you use multipart/form-data then below code shows an example
const request = require('supertest');
const express = require('express');
const fs = require('fs')
const app = express();
app.post('/user', function(req, res) {
// capture the encoded form data
req.on('data', (data) => {
console.log(data.toString());
});
// send a response when finished reading
// the encoded form data
req.on('end', () => {
res.status(200).json({ name: 'tobi' });
});
});
testImage = './package.json'
resp = request(app)
.post('/user')
resp.set('Authorization', `Bearer Test`)
resp.attach("file", testImage)
resp.expect(200)
.then(response => {
console.log("response",response);
}).catch((err) => {
console.log(err)
})

I think you actually want to use fs.createReadStream(testImage) to read that image into your request, since fs.createWriteStream(testImage) would be writing data into the file descriptor provided (in this case testImage). Feel free to checkout Node Streams to see how they work in more detail.
I'm not quite sure where you're getting the finish event from for supertest, but you can see how to use the .pipe() method here.
You might also want to consider using supertest multipart attachments, if that better suits your test.

Related

Undefined body of a fetch request (arraybuffer body) in node server

I've read a lot of similar questions, but any answer worked for me.
I'm trying to send an audio in the body of a fetch request (in vuejs) as an ArrayBuffer, but when I print it in the server side, the body is undefined.
Client code:
export async function makeQuery(data) {
let bufferAudio = await data.arrayBuffer();
console.log(bufferAudio);
const response = await fetch(`/api/query`, {
method: 'POST',
//headers: {'Content-Type': 'audio/mp3'},
body: bufferAudio,
})
.catch(function(e) {
console.log("error", e);
});
return response;
}
Node Server code:
const express = require('express');
const path = require('path');
const app = express(), port = 3080;
app.post('/api/query', async (req, res) => {
query = req.body;
console.log('query ', query); // Here the body is undefined
/*
Do some processing...
*/
});
When I send a simple json string in the request body (and app.use(express.json()) in the server)it works. But not with the arraybuffer audio. Thanks in advance

how can i send custom sms message with twilio api via express

so im trying to make a simple function in a web that has input and button , and when i click the button twilio api send message with the body of input value life if input is hello the message sent is hello, this is the index.js file which is include the simple function that gonna send the message and i don't know if i should use POST method or get just look
let input = document.querySelector("input").value;
document.querySelector("button").addEventListener("click", whatTheHell);
let whatTheHell = () => {
fetch("/sendSms")
.then((res) => res.json())
.then((res) => console.log(res))
.catch((err) => console.log(err));
};
and this the express.js file that contain the twilio api that gonna send the sms
const express = require("express");
if (process.env.NODE_ENV !== "production") {
require("dotenv").config();
}
const accountSid = process.env.accountSid;
const authToken = process.env.authToken ;
const app = express();
const client = require("twilio")(accountSid, authToken);
app.use(express.json());
app.use(express.static("public"));
app.get("/sendSms", (req, res) => {
client.messages
.create({
body: "message from me",
messagingServiceSid: "MGXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
to: "NUMBER",
})
.then((message) => {
res.json({ message: message }).done();
});
});
app.listen(3000, () => {
console.log("Server Started");
});
so what i want here in body: "message from me" is to be something like this body : user.input or something like that , i tried using post method and did req.body.msg and msg is input.value but it dont accept post method .
Twilio developer evangelist here.
I would recommend making this a POST request. You need to update a few things to get your input from the front end to the server. Let's start with the front end.
Instead of getting the input's value straight away, you should wait until the button is clicked to get the value. Then, when it is clicked, you need to make a POST request with the message you want to send in the body of the request. One way to do that is to JSON stringify an object of data.
let input = document.querySelector("input");
document.querySelector("button").addEventListener("click", whatTheHell);
let whatTheHell = () => {
const message = input.value;
fetch("/sendSms", {
method: "POST",
body: JSON.stringify({ message: message }),
headers: {
'Content-Type': 'application/json'
}
})
.then((res) => res.json())
.then((res) => console.log(res))
.catch((err) => console.log(err));
};
Now, on the server side we need to update your endpoint to receive POST requests. You are already using the express JSON parsing middleware, so the message will be available as req.body.message. We can then use that in the request to Twilio.
const express = require("express");
if (process.env.NODE_ENV !== "production") {
require("dotenv").config();
}
const accountSid = process.env.accountSid;
const authToken = process.env.authToken ;
const app = express();
const client = require("twilio")(accountSid, authToken);
app.use(express.json());
app.use(express.static("public"));
app.post("/sendSms", (req, res) => {
const message = req.body.message;
client.messages
.create({
body: message,
messagingServiceSid: "MGXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
to: "NUMBER",
})
.then((message) => {
res.json({ message: message });
})
.catch((error) => {
console.error(error);
res.status(500).json({ error: error.message });
});
});
app.listen(3000, () => {
console.log("Server Started");
});
And that should work.
You can use a query paramater to send the message to your express server and retrieve them in your server as explained here: How to get GET (query string) variables in Express.js on Node.js?.
If you make your method a post method when sending it you also need to make your express from get to post like so: app.get() -> app.post()

Test request with FormData in jest (without mocking)

I want to test a function with jest that makes a http post request to a server by using the javascript FormData. here is a minimal example of what this could look like:
import axios from "axios";
async function foo() {
let bodyFormData = new FormData();
bodyFormData.append("foo", "myfoo");
resp = await axios({
method: "post",
url: "https://postman-echo.com/post",
data: bodyFormData,
});
return resp;
}
describe("Test", () => {
it("foo test", async () => {
return foo().then(function (result) {
console.log(result);
// here some testing stuff ...
});
});
});
from my research jest cannot use the FormData(). How can i workaround that? Ive seen some solutions here on SO but all of them are somehow mocking the FormData but the probmle with that is then never a real request is send but i want to test if the real request to the real server in working. How can I do that with jest? I've also tried https://github.com/form-data/form-data but it is not working too
Actually you need to use supertest module like that
const supertest = require('supertest');
const app = require('../app');
const request = supertest(app)
describe('POST "/" root endpoint for login & authentication', function() {
it('/register method with missing parameter returns', function(done) {
request
.post('/register')
.send('username=john&password=12345') // x-www-form-urlencoded upload
.set('Accept', 'application/json')
.expect(200)
.end((err, res) => {
if(err) return done(err);
expect(res.body.message).toBe("Username or password is not specified");
return done();
});
});
});
Also you can check this post

Piping requests using gaxios (or axios)

At present I'm performing the trick of piping a request req to a destination url, and piping the response back to res, like so:
const request = require('request');
const url = 'http://some.url.com' + req.originalUrl;
const destination = request(url);
// pipe req to destination...
const readableA = req.pipe(destination);
readableA.on('end', function () {
// do optional stuff on end
});
// pipe response to res...
const readableB = readableA.pipe(res);
readableB.on('end', function () {
// do optional stuff on end
});
Since request is now officially deprecated (boo hoo), is this trick at all possible using the gaxios library? I thought that setting responseType: 'stream' on the request would do something similar as above, but it doesn't seem to work.
SImilarly, can gaxios be used in the following context:
request
.get('https://some.data.com')
.on('error', function(err) {
console.log(err);
})
.pipe(unzipper.Parse())
.on('entry', myEntryHandlerFunction);
Install gaxios:
npm install gaxios
And then use request with the Readable type specified and with responseType set to 'stream'.
// script.ts
import { request } from 'gaxios';
(await(request<Readable>({
url: 'https://some.data.com',
responseType: 'stream'
}))
.data)
.on('error', function(err) {
console.log(err);
})
.pipe(unzipper.Parse())
.on('entry', myEntryHandlerFunction);
// first-example.ts
import { request } from 'gaxios';
// pipe req to destination...
const readableA = (await request<Readable>({
url: 'http://some.url.com',
method: 'POST',
data: req, // suppose `req` is a readable stream
responseType: 'stream'
})).data;
readableA.on('end', function () {
// do optional stuff on end
});
// pipe response to res...
const readableB = readableA.pipe(res);
readableB.on('end', function () {
// do optional stuff on end
});
Gaxios is a stable tool and is used in official Google API client libraries. It's based on the stable node-fetch. And it goes with TypeScript definitions out of the box. I switched to it from the deprecated request and from the plain node-fetch library.
I guess if you provide responseType as stream and use res.data, you will get a stream which you could pipe like this
const {request} = require("gaxios");
const fs = require("fs");
const {createGzip} = require("zlib");
const gzip = createGzip();
(async () => {
const res = await request({
"url": "https://www.googleapis.com/discovery/v1/apis/",
"responseType": "stream"
});
const fileStream = fs.createWriteStream("./input.json.gz");
res.data.pipe(gzip).pipe(fileStream);
})();
Looks like you are trying to basically forward requests from your express server to the clients. This worked for me.
import { request } from "gaxios";
const express = require("express");
const app = express();
const port = 3000;
app.get("/", async (req: any, res: any) => {
const readableA = (await request({
url: "https://google.com",
responseType: "stream",
})) as any;
console.log(req, readableA.data);
const readableB = await readableA.data.pipe(res);
console.log(res, readableB);
});
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});
I imagine more complicated responses from A will require more nuiance in how to pipe it. But then you can probably just interact with express's response object directly and set the appropriate fields.

How to perform an HTTP file upload using express on Cloud Functions for Firebase (multer, busboy)

I am trying to upload a file to Cloud Functions, using Express to handle requests there, but i am not succeeding. I created a version that works locally:
serverside js
const express = require('express');
const cors = require('cors');
const fileUpload = require('express-fileupload');
const app = express();
app.use(fileUpload());
app.use(cors());
app.post('/upload', (req, res) => {
res.send('files: ' + Object.keys(req.files).join(', '));
});
clientside js
const formData = new FormData();
Array.from(this.$refs.fileSelect.files).forEach((file, index) => {
formData.append('sample' + index, file, 'sample');
});
axios.post(
url,
formData,
{
headers: { 'Content-Type': 'multipart/form-data' },
}
);
This exact same code seems to break when deployed to Cloud Functions, where req.files is undefined. Does anyone have any idea what is happening here?
EDIT
I also had a go at using multer, which worked fine locally, but once uploaded to Cloud Functions, this got me an empty array (same clientside code):
const app = express();
const upload = multer();
app.use(cors());
app.post('/upload', upload.any(), (req, res) => {
res.send(JSON.stringify(req.files));
});
There was indeed a breaking change in the Cloud Functions setup that triggered this issue. It has to do with the way the middleware works that gets applied to all Express apps (including the default app) used to serve HTTPS functions. Basically, Cloud Functions will parse the body of the request and decide what to do with it, leaving the raw contents of the body in a Buffer in req.rawBody. You can use this to directly parse your multipart content, but you can't do it with middleware (like multer).
Instead, you can use a module called busboy to deal with the raw body content directly. It can accept the rawBody buffer and will call you back with the files it found. Here is some sample code that will iterate all the uploaded content, save them as files, then delete them. You'll obviously want to do something more useful.
const path = require('path');
const os = require('os');
const fs = require('fs');
const Busboy = require('busboy');
exports.upload = functions.https.onRequest((req, res) => {
if (req.method === 'POST') {
const busboy = new Busboy({ headers: req.headers });
// This object will accumulate all the uploaded files, keyed by their name
const uploads = {}
// This callback will be invoked for each file uploaded
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
console.log(`File [${fieldname}] filename: ${filename}, encoding: ${encoding}, mimetype: ${mimetype}`);
// Note that os.tmpdir() is an in-memory file system, so should only
// be used for files small enough to fit in memory.
const filepath = path.join(os.tmpdir(), fieldname);
uploads[fieldname] = { file: filepath }
console.log(`Saving '${fieldname}' to ${filepath}`);
file.pipe(fs.createWriteStream(filepath));
});
// This callback will be invoked after all uploaded files are saved.
busboy.on('finish', () => {
for (const name in uploads) {
const upload = uploads[name];
const file = upload.file;
res.write(`${file}\n`);
fs.unlinkSync(file);
}
res.end();
});
// The raw bytes of the upload will be in req.rawBody. Send it to busboy, and get
// a callback when it's finished.
busboy.end(req.rawBody);
} else {
// Client error - only support POST
res.status(405).end();
}
})
Bear in mind that files saved to temp space occupy memory, so their sizes should be limited to a total of 10MB. For larger files, you should upload those to Cloud Storage and process them with a storage trigger.
Also bear in mind that the default selection of middleware added by Cloud Functions is not currently added to the local emulator via firebase serve. So this sample will not work (rawBody won't be available) in that case.
The team is working on updating the documentation to be more clear about what all happens during HTTPS requests that's different than a standard Express app.
Thanks to the answers above I've built a npm module for this (github)
It works with google cloud functions, just install it (npm install --save express-multipart-file-parser) and use it like this:
const fileMiddleware = require('express-multipart-file-parser')
...
app.use(fileMiddleware)
...
app.post('/file', (req, res) => {
const {
fieldname,
filename,
encoding,
mimetype,
buffer,
} = req.files[0]
...
})
I was able to combine both Brian's and Doug's response. Here's my middleware that end's up mimicking the req.files in multer so no breaking changes to the rest of your code.
module.exports = (path, app) => {
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))
app.use((req, res, next) => {
if(req.rawBody === undefined && req.method === 'POST' && req.headers['content-type'].startsWith('multipart/form-data')){
getRawBody(req, {
length: req.headers['content-length'],
limit: '10mb',
encoding: contentType.parse(req).parameters.charset
}, function(err, string){
if (err) return next(err)
req.rawBody = string
next()
})
} else {
next()
}
})
app.use((req, res, next) => {
if (req.method === 'POST' && req.headers['content-type'].startsWith('multipart/form-data')) {
const busboy = new Busboy({ headers: req.headers })
let fileBuffer = new Buffer('')
req.files = {
file: []
}
busboy.on('field', (fieldname, value) => {
req.body[fieldname] = value
})
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
file.on('data', (data) => {
fileBuffer = Buffer.concat([fileBuffer, data])
})
file.on('end', () => {
const file_object = {
fieldname,
'originalname': filename,
encoding,
mimetype,
buffer: fileBuffer
}
req.files.file.push(file_object)
})
})
busboy.on('finish', () => {
next()
})
busboy.end(req.rawBody)
req.pipe(busboy)
} else {
next()
}
})}
I have been suffering from the same problem for a few days, turns out that firebase team has put the raw body of multipart/form-data into req.body with their middleware. If you try console.log(req.body.toString()) BEFORE processing your request with multer, you will see your data. As multer creates a new req.body object which is overriding the resulting req, the data is gone and all we can get is an empty req.body. Hopefully the firebase team could correct this soon.
To add to the official Cloud Function team answer, you can emulate this behavior locally by doing the following (add this middleware higher than the busboy code they posted, obviously)
const getRawBody = require('raw-body');
const contentType = require('content-type');
app.use(function(req, res, next){
if(req.rawBody === undefined && req.method === 'POST' && req.headers['content-type'] !== undefined && req.headers['content-type'].startsWith('multipart/form-data')){
getRawBody(req, {
length: req.headers['content-length'],
limit: '10mb',
encoding: contentType.parse(req).parameters.charset
}, function(err, string){
if (err) return next(err);
req.rawBody = string;
next();
});
}
else{
next();
}
});
Cloud functions pre-processes the request object before passing it on further. As such the original multer middleware doesn't work. Furthermore, using busboy is too low level and you need to take care of everything on your own which isn't ideal. Instead you can use a forked version of multer middleware for processing multipart/form-data on cloud functions.
Here's what you can do.
Install the fork
npm install --save emadalam/multer#master
Use startProcessing configuration for custom handling of req.rawBody added by cloud functions.
const express = require('express')
const multer = require('multer')
const SIZE_LIMIT = 10 * 1024 * 1024 // 10MB
const app = express()
const multipartFormDataParser = multer({
storage: multer.memoryStorage(),
// increase size limit if needed
limits: {fieldSize: SIZE_LIMIT},
// support firebase cloud functions
// the multipart form-data request object is pre-processed by the cloud functions
// currently the `multer` library doesn't natively support this behaviour
// as such, a custom fork is maintained to enable this by adding `startProcessing`
// https://github.com/emadalam/multer
startProcessing(req, busboy) {
req.rawBody ? busboy.end(req.rawBody) : req.pipe(busboy)
},
})
app.post('/some_route', multipartFormDataParser.any(), function (req, res, next) {
// req.files is array of uploaded files
// req.body will contain the text fields
})
I ran into this issue today, check here for more details on how to handle files on google cloud (basically you don't need multer).
Here is a middleware I use to extract files. This will keep all your files on request.files and other form fields on request.body for all POST with multipart/form-data content type. It will leave everything else the same for your other middlewares to handle.
// multiparts.js
const { createWriteStream } = require('fs')
const { tmpdir } = require('os')
const { join } = require('path')
const BusBoy = require('busboy')
exports.extractFiles = async(req, res, next) => {
const multipart = req.method === 'POST' && req.headers['content-type'].startsWith('multipart/form-data')
if (!multipart) return next()
//
const busboy = new BusBoy({ headers: req.headers })
const incomingFields = {}
const incomingFiles = {}
const writes = []
// Process fields
busboy.on('field', (name, value) => {
try {
// This will keep a field created like so form.append('product', JSON.stringify(product)) intact
incomingFields[name] = JSON.parse(value)
} catch (e) {
// Numbers will still be strings here (i.e 1 will be '1')
incomingFields[name] = value
}
})
// Process files
busboy.on('file', (field, file, filename, encoding, contentType) => {
// Doing this to not have to deal with duplicate file names
// (i.e. TIMESTAMP-originalName. Hmm what are the odds that I'll still have dups?)
const path = join(tmpdir(), `${(new Date()).toISOString()}-${filename}`)
// NOTE: Multiple files could have same fieldname (which is y I'm using arrays here)
incomingFiles[field] = incomingFiles[field] || []
incomingFiles[field].push({ path, encoding, contentType })
//
const writeStream = createWriteStream(path)
//
writes.push(new Promise((resolve, reject) => {
file.on('end', () => { writeStream.end() })
writeStream.on('finish', resolve)
writeStream.on('error', reject)
}))
//
file.pipe(writeStream)
})
//
busboy.on('finish', async () => {
await Promise.all(writes)
req.files = incomingFiles
req.body = incomingFields
next()
})
busboy.end(req.rawBody)
}
And now in your function, make sure that this is the first middleware you use.
// index.js
const { onRequest } = require('firebase-functions').https
const bodyParser = require('body-parser')
const express = require('express')
const cors = require('cors')
const app = express()
// First middleware I'm adding
const { extractFiles } = require('./multiparts')
app.use(extractFiles)
app.use(bodyParser.urlencoded({ extended: true }))
app.use(bodyParser.json())
app.use(cors({ origin: true }))
app.use((req) => console.log(req.originalUrl))
exports.MyFunction = onRequest(app);
I fixed some bugs G. Rodriguez's response. I add 'field' and 'finish' event for Busboy, and do next() in 'finish' event. This is work for me. As follow:
module.exports = (path, app) => {
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))
app.use((req, res, next) => {
if(req.rawBody === undefined && req.method === 'POST' && req.headers['content-type'].startsWith('multipart/form-data')){
getRawBody(req, {
length: req.headers['content-length'],
limit: '10mb',
encoding: contentType.parse(req).parameters.charset
}, function(err, string){
if (err) return next(err)
req.rawBody = string
next()
})
} else {
next()
}
})
app.use((req, res, next) => {
if (req.method === 'POST' && req.headers['content-type'].startsWith('multipart/form-data')) {
const busboy = new Busboy({ headers: req.headers })
let fileBuffer = new Buffer('')
req.files = {
file: []
}
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
file.on('data', (data) => {
fileBuffer = Buffer.concat([fileBuffer, data])
})
file.on('end', () => {
const file_object = {
fieldname,
'originalname': filename,
encoding,
mimetype,
buffer: fileBuffer
}
req.files.file.push(file_object)
})
})
busboy.on('field', function(fieldname, val, fieldnameTruncated, valTruncated) {
console.log('Field [' + fieldname + ']: value: ' + inspect(val));
});
busboy.on('finish', function() {
next()
});
busboy.end(req.rawBody)
req.pipe(busboy);
} else {
next()
}
})}
Thanks for everyone's help on this thread. I wasted a whole day trying every possible combination and all these different libraries... only to discover this after exhausting all other options.
Combined some of the above solutions to create a TypeScript and middleware capable script here:
https://gist.github.com/jasonbyrne/8dcd15701f686a4703a72f13e3f800c0
If you just want to get a single uploaded file from the request, use busboy to get the file as a readable stream:
const express = require('express')
const Busboy = require('busboy')
express().post('/', (req, res) => {
const busboy = new Busboy({ headers: req.headers })
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
// Do something with `file`, e.g. pipe it to an output stream.
// file.pipe(fs.createWriteStream('upload.pdf')
})
// The original input was moved to `req.rawBody`
busboy.write(req.rawBody)
})
Note that, on top of using Busboy on the server and parsing the rawReq, you may also need to add the following config to your Axios request:
{ headers: { 'content-type': `multipart/form-data; boundary=${formData._boundary}` }};
If you only specify the content-type and not the boundary you get a Boundary not found error on the server. If you remove the headers altogether, instead, Busboy won't parse the fields properly.
See: Firebase Cloud Functions and Busboy not parsing fields or files
I experience the same issue when i deployed my app using firebase function. I was using multer to upload image to amazon s3. I resolve this issue by using the above npm https://stackoverflow.com/a/48648805/5213790 created by Cristóvão.
const { mimetype, buffer, } = req.files[0]
let s3bucket = new aws.S3({
accessKeyId: functions.config().aws.access_key,
secretAccessKey: functions.config().aws.secret_key,
});
const config = {
Bucket: functions.config().aws.bucket_name,
ContentType: mimetype,
ACL: 'public-read',
Key: Date.now().toString(),
Body: buffer,
}
s3bucket.upload(config, (err, data) => {
if(err) console.log(err)
req.file = data;
next()
})
Note that this is for a single file image upload.
The next middleware will have the returned object from s3
{
ETag: '"cacd6d406f891e216f9946911a69aac5"',
Location:'https://react-significant.s3.us-west1.amazonaws.com/posts/1567282665593',
key: 'posts/1567282665593',
Key: 'posts/1567282665593',
Bucket: 'react-significant'
}
In this case, you might need the Location url before you save your data in the db.
I've tried Dougs answer, however the finish was never fired, so i tweaked the code a little bit and got this which works for me:
// It's very crucial that the file name matches the name attribute in your html
app.post('/', (req, res) => {
const busboy = new Busboy({ headers: req.headers })
// This object will accumulate all the uploaded files, keyed by their name
const uploads = {}
// This callback will be invoked for each file uploaded
busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
console.log(`File [${fieldname}] filename: ${filename}, encoding: ${encoding}, mimetype: ${mimetype}`)
// Note that os.tmpdir() is an in-memory file system, so should only
// be used for files small enough to fit in memory.
const filepath = path.join(os.tmpdir(), filename)
uploads[fieldname] = { file: filepath }
console.log(`Saving '${fieldname}' to ${filepath}`)
const stream = fs.createWriteStream(filepath)
stream.on('open', () => file.pipe(stream))
})
// This callback will be invoked after all uploaded files are saved.
busboy.on('finish', () => {
console.log('look im firing!')
// Do whatever you want here
res.end()
})
// The raw bytes of the upload will be in req.rawBody. Send it to busboy, and get
// a callback when it's finished.
busboy.end(req.rawBody)
})

Categories

Resources