How to store PDF in mongodb? - javascript

I was successful in storing images in mongodb using the base64 url. But when I tried to do so with PDFs, it gave an url which does not work. Wait, let me explain, when I put the image base64 url in the req.body of the POST request, the special signs would get disappeared, so I tried encodeURIComponent() method to make it error free. After that I found that storing the huge string in mongodb was too short to fit in the db, so I tried: app.use(express.json({limit: '50mb'})); app.use(express.urlencoded({limit: '50mb', extended: false })); and It worked! but when the client requests the base64 url, it would come encoded, so I put decodeURIComponent() to decode it and was not a great issue nd I got the desired result, yet with the Image one.
The main issue issue is when it comes to PDF. I don't know why it's happening with PDF only! when I make base64 url in CLIENT side and test it, it works fine, but when it comes to server side, all the mess happens. please help me deal with this.
Note: "I don't want to use Gridfs, formidabe, multer etc for file things"
here's my piece of code:
$('#seasonForm').submit(async function (e) {
e.preventDefault();
const form = $(this);
const ImgFile = document.getElementById('seasonThumbnail').files[0];
const PDFFile = document.getElementById('seasonPDF').files[0];
const imgurl = encodeURIComponent(await getBase64(ImgFile));
const PDFurl = encodeURIComponent(await getBase64(PDFFile));
const url = '/uploadSeason';
$.ajax({
type: "POST",
url: url,
data: form.serialize()+`&Version=<%- NxtSeasons %>&image=${imgurl}&PDF=${PDFurl}`,
success: data => {
console.log(data.message);
if (data.status == "error") {
showIt(".alert", data.message, "error");
} else {
showIt(".alert", data.message, "success");
}
}
});
})
wait, now don't get confused with getBase64() and showIt. these are my functions. getBase64() is a promice which returns base64 url of the file and showIt() is type of alert which I made. Now if you don't know what is base64 url, this is the getBase64 one's code:
const getBase64 = (file) => {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result);
reader.onerror = error => reject(error);
});
}
My nodejs code:
app.post("/uploadSeason", admin, async (req, res) => {
try {
const { Name, Desctiption, Version, image, PDF } = req.body;
const newSeason = new Season({
Name,
Desctiption,
Version,
image: encodeURIComponent(image),
PDF: encodeURIComponent(PDF),
});
await newSeason.save();
res.json({
status: "success",
message: "Season added successfully"
});
} catch (e) {
console.log(e);
res.json({
status: "error",
message: e
});
}
});

Related

React Share Dynamically generated QR code on social Media

I would like to know how to share a base64 string as an image over WhatsApp primarily.
I have a react app that generates a QR code and renders it using <img src={base64String} />. Now that img is wrapped around an <a><a/> tag with download attribute, it is working fine. But what I want to accomplish is to be able to have a share button that can share the image on WhatsApp.
I don't want to store it locally or involve file reading if possible because I had a lot of issues implementing it since the QR code will be discarded after download and need not be stored as a file. But if that is the right way I'm ok to work with it.
I tried creating a blob and sharing by using atob(), but that returned an error saying "string not encoded properly" which I didn't want to dig into since it was able to render it as an image and also download by which I assumed the base64 string was correct.
code for Generator:
const GenerateQRCode = (e) => {
e.preventDefault()
QRCode.toDataURL(value, {
margin: 2,
color: {
dark: '#000000',
light: '#ffffff'
},
width: 2000,
height:2000
}, (err, value) => {
if (err) return console.error(err)
console.log(value)
setQr(value)
console.log(data);
})
}
const handleShare =(e)=>{
}
code for that renders the image:
{qr && <>
<img src={qr} alt='' className='qrimg' id='can'/>
<a href={qr} download={`${state.name}.png`} ><button value="Download" >Download</button></a>
</>
}
<button type="" onClick={handleShare}></button>
So, I want that share button to send an image over WhatsApp. My previous attempts for handleShare were messy so I wanted to get fresh inputs or a Solution to do this the right way.
Finally success, that was a hard one! You have to upload the file as a Media into the API and then use that uploaded Media ID to send. Check out the code and replace "xxx" with your values.
Note: Your QR code must be a PNG for this to work. This is not a snippet because it doesn't make sense to run
<img src="data:image/png;base64,iVBORw0KGg...VERY_LONG_STRING_HERE...">
<script>
// Your auth token. Keep it scret, keep it safe.
const Authorization = "Bearer xxx"
const type = 'image/png' // Image type (cannot be gif due to API restriction!)
const baseUrl = "https://graph.facebook.com/v15.0/xxx" // Phone number of sender
const recipient = "xxx" // Phone number of recipient
/** This function creates a File object from an img element which has a base64 image */
function fileFromImg(img) {
var byteString = atob(img.src.split(',')[1]);
var arrayBuffer = new ArrayBuffer(byteString.length);
var arr = new Uint8Array(arrayBuffer);
for (var i = 0; i < byteString.length; i++) {
arr[i] = byteString.charCodeAt(i);
}
const blob = new Blob([arrayBuffer], { type });
return new File([blob], "qr.png", { type })
}
/** This function will upload a File and call the given callback with the result */
function sendFile(f, callback) {
const url = baseUrl + "/media";
const formData = new FormData()
formData.append('file', f)
formData.append('type', type)
formData.append('messaging_product', "whatsapp")
fetch(url, {
method: 'POST',
body: formData,
headers: { Authorization }
})
.then(r => r.json())
.then(callback)
.catch(console.error)
}
/** This takes in a successful upload and sends that forward to receiver */
function handleResponse(response) {
// When it works response will look like {"id": "xxx"}
fetch(baseUrl + "/messages", {
method: 'POST',
body: JSON.stringify({
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient,
"type": "image",
"image": response
}),
headers: { Authorization, "Content-Type": "application/json" }
})
.then(r => r.json())
.then(r => console.log(r))
.catch(console.error)
}
// I choose to trigger the sending when anything is clicked
document.addEventListener("click", () => {
const img = document.querySelector("img")
const file = fileFromImg(img)
sendFile(file, handleResponse)
})
</script>
The recipient will see this;

Upload file to google drive after http get request

I have two functions in separate files to split up the workflow.
const download = function(url){
const file = fs.createWriteStream("./test.png");
const request = https.get(url, function(response) {
response.pipe(file);
});
}
This function in my fileHelper.js is supposed to take a URL with an image in it and then save it locally to test.png
function uploadFile(filePath) {
fs.readFile('credentials.json', (err, content) => {
if (err) return console.log('Error loading client secret file:', err);
// Authorize a client with credentials, then call the Google Drive API.
authorize(JSON.parse(content), function (auth) {
const drive = google.drive({version: 'v3', auth});
const fileMetadata = {
'name': 'testphoto.png'
};
const media = {
mimeType: 'image/png',
body: fs.createReadStream(filePath)
};
drive.files.create({
resource: fileMetadata,
media: media,
fields: 'id'
}, (err, file) => {
if (err) {
// Handle error
console.error(err);
} else {
console.log('File Id: ', file.id);
}
});
});
});
}
This function in my googleDriveHelper.js is supposed to take the filePath of call and then upload that stream into my google drive. These two functions work on their own but it seems that the https.get works asynchronously and if I try to call the googleDriveHelper.uploadFile(filePath) function after the download, it doesn't have time to get the full file to upload so instead a blank file will be uploaded to my drive.
I want to find a way so that when the fileHelper.download(url) is called, it automatically uploads into my drive.
I also don't know if there is a way to create a readStream directly from the download function to the upload function, so I can avoid having to save the file locally to upload it.
I believe your goal as follows.
You want to upload a file retrieving from an URL to Google Drive.
When you download the file from the URL, you want to upload it to Google Drive without creating the file.
You want to achieve this using googleapis with Node.js.
You have already been able to upload a file using Drive API.
For this, how about this answer?
Modification points:
At download function, the retrieved buffer is converted to the stream type, and the stream data is returned.
At uploadFile function, the retrieved stream data is used for uploading.
When the file ID is retrieved from the response value of Drive API, please use file.data.id instead of file.id.
By above modification, the file downloaded from the URL can be uploaded to Google Drive without creating a file.
Modified script:
When your script is modified, please modify as follows.
download()
const download = function (url) {
return new Promise(function (resolve, reject) {
request(
{
method: "GET",
url: url,
encoding: null,
},
(err, res, body) => {
if (err && res.statusCode != 200) {
reject(err);
return;
}
const stream = require("stream");
const bs = new stream.PassThrough();
bs.end(body);
resolve(bs);
}
);
});
};
uploadFile()
function uploadFile(data) { // <--- Modified
fs.readFile("drive_credentials.json", (err, content) => {
if (err) return console.log("Error loading client secret file:", err);
authorize(JSON.parse(content), function (auth) {
const drive = google.drive({ version: "v3", auth });
const fileMetadata = {
name: "testphoto.png",
};
const media = {
mimeType: "image/png",
body: data, // <--- Modified
};
drive.files.create(
{
resource: fileMetadata,
media: media,
fields: "id",
},
(err, file) => {
if (err) {
console.error(err);
} else {
console.log("File Id: ", file.data.id); // <--- Modified
}
}
);
});
});
}
For testing
For example, when above scripts are tested, how about the following script?
async function run() {
const url = "###";
const data = await fileHelper.download(url);
googleDriveHelper.uploadFile(data);
}
References:
Class: stream.PassThrough
google-api-nodejs-client

Download and upload image without saving to disk

Using Node.js, I am trying to get an image from a URL and upload that image to another service without saving image to disk. I have the following code that works when saving the file to disk and using fs to create a readablestream. But as I am doing this as a cron job on a read-only file system (webtask.io) I'd want to achieve the same result without saving the file to disk temporarily. Shouldn't that be possible?
request(image.Url)
.pipe(
fs
.createWriteStream(image.Id)
.on('finish', () => {
client.assets
.upload('image', fs.createReadStream(image.Id))
.then(imageAsset => {
resolve(imageAsset)
})
})
)
Do you have any suggestions of how to achieve this without saving the file to disk? The upload client will take the following
client.asset.upload(type: 'file' | image', body: File | Blob | Buffer | NodeStream, options = {}): Promise<AssetDocument>
Thanks!
How about passing the buffer down to the upload function? Since as per your statement it'll accept a buffer.
As a side note... This will keep it in memory for the duration of the method execution, so if you call this numerous times you might run out of resources.
request.get(url, function (res) {
var data = [];
res.on('data', function(chunk) {
data.push(chunk);
}).on('end', function() {
var buffer = Buffer.concat(data);
// Pass the buffer
client.asset.upload(type: 'buffer', body: buffer);
});
});
I tried some various libraries and it turns out that node-fetch provides a way to return a buffer. So this code works:
fetch(image.Url)
.then(res => res.buffer())
.then(buffer => client.assets
.upload('image', buffer, {filename: image.Id}))
.then(imageAsset => {
resolve(imageAsset)
})
well I know it has been a few years since the question was originally asked, but I have encountered this problem now, and since I didn't find an answer with a comprehensive example I made one myself.
i'm assuming that the file path is a valid URL and that the end of it is the file name, I need to pass an apikey to this API endpoint, and a successful upload sends me back a response with a token.
I'm using node-fetch and form-data as dependencies.
const fetch = require('node-fetch');
const FormData = require('form-data');
const secretKey = 'secretKey';
const downloadAndUploadFile = async (filePath) => {
const fileName = new URL(filePath).pathname.split("/").pop();
const endpoint = `the-upload-endpoint-url`;
const formData = new FormData();
let jsonResponse = null;
try {
const download = await fetch(filePath);
const buffer = await download.buffer();
if (!buffer) {
console.log('file not found', filePath);
return null;
}
formData.append('file', buffer, fileName);
const response = await fetch(endpoint, {
method: 'POST', body: formData, headers: {
...formData.getHeaders(),
"Authorization": `Bearer ${secretKey}`,
},
});
jsonResponse = await response.json();
} catch (error) {
console.log('error on file upload', error);
}
return jsonResponse ? jsonResponse.token : null;
}

Google Cloud Speech-to-Text api encoding issues in Node.js

I'm trying to capture audio from a user's microphone and send it to a server where it will get sent to Google's Speech-to-Text-API for translation. I'm accessing the audio using navigator.mediaDevices.GetuserMedia() which I capture using a MediaRecorder object. When I run the following code I get an error from Google that says "INVALID_ARGUMENT: RecognitionAudio not set." I'm not sure how to set it as the relavant page (https://cloud.google.com/speech-to-text/docs/reference/rest/v1/RecognitionAudio) doesn't say much about it.
Relevant client side code that runs after user presses stop:
mediaRecorder.onstop = function(e) {
var blob = new Blob(chunks, { type : 'audio/flac' });
var reader = new FileReader();
reader.readAsBinaryString(blob);
reader.onloadend = function() {
base64data = reader.result;
writeBinaryFile(base64data)
}
chunks = []; //array to store recording
}
//asynchronous binary file write
function writeBinaryFile(content) {
$.ajax({
type: "POST",
url: "/voice_api",
data: { content: content }
}).done(function(data) {
// TODO: display success status somewhere
});
Server side code running node.js:
app.post("/voice_api", (req, res) => {
const audioBytes = req.body;
// The audio file's encoding, sample rate in hertz, and BCP-47 language code
const audio = {
content: audioBytes,
};
const config = {
languageCode: 'en-US'
};
const request = {
audio: audio,
config: config
};
// Detects speech in the audio file
client
.recognize(request)
.then(data => {
const response = data[0];
const transcription = response.results
.map(result => result.alternatives[0].transcript)
.join('\n');
console.log(`Transcription: ${transcription}`);
res.send(transcription);
})
.catch(err => {
console.error('ERROR:', err);
});
});
If I run the server code with the line "const audioBytes = req.body;" changed to "const audioBytes = req.body.content;" I get an error message that there is bad encoding. I'm not sure if I'm encoding it properly on the client side or if I'm accessing it properly on the server side. Any help would be appreciated. Thanks!
const config = {
// "enableAutomaticPunctuation": true,
"encoding": "LINEAR16",
"languageCode": "en-US",
"model": "default",
"sampleRateHertz": 44100,
audioChannelCount: 2,
enableSeparateRecognitionPerChannel: true,
};

Send a file from mobile to Node js server

I'm doing an application with react-native. Now I'm trying to send an image from the mobile to the server (Node Js). For this I'm using react-native-image-picker. And the problem is that when I send the image it save a file but it's empty not contain the photo. I think that the problem probably is that the server can't access to the path of the image because is in a different device. But I don't know how I can do it.
React-Native:
openImagePicker(){
const options = {
title: 'Select Avatar',
storageOptions: {
skipBackup: true,
path: 'images'
}
}
ImagePicker.showImagePicker(options, (imagen) =>{
if (imagen.didCancel) {
console.log('User cancelled image picker');
}
else if (imagen.error) {
console.log('ImagePicker Error: ', imagen.error);
}
else if (imagen.customButton) {
console.log('User tapped custom button: ', imagen.customButton);
}
else {
let formdata = new FormData();
formdata.append("file[name]", imagen.fileName);
formdata.append("file[path]", imagen.path);
formdata.append("file[type]", imagen.type);
fetch('http://X/user/photo/58e137dd5d45090d0b000006', {
method: 'PUT',
headers: {
'Content-Type': 'multipart/form-data'
},
body: formdata
})
.then(response => {
console.log("ok");
})
.catch(function(err) {
console.log(err);
})
}})}
Node Js:
addPhotoUser = function (req, res) {
User.findById(req.params.id, function(err, user) {
fs.readFile(req.body.file.path, function (err, data) {
var pwd = 'home/ubuntu/.../';
var newPath = pwd + req.body.file.name;
fs.writeFile(newPath, data, function (err) {
imageUrl: URL + req.body.file.name;
user.save(function(err) {
if(!err) {
console.log('Updated');
} else {
console.log('ERROR: ' + err);
}
res.send(user);
});
});
});
});
};
Yes, the problem is that the filepath is on the local device and not the server. You want to send the actual data returned to you by react-native-image-picker not the uri. It looks like that library encodes the data with base64 so you're going to want send that to your server, not the uri returned from the library because it won't be accessible on a remote server.
What this means is that you won't be reading any files on your server but instead just decoding a base64 string in the response body and writing that to your filesystem.
For the client side:
let formdata = new FormData();
formdata.append("file[name]", imagen.fileName);
formdata.append("file[data]", imagen.data); // this is base64 encoded!
formdata.append("file[type]", imagen.type);
fetch('http://X/user/photo/58e137dd5d45090d0b000006', {
method: 'PUT',
headers: {
'Content-Type': 'multipart/form-data'
},
body: formdata
})
On the server side atob to decode from base64 before writing to the filesystem:
let decoded = atob(req.body.data)
// now this is binary and can written to the filesystem
From there:
fs.writeFile(newPath, decoded, function (err) {
imageUrl: newPath;
user.save(function(err) {
if(!err) {
console.log('Updated');
} else {
console.log('ERROR: ' + err);
}
res.send(user);
});
});
Note, you don't need the filesystem write that's in your code because you're decoding the image that was sent as a b64 string in your request.
There also seems to be some oddities with how you're using that user object. You seem to be only passing a function that handles errors and not any actual data. I don't know what ORM you're using so it's hard to say how it should work. Maybe something like this?
user.save({imageUrl:uriReturnedByFsWrite}, (err, data)=>{...})
Good luck :)
Make an object then send that object to the server. The object will consist of name,path and type, like this:
var imageData = {name: 'image1', path: uri, type: 'image/jpeg'}
Above is a one way to send the image data. The other way is to convert it into BLOB so that server side programmer doesn't have to do this task on their end. You can make BLOB by use of react-native-fetch-blob.
One more way is to directly upload the images to the amazon server(s3) and send the link to the backend..
Function that returns base64 string:
var RNFetchBlob = require('react-native-fetch-blob').default;
getImageAttachment: function(uri_attachment, mimetype_attachment) {
return new Promise((RESOLVE, REJECT) => {
// Fetch attachment
RNFetchBlob.fetch('GET', config.apiRoot+'/app/'+uri_attachment)
.then((response) => {
let base64Str = response.data;
var imageBase64 = 'data:'+mimetype_attachment+';base64,'+base64Str;
// Return base64 image
RESOLVE(imageBase64)
})
}).catch((error) => {
// error handling
console.log("Error: ", error)
});
},
Cheers :)

Categories

Resources