I am trying to store a json object that contains config info in a Google Drive Appdata file. I am currently writing the app in JS that is run on the client side. Using the Google Drive API, I can currently check for the file in the appdata folder. How would I go about generating a new file and storing it in the appdata folder if the config was not found?
var request = gapi.client.drive.files.list({
'q': '\'appdata\' in parents'
});
request.execute(function(resp) {
for (i in resp.items) {
if(resp.items[i].title == FILENAME) {
fileId = resp.items[i].id;
readFile(); //Function to read file
return;
}
}
//Create the new file if not found
});
The gapi client does not provide a method to upload files to google drive (it does for metadata) but they do still expose an API endpoint.
Here's an example I've been using for the V3 api
function saveFile(file, fileName, callback) {
var file = new Blob([JSON.stringify(file)], {type: 'application/json'});
var metadata = {
'name': fileName, // Filename at Google Drive
'mimeType': 'application/json', // mimeType at Google Drive
'parents': ['appDataFolder'], // Folder ID at Google Drive
};
var accessToken = gapi.auth.getToken().access_token; // Here gapi is used for retrieving the access token.
var form = new FormData();
form.append('metadata', new Blob([JSON.stringify(metadata)], {type: 'application/json'}));
form.append('file', file);
var xhr = new XMLHttpRequest();
xhr.open('post', 'https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart&fields=id');
xhr.setRequestHeader('Authorization', 'Bearer ' + accessToken);
xhr.responseType = 'json';
xhr.onload = () => {
console.log(xhr.response.id); // Retrieve uploaded file ID.
callback(xhr.response);
};
xhr.send(form);
}
And since google drive will allow duplicate filenames since they're unique by ID I use something like this to check if it exists already:
function fileExists(file, fileName){
var request = gapi.client.drive.files.list({
spaces: 'appDataFolder',
fields: 'files(id, name, modifiedTime)'
});
request.execute(function(res){
var exists = res.files.filter(function(f){
return f.name.toLowerCase() === fileName.toLowerCase();
}).length > 0;
if(!exists){
saveFile(file, fileName, function(newFileId){
//Do something with the result
})
}
})
}
Check the documentation about Storing Application Data:
The 'Application Data folder' is a special folder that is only accessible by your application. Its content is hidden from the user, and from other apps. Despite being hidden from the user, the Application Data folder is stored on the user's Drive and therefore uses the user's Drive storage quota. The Application Data folder can be used to store configuration files, saved games data, or any other types of files that the user should not directly interact with.
NOTE:
To be able to use your Application Data folder, request access to the following scope:
https://www.googleapis.com/auth/drive.appdata
If you'll check the sample code on how to insert a file into the Application Data folder(PHP code):
$fileMetadata = new Google_Service_Drive_DriveFile(array(
'name' => 'config.json',
'parents' => array('appDataFolder')
));
$content = file_get_contents('files/config.json');
$file = $driveService->files->create($fileMetadata, array(
'data' => $content,
'mimeType' => 'application/json',
'uploadType' => 'multipart',
'fields' => 'id'));
printf("File ID: %s\n", $file->id);
By adding appDataFolder as a parent for the file will make it write to the appFolder. Then implement you own uploading/cody code to insert the file and its content to the appFolder.
Hope this helps
Related
I am uploading file to S3 bucket using S3 upload function in Node.js. The frontend is built on Angular. But now the client's requirement is that all uploads should direct to s3 bucket via a presigned URL. Does this because of any security concern? The Code that i am currently using to upload files to S3 Bucket is:
async function uploadFile(object){
//object param contains two properties 'image_data' and 'path'
return new Promise(async(resolve, reject) => {
var obj = object.image_data;
var imageRemoteName = object.path+'/'+Date.now()+obj.name;
AWS.config.update({
accessKeyId: ACCESS_KEY,
secretAccessKey: SECRET_KEY,
region: REGION
})
var s3 = new AWS.S3()
s3.upload({
Bucket: BUCKET,
Body: obj.data,
Key: imageRemoteName
})
.promise()
.then(response => {
console.log(`done! - `, response)
resolve(response.Location)
})
.catch(err => {
console.log('failed:', err)
})
})
}
Any Help will be appreciated, Thanks!
Security wise it doesn't make a difference whether you call upload or first create a pre-signed URL, as long as the code you showed does not run within your Angular application, meaning on the client. In that case every client of your application has access to your AWS access key, and secret key. Still, swapping upload with a pre-signed URL won't solve the problem in this case. However, if you use a server such as express and that's where this code is running, you're basically fine.
AWS provides instructions on how to upload objects using a pre-signed URL. The basic steps are:
import { getSignedUrl } from "#aws-sdk/s3-request-presigner";
import { S3Client, PutObjectCommand } from "#aws-sdk/client-s3";
const s3Client = new S3Client({
accessKeyId: ACCESS_KEY,
secretAccessKey: SECRET_KEY,
region: REGION
});
/* ... */
const command = new PutObjectCommand({
Bucket: BUCKET,
Body: obj.data,
Key: imageRemoteName
});
// upload image and return a new signed URL,
// with expiration to download image, if needed.
// Otherwise you can leave `signedUrl` unused.
const signedUrl = await getSignedUrl(s3Client, command, {
expiresIn: 3600,
});
I'm reading this documenation:
https://developers.google.com/drive/api/v3/appdata
This is my code:
var fileMetadata = {
'name': 'config.json',
'parents': ['appDataFolder']
};
var media = {
mimeType: 'application/json',
body: '"sample text"'
};
const request = gapi.client.drive.files.create({
resource: fileMetadata,
media,
fields: 'id'
})
request.execute(function (err, file) {
if (err) {
// Handle error
console.error(err);
} else {
console.log('Folder Id:', file.id);
}
})
I get a 403 error: "The user does not have sufficient permissions for this file."
Does not the user have permission to create a file in his appDataFolder? How to create a file in it?
The scope of gapi client is 'https://www.googleapis.com/auth/drive.appdata' and the user accepted it.
I believe the reason for this error is that you are only using the scope to access the appdata folder, but not the scope to create files. Accessing the app data folder and creating files are two different things. According to your code, you are trying to create a file in the appdata folder.
I suggest you to include both scopes:
https://www.googleapis.com/auth/drive.appdata
https://www.googleapis.com/auth/drive.file
If you are not using incremental authorization, make sure to revoke access and reauthorize again.
Reference: https://developers.google.com/drive/api/v3/about-auth#OAuth2Authorizing
You don't actually need https://www.googleapis.com/auth/drive.file scope to create or delete data inside the appDataFolder. https://www.googleapis.com/auth/drive.appdata scope covers all that.
Try this. Just pass your auth client to the createFile() function.
// Requiring the modular service is much better than requiring the whole GAPI
const GDrive = require('#googleapis/drive');
function createFile(auth) {
const drive = GDrive.drive({version: 'v3', auth});
const fileMetadata = {
'name': 'config.json',
'parents': ['appDataFolder']
};
const media = {
mimeType: 'application/json',
body: '{"TEST": "THIS WORKED"}'
};
drive.files.create({
resource: fileMetadata,
media: media,
fields: 'id'
}).then((resp) => {
console.log('File Id: ', resp.data.id);
}).catch((error) => {
console.error('Unable to create the file: ', error);
});
}
I'm developing a chatbot using Microsoft Bot Framework with Node Js.
My purpose is to send the user a csv file while he asks me something.
I've implemented the following function but while I download the file 'prova.csv' the format is not recognized.
Inspecting well the format in output is of the type: "data"
Anyone can help me what's wrong? Thanks
function (session, results, next) {
if (results.response.entity === 'Si') {
const contentType = 'text/csv';
const response = session.dialogData.far_ric_formato.response.rows;
const csv = response.map(ric => `${ric.num};${ric.pin}`).join('\n');
session.send({
text: 'Ecco il CSV pronto per il download (MOCK)',
attachments: [
{
contentType: contentType,
contentUrl: `data:${contentType};base64,${Buffer.from(csv).toString('base64')}`,
name: 'prova.csv'
}
]
});
Base64 string of a .csv file can't be directly rendered on client side, as a workaround you can for example code like this:
var restify = require('restify');
var fs = require('fs');
bot.dialog('download', (session, result)=>{
fs.readFile('./files/test.csv', function(err, data){
var contentType = 'text/csv';
var base64 = Buffer.from(data).toString('base64');
var msg = new builder.Message(session)
.addAttachment({
contentUrl: 'http://localhost:3978/csv/'+base64, //replace with your server url + base64 string.
contentType: contentType,
name: 'MyTest.csv',
});
session.send(msg);
});
}).triggerAction({matches:/^download/i});
server.get('/csv/:base64code', (req, res, next)=>{
let base64code = req.params.base64code;
res.header('Content-disposition', 'inline; filename=test.csv');
res.header('Content-type', 'application/csv');
res.send(Buffer.from(base64code, 'base64'));
});
When user trigger the download dialog, it will send this file as attachment and when user click on this file, this .csv file will be downloaded in user's client side.
So I have a react native application that's kind of like slack and I'm trying to do image uploads to s3.
I went with getSignedUrl route.
So the client pics a photo,
fetches a signed url to the bucket
then changes the url on the server for that user
then a put request to the signed url that was fetched.
It mostly works the files get in the right bucket and they are photos. but
A) the link makes me download the file instead of displaying it in browser.
B) the file isn't an image...its an xml file and can only be opened in photoshop
I've tried changing the type in the data.append type,
Adding header to the signed request
Adding x-amz- headers to the signed request
hard coding the file type in server
converting image to base64 string with a native module but It still is coming up wrong.
Client Side calls to server
uploadToServer() {
// alert('coming soon!');
//Go back to profile page
this.props.navigation.goBack();
//grab user from navigator params
let user = this.props.navigation.state.params.user
let pic = this.state.selected;
// turn uri into base64
NativeModules.ReadImageData.readImage(pic.uri, (image) => {
console.log(image);
var data = new FormData();
data.append('picture', {
uri: image,
name: pic.filename,
type: 'image/jpeg'
});
//get the signed Url for uploading
axios.post(api.getPhotoUrl, {fileName: `${pic.filename}`}).then((res) => {
console.log("get Photo URL response", res);
//update the user with the new url
axios.patch(api.fetchUserByID(user.id), {profileUrl: res.data.url}).then((resp) => {
console.log("Update User response", resp.data);
}).catch(err => errorHandler(err));
//upload the photo using the signed request url given to me.
//DO I NEED TO TURN DATA INTO A BLOB?
fetch(res.data.signedRequest, {
method: 'PUT',
body: data
}).then((response) => {
console.log("UPLOAD PHOTO RESPONSE: ", response);
}).catch(err => errorHandler(err))
}).catch((err) => errorHandler(err))
})
}
GET SIGNED URL logic from on out
router.post('/users/sign-s3', (req, res) => {
const s3 = new aws.S3({signatureVersion: 'v4', region: 'us-east-2'});
const fileName = `${req.user.id}-${req.body.fileName}`;
const fileType = req.body.fileType;
const s3Params = {
Bucket: AWS_S3_BUCKET,
Key: `images/${fileName}`,
Expires: 60,
ContentType: 'image/jpeg',
ACL: 'public-read'
};
s3.getSignedUrl('putObject', s3Params, (err, data) => {
if (err) {
console.log(err);
return res.end();
}
const returnData = {
signedRequest: data,
url: `https://${AWS_S3_BUCKET}.s3.amazonaws.com/${s3Params.Key}`
};
res.write(JSON.stringify(returnData));
res.end();
return null;
});
});
You need to change content type from image to supported xml format if you want it to be displayed in browser.
Refer this and set content type accordingly.
I'm trying to create and then send zip file to client. I know how to create it but I've got a problem with send it to client. I tried many ways.
I'm sending POST request from Client and as response I want to send a file.
This is my server-site example code
var Zip = require('node-zip');
router.post('/generator', function(req, res, next) {
var zip = new Zip;
zip.file('hello.txt', 'Hello, World!');
var options = {base64: false, compression:'DEFLATE'};
fs.writeFile('test1.zip', zip.generate(options), 'binary', function (error) {
console.log('wrote test1.zip', error);
});
res.setHeader('Content-disposition', 'attachment; filename=test1.zip');
res.download('test1.zip');
}
});
I also tried something like this:
res.setHeader('Content-disposition', 'attachment; filename=' + filename);
res.setHeader('Content-type', mimetype);
var filestream = fs.createReadStream(file);
filestream.pipe(res);
I tried to use such libraries as:
node-zip
archiver
Can anyone explain me how to do that ?
This module works fine too: https://www.npmjs.com/package/adm-zip
Example without creating temporary zip file in server:
var AdmZip = require('adm-zip');
router.get('/zipFilesAndSend', function(req, res) {
var zip = new AdmZip();
// add local file
zip.addLocalFile("./uploads/29/0046.xml");
// get everything as a buffer
var zipFileContents = zip.toBuffer();
const fileName = 'uploads.zip';
const fileType = 'application/zip';
res.writeHead(200, {
'Content-Disposition': `attachment; filename="${fileName}"`,
'Content-Type': fileType,
})
return res.end(zipFileContents);
});
Try this express-easy-zip npm package to generate a zip file from a local folder path and send it as a download to the client.
var zip = require('express-easy-zip');
var app = require('express')();
app.use(zip());
app.get('my-route/zip', async function(req, res) {
var dirPath = __dirname + "/uploads";
await res.zip({
files: [{
path: dirPath,
name: 'Package'
}],
filename: 'Package.zip'
});
});
I haven't worked with node-zip or archiver before (I usually just use the built-in zlib module), but one thing I noticed right away is that you should place res.download inside the callback of writeFile. That way it will only send the file once it has been fully written to disk.
fs.writeFile('test1.zip', zip.generate(options), 'binary', function (error) {
res.download('test1.zip');
});
I hope this solution works for you, if it doesn't feel free to comment.
Also, I think res.download sets the Content-disposition header for you, you don't need to set it manually. Not 100% sure on that one though.
Above solutions work.(above solutions generate zip and send it to frontend as data in response. In order to make it as downloadable following code will work) I was using express-zip. It is compressing files and sending data to frontend from backend(node). But in frontend I was getting only data in response. In my case I want user can be able to download the zip which sent by server. To solve this I followed following approach. For generating download window in browser i used downloadjs (we can follow another approach but i find this easy)
Front-End
const download = require('downloadjs')
return axios({
url:process.env.API_HOST+'/getuploadedfiles',
method:'get',
headers:{
'Content-Type': 'multipart/form-data',
withCredentials:true,
},
responseType:'arraybuffer' // If we don't mention we can't get data in desired format
})
.then(async response => {
console.log("got al files in api ");
let blob = await new Blob([response.data], { type: 'application/zip' }) //It is optional
download(response.data,"attachement.zip","application/zip") //this is third party it will prompt download window in browser.
return response.data;
})
Backe-End
const zip = require('express-zip');
app.use('/getuploadedfiles',function(req,res){
res.zip([
{path:'/path/to/file/file2.PNG',name:'bond.png'},
{path:'/path/to/file/file1.PNG',name:'james.png'}
])