I am working on a service which uses restify and mongoose as odm.
Below is the service response which I get after DB call.
[
{
"_id": "5a548b7c025cfcffdd286e0f",
"createdAt": "2018-01-09T09:29:32.515Z",
"updatedAt": "2018-01-09T10:59:49.159Z",
"subject": "Hello buddy.",
"body": "i am good and fine.",
"category": "email service",
"category_id": "232",
"author": "5a5485834c2274ce5d5e54f8",
"__v": 0,
"images": [
"https://google.com"
],
"email content": [
{
"_id": "5a549dcb80a16b2e822357ae",
"createdAt": "2018-01-09T10:47:39.883Z",
"updatedAt": "2018-01-09T10:47:39.883Z",
"body": "my first comment",
"name": "asdf"
"password":"312312"
"__v": 0,
"photos": []
}
]
}]
Emails.find({_id: emaiId, author_id: author_id}).
populate('emailcontent').exec(function(err, listOfEmails) {
if (err) {
console.error(err);
return next(
new errors.InvalidContentError(err.errors.name.message),
);
}
res.send(listOfEmails);
next();
This is a small part of my response. Now, I do not want to send all the response back to the client. For example, username, password, phone no etc. So I have a lot of data which I do not want to send back. How do I do this in over here?
Delete these fields manually before sending data back:
listOfEmails.forEach(email => {
// for array properties
email['email content'].forEach(content => {
delete content.name;
delete content.password;
// ... and so on
});
// for usual properties
delete email._id;
});
res.send(listOfEmails);
Answers above are about black listing fields, but you can also try whitelisting, e.g.
const croppedEmails = emails.map( (email) => {
body: email.body,
...(rest of needed fields)
});
With this approach when you will add new field which is confidential you will not have to add new delete, on the other hand when adding new public field you will also need to add it to above function.
Oh, and there is also projection Mongoose, find, return specific properties
You can use map() to create a new array that you send as response. Use delete to delete fields.
const newListOfEmails = listOfEmails.map((email) => {
delete email._id;
delete email.createdAt;
delete email.updatedAt;
for (let eIdx = 0; eIdx < email["email content"].length; eIdx += 1) {
delete email["email content"][eIdx].password;
}
return email;
});
res.send(newListOfEmails);
Related
I have been working on the backend of my app. At this point, it can access all data in a data base, and output it. I'm trying to implement some queries, so that the user can filter out the content that is returned. My DAL/DAO, looks like this
let mflix //Creates a variable used to store a ref to our DB
class MflixDAO {
static async injectDB(conn){
if(mflix){
return
}
try{
mflix = await conn.db(process.env.JD_NS).collection("movies")
}catch(e){
console.error('Unable to establish a collection handle in mflixDAO: ' + e)
}
}
// Creates a query to fetch data from the collection/table in the DB
static async getMovies({
mflix.controller
filters = null,
page = 0,
moviesPerPage = 20,
} = {}) {
let query
if (filters){
// Code
if("year" in filters){
query = {"year": {$eq: filters["year"]}}
}
// Code
}
// Cursor represents the returned data
let cursor
try{
cursor = await mflix.find(query)
}catch(e){
console.error('Unable to issue find command ' + e)
return {moviesList: [], totalNumMovies: 0}
}
const displayCursor = cursor.limit(moviesPerPage).skip(moviesPerPage * page)
try{
const moviesList = await displayCursor.toArray() // Puts data in an array
const totalNumMovies = await mflix.countDocuments(query) // Gets total number of documents
return { moviesList, totalNumMovies}
} catch(e){
console.error('Unable to convert cursor to array or problem counting documents ' + e)
return{moviesList: [], totalNumMovies: 0}
}
}
}
export default MflixDAO
Just so you know, I am using a sample database from MongoDB Atlas. I am using Postman to test HTTP requests. All the data follows JSON format
Anyway, when I execute a basic GET request. The program runs without any problems. All the data outputs as expected. However, if I execute something along the lines of
GET http://localhost:5000/api/v1/mflix?year=1903
Then moviesList returns an empty array [], but no error message.
After debugging, I suspect the problem lies either at cursor = await mflix.find(query) or displayCursor = cursor.limit(moviesPerPage).skip(moviesPerPage * page), but the callstacks for those methods is so complex for me, I don't know what to even look for.
Any suggestions?
Edit: Here is an example of the document I am trying to access:
{
"_id": "573a1390f29313caabcd42e8",
"plot": "A group of bandits stage a brazen train hold-up, only to find a determined posse hot on their heels.",
"genres": [
"Short",
"Western"
],
"runtime": 11,
"cast": [
"A.C. Abadie",
"Gilbert M. 'Broncho Billy' Anderson",
"George Barnes",
"Justus D. Barnes"
],
"poster": "https://m.media-amazon.com/images/M/MV5BMTU3NjE5NzYtYTYyNS00MDVmLWIwYjgtMmYwYWIxZDYyNzU2XkEyXkFqcGdeQXVyNzQzNzQxNzI#._V1_SY1000_SX677_AL_.jpg",
"title": "The Great Train Robbery",
"fullplot": "Among the earliest existing films in American cinema - notable as the first film that presented a narrative story to tell - it depicts a group of cowboy outlaws who hold up a train and rob the passengers. They are then pursued by a Sheriff's posse. Several scenes have color included - all hand tinted.",
"languages": [
"English"
],
"released": "1903-12-01T00:00:00.000Z",
"directors": [
"Edwin S. Porter"
],
"rated": "TV-G",
"awards": {
"wins": 1,
"nominations": 0,
"text": "1 win."
},
"lastupdated": "2015-08-13 00:27:59.177000000",
"year": 1903,
"imdb": {
"rating": 7.4,
"votes": 9847,
"id": 439
},
"countries": [
"USA"
],
"type": "movie",
"tomatoes": {
"viewer": {
"rating": 3.7,
"numReviews": 2559,
"meter": 75
},
"fresh": 6,
"critic": {
"rating": 7.6,
"numReviews": 6,
"meter": 100
},
"rotten": 0,
"lastUpdated": "2015-08-08T19:16:10.000Z"
},
"num_mflix_comments": 0
}
EDIT: It seems to be a datatype problem. When I request a data with a string/varchar type, the program returns values that contain that value. Example:
Input:
GET localhost:5000/api/v1/mflix?rated=TV-G
Output:
{
"_id": "XXXXXXXXXX"
// Data
"rated" = "TV-G"
// Data
}
EDIT: The problem has nothing to do with anything I've posted up to this point it seems. The problem is in this piece of code:
let filters = {}
if(req.query.year){
filters.year = req.query.year // This line needs to be changed
}
const {moviesList, totalNumMovies} = await MflixDAO.getMovies({
filters,
page,
moviesPerPage,
})
I will explain in the answer below
Ok so the problem, as it turns out, is that when I make an HTTP request, the requested value is passed as a string. So in
GET http://localhost:5000/api/v1/mflix?year=1903
the value of year is registered by the program as a string. In other words, the DAO ends up looking for "1903" instead of 1903. Naturally, year = "1903" does not exist. To fix this, the line filters.year = req.query.year must be changed to filters.year = parseInt(req.query.year).
I have user object structured like this:
{
"id": "",
"username": "",
"name": "",
"bio": "",
"email": "",
"profileImg": "",
"phoneNumber": 0,
"messagingPoints": 0,
"funds": 0,
"inventoryId": "",
"lastLogin": "2022-02-23T03:27:13.535Z",
"isPrivate": false,
"messagesReceived": []
}
I want to be able to reach a patch endpoint to update any of these fields. For example, /api/user?id=userId&name=John, should be able to grab the field "name" and set it to John. /api/user/id=?id=userId&email=abc#gmail.com should grab the email field and set it to abc#gmail.com
I am struggling to find docs for MongoDB to accomplish this, so I'm wondering if it is not possible? Do I need a specific endpoint for each of these update operations (ex: /api/user/name?id=userId&value=John instead of /api/user?id=userId&name=John)?
If it is possible, how could I accomplish this? Thanks!
You can pass user ID in update filter. Also you can pass data in request body of the PUT request instead of query parameters.
app.put('/api/user', async (req, res) => {
const { id, ...data } = req.body;
// Filter out any invalid fields
// Update documents where id field is equal to value of id in request body
await collection.updateOne(
{ id },
{ $set: data }
);
return res.json({ data: "User updated" })
})
I am total noob to AWS, DataStore, and DynamoDB. I can't figure out why this isn't working.
If I attempt query an object related to my users ID from Cognito this seems to work. If I console log I get an array with one User.
let profile = await DataStore.query(User, (u) =>
u.id('eq', user.attributes.sub)
);
// Returns [Model]
console.log('[LOADING USER DATA]', profile);
If I attempt query an object related to my users ID like this. (as recommended by the docs). I get undefined.
let profile = await DataStore.query(User, user.attributes.sub);
// Returns undefined
console.log('[LOADING USER DATA]', profile);
Dynamo Data looks fine to me but am I using the wrong primary key ID field or something?
{
"id": "929e6e62-e3a0-4b84-b050-72c53eebda93",
"_lastChangedAt": 1644433699,
"status": "Active",
"createdAt": "2022-02-09T19:08:18.548Z",
"name": "Test",
"orgs": [
"b6508155-f5cf-49b7-88d6-0e605677c4e4"
],
"_version": 1,
"role": "User",
"updatedAt": "2022-02-09T19:08:18.548Z",
"orgID": "b6508155-f5cf-49b7-88d6-0e605677c4e4",
"title": ""
}
What am I doing wrong?
thank you for the time you will take to resolve my issue !
I am not sure that Google app script allows to do what I need.
Could you please tell me if it is possible?
If yes, do you have already a script code to do it?
I have created a file which I have shared it with others colleagues (in a shared drive), and it is used as a "template".
When a colleague creates a copy of it, I would like that the script to give me the new Google sheet id created from the model and saved this id in my Google sheet dashboard?
Is it possible with appscript?
Thanks a lot and have a good day !
Copy Spreadsheet and Save Id
function copySpreadsheetAndSaveId() {
const fileId = "fileid";
const ss = SpreadsheetApp.getActive():
const sh = ss.getSheetByName("Dashboard");
sh.getRange(sh.getLastRow() + 1, 1).setValue(DriveApp.getFileById(fileId).makeCopy().getId());//appends the id to the bottom of column one in the sheeet named Dashboard.
}
If you want users to be able to open the Spreadsheet then you can't restrict them copying it by script only
I can think of a couple of workarounds:
Workaround 1:
Make the Spreadsheet private, and create a web app which runs as you but is accessible by other users. On doGet(), create a copy of the Spreadsheet and share it with the email returned from Session.getActiveUser().getEmail():
function doGet() {
// Check if security policy gets email address:
const user = Session.getActiveUser()
if (!user.getEmail()) {
return ContentService.createTextOutput('Unable to retrieve user.')
}
const ss = DriveApp.getFileById("template-spreadsheet-id")
const newFile = ss.makeCopy().addEditor(user)
const html = `File copied, click here to open.`
return HtmlService.createHtmlOutput(html)
}
Pros:
Should work for anyone within the same domain as you
You can directly retrieve the ID on copy and save it to your database
Cons:
Security policy might stop you being able to get the user
What's to stop them from just copying the copy?
Workaround 2:
If you're an admin user, you could use the Drive Audit Activity API to check for domain-wide copy events of a given file ID. It's a bit more involved and assumes you have a client set up in GCP but will have a bigger catch-radius than the first workaround, and also doesn't involve restricting access to the template or creating a Web App:
function getAuditLog() {
const baseUrl = "https://admin.googleapis.com/admin/reports/v1/activity/users/all/applications/drive"
const apiKey = "api-key-obtained-from-gcp"
const params + `eventName=copy&key=${apiKey}`
const headers = {
"Authorization": `Bearer ${ScriptApp.getOAuthToken()}`,
"Accept": "application/json"
}
const response = UrlFetchApp.fetch(`${baseUrl}?${params}`, {
"method": "get",
"headers": headers"
})
const responseData = JSON.parse(response.getContentText())
}
You'll then have to process the response. responseData contains an items key which is an array of copy events in the audit report:
{
"kind": "admin#reports#activities",
"etag": "\"xxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxx\"",
"items": [
{
"kind": "admin#reports#activity",
"id": {
"time": "2022-01-21T10:03:12.793Z",
"uniqueQualifier": "-XXXXXXXXXXXXXX",
"applicationName": "drive",
"customerId": "xxxxxxx"
},
"etag": "\"xxxxxxxxxxxxx/xxxxxxxxxxx\"",
"actor": {
"email": "user#example.com",
"profileId": "XXXXXXXXXXXX"
},
"ipAddress": "0000:0000:0000:0000:0000:0000:0000:0000",
"events": [
{
"type": "access",
"name": "copy",
"parameters": [
{
"name": "primary_event",
"boolValue": false
},
{
"name": "billable",
"boolValue": true
},
{
"name": "old_value",
"multiValue": [
"Spreadsheet Template File Name"
]
},
{
"name": "new_value",
"multiValue": [
"Copy of Spreadsheet Template File Name"
]
},
{
"name": "doc_id",
"value": "new-spreadsheet-id"
},
{
"name": "doc_type",
"value": "spreadsheet"
},
{
"name": "is_encrypted",
"boolValue": false
},
{
"name": "doc_title",
"value": "Copy of Spreadsheet Template File Name"
},
{
"name": "visibility",
"value": "private"
},
{
"name": "actor_is_collaborator_account",
"boolValue": false
},
{
"name": "owner",
"value": "user#example.com"
},
{
"name": "owner_is_shared_drive",
"boolValue": false
},
{
"name": "owner_is_team_drive",
"boolValue": false
}
]
}
]
}
]
...
}
You will have to filter out the reponse from here, however. For each element in the items array, the events key contains the information you will need to look for:
old_value is the original template spreadsheet's name
doc_id is the ID of the new spreadsheet
items.actor is the email of the person that completed the action.
References:
Example Audit request using the Try this API feature
I'm new to JavaScript and Node etc but enjoying the experience of developing - last serious development i did was in the 90s - I used to be a 370 assembler programmer back 30 years ago!
I've been stuck on this all day.
My issue is that I am trying to add a new element to the entries in an array of objects. I have simplified my code here in the hope that I am doing something blindingly stupid and obviously wrong though I have tried this in javascript in the browser and I know I am barking up the right tree.
What I am trying to do is add a new key/value pair to the collectibles returned. I have simplified the code here but in essence the value is coming for a different collection. the issue is that the lines that set the new keys are having no effect on my objects in the array collectibles. I have tried several methods all of which I believe should work.
I can read the key/values in the returned array of objects and I can also set existing keys to new values, but setting a new key is not working.
Here is my code:
var sendJsonResponse = function (res, status, content) {
res.status(status);
res.json(content);
};
var mongoose = require("mongoose");
var Coll = mongoose.model("collectible");
var Img = mongoose.model("image");
module.exports.collectiblesList = function (req, res) {
var firstImage_ids = [];
var imageDir = "https://s3.amazonaws.com/xxxxxxxxxxx/images/";
var defaultImage = imageDir + "default.jpg";
// get the collectibles
Coll.find(req.body).exec(function (err, collectibles) {
if (!collectibles) {
// not found
sendJsonResponse(res, 404, {"message": "no collectibles found"});
return;
}
if (err) {
console.log(collectible);
sendJsonResponse(res, 400, err);
return;
}
//found the collectibles
// now find the list of images we need to pull (each collectible can have multiple images
collectibles.forEach(function (collectible, index) {
if (collectible.image_ids) {
firstImage_ids.push(collectible.image_ids[0]);
collectibles[index].imageUrl = defaultImage; // this is the line that has no effect.
collectible.imageUrl = defaultImage; // tried this too.
collectible["imageUrl"] = defaultImage; // and this. :-(
console.log(collectible);
}
});
sendJsonResponse(res, 200, collectibles);
});
};
Using Postman to test this API. This is what is being returned:
[
{
"_id": "5a43c61134aaea2025158cab",
"name": "A-0_5",
"title": "Occoneechee Lodge 104: A-0_5",
"issueDate": null,
"quantity": null,
"event_id": null,
"type": "A - Arrowhead patches",
"tag_ids": [
"5a401a17b720186ab61c649e"
],
"description": "Elangomat, no name",
"organisation_id": "5a413340b720186ab61c661e",
"supercedes_id": null,
"sortName": "A-000.5",
"image_ids": [
"5a41685ab720186ab61c663d"
],
"deleted": false,
"schemaVersion": "1.0"
},
{
"_id": "5a43c63334aaea2025158cac",
"name": "A-1",
"title": "Occoneechee Lodge 104: A-1",
"issueDate": null,
"quantity": null,
"event_id": null,
"type": "A - Arrowhead patches",
"tag_ids": [
"5a401a17b720186ab61c649e"
],
"description": "Service",
"organisation_id": "5a413340b720186ab61c661e",
"supercedes_id": null,
"sortName": "A-001",
"image_ids": [
"5a41685ab720186ab61c663e"
],
"deleted": false,
"schemaVersion": "1.0"
}]
i.e. it contains no imageUrl keys.
thanks for the assist.
WWW
GT
I might have misunderstood. But if your issue is that collectibles[i].imageUrl is later on returning undefined/null, then i believe it's simply because you need to add
collectibles.save();
after setting the collectibles[i].imageUrl value. you can add that line after the foreach loop, before the json response.
collectibles.forEach(function (collectible, index) {
if (collectible.image_ids) {
firstImage_ids.push(collectible.image_ids[0]);
collectible.imageUrl = defaultImage;
}
});
collectibles.save(); //here, this saves the changes done to the mongoose object.
sendJsonResponse(res, 200, collectibles);
I solved it in the end after a lot of effort. This article explains (but i don't fully yet follow the explanation)
I found that the object has more metadata in it and the actual object is in _doc, so the first (and i suspect frowned upon solution) is to update the code to:
collectible._doc.imageUrl = defaultImage;
The better way was to simply add imageUrl to the mongoose model and leave _doc out of it. The issue there is that this is not a field i intend to populate in the mongoDB.