I'm building a website for my sister so that she can sell her art. I am using Next.js to set everything up. The website renders the artwork by grabbing an array from a database and mapping through it.
Two Example objects
{
id: 8,
path: "images/IMG_0008.jpg",
size: "9x12x.75",
price: "55",
sold: false
}
{
id: 9,
path: "images/IMG_0009.jpg",
size: "9x12x.75",
price: "55",
sold: false
}
pages/Shop.js
import Card from "../Components/Card";
import fetch from 'node-fetch'
import Layout from "../components/Layout";
function createCard(work) {
return (
<Card
key={work.id}
id={work.id}
path={work.path}
size={work.size}
price={work.price}
sold={work.sold}
/>
);
}
export default function Shop({artwork}) {
return (
<Layout title="Shop">
<p>This is the Shop page</p>
{artwork.map(createCard)}
</Layout>
);
}
export async function getStaticProps() {
const res = await fetch('http://localhost:3000/api/get-artwork')
const artwork = await res.json()
return {
props: {
artwork,
},
}
}
The problem I am running into is that when I try to use mongoose in the api/get-artwork. It will only render the page once and once it is refreshed it will break I believe do to the fact the Schema and Model get redone.
pages/api/get-artwork.js/
const mongoose = require("mongoose");
mongoose.connect('mongodb://localhost:27017/ArtDB', {
useNewUrlParser: true,
useUnifiedTopology: true,
useFindAndModify: false
});
const itemsSchema = {
id: String,
description: String,
path: String,
size: String,
price: Number,
sold: Boolean
};
const Art = mongoose.model("Art", itemsSchema);
export default (req, res) => {
Art.find({sold: false}, (err, foundItems)=> {
if (err) {
console.log(err);
} else {
console.log(foundItems);
res.status(200).json(foundItems);
}
});
};
So to try to fix this I decided to use the native MongoDB driver. Like this.
/pages/api/get-artwork/
const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');
// Connection URL
const url = 'mongodb://localhost:27017';
// Database Name
const dbName = 'ArtDB';
// Create a new MongoClient
const client = new MongoClient(url, {useUnifiedTopology: true});
let foundDocuments = ["Please Refresh"];
const findDocuments = function(db, callback) {
// Get the documents collection
const collection = db.collection('arts');
// Find some documents
collection.find({}).toArray(function(err, arts) {
assert.equal(err, null);
foundDocuments = arts;
callback(arts);
});
}
// Use connect method to connect to the Server
client.connect(function(err) {
assert.equal(null, err);
const db = client.db(dbName);
findDocuments(db, function() {
client.close();
});
});
export default (req, res) => {
res.send(foundDocuments);
};
This works for the most part but occasionally the array will not be returned. I think this is because the page is loading before the mongodb part finishes? So I guess my question is how do I make 100% sure that it loads the art correctly every time whether that be using mongoose or the native driver.
Thanks!
The Next.js team has a good set of example code, which they add to regularly, one of them being Next.js with MongoDB and Mongoose. Check it out, https://github.com/vercel/next.js/tree/canary/examples/with-mongodb-mongoose, and hope this helps if you're still searching for solutions.
A little more complete answer might be helpful here. Here's what's working for us.
I would suggest using the next-connect library to make this a little easier and not so redundant.
Noticed unnecessary reconnects in dev so I bind to the global this property in Node. Perhaps this isn't required but that's what I've noticed. Likely tied to hot reloads during development.
This is a lengthy post but not nearly as complicated as it seems, comment if you have questions.
Create Middleware Helper:
import mongoose from 'mongoose';
// Get your connection string from .env.local
const MONGODB_CONN_STR = process.env.MONGODB_CONN_STR;
const databaseMiddleware = async (req, res, next) => {
try {
if (!global.mongoose) {
global.mongoose = await mongoose.connect(MONGODB_CONN_STR, {
useNewUrlParser: true,
useUnifiedTopology: true,
useFindAndModify: false,
});
}
}
catch (ex) {
console.error(ex);
}
// You could extend the NextRequest interface
// with the mongoose instance as well if you wish.
// req.mongoose = global.mongoose;
return next();
};
export default databaseMiddleware;
Create Model:
Typically the path here might be src/models/app.ts.
import mongoose, { Schema } from 'mongoose';
const MODEL_NAME = 'App';
const schema = new Schema({
name: String
});
const Model = mongoose.models[MODEL_NAME] || mongoose.model(MODEL_NAME, schema);
export default Model;
Implement Next Connect:
Typically I'll put this in a path like src/middleware/index.ts (or index.js if not using Typescript).
Note: the ...middleware here just allows you, see below, to pass in additional middleware on the fly when the handler here is created.
This is quite useful as our handler creator here can have things like logging and other useful middleware so it's not so redundant in each page/api file.
export function createHandler(...middleware) {
return nextConnect().use(databaseMiddleware, ...middleware);
}
Use in Api Route:
Putting it together, we can now use our App Model with ease
import createHandler from 'path/to/above/createHandler';
import App from 'path/to/above/model/app';
// again you can pass in middleware here
// maybe you have some permissions middleware???
const handler = createHandler();
handler.get(async (req, res) => {
// Do something with App
const apps = await App.find().exec();
res.json(apps);
});
export default handler;
In pages/api/get-artwork.js/
const mongoose = require("mongoose");
mongoose.connect("mongodb://localhost:27017/ArtDB", {
useNewUrlParser: true,
useUnifiedTopology: true,
useFindAndModify: false,
});
const itemsSchema = {
id: String,
description: String,
path: String,
size: String,
price: Number,
sold: Boolean,
};
let Art;
try {
// Trying to get the existing model to avoid OverwriteModelError
Art = mongoose.model("Art");
} catch {
Art = mongoose.model("Art", itemsSchema);
}
export default (req, res) => {
Art.find({ sold: false }, (err, foundItems) => {
if (err) {
console.log(err);
} else {
console.log(foundItems);
res.status(200).json(foundItems);
}
});
};
It works just fine for me.
When we use mongoose with express.js, we connect to mongodb and run the code once. But in next.js everytime we need to connect to mongodb, we have to run connection code.
Connection to mongodb
import mongoose from "mongoose";
const dbConnect = () => {
if (mongoose.connection.readyState >= 1) {
// if it is not ready yet return
return;
}
mongoose
.connect(process.env.DB_LOCAL_URI, {
//****** since mongoose 6, we dont need those******
// useNewUrlParser: true,
// useUnifiedTopology: true,
// useFindAndModify: false,
// useCreateIndex: true,
})
.catch((err) => console.log(err))
.then((con) => console.log("connected to db"));
};
export default dbConnect;
Use it inside next api
Before running handler code, you need to connect first
import dbConnect from "../../../config/dbConnect";
dbConnect();
... then write handler code
A concise OO approach
Inspired by #Blujedis answer this is what I ended up with.
/* /api/util/handler.js */
import nextConnect from "next-connect";
import mongoose from "mongoose";
export class Handler {
static dbConnection = null
static dbMiddleware = async (req, res, next) => {
try {
Handler.dbConnection ||= await mongoose.connect(process.env.MONGO_URL)
next()
} catch (err) {
console.error(err);
next(err)
}
}
constructor(...middleware) {
return nextConnect().use(Handler.dbMiddleware, ...middleware);
}
}
Example handler:
/* /api/people.js */
import { Handler } from "./util/handler"
import { Person } from "./models/person"
export default
new Handler()
.get(async (req, res) => {
const people = await Person.find({})
return res.status(200).json(people)
})
Related
I have created the following user schema, including two methods:
getSnapshot()
getLastTweetId()
user.js
const mongoose = require('mongoose')
const getLastTweetId = require('../utilities/getLastTweetId')
const getFollowers = require('../utilities/getFollowers')
const userSchema = new mongoose.Schema({
twitterId: {
type: String,
required: true
},
screenName: {
type: String
},
snapshots: {
type: [snapshotSchema],
default: []
},
createdAt: {
type: Date
},
})
userSchema.method('getSnapshot', async function () {
const { user, snapshot } = await getFollowers({user: this})
await user.save()
return snapshot
})
userSchema.method('getLastTweetId', async function () {
const tweetId = await getLastTweetId({user: this})
return tweetId
})
const User = mongoose.model('User', userSchema)
module.exports = User
When I define a user instance in passport.js, I can call getSnapshot() on user with no problems. (see below)
passport.js
const passport = require('passport')
const mongoose = require('mongoose')
const needle = require('needle')
const { DateTime } = require('luxon')
const User = mongoose.model('User')
// Setup Twitter Strategy
passport.use(new TwitterStrategy({
consumerKey: process.env.TWITTER_CONSUMER_API_KEY,
consumerSecret: process.env.TWITTER_CONSUMER_API_SECRET_KEY,
callbackURL: process.env.CALLBACK_URL,
proxy: trustProxy
},
async (token, tokenSecret, profile, cb) => {
const twitterId = profile.id
const screenName = profile.screen_name
const existingUser = await User.findOne({ twitterId })
if (existingUser) {
// Track if this is a new login from an existing user
if (existingUser.screenName !== screenName) {
existingUser.screenName = screenName
await existingUser.save()
}
// we already have a record with the given profile ID
cb(undefined, existingUser)
} else {
// we don't have a user record with this ID, make a new record
const user = await new User ({
twitterId ,
screenName,
}).save()
**user.getSnapshot()**
cb(undefined, user)
}
}
)
However, when I call getLastTweetId() on a user instance in tweet.js, I receive the following error in my terminal:
TypeError: user.getLastTweetId is not a function
Then my app crashes.
tweets.js
const express = require('express')
const mongoose = require('mongoose')
const User = mongoose.model('User')
const Tweet = mongoose.model('Tweet')
const { DateTime } = require('luxon')
const auth = require('../middleware/auth')
const requestTweets = require('../utilities/requestTweets')
const router = new express.Router()
const getRecentTweets = async (req, res) => {
const twitterId = req.user.twitterId
const user = await User.find({twitterId})
*const sinceId = user.getLastTweetId()*
let params = {
'start_time': `${DateTime.now().plus({ month: -2 }).toISO({ includeOffset: false })}Z`,
'end_time': `${DateTime.now().toISO({ includeOffset: false })}Z`,
'max_results': 100,
'tweet.fields': "created_at,entities"
}
if (sinceId) {
params.since_id = sinceId
}
let options = {
headers: {
'Authorization': `Bearer ${process.env.TWITTER_BEARER_TOKEN}`
}
}
const content = await requestTweets(twitterId, params, options)
const data = content.data
const tweets = data.map((tweet) => (
new Tweet({
twitterId,
tweetId: tweet.id,
text: tweet.text,
})
))
tweets.forEach(async (tweet) => await tweet.save())
}
// Get all tweets of one user either since last retrieved tweet or for specified month
router.get('/tweets/user/recent', auth, getRecentTweets)
module.exports = router
I would really appreciate some support to figure out what is going on here.
Thank you for bearing with me!
My first guess was that the user instance is not created properly in tweets.js, but then I verified via log messages that the user instance is what I expect it to be in both passport.js as well as tweets.js
My second guess was that the problem is that the user instance in the database was created before I added the new method to the schema, but deleting and reinstantiating the entire collection in the db changed nothing.
Next I went about checking if the issue is related to instantiating the schema itself or just importing it and it seems to be the latter, since when I call getLastTweetId in passport.js it also works, when I call getSnapshot() in tweets.js it also fails.
This is where I'm stuck, because as far as I can tell, I am requiring the User model exactly the same way in both files.
Even when I print User.schema.methods in either file, it shows the following:
[0] {
[0] getSnapshot: [AsyncFunction (anonymous)],
[0] getLastTweetId: [AsyncFunction (anonymous)]
[0] }
It looks like my first guess regarding what was wrong was on point, and I was just sloppy in verifying that I'm instantiating the user correctly.
const user = await User.find({twitterId})
The above line was returning an array of users.
Instead, I should have called:
const user = await User.findOne({twitterId})
I did not detect the bug at first, because logging an array that contains only one object looks nearly the same as just logging the object itself, I simply overlooked the square brackets.
Changing that single line fixed it.
created a nodejs module called role-management, and I published it in azure artifacts(the equivalent of npm repository).
here's the code of this model :
const mongoose = require('mongoose');
exports.getRoles = async (db_url,key) => {
await mongoose.connect(db_url, { useNewUrlParser: true });
const UserSchema= new mongoose.Schema({ },{ strict: false });
const User = mongoose.model('User', UserSchema,'users');
const result = await User.find({ $or: [{ name: key }, { email: key }] });
console.log(result[0].roles)
return result[0].roles
}
I tried to install this module on my project, here's the code which use this module :
const getRoles=required('role-management')
const getItems = async (req, res) => {
try {
const roles = await getRoles(
process.env.db-url,
req.Email
)
res.status(200).json(roles)
} catch (error) {
handleError(res, error)
}
}
module.exports = { getItems }
now the problem is, when I try to hit getItem from an api the first time, it works and i'm getting the roles, but when I try the second time i got this error :
(node:21540) UnhandledPromiseRejectionWarning: RangeError [ERR_HTTP_INVALID_STATUS_CODE]: Invalid status code: undefined
is there something wrong with the role-management module ? mongoose connectivity problem ? could help please ?
(1/9/2023) Update : SvelteKit now supports server only load functions and Form actions to send requests to the server.
I want to call my database, but I don't want it be able to get accessed by end users by them going to the API endpoint that I set up. I was wondering how I would be able to just call my database from a file in the lib folder and just returning the data there. When I try it I get the error global not defined:
lib/db.js:
import dotenv from "dotenv";
dotenv.config();
import { MongoClient } from "mongodb";
const uri = process.env["MONGODB_URI"];
const options = {
useUnifiedTopology: true,
useNewUrlParser: true,
};
let client;
let clientPromise;
if (!uri) {
throw new Error("Please add your Mongo URI to .env.local");
}
if (process.env["NODE_ENV"] === "development") {
if (!global._mongoClientPromise) {
client = new MongoClient(uri, options);
global._mongoClientPromise = client.connect();
}
clientPromise = global._mongoClientPromise;
} else {
client = new MongoClient(uri, options);
clientPromise = client.connect();
}
export default clientPromise;
routes/items/index.js:
import clientPromise from "$lib/db";
export async function get() {
const client = await clientPromise;
const db = client.db();
const data = await db.collection("items").find({}).toArray();
const items = data.map(({ name }) => ({ name }));
if (items) {
return {
body: {
items,
},
};
}
}
My attempt:
lib/stores/items.js
import clientPromise from "$lib/db";
import { writable } from "svelte/store";
export const items= writable([]);
const fetchItems = async () => {
const client = await clientPromise;
const db = client.db();
const data = await db.collection("items").find({}).toArray();
const items = data.map(({ name }) => ({ name }));
substances.set(items);
};
fetchItems();
Trying the above code in various places always yields a global not defined error in the client.
I found one question from someone with the same problem, but I couldn't figure out how to create a helper file.
Protecting API is done on back-end side. Usually it either server (like NodeJS) or tools Nginx/Apache (proxy, etc.). You're basically looking for Content-Security-Policy topic, which is vaporous but not related to SvelteKit.
Btw, calling DB directly from the Front-end wouldn't be secure and is not possible.
To get data from any database, you should create enpoint
For user authentication, you can create handle hook:
export async function handle({ request, resolve }) {
let user = await authenticate(request)
request.locals.user = user
request.locals.isAuthenticated = !!user
if (request.path.startsWith('/api')) {
if (!user) {
return {
status: 401,
body: JSON.stringify({
error: {
message: 'Unauthorized'
}
})
}
}
const response = await resolve(request)
return response
}
Good evening all!
I have been stuck on this issue for a while and I can't seem to solve it through sheer Googling and so I am reaching out to you all.
Context:
I am writing a small application that handles all the calendars and basic project information for all the interns at our company because my boss is constantly asking me what they're up to and I wanted to give him something that he could look at, so I decided to solve it with code whilst also learning a new framework in the process(Express).
Right now I have my routes all set up, I have my controllers all set up, and I have my DB cursor all set up. When I make the call to the route I have defined, it runs the getAllUsers() controller function and inside that controller function it makes a call to the database using the getAllUsers() function on the DB cursor, I want the code to wait for the DB cursor to return its result before continuing but it isn't and I can't work out why. The DB cursor code does work because it fetches the data and logs it out fine.
Any help would be greatly appreciated, I have put the three bits of code in question below, let me know if you need me to show more.
p.s ignore the 'here1', 'here2', etc calls, this is how I have been working out what's happening at any point in time.
routes.ts
import express from 'express';
import controllers from './controller.js';
export default (app: express.Application) => {
// Users
app.route('/users').get(controllers.getAllUsers)
app.route('/users').post(controllers.postNewUser)
app.route('/users').delete(controllers.deleteUser)
app.route('/user/:emailAddress').get(controllers.getUser)
app.route('/user/:emailAddress').put(controllers.updateUser)
}
controllers.ts
import express from 'express';
import dbcursor from '../services/dbcursor.js';
// Interfaces
import { Project, User } from '../services/interfaces.js'
const controllers = {
// Users
getAllUsers: async (req: express.Request, res: express.Response) => {
try {
const dbRes = await dbcursor.getAllUsers();
console.log('here 3', dbRes)
res.status(200).json({
message: 'Users fetched succesfully!',
dbRes: dbRes
});
} catch (err) {
res.status(400).json({
message: 'Failed to get users.',
dbRes: err
});
}
},
}
dbcursor.ts
import dotenv from 'dotenv';
import mongodb from 'mongodb'
dotenv.config();
// Interfaces
import { User, Project } from './interfaces'
// DB Client Creation
const { MongoClient } = mongodb;
const uri = process.env.DB_URI || ''
const client = new MongoClient(uri, { useNewUrlParser: true, useUnifiedTopology: true });
const dbcursor = {
// Users
getAllUsers: async () => {
let dbRes;
try {
await client.connect(async err => {
if (err) throw err;
console.log("here 1", dbRes)
const collection = client.db("InternManager").collection("Users");
dbRes = await collection.find().toArray()
console.log("here 2", dbRes)
return dbRes;
});
} catch(err: any) {
return err;
}
},
}
It's generally not a good idea to mix callbacks and promises. Try not passing a callback to the client.connect method, and you should be able to await the promise as expected
getAllUsers: async () => {
let dbRes;
try {
await client.connect();
console.log("here 1", dbRes)
const collection = client.db("InternManager").collection("Users");
dbRes = await collection.find().toArray()
console.log("here 2", dbRes)
return dbRes;
} catch(err: any) {
throw err; // If you're just catching and throwing the error, then it would be okay to just ignore it
}
},
Getting metric has already been registered when trying to publish metrics from service. To avoid that I used register.removeSingleMetric("newMetric"); but problem is that it clear register and past records every time a new call comes in.
Wondering what to address this problem:
export function workController(req: Request, res: Response) {
resolveFeaturePromises(featurePromises).then(featureAggregate => {
return rulesService.runWorkingRulesForHook(featureAggregate, hook.name, headers)
.then(([shouldLog, result]) => {
...
// publish metric
publishHookMetrics(result);
// publish metric
sendResponse(res, new CoreResponse(result.isWorking, result.responseData), req, featureAggregate, hook);
});
}).catch(err => {
sendResponse(res, new CoreResponse(Action.ALLOW, null), req, null, hook);
console.error(err);
});
}
function publishCustomMetrics(customInfoObject: CustomInfoObject) {
const counter = new promClient.Counter({
name: "newMetric",
help: "metric for custom detail",
labelNames: ["name", "isWorking"]
});
counter.inc({
name: customInfoObject.hook,
isWorking: customInfoObject.isWorking
});
}
stack trace
[Node] [2020-07-30T17:40:09+0500] [ERROR] Error: A metric with the name newMetric has already been registered.
application.ts
export async function startWebServer(): Promise<Server> {
if (!isGlobalsInitilized) {
throw new Error("Globals are noit initilized. Run initGlobals() first.");
}
// Setup prom express middleware
const metricsMiddleware = promBundle({
includeMethod: true,
includePath: true,
metricsPath: "/prometheus",
promClient: {
collectDefaultMetrics: {
}
}
});
// start http server
const app = require("express")();
app.use(bodyParser.json());
app.use(metricsMiddleware);
const routeConfig = require("./config/route-config");
routeConfig.configure(app);
const port = process.env.PORT || 3000;
return app.listen(port, function () {
console.log("service listening on port", port);
});
}
versions:
express-prom-bundle: 6.0.0
prom-client: 12.0.0
The counter should be initialize only once, then used for each call. Simplest modification from code you gave would be something like this.
const counter = new promClient.Counter({
name: "newMetric",
help: "metric for custom detail",
labelNames: ["name", "isWorking"]
});
export function publishCustomMetrics(customInfoObject: CustomInfoObject) {
counter.inc({
name: customInfoObject.hook,
isWorking: customInfoObject.isWorking
});
}
BTW, I found this thread while searching a way to avoid the "already been registered" error in unit tests. Many tests was instantiating the same class (from a library) that was initializing a counter in the constructor.
As it is in tests, and I didn't need consistency over the metrics, I found an easy solution is to clear metrics registers at the beginning of each test.
import { register } from "prom-client";
// ...
register.clear();