I have this code:
app.post('/pst', function(req, res) {
var url = req.body.convo;
myAsyncFucntion(url).then(result => {
console.log('TAKE A LOOK AT THIS!');
//transforming array to string to pass to Buffer.from()
//then we remove ',' with newlines, so each index of array is a new line
var str = result.toString();
result = str.split(',').join('\r\n');
//clever way to send text file to client from the memory of the server
var fileContents = Buffer.from(result, 'ascii');
var readStream = new stream.PassThrough();
readStream.end(fileContents);
res.set('Content-disposition', 'attachment; filename=' + fileName);
res.set('Content-Type', 'text/plain');
readStream.pipe(res);
//garbage collecting. i don't know if it's neccessary
result = '';
str = '';
}).catch(err => {
console.log(err);
res.render('error.ejs');
})
});
This code will run an async function and serve the user some data from the memory, as a text file.
I am planing on using sockets and notifying the client that the work is done.
The client will enter a link and will download a file.
So i plan to take the local variable result and export it in a global variable.
This way, the app.get() will have access to it, and when the user follos that link, it will serve the file.
But a user told me that global variables are shared between instances.
Is this true? So if two (or more) users try to get their results at the same time, the global variables will be the same
for both of them?
Is this true?
Yes. (Well, effectively yes. The real answer is that there aren't "multiple instances" in the first place: You have one server which multiple users are making multiple requests to.)
If you want to associate data with a particular browser session, then use a session (NPM modules to handle sessions exist for Express).
This really depends on where you declare those variables.
Please read some documentation on "variable scope" in JS.
It's hard to explain it here when there are plenty of nicely written and pictured explanations out there.
If some variable is declared within the code-area that is read anew for every user-request, then this variable would be different for each user.
If you declare the variable ourside oth that part, then that content is the same for every action of your Node-application.
Related
I'm creating a Node/Express application that is dependent on data that is fetched from an API. The issue is that my application has to fetch quite often from the API and do some intensive calculations every time, so I want to cache results from both the API and the intensive calculations.
I already wrote out some very rudimentary caching functionality (i.e. writing values into a text file), but I'm curious if there's a way to keep the values in memory in some sort of object for easy access across the whole application (e.g. across multiple files) instead of having a util function to read in the values from a text file every time.
For example, maybe I have a paradigm that looks like this:
When the server first initializes, I fetch for the data, and create some sort of cached value:
const data = await api.get('colors');
const map = {
red: data.red,
blue: data.blue,
green: data.green,
}
In one of my controllers, I want to access map and be able to use the values. So maybe I'd be able to do something like this:
import { map } from './utils/mapManager.js'
const getGreenValue = (req, res) => {
res.send(map.green);
}
What is the best way to design a system that is trying to achieve this behavior?
Thanks in advance, and happy to answer any clarifying questions.
You can use expressjs app.locals property.
To write data:
app.locals.map = map;
To read data:
let map = app.locals.map;
Docs:
https://expressjs.com/en/api.html#app.locals
Please refer this doc -> Global
You can use global object in this case.
console.log(globalString); // Output: "This can be accessed anywhere!"
globalString = "Check me out now";
console.log(globalString); // Output: "Check me out now"
globalString = undefined;
console.log(globalString); // Output: undefined```
I suggest you to try express specific app.locals or res.locals.
Say you have an app initialized like
const app = express();
const data = await api.get('colors');
const map = {
red: data.red,
blue: data.blue,
green: data.green,
}
//Assign it to locals
app.locals.map = map;
Now the map is available throughout the lifetime of application and you can access the same in views just by calling the variable map.
res.locals does the same thing but it is available only until the lifecycle of request. Say, you want to send a particular color based on the request. You can do it like this.
app.use(function(req, res, next){
if(req.params.id==='admin')
res.locals.color = req.app.locals.map.red;
else
res.locals.color = req.app.locals.map.blue;
next();
});
With the above middleware, you will be able to have a variable color that is available to all the succeeding middlewares and the final view. You can access the same in view using color. This is persistent till the lifetime of that request.
Dear all,
Im working with JS for some weeks and now I need a bit of clarification. I have read a lot of sources and a lot of Q&A also in here and this is what I learned so far.
Everything below is in connection with Node.js and Socket.io
Use of globals in Node.js "can" be done, but is not best practice, terms: DONT DO IT!
With Sockets, everything is treated per socket call, meaning there is hardly a memory of previous call. Call gets in, and gets served, so no "kept" variables.
Ok I build up some chat example, multiple users - all get served with broadcast but no private messages for example.
Fairly simple and fairly ok. But now I am stuck in my mind and cant wrap my head around.
Lets say:
I need to act on the request
Like a request: "To all users whose name is BRIAN"
In my head I imagined:
1.
Custom object USER - defined globally on Node.js
function User(socket) {
this.Name;
this.socket = socket; }
2.
Than hold an ARRAY of these globally
users = [];
and on newConnection, create a new User, pass on its socket and store in the array for further action with
users.push(new User(socket));
3.
And on a Socket.io request that wants to contact all BRIANs do something like
for (var i = 0; i < users.length; i++) {
if(user[i].Name == "BRIAN") {
// Emit to user[i].socket
}}
But after trying and erroring, debugging, googling and reading apparently this is NOT how something like this should be done and somehow I cant find the right way to do it, or at least see / understand it. can you please help me, point me into a good direction or propose a best practice here? That would be awesome :-)
Note:
I dont want to store the data in a DB (that is next step) I want to work on the fly.
Thank you very much for your inputs
Oliver
first of all, please don't put users in a global variable, better put it in a module and require it elsewhere whenever needed. you can do it like this:
users.js
var users = {
_list : {}
};
users.create = function(data){
this._list[data.id] = data;
}
users.get = function(user_id){
return this._list[user_id];
};
users.getAll = function(){
return this._list;
};
module.exports = users;
and somewhere where in your implementation
var users = require('users');
For your problem where you want to send to all users with name "BRIAN",
i can see that you can do this good in 2 ways.
First.
When user is connected to socketio server, let the user join a socketio room using his/her name.
so it will look like this:
var custom_namespace = io.of('/custom_namespace');
custom_namespace.on('connection', function(client_socket){
//assuming here is where you send data from frontend to server
client_socket.on('user_data', function(data){
//assuming you have sent a valid object with a parameter "name", let the client socket join the room
if(data != undefined){
client_socket.join(data.name); //here is the trick
}
});
});
now, if you want to send to all people with name "BRIAN", you can achieve it by doing this
io.of('/custom_namespace').broadcast.to('BRIAN').emit('some_event',some_data);
Second.
By saving the data on the module users and filter it using lodash library
sample code
var _lodash = require('lodash');
var Users = require('users');
var all_users = Users.getAll();
var socket_ids = [];
var users_with_name_brian = _lodash.filter(all_users, { name : "BRIAN" });
users_with_name_brian.forEach(function(user){
socket_ids.push(user.name);
});
now instead of emitting it one by one per iteration, you can do it like this in socketio
io.of('/custom_namespace').broadcast.to(socket_ids).emit('some_event',some_data);
Here is the link for lodash documentation
I hope this helps.
How simultaneously to render a page and transmit my custom data to browser. As i understood it needs to send two layers: first with template and second with JSON data. I want to handle this data by backbone.
As i understood from tutorials express and bb app interact as follows:
res.render send a page to browser
when document.ready trigger jQuery.get to app.get('/post')
app.get('/post', post.allPosts) send data to page
This is three steps and how to do it by one?
var visitCard = {
name: 'John Smit',
phone: '+78503569987'
};
exports.index = function(req, res, next){
res.render('index');
res.send({data: visitCard});
};
And how i should catch this variable on the page- document.card?
I created my own little middleware function that adds a helper method called renderWithData to the res object.
app.use(function (req, res, next) {
res.renderWithData = function (view, model, data) {
res.render(view, model, function (err, viewString) {
data.view = viewString;
res.json(data);
});
};
next();
});
It takes in the view name, the model for the view, and the custom data you want to send to the browser. It calls res.render but passes in a callback function. This instructs express to pass the compiled view markup to the callback as a string instead of immediately piping it into the response. Once I have the view string I add it onto the data object as data.view. Then I use res.json to send the data object to the browser complete with the compiled view :)
Edit:
One caveat with the above is that the request needs to be made with javascript so it can't be a full page request. You need an initial request to pull down the main page which contains the javascript that will make the ajax request.
This is great for situations where you're trying to change the browser URL and title when the user navigates to a new page via AJAX. You can send the new page's partial view back to the browser along with some data for the page title. Then your client-side script can put the partial view where it belongs on the page, update the page title bar, and update the URL if needed as well.
If you are wanting to send a fully complete HTML document to the browser along with some initial JavaScript data then you need to compile that JavaScript code into the view itself. It's definitely possible to do that but I've never found a way that doesn't involve some string magic.
For example:
// controller.js
var someData = { message: 'hi' };
res.render('someView', { data: JSON.stringify(someData) });
// someView.jade
script.
var someData = !{data};
Note: !{data} is used instead of #{data} because jade escapes HTML by default which would turn all the quotation marks into " placeholders.
It looks REALLY strange at first but it works. Basically you're taking a JS object on the server, turning it into a string, rendering that string into the compiled view and then sending it to the browser. When the document finally reaches the browser it should look like this:
// someSite.com/someView
<script type="text/javascript">
var someData = { "message": "hi" };
</script>
Hopefully that makes sense. If I was to re-create my original helper method to ease the pain of this second scenario then it would look something like this:
app.use(function (req, res, next) {
res.renderWithData = function (view, model, data) {
model.data = JSON.stringify(data);
res.render(view, model);
};
next();
});
All this one does is take your custom data object, stringifies it for you, adds it to the model for the view, then renders the view as normal. Now you can call res.renderWithData('someView', {}, { message: 'hi' });; you just have to make sure somewhere in your view you grab that data string and render it into a variable assignment statement.
html
head
title Some Page
script.
var data = !{data};
Not gonna lie, this whole thing feels kind of gross but if it saves you an extra trip to the server and that's what you're after then that's how you'll need to do it. Maybe someone can think of something a little more clever but I just don't see how else you'll get data to already be present in a full HTML document that is being rendered for the first time.
Edit2:
Here is a working example: https://c9.io/chevex/test
You need to have a (free) Cloud9 account in order to run the project. Sign in, open app.js, and click the green run button at the top.
My approach is to send a cookie with the information, and then use it from the client.
server.js
const visitCard = {
name: 'John Smit',
phone: '+78503569987'
};
router.get('/route', (req, res) => {
res.cookie('data', JSON.stringify(pollsObj));
res.render('index');
});
client.js
const getCookie = (name) => {
const value = "; " + document.cookie;
const parts = value.split("; " + name + "=");
if (parts.length === 2) return parts.pop().split(";").shift();
};
const deleteCookie = (name) => {
document.cookie = name + '=; max-age=0;';
};
const parseObjectFromCookie = (cookie) => {
const decodedCookie = decodeURIComponent(cookie);
return JSON.parse(decodedCookie);
};
window.onload = () => {
let dataCookie = getCookie('data');
deleteCookie('data');
if (dataCookie) {
const data = parseObjectFromCookie(dataCookie);
// work with data. `data` is equal to `visitCard` from the server
} else {
// handle data not found
}
Walkthrough
From the server, you send the cookie before rendering the page, so the cookie is available when the page is loaded.
Then, from the client, you get the cookie with the solution I found here and delete it. The content of the cookie is stored in our constant. If the cookie exists, you parse it as an object and use it. Note that inside the parseObjectFromCookie you first have to decode the content, and then parse the JSON to an object.
Notes:
If you're getting the data asynchronously, be careful to send the cookie before rendering. Otherwise, you will get an error because the res.render() ends the response. If the data fetching takes too long, you may use another solution that doesn't hold the rendering that long. An alternative could be to open a socket from the client and send the information that you were holding in the server. See here for that approach.
Probably data is not the best name for a cookie, as you could overwrite something. Use something more meaningful to your purpose.
I didn't find this solution anywhere else. I don't know if using cookies is not recommended for some reason I'm not aware of. I just thought it could work and checked it did, but I haven't used this in production.
Use res.send instead of res.render. It accepts raw data in any form: a string, an array, a plain old object, etc. If it's an object or array of objects, it will serialize it to JSON for you.
var visitCard = {
name: 'John Smit',
phone: '+78503569987'
};
exports.index = function(req, res, next){
res.send(visitCard};
};
Check out Steamer, a tiny module made for this this exact purpose.
https://github.com/rotundasoftware/steamer
Most elegant and simple way of doing this is by using rendering engine (at least for that page of concern). For example use ejs engine
node install ejs -s
On server.js:
let ejs = require('ejs');
app.set('view engine', 'ejs');
then rename desired index.html page into index.ejs and move it to the /views directory. After that you may make API endpoit for that page (by using mysql module):
app.get('/index/:id', function(req, res) {
db.query("SELECT * FROM products WHERE id = ?", [req.params.id], (error, results) => {
if (error) throw error;
res.render('index', { title: results[0] });
});
});
On the front-end you will need to make a GET request, for example with Axios or directly by clicking a link in template index.ejs page that is sending request:
<a v-bind:href="'/index/' + co.id">Click</a>
where co.id is Vue data parameter value 'co' that you want to send along with request
I'm working on a NoDB CMS in Meteor, but I'm new to both Meteor and JavaScript frameworks.
How do I go about reading and writing files to the server?
Within the Node fs module you have a writeFile function.
getUser = Meteor.users.findOne({_id : Meteor.userId()});
userObject = JSON.stringify(getUser);
var path = process.env["PWD"] + "/public/";
fs.writeFile(process.env["PWD"] + "/public/"+Meteor.userId()+'.txt', userObject,
function (err) {
if (err) throw err;
console.log('Done!');
}
);
The above snippet would create a file with all the information of the user. You could access the properties of the result of your query with something like getUser._id to prepare your data parameter (String or Buffer) to print pretty.
All this of course is server side.
you can try to use Npm.require inside the startup function. Like so
Meteor.startup(function () {
fs = Npm.require('fs');
}
But you should definitely have a look at collectionFS that does what you are looking for: storing files on the server and allowing you to retrieve them
an added advantage is that you can distribute everything over many nodes of a MongoDB cluster
to manipulate image files, you can use imagemagick with nodejs this should allow you to transform in any way you need.
The node fs module is a start. http://nodejs.org/api/fs.html
You might want to be a bit more specific with your question though, as it's kind of broad.
I have been searching for this particular problem for the past week, and since I couldn't find any information on the subject(that wasnt outdated), I just decided to work on other things. But now I am at the point where I need to be able to send data(that I constructed) to specific clients using their ID who are connected to my server using node.js and socket.io. I already store the ID in an object for each new connection. What I need to know is a way to just send it to a connection ID I choose.
Something like: function send(data, client.id) {};
I am using an http server, not TCP.
Is this possible?
edit:
server = http_socket.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(respcont);
client_ip_address = req.header('x-forwarded-for');
});
socket = io.listen(1337); // listen
//=============================================
// Socket event loop
//=============================================
socket.on ('connection', function (client_connect) {
var client_info = new client_connection_info(); // this just holds information
client_info.addNode(client_connect.id, client_connect.remoteAddress, 1); // 1 = trying to connet
var a = client_info.getNode(client_connect.id,null,null).socket_id; // an object holding the information. this function just gets the socket_id
client_connect.socket(a).emit('message', 'hi');
client_connect.on('message', function (data) {
});
client_connect.on ('disconnect', function () {
);
});
solution: I figured it out by just experimenting... What you have todo is make sure you store the SOCKET, not the socket.id (like i was doing) and use that instead.
client_info.addNode(client_connect.id, client_connect.remoteAddress, client_connect, 1)
var a = client_info.getNode(client_connect.id,null,null,null).socket;
a.emit('message', 'hi');
If you need to do this, the easiest thing to do is to build and maintain a global associative array that maps ids to connections: you can then look up the appropriate connection whenever you need and just use the regular send functions. You'll need some logic to remove connections from the array, but that shouldn't be too painful.
Yes, it is possible.
io.sockets.socket(id).emit('message', 'data');
Your solution has one major drawback: scaling. What will you do when your app needs more the one machine? Using internal IDs also could be difficult. I advice using external IDs (like usernames).
Similarly to this post I advice using rooms (together with Redis to allow scaling). Add private room for every new socket (basing on user's name for example). The code may look like this:
socket.join('priv/' + name);
io.sockets.in('priv/' + name).emit('message', { msg: 'hello world!' });
This solution allows multiple machines to emit events to any user. Also it is quite simple and elegant in my opinion.