Currently our project is handling logging through static methods in a class. We use a transaction id to track the logs of a request and don't confuse with the logs of another.
But this has one concurrency problem.
request1 set up the transaction id (static)
request2 set up the transaction id (another one)
request1 logs, then it uses the id of request2.
Our architect propose to instantiate a logger and pass it as parameter in all functions, but that is quite annoying since there are a lot of functions calls, so that means adding a new parameter, not meaningful from a functional viewpoint, to hundreds of functions.
Any other solution I can think needs a way to, in any function, differentiate which context it belongs to, I mean, which request called the function.
I am very novice in nodejs, is there anykind of global variable or other mechanism that could me help here?
There is a blog sharing how this can be achieved.
I took a look on the library it used, it's referring to cls-hooked which uses some internal/experimental Node.js features.
Hence, it's not quite recommended to use in production.
Edited
Recently, I discover a library when learning https://zipkin.io. It uses https://github.com/othiym23/node-continuation-local-storage to keep track of value like thread-local storage. Worth to take a look
Related
TL;DR: Is it bad practice to use a global variable to prevent code executing unnecessarily, and if so what are the alternatives?
I have an Electron application that reads a stream of real time data from another application and outputs some elements of it on screen.
There are two types of data received, real time (telemetry) data of what is currently going on and more static data that updated every few seconds (sessionInfo).
The first time a sessionInfo packet is received I need to position and size some of the UI elements according to some of the data in it. This data will definitely not change during the course of the application being used, so I do not want the calculations based on it to be executed more than once*.
I need to listen for all sessionInfo packets, there are other things I do with them when received, it is just this specific part of the data that only needs to be considered once.
Given the above, is this a scenario where it would be appropriate to use a global variable to store this information (or even just a flag to say that this info had been processed) and use this to prevent the code executing multiple times? All my reading suggests that Global Variables are never a good idea, but short of allowing this code to execute repeatedly I am unsure what alternatives I have here.
*I recognise that allowing this would probably make no practical difference to my application, but this is a learning experience for me as well as producing something useful so I would like to understand the 'right' approach rather than just bodge something inefficient together and make it work.
Global variables are generally a bad practice for many reasons like:
They pollute the global scope, so if you create a global i, you
can't use that name anywhere else.
They make it impossible or very difficult to unit test your code.
You usually end up needing more of them than you think, to the point where many things become global.
A better practice is to create singleton services. For example, you might create a SessionService class that handles all your auth stuff. Or, perhaps a DataStreamService that holds the state variables of your data stream. To access that singleton, you'd import it into whatever file needs it. Some frameworks go even farther and have a single global data store, like react/redux.
There is a similar thread that is currently two years old, and it pretty much sums up what I want.
To give you an example, for the C# developers there is a single Session object which stores user's session and is accessible anywhere, in each class, .dll, you name it. It's stored per-request.
I need basically the same thing. My users have simple permissions, they are either admins or have roughly 5 assigned elements. I'm storing these 5 elements in an access token, so basically in the first middleware I have an array of ids which are user's permissions.
What I want is not to pass a permissions object around, especially not to pass them to service/DAL layers which are responsible for querying the database since it's pretty ridiculous for getAllCities function to receive permissions object and a callback as parameters. Or whatever object for that matter.
I cannot store the values in global variables since the globals are, well, global. I don't want to pass the variables to each function, so I would probably fall back to rounding the databases for user's permissions per each call, but that's just.. Meh.
I also don't want to mess to much with dozens of libraries etc., because I believe this is a common problem that people stumble upon and I wonder what are the best solutions.
On the thread linked above there is a robust solution which I would frankly want to avoid. Everything I've used in Node has been plug-and-play, and implementing custom middlewares and pipelines for this "small" thing seems like an overkill.
Two years later, is there a better solution out there to this problem?
You would typically use something like express-session to manage a session for each user and then each incoming request would automatically have access to that session object for that user in the request handler. You can then store any user-specific information in that session object.
Any functions that you want to execute in a given request should either be passed the request object (from which they can get access to the session) or just pass them the session object or some user-specific object you pulled out of the session. In node.js the place for user-specific information is in the request object. There is no "global" space to store it that all other functions can access it.
If you want a function to be able to access request or session specific data, then pass that data to the function. That's how it works. If this appears to be a problem to you, then you have to rethink how your code is architected and perhaps use objects/methods that can maintain state more easily.
node.js does not have a thread per request that can have it's own thread state. Instead, in an http server, you use the request object as the core state for a given request and you pass it to functions that need it. Or, if you're using a user session, you can extract that from the request object and then pass the session around as needed.
I don't want to pass the variables to each function, so I would
probably fall back to rounding the databases for user's permissions
per each call, but that's just.. Meh.
It sounds to me like too much of your code is just plain functions rather than methods on an object. If you think of your code as a user object that has methods on it, then you can form a user object in the initial request and then call methods on that user object for most things you would want to do that are user-specific. The user object then is a nice simple repository for user-specific information (such as permissions) and ALL methods on the object will have access to it without having to pass it to every function.
I repeat, if passing the user state around seems like a burden, then it's probably because your code is not architected in a way that makes it easy. Rather than looking for a work-around for something that is not the way node.js works, you should think about architecting your code to make these kinds of things easy so your architecture more closely matches the needs of your code given the way the underlying system works.
implementing custom middlewares and pipelines for this "small" thing
seems like an overkill.
Session middleware is not a small thing. It's core to any application that maintains user-specific data across requests and wants easy access to it on each new request from that same user. It's generally how you manage user state in node.js.
To give you an example, for the C# developers there is a single
Session object which stores user's session and is accessible anywhere,
in each class, .dll, you name it. It's stored per-request.
That simply isn't how node.js works at the lowest architectural level. And, the sooner you realize that and start designing your code to align with the way node.js actually works (rather than the way some other system works), the sooner you will find node.js coding flows naturally. It's something new to learn, not to be avoided or worked-around.
Let's imagine, in a OO world, I want to build a Torrent object which listens to the network and lets me interact with it. It would inherit an EventEmitter and would look something like this:
var torrent = new Torrent(opts)
torrent.on('ready', cb) // add torrent to the UI
torrent.on('metadata', cb) // update data in the UI
and I can also make it do things:
torrent.stop()
torrent.resume()
Then of course if I want to do delete the torrent from memory I can call torrent.destroy().
The cool thing about this OO approach is that I can easily package this functionality in its own npm module, test the hell out of, and give users a nice clean reusable API.
My question is, how do I achieve this with Cycle.js apps?
If I create a driver it's unclear how I would go about creating many torrents and having their own independent listeners. Also consider I'd like to package functionality in a way that others get to easily reuse it in other Cycle.js apps.
It seems to me that you are trying to solve a problem thinking about it as you would write "imperative code".
I think creating Torrent instances with their own listeners is not something you should be using in cycle components.
I would go about it differently - creating Torrent module and figuring out what would be its sources and sinks. If this module should be reusable and published, you can create it as a function that would receive streams as arguments. Maybe something similar to TodoMVC Task component (which is then used in its parent component).
Since this module can be created as a pure function, testing it should be at least just as easy.
This implementation of course depends on your requirements but communication with the module would then be done only with streams and since it would be declarative there would be no need for methods like stop() and destroyed() which you would call from elsewhere.
How do I test it?
In cycle.js you'd write a component with intent model and view functions.
You'd test intent(), for given input Streams, produces Streams of actions that you want. For models, you'd test that given http and action streams, you get the state you want, and for view, you test that given a state you get the VDom you want.
One tricky bit with cycle.js is that since it passes functions around, normal JavaScript objects that use the 'this' keyword are not worth the trouble due to 'this' context problems. If you are working with cycle.js and you think you might write a JS class for use with Isolate, Onionify, or Collections most likely, you are going in the wrong direction. See MDN docs about 'this'
how I would go about creating many torrents
The Cycle.js people have several ways to deal with groups of things like this.
This ticket describes some things that might work for that:
Wrap subapp in Web Component
Stanga and similars.
Cycle Collections
Cycle Onionify
I have a basic express server that needs to store some global variables during each request handling.
More in depth, request handling involves many operation that need to be stored in a variable such as global.transaction[]
Of course if I use the global scope, every connection will share information of its transaction and I need a global scope because I need to access the transaction array from many other modules, during my execution.
Any suggestion on this problem? I feel like is something very trivial but I'm looking for complicated solutions :)
Many thanks!
UPDATE
This is a case scenario, to be more clear.
On every request I have 3 modules (ModuleA, ModuleB, ModuleC) which read the content of 10 random files in one directory. I want to keep track of the list of file names read by every request, and send back with res.write the list.
So ModuleA/B/C need to access a sort of global variable but the lists of request_1, request_2, request_3 etc... don't have to mix up.
Here is my suggestion avoid global state like fire.
It's the number one maintenance problem in Node servers from my experience.
It makes your code not composable and harder to reuse.
It creates implicit dependencies in your code - you're never sure which piece depends on which and it's not easy to verify.
You want the parts of code that each piece of an application uses to be as explicit as possible. It's a huge issue.
The issue
We want to synchronize state across multiple requests and act accordingly. This is a very big problem in writing software - some say even the biggest. The importance of the way objects in the application communicate can not be overestimated.
Some solutions
There are several ways to accomplish sharing state across requests or server wide in a Node server. It depends on what you want to do. Here are the two most common imo.
I want to observe what the requests do.
I want one request to do things based on what another request did.
1. I want to observe what the requests do
Again, there are many ways to do this. Here are the two I see most.
Using an event emitter
This way requests emit events. The application reads events the requests fire and learns about them accordingly. The application itself could be an event emitter you can observe from the outside.
You can do something like:
request.emit("Client did something silly", theSillyThing);
And then listen to it from the outside if you choose to.
Using an observer pattern
This is like an event emitter but reversed. You keep a list of dependencies on the request and call a handler method on them yourself when something interesting happens on the request.
Personally, I usually prefer an event emitter because I think they usually solve the case better.
2. I want one request to do things based on what another request did.
This is a lot tricker than just listening. again, there are several approaches here. What they have in common is that we put the sharing in a service
Instead of having global state - each request gets access to a service - for example when you read a file you notify the service and when you want a list of read files - you ask the service. Everything is explicit in the dependency.
The service is not global, only dependencies of it. For example, it can coordinate resources and the data, being some form of Repository).
Nice theory! Now what about my use case?
Here are two options for what I would do in your case. It's far from the only solution.
First option:
Each of the modules are an event emitter, whenever they read a file they emit an event.
A service listens to all their events and keeps count.
Requests have access to that service explicitly and can query it for a list of files.
Requests perform writes through the modules themselves and not the added service.
Second option:
Create a service that owns a copy of module1, module2 and module3. (composition)
The service delegates actions to the modules based on what is required from it.
The service keeps the list of files accessed since the requests were made through it.
The request stops using the modules directly - uses the service instead.
Both these approaches have advantages and disadvantages. A more complicated solution might be required (those two are in practice pretty simple to do) where the services are abstracted further but I think this is a good start.
One simple way is storing data on the request object.
Here is an example (using Express):
app.get('/hello.txt', function(req, res){
req.transaction = req.transaction || [];
if (req.transaction.length) {
// something else has already written to this array
}
});
However, I don't really see how you can need this. When you call moduleA or moduleB, you just have to pass an object as argument, and it solves your issue. Maybe you're looking for dependency injection?
using koa ctx.state doc for this scenario, in express I believe this Plugin should serve your needs.
in order to keep some data that will be resused by another request on the save server app, I propose to use session in expresse and avoid any global state or any props drilling from one request to another.
In order to manage session state in express you could use :
session-file-store save the session in a file
express-mongodb-session : save the session in mongoDb
mssql-session-store -> for a relation db
Of course there is another technique ti manage session in NodeJs.
I have a web application based on apache. php, js and jquery. All works fine.
On the client side there is a small library in JS/jquery, offering some generic list handling methods. In the past I used callbacks to handle those few issues where those methods had to behave slightly different. That way I can reuse methods like list handling, dialog handling and stuff for different part of the application. However recently the number of callbacks I had to hand through when stepping into the library grew and I am trying a redesign:
Instead of specifying all callbacks as function arguments I created a central catalog object in the library. Each module of the application registers its own variant of callbacks into that catalog upon initialization. At runtime the methods lookup the required callbacks in that catalog instead of expecting it specified in their list of arguments. This cleans up things quite a lot.
However I have one thing I still cannot get rid of: I require a single argument (I call it context, mode might be another term) that is used by the methods to lookup the required callback in the catalog. This context has to be handed through to all methods. Certainly better than all sorts of different callbacks being specified everywhere, but I wonder if I can get rid of that last one to.
But where do I specify that context, if not as method argument ? I am pretty new to JS and jquery, so I failed to find an approach for this. Apparently I don't want to use global vars, and to be frank I doubt that I can simply store a context in a single variable anyway, since because of all the event handlers and external influences methods might be called in different contexts at the same time, or at least interleaving. So I guess I need something closer to a function stack. Maybe I can simply push a context object to the stack and read that from within the layers of the library that need to know ? The object would be removed when I leave the library again. Certainly other approaches exist too.
Here are so many experienced coders how certainly can give a newbie like a short hint, a starting point that leads to an idea, how to implement this. How is such thing 'usually' done ?
I tried round a while, exploring the arguments.callee.caller hierarchy. I thought maybe I could set a prototype member inside a calling function, then, when execution steps further down I could simply traverse the call stack upwards until I find a caller holding such property and use that value as context.
However I also saw the ongoing discussions that reveal two things: 1.) arguments.callee appears to be depreciated and 2.) it appears to be really expensive. So that is a no go.
I also read about the Function.caller alternative (which appears not to be depreciated and much more efficient, however until now I failed to explore that trail...
As written currently passing the context/mode down simply works by specifying an additional argument in the function calls. It carries a unique string that is used as a key when consulting the catalog. So something like this (not copied, but written as primitive example):
<!-- callbacks -->
callback_inner_task_base:function(arg1,arg2){
// do something with args
}
callback_inner_task_spec:function(arg1,arg2){
// do something with args
}
<!-- catalog -->
Catalog.Callback:function(context,slot){
// some plausibility checks...
return Catalog[context][slot];
}
Catalog.base.slot=callback_inner_task_base;
Catalog.spec.slot=callback_inner_task_spec;
<!-- callee -->
do_something:function(arg1,arg2,context){
...
// callback as taken from the catalog
Catalog.Callback(callback,'inner_task')(arg1,arg2);
...
}
<!-- caller -->
init:function(...){
...
do_something('thing-1',thing-2','base');
do_something('thing-1',thing-2','spec');
...
}
But where do I specify that context, if not as method argument ?
Use a function property, such as Catalog.Callback.context
Use a monad