I have a file called utils.js which has some constants and I'm building a socket app which uses Node for backend work and regular Javascript for frontend. The names of the socket events are identical in both frontend and backend so I want to use the constants both in backend and frontend. But the syntax for exporting/importing is different in ecmascript6 as opposed to Node so how do I pull this off?
This is the contents of utils.js:
const EVENTS = {
CONNECTION: 'connection',
MESSAGE: 'message'
};
Object.freeze(EVENTS);
module.exports = EVENTS;
You can put the objects in JSON file, like events.json, or even event.js is you want plain JS objects. Both the Node and JS have access to this. Then you require/import this file in utils.js and you can do the same on the front end. Front and back ends can handle these however they want.
Let me preface by saying that I have spent a considerable amount of time trying to figure out the solution to this problem but I have not discovered something that works. I am using node and want to share a variable between my app.js server file and a client side javascript file (demo.js).
I run node app.js to launch the server and demo.js runs in the client. I have tried using module.exports and export but when I try importing in the demo.js file or referring to the module.exports var I get errors. Maybe I'm approaching this is in the wrong way.
For example, I am trying to use the node wikipedia package to scrape data. I have the following in my app.js file:
var wikipedia = require('node-wikipedia');
wikipedia.page.data('Clifford_Brown', { content: true }, function(response) {
console.log(response);
export const response = response;
module.exports.data = response
});
In my demo.js file I have tried importing this response var and using the module.exports var but I have been unsuccessful.
Anyone have any solutions to this issue or different approaches I should take?
Browser javascript files run in the browser. node.js javascript files run on the server. You cannot directly export things from one to the other. They are on completely different computers in different locations.
It is very important for developers to understand the notion that server-side code runs on the server and client-side code runs on the browser. The two cannot directly call each other or reach the other's variables. Imagine your server is in a data center in Seattle and the browser is running on a computer in Venice.
See How to access session variables in the browser for your various choices described for a previous answer.
In a nutshell, you can have the server insert a javascript variable into the generated web page so that when the javascript runs in the web page on the browser, it can then access that variable in its own page. Or, you can create an Ajax call so the client can request data directly from the server. Or you can have the server put some data in a cookie which the Javascript in the browser can then access.
If the data is easily known by the server at the time the page is generated and you are using some sort of page template system, then it is very easy to just add a <script> tag to the generated page that defines one or more Javascript variables that contain the desired information. Then, the client-side Javascript can just refer to those variables to have access to the data.
To pass data in http there is a request message and response message and the data needs to be inside that message.
In the request you can either pass variables in the request URL
http://host_name/path?key=value
Or inside the request body or headers.
In the response you pass back variables in the response header or response body
First Example:
One way of processing a URL request from the browser explicitly while passing variables is to set up your server to render a html page with those variables embedded.
If you use a templating engine like jade, you can consume the sent variables directly into the template using res.render({ key: 'value' }) rather than using a promise based api call which would run when the user performs some action on the client.
For instance.
// SERVER setup rendering engine
app.get('/', function(req, res) {
res.render( 'index', { key: 'value' })
}
Which will render index.html to the client with the key-value pair passed to the template file used to serve up the index.html (for example jade or ejs).
Second Example:
Using axios you can set up an action to call a server api (you can also pass variables in the URL, headers or body). Using the promise pattern you can then use these variables after the server api has responded.
// CLIENT setup axios
axios.get(URL + '/getkeyvalue')
.then(function(response) {
const value = response.data.key
})
On you server using express you (this is where you would get the optional request variables mentioned above) send back response variables in the body like this.
// SERVER setup express
app.get('/getkeyvalue', function(req, res) {
res.send({ key: 'value' })
}
Note that these are simple examples.
They are too completely different systems. The best way to accomplish what you're trying to do is the create a variable in your html on the server side by stringifying your data
<script> var my_data = <%= JSON.stringify(data) %> </script>
Thats an example using ejs, a common templating language in expressjs
I'm trying to use serialport.js, even just including it at the moment causes a runtime error TypeError: undefined is not an object (evaluating 'stream._ansicursor').
var React = require('react')
var SerialPort = require('serial port')
The context of this is within a react app, so its being compiled from jsx. I have no idea what or how this error came about, looking in at the line given in the error its coming from this code thats coming from serial port.js.
/**
* Creates a Cursor instance based off the given `writable stream` instance.
*/
function ansi (stream, options) {
if (stream._ansicursor) {
return stream._ansicursor
} else {
return stream._ansicursor = new Cursor(stream, options)
}
}
module.exports = exports = ansi
The serialport package you are trying to use operates on nodejs streams, this means it will need to run on the server within the nodejs context.
Depending on what you're trying to do with the serial port package, if you want to continue to have a React based web UI for this task, you will need to separate out the serialport actions and write an API using a nodejs framework. I'd suggest something like ExpressJS.
This way you can send requests to API urls that will perform the serialport tasks on the server and return JSON feedback that your web application, written in React, can interact with.
I have web application backed end in NodeJS and logstash/elasticsearch/kibana to handle system logs like (access_error.log, messages.log etc).
Right now I need to record all JavaScript client side errors into kibana also. What is the best way to do this?
EDIT: I have to add additional information to this question. As #Jackie Xu provide partial solution to my problem and as follows from my comment:
I'm most interested in realizing server-side error handling. I think it's not effective write each error into file. I'm looking for best practices how to make it more performance.
I need to handle js error records on server-side more effective than just write into file. May you provide some scenarios how could I increase server-side logging performance?
When you say client, I'm assuming here that you mean a logging client and not a web client.
First, make it a habit to log your errors in a common format. Logstash likes consistency, so if you're putting text and JSON in the same output log, you will run into issues. Hint: log in JSON. It's awesome and incredibly flexible.
The overall process will go like this:
Error occurs in your app
Log the error to file, socket, or over a network
Tell logstash how to get (input) that error (i.e. from file, listen over network, etc)
Tell logstash to send (output) the error to Elasticsearch (which can be running on the same machine)
In your app, try using the bunyan logger for node. https://github.com/trentm/node-bunyan
node app index.js
var bunyan = require('bunyan');
var log = bunyan.createLogger({
name: 'myapp',
streams: [{
level: 'info',
stream: process.stdout // log INFO and above to stdout
}, {
level: 'error',
path: '/var/log/myapp-error.log' // log ERROR and above to a file
}]
});
// Log stuff like this
log.info({status: 'started'}, 'foo bar message');
// Also, in express you can catch all errors like this
app.use(function(err, req, res, next) {
log.error(err);
res.send(500, 'An error occurred');
});
Then you need to configure logstash to read those JSON log files and send to Elasticsearch/Kibana. Make a file called myapp.conf and try the following:
logstash config myapp.conf
# Input can read from many places, but here we're just reading the app error log
input {
file {
type => "my-app"
path => [ "/var/log/myapp/*.log" ]
codec => "json"
}
}
# Output can go many places, here we send to elasticsearch (pick one below)
output {
elasticsearch {
# Do this if elasticsearch is running somewhere else
host => "your.elasticsearch.hostname"
# Do this if elasticsearch is running on the same machine
host => "localhost"
# Do this if you want to run an embedded elastic search in logstash
embedded => true
}
}
Then start/restart logstash as such: bin/logstash agent -f myapp.conf web
Go to elasticsearch on http://your-elasticsearch-host:9292 to see the logs coming in.
If I understand correctly, the problem you have is not about sending your logs back to the server (or if it was #Jackie-xu provided some hints), but rather about how to send them to elastiscsearch the most efficiently.
Actually the vast majority of users of the classic stack Logstash/Elasticsearch/Kibana are used to having an application that logs into a file, then use Logstash's plugin for reading files to parse that file and send the result to ElasticSearch. Since #methai gave a good explanation about it I won't go any further this way.
But what I would like to bring on is that:
You are not forced to used Logstash.
Actually Logstash's main role is to collect the logs, parse them to identify their structure and recurrent field, and finally output them in a JSON format so that they can be sent to ElasticSearch. But since you are already manipulating javascript on the client side, one can easily imagine that you would talk directly to the Elasticsearch server.
For example once you have caught a javascript exception, you could do the folowing:
var xhr = new XMLHttpRequest();
xhr.open("PUT", http://your-elasticsearch-host:9292, true);
var data = {
lineNumber: lineNumber,
message: message,
url: url
}
xhr.send(JSON.stringify(data));
By doing this, you are directly talking from the client to the ElasticSearch Server. I can't imagine a simpler and faster way to do that (But note that this is just theory, I never tried myself, so reality could be more complex, especially if you want special fields like date timestamps to be generated ;)). In a production context you will probably have security issues, probably a proxy server between the client and the ES server, but the principle is there.
If you absolutely want to use Logstash you are not forced to use a file input
If, for the purpose of harmonizing, doing the same as everyone, or for using advanced logstash parsing confifuration you want to stick to Logstash, you should take a look at all the alternative inputs to the basic file input. For example I used to use a pipe myself, with a process in charge of collecting the logs and outputting these to the standard output. There is also the possibilty to read on an open tcp socket, and a lot more, you can even add your own.
You would have to catch all client side errors first (and send these to your server):
window.onerror = function (message, url, lineNumber) {
// Send error to server for storage
yourAjaxImplementation('http://domain.com/error-logger/', {
lineNumber: lineNumber,
message: message,
url: url
})
// Allow default error handling, set to true to disable
return false
}
Afterwards you can use NodeJS to write these error messages to a log. Logstash can collect these, and then you can use Kibana to visualise.
Note that according to Mozilla window.onerror doesn't appear to work for every error. You might want to switch to something like Sentry (if you don't want to pay, you can directly get the source from GitHub).
Logging errors trough the default built-in file logging allows your errors to be preserved and it also allows your kernel to optimize the writes for you.
If you really think that it is not fast enough (you get that many errors?) you could just put them into redis.
Logstash has a redis pub/sub input so you can store the errors in redis and logstash will pull them out and store them in your case in elasticsearch.
I'm presuming logstash/es are on another server, otherwise there really is no point doing this, es has to store the data on disc also, and it is not nearly as efficient as writing a logfile.
With whatever solution you go with, youll want to store the data, eg. writing it to disc. Append-only to a single (log) file is highly efficient, and when preserving data the only way you can handle more is to shard it across multiple discs/nodes.
I am new to Meteorjs and I am trying to retrieve data from an already existing MongoDB.
Heres what I have so far:
I set the env variable MONGO_URL to the mongoDB url
export MONGO_URL="mongodb://username:password#address:port/dbname"
Created a new meteor project with the following code:
MyCollection = new Meteor.Collection('mycollection');
if (Meteor.isClient) {
//Meteor.subscribe("mycollection");
console.log(MyCollection.findOne());
Template.hello.greeting = function () {
return MyCollection.findOne();
};
}
if (Meteor.isServer) {
Meteor.startup(function () {
// code to run on server at startup
console.log(MyCollection.findOne());
});
}
I know the server side console.log(MyCollection.findOne()); works as it prints out the correct data on the terminal.
The problem is with the client side. When I view the page on my browser, the data is blank and console.log(MyCollection.findOne()); shows 'undefined'.
I know that autopublish is on and I dont have to manually publish the collection from the server side.
I would like to know how I could make the client read from my external mongoDB directly.
Let me know if you have any suggestions!
Even with autopublish on, there is a lag between the client starting and the data being published. At the time that your first console.log is run, the documents haven't finished syncing so the findOne will return undefined. It turns out this isn't a big deal; as you get more familiar with meteor, you will see that the results of find operations are often used in non time-sensitive ways. An easy way to check if the client has the data is just to wait for the page to load, then start the browser console, and manually type:
console.log(MyCollection.findOne());
As for your other problem, the greeting needs to be something that can be displayed in html - a string for example. It can't be a document. Assuming your document had a message property you could do:
return MyCollection.findOne().message;