redis php and javascript connection - javascript

I am making a web chat client where I use redis for pub/sub. I am having trouble with the subscribe part. I am able to publish but I am not sure how to subscribe. I have a php script written to subscribe (it work when I run php) it listens and echos the message. I want to be able to get that message in javascript. How do I call the php file and listen? I tried ajax in jquery and listening for the echo in the success function but it does not seem to work. I am new to this any advice is helpful
EDIT: Here is the javascript
$.ajax({
url:"http://localhost/redisphp.php",
type: GET,
success: function(response){ ...},
...
Here is the redis. I modeled after this link https://xmeng.wordpress.com/2011/11/14/pubsub-in-redis-using-php/
<?php
function f($redis, $chan, $msg) {
switch($chan) {
case 'chan-1':
echo $msg;
}
}
ini_set('default_socket_timeout', -1);
$redis = new Redis();
$redis->pconnect('128.0.0.0',6378);
$redis->subscribe(array('chan-1'), 'f');
print "\n";
?>

I have a php script written to subscribe (it work when I run php) it listens and echos the message. I want to be able to get that message in javascript.
It sounds like you would need to create an Express API that would connect to your Redis server.
Please keep in mind I am going on very limited information that you have provided. So first thing with your Express API is to create your package.json file like so:
{
"dependencies": {
"express": "4.16.3",
"redis": "2.8.0",
"nodemon": "1.18.3"
},
"scripts": {
"dev": "nodemon",
"start": "node index.js"
}
}
You don't want it to look exactly like this of course, but just pointing you in the right of what you want to be doing. Of course you will need to do an npm install for those particular dependencies.
Then if I were you I would create a server/keys.js file like so:
module.exports = {
redisHost: process.env.REDIS_HOST,
redisPort: process.env.REDIS_PORT,
};
And then require that inside a server/index.js file where you will also add the following:
const keys = require("./keys");
// Express App Setup
const express = require("express");
const app = express();
// Redis Client Setup
const redis = require(‘redis’);
const redisClient = redis.createClient({
host: keys.redisHost,
port: keys.redisPort,
retry_strategy: () => 1000
});
const redisPublisher = redisClient.duplicate();
So this retry_strategy takes an arrow function and it is saying that if we ever lose connection to the redis server try to reconnect to it once every second.
The key for retry_strategy is separated by an underscore as opposed to the JavaScript standard of camelCase.
According to the Redis documentation for the Javascript library, if we ever have a client that is listening or publishing information on Redis we have to make a duplicate connection between when a connection is turned into one that is going to listen, subscribe or publish information, it cannot be used for other purposes.
So thats why I am doing this duplicate() thing at the end of redisClient.
So that is pretty much it.

Related

SCRAM-SERVER-FIRST-MESSAGE: client password must be a string

Ive read documentation from several pages on SO of this issue, but i havent been able to fix my issue with this particular error.
throw new Error('SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string')
^
Error: SASL: SCRAM-SERVER-FIRST-MESSAGE: client password must be a string
at Object.continueSession (C:\Users\CNFis\Desktop\WulfDevelopments\ThePantry\node_modules\pg\lib\sasl.js:24:11)
at Client._handleAuthSASLContinue (C:\Users\CNFis\Desktop\WulfDevelopments\ThePantry\node_modules\pg\lib\client.js:257:10)
at Connection.emit (events.js:400:28)
at C:\Users\CNFis\Desktop\WulfDevelopments\ThePantry\node_modules\pg\lib\connection.js:114:12
at Parser.parse (C:\Users\CNFis\Desktop\WulfDevelopments\ThePantry\node_modules\pg-protocol\dist\parser.js:40:17)
at Socket.<anonymous> (C:\Users\CNFis\Desktop\WulfDevelopments\ThePantry\node_modules\pg-protocol\dist\index.js:11:42)
at Socket.emit (events.js:400:28)
at addChunk (internal/streams/readable.js:290:12)
at readableAddChunk (internal/streams/readable.js:265:9)
at Socket.Readable.push (internal/streams/readable.js:204:10)
its as if in my connectDB() function its not recognizing the password to the database. I am trying to run a seeder.js script to seed the database with useful information for testing purposes, and if i run npm run server which is a script that just starts a nodemon server, itll connect to the DB just fine. but when i try to run my script to seed data, i am returning this error.
import { Sequelize } from "sequelize";
import colors from "colors";
import dotenv from "dotenv";
dotenv.config();
const user = "postgres";
const host = "localhost";
const database = "thePantry";
const port = "5432";
const connectDB = async () => {
const sequelize = new Sequelize(database, user, process.env.DBPASS, {
host,
port,
dialect: "postgres",
logging: false,
});
try {
await sequelize.authenticate();
console.log("Connection has been established successfully.".bgGreen.black);
} catch (error) {
console.error("Unable to connect to the database:".bgRed.black, error);
}
};
export default connectDB;
above is my connectDB() file, and again, it works when i run the server normally. but i receive this error only when trying to seed the database. Ill post my seeder script below:
import dotenv from "dotenv";
import colors from "colors";
import users from "./data/users.js";
import User from "./models/userModel.js";
import connectDB from "./config/db.js";
dotenv.config();
console.log(process.env.DBPASS);
connectDB();
const importData = async () => {
try {
await User.drop();
await User.sync();
await User.bulkCreate(users);
console.log("Data Imported".green.inverse);
process.exit();
} catch (e) {
console.error(`${e}`.red.inverse);
process.exit(1);
}
};
const destroyData = async () => {
try {
await User.bulkDestroy();
console.log("Data Destroyed".red.inverse);
process.exit();
} catch (e) {
console.error(`${e}`.red.inverse);
process.exit(1);
}
};
if (process.argv[2] === "-d") {
destroyData();
} else {
importData();
}
Add your .env file in your project, I think your .env file is missing in your project folder.
add like this:
So, i may have figured this out by playing around in another project with sequelize, as it turns out, the initial connection to the database in my server.js file, honestly means nothing. Unlike Mongoose where the connection is available across the whole app. its not the same for Sequelize this connection that it creates is only apparent in certain places, for example i was trying the same process in my other project as i am here, except i was trying to read data from my DB using the model that i built with sequelize and i was receiving the same type error, i went into where i defined the model and made a sequelize connection there, and i was then able to read from the database using that object model.
Long story short, to fix the error in this app i have to place a connection to the database in the seeder.js file or i have to place a connection in the User model (this is ideal since ill be using the model in various places) to be able to seed information or read information from the database.
today i have same problem like this, so if you use database with type relational. you must define password from database.
const user = "postgres";
const host = "localhost";
const database = "thePantry";
const password = "yourdatabasepassword"; if null > const password = "";
const port = "5432";
but, if you use database with type non-relational, as long as the attributes are the same, you can immediately run the program as you defined it
I also faced this issue and another solution different from the accepted solution here solved my issue, so I wanted to explain that to this lovely community, too.
Firstly, when I faced the issue, ran my project in debug mode and reached the code below.
let sequelize;
if (config.use_env_variable) {
sequelize = new Sequelize(process.env[config.use_env_variable], config);
} else {
sequelize = new Sequelize(config.database, config.username, config.password, config);
}
The problem here is actually obvious when I saw first, there is a problem in .env file as mentioned in the solutions above. In my process.env is defined as like as following line: DATABASE_URL=postgres://username:password#IP_adress:port/db_name and my config.js file is in the following format:
module.exports = {
"development": {
"url":"postgres://username:password#IP_adress:port/db_name",
"dialect": "postgres",
}, ...
}
So as a solution, I come with the following fix for the parameters that are inside Sequelize(...). My solution below is totally worked for me and I hope it also works for you too.
let sequelize;
if (config.use_env_variable) {
sequelize = new Sequelize(process.env[config.use_env_variable], config);
} else {
sequelize = new Sequelize(config.url, config);
}
Finally, the point you need to be careful about what you have written to the config file. That's the most important in this case.
Farewell y'all.
Here is my case. I have postgresql connection url in my enviroment like:
POSTGRES=postgres://postgres:test#localhost:5432/default
But my config getting like:
POSTGRES_DB_HOST=localhost
POSTGRES_DB_PORT=5432
...rest of configs
Now it has resolved.
I faced this issue because I was trying to execute nodemon from a parent folder. Once I changed my pwd, the error was resolved.
For your seeder script, i'm doing something similar but not using Sequilize, just the node-postgres package in an ExpressJS app.
To give context (so you know if this applies to your situation)
I run a separate script for testing, which uses database credentials to test batched emailing. So, I need to access my database (eventually will migrate it to an AWS lambda function).
I need to access my database and run sequential actions, since I'm not spinning up my server, all that 'under the hood' processes that would normally start your connection pool is probably not running. My guess ( I know it's an old post but this may help others).
Try passing your hardcoded password credentials. first on your seeder.js file. (i'm sure you've tried this already).
Try creating a new Pool within your seeder script and pass it your credentials (try hard coding it first to see if it works).
Pool in postgres takes a client config with the following properties (i use this to get mine to work).
const pool = new Pool({
user: '****',
database: '****',
password: '****',
port: 5432,
host: '****',
max: 5,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 5000,
})
I imagine sequilize will have a similar configuration, so try playing around with that.
Then I just connect to the pool and do everything I'd normally do.
Hope this helps with a bit of the troubleshooting. I had the EXACT same error message earlier. Ultimately I had to restructure my code to 'boot up' the Client/Connection Pool for the database. It sounds like you're not properly 'booting up' your connection so try doing it manually within your seeder script (don't pass process.env.DB_PASSWORD at first).
I saw this error when running a npx sequelize-cli db:... command
and my postgres server wasn't running or able to accept connections.
To fix it, I had to be running: postgres -D /usr/local/var/postgres in the background.

how to get a request in node repl

I've started to build a typescript library (intended to be used on the server side) and right now I'm trying to use the node repl to play around with my code and see what happens in certain situations... I've built and required the file, but now I'm having a problem: I have a function that takes a http Request (type Request from express.js), and I'd like to try and run it in the repl providing it with a copy of a request that I previously made from my browser. Is this feasible?
I thought maybe I could do it by either:
doing regex magic on the request exported as cURL or
sending the request to node, but then how am I going to receive it while in the repl?
I'm not sure I understand your use-case, but you can try something like this:
In some temp folder type:
npm install "request-promise"
Then from the same temp folder, enter the REPL and type:
(async () => {const response = await require("request-promise").get("https://cnn.com"); console.log(response)})()
This example is for get, but it can be easily changed to other HTTP methods.
I've found a fairly simple way to do what I want... It involved quickly setting up a basic express server (set up following this tutorial):
mkdir scratch && cd scratch && npm init
(select defaults except entrypoint app.js)
npm i express
Create an app.js (vi app.js) with the following contents:
var express = require('express');
var app = express();
var circ = {};
circ.circ = circ;
var cache = [];
app.get('/', function (req, res) {
res.send(JSON.stringify(req, (key, value) => {
if (typeof value === 'object' && value !== null) {
// Duplicate reference found, discard key
if (cache.includes(value)) return;
// Store value in our collection
cache.push(value);
}
return value;
}));
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
(See this answer for JSON.stringify custom replacer resp. second argument to JSON.stringify). You can optionally use flatted instead, which I discovered later and is surely better.
Now do the following:
Run the app with node app.js
In your browser, navigate to the website where your desired request is posted to.
Open your browsers development tools (Ctrl+shift+c works for me).
Go to the network tab.
Find the request that interests you and right click on it.
Click on copy > copy as curl (or similar, depending on which browser you're using).
Run that curl request, but change the url it's posted to to 127.0.0.1:3000 (e.g. change curl 'example.com' \...etc to curl '127.0.0.1:3000' \...etc
You should now get that request on standard output as a JSON object and it's in the format that express usually deals with. Yay! Now, pipe it into your clipboard (likely xclip -selection c on linux) or probably even better, redirect it to a file.
...
Step 2 - ?
Step 3 - Profit :)

Should this function be ran asynchronously on the server

im not that experienced with node js but im developing something similar to how uber displays their cars in real time on a map.
So i have an sql database with a ton of cars and their gps location. The client sends their gps coordinates and a radius to the following function. some is in pseudo code for now.
var mysql = require('mysql');
var express = require('express');
var connection = mysql.createConnection({
host: "",
port: ,
user: "",
password: "",
database: ""
});
user.on('returnCars', function(gps, radius){
connection.query({
sql: "SELECT * FROM cars WHERE radius = ?",
values: [username] },
function(error, results, fields)
{
if(results)
{
user.emit('returnCars', results);
}
}
});
});
});
So as sql querys arnt instant, if there was 1000 people running this function at once it would surely clog up. All my research is telling me that this is the right way to do it so the only option would for it to be ran asnync right?
Also would it just be the returnCars function to run asynchronously? Im not sure if because the connection/ sql details variable isnt in a function or anything it would all try and read it at once so maybe it should go inside the function or something.
The code is far too fragmentary to really help you with, but in general:
If you're using Node's built-in HTTP serving or something layered on top of it like Express, your code for when a request is received is expected to run asynchronously and there's nothing special you need to do to make that happen.
If you're using any of the main npm modules for MySQL access, the functions you use to make queries will run asynchronously and there's nothing special you have to do to make that happen.
In your pseudocode example, you've shown a callback for your sqlQuery function stand-in. That's likely how you would use the MySQL access module you choose to use, either with direct callbacks like that or promises.

Using NodeJS with Express 3.x and Jade templates is it possible to just re-render one item for a previously rendered list?

I have been trying to find any post that can explain if it is possible to re-render one 'new' item (append) to a jade template list.
Say that we have a list of log-entries and upon first request we render a fetched list from a MongoDB collection 'logs', using res.render and Jades each functionality.
Since we like to retrieve updates from the database we also have a MongoWatch attached to that collection that listens for changes. Upon update can we execute some code that appends to that first list in the Jade-template?
/* app.js */
/*
Display server log
*/
app.get ('/logs', function(req, res, next) {
// Using Monk to retrieve data from mongo
var collection = db.get('logs');
collection.find({}, function(e,docs){
// watch the collection
watcher.watch('application.logs', function(event){
// Code that update the logs list with the new single entry event.data?
});
// Request resources to render
res.render('logs', { logs: docs } );
});
});
<!-- logs.jade -->
extends layout
block content
div
each log in logs
div.entry
p.url= log.url
Maybe i should use the template engine in another fashion, i am quite new to Express, Jade and really appreciate all you guys that spends your time answering problems like these..
// Regards
Ok, so i have looked up the suggestion from Jonathan Lenowski, thanks by the way!, and i came up with a solution to my problem. Thought i'd follow up and perhaps help someone else along the way..
Basically i am now using as suggested socket.io
So first install the socket.io npm module by adding it to package.json and run npm install, i used 'latest' as version.
Next to use the 'socket.io.js' on the client-side you actually have to copy the file from the installed socket.io module to your javascript folder.
Path (seen from project root is): 'node_modules/socket.io/node_modules/socket.io-client/dist/'
Setup DB, Watcher, Webserver, Socket and controller on server-side
/*
SETUP DATABASE HANDLE
in app.js
*/
var mongo = require('mongodb');
var monk = require('monk');
var db = monk('localhost:'+app.get('port')+'/application');
/* SETUP DATABASE UPDATE WATCH */
var watcher = new MongoWatch({ format: 'pretty', host: 'localhost', port: app.get('port') });
/* START WEBSERVER AND SETUP WEBSOCKET */
var server = Https.createServer({key: certData.serviceKey, cert: certData.certificate}, app);
var io = require('socket.io').listen(server);
server.listen(app.get('port'), function(){
console.log('Express server listening on port ' + app.get('port'));
});
/*
Display server log - controller
*/
app.get ('/logs', function(req, res, next) {
// Using Monk to retrieve data from mongo
var collection = db.get('logs');
collection.find({}, function(e,docs){
// watch the collection logs in database application
watcher.watch('application.logs', function(event){
io.sockets.emit('logs', { log: event.data });
});
// Request resources to render
res.render('logs', { logs: docs } );
});
});
Include the socket.io javascript in layout
/*
Add client side script
in layout.jade
*/
script(type='text/javascript' src='/javascripts/socket.io.js')
Use the client
/*
SETUP DATABASE HANDLE
in logs.jade
*/
extends layout
block content
script.
var socket = io.connect('https://localhost:4431');
socket.on('logs', function (data) {
console.log(data.log);
// Here we use javascript to add a .log-entry to the list
// This minor detail i leave to the developers own choice of tools
});
div.row#logs
div.col-sm-12
div.header-log Some application
div.logs-section
each log in logs
div.log-entry.col-sm-12(data-hook=log.status)
p.method= log.method
p.url= log.url
p.status(style='color: #'+log.color+' !important')= log.status
p.response-time= log.time
p.content-length= log.length
p.datetime= log.date
Use the functionality, remember that this flow is triggered by actually adding a row in the database 'application' and the collection 'logs'.
I use ssl thus with regular http we create a 'http' server instead and connect from the client with a standard address prefix of http://...
Also as an additional note, in order to use MongoWatch it is required of you to setup the MongoDB with replication set. Which is a mirror database that can be used as a fallback (dual purpose).
Cheers! And once again thanks to Jonathan!

Socket.io sanitize incoming data (xss)

im using socket.io in expressjs 3. And i want to sanitize incoming messages with express-validator. I have this code:
var expressValidator = require('express-validator')
, sanitize = require('express-validator').sanitize;
socket.on('chat', function (data) {
io.sockets.in('test').emit('chat', {
user: sh.session.user,
message: data.message,
time: new Date()
});
});
how do i use sanitize(data.message).xss? Because this does not work.
In this case you want to use validator instead of express-validator. First install it thru npm:
npm install validator
Then use it pretty much the same way:
var sanitize = require('validator').sanitize;
// later on
message = sanitize(data.message).xss()
The reason for this is because express-validator is used for when you are dealing with an HTTP request that went thru expressjs. In the case of Websockets, you are not going thru expressjs, but rather just listening on the same port as it. So express-validator is not actually "present" in the context of your Websocket's data event.

Categories

Resources