I've started to build a typescript library (intended to be used on the server side) and right now I'm trying to use the node repl to play around with my code and see what happens in certain situations... I've built and required the file, but now I'm having a problem: I have a function that takes a http Request (type Request from express.js), and I'd like to try and run it in the repl providing it with a copy of a request that I previously made from my browser. Is this feasible?
I thought maybe I could do it by either:
doing regex magic on the request exported as cURL or
sending the request to node, but then how am I going to receive it while in the repl?
I'm not sure I understand your use-case, but you can try something like this:
In some temp folder type:
npm install "request-promise"
Then from the same temp folder, enter the REPL and type:
(async () => {const response = await require("request-promise").get("https://cnn.com"); console.log(response)})()
This example is for get, but it can be easily changed to other HTTP methods.
I've found a fairly simple way to do what I want... It involved quickly setting up a basic express server (set up following this tutorial):
mkdir scratch && cd scratch && npm init
(select defaults except entrypoint app.js)
npm i express
Create an app.js (vi app.js) with the following contents:
var express = require('express');
var app = express();
var circ = {};
circ.circ = circ;
var cache = [];
app.get('/', function (req, res) {
res.send(JSON.stringify(req, (key, value) => {
if (typeof value === 'object' && value !== null) {
// Duplicate reference found, discard key
if (cache.includes(value)) return;
// Store value in our collection
cache.push(value);
}
return value;
}));
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
(See this answer for JSON.stringify custom replacer resp. second argument to JSON.stringify). You can optionally use flatted instead, which I discovered later and is surely better.
Now do the following:
Run the app with node app.js
In your browser, navigate to the website where your desired request is posted to.
Open your browsers development tools (Ctrl+shift+c works for me).
Go to the network tab.
Find the request that interests you and right click on it.
Click on copy > copy as curl (or similar, depending on which browser you're using).
Run that curl request, but change the url it's posted to to 127.0.0.1:3000 (e.g. change curl 'example.com' \...etc to curl '127.0.0.1:3000' \...etc
You should now get that request on standard output as a JSON object and it's in the format that express usually deals with. Yay! Now, pipe it into your clipboard (likely xclip -selection c on linux) or probably even better, redirect it to a file.
...
Step 2 - ?
Step 3 - Profit :)
Related
I'm new to javascript and trying to follow a Udemy tutorial and upload the code to github along the way. I need to hide an API key used in a URL that looks like the following:
https://api.darksky.net/forecast/api-key-here/37.8267,-122.4233
I created a .env file that contains a single line API_KEY=my-key-of-numbers-here
My entire code looks like this:
const request = require('request');
require('dotenv').config();
const api_key = process.env.API_KEY;
const url = 'https://api.darksky.net/forecast/${api_key}/37.8267,-122.4233';
request({ url: url }, (error, response) => {
const data = JSON.parse(response.body);
console.log(data.currently);
});
When I run node app.js in the terminal I get back undefined. However, if I use the actual key everything works fine, but I obviously can't make the key public. How can I fix this?
try ` instead ' – Estradiaz
Per the comment by #Estradiaz. I was using an apostrophe ' around the URL instead of backticks `. Solved.
Are you sure that you have dotenv installed?
try npm install dotenv in the terminal.
I want to download file using absolute FTP URL, like ftp://host:port/dir/file.extension
I've tried node-libcurl, wget, wget-improved, request. All failed saying that the protocol must be either HTTP or HTTPS.
There are FTP clients available for Node (available on npmjs). But, as per their documentation, they require creating a connection to FTP Server, change directory and then download it.
Is there any simple solution?
I will outline a simple approach here (and no complete solution with code!). FTP is based upon TCP with a simple human readable protocol. In order to fetch a file from an FTP server you need to do the following:
Create a TCP socket using net.Socket
Use socket.connect to connect to your FTP server on port 21
Communicate with the server using socket.write to send data and socket.on('data') to read data
An example of FTPs protocol for a simple file retrieval is provided in this blog post and can be summarized as follows:
Connect to server using net.Socket.connect
Set user with USER command
Authenticate with PASS
Go to desired directory using CWD
Change to passive mode using PASV
Read server reply to find out IP and port to connect to in order to fetch the file
Open another socket on IP and port of previous step
Voilà!
Anyone wanting to have simple single line solution that in addition to FTP would also work with FTPS and SFTP, you can try ftp-any-get
import { getFile } from "#tpisto/ftp-any-get"
async function main() {
// Fetch from FTP server
let ftpFile = await getFile("ftp://demo:password#my-ftp-server.net/my-file.txt");
// Fetch from FTP server using TLS
let ftpsFile = await getFile("ftps://demo:password#my-ftp-server.net/my-file.txt");
// Fetch file using SFTP. SFTP runs over the SSH protocol.
let sftpFile = await getFile("sftp://demo:password#my-ftp-server.net/my-file.txt");
}
main();
You can use node-libcurl, I don't know exactly how you did it, but here is some working code.
var Curl = require( 'node-libcurl' ).Curl,
Easy = require( 'node-libcurl' ).Easy,
path = require( 'path' ),
fs = require( 'fs' );
var handle = new Easy(),
url = 'ftp://speedtest.tele2.net/1MB.zip',
// Download file to the path given as first argument
// or to a file named 1MB.zip on current dir
fileOutPath = process.argv[2] || path.join( process.cwd(), '1MB.zip' ),
fileOut = fs.openSync( fileOutPath, 'w+' );
handle.setOpt( Curl.option.URL, url );
handle.setOpt( Curl.option.WRITEFUNCTION, function( buff, nmemb, size ) {
var written = 0;
if ( fileOut ) {
written = fs.writeSync( fileOut, buff, 0, nmemb * size );
}
return written;
});
handle.perform();
fs.closeSync( fileOut );
The repository currently has one example showing how to download a file using wildcard matching, I just changed the URL to point directly at the file, and removed the WILDCARDMATCH and CHUNK_*_FUNCTION options.
I was trying to find an online tool to generate a new link which randomly redirects to user defined array of links.
Please check out http://mathstatic.co.nz/auto to see what I am trying to do. Problem with that site is that it is wordpress and you can only enter 2 links. I want to do something similar but with user defined number of links.
Please take a look at the code I have for what I put together today: http://rpantaev.com/RedirectLinkTool/
After researching, it seems that I need to use PHP and perhaps a database like mysql to store users' data, but I am not familiar with PHP so any pointers will be greatly appreciated.
So far php is installed on my server and I added .htaccess so html can read php. I put together a webpage with html/css/js that takes in user defined number of links, checks for http://, stores in array and redirects to randomly chosen url (array is not persistent and no new url is generated).
How can I recreate what the first link does and incorporate that into my website?
Thank you!
So the language you use will depend on what languages your server supports or whether you can install your own. If your looking to run a server on your computer to just test this out, you just need to install the language you want to use then find a way to run as server on your computer.
You can use any database (postgres, mongodb), to store your urls. If it's ok to use memory for this project then you can use a dictionary to store the random urls.
Here's a way to do this using Node.js. In this solution, I create a server on port 9898 and create an in memory database using a dictionary. When there are entries, the database will look like this:
{
"uuid1": { id: "uuid1", urls: ["https://www.google.com", "https://youtube.com"] },
"uuid2": { id: "uuid2", urls: ["https://www.google.com", "https://youtube.com"] }
}
uuid1 and uuid2 would be replaced with actual unique IDs.
In this solution, I'm expecting that the client will send a json object.
The solution depends on expressjs and body-parser packages. Here's actual code that I test on my computer.
const express = require('express')
const bodyParser = require('body-parser');
const app = express()
app.use(bodyParser.json()); // for parsing application/json
app.use(bodyParser.urlencoded({ extended: true })); // for parsing application/x-www-form-urlencoded
const port = 9898
const host = `http://localhost:${port}`
// you would replace this with an actual database
const db = {}
// to generate a uuid
// from: https://stackoverflow.com/a/2117523
function uuidv4() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
const r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
}
function success(message) {
return JSON.stringify({
success: message
})
}
function error(message) {
return JSON.stringify({
error: message
})
}
// assuming body looks like this { urls: ['www.google.com', 'www.gmail.com'] }
app.post('/gen', (req, res) => {
const gen = req.body
// check if the json in the request has the list of urls
if(!(gen && gen.urls && gen.urls.length > 0)) {
// if not then log the body and return a json object with the error object
console.log('body', req.body)
return res.send(error('I cannot find the list of urls in json body.'))
}
// generate a uuid
const id = uuidv4()
// save the urls with the new id
db[id] = {
id: id,
urls: gen.urls
}
// respond with the url that the user should use
res.send(success(host + '/' + id))
})
app.get('/:id', (req, res) => {
// get the id paramater from the request
const id = req.params.id
// check if that id was saved, if not send an error message
if(!db[id])
return res.send(error('that url does not exist'))
// grab a random element from the list of urls
const urls = db[id].urls
const url = urls[Math.floor(Math.random() * urls.length)]
// redirect the user to the random url
res.redirect(url)
})
// start the server
app.listen(port, () => {
console.log('listening on port: '+port);
})
/*
example request:
curl -X POST -H "Content-Type: application/json" -d '{"urls": ["https://www.google.com", "https://www.yahoo.com", "https://www.youtube.com"]}' localhost:9898/gen
after doing this you grab the url generated and paste into your browser.
for example when I ran this it generated: http://localhost:9898/aa582177-4ab5-4ede-99c5-a06b43976c12
*/
If you've never setup a nodejs project before this page can get you up to speed: https://docs.npmjs.com/getting-started/installing-node
After install nodejs and npm. create a folder for the project. In your command line, go to this directory. To setup the project and install the two packages, write:
npm init -y
npm install express body-parser
After doing that create a file named index.js in that folder. Then paste the code in. Then to run the server use this command:
node index.js
If your on MacOS or Linux, or have a way to use curl on Windows you can use this example request:
curl -X POST -H "Content-Type: application/json" -d '{"urls": ["https://www.google.com", "https://www.yahoo.com", "https://www.youtube.com"]}' localhost:9898/gen
Then that should return a url for you to use. For example for me it generated this:
http://localhost:9898/aa582177-4ab5-4ede-99c5-a06b43976c12
You would paste the url generated into your browser and it should reroute you to one of the urls in the list.
I want to send asynchronous data to the node on configuration. I want to
perform a SQL request to list some data in a .
On node creation, a server side function is performed
When it's done, a callback send data to the node configuration
On node configuration, when data is received, the list is created
Alternatively, the binary can request database each x minutes and create a
cache that each node will use on creation, this will remove the asynchronous
part of code, even if it's no longer "live updated".
In fact, i'm stuck because i created the query and added it as below :
module.exports = function(RED) {
"use strict";
var db = require("../bin/database")(RED);
function testNode(n) {
// Create a RED node
RED.nodes.createNode(this,n);
// Store local copies of the node configuration (as defined in the
.html
var node = this;
var context = this.context();
this.on('input', function (msg) {
node.send({payload: true});
});
}
RED.nodes.registerType("SQLTEST",testNode);
}
But I don't know how to pass data to the configuration node. I thought of
Socket.IO to do it, but, is this a good idea and is it available? Do you know any solution ?
The standard model used in Node-RED is for the node to register its own admin http endpoint that can be used to query the information it needs. You can see this in action with the Serial node.
The Serial node edit dialog lists the currently connected serial devices for you to pick from.
The node registers the admin endpoint here: https://github.com/node-red/node-red-nodes/blob/83ea35d0ddd70803d97ccf488d675d6837beeceb/io/serialport/25-serial.js#L283
RED.httpAdmin.get("/serialports", RED.auth.needsPermission('serial.read'), function(req,res) {
serialp.list(function (err, ports) {
res.json(ports);
});
});
Key points:
pick a url that is namespaced to your node type - this avoids clashes
the needsPermission middleware is there to ensure only authenticated users can access the endpoint. The permission should be of the form <node-type>.read.
Its edit dialog then queries that endpoint from here: https://github.com/node-red/node-red-nodes/blob/83ea35d0ddd70803d97ccf488d675d6837beeceb/io/serialport/25-serial.html#L240
$.getJSON('serialports',function(data) {
//... does stuff with data
});
Key points:
here the url must not begin with a /. That ensures the request is made relative to wherever the editor is being served from - you cannot assume it is being served from /.
I am making a web chat client where I use redis for pub/sub. I am having trouble with the subscribe part. I am able to publish but I am not sure how to subscribe. I have a php script written to subscribe (it work when I run php) it listens and echos the message. I want to be able to get that message in javascript. How do I call the php file and listen? I tried ajax in jquery and listening for the echo in the success function but it does not seem to work. I am new to this any advice is helpful
EDIT: Here is the javascript
$.ajax({
url:"http://localhost/redisphp.php",
type: GET,
success: function(response){ ...},
...
Here is the redis. I modeled after this link https://xmeng.wordpress.com/2011/11/14/pubsub-in-redis-using-php/
<?php
function f($redis, $chan, $msg) {
switch($chan) {
case 'chan-1':
echo $msg;
}
}
ini_set('default_socket_timeout', -1);
$redis = new Redis();
$redis->pconnect('128.0.0.0',6378);
$redis->subscribe(array('chan-1'), 'f');
print "\n";
?>
I have a php script written to subscribe (it work when I run php) it listens and echos the message. I want to be able to get that message in javascript.
It sounds like you would need to create an Express API that would connect to your Redis server.
Please keep in mind I am going on very limited information that you have provided. So first thing with your Express API is to create your package.json file like so:
{
"dependencies": {
"express": "4.16.3",
"redis": "2.8.0",
"nodemon": "1.18.3"
},
"scripts": {
"dev": "nodemon",
"start": "node index.js"
}
}
You don't want it to look exactly like this of course, but just pointing you in the right of what you want to be doing. Of course you will need to do an npm install for those particular dependencies.
Then if I were you I would create a server/keys.js file like so:
module.exports = {
redisHost: process.env.REDIS_HOST,
redisPort: process.env.REDIS_PORT,
};
And then require that inside a server/index.js file where you will also add the following:
const keys = require("./keys");
// Express App Setup
const express = require("express");
const app = express();
// Redis Client Setup
const redis = require(‘redis’);
const redisClient = redis.createClient({
host: keys.redisHost,
port: keys.redisPort,
retry_strategy: () => 1000
});
const redisPublisher = redisClient.duplicate();
So this retry_strategy takes an arrow function and it is saying that if we ever lose connection to the redis server try to reconnect to it once every second.
The key for retry_strategy is separated by an underscore as opposed to the JavaScript standard of camelCase.
According to the Redis documentation for the Javascript library, if we ever have a client that is listening or publishing information on Redis we have to make a duplicate connection between when a connection is turned into one that is going to listen, subscribe or publish information, it cannot be used for other purposes.
So thats why I am doing this duplicate() thing at the end of redisClient.
So that is pretty much it.