I have a Frontend application sending file over HTTP to Nodejs Backend. The Nodejs application acts as a socketio server and needs to send the file to a python socketio client. I have used multer to parse the HTTP request. Here are the nodejs and python code I have written so far (only pasted required parts)
// NodeJS server code I have written so far (could be completely wrong)
// req.file contains the file parsed from multer
var socket = req.app.get('socket');
let readStream = createReadStream(req.file.path);
readStream.on('ready', () => {
socket.emit('image', readStream, data => console.log(data));
readStream.pipe(createWriteStream(req.file.originalname));
});
# Python socketio client
#sio.event
async def image(data):
try:
print(data)
# want to save data preferably in a local file or a buffer is also fine
except:
return 'NOT OK'
finally:
return 'OK'
I am facing issues on both fronts, for both of which I have almost no idea how to procede:
sending image from nodejs (thought of using fs.createReadStream but don't know how to use the stream over a socketio event)
receiving image on python and storing it locally.
Related
I have a node.js server and client using graphql. I need to upload file from a client, send it to the node server and then forward this file to another service, lets name it X.
When i upload file to node.js server i can get a readable stream of file using apollo graphql API and it works fine.
But when i try send this stream to X using axios, on X i can read only one chunk of the file.
Here's code from my node.js server:
const { createReadStream } = await file;
const stream = createReadStream();
axios.post('http://localhost:7000', stream, {
maxBodyLength: Infinity,
maxContentLength: Infinity
});
And here is a receiver (X server) controller code
const writeStream = fs.createWriteStream('final.csv');
req.pipe(writeStream);
res.send('Hello');
In final.csv file i can see only small part of data i send. I want to be able to send files up to 1GB size.
TL;DR:
How can I send data from client-side javascript to server-side javascript file (node.js or whatever), receive it there (and do some stuff with it), then send it back to client-side javascript file for further usage?
Full description of the issue
There is an HTML page with a form, from which myjavascript.js collect user input and process it on client side (using FormData). Then myjavascript.js sends the processed input to myphp.php (server-side) this way:
myjavascript.js:
ajax = new XMLHttpRequest();
ajax.open("POST", "../../myphp.php",false);
ajax.send(formdata);
return ajax.responseText;
The data (formdata) is then received by myphp.php, undergoes some further processing, and then is sent back to the myjavascript.js:..
myphp.php:
$fieldOne = $_POST["fieldOne"];
$fieldTwo = $_POST["fieldTwo"];
...
($fieldOne, $fieldTwo etc. are processed, and the result is assigned to $results)
...
echo json_encode($results);
exit();
... where it is returned by return ajax.responseText; (as in myjavascript.js code above). And then this data is displayed on the HTML page etc.
The question
I used to do this way before. Now I would like to use server-side javascript (say, myserver.js) instead of myphp.php. I have the code for processing the data in myserver.js, I just do not understand how to receive the data from myjavascript.js and send the response back. Is it possible, and if yes, how the code for it should look like? Generally, I do not quite understand how the mechanism of receiving POST data in server-side javascript (and sending the response back) differs from the PHP one. Thank you for any suggestions and comments.
In PHP, the web server portion is typically implemented by some other framework, and your PHP scripts are called by that framework. For example, an Apache module may call your blah.php script incoming web request is for /blah.php. This means PHP developers often think in terms of the browser calling a PHP file, but it's actually more complicated than that behind the scenes.
A typical Node.js setup, in contrast, is that the JavaScript code itself implements the web server listener. You run node with your server JS script, and that script starts the server. The built-in http module, for example, lets you implement an low-level HTTP listener through createServer:
const { createServer } = require('http');
const { once } = require('events');
const server = createServer(async (req, res) => {
const chunks = [];
req.on('data', chunk => chunks.push(chunk));
await once(req, 'end');
const body = JSON.parse(chunks.join(''));
// Do stuff with the body
res.statusCode = 200;
res.end('Success!');
});
server.listen(8080);
console.log('Server is listening at http://0.0.0.0:8080/');
So basically, stop thinking in terms of a client-side JavaScript file sending things to a server-side JavaScript file and think in terms of your browser sending an HTTP request to your HTTP server.
I'd recommend you use a framework instead like express or koa to build your server since it lets you more easily configure route-specific handling for requests, and they have the ability to use pre-built "middlewares" to do the common stuff like JSON body parsing and much more.
You need to use node js and express js so that you can reach your goal easily.
Run the following commands after installed node js:
$ npm install express --save
$ npm install body-parser --save
myserver.js
const express = require('express')
const app = express()
const port = 3000
const bodyParser = require('body-parser')
app.use( bodyParser.json() )
app.use(bodyParser.urlencoded({
extended: true
}))
app.get('/', function (req, res) {
console.log(req.body.fieldOne)
console.log(req.body.fieldTwo)
res.send('Hello World!')
})
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
And then, run your server with the bellow command:
$ node myserver.js
I've set up an API with nodejs express for a real-time chat application. For it to be real-time I am using primus but I'm currently stuck at trying to connect primus to my frontend.
I have a folder structure for the whole backend and then another folder structure for my frontend. So they are both separate.
Here I connect the server to Primus
var server = http.createServer(app);
const primus = require('../primus/live').go(server);
This then goes as you can see to the folder primus with a file live.js
//BACKEND
const Primus = require('primus');
let go = (server) => {
let primus = new Primus(server, {/* options */});
primus.on('connection', (spark) => {
console.log('Received spark 🔥');
});
}
module.exports.go = go;
Now in my frontend, I am trying to call Primus via the script tag
//FRONTEND
<script src="http://localhost:3000/primus/live.js"></script>
but this just gives me a 404 Not Found error. Also when I just try to connect through this in my browser it doesn't work. So I am unsure what my problem here is. Any ideas?
https://github.com/primus/primus#how-do-i-use-primus-with-express
make sure to call .listen on the http server, not the Express server
I am trying to build a real-time Django application.
Because of the way my hosting service works, I am unable to start a Websocket server in parallel of my Django server.
I managed to have user-to-user interactions by creating an express server on a separate NodeJS website with socket.io, and having clients on the Django server also connect to the remote socket.io server.
However, I'dlike to have my Django server directly send events to users. To do this, I would like to create a connection between the Django server and the NodeJS server. Something like that in python:
socket = io("http://socket.io.server")
socket.emit('eventForUsers')
Is there anyway for me to achieve this?
The only information I found seemed to require me to run a parallel server from my Django app, which I can't do because my host doesn't allow me to run long-term processes.
It really depends what is the most simple solution for you, and what are your requirements. (If you want realtime bidirectional messaging then I suggest to use the socket.io-client instead of the POST example)
POST (GET,PUT,...)
You can (for example) use POST requests from your Django to Node
Django: python example (there are other ways to perform a POST to Node from Django)
reqParams = {"action":"doThis","data":"put the pizza in the oven"}
import requests
requests.post('http://ip:port/route', params = reqParams)
Node example : Listens for post at /route (express) and prints the params of the Django request
var express = require('express');
var app = express();
app.post('/route', function (req, res) {
console.log(req.query); res.end();
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
Then you can use the data in req.params to perform action like broadcasting something to the socket.io clients (or a specific client)
This can also be done the other way around, performing requests to send data from Node to Django using POST (GET,...)
Socket.io-client in Django
Another (easier) solution is to include the socket.io-client in your Django webapp so it connects to your Node socket.io server as a client when the webapp is opened by browsers.
(using javascript in django)
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io('http://ip:port');
socket.on('connect', function(){alert("Hello World!");});
socket.on('event', function(data){});
socket.on('disconnect', function(){});
</script>
More Resources :
1. JavaScript (Django Docs)
2. Including the static files in your template
I have some data that I want to store locally and to be able to pull it dynamically, maybe in another session or after the browser was closed and all browser data was cleared.
I run the site with http-server CLI command and navigate to localhost to access it from the browser.
How can I send data to the server side so the server side will save the data as a file?
I tried to do an ajax post request to see if something happens in the console, but it just returned 404 and nothing came up in the console.
The docs don't mention anything about post requests: https://www.npmjs.com/package/http-server
PS: I have to run this with http-server, this is an offline project.
You will not be able to do this with http-server alone, because http-server can only serve static content and cannot be used to run any code on the server side.
You will have to write a backend yourself, possibly using a framework like Express, Hapi, Restify, Loopback etc. and serve your static files that you need with your new backend, or keep it served as you do now but then you will probably need to take CORS into account if you use different ports for your data saving/retrieving endpoints and your static content - unless you run a reverse proxy that makes it all appear on the same host name and port.
You can use the file system to save the data or you can use a database - either a standalone database like Mongo or Postgres or an embedded database like SQLite or Loki.
For examples on how to serve static content in your own backend see:
How to serve an image using nodejs
You should use express for this kind of stuff. You can easily make methods that handle certain requests.
Here is an exmaple on how to handle a get request by just sending some data
var express = require('express')
var app = express()
app.get('/', function (req, res) {
res.send('Hello World')
})
app.listen(3000)
And you can use the fs api from node itself to write data.
var fs = require('fs')
fs.writeFile('message.txt', 'Hello Node.js', (err) => {
if (err) throw err;
console.log('It\'s saved!');
});
Note: the fs example uses arrow functions. You can find more information here