I'm trying to get back from the remote site(URL) in the following node.js codes.
Case 1: Using basic HTTP module + request module
var http = require('http');
var fs = require('fs');
var request = require('request');
var server = http.createServer(function(req, res) {
// Can't get back any response from remote URL
// Can't use 'res' or nothing have in res
request('http://remote.foo.com/app/lists/dosomething.json',
function(err, response, body) {
console.log(body);
}
);
var host = 'http://remote.foo.com';
var app = '/app/lists/dosomething.json?';
var url = 'param1=a¶m2=b';
// I can get something to print out back
request.get(host+app+url).pipe(res);
});`
Case 2: Using Express + request module
var express = require('express')
, http = require('http')
, path = require('path')
, net = require('net')
, url = require('url')
, qs = require('qs');
var request = require('request');
//.....
app.get('/test/foo.json', function(req, res) {
var url_parts = url.parse(req.url, true);
var query = qs.parse(url_parts.query);
var host = 'http://remote.foo.com';
var app = '/path/dosomething.json?';
var url = 'param1=a¶m2=b';
//the following line, the same code above Case 1
//can't get any result, just blank..
request.get(host+app+url).pipe(res);
res.end();
});
//....
Can anyone explain it?
Thanks
Expanding on my comment a little bit:
When you pipe() a readable stream into a writable stream, by default writable.end() is called when the readable stream emits the end event (see documentation). This means that when you pipe streams, you don't have to explicitly close the destination stream by yourself: it is automatically closed when the source closes. This should make sense: when the source stream ends, the destination stream doesn't have anything to do, and should indeed close.
In your second example, you first pipe a readable stream (request) into a writable stream (res), but the very next instruction closes the res stream, without even reading anything, since the first read() call is executed on the next tick (see source). The solution is then to remove res.end(), since it will be handled automatically.
As a side note: if you need to keep the destination stream open for some reason, you can pass {end: false} to pipe.
Related
I have this URL here: https://viacep.com.br/ws/01001000/json/ , that lets me retrieve JSON data whenever i change the given number on it(the number is unique). For example: 01001000 has its own data, and if i change it for 49140000, it will have its own data as well. What i want to do is: i want to save the JSON data into a database, and somehow cache/save the request, so if in the future i search for the same number again, it won't have to retrieve the data from the URL again.
I have this right now:
My city.routes.js where i make the request to the API:
const express = require('express');
const axios = require('axios');
const cityRouter = express.Router();
cityRouter.get('/:cep', async (request, response) => {
try {
const { cep } = request.params;
const resp = await axios.get(`https://viacep.com.br/ws/${cep}/json/`);
return response.json(resp.data);
} catch (error) {
return response.status(400);
}
});
module.exports = cityRouter;
An index.js to make easier to the server to import and use the routes:
const express = require('express');
const routes = express.Router();
const cityRoutes = require('./city.routes');
routes.use(cityRoutes);
module.exports = routes;
My server.js:
const express = require('express');
const routes = require('./routes');
const app = express();
app.use(express.json());
app.use(routes);
app.listen(3333, () => {
console.log('Server is on!');
});
I can retrieve the JSON data that i want without problems:
enter image description here
You can do this via using caching libraries or using db and indexing on based of number for faster retrieval.
My suggestion:
If you need to cache for smaller amount of time lets say week or so prefer caching libraries like redis or memcache.
There u can do something like:
await redis.set(key, JSON.stringify(data), { expiry: '1W'});
The above code varies depending on library you use. But the idea remains the same you cache the data with key(Number).
And next time before making request you first tries to get the key from cache provider.
await redis.get(key)
if above value is present you will get json string of the desired result and dont need to make the network call.
If not present you make the network call and cache the result for future use.
In case of DB approach you simply make a get request via key to the db.
But do index the key in collection or relation when initializing the structure for faster retrieval.
I am unable to make a POST method request with . I am unable to send a request object with it.
For example see this example. Here only GET request is being made. But I want to make a POST request.
If you want to send a POST request to an another host it would be easier to use some libraries dedicated to this task like axios or request.
I was facing the same issue. When I tried to make a POST request I was receiving undefined as the value of the body object.
I ended up not using koa-http-request as the example provided in your link. I have used both koa-bodyparser and koa-router middlewares instead:
const Koa = require('koa');
const KoaRouter = require('koa-router');
const bodyParser = require('koa-bodyparser');
const app = new Koa();
const router = new KoaRouter();
// ===== M i d d l e w a r e s =====
// Body parser
app.use(bodyParser());
// Router
app.use(router.routes()).use(router.allowedMethods());
// ...
const addItem = async (context) => {
const body = await context.request.body;
// Now you can access the body object
console.log(body)
}
// ...
router.post('/', addItem);
In my case it had worked as it supposed to. :)
I'm trying to create a small service that converts images from one type to another.
I'll get to the conversion part later, right now i cant even send it to the node server properly.
i have a simple script that's supposed to use filesystem to read an image and then POST it to a node server endpoint.
I couldn't find a way to do this anywhere online
i tried all kinds of different ways and formats, but on the endpoint the req.body is always an empty object or just an error
const fse = require('fs-extra');
var XMLHttpRequest = require("xmlhttprequest").XMLHttpRequest;
var xhr = new XMLHttpRequest();
fse.readFile('testimg.png', function (err, data) {
if (err) {
throw err
}
console.log(data)
xhr.open("POST", 'http://localhost:3000/convert', true); // true for asynchronous
xhr.setRequestHeader('Content-Type', 'application/upload');
xhr.send(data)
})
and this is the server endpoint:
var express = require('express');
var router = express.Router();
router.get('/', (req,res)=>{
res.send("hello")
})
router.post('/', async(req,res)=>{
console.log(req.body);
res.send("Hello i work")
})
module.exports = router;
What i want is to get the data on the server endpoint and to be able to process it and convert it for example: upload jpg and convert and send back as png or the opposite.
Help highly appreciated
I am doing node.js exercises from nodeschool.io (learnyounode). One of the exercises involves creating a http server which serves a text file from a readable file stream. I'm very new to asynchronous programming. The solution I came up with is:
var http = require('http');
var fs = require('fs');
var readable = fs.createReadStream(process.argv[3]);
var server = http.createServer(function(request, response) {
readable.on('data', function(chunk) {
response.write(chunk);
})
});
server.listen(process.argv[2]);
This works, however the official solution uses a pipe instead of on-data event:
var http = require('http')
var fs = require('fs')
var server = http.createServer(function (req, res) {
res.writeHead(200, { 'content-type': 'text/plain' })
fs.createReadStream(process.argv[3]).pipe(res);
})
server.listen(Number(process.argv[2]))
What are the (potential) differences and/or benefits of doing it either way?
Well, there's more code in your version, and that usually means you have more options to make mistakes. Take into account some edge cases, like what happens when the stream throws an error?
I'm not exactly sure what the behavior would be (you can check yourself by e.g. inserting some non-existing filename) but chances are that in your version the error handling is not working very well, potentially ignoring errors (because you're not listening for error events).
I saw a similar question, but I'm looking for a way to do it manually. I don't want to use express or another library to do it.
var http = require('http');
var server = http.createServer(function(req, res) {
res.end('<h1 >Hi!</h1>'); //I want to to fetch a file ex: index.html
});
server.listen(9334);
How would i do that? Also as a sub-question, just because I'm curious. Is it possible to use jQuery ajax to fetch this file?
Here is one way to do is using 'fs'.
var http = require('http');
var fs = require('fs');
var server = http.createServer(function(req, res) {
fs.createReadStream("filename.ext").pipe(res);
});
server.listen(9334);
This is also good because if the file is big the data is streamed to the response instead of waiting for the read operation to be completed. Although you might have to set 'Content-Type' header in some cases.