Develop Reference

Node & Angular, Feedly RSS API call, get data from service and pass to controller. Callback or forming url with auth header for https.get() request

I have a controller.js file that I am trying to get data from my feed-service.js file. FEEDLY_USER_ID and FEEDLY_ACCESS_TOKEN are accessible and defined in a separate config.js file.
controller.js:
$scope.feedlyGlobalAll = FeedService.getGlobalAll();
feed-service.js:
var request = require('request');
var globalOptions = {
url: 'https://cloud.feedly.com/v3/streams/contents?streamId=user/' + FEEDLY_USER_ID + '/category/global.all',
auth: {
'bearer': FEEDLY_ACCESS_TOKEN
}
};
service.getGlobalAll = function(){
request.get(globalOptions, function(error, response, body){
if(!error && response.statusCode == 200){
service.globalAll = JSON.parse(body);
return service.globalAll;
}
// error code here //
})
}
I'm using an npm package called "request" to make the GET because I couldn't get https.get() work. This Feedly API call requires the URL, my user ID, and an access token passed in the header.
I've been reading and apparently I'm supposed to use callbacks or promises, but I can't get either of them to work. With http.get(), I can utilize promises by http.get().then(yada yada), which works for the forecast.io call I'm making elsewhere. The request module apparently doesn't allow .then(). I typically run into TypeError and .then() is not a function.
When I tried doing https.get(), here is the code I was using. I was never able to acquire a successful response.
var url = {
url: 'https://cloud.feedly.com/v3/streams/contents?streamId=user/' + FEEDLY_USER_ID + '/category/global.all',
'headers': {
'Authorization': 'Bearer ' + FEEDLY_ACCESS_TOKEN
}
};
https = require('https');
https.get(url).then(yada yada)
I tried numerous things in the var url for the https.get call. I tried with and without quotes, I tried just auth: bearer token as it works with the request module, but wasn't able to get it to work.
Solution to this issue would either be:
Callback from feed-service to controller using existing request module code.
Figuring out the correct way to form the url for the https.get request and then I can utilize its promise function.

Use a promise inside your request callback:
service.getGlobalAll = function() {
return $q(function(resolve, reject) {
request.get(globalOptions, function(error, response, body){
if(!error && response.statusCode == 200){
service.globalAll = JSON.parse(body);
resolve(service.globalAll);
} else {
reject();
}
});
});
};
$q (angular's promise api) can be injected as a dependency in your service. The above code will return a promise that will resolve when your library's ajax call returns, so you can access it with .then().

Relative uri for node.js request library

I have the following code, and node.js can't resolve the url:
const request = require('request')
const teamURL = `/users/${user._id}/teams`;
const req = request({
url: teamURL,
json: true
},
function(error, response, body) {
if (!error && response.statusCode == '200') {
res.render('userHome.html', {
user: user,
teams: body
});
}
else {
console.error(error);
next(error);
}
});
is there a good way to use relative paths/urls with the request library on a server-side node.js Express app?

Giving just a relative url only works if it is clear from context what the root part of the url should be. For instance, if you are on stackoverflow.com and find a link /questions, it's clear from context the full url should be stackoverflow.com/questions.
The request library doesn't have this kind of context information available, so it needs the full url from you to do be able to make the request. You can build the full url yourself of course, for instance by using url.resolve():
var url = require('url');
var fullUrl = url.resolve('http://somesite.com', '/users/15/teams');
console.log(fullUrl); //=> 'http://somesite.com/users/15/teams');
But of course this will still require you to know the root part of the url.

Jasper 's answer is correct -- the request module needs full URL. if you are in a situation where you have a single page application, with lots of requests to an API with the same base URL, you can save a lot of typing by creating a module like this:
var url = require('url');
var requestParser = (function() {
var href = document.location.href;
var urlObj = url.parse(href, true);
return {
href,
urlObj,
getQueryStringValue: (key) => {
let value = ((urlObj && urlObj.query) && urlObj.query[key]) || null;
return value;
},
uriMinusPath: urlObj.protocol + '//' + urlObj.hostname
};
})();
then, to grab the base URL anytime you need it: requestParser.uriMinusPath
and grab the value of an arbitrary query parameter: RequestParser.getQueryStringValue('partner_key');

Node.js minimal function for parsing route

I have a Node.js / Express app working, that receives routes like so:
app.get('/resource/:res', someFunction);
app.get('/foo/bar/:id', someOtherFunction);
This is great and works fine.
I am also using Socket.IO, and want to have some server calls use websockets instead of traditional RESTful calls. However, I want to make it very clean and almost use the same syntax, preferrably:
app.sio.get('/resource/:res', someFunction);
This will give a synthetic 'REST' interface to Socket.IO, where, from the programmer's perspective, he isn't doing anything different. Just flagging websockets: true from the client.
I can deal with all the details, such as a custom way to pass in the request verbs and parse them and so and so, I don't have a problem with this. The only thing I am looking for is some function that can parse routes like express does, and route them properly. For example,
// I don't know how to read the ':bar',
'foo/:bar'
// Or handle all complex routings, such as
'foo/:bar/and/:so/on'
I could dig in real deep and try to code this myself, or try to read through all of express' source code and find where they do it, but I am sure it exists by itself. Just don't know where to find it.
UPDATE
robertklep provided a great answer which totally solved this for me. I adapted it into a full solution, which I posted in an answer below.

You can use the Express router class to do the heavy lifting:
var io = require('socket.io').listen(...);
var express = require('express');
var sioRouter = new express.Router();
sioRouter.get('/foo/:bar', function(socket, params) {
socket.emit('response', 'hello from /foo/' + params.bar);
});
io.sockets.on('connection', function(socket) {
socket.on('GET', function(url) {
// see if sioRouter has a route for this url:
var route = sioRouter.match('GET', url);
// if so, call its (first) callback (the route handler):
if (route && route.callbacks.length) {
route.callbacks[0](socket, route.params);
}
});
});
// client-side
var socket = io.connect();
socket.emit('GET', '/foo/helloworld');
You can obviously pass in extra data with the request and pass that to your route handlers as well (as an extra parameter for example).

robertklep provided a great answer which totally solved this for me. I adapted it into a full solution, which is below in case others want to do something similar:
Node (server side):
// Extend Express' Router to a simple name
app.sio = new express.Router();
app.sio.socketio = require('socket.io').listen(server, { log: false });
// Map all sockets requests to HTTP verbs, which parse
// the request and pass it into a simple callback.
app.sio.socketio.sockets.on('connection', function (socket) {
var verbs = ['GET', 'POST', 'PUT', 'PATCH', 'DELETE'];
for (var i = 0; i < verbs.length; ++i) {
var go = function(verb) {
socket.on(verb, function (url, data) {
var route = app.sio.match(verb, url);
if (route && route.callbacks.length) {
var req = {url: url, params: route.params, data: data, socket:socket}
route.callbacks[0](req);
}
});
}(verbs[i]);
}
});
// Simplify Socket.IO's 'emit' function and liken
// it traditional Express routing.
app.sio.end = function(req, res) {
req.socket.emit('response', req.url, res);
}
// Here's an example of a simplified request now, which
// looks nearly just like a traditional Express request.
app.sio.get('/foo/:bar', function(req) {
app.sio.end(req, 'You said schnazzy was ' + req.data.schnazzy);
});
Client side:
// Instantiate Socket.IO
var socket = io.connect('http://xxxxxx');
socket.callbacks = {};
// Similar to the server side, map functions
// for each 'HTTP verb' request and handle the details.
var verbs = ['get', 'post', 'put', 'path', 'delete'];
for (var i = 0; i < verbs.length; ++i) {
var go = function(verb) {
socket[verb] = function(url, data, cb) {
socket.emit(String(verb).toUpperCase(), url, data);
if (cb !== undefined) {
socket.callbacks[url] = cb;
}
}
}(verbs[i]);
}
// All server responses funnel to this function,
// which properly routes the data to the correct
// callback function declared in the original request.
socket.on('response', function (url, data) {
if (socket.callbacks[url] != undefined) {
socket.callbacks[url](data);
}
});
// Implementation example, params are:
// 1. 'REST' URL,
// 2. Data passed along,
// 3. Callback function that will trigger
// every time this particular request URL
// gets a response.
socket.get('/foo/bar', { schnazzy: true }, function(data){
console.log(data); // -> 'You said schnazzy was true'
});
Thanks for your help, robertklep!

How do I set a MIME type before sending a file in Node.js?

When sending scripts from my Node.js server to the browser, in Google Chrome, I get this warning:
Resource interpreted as Script but transferred with MIME type
text/plain
I Google'd around, and found out that it's a server-side problem, namely, I think that I should set the correct MIME type to things, before sending them. Here's the HTTP server's handler:
var handler = function(req, res)
{
url = convertURL(req.url); //I implemented "virtual directories", ignore this.
if (okURL(url)) //If it isn't forbidden (e.g. forbidden/passwd.txt)
{
fs.readFile (url, function(err, data)
{
if (err)
{
res.writeHead(404);
return res.end("File not found.");
}
//I think that I need something here.
res.writeHead(200);
res.end(data);
});
}
else //The user is requesting an out-of-bounds file.
{
res.writeHead(403);
return res.end("Forbidden.");
}
}
Question: How do I correct my server-side code to configure the MIME type correctly?
(Note: I already found https://github.com/broofa/node-mime, but it only lets me determine the MIME type, not to "set" it.)

I figured it out!
Thanks to #rdrey's link and this node module I managed to correctly set the MIME type of the response, like this:
function handler(req, res) {
var url = convertURL(req.url);
if (okURL(url)) {
fs.readFile(url, function(err, data) {
if (err) {
res.writeHead(404);
return res.end("File not found.");
}
res.setHeader("Content-Type", mime.lookup(url)); //Solution!
res.writeHead(200);
res.end(data);
});
} else {
res.writeHead(403);
return res.end("Forbidden.");
}
}

Search google for the Content-Type HTTP header.
Then figure out how to set it with http://expressjs.com/api.html#res.set
Oops, the example includes your answer ;)
Simply check the file ending, if it's .js, set the appropriate MIME type to make browsers happy.
EDIT: In case this is pure node, without express, look here: http://nodejs.org/api/http.html#http_response_setheader_name_value

mime.lookup() is now renamed to mime.getType().
So you can do like this:
res.set('Content-Type', mime.getType('path/file'));
https://www.npmjs.com/package/mime

You can copy the following JSON, containing a comprehensive list of mime types,
{".x3d":"application/vnd.hzn-3d-crossword",".3gp":"video/3gpp",".3g2":"video/3gpp2",".mseq":"application/vnd.mseq",".pwn":"application/vnd.3m.post-it-notes",".plb":"application/vnd.3gpp.pic-bw-large",".psb":"application/vnd.3gpp.pic-bw-small",".pvb":"application/vnd.3gpp.pic-bw-var",".tcap":"application/vnd.3gpp2.tcap",".7z":"application/x-7z-compressed",".abw":"application/x-abiword",".ace":"application/x-ace-compressed",".acc":"application/vnd.americandynamics.acc",".acu":"application/vnd.acucobol",".atc":"application/vnd.acucorp",".adp":"audio/adpcm",".aab":"application/x-authorware-bin",".aam":"application/x-authorware-map",".aas":"application/x-authorware-seg",".air":"application/vnd.adobe.air-application-installer-package+zip",".swf":"application/x-shockwave-flash",".fxp":"application/vnd.adobe.fxp",".pdf":"application/pdf",".ppd":"application/vnd.cups-ppd",".dir":"application/x-director",".xdp":"application/vnd.adobe.xdp+xml",".xfdf":"application/vnd.adobe.xfdf",".aac":"audio/x-aac",".ahead":"application/vnd.ahead.space",".azf":"application/vnd.airzip.filesecure.azf",".azs":"application/vnd.airzip.filesecure.azs",".azw":"application/vnd.amazon.ebook",".ami":"application/vnd.amiga.ami","N/A":"application/andrew-inset",".apk":"application/vnd.android.package-archive",".cii":"application/vnd.anser-web-certificate-issue-initiation",".fti":"application/vnd.anser-web-funds-transfer-initiation",".atx":"application/vnd.antix.game-component",".dmg":"application/x-apple-diskimage",".mpkg":"application/vnd.apple.installer+xml",".aw":"application/applixware",".les":"application/vnd.hhe.lesson-player",".swi":"application/vnd.aristanetworks.swi",".s":"text/x-asm",".atomcat":"application/atomcat+xml",".atomsvc":"application/atomsvc+xml",".ac":"application/pkix-attr-cert",".aif":"audio/x-aiff",".avi":"video/x-msvideo",".aep":"application/vnd.audiograph",".dxf":"image/vnd.dxf",".dwf":"model/vnd.dwf",".par":"text/plain-bas",".bcpio":"application/x-bcpio",".bin":"application/octet-stream",".bmp":"image/bmp",".torrent":"application/x-bittorrent",".cod":"application/vnd.rim.cod",".mpm":"application/vnd.blueice.multipass",".bmi":"application/vnd.bmi",".sh":"application/x-sh",".btif":"image/prs.btif",".rep":"application/vnd.businessobjects",".bz":"application/x-bzip",".bz2":"application/x-bzip2",".csh":"application/x-csh",".c":"text/x-c",".cdxml":"application/vnd.chemdraw+xml",".css":"text/css",".cdx":"chemical/x-cdx",".cml":"chemical/x-cml",".csml":"chemical/x-csml",".cdbcmsg":"application/vnd.contact.cmsg",".cla":"application/vnd.claymore",".c4g":"application/vnd.clonk.c4group",".sub":"image/vnd.dvb.subtitle",".cdmia":"application/cdmi-capability",".cdmic":"application/cdmi-container",".cdmid":"application/cdmi-domain",".cdmio":"application/cdmi-object",".cdmiq":"application/cdmi-queue",".c11amc":"application/vnd.cluetrust.cartomobile-config",".c11amz":"application/vnd.cluetrust.cartomobile-config-pkg",".ras":"image/x-cmu-raster",".dae":"model/vnd.collada+xml",".csv":"text/csv",".cpt":"application/mac-compactpro",".wmlc":"application/vnd.wap.wmlc",".cgm":"image/cgm",".ice":"x-conference/x-cooltalk",".cmx":"image/x-cmx",".xar":"application/vnd.xara",".cmc":"application/vnd.cosmocaller",".cpio":"application/x-cpio",".clkx":"application/vnd.crick.clicker",".clkk":"application/vnd.crick.clicker.keyboard",".clkp":"application/vnd.crick.clicker.palette",".clkt":"application/vnd.crick.clicker.template",".clkw":"application/vnd.crick.clicker.wordbank",".wbs":"application/vnd.criticaltools.wbs+xml",".cryptonote":"application/vnd.rig.cryptonote",".cif":"chemical/x-cif",".cmdf":"chemical/x-cmdf",".cu":"application/cu-seeme",".cww":"application/prs.cww",".curl":"text/vnd.curl",".dcurl":"text/vnd.curl.dcurl",".mcurl":"text/vnd.curl.mcurl",".scurl":"text/vnd.curl.scurl",".car":"application/vnd.curl.car",".pcurl":"application/vnd.curl.pcurl",".cmp":"application/vnd.yellowriver-custom-menu",".dssc":"application/dssc+der",".xdssc":"application/dssc+xml",".deb":"application/x-debian-package",".uva":"audio/vnd.dece.audio",".uvi":"image/vnd.dece.graphic",".uvh":"video/vnd.dece.hd",".uvm":"video/vnd.dece.mobile",".uvu":"video/vnd.uvvu.mp4",".uvp":"video/vnd.dece.pd",".uvs":"video/vnd.dece.sd",".uvv":"video/vnd.dece.video",".dvi":"application/x-dvi",".seed":"application/vnd.fdsn.seed",".dtb":"application/x-dtbook+xml",".res":"application/x-dtbresource+xml",".ait":"application/vnd.dvb.ait",".svc":"application/vnd.dvb.service",".eol":"audio/vnd.digital-winds",".djvu":"image/vnd.djvu",".dtd":"application/xml-dtd",".mlp":"application/vnd.dolby.mlp",".wad":"application/x-doom",".dpg":"application/vnd.dpgraph",".dra":"audio/vnd.dra",".dfac":"application/vnd.dreamfactory",".dts":"audio/vnd.dts",".dtshd":"audio/vnd.dts.hd",".dwg":"image/vnd.dwg",".geo":"application/vnd.dynageo",".es":"application/ecmascript",".mag":"application/vnd.ecowin.chart",".mmr":"image/vnd.fujixerox.edmics-mmr",".rlc":"image/vnd.fujixerox.edmics-rlc",".exi":"application/exi",".mgz":"application/vnd.proteus.magazine",".epub":"application/epub+zip",".eml":"message/rfc822",".nml":"application/vnd.enliven",".xpr":"application/vnd.is-xpr",".xif":"image/vnd.xiff",".xfdl":"application/vnd.xfdl",".emma":"application/emma+xml",".ez2":"application/vnd.ezpix-album",".ez3":"application/vnd.ezpix-package",".fst":"image/vnd.fst",".fvt":"video/vnd.fvt",".fbs":"image/vnd.fastbidsheet",".fe_launch":"application/vnd.denovo.fcselayout-link",".f4v":"video/x-f4v",".flv":"video/x-flv",".fpx":"image/vnd.fpx",".npx":"image/vnd.net-fpx",".flx":"text/vnd.fmi.flexstor",".fli":"video/x-fli",".ftc":"application/vnd.fluxtime.clip",".fdf":"application/vnd.fdf",".f":"text/x-fortran",".mif":"application/vnd.mif",".fm":"application/vnd.framemaker",".fh":"image/x-freehand",".fsc":"application/vnd.fsc.weblaunch",".fnc":"application/vnd.frogans.fnc",".ltf":"application/vnd.frogans.ltf",".ddd":"application/vnd.fujixerox.ddd",".xdw":"application/vnd.fujixerox.docuworks",".xbd":"application/vnd.fujixerox.docuworks.binder",".oas":"application/vnd.fujitsu.oasys",".oa2":"application/vnd.fujitsu.oasys2",".oa3":"application/vnd.fujitsu.oasys3",".fg5":"application/vnd.fujitsu.oasysgp",".bh2":"application/vnd.fujitsu.oasysprs",".spl":"application/x-futuresplash",".fzs":"application/vnd.fuzzysheet",".g3":"image/g3fax",".gmx":"application/vnd.gmx",".gtw":"model/vnd.gtw",".txd":"application/vnd.genomatix.tuxedo",".ggb":"application/vnd.geogebra.file",".ggt":"application/vnd.geogebra.tool",".gdl":"model/vnd.gdl",".gex":"application/vnd.geometry-explorer",".gxt":"application/vnd.geonext",".g2w":"application/vnd.geoplan",".g3w":"application/vnd.geospace",".gsf":"application/x-font-ghostscript",".bdf":"application/x-font-bdf",".gtar":"application/x-gtar",".texinfo":"application/x-texinfo",".gnumeric":"application/x-gnumeric",".kml":"application/vnd.google-earth.kml+xml",".kmz":"application/vnd.google-earth.kmz",".gqf":"application/vnd.grafeq",".gif":"image/gif",".gv":"text/vnd.graphviz",".gac":"application/vnd.groove-account",".ghf":"application/vnd.groove-help",".gim":"application/vnd.groove-identity-message",".grv":"application/vnd.groove-injector",".gtm":"application/vnd.groove-tool-message",".tpl":"application/vnd.groove-tool-template",".vcg":"application/vnd.groove-vcard",".h261":"video/h261",".h263":"video/h263",".h264":"video/h264",".hpid":"application/vnd.hp-hpid",".hps":"application/vnd.hp-hps",".hdf":"application/x-hdf",".rip":"audio/vnd.rip",".hbci":"application/vnd.hbci",".jlt":"application/vnd.hp-jlyt",".pcl":"application/vnd.hp-pcl",".hpgl":"application/vnd.hp-hpgl",".hvs":"application/vnd.yamaha.hv-script",".hvd":"application/vnd.yamaha.hv-dic",".hvp":"application/vnd.yamaha.hv-voice",".sfd-hdstx":"application/vnd.hydrostatix.sof-data",".stk":"application/hyperstudio",".hal":"application/vnd.hal+xml",".html":"text/html",".irm":"application/vnd.ibm.rights-management",".sc":"application/vnd.ibm.secure-container",".ics":"text/calendar",".icc":"application/vnd.iccprofile",".ico":"image/x-icon",".igl":"application/vnd.igloader",".ief":"image/ief",".ivp":"application/vnd.immervision-ivp",".ivu":"application/vnd.immervision-ivu",".rif":"application/reginfo+xml",".3dml":"text/vnd.in3d.3dml",".spot":"text/vnd.in3d.spot",".igs":"model/iges",".i2g":"application/vnd.intergeo",".cdy":"application/vnd.cinderella",".xpw":"application/vnd.intercon.formnet",".fcs":"application/vnd.isac.fcs",".ipfix":"application/ipfix",".cer":"application/pkix-cert",".pki":"application/pkixcmp",".crl":"application/pkix-crl",".pkipath":"application/pkix-pkipath",".igm":"application/vnd.insors.igm",".rcprofile":"application/vnd.ipunplugged.rcprofile",".irp":"application/vnd.irepository.package+xml",".jad":"text/vnd.sun.j2me.app-descriptor",".jar":"application/java-archive",".class":"application/java-vm",".jnlp":"application/x-java-jnlp-file",".ser":"application/java-serialized-object",".java":"text/x-java-source,java",".js":"application/javascript",".json":"application/json",".joda":"application/vnd.joost.joda-archive",".jpm":"video/jpm",".pjpeg":"image/pjpeg",".jpgv":"video/jpeg",".ktz":"application/vnd.kahootz",".mmd":"application/vnd.chipnuts.karaoke-mmd",".karbon":"application/vnd.kde.karbon",".chrt":"application/vnd.kde.kchart",".kfo":"application/vnd.kde.kformula",".flw":"application/vnd.kde.kivio",".kon":"application/vnd.kde.kontour",".kpr":"application/vnd.kde.kpresenter",".ksp":"application/vnd.kde.kspread",".kwd":"application/vnd.kde.kword",".htke":"application/vnd.kenameaapp",".kia":"application/vnd.kidspiration",".kne":"application/vnd.kinar",".sse":"application/vnd.kodak-descriptor",".lasxml":"application/vnd.las.las+xml",".latex":"application/x-latex",".lbd":"application/vnd.llamagraphics.life-balance.desktop",".lbe":"application/vnd.llamagraphics.life-balance.exchange+xml",".jam":"application/vnd.jam",".123":"application/vnd.lotus-1-2-3",".apr":"application/vnd.lotus-approach",".pre":"application/vnd.lotus-freelance",".nsf":"application/vnd.lotus-notes",".org":"application/vnd.lotus-organizer",".scm":"application/vnd.lotus-screencam",".lwp":"application/vnd.lotus-wordpro",".lvp":"audio/vnd.lucent.voice",".m3u":"audio/x-mpegurl",".m4v":"video/x-m4v",".hqx":"application/mac-binhex40",".portpkg":"application/vnd.macports.portpkg",".mgp":"application/vnd.osgeo.mapguide.package",".mrc":"application/marc",".mrcx":"application/marcxml+xml",".mxf":"application/mxf",".nbp":"application/vnd.wolfram.player",".ma":"application/mathematica",".mathml":"application/mathml+xml",".mbox":"application/mbox",".mc1":"application/vnd.medcalcdata",".mscml":"application/mediaservercontrol+xml",".cdkey":"application/vnd.mediastation.cdkey",".mwf":"application/vnd.mfer",".mfm":"application/vnd.mfmp",".msh":"model/mesh",".mads":"application/mads+xml",".mets":"application/mets+xml",".mods":"application/mods+xml",".meta4":"application/metalink4+xml",".mcd":"application/vnd.mcd",".flo":"application/vnd.micrografx.flo",".igx":"application/vnd.micrografx.igx",".es3":"application/vnd.eszigno3+xml",".mdb":"application/x-msaccess",".asf":"video/x-ms-asf",".exe":"application/x-msdownload",".cil":"application/vnd.ms-artgalry",".cab":"application/vnd.ms-cab-compressed",".ims":"application/vnd.ms-ims",".application":"application/x-ms-application",".clp":"application/x-msclip",".mdi":"image/vnd.ms-modi",".eot":"application/vnd.ms-fontobject",".xls":"application/vnd.ms-excel",".xlam":"application/vnd.ms-excel.addin.macroenabled.12",".xlsb":"application/vnd.ms-excel.sheet.binary.macroenabled.12",".xltm":"application/vnd.ms-excel.template.macroenabled.12",".xlsm":"application/vnd.ms-excel.sheet.macroenabled.12",".chm":"application/vnd.ms-htmlhelp",".crd":"application/x-mscardfile",".lrm":"application/vnd.ms-lrm",".mvb":"application/x-msmediaview",".mny":"application/x-msmoney",".pptx":"application/vnd.openxmlformats-officedocument.presentationml.presentation",".sldx":"application/vnd.openxmlformats-officedocument.presentationml.slide",".ppsx":"application/vnd.openxmlformats-officedocument.presentationml.slideshow",".potx":"application/vnd.openxmlformats-officedocument.presentationml.template",".xlsx":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",".xltx":"application/vnd.openxmlformats-officedocument.spreadsheetml.template",".docx":"application/vnd.openxmlformats-officedocument.wordprocessingml.document",".dotx":"application/vnd.openxmlformats-officedocument.wordprocessingml.template",".obd":"application/x-msbinder",".thmx":"application/vnd.ms-officetheme",".onetoc":"application/onenote",".pya":"audio/vnd.ms-playready.media.pya",".pyv":"video/vnd.ms-playready.media.pyv",".ppt":"application/vnd.ms-powerpoint",".ppam":"application/vnd.ms-powerpoint.addin.macroenabled.12",".sldm":"application/vnd.ms-powerpoint.slide.macroenabled.12",".pptm":"application/vnd.ms-powerpoint.presentation.macroenabled.12",".ppsm":"application/vnd.ms-powerpoint.slideshow.macroenabled.12",".potm":"application/vnd.ms-powerpoint.template.macroenabled.12",".mpp":"application/vnd.ms-project",".pub":"application/x-mspublisher",".scd":"application/x-msschedule",".xap":"application/x-silverlight-app",".stl":"application/vnd.ms-pki.stl",".cat":"application/vnd.ms-pki.seccat",".vsd":"application/vnd.visio",".vsdx":"application/vnd.visio2013",".wm":"video/x-ms-wm",".wma":"audio/x-ms-wma",".wax":"audio/x-ms-wax",".wmx":"video/x-ms-wmx",".wmd":"application/x-ms-wmd",".wpl":"application/vnd.ms-wpl",".wmz":"application/x-ms-wmz",".wmv":"video/x-ms-wmv",".wvx":"video/x-ms-wvx",".wmf":"application/x-msmetafile",".trm":"application/x-msterminal",".doc":"application/msword",".docm":"application/vnd.ms-word.document.macroenabled.12",".dotm":"application/vnd.ms-word.template.macroenabled.12",".wri":"application/x-mswrite",".wps":"application/vnd.ms-works",".xbap":"application/x-ms-xbap",".xps":"application/vnd.ms-xpsdocument",".mid":"audio/midi",".mpy":"application/vnd.ibm.minipay",".afp":"application/vnd.ibm.modcap",".rms":"application/vnd.jcp.javame.midlet-rms",".tmo":"application/vnd.tmobile-livetv",".prc":"application/x-mobipocket-ebook",".mbk":"application/vnd.mobius.mbk",".dis":"application/vnd.mobius.dis",".plc":"application/vnd.mobius.plc",".mqy":"application/vnd.mobius.mqy",".msl":"application/vnd.mobius.msl",".txf":"application/vnd.mobius.txf",".daf":"application/vnd.mobius.daf",".fly":"text/vnd.fly",".mpc":"application/vnd.mophun.certificate",".mpn":"application/vnd.mophun.application",".mj2":"video/mj2",".mpga":"audio/mpeg",".mxu":"video/vnd.mpegurl",".mpeg":"video/mpeg",".m21":"application/mp21",".mp4a":"audio/mp4",".mp4":"application/mp4",".m3u8":"application/vnd.apple.mpegurl",".mus":"application/vnd.musician",".msty":"application/vnd.muvee.style",".mxml":"application/xv+xml",".ngdat":"application/vnd.nokia.n-gage.data",".n-gage":"application/vnd.nokia.n-gage.symbian.install",".ncx":"application/x-dtbncx+xml",".nc":"application/x-netcdf",".nlu":"application/vnd.neurolanguage.nlu",".dna":"application/vnd.dna",".nnd":"application/vnd.noblenet-directory",".nns":"application/vnd.noblenet-sealer",".nnw":"application/vnd.noblenet-web",".rpst":"application/vnd.nokia.radio-preset",".rpss":"application/vnd.nokia.radio-presets",".n3":"text/n3",".edm":"application/vnd.novadigm.edm",".edx":"application/vnd.novadigm.edx",".ext":"application/vnd.novadigm.ext",".gph":"application/vnd.flographit",".ecelp4800":"audio/vnd.nuera.ecelp4800",".ecelp7470":"audio/vnd.nuera.ecelp7470",".ecelp9600":"audio/vnd.nuera.ecelp9600",".oda":"application/oda",".ogx":"application/ogg",".oga":"audio/ogg",".ogv":"video/ogg",".dd2":"application/vnd.oma.dd2+xml",".oth":"application/vnd.oasis.opendocument.text-web",".opf":"application/oebps-package+xml",".qbo":"application/vnd.intu.qbo",".oxt":"application/vnd.openofficeorg.extension",".osf":"application/vnd.yamaha.openscoreformat",".weba":"audio/webm",".webm":"video/webm",".odc":"application/vnd.oasis.opendocument.chart",".otc":"application/vnd.oasis.opendocument.chart-template",".odb":"application/vnd.oasis.opendocument.database",".odf":"application/vnd.oasis.opendocument.formula",".odft":"application/vnd.oasis.opendocument.formula-template",".odg":"application/vnd.oasis.opendocument.graphics",".otg":"application/vnd.oasis.opendocument.graphics-template",".odi":"application/vnd.oasis.opendocument.image",".oti":"application/vnd.oasis.opendocument.image-template",".odp":"application/vnd.oasis.opendocument.presentation",".otp":"application/vnd.oasis.opendocument.presentation-template",".ods":"application/vnd.oasis.opendocument.spreadsheet",".ots":"application/vnd.oasis.opendocument.spreadsheet-template",".odt":"application/vnd.oasis.opendocument.text",".odm":"application/vnd.oasis.opendocument.text-master",".ott":"application/vnd.oasis.opendocument.text-template",".ktx":"image/ktx",".sxc":"application/vnd.sun.xml.calc",".stc":"application/vnd.sun.xml.calc.template",".sxd":"application/vnd.sun.xml.draw",".std":"application/vnd.sun.xml.draw.template",".sxi":"application/vnd.sun.xml.impress",".sti":"application/vnd.sun.xml.impress.template",".sxm":"application/vnd.sun.xml.math",".sxw":"application/vnd.sun.xml.writer",".sxg":"application/vnd.sun.xml.writer.global",".stw":"application/vnd.sun.xml.writer.template",".otf":"application/x-font-otf",".osfpvg":"application/vnd.yamaha.openscoreformat.osfpvg+xml",".dp":"application/vnd.osgi.dp",".pdb":"application/vnd.palm",".p":"text/x-pascal",".paw":"application/vnd.pawaafile",".pclxl":"application/vnd.hp-pclxl",".efif":"application/vnd.picsel",".pcx":"image/x-pcx",".psd":"image/vnd.adobe.photoshop",".prf":"application/pics-rules",".pic":"image/x-pict",".chat":"application/x-chat",".p10":"application/pkcs10",".p12":"application/x-pkcs12",".p7m":"application/pkcs7-mime",".p7s":"application/pkcs7-signature",".p7r":"application/x-pkcs7-certreqresp",".p7b":"application/x-pkcs7-certificates",".p8":"application/pkcs8",".plf":"application/vnd.pocketlearn",".pnm":"image/x-portable-anymap",".pbm":"image/x-portable-bitmap",".pcf":"application/x-font-pcf",".pfr":"application/font-tdpfr",".pgn":"application/x-chess-pgn",".pgm":"image/x-portable-graymap",".png":"image/x-png",".ppm":"image/x-portable-pixmap",".pskcxml":"application/pskc+xml",".pml":"application/vnd.ctc-posml",".ai":"application/postscript",".pfa":"application/x-font-type1",".pbd":"application/vnd.powerbuilder6",".pgp":"application/pgp-signature",".box":"application/vnd.previewsystems.box",".ptid":"application/vnd.pvi.ptid1",".pls":"application/pls+xml",".str":"application/vnd.pg.format",".ei6":"application/vnd.pg.osasli",".dsc":"text/prs.lines.tag",".psf":"application/x-font-linux-psf",".qps":"application/vnd.publishare-delta-tree",".wg":"application/vnd.pmi.widget",".qxd":"application/vnd.quark.quarkxpress",".esf":"application/vnd.epson.esf",".msf":"application/vnd.epson.msf",".ssf":"application/vnd.epson.ssf",".qam":"application/vnd.epson.quickanime",".qfx":"application/vnd.intu.qfx",".qt":"video/quicktime",".rar":"application/x-rar-compressed",".ram":"audio/x-pn-realaudio",".rmp":"audio/x-pn-realaudio-plugin",".rsd":"application/rsd+xml",".rm":"application/vnd.rn-realmedia",".bed":"application/vnd.realvnc.bed",".mxl":"application/vnd.recordare.musicxml",".musicxml":"application/vnd.recordare.musicxml+xml",".rnc":"application/relax-ng-compact-syntax",".rdz":"application/vnd.data-vision.rdz",".rdf":"application/rdf+xml",".rp9":"application/vnd.cloanto.rp9",".jisp":"application/vnd.jisp",".rtf":"application/rtf",".rtx":"text/richtext",".link66":"application/vnd.route66.link66+xml",".shf":"application/shf+xml",".st":"application/vnd.sailingtracker.track",".svg":"image/svg+xml",".sus":"application/vnd.sus-calendar",".sru":"application/sru+xml",".setpay":"application/set-payment-initiation",".setreg":"application/set-registration-initiation",".sema":"application/vnd.sema",".semd":"application/vnd.semd",".semf":"application/vnd.semf",".see":"application/vnd.seemail",".snf":"application/x-font-snf",".spq":"application/scvp-vp-request",".spp":"application/scvp-vp-response",".scq":"application/scvp-cv-request",".scs":"application/scvp-cv-response",".sdp":"application/sdp",".etx":"text/x-setext",".movie":"video/x-sgi-movie",".ifm":"application/vnd.shana.informed.formdata",".itp":"application/vnd.shana.informed.formtemplate",".iif":"application/vnd.shana.informed.interchange",".ipk":"application/vnd.shana.informed.package",".tfi":"application/thraud+xml",".shar":"application/x-shar",".rgb":"image/x-rgb",".slt":"application/vnd.epson.salt",".aso":"application/vnd.accpac.simply.aso",".imp":"application/vnd.accpac.simply.imp",".twd":"application/vnd.simtech-mindmapper",".csp":"application/vnd.commonspace",".saf":"application/vnd.yamaha.smaf-audio",".mmf":"application/vnd.smaf",".spf":"application/vnd.yamaha.smaf-phrase",".teacher":"application/vnd.smart.teacher",".svd":"application/vnd.svd",".rq":"application/sparql-query",".srx":"application/sparql-results+xml",".gram":"application/srgs",".grxml":"application/srgs+xml",".ssml":"application/ssml+xml",".skp":"application/vnd.koan",".sgml":"text/sgml",".sdc":"application/vnd.stardivision.calc",".sda":"application/vnd.stardivision.draw",".sdd":"application/vnd.stardivision.impress",".smf":"application/vnd.stardivision.math",".sdw":"application/vnd.stardivision.writer",".sgl":"application/vnd.stardivision.writer-global",".sm":"application/vnd.stepmania.stepchart",".sit":"application/x-stuffit",".sitx":"application/x-stuffitx",".sdkm":"application/vnd.solent.sdkm+xml",".xo":"application/vnd.olpc-sugar",".au":"audio/basic",".wqd":"application/vnd.wqd",".sis":"application/vnd.symbian.install",".smi":"application/smil+xml",".xsm":"application/vnd.syncml+xml",".bdm":"application/vnd.syncml.dm+wbxml",".xdm":"application/vnd.syncml.dm+xml",".sv4cpio":"application/x-sv4cpio",".sv4crc":"application/x-sv4crc",".sbml":"application/sbml+xml",".tsv":"text/tab-separated-values",".tiff":"image/tiff",".tao":"application/vnd.tao.intent-module-archive",".tar":"application/x-tar",".tcl":"application/x-tcl",".tex":"application/x-tex",".tfm":"application/x-tex-tfm",".tei":"application/tei+xml",".txt":"text/plain",".dxp":"application/vnd.spotfire.dxp",".sfs":"application/vnd.spotfire.sfs",".tsd":"application/timestamped-data",".tpt":"application/vnd.trid.tpt",".mxs":"application/vnd.triscape.mxs",".t":"text/troff",".tra":"application/vnd.trueapp",".ttf":"application/x-font-ttf",".ttl":"text/turtle",".umj":"application/vnd.umajin",".uoml":"application/vnd.uoml+xml",".unityweb":"application/vnd.unity",".ufd":"application/vnd.ufdl",".uri":"text/uri-list",".utz":"application/vnd.uiq.theme",".ustar":"application/x-ustar",".uu":"text/x-uuencode",".vcs":"text/x-vcalendar",".vcf":"text/x-vcard",".vcd":"application/x-cdlink",".vsf":"application/vnd.vsf",".wrl":"model/vrml",".vcx":"application/vnd.vcx",".mts":"model/vnd.mts",".vtu":"model/vnd.vtu",".vis":"application/vnd.visionary",".viv":"video/vnd.vivo",".ccxml":"application/ccxml+xml,",".vxml":"application/voicexml+xml",".src":"application/x-wais-source",".wbxml":"application/vnd.wap.wbxml",".wbmp":"image/vnd.wap.wbmp",".wav":"audio/x-wav",".davmount":"application/davmount+xml",".woff":"application/x-font-woff",".wspolicy":"application/wspolicy+xml",".webp":"image/webp",".wtb":"application/vnd.webturbo",".wgt":"application/widget",".hlp":"application/winhlp",".wml":"text/vnd.wap.wml",".wmls":"text/vnd.wap.wmlscript",".wmlsc":"application/vnd.wap.wmlscriptc",".wpd":"application/vnd.wordperfect",".stf":"application/vnd.wt.stf",".wsdl":"application/wsdl+xml",".xbm":"image/x-xbitmap",".xpm":"image/x-xpixmap",".xwd":"image/x-xwindowdump",".der":"application/x-x509-ca-cert",".fig":"application/x-xfig",".xhtml":"application/xhtml+xml",".xml":"application/rss+xml",".xdf":"application/xcap-diff+xml",".xenc":"application/xenc+xml",".xer":"application/patch-ops-error+xml",".rl":"application/resource-lists+xml",".rs":"application/rls-services+xml",".rld":"application/resource-lists-diff+xml",".xslt":"application/xslt+xml",".xop":"application/xop+xml",".xpi":"application/x-xpinstall",".xspf":"application/xspf+xml",".xul":"application/vnd.mozilla.xul+xml",".xyz":"chemical/x-xyz",".yaml":"text/yaml",".yang":"application/yang",".yin":"application/yin+xml",".zir":"application/vnd.zul",".zip":"application/zip",".zmm":"application/vnd.handheld-entertainment+xml",".zaz":"application/vnd.zzazz.deck+xml",".atom":"application/atom+xml",".jpeg":"image/jpeg",".jpg":"image/jpeg",".rss":"application/rss+xml"}
to a file on your server called mimeTypes.json (for example), read it with var mimes = require("mimeTypes.json"), then before res.end simply do
var extension = req.url.substring(
req.
url.lastIndexOf(".")
);
var type = mimes[extension];
if(type) {
req.setHeader("content-type", type);
}
P.S.
The JSON was parsed from the JavaScript console from the link above. In case of changes, simply run the following in your console at that page to get the new JSON:
var j =
Array.apply(
null,
document
.getElementsByTagName("table")[0].rows
).map(x => [
x.cells[2].innerText,
x.cells[1].innerText
]).slice(1)
.filter(x=>
!x[1].includes("x-citrix-jpeg")
)
j.forEach(x => {
x[0].indexOf(",") > -1 && (
x[0].split(",").forEach(y =>
j.push([y.trim(), x[1]])
)
)
})
j = Object.fromEntries(
j.filter(x=>
x[0].indexOf(",") === -1
)
);

I had problems using your handler function because convertURL and okURL functions where not defined. I modified the code a little and finished looking like this
function handler(req, res)
{
// /home/juan/Documentos/push-api-demo is the path of the root directory of the server
var url = '/home/juan/Documentos/push-api-demo' + req.url;
var file_exists = fs.existsSync(url);
if (file_exists)
{
fs.readFile(url, function(err, data)
{
if (err)
{
res.writeHead(404);
return res.end("File not found.");
}
res.setHeader("Content-Type", mime.lookup(url));
res.writeHead(200);
res.end(data);
});
}
else
{
res.writeHead(403);
return res.end("Forbidden.");
}
}

Node.js copy remote file to server

Right now I'm using this script in PHP. I pass it the image and size (large/medium/small) and if it's on my server it returns the link, otherwise it copies it from a remote server then returns the local link.
function getImage ($img, $size) {
if (#filesize("./images/".$size."/".$img.".jpg")) {
return './images/'.$size.'/'.$img.'.jpg';
} else {
copy('http://www.othersite.com/images/'.$size.'/'.$img.'.jpg', './images/'.$size.'/'.$img.'.jpg');
return './images/'.$size.'/'.$img.'.jpg';
}
}
It works fine, but I'm trying to do the same thing in Node.js and I can't seem to figure it out. The filesystem seems to be unable to interact with any remote servers so I'm wondering if I'm just messing something up, or if it can't be done natively and a module will be required.
Anyone know of a way in Node.js?

You should check out http.Client and http.ClientResponse. Using those you can make a request to the remote server and write out the response to a local file using fs.WriteStream.
Something like this:
var http = require('http');
var fs = require('fs');
var google = http.createClient(80, 'www.google.com');
var request = google.request('GET', '/',
{'host': 'www.google.com'});
request.end();
out = fs.createWriteStream('out');
request.on('response', function (response) {
response.setEncoding('utf8');
response.on('data', function (chunk) {
out.write(chunk);
});
});
I haven't tested that, and I'm not sure it'll work out of the box. But I hope it'll guide you to what you need.

To give a more updated version (as the most recent answer is 4 years old, and http.createClient is now deprecated), here is a solution using the request method:
var fs = require('fs');
var request = require('request');
function getImage (img, size, filesize) {
var imgPath = size + '/' + img + '.jpg';
if (filesize) {
return './images/' + imgPath;
} else {
request('http://www.othersite.com/images/' + imgPath).pipe(fs.createWriteStream('./images/' + imgPath))
return './images/' + imgPath;
}
}

If you can't use remote user's password for some reasons and need to use the identity key (RSA) for authentication, then programmatically executing the scp with child_process is good to go
const { exec } = require('child_process');
exec(`scp -i /path/to/key username#example.com:/remote/path/to/file /local/path`,
(error, stdout, stderr) => {
if (error) {
console.log(`There was an error ${error}`);
}
console.log(`The stdout is ${stdout}`);
console.log(`The stderr is ${stderr}`);
});