Develop Reference

The relationship between front end and middleware

I have a front end application, which I would like to return results with from an Express backend. Let's just call those results country and city for this reference.
I have done a bunch of searching, but I cannot find any solid resources on the relationship between the front end and middleware. Yes, I know what these things are, and the order in which they should flow, but the confusion sits with :
Do I need to connect my front end and middleware? How so?
If I am already connected to my backend from the front end, would I also have to connect to middleware?
How would I return the country and city from the middleware and/or express backend?
Any other info you think would be helpful for the greater dev community who is still learning would be beneficial.

While you could return data from a middleware, it's probably not what you are trying to do. A middleware is a piece of code that is executed between the time the request is receive by your backend, and the resource is fetch. In a middleware you could do things such as check if a user has access to a certain resource or authenticate a user by some sort of credential passed with the request.
Either way, the way you would, typically, do request from your front-end to your backend is via an XmlHttpRequest. Those request are usually Asynchronous, so they usage will not block the whole page while being executed. There are many ways you could create XmlHttpRequest. The native Javascript way is kinda ugly so I would suggest using the fetch api instead. You could also go with third party library if you need to do more complex stuff. I personnally like axios but this is up to you.
To give you a better understanding of what Express is doing, it's basically an infinite loop that waits for http request. You need to defined routes, that execute function that returns data.
Here is a basic example. Note that this script is executed via NodeJS :
// myserver.js
const express = require('express')
const app = express()
app.get('/cities', (req, res) => {
const cities = /** somehow get all the cities **/
res.json(cities);
})
/** the rest of the server... **/
/** For example, the route for Countries **/
In the previous example, we've built a basic server that listen to the url localhost:3000/cities and execute a function when this url is fetched. The said function will fetch all the cities and return them as JSON.
In your frontend, You would need to do a XmlHttpRequest that would call this url, to get the server to execute the function, which will return the data. Phew... I hope I did not lost you there.
A typical example would be a simple call using the fetch api.
Please note that this script is executed in the browser.
// myclient.js
async fetchAllCities() {
const cities = await fetch('http://localhost:3000/cities');
console.log(cities);
}
// just for fun, we add a click listener on a button and call the function defined above.
document.getElementById('myButton').addEventListener('click', async function() {
// we fetch the cities when we click on the button !
await fetchAllCities();
});
In the previous example, I am using the fetch function to call the url we declared in our Express server.
I'm also using Async / Await, which can be a little tricky, but it just mean Wait for the data to be there before going forward.
I highly suggest reading on the subject. Here are some references.
How do I return the response from an asynchronous call?
Understanding async/await on NodeJS.
Await from MDN
I hope this brief overview of XmlHttpRequest helped you get the base of how an API works.

Middleware is used to help the back-end do its job in processing incoming requests. It does not exist separate from the back-end. It's part of the back-end. For example, you might have middleware that checks to see if an incoming request is properly authorized/authenticated before the route can be handled by it's regular route handler.
Do I need to connect my front end and middleware? How so?
No. Your front-end sends requests to the back-end. The back-end may or may not use middleware to service the request. That's entirely up to the implementation in the back-end and what it needs to do for any given request.
If I am already connected to my backend from the front end, would I also have to connect to middleware?
No. You don't separately connect to middleware. You connect to your back-end and the back-end may or may not use middleware to do its job (something the front-end will have no knowledge of).
How would I return the country and city from the middleware and/or express backend?
You would have to show more details about what you're actually trying to return back from a request, but a common data format is JSON so you could construct a Javascript object with your desired response and then send it back to the client as the response from the incoming request using either res.json(someObj) or res.send(someObj) (both do the same thing if someObj is a Javascript object).
For example:
app.get("/getsomething", (req res) => {
// do some processing here to get cityResult and countryResult
// construct object to send back to client
const obj = { city: cityResult, country: countryResult};
// send this object as JSON back the the client as the response to this
// incoming request
res.json(obj);
});

calling a function itself in a controller sails js

I know this same question has been asked before but the previous questions were in different languages and on different frameworks, and I tried all the methods mentioned in previous questions' answers but found no luck.
so here it is again.
I have a controller in my sails js app named orderController
code is:
module.exports = {
order_history: async function(req, res){
//get a response from an API and it is saved in nData
if(nData.status_code == 0 && nData.result == 'succeed'){
res.ok(nData.data);
}else{
updateCredentials();
order_history(req, res);
}
},
};
now if there are expired credentials then it updates the credentials and I want it to repeat itself so it can return the proper response in case of expired credentials. I do not want another request to be sent for this.

My advice would be to create a helper with the logic for order_history that you call from within the controller. If it fails, you can do the update_credentials, then call it again. I say this because sails is built with the idea that a controller's purpose is to be a specific response to a specific request endpoint and not meant to be reusable code. That's where the helper comes in, it's purpose is to be a globally available function. The benefit of doing it this way is that you won't end up in a situation where the order-history fails for a reason other than needing to update credentials, calls itself, fails for that reason again, updates credentials, calls itself... You can have it try updating credentials, then if it fails again respond with an error message.

NodeJS http module: what is requestListener?

I'm new to JS and more specifically Node. Even after reading the api docs, I'm confused about what 'requestListener' is in the following method.
http.createServer([requestListener]);
Searching google revealed that 'requestListener' is a(n) (anonymous) function with the following signature:
function (request, response) { };
I suppose I'm reading the docs incorrectly, hopefully someone can point me in the right direction.

The docs say that the method call takes a function which will be called when a new request is received by your application. This function, as you correctly stated in your question, takes two arguments - a request and response objects.
You should inspect the contents of these objects to learn what information is available to you. Also, take a look at the API docs for request and response.
The function is optional; you could also attach the request handler in the following way:
var server = http.createServer()
server.on('request', function (req, res) {
// Process the request here
})
In practice, this function is called when someone opens up your website in their browser (i.e. issues a GET http request). The purpose of that function is to provide a HTTP response body back to the client, i.e. render a web page or perform any business logic as necessary.

To directly answer your question: it's a function that gets called when a request is received by the server and is given those two parameters.
At the very least you can experiment with doing a console.log(request, response) inside the function and see what is spit out in your terminal.
But that's only the beginning of the rabbit hole. You should read up on "callback functions", as they are integral to how Node (and quite a bit of client-side javascript) operates (asynchronously).

The http.createServer method creates a server object.
The server object has a listen method. If you invoke the listen method, for example:
createServer(requestListener).listen({ port:80 });
the server object will be listening on port 80, and when a Http request is received on that port, the server object will invoke the function requestListener, passing it two objects, of type request and response. So you can write your requestListener like, for example:
function requestListener(req, res) {
res.write("Hello world");
res.end();
}
and it will push out the string Hello world to every url that is hitting your this simple web server.
write is one of the many methods of the response object.
You can run the above few lines of code on your PC, and point your browser to http://localhost.
If you have other applications listening on port 80, then use a different port number in your listen method.
Using an anonymous function for requestListener is merely a different coding pattern, for better or for worse. My code above can be re-written as:
createServer((req, res) => {
res.write("Hello world");
res.end();
}).listen({ port:80 });
The above code is very rudimentary and will send the same response to every Http request that hits it. The code does not differentiate between localhost/page1 or localhost/page2, etc. So to do anything more, the requestListener have to be expanded significantly to parse different paths in the url and to decide on what content to send for each. But to do anything useful as a real web server without writing too much code, you will need a package. Express is an excellent one that drives many real life web servers.

Make Parse Data Modification Only Accessible from the Cloud

I am trying to create an app on Parse. That app uses data, but in order to make the data storage more secure, you would not like the clients to be able to run it. Instead, only the server should be able to modify data,
So far, I haven't seen any options as to how to achieve that, except by using user/role-based authentication, and that is something I'd rather avoid because it is the environment, not the user or role I would like to make the data access depend on.
Are there any ways to do that?

Turn off write access for everyone on each class, then in your Cloud Code use the master key which lets you bypass permissions.

You could use a beforeSave handler in Cloud Code...
Parse.Cloud.beforeSave('myClassName', function(req, res) {
if (req.master) {
res.success();
} else {
res.error('Cannot change this data.');
}
}
Then only requests made using the Master Key can alter this data.
In other places in Cloud Code, you can pass this option for individual requests like this:
obj.save(null, { useMasterKey: true });
Or turn it on for actions that follow:
Parse.Cloud.useMasterKey();

Understanding how to use NodeJS to create a simple backend

I have been trying to develop a rather simple server in nodejs. Basically, what I am going for is a simple API that requires authentication (simple username/password style). What I do not need is any kind of frontend functionality (templating etc.). My problem is, I can't seem to get my head around the approach of express/node.
Specifically, my questions are:
How do I wire in the authentication? Do I pass several handlers into every route that requires authentication, or is there a more elegant way to do this?
How does the Express middleware (like app.use(express.bodyParser())) work? Do they alter contents of the request or response object? Specifically, if I use the body parser (internally formidable?), where do I access the request data this is supposed to parse?
When using authentication and I have, say, credentials stored in a database with more information about the individual client associated, at what point do I extract that information? I.e., when a user logs in, do I fetch the user record on login and pass it on, or do I fetch it in every handler that requires the information?
Ultimately, do you know of an open source application that I could take a look at? I'd like to see something that has simple authentication and maybe even utilizes formidable, since uploading a file is one of my requirements.
As I mentioned earlier, I believe my problem is ultimately a difficulty with the function-oriented approach in node (also, I have rather limited experience in webservice programming). If you know a resource where I could read up on how to approach architecting a nodejs app, please don't hesitate to point me to it.

How do I wire in the authentication? Do I pass several handlers into
every route that requires authentication, or is there a more elegant
way to do this?
You should use the session middleware. Here is some pseudo code:
var http = require('http');
var app = express();
var authorize = function(req, res, next) {
if(req.session && req.session.appname && req.session.appname === true) {
// redirect to login page
return;
}
next();
}
app.use(express.session());
app.all('/admin*', authorize, function(req, res, next) {
});
How does the Express middleware (like app.use(express.bodyParser()))
work? Do they alter contents of the request or response object?
Specifically, if I use the body parser (internally formidable?), where
do I access the request data this is supposed to parse?
Every middleware have an access to the request and response object. So, yes, it modifies it. Normally attach properties to it. This means that inside your handler (which is also a middleware) you may write:
if(req.body && req.body.formsubmitted && req.body.formsubmitted === 'yes') {
var data = {
title: req.body.title,
text: req.body.text,
type: req.body.type
}
// store the data
}
When using authentication and I have, say, credentials stored in a
database with more information about the individual client associated,
at what point do I extract that information? I.e., when a user logs
in, do I fetch the user record on login and pass it on, or do I fetch
it in every handler that requires the information?
I think that you should do the things the same way as in any other server side language. Keep the state of the user (logged/not-logged) inside a session. You may also keep the user's id and fetch the data for him whatever you need. It depends of your case, but you have the ability to cache information. Because node is not like PHP for example, I mean it's not dieing.
Ultimately, do you know of an open source application that I could
take a look at? I'd like to see something that has simple
authentication and maybe even utilizes formidable, since uploading a
file is one of my requirements.
Yep. I wrote an article about really simple MVC web site with admin panel. It is available here. And the code of it is here.

A simple way to implement authentication (if you don't want to use additional modules):
var checkAuth = function(req, res, next) {
if(!req.session.user)
{
// Redirect to login form
res.redirect("/login");
}
else
{
// Proceed to member's area
next();
}
};
app.get("/member/page", checkAuth, function(req, res) {
// render view, etc
});
bodyParser parses / converts the body of a POST request into an object, which helps with getting form submission values.
The route that handles your login form submission can access username / password like this:
var username = req.body.username;
var password = req.body.password;
At this point you'd query your database to select from users where the username and password matches (you'd want to use password encryption in a production environment).
If you get a record back in the query result, set it in the session. A simple way to do this is:
req.session.user = userRecord
(Adjust for your session middleware)

If you are looking for REST, I recommend using either Restify or booster
For authentication (distinct from authorization), use standard Basic, which can be handled by express.basicAuth() just to parse it and place it on the req object. Personally, I don't like basicAuth because it returns a 401 if there is no login, whereas the process of authenticating is different than determining if authentication is necessary.
For more advanced authentication, as well as session management, use cansecurity or passport. For authorization, you either can put individual middleware in each route, use cansecurity's middlewares, or use its declarative authorization.
Disclosure: I am the author of both booster and cansecurity.

If your goal is to build a RESTful API in Node.js, my best bet would be Restify, which uses a similar aproach of routes like Express, but eliminates all the high level stuff(templating, etc.) and ads backend functionalities(ie: body parser, ip blacklist, requests per hour).
For the authentication part, I would use another library perhaps, and wire it to a particular route. There are ORM's too that can solve your database needs(mongo and mysql are well supported, both for the "noSQL" fans and the classic db aproach ones).