Develop Reference

The relationship between front end and middleware

I have a front end application, which I would like to return results with from an Express backend. Let's just call those results country and city for this reference.
I have done a bunch of searching, but I cannot find any solid resources on the relationship between the front end and middleware. Yes, I know what these things are, and the order in which they should flow, but the confusion sits with :
Do I need to connect my front end and middleware? How so?
If I am already connected to my backend from the front end, would I also have to connect to middleware?
How would I return the country and city from the middleware and/or express backend?
Any other info you think would be helpful for the greater dev community who is still learning would be beneficial.

While you could return data from a middleware, it's probably not what you are trying to do. A middleware is a piece of code that is executed between the time the request is receive by your backend, and the resource is fetch. In a middleware you could do things such as check if a user has access to a certain resource or authenticate a user by some sort of credential passed with the request.
Either way, the way you would, typically, do request from your front-end to your backend is via an XmlHttpRequest. Those request are usually Asynchronous, so they usage will not block the whole page while being executed. There are many ways you could create XmlHttpRequest. The native Javascript way is kinda ugly so I would suggest using the fetch api instead. You could also go with third party library if you need to do more complex stuff. I personnally like axios but this is up to you.
To give you a better understanding of what Express is doing, it's basically an infinite loop that waits for http request. You need to defined routes, that execute function that returns data.
Here is a basic example. Note that this script is executed via NodeJS :
// myserver.js
const express = require('express')
const app = express()
app.get('/cities', (req, res) => {
const cities = /** somehow get all the cities **/
res.json(cities);
})
/** the rest of the server... **/
/** For example, the route for Countries **/
In the previous example, we've built a basic server that listen to the url localhost:3000/cities and execute a function when this url is fetched. The said function will fetch all the cities and return them as JSON.
In your frontend, You would need to do a XmlHttpRequest that would call this url, to get the server to execute the function, which will return the data. Phew... I hope I did not lost you there.
A typical example would be a simple call using the fetch api.
Please note that this script is executed in the browser.
// myclient.js
async fetchAllCities() {
const cities = await fetch('http://localhost:3000/cities');
console.log(cities);
}
// just for fun, we add a click listener on a button and call the function defined above.
document.getElementById('myButton').addEventListener('click', async function() {
// we fetch the cities when we click on the button !
await fetchAllCities();
});
In the previous example, I am using the fetch function to call the url we declared in our Express server.
I'm also using Async / Await, which can be a little tricky, but it just mean Wait for the data to be there before going forward.
I highly suggest reading on the subject. Here are some references.
How do I return the response from an asynchronous call?
Understanding async/await on NodeJS.
Await from MDN
I hope this brief overview of XmlHttpRequest helped you get the base of how an API works.

Middleware is used to help the back-end do its job in processing incoming requests. It does not exist separate from the back-end. It's part of the back-end. For example, you might have middleware that checks to see if an incoming request is properly authorized/authenticated before the route can be handled by it's regular route handler.
Do I need to connect my front end and middleware? How so?
No. Your front-end sends requests to the back-end. The back-end may or may not use middleware to service the request. That's entirely up to the implementation in the back-end and what it needs to do for any given request.
If I am already connected to my backend from the front end, would I also have to connect to middleware?
No. You don't separately connect to middleware. You connect to your back-end and the back-end may or may not use middleware to do its job (something the front-end will have no knowledge of).
How would I return the country and city from the middleware and/or express backend?
You would have to show more details about what you're actually trying to return back from a request, but a common data format is JSON so you could construct a Javascript object with your desired response and then send it back to the client as the response from the incoming request using either res.json(someObj) or res.send(someObj) (both do the same thing if someObj is a Javascript object).
For example:
app.get("/getsomething", (req res) => {
// do some processing here to get cityResult and countryResult
// construct object to send back to client
const obj = { city: cityResult, country: countryResult};
// send this object as JSON back the the client as the response to this
// incoming request
res.json(obj);
});

Pass Twilio SMS Post Data from Node to React

I'm currently using Twilio to receive SMS messages on my server. I need to display the return data in React. Twilio only sends the data server side through a POST request. If I send a text from my phone. The Twilio POST request will be send the data to my server. How can I get the POST data sent to my React app? The below code is how I receive the data from Twilio.
app.post('/sms', (req, res) => {
var msgFrom = req.body.From;
var msgBody = req.body.Body;})

yes you can and there are two ways:
1- you can use the the HTTP regular request model, you can achieve this by sending a request checking for updates and if there is what and update the front end.
2- long polling or streams sockets to emit the data to your applications from the back end.

Websocket on Reddit Endpoint to detect new post

I have a JavaScript script to check if there is a new post in a certain subreddit. Reddit magicaly provides a JSON endpoint on every link. In this case I have the following endpoint:
https://www.reddit.com/r/webdev/new.json?limit=1
I then used request module for node.js to get the specefic data I need eg domain, selftext, author and domain. However this changes every single time a new post is published and I therefore use a setInterval function to check every few seconds if a new post is released (Its like some sort of polling). I am saving every posts unique id to mongo to prevent double posting since that would assume every post is new on every request.
So the structure is something like:
setInterval(function () {
request({
uri: redditEndpoint,
json: true
}, function (error, response, body) {}) });
// I then save the data sent to mongo as unique to prevent double posting
// I consume the data here
}, 1000);
I would like to move away from this method and move over to websockets. However I am not sure how to correctly implement a websocket on such an endpoint, preferably with socket.io

That websocket endpoint would need to be provided by the server - in this case from the reddit server. You can't create a websocket connection to a random server which does not offer websocket support.
The only thing you could do is build a proxy server, which polls the reddit server for new events (like what you are currently doing) and which then offers a websocket endpoint itself for other clients.

Persisting a security token between calls

We are creating a prototype application as follows:
We have a html web site using knockoutjs
Using qQuery/Ajax it communicates with Web Api services
We only want the services to be accessed by authorised users. So we have written in security that can validate the user based on username/password
So next I guess we need to pass back some type of token to the client which it uses in further communications with the API services.
What I would like to know is how this is stored on the client so it can be passed back to the server again for the next call?

I assume the client makes an initial call passing in the user name and password over HTTPS and gets back a token. You question is to how to store the token? I assume your application is an SPA. If so, why not just store it in a JavaScript variable? If you do not use a cookie, you avoid XSRF. Of course, you must ensure the user name and password are never stored in the client side and that the life time token of your token is finite and preferably as small as possible.
EDIT:
If you can regenerate the token with every page (since it is not SPA), it is even better and you make the life time of token very small. You can use code like this. I use Authorization header and bearer scheme but you can use your own scheme as well, if no standardization is needed.
var accessToken = ''; // Write out the token into this variable in the server side (view)
$.ajax({
type: 'GET',
url: 'http://whatever',
dataType: 'json',
contentType: 'application/json; charset=utf-8',
headers: { 'Authorization=': ='Bearer ' + accessToken },
success: function (data) {
}
});

So we have written in security that can validate the user based on username/password
This sentence basically means that you need to store the username and password in your javascript file in order to be able to call the service, unless of course you want to ask the user to enter his credentials on every single action he performs. I hope this is not something you are wiling to do at the moment. If it is then you can stop reading my answer and store the username and password in your javascript file.
At this stage it is more than clear that your security should be handled differently. Your Web API should not be protected by a username and password but by a token. Here's how this could work in practice. You will have an action that will take the username and password, validate them and if successful it will return a token. This token could contain the encrypted username. So your javascript will ask the user for his username and password, call the Login method and it could store the token. Then it will use this token on subsequent calls. The API will in turn decrypt it in order to extract the username.

What I would like to know is how this is stored on the client so it can be passed back to the server again for the next call?
Cookies. You will send token as a cookie, and it will be sent automatically when user requests your page.

create a server side session, for the once authorised md5(username) md5(password).
generate an uuid per request, and return it in the response.
basic model is called token exchange and it is reliable (no m.i.t.m) even w/o SSL.

Pass query params from node.js server to socket.io

I'm writing a node.js app, that can generate url's like this:
http://examle.com/?param1=val&param2=val
I wonder if users will follow this urls to my app - is this possible to get params from this url and pass it to socket.io socket object that represents connection with user that was brought to site by that url.
I.E.:
user followed this url;
node.js express server handles that url and get the query params from it;
Now I want to pass these params back to user but with socket.io emit(), not to whole sockets but only to user that was brought by url.
Is this goal achivable?
Should I use some id that will be passed with url and along socket.io handshake process and than store socket in array with id as key, to get this socket later in get request of express server?