I'm building my first Express app, which needs to interact with an API, using an API key that ideally remains secure.
So I wanted to follow a basic pattern of keeping the key (and any future environment variables), in a .gitignored .env file in the root directory.
To not reinvent the wheel, I used this package, and set my env variables like so, in my app.coffee file (the root file of the application):
env = require('node-env-file')
env __dirname + '/.env'
console.log process.env.MY_API_KEY
That console.log prints out the right key to the server logs. The problem arises later:
If I try to access that same variable in one of the JS files loaded later on by my app, process.env is an empty object, so the API key is undefined. This doesn't appear to be a problem with the above package, because if I define the variable in the CL (API_KEY=whatever npm start), the behavior is the same -- it console logs correctly from app.coffee but is unavailable later.
Some information on how the files in which the key is unavailable are being loaded:
The app is running React, which I write to a few .jsx files in public/javascripts/src, and which are compiled by gulp into public/javascripts/build/*.js.
I'm trying to access the key in a .js file in public/javascripts/ which is required by one of the .jsx files.
In that required .js file, process.env returns an empty object. When I try to access process.env in the .jsx files, I'm actually told that process itself is undefined.
Any ideas what's going on here? I'm new to Express/React, and unclear where this process object, which I thought was global and defined on npm start is defined, and what's happening to all the env info in it.
Thanks! Please let me know if any other information would be helpful, orif anyone has any suggestions for how better to handle private env info in my situation.
EDIT:
I tried the suggestions below, and created a separate endpoint internally, which hits the external API and then returns a response. I've strung things up correctly, so that this responds correctly:
router.get '/images', (req, res, next) ->
res.json({ some: 'json' });
but this (which uses a separate class to make a request to an external API), throws an error:
router.get '/images', (req, res, next) ->
new Images('nature').fetch (images) ->
res.json({ some: 'json' })
Essentially, it looks like the asynchrony of the response from the external API (and not even the data itself, which I ignored), is creating a problem. How do I hit this external endpoint and then respond to the internal request with the incoming data?
Back-end vs Front-end
It seems like you are trying to access back-end data from a front-end location, in a wrong way.
The great power of Node.js is having JavaScript in the front and in the back, but it is quite confusing in the beginning to understand on which side each script is executed.
In an Express project, all Javascript files that are sent to the front-end, those that will directly interact with the client's page, are located in public/javascripts/. Generally you will have some AJAX functions in some of those files to exchange data and communicate with the back-end.
These back-end files are located everywhere else : in the root directory, in routes/, and all the other folders you create. Those files are pretty much all connected to your Node instance, and therefore can communicate with each other using global objects like process for example.
Your script in public/javascripts/, that is executed on the client's computer, is trying to directly access a variable located on the server running your Node instance : that's why your code doesn't work. If you wish to access data from the back-end, you must use AJAX calls in the front-end.
Server <---(AJAX only)--- Client
------ ------
app.js public/javascripts/script.js
routes.js
...
That being said, you wanted to keep your API key private, which will not happen if you send it to every client who's on that specific page. What you should do is make the call from the back-end, using the xhr module for example, and then delivering the data to front-end, without the secret API key.
I hope I was clear, Node is quite confusing at first but very soon you will get over these little mistakes !
All .jsx is, is some code, what matters is where the code is being executed. process.env is a variable that is accessible inside the Node.js runtime. When your .jsx code gets transpiled down to .js and served to the browser, the process.env variable will no longer exist. If you're making an API call inside the browser, the API key will be fundamentally available to the client. If you want to secure the key, you have to have your Node.js server expose an API route, which your React app will hit. That Node.js server will then make the call to the external service using the API key. Because that call is being made by the server, process.env will be available, and will remain hidden from the client. You can then forward the result of the API call back to the user.
Related
I'm trying to develop a desktop app which would need to make a few private API calls, authenticated using some secret keys.
The keys are created for me by external IT service providers outside of my organisation - they are responsible for the security so there are a few constraints:
They said even though they have already taken steps on their end to secure the API and there are mitigation strategies in place even if a breach happens, but still they would like to make sure that I treat the keys with a security-conscious mindset and take whatever steps possible on my end as well to make sure they remain secured.
I'm not allowed to just create random middleware / gateway on a private server or serverless platform to perform the API calls on my app's behalf as these calls may contain business data.
I have done some research and from what I can find, the general recommendation is to set up a ".env" file in the project folder and use environment variables in that file to store the API keys.
But upon reading the Vue CLI documentation I found the following:
WARNING
Do not store any secrets (such as private API keys) in your app!
Environment variables are embedded into the build, meaning anyone can
view them by inspecting your app's files.
So, given the constraints, is there a way to store these keys securely in a Vue CLI 4 + Electron Desktop app project?
Thanks.
In general, especially if you have a lot of environment variables, it would be better practice to store environment variables in a dot env file (.env), however, it's possible that this file could be leaked when you package your electron app. So, in this case it would be better to store your environment variables from the terminal/command line. To do so follow this guide (https://www.electronjs.org/docs/api/environment-variables).
Keep in mind anything that requires the API key/private information try to keep it on the backend, i.e., the electron process and send the results to the Vue front end.
Here's an example of how you could implement this:
On windows from CMD:
set SOME_SECRET="a cool secret"
On POSIX:
$ export SOME_SECRET="a cool secret"
Main process:
// Other electron logic
const { ipcMain } = require("electron");
// Listen for an event sent from the client to do something with the secret
ipcMain.on("doSomethingOnTheBackend", (event, data) => {
API.post("https://example.com/some/api/endpoint", {token: process.env.SOME_SECRET, data});
});
Client side:
const { ipcRenderer } = require("electron");
ipcRenderer.send("doSomethingOnTheBackend", {username: "test", password: "some password"});
Also note, to use the ipcRenderer on the client side nodeIntegration needs to be enabled.
Here are some more resources to help you get started:
https://www.electronjs.org/docs/api/ipc-renderer
https://www.electronjs.org/docs/api/ipc-main
I have following Node.js app structure:
- main.js - main file
- ./requests/requestSetName.js - different functions sending different requests to API, so I have e.g. userRequests.js, vechicleRequests.js etc.
First thing I do in main.js is I send a request with login credentials to get access token. For all other requests I need to put this token in the request header. So each function in all modules in request folder needs to have access to it.
What is the best way to store this token and to be able to use it across different modules in Node.js ?
I think you shoud use classes for each request file and inject another class (Globals) witch contains the token , in the constructor of each request class, typescript should make It clean.
Let me preface by saying that I have spent a considerable amount of time trying to figure out the solution to this problem but I have not discovered something that works. I am using node and want to share a variable between my app.js server file and a client side javascript file (demo.js).
I run node app.js to launch the server and demo.js runs in the client. I have tried using module.exports and export but when I try importing in the demo.js file or referring to the module.exports var I get errors. Maybe I'm approaching this is in the wrong way.
For example, I am trying to use the node wikipedia package to scrape data. I have the following in my app.js file:
var wikipedia = require('node-wikipedia');
wikipedia.page.data('Clifford_Brown', { content: true }, function(response) {
console.log(response);
export const response = response;
module.exports.data = response
});
In my demo.js file I have tried importing this response var and using the module.exports var but I have been unsuccessful.
Anyone have any solutions to this issue or different approaches I should take?
Browser javascript files run in the browser. node.js javascript files run on the server. You cannot directly export things from one to the other. They are on completely different computers in different locations.
It is very important for developers to understand the notion that server-side code runs on the server and client-side code runs on the browser. The two cannot directly call each other or reach the other's variables. Imagine your server is in a data center in Seattle and the browser is running on a computer in Venice.
See How to access session variables in the browser for your various choices described for a previous answer.
In a nutshell, you can have the server insert a javascript variable into the generated web page so that when the javascript runs in the web page on the browser, it can then access that variable in its own page. Or, you can create an Ajax call so the client can request data directly from the server. Or you can have the server put some data in a cookie which the Javascript in the browser can then access.
If the data is easily known by the server at the time the page is generated and you are using some sort of page template system, then it is very easy to just add a <script> tag to the generated page that defines one or more Javascript variables that contain the desired information. Then, the client-side Javascript can just refer to those variables to have access to the data.
To pass data in http there is a request message and response message and the data needs to be inside that message.
In the request you can either pass variables in the request URL
http://host_name/path?key=value
Or inside the request body or headers.
In the response you pass back variables in the response header or response body
First Example:
One way of processing a URL request from the browser explicitly while passing variables is to set up your server to render a html page with those variables embedded.
If you use a templating engine like jade, you can consume the sent variables directly into the template using res.render({ key: 'value' }) rather than using a promise based api call which would run when the user performs some action on the client.
For instance.
// SERVER setup rendering engine
app.get('/', function(req, res) {
res.render( 'index', { key: 'value' })
}
Which will render index.html to the client with the key-value pair passed to the template file used to serve up the index.html (for example jade or ejs).
Second Example:
Using axios you can set up an action to call a server api (you can also pass variables in the URL, headers or body). Using the promise pattern you can then use these variables after the server api has responded.
// CLIENT setup axios
axios.get(URL + '/getkeyvalue')
.then(function(response) {
const value = response.data.key
})
On you server using express you (this is where you would get the optional request variables mentioned above) send back response variables in the body like this.
// SERVER setup express
app.get('/getkeyvalue', function(req, res) {
res.send({ key: 'value' })
}
Note that these are simple examples.
They are too completely different systems. The best way to accomplish what you're trying to do is the create a variable in your html on the server side by stringifying your data
<script> var my_data = <%= JSON.stringify(data) %> </script>
Thats an example using ejs, a common templating language in expressjs
Can I access the inbuilt navigator functions like isinNet() or DomainNameorHost() from nodejs?
Since nodeJS runs on the server, not the browser, you can't access functions that are only provided in a browser.
Most developers use a middleware like Express to create a web service on nodejs.
In a route, such as
app.route("/play", function(req,res){
// code that handles URL /play
});
there is a callback function that is called when a request arrives for that route.
The req object parameter contains everything about the request.
req.ip is the upstream (incoming) ip address.
I looked around in npm for a module that might map remote ips to hostnames and could not find one. Presumably all it would do is reverseDNS, which could take time and hold up processing requests.
I have a to access the config variables defined in the file called
test.js which has --
var aws = require('aws-sdk');
exports.connect = function(){
return aws;
}
Now I need to access it when the OnClick event occurs on the browser. I have this script but the require module does not work.
clientScript.js
var aws = require('../scripts/test.js').connect();
function getValue() {
aws.describe({},function(){...})
}
How can I access this aws variable?
Hopefully I'm not too far off the mark with what you're trying to do here. My understanding (cobbled together between this and your previous question is that you would like something in the browser that upon click will retrieve some status information from an external API, which will then be displayed in the client.
What I would recommend doing (based on the above assumption) is defining your desired function as something to be triggered by an HTTP request to the Express server, which can perform your function and send whatever you'd like from its process back to the client.
In your server define (assuming your Express variable is app)
app.get('/request', someFunction);
In someFunction define what it is you'd like to do, how it relates to the request and response, and what to send back to the client. Express will expect the function to take request and response as arguments, but you don't necessarily need to use them:
someFunction(req,res) {
//do whatever I'd like with aws or anything else
res.send(foo); //where foo is whatever JSON or text or anything else I'd like the client to have to manipulate
}
On the client, you would have a function bound to onclick that would make the request to that /request endpoint. This function could use AJAX or simply render another page entirely.
This sort of organization also leaves display and behavior to the client, while the server deals with retrieving and manipulating data. This layout also resolves any concerns about require() statements on the clientside (which, while possible with things like Browserify, are not necessary and may make the code more confusing).
You can't use require() on the client side, that is only a server side function provided by node.js which runs on the server. If you need config options that are shared between server and client, then you will need to make a few changes to your test.js file so it will work in both. I'm sure there are a number of ways to do this, but the way I prefer is:
Put all your configuration variables inside test.js into an object like:
this.ConfigOptions = {option1:value1, option2:value2};
The client would include the file like this:
<script src="test.js" type="text/javascript"></script>
and can access the config options via the ConfigOptions object
while the server would use require() to include the file and access the config options like this:
var ConfigOptions = require('test.js').ConfigOptions;