I'm developing server side app on Node.js and I thinking about config file usage. But when I google it.. well as for now I have more questions then answers.
In my case I have a-lot-of functions with arguments different types of
sensitivity not just only secret, tokens, keys. Most of this
argument as user-depended. For example, user that launch app use to
define (via CLI [promtly]): number of days, how much times something
will execute, when and where, etc
For example I just check this question and that article. And my question is simple:
Which one to use and why? (or where can I read more about it) Using only .env file, or only config.json or probably both like in article above? Is it OK to store all settings in package.json or not? (like in this question)
I'd be very pleased if more experienced JS devs tell me what should I use and why.
Even if this question is marked as answered feel free to tell me (and everyone on SO) about your experience with such problem. But please, remember, that I'm asking not about *How to do it* or about DotEnv usage (or any other npm module). It's all about patterns & practices.
Is it OK to store all settings in package.json or not?
For sure you could theoretically do that. But you want to configure your service, and you usually run the same package with different configurations. Therefore it really makes sense to have some kind of configuration file, then you can set up different configurations for your local testserver, your development server, your production server and so on.
.env file or config.json ?
That is really up to you. However the .env file seem to not support any nested data, which I think is really useful for configuration. I always used config jsons, and that worked quite well so far.
In my case I have a-lot-of functions with arguments different types of sensitivity not just only secret, tokens, keys.
For sensitive data inside of a production environment you should really use a secret store (e.g. Vault). For local testing
environments this is irrelevant though.
Most values are user-depended
Then either store them in a config file, or use a database.
Related
hey im not really familar with JavaScript or react.
So i hope i dont a too easy question:
i want to have a "one-page"-website, and want to change this page dynamically with ajax-request.
I have coded for example code for four visibility-levels (guest-user, normal user, moderator, administrator)
if you log in into my page and you are an admin, you get the JS-Code from all levels. For example in the json-response there is a list with URLs to the Javascriptcode destination.
If you log in as a normal user you should get only the normal-user js-code. The guest-user-js-code you already have; you got that at the time you entered the page.
So i guess the thing is clear, what i want.
But how i should implement this?
Are there some ready solutions out there?
https://reactjs.org/docs/code-splitting.html
maybe i have to adjust this here?
and maybe there are some good bundlers out there, that i can use, doing that splitting with hiding the endpoint urls (which i get if i have the rights from an ajax-request)?
lg knotenpunkt
As I said in the comments, I think that the question is very, very broad. Each one of the requests is a full standalone argument.
Generally speaking, I hope that this will led you to the right way.
You can split your code by using CommonJS or ES6 modules (read more here). That is to keep it "modular". Then, during the bundling process, other splitting techniques may be applied, but this will depend on your development environment and used tools.
Your best option for bundling would be Webpack without any doubt. However, directly dealing with Webpack or setting up a custom development environment is not an easy task. You'll certainly want to read about Create React App, which is a good place to start for a Single Page Application. It will allow you to write your code in a "modular" fashion and will bundle, split and process it automatically (uses Webpack under the hood).
Finally securing access must be done server-side (there is another world of available options there).
This is more of a conceptual question. I am trying to build an expressjs app in which when a user starts I would need to make a temp folder for each user to store some files. I am not sure how to handle this scenario.
I am not much experienced on server side development and this would be my first node/expressjs project.
I am sorry no code yet as I am still clueless how to approach the above problem. I looked into express-session and my current approach would be to create a session and use session id a the folder name.
can anybody guide me towards a more robust approach.
You would need to use Node file system:
https://nodejs.org/api/fs.html#fs_fs_mkdtemp_prefix_callback
There are also lots of npm packages for this you could use which might make it even simpler.
something like this: https://www.npmjs.com/package/tmp
Since you are using sessions, get the username out of the session and then use that to along with a timestamp to create a folder name that wouldn't run into any duplicate names.
I'm a Git newbie.
I have a BitBucket account, and I have Atlassian SourceTree installed on my Windows 7 PC.
Periodically, I ask our IRT team to put fresh code up at www.example.com/extrajs/myscript.js
I'm not interested in tracking any copies of the data on my own PC - I just want to have a button I can click that will copy whatever is currently at www.example.com/extrajs/myscript.js into my repository.
Is it possible to copy directly from the web to a Git repository? If so, how? (SourceTree-specific or BitBucket-web-UI-specific answers even better! I like to avoid the command line when possible.)
Thanks!
In the general case1, a file can't be pushed to a repo in isolation. It needs to be part of a tree, which needs to be part of a commit, which needs to be related to other commits in the repository. This requires a local copy of at least some parts of the repository, even if it's just in-memory.
If you don't want a persistent local copy, the tool would need to fetch some information from the repository each time you want to push an update, so it knows how to structure it. This could be done, but it's quite far from the workflow that most distributed version control systems are designed for. Unfortunately, this means that it's unlikely that any tools will provide functionality like this right out-of-the-box. I haven't heard of anything like this from GitHub or BitBucket.
It would probably be necessary for you to write a custom script to handle this. If you're worried about forgetting to clean it up (and accidentally pushing inconsistent/stale data), I suggest using an OS API to use a unique temporary folder each time you run the script, such as mktemp -d on Linux. You should also be aware of git clone's --shallow-clone option, which allows you to only clone the most recent commits of a repository, letting you make changes without requiring the entire history to be redownloaded.
1 Some Git servers are really just standard Linux machines which you're using a standard SSH connection to interact with, potentially with broad permissions. In cases like this, it may be possible to upload the file to the server using SSH/SFTP, and the commit it directly on the server. However, this type of configuration has become rare, and I would recommend against it.
One big benefit I've always perceived with using NodeJS on the server is the potential for sharing bits of code between the server and client side (ex. input validation). Now that I'm actually developing using NodeJS one difficulty that I've found is determining the responsibility and context in which each body of code is executed. Below I'll list a few of the difficulties I've had in hopes gain some enlightenment on conventions or guidance that I may be overlooking that could help elevate these issues.
Build-Time Code
Build time code for projects that use Gulp, Grunt, or vanilla NPM in a way that follow the basic documentation are generally pretty easy to follow. Most smaller projects tend to keep all of the code within a single file and the file tends to be named a conventional name like gulpfile.js, however with bigger projects I've seen these scripts begin to be split out. I've seen some cases where the gulp file is split into multiple files and placed under a separate directory. Even worse I've found cases where the gulpfile.js file isn't even named as such causing new developers to hunt around to find where the gulpfile is located and once it is located the gulp command always has to be run with the specific --gulpfile option.
Run-Time Server-Side Code
The entry point for basic node applications appear to simply require pointing out a specific JavaScript file when running the node command (ex. node script.js). For web server applications, such as those using Express, I've noticed that by convention the entry point file is often called server.js and can usually be found in the root directory of the application. In some other cases however such as when running the web server in a developer environment I've seen gulp tasks take on the responsibility of launching Node. In these cases there seems to be multiple ways to include the entry point but one example I've found is just starting up the webpack complier followed by a require statement for the entry point script. Figuring out how to incorporate normal guidance on how to accomplish a typical node debug command is non-trivial in this type of setup. Besides the entry point of the application, there doesn't seem to be any general guidance on directory structure for NodeJS/Express applications that keeps server-side specific code in it's place to help locate it and to keep it separate from build-time and client-side code.
The server-side story becomes even more complex in cases where the server side code is used both for the purpose for serving up static content, server-side generated views (such as with MVC), as well as for providing an API to the client side. My preference is to separate API from the application project but I get the feeling from others that there is a sense of overcomplexity involved in doing so where I see it as a reasonable separation of concerns.
Run-Time Client-Side Code
Since client-side code can often have various entry points based on the first page that is requested this can be tricky. However because of the general transparency of URLs with how they map to resources for typical cases it as well as how powerful the debugging tools have become in modern browsers, it isn't too much trouble following the follow of the scripts. The difficult instead for the client side code comes more for typical build processes which generally end up copying the files around and placing them into a production like structure under a different name. An example would be a project that has a folder called src or js that holds client-side and server-side code intermingled except for that only a portion of the files happen to be included in a build task which transforms and often concatenates the files and places them in a distribution folder. Common names of these distribution folders that I've seen are dist, public, www, and wwwroot. Often if not always these directories are at the root of the project which at least makes it a bit easier to locate without having to interrogate the build scripts.
My hope is that there is some general guidance on how to put all of this together in a sane way perhaps by an authoritative source mainly to give guidance to those like myself who may want start off on the right foot. As a side effect perhaps being able to reference some sort of standard even if it is a loose one may also reduce the amount of boilerplate a team has to invent and discuss as they get started. Within each of the contexts listed above there will obviously be some technology specific conventions such as those followed for AngularJS, Meteor, or ReactJS on the client-side. The conventions I'm looking for are more specific to separating the main highlevel contexts in end-to-end JavaScript applications where the language and platform no longer become obvious way to differentiate between each.
Build-Time Code
IMHO if you have so much build-time code that it's more than say 1000 lines and requires more than a handful of files, something has gone off the rails. Either you don't know how to make good use of existing packages from npm or you don't understand how to refactor generic code and publish as independent npm packages. If you feel like you need guidance about a project's build-time code because it is so large and complex, my suggestion is to modularize and split out into separate projects - each published independently to npm. Also just check your overall approach. What are you doing that is so custom and requires so much machinery?
Run-Time Server Side Code
Please see my other answer to ExpressJS How to structure an application?
Generally I'd rather see client side code and server side code either completely separate npm packages (separate git repos, separate package.json files, published independently) (if they are large enough) or otherwise co-mingled in the same module and grouped by coupling (all the code relating to a feature kept together including front and back end code), especially if your code base has a substantial amount of code that works in both environments.
I have an open-source full-stack node/JS application called mjournal that keeps browser code and node code alongside each other. You can have a look and see if it feels logical to you and easy to understand where code lives. It is by no means a popular approach, so many folks will dislike it, but it feels good to me personally since I've embraced "group by coupling" as a general principle.
Figuring out how to incorporate normal guidance on how to accomplish a typical node debug command is non-trivial in this type of setup
Yeah, that's nonsense. Your app should start with npm start or something like node server.js. Elaborate gulp/grunt setups that confuse new developers are unnecessary complexity you should just eliminate.
The server-side story becomes even more complex in cases where the server side code is used both for the purpose for serving up static content
In my experience, the code to serve up static content boils down to 5 lines or less, so no big deal. If you have a non-microscopic amount of code dealing with serving static content, again something has gone way off the rails.
Run-Time Client Side Code
My hope is that there is that there is some general guidance on how to put all of this together in a sane way perhaps by an authoritative source mainly to give guidance to those like myself who may want start off on the right foot.
There are some people in the node community that have adopted the "convention over configuration" approach in use by Ruby on Rails, EmberJS, and some other large projects. If you like that approach, check out tools that use it like SailsJS, EmberJS, Yeoman generators, etc.
But in general, looking for a "standard" is not how the node.js/javascript/web community rolls. Small npm packages. File layouts that are forced into obviousness due to smallness. I feel your frustration here as front-end toolchains are so complex, but ultimately it's because JavaScript in the browser took too many decades to create a reasonable module system. Things may start to standardize in the next few years now that ES6 modules are an official spec, but with so much code already written in CommonJS and it's terrible precursors like RequireJS/AMD, we'll be dealing with them probably for the foreseeable future.
I'm currently working on a new google polymer web application and wondered if I should use firebase as the backend/db. I took a look at the project, made some test applications and really liked it! But to fully convince me, that firebase is the way to go I need the following questions answered:
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
Is there an other way (like Java approach) to implement server business logic?
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
Thanks & kind regards
Marc
So, I asked the firebase supprt and got the following answer:
Great to meet you.
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
You can implement extremely complex and effective rules using our security rules language. You can deploy security rules as part of your hosting deploy process, or via the REST API. It's not possible to break the contents into multiple files on the server, but you could certainly build your own process for merging multiple files into a single JSON result.
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
Generally speaking, synchronizing Firebase and a SQL back end is not very practical and they don't translate well. It's probably entirely redundant as well.
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
I don't know what 100% supported means in this context. We offer a JavaScript SDK so they should play fine together.
Is there an other way (like Java approach) to implement server business logic?
We offer official SDKs in Java, Objective-C/Swift, Android, Node.js, JavaScript, and a REST API for use with other languages.
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
I hope that helps!
I responded:
Thank you for the information, it helped me very much! After reading your response on question number 5 one further question popped into my mind:
…
5. Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
Is there like a best practice on how to handle the database schema? I only have one web application (without apps, etc.) in my case... I expect, that the database will change drastically over time and releases. Should I write JS logic, that checks the current database version and update it, if it's necessary? Maybe this would make a nice feature...
For example: I deployed Version 1.0 of my application and everything works fine. After 3 months of programming I notice, that the user data needs a further attribute: address, which is a 'not null' attribute. I now deploy Version 2.0 of my application and every new registered user has a address, but the old users (from Version 1.0) do not have this field or a value.
How should I handle this?
Support responded:
Hi Marc,
There’s no best practice here, but your ideas seem fairly sound. You probably don’t need to check in your JavaScript. You can probably store a version number in the user’s profiles, and when they upgrade to the latest software, you can upgrade that in their profile data.
Then your validation rules could use something like the following:
{
"user": {
".write": "newData.hasChild('address') || newData.child('appVersion') < 4",
"address": {
".validate": "newData.isString() && newData.val().length < 1000"
}
}
}
So if you are concerned about versioning, this could be used to deal with legacy releases.
Another popular approach I’ve seen from devs is to do intermediate upgrades by duplicating data. Thus, you release an intermediate version that writes to the old path and to the new path with the updated data structure (which keeps the app working for old users till they upgrade). Once a reasonable percent of clients are upgraded, then release a final version that no longer does a dual write to the old structure and newer structure.
Of course, flattening data, while it makes joining and fetching data bit more of a pain, will make upgrades much easier as the modular data structure adapts more easily to changes. And, naturally, a pragmatic design where you wrap the various records in a class (e.g. the UserProfile class with getter/setter methods) makes transitions simpler as you can easily hack in versioning at one place.
Hope this helps someone :)