Develop Reference

Is an API and Router two separate things?

I have a Node.js with Express.js app. I have folders like such:
src
/models
/router
/store
/api/v1
index.js
My router folder has a file called index.router.js which contains my app's routes e.g:
import UserAPI from '../api/v1/user.api.js'
expressRouter.post('/register', function async function (req, res) {
await UserAPI.registerUser({
EMail: req.body.EMail,
Password: req.body.Password,
Name: req.body.Name
});
The above is a route to an API so it went into my index.router.js file. To perform an API action on this endpoint, I created another file for API functionality called user.api.js and it would contain something like:
async function registerUser({EMail, Password, Name}) {
// Import method from model file and perform DB action
});
As the application has grown I have come to wonder whether I have made it too complex and created an unecessary layer with a separate file of user.api.js which could possibly be refactored to be part of index.router.js.
What I do not understand is what is the standard practice of file structuring an API for scalability and should the API endpoints be in a separate file/folder of api/v1, api/v2 or should they be part of the router?
One thing that could be advantagous of having separate api files is resuability. Because it only contains functions and no routing, the functions could be reused across many different router files.

You have basically re-invented the MVC design pattern.
It's not overly complicated. It is considered good practice and encouraged.
C
Traditionally, what you call Routers is usually called the Controller. The job of the controller is simply to handle routing, handle argument parsing (request body, query parameters, is user logged in etc.) and sometimes handle validation. This is exactly what Express is designed to do. And Express allows controller functionality like authentication and validation to be refactored into middlewares.
Note: Sometimes you will see tutorials on the internet where people separate controllers and routes. My personal recommendation is do not do this. Express routing has been designed to be perfect for writing controllers. About the ONLY reason to separate them is if you have two different URLs that do the exact same thing. In my opinion that is better handled by a redirect.
M
Traditionally what you call API is called the Model. The model is your traditional collection of objects or data structures that you learned to program with. The model is what performs the application logic. Normally classes or modules that implement models are not labeled with anything. For example a user model would not be called UserAPI or UserModel but simply called User. However, what you name things is just a convention. Stick with what makes sense to you.
V
The final part of MVC is the View. In Express the view is simply res.json() or res.render() with its associated HTML template. This part is 99% already written by Express developers - you just need to tell the view functions what to send to the front-end.
Your architecture is good
There are very good reasons for separating the model (API) from the controller (router). First is that it allows you to solve your problems without polluting your core logic with parameter parsing logic. Your models should not need to worry about weather the user is logged in or how data is passed to it.
Second is that it allows you to use your model (API) logic outside of Express. The most obvious use for this is unit testing. This allows you to unit test your core logic without the web parts of the code. I also usually write utility scripts that I can use to do things like create a new user, dump user data, generate authentication token so I can use it with Postman etc.
For example you can create a script like:
#! /usr/bin/env node
// register-user.js
import UserAPI from '../api/v1/user.api.js'
UserAPI.registerUser({
EMail: process.argv[2],
Password: process.argv[4],
Name: process.argv[3]
})
.then(x => {console.log(x); process.exit()})
.catch(console.error);
Which you can then execute on the command line to create new users without needing to run the server:
$ ./register-user.js myemail#address.com 'My Name' 123456
It looks like your software is already structured according to MVC. Keep it that way. It will make maintaining and modifying the software a little easier.

API and router are two different things and even from different worlds.
Most of the applications are composed of two basic building blocks, UIs and APIs. UIs is supposed to be for humans and APIs for machines. You can have 0-n UIs and 0-n APIs. There is no rule for that.
For example UI you might have a webpage for common visitiors, application for paying visitors or those who bought your product and application for administrators. Those are three separate UIs. If one is malfunctioning, others are working.
The other example are APIs. There can be separate API for each of these UI. Also there can be APIs for third parties or other microservices of your own system. And again, if one API doesn't work, it doesn't impact others.
Router is the design pattern for URL control. Or a specific implementation of the design pattern if you wish.
While both UIs and APIs might require URL control, the same pattern can be used in both parts of the application system. But it doesn't mean it should be in the same folder. Things for building blocks should be in separate folders. Router, unlike the model, is not a common thing you would share among building blocks.
The folder structure you present here, Im sure you have seen it somewhere on the interent. But I do not consider it mature. I would consider something like that more mature:
├── model
├── website-ui
│ ├── routers
│ ├── templates
│ └── index.js // this is router
├── website-api
│ ├── user
│ │ └── index.js // this might be a router (API dependent
│ ├── item
│ │ └── index.js // this might be a router
│ └── index.js // this is router
├── index.js
└── main-router.js // this is router
Usually you don't even do main-router like that, because this responsibility goes often to load balancer outside of Nodejs. But everyone must start somewhere. This implementation is easy to upgrade later.
Do not confuse multiple APIs with API versions. In the best case scenario, you never want API versions, ever. You would proceed like this:
├── website-api
│ ├── user-old
│ │ └── index.js
│ ├── user
│ │ └── index.js
│ ├── item
│ │ └── index.js
│ ├── index-v1.js
│ └── index-v2.js
or that:
├── website-api-v1
│ ├── user
│ │ └── index.js
│ ├── item
│ │ └── index.js
│ └── index.js
├── website-api-v2
│ ├── user
│ │ └── index.js
│ ├── item
│ │ └── index.js
│ └── index.js
You can't tell now. And you shouldn't care. When you do changes in API, you do them backward compatible. If you can't do that anymore, it means you have done some critical mistakes in the past or large business changes comes in. This is not predictable.
One thing that could be advantagous of having separate api files is resuability.
Yes, but you can't tell now.
Regarding to your other questions. Stick to the SOLID rinciples. Each part of the code should have one specific purpose.
When I see your router folder, I have no idea what is there. Well I know routers, but routers of what? There can be everything, yet nothing. Thus not easily extensible = bad.
When I look at my structure, I can more easily predict what is in there.
You should design your architecture regarding to the purpose and not the specific implementations. Lets say you have two APIs because you have two purposes. Are they both REST or GraphQL? Can I share code and remove duplicities? Not that important. Sharing code is actually very dangerous if not done properly. Shared code is the worst part to refactor and it often doesn't provide as many advantages.
... I have come to wonder whether I have made it too complex...
Depends, is it a 14 days project? Yes you did. Is it for a year or more? You should go even deeper.

How to Dynamically Fetch Files with WebPack and Exclude Directories (node_modules)

TL;DR - We have ran into an issue with Storybook inside of a Monorepo
whose packages are independently versioned where the problem surrounds
Storybook running out of memory when it tries to scan for
*.stories.js due to it scanning every package's node_modules
directory. require.context() doesn't allow for exclusion to the
point where it won't scan the node_modules directories and we
couldn't find a native webpack solution to this problem - nor could we
find any existing discussion points around this problem, whose
solution wasn't just require.context('.', true,
/^\.\/((?!node_modules).)*\.stories\.js$/). We're posting our
discovery and potentially solution to see if anyone in the community
has had to solve a similar problem, and see if we are missing
something/making it harder than it needs to be.
The problem that we are trying to solve for is that Webpack's require.context() method does not allow for excluding specific directories from being scanned. For our project set-up, we are utilizing Lerna to manage a monorepo a JavaScript packages that are all independently versioned. Our file tree looks like this:
.
├── config
│ └── storybook
└── packages
└── vue
└── components
├── example-component
│ ├── node_modules
│ │ └── ...
│ ├── test
│ │ ├── example.stories.js
│ │ └── index.spec.js
│ ├── index.vue
│ └── package.json
└── example-parent
├── example-component-2
│ ├── node_modules
│ │ └── ...
│ ├── test
│ │ ├── example.stories.js
│ │ └── index.spec.js
│ ├── index.vue
│ └── package.json
└── example-component-3
├── node_modules
│ └── ...
├── test
│ ├── example.stories.js
│ └── index.spec.js
├── index.vue
└── package.json
For our storybook configuration, we need to tell Storybook where each of the components' *.stories.js file lives. We want/need the solution to be dynamic in a sense where if we add a new component into the directory tree in the same fashion, Storybook should automatically find the new *.stories.js without having to update Storybook's config file.
The option that Storybook suggests, is to use the require.context(String directory, boolean isDeep, RegEx fileMatch) method that is a part of webpack. However, when isDeep is set as true you are unable to restrict the method from scanning particular directories, like node_modules, due to the fact that the directory parameter is a string and not RegEx. Understandably, a regex match for the directory would make this method more expensive. However, it means that even if I set up my method to look like require.context('.', true, /^\.\/((?!node_modules).)*\.stories\.js$/) WebPack is still going to traverse my node_modules directories but won't match any internal *.stories.js files.
This becomes an issue when we use Lerna to install all the packages' depenencies via lerna bootstrap. Since by default that command will install ALL dependencies, including devDependencies for stuff like automated tests, an individual node_modules directory for a component could be quite large. So as we are adding more and more components, we have ran into a memory issue with running Storybook when components have all of their dependencies installed.
Our initial band-aid solution, was to restrict Lerna from install devDependencies by using lerna bootstrap -- --production --no-optional however this makes us unable to run tests. To run tests, we then have to install all dependencies... which means we can't run Storybook until we clean up the node_modules.
Our current solution is to use a fs type of library/package that allows us to traverse the directory tree with more specificity, and manually ignore node_modules when scanning deep directories. This allows us to still nest components within parent directories for better organization of similar components, and still allows Storybook to automatically find new *.stories.js files. So now our story discovery and Storybook config looks like:
const path = require('path');
const directoryTree = require('directory-tree');
let isPopulatedDir = (item) => item.type === 'directory' && item.children.length > 0;
let isStoryFile = (item) => item.type === 'file' && /\.stories\.js$/.test(item.name);
function parseChildren(children, response) {
for(var i = 0; i < children.length; i++) {
let child = children[i];
if(child.name === 'node_modules') continue;
if (isPopulatedDir(child)) {
parseChildren(child.children, response);
} else if(isStoryFile(child)) {
response.push(child.path);
}
}
return response;
}
function getStories() {
const tree = directoryTree("./packages/vue/components"),
storyPaths = [];
tree.children.forEach(item => {
if (isPopulatedDir(item)) {
storyPaths.push(...parseChildren(item.children, []));
}
});
return storyPaths;
}
module.exports = {
stories: getStories(),
webpackFinal: async (config) => {
config.node = {
fs: "empty"
};
config.resolve.modules = [
...(config.resolve.modules || []),
path.resolve('./'),
];
// Return the altered config
return config;
}
}
This solution works, it doesn't traverse any node_module directories, and allow us to dynamically find the *.stories.js files, no matter how their directory may be set up. But we can't help but feel like this is a problem that other people have had to of ran into, and maybe we are just overlooking a solution that Webpack already offers. So please, if you have had similar obstacles to overcome whose solution may work better for us we'd love to hear your approach or thoughts on our solution.

Navigating from Feature file to step defination: Any Plugins

In visual studio cod, How to navigate from feature to step definition. Do we need any additional plugins or any configuration needs to be added. I have downloaded the Cucumber (Gherkin) Full Support plugin but still cannot navigate from .feature to the step definition.

The documentation of Cucumber (Gherkin) Full Support plugin has the explanation for it.
You need to add the below in your settings:
{
"cucumberautocomplete.steps": [
"test/features/step_definitions/*.js",
"node_modules/qa-lib/src/step_definitions/*.js"
],
"cucumberautocomplete.syncfeatures": "test/features/*feature",
"cucumberautocomplete.strictGherkinCompletion": true
}
cucumberautocomplete.steps => provide the path of the step definitions.
cucumberautocomplete.syncfeatures => provide the path of the feature files
After this(might be after a restart), cmd + click(on mac) would take to the step definition.
Thanks,
Naveen

Having installed the extension alexkrechik.cucumberautocomplete, I tried modifying the settings from both the UI of the extension and its corresponding settings JSON (by default, mine were in ~/.config/Code/User/settings.json). But this didn't work because I got this error in the *.feature files: Was unable to find step for "Some feature description".
I noticed I had skipped a step mentioned in the extension docs... By default, it was getting the settings.json from my userspace and not my work(project)space.
For me, the solution was to go to the root directory of my project (usually outside of /src, where you have the package.json and node_modules/) and create a .vscode/ folder. Then, create a settings.json file and paste there the cucumberautocomplete configuration with the paths relative to this brand new file.
Below I show a schema:
myProject/
├── node_modules
├── package.json
├── subdir1
│ ├── src
│ └── test
│ └── e2e
│ └── src
│ ├── features
│ │ └── myfeature1.feature
│ ├── mypageobject1.po.ts
│ └── steps
│ └── mystep1.step.ts
└── .vscode
└── settings.json
An example of configuration would be:
{
"editor.detectIndentation": false,
"window.zoomLevel": 0,
"cucumberautocomplete.steps": [
"subidr1/test/e2e/src/steps/*.steps.ts"
],
"cucumberautocomplete.syncfeatures": "subidr1/test/e2e/src/feature/*.feature"
}
Note that you could use **/*.steps.ts and **/*.feature paths but every time the extension settings file changes, when you Ctr + Click on a feature description, you will need to wait for the editor to resolve the paths. Otherwise, there is no waiting time.

How do next.js applications optimize for mobile screens?

I just figured out that since I built my app with Next.js, I can't use CRA's folder structure framework to build or diagnose my application.
Unfortunately, I'm completely at a loss at the moment with respect to how Next.js applications are supposed to properly scale a website for mobile devices. I've always been under the impression that it was the job of index.html to do that (which I've written, but my app can't seem to bother to find it or use it). I've looked at the default folder structure for a Next.js app:
├── README.md
├── components
│ ├── head.js
│ └── nav.js
├── next.config.js
├── node_modules
│ ├── [...]
├── package.json
├── pages
│ └── index.js
├── static
│ └── favicon.ico
└── yarn.lock
source
but there doesn't seem to be a place for index.html.
My question is simply, how do Next.js apps optimize for mobile screens? Are they even supposed to have an index.html, and if so where? And how do favicons work, because I've created the static folder and put the favicon inside, but I'm pretty sure to have the favicon do anything, it has to be referenced by a file (conventionally index.html).
repo

Next has this Document component which you can customize to your own likings.
As their official docs say:
Is used to change the initial server side rendered document markup
You can use it to customize your head tag content as you would do anyway in your index.html.
Don't forget to add <meta name="viewport"content="width=device-width, initial-scale=1.0" /> in head tag if you want your app to use media queries.
Next project is not supposed to have a particular index.html file. Instead, the initial page is supposed to be a component located specifically in pages/index.js.

How to include library files (css/js) in angular-fullstack based app

I used a Yo Angular-Fullstack generator (https://github.com/DaftMonk/generator-angular-fullstack) and started an app and then tried installing Toastr from bower by doing -
bower install angular-toastr
and now I want to add the toastr css and js files. They are located in the
bower_components/angular-toastr/dist
now how do i include them in my current project, so that they are included in the dist folder when i build the application using grunt.
The folder structure is as follows -
├── client
│ ├── app - All of our app specific components go in here
│ ├── assets - Custom assets: fonts, images, etc…
│ ├── components - Our reusable components, non-specific to to our app
│
├── e2e - Our protractor end to end tests
│
└── server
├── api - Our apps server api
├── auth - For handling authentication with different auth strategies
├── components - Our reusable or app-wide components
├── config - Where we do the bulk of our apps configuration
│ └── local.env.js - Keep our environment variables out of source control
│ └── environment - Configuration specific to the node environment
└── views - Server rendered views

I use a grunt task called wiredep. It looks for the bower components that my app uses and add a reference to the css/js files to the file I specify.
I'm using .NET BundleConfig for minification so my task setup looks like this:
wiredep: {
task: {
src: [
'App_Start/BundleConfig.cs'
],
ignorePath: '..',
fileTypes: {
cs: {
block: /(([ \t]*)\/\/\s*bower:*(\S*))(\n|\r|.)*?(\/\/\s*endbower)/gi,
detect: {
js: /<script.*src=['"](.+)['"]>/gi,
css: /<link.*href=['"](.+)['"]/gi
},
replace: {
js: '.Include("~{{filePath}}")',
css: '.Include("~{{filePath}}")'
}
}
},
dependencies: true,
devDependencies: false
}
},
The end result is something like this:
bundles.Add(new ScriptBundle("~/bundles/thirdparty")
//NOTE: auto-generated by a grunt task
//anything between 'bower:js' and 'endbower' WILL BE LOST!
//bower:js
.Include("~/assets/angular/angular.js")
.Include("~/assets/moment/moment.js")
//endbower
);
bundles.Add(new StyleBundle("~/bundles/css")
//NOTE: auto-generated by a grunt task
//anything between 'bower:css' and 'endbower' WILL BE LOST!
//bower:css
.Include("~/assets/nouislider/distribute/nouislider.min.css")
//endbower
.Include("~/Content/css/app.css")
);
As I said, I'm using .NET BundleConfing, However, you can use and tags. I think you just need to remove the option replace from the grunt task configuration.