npm global installed packages and a automated and recursive build process - javascript

OK so, I have a WordPress site which a custom theme and several custom plugins (mu-plugins to be exact) which all use the exact same build processes. That is to say, they all have package.json files and use webpack. Technically there are a couple gulpfiles but I am migrating those to webpack configs.
So I want to be able to deploy my app, and after composer install all the plugins and theme, I would have a node script run which would:
Install all the NPM packages the build processes need - but install them globally
CD into the custom theme's src directory and run webpack --config webpack.production.js to build the theme's assets
CD into each of the custom plugins' src directories and run web run webpack --config webpack.production.js to build each plugin's assets in their respective locations
By doing this, I hope to
avoid committing compiled assets to my repos
avoid committing 3rd party packages to my repos
cut down on deployment time (because all packages would be installed globally, and only 1 time vs in each location)
I have already written the node scripts which:
installs the NPM packages
sets the node_path and node_prefix settings so my webpack (and gulp) scripts know where to find things
and CDs into each location and runs something like
This script simply runs child_process('webpack --config webpack.production.js' {cwd:' whatever/plugin/src'})
Now, the issues I having are (and how I have fixed some of them):
while my init node script could find node modules, the spawned processes that ran webpack could not
fixed this by setting NODE_PATH before the npm run command or by adding the node path via export in my .profile scrip
webpack files were unable to find the various loaders
fixed this by adding the global node path to the resolveLoaders option
my scss files couldn't find 3rd party packages like bulma
I fixed this by including the global node_modules path into the includePaths arg for sass-loader
The last, well most current issue I am facing is with webpack and vuejs.
I get this error:
ERROR in (webpack)/node_modules/timers-browserify/main.js
Module not found: Error: Can't resolve 'setimmediate' in '/usr/local/bin/npm-global/lib/node_modules/webpack/node_modules/timers-browserify'
# (webpack)/node_modules/timers-browserify/main.js 54:0-23
# /usr/local/bin/npm-global/lib/node_modules/vue/dist/vue.js
# ./src/js/entry.js
# ./src/entry.js
timers-browserify is apparently a dependency of VueJs. My entry js file has this vue import:
import Vue from 'vue/dist/vue';
and that appears to be throwing the error. If I remove it then webpack compiles everything as expected. If I install my node_modules in that folder (there is a package.json file in there) then it also compiles as expected.
I have already updated my webpack config to "resolve" module and loader dependencies to use their globally installed locations:
resolve: {
// Configure how Webpack finds modules imported with `import`.
modules: [
'/usr/local/bin/npm-global/lib/node_modules',
path.resolve(__dirname, 'node_modules'),
],
},
resolveLoader: {
// Configure how Webpack finds `loader` modules.
modules: [
'/usr/local/bin/npm-global/lib/node_modules',
path.resolve(__dirname, 'node_modules'),
],
},```
So it seems that Vue, or maybe some npm packages in general seem to have some expectations of where certain node_modules live?
This is likely going to be a continuous issue with the path I have decided on so I want to know whats causing it or what to look for?
One way I can fix this is by symlinking my globally install node_module folder into each plugin/theme. But I still want to know what is causing this as this entire process has exposed me to a couple concepts that have just helped me in general so I feel like the answer to this problem might also do the same.

Related

Deploying a NestJs project on a single board computer (Raspberry or similar)

even if it seems a simple task I'm having some trouble finding a solution. I know that with Nest CLI I can use the command "nest build" in order to create a dist folder that contains the production files of my project.
The problem is when I move the folder on my Raspberry and i try to run the project with the command "node dist/main" following NestJs instructions. Nothing starts because node says that it cannot find #nestjs/core and other modules.
I did't find nothing clear in the official guide about deploying the app, so my question is: what do I need to move onto my rasperry in addition to dist folder? Do I need to reinstall all node_modules folder or it's possible to have a running project without have to reinstall 800Mb of modules?
Yes you need to run yarn or npm install on your production environment, as your dist folder only contains your own code.
Because unlike compiled language like Golang where all dependencies are bundled and compiled in you executable file, Javascript bundlers don't. Your bundled code in dist still contains require or import statement to get dependencies from node_modules.
You can also run npm prune --production to remove any developpement dependencies and thus reduce the size of your node_modules folder. (I believe that yarn does it by default.)

Organize multiple Webpack entry file dependencies

How to organize JS reusable components with their own dependencies in 3rd part bundles (e.g. vendor/ subdirectories) with webpack different entry files/configs?
Description:
I have following structure with entrypoints and components
app/
assets/
index.js
package.json
webpack.config.js
vendor/
my-utils-bundle/
assets/
components/
MyMath.js
package.json
webpack.config.js
In my includable within different entry files/webpack configs MyMath.js components there's node_modules/ dependency to package e.g. mathjs with following line:
import { pi, pow, round, sqrt } from 'mathjs'
....some code below...
P.S. mathjs dependency package provided in this bundle (vendor/my-utils-bundle/package.json)
Then when i run yarn dev to compile my assets in app/assets/index.js there's following error:
"./vendor/my-utils-bundle/assets/components/MyMath.js" contains a reference to the file "mathjs".
This file can not be found, please check it for typos or update it if the file got moved.
Question:
How can i use this MyMath.js components as includable class in app/assets/index.js and maybe also in other "bundles" (e.g. vendor/my-other-bundle)?
Looks that I could install this mathjs dependency also in app/package.json but then it seems that on application level i should think about bundles JS dependencies that sounds not as the best solution.
This is an open question that has been going on pretty much since the launch of Encore. Even earlier, given that according to the best practices for bundles, third party dependencies shouldn't be distributed, and left for the app to manage.
That being said, there is at least one bundle (foxy) that attempts to solve this, by hooking into the composer commands and merge dependencies when packages are installed/updated.
From its README (emphasis mine):
Foxy focuses solely on automation of the validation, addition,
updating and deleting of the dependencies in the definition file of
the asset package, while restoring the project state, as well as PHP
dependencies if NPM or Yarn terminates with an error.
Foxy retrieves the list of all Composer dependencies to inject the
asset dependencies in the file package.json
[...]
Given that Foxy does not manipulate any asset
dependencies, and let alone the version constraints, this allows NPM
or Yarn to solve the asset dependencies without any intermediary

where is create-react-app webpack config and files?

I create a ReactJS project with the create-react-app package and that worked well, but I cannot find webpack files and configurations.
How does react-create-app work with webpack? Where are the webpack configuration files located in a default installation with create-react-app? I'm unable to find configuration files in my project's folders.
I have not created an override config file. I can manage config settings with other articles but I want to find the conventional config file(s).
If you want to find webpack files and configurations go to your package.json file and look for scripts
You will find that scripts object is using a library react-scripts
Now go to node_modules and look for react-scripts folder react-script-in-node-modules
This react-scripts/scripts and react-scripts/config folder contains all the webpack configurations.
The files are located in your node_modules/react-scripts folder:
Webpack config:
https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/config/webpack.config.js
Start Script:
https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/scripts/start.js
Build Script:
https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/scripts/build.js
Test Script:
https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/scripts/test.js
and so on ...
Now, the purpose of CRA is not to worry about these.
From the documentation:
You don’t need to install or configure tools like Webpack or Babel.
They are preconfigured and hidden so that you can focus on the code.
If you want to have access to the config files, you need to eject by running:
npm run eject
Note: this is a one-way operation. Once you eject, you can’t go back!
In most scenarios, it is best not to eject and try to find a way to make it work for you in another way.
If you need to override some of the config options, you can have a look at https://github.com/gsoft-inc/craco
A lot of people come to this page with the goal of finding the webpack config and files in order to add their own configuration to them. Another way to achieve this without running npm run eject is to use react-app-rewired. This allows you to overwrite your webpack config file without ejecting.
Assuming you don't want to eject and you just want to look at the config you will find them in /node_modules/react-scripts/config
webpack.config.dev.js. //used by `npm start`
webpack.config.prod.js //used by `npm run build`
Webpack config used by create-react-app is here:
https://github.com/facebook/create-react-app/tree/master/packages/react-scripts/config
You can find it inside the /config folder.
When you eject you get a message like:
Adding /config/webpack.config.dev.js to the project
Adding /config/webpack.config.prod.js to the project
Webpack configuration is being handled by react-scripts.
You can find all webpack config inside node_modules react-scripts/config.
And If you want to customize webpack config, you can follow this customize-webpack-config
Ejecting is not the best option as well as editing something under node_modules.
react-app-rewired is not maintained and has the warning:
...you now "own" the configs. No support will be provided. Proceed
with caution...
Solution - use craco.
Try ejecting the config files, by running:
npm run eject
then you'll find a config folder created in your project. You will find your webpack config files init.
I'm using create-react-app with craco, and I was able to override my webpack configuration when updating to webpack 5 with the craco.config.js:
module.exports = {
style: {
postcssOptions: {
plugins: [
require('tailwindcss'),
require('autoprefixer'),
],
},
},
webpack: {
configure: (webpackConfig, { env, paths }) => {
// eslint-disable-next-line no-param-reassign
webpackConfig.resolve.fallback = {
fs: false,
};
return webpackConfig;
},
},
}
I know it's pretty late, but for future people stumbling upon this issue, if you want to have access to the webpack config of CRA, there's no other way except you have to run:
$ npm run eject
However, with ejection, you'll strip away yourself from CRA pipeline of updates, therefore from the point of ejection, you have to maintain it yourself.
I have come across this issue many times, and therefore I've created a template for react apps which have most of the same config as CRA, but also additional perks (like styled-components, jest unit test, Travis ci for deployments, prettier, ESLint, etc...) to make the maintenance easier. Check out the repo.

Using stellar-lib api with Meteor

this is probably a silly question but am new to Meteor and struggling a bit. I want to build a stellar app that tweets when you get stellar. There is a nice Javascript API stellar-lib that works on node, but im unsure how to access the modules in Meteor...
I'm also building an app with Meteor and stellar-lib and have found two options:
1) You can manually copy over the built stellar-lib.js or stellar-lib-min.js from the build/ directory from either the github repo or the node_modules folder you installed it to when you ran the npm install command.
If you do this, you will have to copy the .js file to client/compatibility, otherwise stellar-lib will not work (note: this means you can only use Stellar on the client).
2) If you need it on the server, you can also have browserify wrap stellar-lib for you, then copy the output to lib/ in your Meteor app's root directory. I did this in my repo here with gulp.
Here's a short explanation to how 2) works:
.gulp is where I'll install my NPM modules where they will be ignored by the Meteor build environment (because the folder is hidden).
In deps.js, I require the modules I would want to use in my Meteor app as I would if I was using them in a traditional node.js app. package.json defines the modules I'll install with NPM and the gulpfile.js describes a build task that will resolve my require statements and output a single deps.js file that includes my dependencies to my Meteor app's lib/ folder.

Running grunt on dependencies

I have 3 projects all using grunt.
Project a depends on project c and b
Project b depends on project c
Project c depends on nothing
Project a and b both require a step that compiles project c (which is a style repo that contains global styles for our org).
I am attempting to run grunt post install for project b.
There are a couple problems here.
Project a and b both try to build project c in their build processes. This takes sometime and I'd rather avoid it.
Project b expects different paths when running alone. loadNpmTasks fails unless I grunt.file.setBase, but then I have other paths that are broken as well. This means I need to manually track all of those down and make sure they are correct in both situations. This is rather flimsy.
As a result, I am thinking I am not doing this correctly or in a "normal way". What is the appropriate way to manage dependencies that use grunt?
Update
The main problem is that I get the following errors no matter what solution I've been choosing:
Local Npm module "grunt-contrib-compass" not found. Is it installed?
Local Npm module "grunt-contrib-handlebars" not found. Is it installed?
Local Npm module "grunt-contrib-requirejs" not found. Is it installed?
Local Npm module "grunt-contrib-watch" not found. Is it installed?
Local Npm module "grunt-notify" not found. Is it installed?
Local Npm module "grunt-curl" not found. Is it installed?
Local Npm module "grunt-shell" not found. Is it installed?
I tried spawn a task and calling grunt via the command line. I tried using various plugins to help. I think the issue is that both the top level and the dependencies require those tasks. This means they get pulled up into the parent node_modules folder. As a result the dependencies are missing the above modules in their node_modules folder.
I've built grunt-grunt with exactly this kind of scenario in mind.
grunt.initConfig({
grunt: {
lintsome: {
gruntfile: 'node_modules/some/Gruntfile.js',
task: 'jshint'
}
}
});
The README contains enough documentation to get you going.

Categories

Resources