I am trying to package code for AWS Lambda. Lambda has various restrictions, such as using Node 6.10, and not having a build step, like AWS EB does. I also am using NPM modules, so these will need to be bundled with the AWS Lambda handler.
Here is what I would like to do:
Define and use NPM modules (pure JS modules only)
Transpile all code (including NPM modules) to a JS version that Node 6.10 supports
Statically link all NPM modules into one big JS file
Upload that single file to AWS Lambda
For example, suppose I have an NPM module foo (node_modules/foo/index.js):
export default { x: 1 };
and I have my own code ('index.js'):
import foo from 'foo';
export const handler = (event, context, callback) => {
console.log(foo); // Will appear in CloudWatch logs
callback(null, 'OK');
};
The output would be something like this ('dist/bundle.js'):
var foo = { x: 1 };
exports.handler = function(event, context, callback) {
console.log(foo);
callback(null, 'OK');
};
I should be able to upload and run bundle.js on AWS Lambda without further modification.
How can I achieve this using existing JS tools?
You can use serverless with serverless-webpack
Then you deploy your bundle with serverless deploy
It turns out that this is possible, but it requires some tricky configuration to achieve. I have created a boiler-plate repo for others to use.
Here are the important bits...
First, you need a .babelrc that targets Node.js 6.10:
{
"presets": [
[
"env", {
"targets": {
"node": "6.10"
},
"loose": false,
"spec": true
}
]
]
}
Next, you need to configure Webpack to generate a commonjs library targetting node:
const path = require('path');
const webpack = require('webpack');
const debug = process.env.NODE_ENV !== 'production';
module.exports = {
context: __dirname,
entry: [ 'babel-polyfill', './index.js' ],
output: {
path: path.join(__dirname, 'out'),
filename: 'index.js',
libraryTarget: 'commonjs'
},
devtool: debug ? 'source-map' : false,
module: {
rules: [
{
test: /\.js$/,
use: {
loader: 'babel-loader',
options: {
babelrc: true,
compact: !debug
}
}
}
],
},
target: 'node',
plugins: [
new webpack.DefinePlugin({ 'global.GENTLY': false })
]
};
Note that you do not want to ignore the node_modules folder, since that would prevent static-linking.
The babel-polyfill plugin is also crucial if you want to use modern JS features.
Your actual handler code should have a named export that matches what you have set in the AWS console:
export const handler = (event, context, callback) => callback(null, 'OK');
Do not do it like this!
// Bad!
export default {
handler: (event, context, callback) => callback(null, 'OK'),
};
When packaging the code, make sure you add index.js to the top level of the zip:
zip -j bundle.zip ./out/index.js
Related
I am trying to use Wallaby in conjunction with the dotenv-flow package. I currently have my wallaby.js config file setup like below:
require("dotenv-flow").config()
module.exports = function (wallaby) {
return {
files: [
'api/*',
'controllers/*',
'config/*',
'firebase/*',
'helpers/*',
'models/*',
'services/*',
'smtp/*',
'sockets/*'
],
tests: [
"test/**/*.test.mjs"
],
testFramework: "mocha",
env: {
type: "node",
params: {
env: "NODE_ENV=test"
}
}
};
};
I've tried a few other ways of writing the file including in esm module format. However, my tests run and my sequelize code complains that it wasn't passed environment variables to use for connecting to the development DB.
You are loading your .env file but then never using it's contents. Another problem is that wallaby doesn't understand the dotenv output so you have to massage it a little bit.
const environment = Object.entries(
require("dotenv-flow").config()['parsed']).
map( x => `${x[0]}=${x[1]}`).join(';'),
Then change your environment to something like this
env: {
runner: 'node',
params: {
env: environment
}
}
As per the documentation, I am not using worker-loader, I am trying to use the native way suggested by the webpack-5 documentation.
Below is the usage of the worker script in the main thread.
const worker = new window.Worker(
new URL("../workers/listOperation.worker.js", import.meta.url),
{
type: "module",
},
);
worker.postMessage({ list: hugeList, params: reqData });
worker.onerror = err => console.error(err);
worker.onmessage = e => {
const { list } = e.data;
// Usage of `list` from the response
worker.terminate();
};
return worker;
It works fine if there are no imports used in the script. But when I import any node modules (e.g. loadash/get) or any other function/constants from local, it does not work as the output webWorker bundle file doesn't transpile and bundle the imported code. It keeps the "import" statement as it is.
Below is the worker script (listOperation.worker.js)
import get from "lodash/get";
import { ANY_CONSTANT } from "../constants"; // This is some local constant
addEventListener("message", e => {
const { list, params } = e.data;
// Here I have some usage of `get` method from `lodash/get` and ANY_CONSTANT
self.postMessage({
list: list,
});
});
Webpack outputs the bundle file like below which won't be usable by the browser, if I put the /\.worker.js$/ pattern in the the exclude of babel-loader rule.
import get from "lodash/get";import{ANY_CONSTANT}from"../constants";addEventListener("message",(e=>{const{list:s,params:t,.......
And even if I don't put the /\.worker.js$/ pattern in the the exclude of babel-loader rule, the output bundle still doesn't include the implementation of get from lodash/get or the value of the constant. It just outputs it in cjs using require.
Also, I made use of asset module so that I can put the output file inside a directory, not directly in the root of the dist folder. Configuration changes in my webpack.config looks like this.
module.exports = {
entry: {...},
module: {
rules: [
{
test: /\.js$/,
exclude: [/(node_modules)/, /\.worker.js$/],
use: {
loader: "babel-loader", // This uses the config defined in babel.config.js
},
},
{
test: /\.worker.js$/,
exclude: /(node_modules)/,
type: "asset/resource",
generator: {
filename: "js/workers/[hash][ext][query]",
},
},
],
},
}
I'm working on Arrow project and I'm in phase to create the Bundle file using Webpack.
I grouped my modules in a folder. and for each folder I have index.js where I export all modules.
Also, I have global index.js that imports all index.js like this :
import * as Has from "./scripts/array/has"
import * as Math from "./scripts/array/math"
import * as Arrow from "./scripts/array"
export { Has, Math, Arrow }
Now, I want to create my bundle from this global index.js .
My Webpack configuration looks like :
const path = require("path")
const { CleanWebpackPlugin } = require("clean-webpack-plugin")
module.exports = {
mode: "development",
entry: {
arrow: "./src/1.x.x/index",
},
plugins: [
new CleanWebpackPlugin({ cleanStaleWebpackAssets: false }),
],
devtool: 'inline-source-map',
devServer: {
contentBase: './build',
},
output: {
filename: "[name]-alpha.js",
path: path.resolve(__dirname, 'build'),
},
optimization: {
splitChunks: {
chunks: 'all',
},
},
}
The Issue is when I try to import my functions from the build the functions do not appear in the Autocomplete and I received errors that these functions are undefined!
I have just ___esModule :
import { __esModule } from "./arrow-alpha"
I want to let developers use and import the functions like the example
import {omit} from "arrow" // arrow is the bundled file
const frameworks = ["react", "vue", "ember", "angular"];
const withoutAngular = omit(frameworks,"angular");
console.log(withoutAngular); // ["react", "vue", "ember"]
I blocked in this step for days and I could not figure out the issue.
It sounds like you are looking to export your code as library with webpack. In order to do so, you should export your library module as umd which can work for backend/client side so IDE can also understand what you export. Just add following configuration in your webpack.config.js:
output: {
libraryTarget: 'umd', // style for your library
library: 'yourLibName', // if you keen to have a name for library
// ...
},
NOTE: this only works if you pass to webpack a cjs style code, otherwise it will convert to harmony module which you can't export accordingly (such as esm module)
In order to make sure babel would convert your esm module to commonjs, you have to set modules to commonjs like this:
babel.config.js or .babelrc
[
"#babel/preset-env",
{
"modules": "commonjs",
}
]
Adding below Configuration. Will eject your final bundle to be accessed as Window.Arrow on adding script to index.html page
output: {
filename: "[name]-alpha.js",
path: path.resolve(__dirname, 'build'),
library: 'Arrow',
libraryTarget: 'var'
},
and your index.js page inside src folder to be as
module.exports = require('./scripts/array');
I have a rather large React application built with webpack 2. The application is embedded into a Drupal site as a SPA within the existing site. The Drupal site has a complex gulp build setup and I can't replicate it with webpack, so I decided to keep it.
I have split my React application into multiple parts using the DllPlugin / DllReferencePlugin combo which is shipped out of the box in webpack 2. This works great, and I get a nice vendor-bundle when building with webpack.
The problem is when I try to run my webpack configuration in gulp, I get an error. I might be doing it wrong, as I have not been able to find much documentation on this approach, but nevertheless, it's not working for me.
It looks like it's trying to include the the manifest file from my vendor-bundle before creating it.
Whenever I run one of my defined gulp tasks, like gulp react-vendor I get an error, saying that it cannot resolve the vendor-manifest.json file.
If I on other hand run webpack --config=webpack.dll.js in my terminal, webpack compiles just fine and with no errors.
I have included what I think is the relevant files. Any help on this is appreciated.
webpack.config.js
// Use node.js built-in path module to avoid path issues across platforms.
const path = require('path');
const webpack = require('webpack');
// Set environment variable.
const production = process.env.NODE_ENV === "production";
const appSource = path.join(__dirname, 'react/src/');
const buildPath = path.join(__dirname, 'react/build/');
const ReactConfig = {
entry: [
'./react/src/index.jsx'
],
output: {
path: buildPath,
publicPath: buildPath,
filename: 'app.js'
},
module: {
rules: [
{
exclude: /(node_modules)/,
use: {
loader: "babel-loader?cacheDirectory=true",
options: {
presets: ["react", "es2015", "stage-0"]
},
},
},
],
},
resolve: {
modules: [
path.join(__dirname, 'node_modules'),
'./react/src/'
],
extensions: ['.js', '.jsx', '.es6'],
},
context: __dirname,
devServer: {
historyApiFallback: true,
contentBase: appSource
},
// TODO: Split plugins based on prod and dev builds.
plugins: [
new webpack.DllReferencePlugin({
context: path.join(__dirname, "react", "src"),
manifest: require(path.join(__dirname, "react", "vendors", "vendor-manifest.json"))
}),
new webpack.optimize.CommonsChunkPlugin({
name: 'manifest',
filename: 'webpack-loader.js'
}),
]
};
// Add environment specific configuration.
if (production) {
ReactConfig.plugins.push(
new webpack.optimize.UglifyJsPlugin()
);
}
module.exports = [ReactConfig];
webpack.dll.js
const path = require("path");
const webpack = require("webpack");
const production = process.env.NODE_ENV === "production";
const DllConfig = {
entry: {
vendor: [path.join(__dirname, "react", "vendors", "vendors.js")]
},
output: {
path: path.join(__dirname, "react", "vendors"),
filename: "dll.[name].js",
library: "[name]"
},
plugins: [
new webpack.DllPlugin({
path: path.join(__dirname, "react", "vendors", "[name]-manifest.json"),
name: "[name]",
context: path.resolve(__dirname, "react", "src")
}),
// Resolve warning message related to the 'fetch' node_module.
new webpack.IgnorePlugin(/\/iconv-loader$/),
],
resolve: {
modules: [
path.join(__dirname, 'node_modules'),
],
extensions: ['.js', '.jsx', '.es6'],
},
// Added to resolve a dependency issue in this build #https://github.com/hapijs/joi/issues/665
node: {
net: 'empty',
tls: 'empty',
dns: 'empty'
}
};
if (production) {
DllConfig.plugins.push(
new webpack.optimize.UglifyJsPlugin()
);
}
module.exports = [DllConfig];
vendors.js (to determine what to add to the Dll)
require("react");
require("react-bootstrap");
require("react-dom");
require("react-redux");
require("react-router-dom");
require("redux");
require("redux-form");
require("redux-promise");
require("redux-thunk");
require("classnames");
require("whatwg-fetch");
require("fetch");
require("prop-types");
require("url");
require("validator");
gulpfile.js
'use strict';
const gulp = require('gulp');
const webpack = require ('webpack');
const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');
// React webpack source build.
gulp.task('react-src', function (callback) {
webpack(reactConfig, function (err, stats) {
callback();
})
});
// React webpack vendor build.
gulp.task('react-vendor', function (callback) {
webpack(vendorConfig, function (err, stats) {
callback();
})
});
// Full webpack react build.
gulp.task('react-full', ['react-vendor', 'react-src']);
NOTE:
If I build my vendor-bundle with the terminal with webpack --config=webpack.dll.js first and it creates the vendor-manifest.json file, I can then subsequently successfully run my gulp tasks with no issues.
This is not very helpful though, as this still will not allow me to use webpack with gulp, as I intend to clean the build before new builds run.
I ended up using the solution mentioned in the end of my question. I build my DLL file first and then I can successfully run my gulp webpack tasks.
One change that can make it easier to debug the issue, is to use the Gulp utility module (gulp-util) to show any webpack errors that might show up during build of webpack, using gulp.
My final gulp setup ended up looking like this:
gulpfile.js
'use strict';
const gulp = require('gulp');
const gutil = require('gulp-util');
const webpack = require('webpack');
const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');
// React webpack source build.
gulp.task('react', function (callback) {
webpack(reactConfig, function (err, stats) {
if (err) {
throw new gutil.PluginError('webpack', err);
}
else {
gutil.log('[webpack]', stats.toString());
}
callback();
});
});
// React webpack vendor build.
gulp.task('react-vendor', function (callback) {
webpack(vendorConfig, function (err, stats) {
if (err) {
throw new gutil.PluginError('webpack', err);
}
else {
gutil.log('[webpack]', stats.toString());
}
callback();
});
});
// React: Rebuilds both source and vendor in the right order.
gulp.task('react-full', ['react-vendor'], function () {
gulp.start('react');
});
I hope this might help someone in a similar situation.
Whenever I run one of my defined gulp tasks, like gulp react-vendor I get an error, saying that it cannot resolve the vendor-manifest.json file.
Your gulpfile.js contains this:
const reactConfig = require('./webpack.config.js');
const vendorConfig = require('./webpack.dll.js');
And webpack.config.js contains this:
new webpack.DllReferencePlugin({
context: path.join(__dirname, "react", "src"),
manifest: require(path.join(__dirname, "react", "vendors", "vendor-manifest.json"))
}),
The require() calls are currently all executed immediately. Whenever you run Gulp, it will evaluate both Webpack configuration files. As currently configured, Node runs the code in webpack.config.js at startup, and from there it sees the require() used in your creation of the DllReferencePlugin, so it will also try to read manifest.json at that time and turn it into an object...which is before it has been built.
You can solve this in one of two ways:
The DllReferencePlugin's manifest option supports either an object (which is what you are currently providing), or else a string containing the path of the manifest file. In other words, it should work if you remove the require() from around your path.join(...) call.
Alternatively, you can also defer the loading of the Webpack config files. Moving your const reactConfig = require('./webpack.config.js'); from the top of the file directly into the gulp task function should be sufficient, assuming that this function is not invoked until after the manifest has been built.
I have a this file backend-dev.js which is mostly webpack configuration. I use it to run my express server from the bundled file. It stays on and restarts the server on any change. Is there any possible configuration can be added to auto refresh the browser too whenever I change the code?
This is what I have in the backend-dev.js:
const path = require('path');
const webpack = require('webpack');
const spawn = require('child_process').spawn;
const HtmlWebpackPlugin = require('html-webpack-plugin');
const compiler = webpack({
// add your webpack configuration here
entry: './app.js',
output: {
path: path.resolve(__dirname, 'dist'),
filename: "bundle.js"
},
module: {
rules: [
{
test: /\.js$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
options: {
presets: ['env', 'es2015', 'stage-2']
}
}
}
]
},
/*plugins: [
new HtmlWebpackPlugin({
title: 'Project Demo',
hash: true,
template: './views/index.ejs' // Load a custom template (ejs by default see the FAQ for details)
})
],*/
node: {
__dirname: false
},
target: 'node'
});
const watchConfig = {
// compiler watch configuration
// see https://webpack.js.org/configuration/watch/
aggregateTimeout: 300,
poll: 1000
};
let serverControl;
compiler.watch(watchConfig, (err, stats) => {
if (err) {
console.error(err.stack || err);
if (err.details) {
console.error(err.details);
}
return;
}
const info = stats.toJson();
if (stats.hasErrors()) {
info.errors.forEach(message => console.log(message));
return;
}
if (stats.hasWarnings()) {
info.warnings.forEach(message => console.log(message));
}
if (serverControl) {
serverControl.kill();
}
// change filename to the relative path to the bundle created by webpack, if necessary(choose what to run)
serverControl = spawn('node', [path.resolve(__dirname, 'dist/bundle.js')]);
serverControl.stdout.on('data', data => console.log(data.toString()));
serverControl.stderr.on('data', data => console.error(data.toString()));
});
Excellent question: I solved it in the following way:
express has two modules you can install to refresh in hot without refreshing the browser, and it will refresh automatically: I leave you my configuration.
npm i livereload
npm i connect-livereload
const livereload = require('livereload');
const connectLivereload = require('connect-livereload');
const liveReloadServer = livereload.createServer();
liveReloadServer.watch(path.join(__dirname, '..', 'dist', 'frontend'));
const app = express();
app.use(connectLivereload());
Note that the folder you want to monitor is in dist/frontend. Change the folder you want to monitor so that it works: To monitor the backend I am using NODEMON
livereload open a port for the browser in the backend to expose the changes: how does the connection happen? It's easy; express with "connect-livereload" and inject a script that monitors that port: if a change occurs, the browser is notified by express, and the browser will refresh for you.
I leave The information as simple as possible to test, and I do not recommend using it like this: To use it you must separate the development and production environments. I keep it simple so that it is easy to understand.
Here I leave the most relevant link I found: I hope it will be helpful too.
https://bytearcher.com/articles/refresh-changes-browser-express-livereload-nodemon/