I'am working on app with React, TS and Webpack stack.
I need to implement feature that allows my app work with client plugins - js files that override existing functionality of some classes. It can be loaded from anywhere - local file system or remote repository and should be fetched in the runtime, because i need to have an option to specify new extension in config and just press F5.
Dynamic import is not my case, because as far as i understand Webpack needs to be able to at least guess roughly what an import() is meant to be referencing. Using simple 'get' request might be an option, but how can i use loaded script as CommonJS module in this case? And am i correct about dynamic import behavior?
You can use #paciolan/remote-module-loader to remotely load a common js module.
import { createLoadRemoteModule } from "#paciolan/remote-module-loader"
const main = async() => {
const loadRemoteModule = createLoadRemoteModule()
const myModule = await loadRemoteModule("http://fake.url/modules/my-module.js")
const value = myModule.default()
console.log({ value })
}
main()
If you need to pass dependencies to the module:
import {
createLoadRemoteModule,
createRequires
} from "#paciolan/remote-module-loader"
const dependencies = {
react: require("react")
}
const main = async() => {
const requires = createRequires(dependencies)
const loadRemoteModule = createLoadRemoteModule({ requires })
const myModule = await loadRemoteModule("http://fake.url/modules/my-module.js")
const value = myModule.default()
console.log({ value })
}
main()
If need to load a React Component, check out #paciolan/remote-component
You may have to take extra steps if you have a Content Security Policy (CSP) set.
Related
I'm working on a javascript app using the classic node/babel/webpack stack that makes heavy use of lookup tables. The typical code look like this:
const computeLookupTable = () => {
// Expensive computation here
};
export const LookupTable = computeLookupTable();
This work well overall, but there is a major downside: the user's device has to do all the precomputation when the app starts. This seems rather unnecessary as the result of the computation will always be the same. It is worth specding some build time for production build to improve the user's experience.
How can I change this behavior, such as computeLookupTable is run during the build process, and directly get the data in there in the generated blob that is shipped to users?
Option 1
Not sure if this is the best way, but you can use Webpack's define plugin to replace the values:
In webpack.config.js:
import webpack from "webpack";
import { LookupTable } from "./computeLookupTable";
module.exports = {
// ...
plugins: {
new webpack.DefinePlugin({
"lookupResults": JSON.stringfify(LookupTable)
})
}
}
Do realize though, that you can't pass functions or anything, just plain objects or primitives. In your code you could just pretend the variable magically exists:
export default function Something() {
const results = lookupResults;
return results.data;
}
One caveat is that you have to create a new variable to avoid invalid syntax like { data: "..." }.data, but instead const results = { data: "..." }; results.data.
Option 2
The more obvious and not-as-eeky/natural way to go this is to generate static files before bundling, then allow Webpack to bundle it up for us.
In webpack.config.js:
import fs from "fs";
import webpack from "webpack";
import { LookupTable } from "./computeLookupTable";
module.exports = (staticData => {
fs.writeFileSync("./src/data/generated.js", "export default " + JSON.stringify(staticData)); // change this to something
// or with JSON (needs plugin): fs.writeFileSync("./src/data/generated.json", JSON.stringify(staticData));
return { /* webpack configs... */ };
})(LookupTable);
And then let's say your JS file is src/index.js:
import table from "./data/generated";
// do something with it...
I'm building on Next.js app and when I install / import Plaiceholder (for generating placeholder images), I get the following error: Module not found: Can't resolve 'child_process'
Node v14.18.0
Next.js v11.1.2
Plaiceholder v2.2.0
Sharp v0.29.2
I understand this error message to mean that webpack5 is trying to bundle node packages that aren't available to the client. But I'm not clear why it is doing this. I haven't customized any of the webpack configs, and I can't find any mention of this issue in the Plaiceholder docs. How do I fix it?
NOTE: I want the base64 data URL to get created during the build, so that it available as soon as the page loads (not fetched asynchronously at run time).
Here's my next.config.js
module.exports = {
reactStrictMode: true,
};
My package.json only has scripts, dependencies, and devDependencies (nothing to change module resolution)
In case it's relevant, here's a simplified example using Plaiceholder:
import Image from "next/image";
import { getPlaiceholder } from "plaiceholder";
import React, { useState } from "react";
...
const { base64 } = await getPlaiceholder(imgUrl);
...
return (<Image
src={imgUrl}
placeholder="blur"
blurDataURL={base64}
/>);
It seems like plaiceholder is not suitable for client-side rendering. I believe that package is for the Node.js environment. That's why you get this error when you try to render your component on the client side.
To solve this problem, you need to move import { getPlaiceholder } from 'plaiceholder' to the NextJS API section. Then you can call that API with your URL data in the body. Then get the base64.
/api/getBase64.js
import { getPlaiceholder } from "plaiceholder";
export default async (req, res) => {
const { body } = req;
const { url } = body;
const { base64 } = getPlaiceholder(url);
res.status(200).send(base64);
};
/component.js
import Image from "next/image";
import React, { useState, useEffect } from "react";
const [base64, setBase64] = useState()
useEffect(() => {
(async () => {
const _base64 = await fetch.post('/api/getBase64', {url: imgUrl}); // wrote for demonstration
setBase64(_base64);
})()
})
return (<Image
src={imgUrl}
placeholder="blur"
blurDataURL={base64}
/>);
I know blurDataURL will be undefined until you fetch the data but this is the way how you can use plaiceholder library to manage your images. It should be imported only for the NodeJS environment. If you do not like this approach, you can try to find another library that also works for the browser environment (client)
UPDATED according to the comment:
If you want to generate this base64 at build time, you can use getStaticProps in the pages that use this Image component. NextJS is smart enough to understand which libraries are used in the client-side or server-side. So you can do this:
import { getPlaiceholder } from "plaiceholder"; // place it at the root of file. This will not be bundled inside of client-side code
export async function getStaticProps(context) {
const { base64 } = await getPlaiceholder(imgUrl);
return {
props: { base64 }, // will be passed to the page component as props
}
}
This way, by using getStaticProps, the page will be created at build time. You can get the base64 prop inside of the page that uses the image component and pass that prop to blurDataURL. Also, you can use this approach with getServerSideProps too.
This is from NextJS website:
Note: You can import modules in top-level scope for use in
getServerSideProps. Imports used in getServerSideProps will not be
bundled for the client-side.
https://nextjs.org/docs/basic-features/data-fetching#getserversideprops-server-side-rendering
It's necessary to Install plugin for Next Js dependency and configure next config based on Plaiceholder Docs for using getPlaiceholder() function in getStaticProps like the answer by #oakar.
npm i #plaiceholder/next
const { withPlaiceholder } = require("#plaiceholder/next");
module.exports = withPlaiceholder({
// your Next.js config
});
I support a relatively complex legacy codebase, but am looking to modernise it a little by bringing in Webpack so that we'd have import & export capabilities in JS.
The problem I'm having is that we use a global object called App where we define and add different properties depending on the page. So for example we have the following file where we instantiate App (loaded on all pages):
app.js
const App = (() => {
const obj = {
Lib: {},
Util: {},
// etc
}
return obj;
})();
Then in another file we add to App.Lib just for the specific page that needs it:
lazyload.js
App.Lib.Lazyload = (() => {
// lazyload logic
})();
We simply concatenate the files during the bundling process, but obviously this is not ideal as none of the files have no knowledge of what goes on outside of it.
Exporting only seems to work for the top level object (where the object is defined), so anything I add to it elsewhere cannot be exported again. For example if I add export default App.Lib.Lazyload; at the end of lazyload.js and then try to import it elsewhere it will not import the Lazyload property.
Is there any way to get this to work without major refactor? If not, would you have any suggestions about the best way to handle it?
I don't think you can import Object.properties in JS. If you want to bundle specific packages (say Lazyload) for packages that need them, you might try:
//lazyload.js
export const LazyLoad = {
//lazyload logic
}
then somewhere else...
import {LazyLoad} from 'path/to/lazyload.js';
// assuming App has already been created/instantiated
App.Lib.Lazyload = LazyLoad;
Using Export Default...
//lazyload.js
const LazyLoad = {};
export default LazyLoad;
then...
import LazyLoad from 'path/to/lazyload.js';
App.Lib.LazyLoad = LazyLoad;
You can find help with Imports and Exports at MDN.
I'm trying to make an improvement to an existing Gatsby plug-in, and I want to pass a React Component to the plug-in, through its configuration entry in gatsby-config.js:
plugins: [
{
resolve: `gatsby-plugin-modal-routing`,
options: { someComponent: SomeComponentClassOrFunction }
},
However, the problem I'm running into is that I can't figure out how to make it work.
If I try to pass the component itself as part of the plug-in's configuration, it seems to get serialized to/from JSON, resulting in the class becoming a useless object. So it seems I have to pass a path string instead.
plugins: [
{
resolve: `gatsby-plugin-modal-routing`,
options: {
modalComponentPath: path.join(__dirname, 'src/components/SomeComponent.js')
}
},
However, if I try to pass the path instead, I can't figure out how to use it to load the component inside the plug-in. I've tried using a dynamic Node import (ie. import(path).then(component => ...)) ...
with a path that's path.join-ed with __dirname
with a relative path (src/components/SomeComponent)
with a local-path-relative path (./src/components/SomeComponent)
with and without a trailing .js
I'm not sure if this is some sort of issue with the different paths of the app vs. the plug-in or whether there's some other problem, but using import seems like an un-Gatsby-like solution anyway.
So, then I discovered the loadPage and loadPageSync functions which are passed into the plug-in ... but those failed also. Every path I try results in component coming back ... but it's a "page not found" component (presumably because the component I'm trying to pass in hasn't been added as a page).
This seems like it should be a simple question, at least to anyone who has worked on Gatsby plug-ins before: if you want a plug-in to take a component as an input (either as a function/class or as a string of a path to a module) ... how can you actually use that component in your plug-in?
All I'm looking for is a basic pattern or reference to a line in an existing Gatsby plugin that takes a component, or something simple like that (I can look up any details).
This seems like it should be a simple question
I had the same thought while trying this out myself. Oh boy.
TL:DR
// gatsby-node.js
const { DefinePlugin } = require('webpack')
const path = require('path')
exports.onCreateWebpackConfig = ({ actions }, { componentPath }) => {
actions.setWebpackConfig({
plugins: [
new DefinePlugin({
'___COMPONENT___': JSON.stringify(componentPath)
})
]
})
}
// gatsby-ssr
export const onRenderBody = ({ setPreBodyComponents }) => {
const Component = require(___COMPONENT___).default
setPreBodyComponents([<Component />])
}
Long read
Gatsby config doesn't seem to pass functions around (I could have sworn it used to), so passing a React component directly to your custom plugin is out the window. It has to be a path to your component.
// gatsby-config.js
{
resolve: 'my-custom-plugin',
options: {
componentPath: path.join(__dirname, './my-component.js')
}
}
You didn't say if you're using the component in gatsby-node or gatsby-browser/ssr, but I assume it's the later since requiring stuff dynamically in Node is dead simple:
Gatsby Node
// gatsby-node.js
function consume(component) {
const Component = require(component)
}
...although it doesn't understand JSX or ESM, but that's a different problem.
Gatsby Browser
gatsby-browser/ssr is run with webpack, so the module format is not a problem. But import(componentPath) won't work:
Dynamic expressions in import()
It is not possible to use a fully dynamic import statement, such as import(foo). Because foo could potentially be any path to any file in your system or project.
webpack doc
Ok, I suppose so something like this should work:
// gatsby-browser
import('./my-dir' + componentPath)
Nope, because webpack will try to resolve this from wherever the plugin live, i.e node_modules or plugins directory & we're not about to ask our users to put their custom components in node_modules.
What about this, then?
// gatsby-browser
import(process.cwd() + componentPath) // nope
We're right back at the beginning — webpack doesn't like full dynamic path! And also even if this works, this is a terrible idea since webpack will try to bundle the whole working directory.
Only if we could encode the path as a static string beforehand, so webpack can just read that code — like using webpack.DefinePlugin to define environment variables. Fortunately we can do that in gatsby-node.js:
// gatsby-node.js
const { DefinePlugin } = require('webpack')
const path = require('path')
exports.onCreateWebpackConfig = ({ actions }) => {
actions.setWebpackConfig({
plugins: [
new DefinePlugin({
'___CURRENT_DIR___': JSON.stringify(process.cwd())
})
]
})
}
And finally
// gatsby-browser
// eslint throw error for unknown var, so disable it
// eslint-disable-next-line
import(___CURRENT_DIR___ + componentPath) // works, but don't do this
But since we can access user options right in gatsby-node, let's just encode the whole path:
// gatsby-node.js
const { DefinePlugin } = require('webpack')
- const path = require('path')
- exports.onCreateWebpackConfig = ({ actions }) => {
+ exports.onCreateWebpackConfig = ({ actions }, { componentPath }) => {
actions.setWebpackConfig({
plugins: [
new DefinePlugin({
- '___CURRENT_DIR___': JSON.stringify(process.cwd())
+ '___COMPONENT___': JSON.stringify(componentPath)
})
]
})
}
Back in gatsby-browser.js:
// gatsby-browser
// I pick a random API to test, can't imagine why one would import a module in this API
export const onRouteUpdate = async () => {
// eslint-disable-next-line
const { default: Component } = await import(___COMPONENT___)
console.log(Component) // works
}
Gatsby SSR
For the sake of completeness, let's try the same trick in gatby-ssr:
// gatsby-ssr
export const onRenderBody = async ({ setPreBodyComponents }) => {
// const Component = require(___COMPONENT___).default
const { default: Component } = await import(___COMPONENT___)
setPreBodyComponents([<Component />])
}
...and it failed.
Why? If one's curious enough they might go and dig around Gatsby code to see how gatsby-ssr is treated differently than gatsby-browser, but alas I just don't feel like doing that.
Fear not, we still have one trick up our sleeve. Webpack's require can import module dynamically too, though not asynchronously. Since gatsby-ssr doesn't run in the browser, I couldn't care less about asynchronicity.
export const onRenderBody = ({ setPreBodyComponents }) => {
const Component = require(___COMPONENT___).default
setPreBodyComponents([<Component />]) // works
}
And now it works.
Sharing code between gatsby-ssr & gatsby-browser
Let's just say we need this component in both gatsby-ssr and gatsby-browser — would require(...) works in gatsby-browser too?
export const onRouteUpdate = async () => {
// eslint-disable-next-line
const { default: Component } = require(___COMPONENT___)
console.log(Component) // yes
}
It works.
import(..) vs require()
While import() does load stuff dynamically, it is more of a tool for code-splitting. Here's some different, other than asynchronicity:
using import('./my-dir' + componentPath) will bundle all files inside ./my-dir into a chunk. There's magic comment we can use to exclude/include stuff.
require(...) will just inline the required component into whatever chunk's calling it.
In the getServerSideProps function of my index page, I'd like to use a function foo, imported from another local file, which is dependent on a certain Node library.
Said library can't be run in the browser, as it depends on "server-only" modules such as fs or request.
I've been using the following pattern, but would like to optimize it. Defining foo as mutable in order to have it be in scope is clunky and seems avoidable.
let foo;
if (typeof window === "undefined") {
foo = require("../clients/foo");
}
export default function Index({data}) {
...
}
export async function getServerSideProps() {
return {
props: {data: await foo()},
}
}
What would be the best practice here? Is it somehow possible to leverage ES6's dynamic import function? What about dynamically importing within getServerSideProps?
I'm using Next.js version 9.3.6.
Thanks.
UPDATE:
It seems as if Next.js's own dynamic import solution is the answer to this. I'm still testing it and will update this post accordingly, when done. The docs seem quite confusing to me as they mentionn disabling imports for SSR, but not vice versa.
https://nextjs.org/docs/advanced-features/dynamic-import
When using getServerSideProps/getStaticProps, Next.js will automatically delete any code inside those functions, and imports used exclusively by them from the client bundle. There's no risk of running server code on the browser.
However, there are a couple of considerations to take in order to ensure the code elimination works as intended.
Don't use imports meant for the server-side inside client-side code (like React components).
Ensure you don't have unused imports in those files. Next.js won't be able to tell if an import is only meant for the server, and will include it in both the server and client bundles.
You can use the Next.js Code Elimination tool to verify what gets bundled for the client-side. You'll notice that getServerSideProps/getStaticProps gets removed as do the imports used by it.
Outside of getServerSideProps/getStaticProps, I found 2 fairly similar solutions.
Rely on dead code elimination
In next.config.js:
config.plugins.push(
new webpack.DefinePlugin({
'process.env.RUNTIME_ENV': JSON.stringify(isServer ? 'server' : 'browser'),
}),
);
export const addBreadcrumb = (...params: AddBreadcrumbParams) => {
if (process.env.RUNTIME_ENV === 'server') {
return import('./sentryServer').then(({ addBreadcrumb }) => addBreadcrumb(...params));
}
return SentryBrowser.addBreadcrumb(...params);
};
Note that some for reason I don't understand, dead code elimination does not work well if you use async await, or if you use a variable to store the result of process.env.RUNTIME_ENV === 'server'. I created a discussion in nextjs github.
Tell webpack to ignore it
In next.config.js
if (!isServer) {
config.plugins.push(
new webpack.IgnorePlugin({
resourceRegExp: /sentryServer$/,
}),
);
}
In that case you need to make sure you will never import this file in the client otherwise you would get an error at runtime.
You can import the third party library or a serverside file inside getServerSideProps or getInitialProps since these functions run on server.
In my case I am using winston logger which runs on server only so importing the config file only on server like this
export async function getServerSideProps (){
const logger = await import('../logger');
logger.info(`Info Log ->> ${JSON.stringify(err)}`);
}
You can also import library/file which has default export like this
export async function getServerSideProps(context) {
const moment = (await import('moment')).default(); //default method is to access default export
return {
date: moment.format('dddd D MMMM YYYY'),
}
}