Develop Reference

A single payload.js for all nuxt static generate'd routes when data is the same

I have a NuxtJS site with only one page /pages/matrix/index.vue but quite a lot of dynamic routes pointing to this page, each route using the same set of data. When generating a static build for deployment on Netlify, the dist folder currently reaches ~1.2 GB, consisting of
3125 .html files in dist/matrix (occupying ~39% of the space)
3125 folders for payload.js files in dist/_nuxt/static/[random]/matrix/ in subfolders for routes (occupying ~61% of the space)
Those 61% are 3125 copies of a 220kB payload.js with exactly the same set of data: [{}], while only the route changes:
__NUXT_JSONP__("/matrix/place2/time3,time14,time29", (function(a, b, ...) {
return {
data: [{ /* data does not change */ }],
fetch: {},
mutations: void 0
}
}("nor", "does", "this")));
I wonder if there is a way to reduce this redundancy by somehow extracting the data part? Reducing ~665 MB to just 220kB sounds alluring.
Some more background:
Routes are /matrix, /matrix/place1 or /matrix/place8/time1,time7,time18. When generating, I pull all data from a headless CMS and feed it to my page component via the payload option. First, I used File System Routing and imported the pages/matrix/index.vue like this:
// pages/matrix/_places/index.vue
<script>
import Index from '../index'
export default Index
</script>
which felt wrong but worked. I blamed this approach to the "duplication" of those payload files (frankly without completely understanding the mechanics of static generation). I now switched to extendRoutes with this nuxt.config.js setting:
router: {
extendRoutes (routes, resolve) {
routes.push(
{
name: 'matrix-place-times',
path: '/matrix/:place/:times',
component: resolve(__dirname, 'pages/matrix/index.vue')
},
{
name: 'matrix-place',
path: '/matrix/:place',
component: resolve(__dirname, 'pages/matrix/index.vue')
}
)
}
}
The amount of payload files spread across route subfolders stays the same.
Any advice on this? Using Nuxt v2.15.7.

Few things I can think about:
using SSR would solve some of those issues (you told that you wish to stay on Netlify tho)
using some aliased routes may be a good idea, especially if you have the exact same data at 2 differently named endpoints
this video also gives some leads regarding ISG or other ways to have more flexibility towards big page amounts
Nuxt3 is able to run on Cloudflare workers, this is not SSR nor exactly ISG but a middle ground, a different approach but it could render pages for not so expensive while being quick

How to create multiple node types with mdx in Gatsby?

I am trying to create multiple content types in Gatsby using mdx (instead of remark). I am having trouble sifting through old methods of handling remark and new methods with mdx. I have seen multiple instances of allXYZ root nodes being queried in various tutorials, but I am having trouble understanding how those were created in the first place. I know in some cases Gatsby or plugins create these nodes (allMdx, allMarkdownRemark, allFile, etc), but I want to learn how to create something like allPosts or allProjects with relevant field nodes for that content type myself.
My end goal is to achieve the following:
Have mdx content stored in different folders (posts, projects, pages) that indicate their content type.
Have a matching folder structure in src/pages with an index.js file in each (the landing page for posts, projects, or pages) and a template file using the new syntax {type.field}.js.
Be able to query allPosts / allProjects / allPages in the respective index.js files and use those types for the template file ({Post.slug}.js, {Project.slug}.js, etc)
The template file would query the child node (post, project, page) the same way that allMdx and mdx are used in the basic tutorial.
Not need to tap into the createPages hook in gatsby-node.js because the gatsby-source-filesystem should do it for me with the above structure.
I found this stackoverflow post that posed a similar question, but the answer seems to imply these custom nodes (allPosts, etc) should be automatically created when you setup the plugin options like this:
{
resolve: `gatsby-source-filesystem`,
options: {
name: `posts`,
path: `${__dirname}/content/posts`,
},
},
That does not work for me however. When I use the __graphql interface, these nodes don't exist, and if I try to query them anyways, I get an error (note: I have tried naming the template file using projects, Project, project, etc. without success):
PageCreator: Tried to create pages from the collection builder.
Unfortunately, the query came back empty. There may be an error in your query:
Cannot query field "allProjects" on type "Query".
File: src/pages/projects/{projects.slug}.js
I also found this Gatsby guide which seems to address part of my question, but I don't understand how to source my data locally instead of through the API requests they are using. I also think this might be overcomplicating something very simple that should work natively with the mdx and filesystem plugins? Not sure!
I feel like I am missing something very basic. I am still new to Gatsby, so it's completely possible I am wrong in thinking this will work how I want it to, but I have spent hours trying to figure this out and think it's time to finally ask for help lol.
Any advice is appreciated!!

You can try this Gatsby plugin https://www.gatsbyjs.com/plugins/gatsby-plugin-mdx-source-name/?=mdx
When do I use this plugin?
This plugin is very useful if you are using multiple instances of gatsby-source-filesystem as it will allow you to query the name field from the source plugin on your Mdx nodes.
plugins: [
`gatsby-plugin-mdx-source-name` ,
{
resolve: `gatsby-source-filesystem` ,
options: {
path: `${__dirname}/src/blog` ,
name: `blog` // this name will be added to the Mdx nodes
}
}
]
The source name will now be available to query via GraphQL:
const query = graphql`
query {
allMdx(){
nodes {
id
fields {
source
}
}
}
}`
For example, if you wanted to filter by this new source field on page creation, you could do the following:
// gatsby-node.js
exports.createPages = ({actions, graphql}) => {
const {createPage} = actions
// query for all Mdx pages
const query = graphql(`
query {
allMdx(){
nodes {
fields {
source
}
frontmatter {
slug
}
}
}
}
`)
return query.then(result => {
// filter by source name "blog"
const posts = result.data.allMdx.nodes.filter(node => node.fields.source === 'blog')
posts.forEach(node => {
createPage({
path: `/blog/${node.frontmatter.slug}`,
component: path.resolve('src/templates/blog-post.js'),
context: {
slug: node.frontmatter.slug
}
})
})
})
}

You create your own custom types using onCreateNode
https://www.gatsbyjs.com/docs/reference/config-files/gatsby-node/#onCreateNode
https://www.gatsbyjs.com/docs/reference/graphql-data-layer/schema-customization/
My solution is to use fileAbsolutePath to determine whether the .mdx file is a page or post etc. based on the folders that my files are located in.
if (node.internal.type == 'Mdx' && node.fileAbsolutePath.indexOf('pages') !== -1) {
actions.createNode({
id: createNodeId(`MdxPage-${node.id}`),
parent: node.id,
internal: {
type: `MdxPage${node.internal.type}`,
contentDigest: node.internal.contentDigest,
},
})
}

Another option is doing the folder structure as you described and than using allMdx to filter based on fileAbsolutePath:
allMdx(
sort: { fields: frontmatter___date, order: DESC }
filter: { fileAbsolutePath: {regex: "/content/projects/"} }
)
or if you'd want to have multiple calls to allMdx in the same query that using an alias:
projects: allMdx(
filter: { fileAbsolutePath: {regex: "/content/posts/"} }
) { ... }
posts: allMdx(
filter: { fileAbsolutePath: {regex: "/content/projects/"} }
) { ... }

Gatsby Site Requires Refresh to View New Data

I have a blog run on Gatsby, and every time I push and deploy new blog posts I need to do a refresh on my page to see the new data.
I tried following the suggestions from this post and adding an onServiceWorkerUpdate function but that doesn't seem to have done anything.
Anyone have workarounds to this issue, and if so will there be a way to test locally? Changes already automatically update when I test in gatsby develop mode.
This is the entirety of my gatsby-browser.js file
export const onServiceWorkerUpdateReady = () => window.location.reload();

You need to install gatsby-plugin-offline first. Leaving your gatsby-config.js with something similar to:
{
plugins: [
{
resolve: `gatsby-plugin-manifest`,
options: {
...
}
},
'gatsby-plugin-offline'
]
}
Note: plugin's order matters in this case.
The plugin will register a service worker and will load it into the client.
Then, in your gatsby-browser.js file you can simply add:
export const onServiceWorkerUpdateReady = () => {
const answer = window.confirm(
`This application has been updated. ` +
`Reload to display the latest version?`
)
if (answer === true) window.location.reload()
}
Additionally, you may want to add the following hack (very common across the repositories):
export const onRouteUpdate = () => {
navigator.serviceWorker.register('/sw.js').then((reg) => {
reg.update();
});
};
Basically, it forces the registration of the service-worker across the site upon the onServiceWorkerUpdateReady refresh.
Check this interesting thread for further caveats and workarounds for some specific scenarios: https://github.com/gatsbyjs/gatsby/issues/9087#issuecomment-774680408

It will not work using Link provided by gatsby
import { Link as GatsbyLink } from 'gatsby';
<GatsbyLink to={to} {...props} />
It works using traditional a tag:
<a href={props.to} target="_blank">
{props.children}
</a>

warn Non-deterministic routing danger: Attempting to create page: [...] already exists

I finished writing a small blog application using Gatsby and React.
Everything works fine when I try it locally. So I proceed with gatsby build and deploy the build folder into Netlify. However, after the deployment, the content of some of the pages is not shown despite locally everything works fine.
Description of the problem: I have a navbar with "Home", "Healthcare", "Technology", "Robotics", "Posts", "NewsLetter", and every time a user clicks on for example "Robotics" a series of posts from that category is shown.
Now locally everything works fine but as soon as I deploy I can only see "Posts" page which carries all the posts. The other choices from the navbar are not rendering the other posts categories.
Below the error I am getting from the terminal:
warn Non-deterministic routing danger: Attempting to create page: "/healthcare/", but page "/healthcare" already exists This could lead to non-deterministic routing behavior
warn Non-deterministic routing danger: Attempting to create page: "/robotics/", but page "/robotics" already exists This could lead to non-deterministic routing behavior
warn Non-deterministic routing danger: Attempting to create page: "/technology/", but page "/technology" already exists This could lead to non-deterministic routing behavior
This leads me to think that some pages are not rendered at the proper time, however, this does not explain the difference between localhost perfectly working and the deployed version not working properly.
Below my gatsby-node.js
const path = require('path')
// create pages dynamically
exports.createPages = async ({ graphql, actions }) => {
// from actions we can destructure createPage
const { createPage } = actions
// below we will be running two queries at the same time
// instead of one query only
const result = await graphql(`
{
allMdx {
nodes {
frontmatter {
slug
}
}
}
category: allMdx {
distinct(field: frontmatter___category)
}
}
`)
result.data.allMdx.nodes.forEach(({ frontmatter: { slug } }) => {
createPage({
path: `/posts/${slug}`,
component: path.resolve(`src/templates/post-template.js`),
context: {
slug,
},
})
})
result.data.category.distinct.forEach(category => {
createPage({
path: `/${category}`,
component: path.resolve(`src/templates/category-template.js`),
context: {
category,
},
})
})
}
Below is also the file post-template.js
const PostTemplate = ({data}) => {
const {
mdx: {
frontmatter: {title, category, image, date},
body,},
} = data;
return (
<Layout>
<Hero/>
<Wrapper>
{/* post info */}
<article>
<Image fluid={image.childImageSharp.fluid} />
<div className="post-info">
<span>{category}</span>
<h2>{title}</h2>
<p>{date}</p>
<div className="underline"></div>
</div>
<MDXRenderer>{body}</MDXRenderer>
</article>
{/* banner */}
<article>
<Banner/>
</article>
</Wrapper>
</Layout>
)
}
export const query = graphql`
query GetSinglePost($slug: String) {
mdx(frontmatter: {slug: {eq: $slug}}) {
frontmatter {
title
category
date(formatString: "MMMM Do, YYYY")
image {
childImageSharp {
fluid {
...GatsbyImageSharpFluid
}
}
}
}
body
}
}
`
const Wrapper = styled.section`
// style goes here ...
`
export default PostTemplate
and the file category-template.js
const CategoryTemplate = props => {
console.log(props);
const {
pageContext: { category },
} = props;
const {
data: {
categories: {nodes:posts}
}
} = props;
return (
<Layout>
<Hero/>
<Posts posts={posts} title={`category / ${category}`}/>
</Layout>
)
}
export const query = graphql`
query GetCategories($category: String) {
categories: allMdx(sort: {fields: frontmatter___date, order: DESC}, filter: {frontmatter: {category: {eq: $category}}}) {
nodes {
excerpt
frontmatter {
title
author
category
date(formatString: "MMMM, Do, YYYY")
slug
readTime
image {
childImageSharp {
fluid {
...GatsbyImageSharpFluid
}
}
}
}
id
}
}
}
`
export default CategoryTemplate
Below the structure of the robotics page component (all the other page components healthcare and technology have the same exact structure so I am only including one)
robotics.js
const robotics = () => {
return (
<Layout>
<SEO title="Robotics"/>
<Hero />
<h2>robotics page</h2>
</Layout>
)
}
export default robotics
In order to solve this problem I did a lot of research but before I would like to pint out that I already clean and redeploy the same application a couple of times before. So what I tried so far:
sudo gatsby clean && sudo gatsby develop after it was working I did sudo gatsby build
After deploying the build folder some pages were not rendering properly so I:
sudo m -rf node_modules sudo rm -rf public/ and sudo rm -rf package-lock.json
and procedeed with:
sudo npm install (if necessary you have to do sudo nom install --unsafe-perm) and
sudo gatsby develop && sudo gatsby build
But unfortuantely the same exact outcome: localhost everthing works fine and on Netlify some pages were not rendering at all.
So I did additional research and I studied this post which represents the same exact problem I have. Unfortunately no answer was accepted. But I tried to apply the suggestion and in fact on my gatsby-node.js I had:
exports.createPages = async ({ graphql, actions, reporter }) => {
const yourQuery= await graphql(
`
{
parameters here
}
`
if (yourQuery.errors) {
reporter.panicOnBuild(`Error while running GraphQL query.`);
return;
}
But nothing happened.
I studied this and this too. This post says that "MDX pages cant currently be in the same directory as JSX pages" but I am not sure this is accurate and did not see it in the official documentation an example showing not to do that.
Please guide me to a potential solution on this point.

Your issue relies on the naming, and I assume that it will affect all pages (robotics, healthcare, technology, etc). This:
const robotics = () => {
return (
<Layout>
<SEO title="Robotics"/>
<Hero />
<h2>robotics page</h2>
</Layout>
)
}
export default robotics
Should be:
const Robotics = () => {
return (
<Layout>
<SEO title="Robotics"/>
<Hero />
<h2>robotics page</h2>
</Layout>
)
}
export default Robotics
Notice the capitalized component (Robotics). This is because, in React, all component names must be capitalized otherwise, React's assignation will never work.
The rest of the code, without a CodeSandbox it's difficult to check but it looks good.
In addition, something that can also lead to this behavior is the use of plugins such as gatsby-plugin-remove-trailing-slashes, so double-check that if you are using it.
Outside the scope of the question (and because of sudo gatsby develop && sudo gatsby build) it seems that you don't know the difference between gatsby develop and gatsby build, they are not complementary and it doesn't make sense to use that command.
To summarize, gatsby build creates a version of your site with production-ready optimizations like packaging up your site’s config, data, and code, and creating all the static HTML pages that eventually get rehydrated into a React application.
gatsby develop just runs a server in the background, enabling useful features and it is optimized for rapid feedback and extra debugging information. It doesn't bundle and compile your code production environment, potentially hiding some issues that will appear in Netlify, since it runs gatsby build.
With that command, you are developing your site and quitting the process by running the gatsby build, without serving it.
So, to double-check your site before pushing it to Netlify, I would suggest running a gatsby build && gatsby serve locally, since, if the Node versions in Netlify and locally are the same, the project must behave equally in both environments.
You can read the Gatsby docs for further explanations.
I did additional research and I studied this post which represents the
same exact problem I have. Unfortunately no answer was accepted.
I answered that question. The error output may be the same but it's caused but a completely different use-case. In that case, the issue relies on a wrong/unfinished promise approach, and it only affects the dynamic pages, in your case are the opposite. In your scenario, the "static" pages are the problem due to bad naming because React can't resolve the component assignation.

Gatsby JS - Multiple pages pulling in markdown files with different categorys

I am using GatsbyJS to build a personal webpage. I have multiple pages, the two relevant ones are a projects/portfolio page and a blog page.
I have setup gatsby so on the blog page, it pulls in markdown files from a specific folder, and displays them using a template on the blog page. I wanted to display my projects in a similar fashion on the projects/portfolio page.
My folder structure is as follows:
-src
-pages
-BlogPostOne
-BlogPostTwo
projects.js
blog.js
-templates
BlogPostTemplate.js
ProjectTemplate.js
-projects
-project1
-project2
I want the projects page to grab the project markdown files from the projects folder and display them using the project template
The blog page grabs the blog post markdown files from the pages folder and displays them using the blog post template, this works fine.
I basically used the same code as I did to grab the blog post files with different variable names and paths but it doesn't work. Is it possible to even do this with Gatsby? I have searched through their documentation and can't find anything similar to what I am trying to do. Does anyone have any experience doing a similar thing with Gatsby?

Yes, this is totally possible.
The solution is actually very simple, but requires a little understanding of Gatsby's internals to figure out. See this snippet on GatsbyCentral if you already know Gatsby a bit.
Otherwise, here's a more verbose explanation.
In your gatsby-node.js file, you need to add this code:
exports.onCreateNode = ({ node, boundActionCreators, getNode }) => {
const { createNodeField } = boundActionCreators;
if (_.get(node, "internal.type") === `MarkdownRemark`) {
// Get the parent node
const parent = getNode(_.get(node, "parent"));
// Create a field on this node for the "collection" of the parent
// NOTE: This is necessary so we can filter `allMarkdownRemark` by
// `collection` otherwise there is no way to filter for only markdown
// documents of type `post`.
createNodeField({
node,
name: "collection",
value: _.get(parent, "sourceInstanceName")
});
}
};
Ensure you also have the required require() statement for lodash:
const _ = require("lodash")
Now ensure that you have two plugin sections in gatsby-config.js for both blog posts and projects. Ensure that each one has a name option, something like:
plugins: [
{
resolve: "gatsby-source-filesystem",
options: {
name: "pages",
path: `${__dirname}/src/pages`
}
},
{
resolve: "gatsby-source-filesystem",
options: {
name: "projects",
path: `${__dirname}/src/projects`
}
},
Then you can query on the allMarkdownRemark collection and filter for the field collection. It will either be pages or projects.
An example query might look like:
query {
allMarkdownRemark(filter: {fields: {collection: {eq: "pages"}}}) {
...
Hope that's helpful.