Nuxt3 Robots.txt - #nuxtjs/robots not generating robots.txt file - javascript

I'm building a Nuxt 3 project. I need my build to generate a robots.txt file, just like this package states it does -> https://github.com/nuxt-community/robots-module
After running "nuxt build" and/or "nuxt generate", the robots.txt does not appear in the output or public folders as I'd expect.
I'm definitely missing something and likely being an idiot here.. Does anyone know what I'm missing? Here's my code:
package.json
"dependencies": {
...
"#nuxtjs/robots": "^2.5.0",
}
nuxt.config.ts
target: "static",
runtimeConfig: {
NUXT_STORYBLOK_PRODUCTION_KEY: process.env.NUXT_STORYBLOK_PRODUCTION_KEY,
public: {
CDN: process.env.CDN,
NUXT_STORYBLOK_PREVIEW_KEY: process.env.NUXT_STORYBLOK_PREVIEW_KEY,
NUXT_DOMAIN_NAME: process.env.NUXT_DOMAIN_NAME,
},
},
modules: [
...
"#nuxtjs/robots",
],
robots: {
UserAgent: "*",
Disallow: "",
},
}

I just successfully installed this module on Nuxt 3, and I think there are a couple of things to note here.
The first is that I've never managed to get the module options for any module to work in Nuxt 3 the way you have shown (top-level options, according to the module docs):
modules: [
...
"#nuxtjs/robots",
],
robots: {
UserAgent: "*",
Disallow: "",
}
Have you tried the other options? You can also try using the Robots Config file, or pass the options when declaring the module (from the README.md for the repo):
export default {
modules: [
// Simple usage
'#nuxtjs/robots',
// With options
['#nuxtjs/robots', { /* module options */ }]
]
}
The other thing is that I also do not see a generated robots.txt file anywhere after running the build or dev, but if I go to '/robots.txt' in the dev or build preview, I can see the output working as intended. Have you tried visiting the path?
It looks like something the server generates when the route is visited rather than a static file generated on build by default. I think you can change that in the options, but it's not something I need so I haven't dug into it.
Hopefully, that helps somewhat!

You are using an incompatible version of nuxt robots for your project. Ensure you are using version ^3.0.0. Any versions below this are not compatible with nuxt 3.
npm install #nuxtjs/robots#3.0.0
Then ensure your nuxt.config.ts looks similar to below. As John Overstreet mentioned above the simple method appears flawed.
I created a config folder in the root directory of my project and placed my robots.txt file within it:
export default {
modules: [
['#nuxtjs/robots', { configPath: "~/config/robots.config" }]
]
}
You will also need to delete the robots.txt file from your public folder
Build/Generate your project and go to '/robots.txt'
The file is generated dynamically upon request of the above route.
With mine and John's help above I hope this helps resolve your issues :)

Related

Dependency not found when using package.json exports key

I have built a very simple component library for Vue in Vite and published it to NPM. When consuming components from the lib.js file it works just fine.
However, I now need to be able to import from my library (or rather the NPM package) a single JS file.
In the consuming site however, I am getting an error and I think it's all to do with the setup of my exports key in the package.json
package.json (Library)
{
"name": "#michaelpumo/components-vue",
"version": "0.0.22",
"types": "./dist/types/lib.d.ts",
"files": [
"dist"
],
"main": "./dist/lib.js",
"module": "./dist/lib.js",
"exports": {
".": {
"import": "./dist/lib.js",
"require": "./dist/lib.js"
},
"./tailwind": "./dist/tailwind.config.js"
}
}
In my consuming site, I'm trying to import the Tailwind file like so:
const { units, config } = require('#michaelpumo/components-vue/tailwind')
console.log(units) // This outputs correctly, so the require worked.
module.exports = { config, units }
This actually logs the contents of 'units' from the Tailwind file, so I know it's been found correctly.
However, in the terminal, I get an error:
ERROR Failed to compile with 1 errors
This dependency was not found:
* #michaelpumo/components-vue/tailwind in ./tailwind.config.js
I'm not sure what is going wrong here.
Here is the structure of my NPM package:
It might be worth mentioning that the consuming site is a Nuxt.js site but I'm not sure that makes any difference here.
Any help greatly appreciated.

How to add import shortcuts - alias

I've created a brand new react app
create-react-app demo
I need to create alias for some directories/components like:
import { Header } from '#uicomponents'
Where #uicomponents is shortcut for the path 'src/hello/this/is/the/path/to/ui/components '
All online tutorials telling that I can use alias imports using resolve.alias, but I'm wondering how to do this with brand-new react app?
Theres is no .babelrc or webpack.config.js files.
Any help please?
If you haven't ejected your application from CRA, then you can alias your source directory with using NODE_PATH in your .env file. NODE_PATH=/src/hello/this/is/the/path/to/ui/components.
If you alias without ejecting, it won't allow you to do the #uicomponents or have multiple aliases. Here's an issue on GitHub that talks about it and the relevant CRA page on environment variables.
If you have ejected you can set the alias in the Webpack configuration files and be able to import like you want:
...
resolve: {
alias: {
'#uicomponents': '/src/hello/this/is/the/path/to/ui/components'
}
},
...
UPDATED:
I recommend you to use Craco.
It allows you to customize webpack / babel / any other tool that used in react-scripts internally.
Webpack and Jest aliases is not an exception.
And recently I created a plugin called craco-alias specially for these purposes.
Links: GitHub, npm.
This plugin generates webpack and jest aliases for you automatically.
You just need to install Craco, write a small config for it, then create (or edit) your jsconfig.json / tsconfig.json (it works with TS as well) and specify aliases source in craco-alias config object.
It's easy - you can find all examples on README page.
Of course it works with every command (npm start, npm test, run build build) and allows to pass any CLI arguments.
But, the only problem is that plugin only supports path aliasing, and it doesn't work with module aliases properly.
I hope it will help somebody :)
The alias solution for craco or rewired create-react-app is react-app-alias for systems as: craco, react-app-rewired, customize-cra
According docs of mentioned systems replace react-scripts with craco in package.json and configure next:
// craco.config.js
const {CracoAliasPlugin} = require('react-app-alias')
module.exports = {
plugins: [
{
plugin: CracoAliasPlugin,
options: {}
}
]
}
Configure aliases in json like this:
// tsconfig.paths.json
{
"compilerOptions": {
"baseUrl": ".",
"paths": {
"example/*": ["example/src/*"],
"#library/*": ["library/src/*"]
}
}
}
And add this file in extends section of main typescript config file:
// tsconfig.json
{
"extends": "./tsconfig.paths.json",
// ...
}

Raven doesn't use source maps in Sentry for TS/JS code

I already used Sentry in a C# project and was very pleased about it. So I also want to use it in a new NodeJS project with TypeScript. Sadly, the required sourcemaps doesn't work fine here. I first tried it in a plain TS project without success. Even in a plain raw JS project it doesn't do any source code mapping.
For the test project, I exactly followed the documentation on node/sourcemaps:
https://docs.sentry.io/clients/node/config/
https://docs.sentry.io/clients/node/sourcemaps/#webpack
https://docs.sentry.io/clients/node/typescript/
The result is always the same, Sentry shows me the ugly code without using sourcemaps:
It seems that Sentry doesn't use the sourcemaps for unknown reasons cause the corresponding Release contains the required sourcemap:
I spend a lot of hours on this. Tried several things, including different paths/file names for the files pushed to sentry. I also used the Sentry cli tool directly, to make sure, that there is no issue with the webpack plugin. Nothing works, Sentry always shows me the raw source code and ignores the sourcemap.
The following files are from my second nodejs program using plain raw JavaScript. It's as minified as possible to work. I don't know whats wrong here cause I did exactly what the documentation requires.
What is the problem?
Sourcemaps were generated and uploaded. First I thought that the file names/paths doesn't match. This older question shows a similar issue, where the paths doesn't match since they were not relative using ~/ but in SentryPlugin this is the default prefix. On the other side, dataCallback make sure that all paths are relative to the project root, instead of having absolut paths like /home/user/project/dist/app.js which would be the default behavior. I did a console.log there to see that they're correctly relative.
Simple NodeJS test project using plain raw JavaScript
src/app.js (Main programm)
var Raven = require('raven')
const path = require('path')
let options = {
release: process.env.RELEASE,
dataCallback: function (data) {
var stacktrace = data.exception && data.exception[0].stacktrace;
if (stacktrace && stacktrace.frames) {
stacktrace.frames.forEach(function (frame) {
if (frame.filename.startsWith('/')) {
frame.filename = 'app:///' + path.basename(frame.filename);
}
});
}
return data;
}
}
Raven.config('http://<DnKey>#<SentryHost>:9000/3', options).install();
//Raven.captureException(new Error('abc'))
throw new Error('Hello123')
webpack.config.js
In this file, I changed the entry path from src/app.js to ./src/app.js. The Sentry docs use ./src/app.js but this result in an error that the path cannot be resolved. It appears to be an mistake of the docs.
const path = require('path')
const fs = require('fs')
const SentryPlugin = require('#sentry/webpack-plugin')
module.exports = {
target: 'node',
devtool: 'source-map',
entry: {
"app": './src/app.js'
},
output: {
path: path.join(__dirname, 'dist'),
filename: '[name].js'
},
plugins: [
new SentryPlugin({
release: process.env.RELEASE,
configFile: 'sentry.properties',
include: './dist',
ignore: ['node_modules', 'webpack.config.js'],
})
]
};
sentry.properties
I added the configFile attribute since SentryPlugin uses internally the CLI of Sentry. The CLI needs to know some information like url or auth-token from Sentry. This could be passed as environment variables, but I like the approach of using a config file, which is completely valid according to the docs. Token has project.write permission as required.
defaults.url=http://SentryHost:9000
defaults.org=sentry
defaults.project=js-test
auth.token=xxx
package.json
{
"name": "sentry-js-test",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"build": "rm -rf dist/* && webpack --mode production"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"#sentry/cli": "^1.30.2",
"#sentry/webpack-plugin": "^1.3.3",
"raven": "^2.4.2",
"webpack": "^4.0.1"
},
"devDependencies": {
"webpack-cli": "^2.0.10"
}
}
Start the test programm
export RELEASE=3.0.20 && npm run build && node dist/app.js
This use webpack to minify src/app.js and place the minified version in dist/app.js. Furthermore, it creates a sourcemap called dist/app.js.map.
On every run, I changed RELEASE (e.g. increment to 3.0.21). That's important since Sentry assumes, that releases and source files (also sourcemaps) are a 1:1 relation. So if we upload our files to release with version 3.0.20 this is done on the first time. On the second time, the files keeps unchanged cause the cli sees that they're already existing. Using a new version make sure that our latest changed files are always uploaded and we don't mess with old ones from playing around.
I spend a lot of hours on this issue and have no Idea what I'm doing wrong. Summarized, I did every step from the tutorials of the documentation, with some small fixes on obvious errors.
Information about the environment
I'm using self hosted Sentry 8.22 in docker containers on Arch Linux. NodeJS is used in Version 9.7.1.
After months struggling with that we finally found the issue.
You can see that the artifacts are here with the sourcemaps, that you correctly tag the versions and everything seems fine?
Running Sentry on your own I bet? We are too.
Well that's the issue.
Sentry is divided into different services and in order to have them sharing the data (sourcemaps...) you need to share a volume between them. Please read that post where it's correctly explained.
Also, if you prefer, you can host them on S3 and Sentry can work with that too!

Testing TypeScript modules with package dependencies

I have a class written in TypeScript:
import * as uuid from "uuid";
export class IdGenerator {
getId() {
return uuid.v4();
}
}
This has a dependency on the uuid package that I have installed with npm.
I deploy this with other code written in TypeScript to the browser using webpack. Dependencies are resolved, all is well.
I want to test the class, so I write the following test:
import { IdGenerator } from "../src/IdGenerator";
describe("An id generator", () => {
const idGenerator = new IdGenerator();
it("generates an id", () => {
expect(idGenerator.getId()).not.toBeNull();
});
});
I use Chutzpah with the following configuration:
{
"Framework": "jasmine",
"TestHarnessReferenceMode": "AMD",
"TestHarnessLocationMode": "SettingsFileAdjacent",
"EnableTestFileBatching": true,
"References": [
{
"Path": "node_modules/requirejs/require.js",
"IsTestFrameworkFile": true
}
],
"Compile": {
"Mode": "External",
"Extensions": [ ".ts" ],
"ExtensionsWithNoOutput": [ ".d.ts" ]
},
"Tests": [
{ "Path": "Components/test" }
]
}
When trying to test with Chutzpah, I get the following error:
Unhandled exception [...] in [...] /node_modules/requirejs/require.js [...] Script error for "uuid", needed by: Components/src/IdGenerator
So I add a reference to the path in chutzpah.json, which gets rid of this error but raises an error for one of uuid's dependencies. I add a reference for the dependency, then get a further dependency error, and so on.
Ideally I would like all dependencies to be resolved in the same way as they are with the bundle deployed to the browser.
Should I abandon the idea of trying to test my TypeScript files with package dependencies in this way and instead look into using dependency injection and mock the package dependencies in the tested TypeScript files? Perhaps also create separate JavaScript tests for the bundle? Or is there another approach that will allow the testing of my TypeScript code with the package dependencies?
This looks like it won't work since requireJS and Node are not compatible in this way. See here. From that answer:
It is not possible to use RequireJS on the client (browser) to access files from node_modules. Files under node_modules must first be copied to a location that is accessible (under the public folder) before the client can access them. The documentation says RequireJS can access Node modules, but this only works for server-side JavaScript (when you want to use RequireJS-style module syntax in Node).
To use your config module in the client app, you must first transform it into a RequireJS-compatible module and copy it under public. This article explains how to automate this with package managers and build tools, and contains the quote that finally fixed my broken understanding of RequireJS + Node:
In order to use node modules on the browser you will need a build
tool. This might bother you. If this is a deal breaker then by all
means continue wasting your life copying and pasting code and
arranging your script tags.

WebStorm settings for local modules import/require

Is it possible to configure where WebStorm is looking for modules without adding the project itself as a library in the settings?
I have added the following part to webpack and compiling works fine, but I also need code completion
resolve: {
root: [
path.resolve('./asset/js')
]
},
Now I can change something like this:
import { ViewContainer } from './../../application';
to:
import { ViewContainer } from 'application';
But with this I have no code completion and adding /asset/js to the Library seems not to work. It also disables checking for errors on this directory.
WebStorm gives me the warning that no module 'application' is found in package.json
package.json only accept npm modules or git repositories
Do you have any solutions for this problem?
Right click on the directory and go to 'Mark Directory as' and select 'Resource Root'

Categories

Resources