I've got a nodejs web project running with express-js and ejs. After making it, it appered to me that it can be host throught static html files on Netlify instead of an nodejs app on Heroku. Indeed, the data only change each month so I only have to build it once a month. Like that, it will be the best hosting option regarding the price.
Is there a way (npm package, GitHub action, ...) to compile an entire express server + ejs application into a folder of static html/css files in order to be hosted on Netlify ?
I've been looking for a while and I couldn't find anything solving my problem.
Thanks for your help.
You can loop through all of the routes, and execute app.render for each of them, then store results in a file.
Sample code:
//express server should be started before this
const fs = require('fs')
['/', '/about', '/contact'].forEach( path => {
app.render(path, {
// optional metadata here
}, (err, res) =>{
if (err)
console.log('Error rendering ' + path, err)
else {
fs.writeFile(__dirname + '/public/' + path + '.html', res, function(err, res) {
if (err)
console.log('error saving html file', path, err)
})
}
})
})
Related
Background
I am migrating an Angular app in GKE cluster. The base docker image that I must use(company policy) does not have any options to install any new softwares like shell, Angular cli command ng etc. The base docker image has only Node installed.
There is a shared base url, let's say, www.my-company.com, that everyone has to use for app deployment with a path added after the base url like www.my-company.com/my-angular-app/ - all the other Angular apps must be differentiated using the path of the app.
What I did
Since I can't run ng serve command in the base image, I added Express dependency in the package.json in Angular application and created an express server to route the traffic to Angular app.
I was following this youtube video to configure the application - https://www.youtube.com/watch?v=sTbQphoYbK0&t=303s. The problem I am facing is to how I load the the static files in the application.
If I define absolute path inside sendFile method of server.js file, although the application is working, but in future, if I need to add any other files in the application, I have to create another route in server.js file.
I don't know how Express can search a file automatically from the static folder(and sub folders) and return only that file when needed. I defined a static folder too, but seems like it is not working.
Following is my server.js code
==============================
const express = require('express');
const http = require('http');
const path = require('path');
const port = 8080;
const contextPath = '/my-angular-app';
const router = express.Router();
const app = express();
app.use(contextPath, router);
app.listen(port, ()=> {
console.log("Listening on port: ", port);
});
app.use(express.static(__dirname + '/dist/testapp/'));
router.get('/', function(req, res) {
// to get index.html file
res.sendFile(path.resolve(__dirname + '/dist/testapp/index.html'));
});
router.get('/*', function(req, res) {
let path = __dirname +'/dist/testapp/' + req.path
console.log('full path: ', path);
// To return static files based on incoming request, I am facing problem here(I think)
res.sendFile(path);
});
==============================
I want Express will send any files based on file name in the request. It should also take care of nested directories in the /dist/testapp/ directory
/dist/testapp/ -> This is the directory where Angular generates code for my app after I execute ng build command
WEBAPP.get("/admin/script.js", (req, res) => {
console.log(req.path);
if (req.session.username !== "Admin") return res.render("error");
res.sendFile(__dirname + "/admin/admin.js")
});
WEBAPP.get("/admin", (req, res) => {
if (!req.session.loggedin) return res.render("error");
if (req.session.username !== "Admin") return res.render("error",);
res.render("admin", {
csrfToken: req.csrfToken(),
title: "ADMIN PORTAL",
username: req.session.username,
nav_avatar: GetImageURL(req.session.avatar, "small")
});
});
There's no need to publically share /admin/script.js in my case but if a user requests this URL say example.com/admin/script.js a check for username equaling "Admin" if all is okay we sendFile.
I would maybe assume that you're not properly targeting your static files. Perhaps console.log the target.
Expressjs's "express.static()" prevents directory/path traversal by default but I thought Nodejs does not have any protection towards directory/path traversal by default?? Recently trying to learn some web development security(directory/path traversal) and I created this:
const http = require("http");
const fs = require("fs");
http
.createServer(function (req, res) {
if (req.url === "/") {
fs.readFile("./public/index.html", "UTF-8", function (err, data) {
if (err) throw err;
res.writeHead(200, { "Content-Type": "text/html" });
res.end(data);
});
} else {
if (req.url === "/favicon.ico") {
res.writeHead(200, { "Content-Type": "image/ico" });
res.end("404 file not found");
} else {
fs.readFile(req.url, "utf8", function (err, data) {
if (err) throw err;
res.writeHead(200, { "Content-Type": "text/plain" });
res.end(data);
});
}
}
})
.listen(3000);
console.log("The server is running on port 3000");
to simulate directory/path traversal security vulnerability but I tried to use "../../../secret.txt" and when I check "req.url", it shows "/secret.txt" instead of "../../../secret.txt" and I also tried using "%2e" & "%2f", it still doesn't work, I still can't get "secret.txt"
(My folder Structure)
- node_modules
- public
- css
- style.css
- images
- dog.jpeg
- js
- script.js
index.html
- package.json
- README.md
- server.js
- secret.txt
As per the documentation of express.static [1], which leads to the docs of the serve-static module [2], the directory you provide is the root directory, meaning it's intentionally made impossible to access anything outside of it.
To serve static files such as images, CSS files, and JavaScript files, use the express.static built-in middleware function in Express.
The function signature is:
express.static(root, [options])
The root argument specifies the root directory from which to serve static assets. For more information on the options argument, see express.static. [3]
[1] https://expressjs.com/en/starter/static-files.html
[2] https://expressjs.com/en/resources/middleware/serve-static.html#API
[3] https://expressjs.com/en/4x/api.html#express.static
Not related, but fyi: the path you're providing to fs etc. is relative to where the script is called from.
For example, if you call node server.js from the root folder of the application, the path "./public/index.html" should work fine, but if you're calling it from a different path, it will fail, e.g. node /home/user/projects/this-project/server.js.
Thus, you should always join the path with __dirname, like so:
+const path = require("path");
-fs.readFile("./public/index.html", "UTF-8", function (err, data) {
+fs.readFile(path.join(__dirname, "./public/index.html"), "UTF-8", function (err, data) {
}
this makes the path relative to the directory of the current file you're trying to access it from, which is what you expect.
I am attempting to create a basic webserver with Node.JS, but running into an issue with properly serving images.
var server = http.createServer(function (request, response){
if(request.url === '/') {
fs.readFile('public/index.html', 'utf8', function(errors, contents){
response.write(contents);
response.end();
});
} else {
fs.readFile("public" + request.url, 'utf8', function(errors, contents){
if (!errors) {
response.end(contents);
} else {
console.log('Failed to read file: /public' + request.url);
response.writeHead(404);
response.end();
}
});
}
});
Everything works fine, apart from if you go to view the image it attempts to download it (which I believe is corrupted - cant open it), which is not what I want, I wish for the images to be served properly in the browser (not via a tag)
Bonus points: I need to be able to correctly give the proper headers (Do I need just a switch statement and set them via that?), as chrome is giving off warnings
Resource interpreted as Stylesheet but transferred with MIME type text/plain: "http://localhost/css.css".
You can use expressjs : http://expressjs.com/
const express = require('express')
const app = express()
app.get('/', (req, res) => res.sendFile('public/index.html'))
app.use(express.static('public'))
app.listen(8080, () => console.log('Example app listening on port 8080!'))
NB :
Your code is unsafe : "public" + request.url. Your users can get all files of your server : fs.readFileSync("public/../../../README.txt");
You will get error on parent folder.
Error: ENOENT: no such file or directory, open 'C:\README.txt'.
So I'm very new to node.js and javascript, and i made a server that works great by loading up an html file on request. This html file does not contain any of it's own data, it simply sources from the internet and displays some images and text i wrote. I've decided to make the site play an audio file when it is opened. I know this is done easily with the <audio> tag in html5 but the src="" lets me take a file from the computer and place it there, of course when i open the site from another computer the file obviously isn't found and thus isn't played. I figure the audio file must be kept as a variable on the server and passed into the html file's <audio src= > tag. How do i do this? It is an .mp3(but i can get it to any other audio format) file about 30 seconds long. I just want it to play when the site is loaded from another computer(over the internet). Also how would i go about doing the same with pictures or any other data that i don't want to source from the internet but rather keep as data in my server?
var http = require('http');
var fs = require('fs');
var simpleServer = http.createServer(function(request, response){
response.writeHead(200, {"Content-Type":"text/html"});
fs.readFile('./Picture.html', null, function(error, data){
if(error){
response.writeHead(404);
} else{
response.write(data);
}
response.end();
})
});
simpleServer.listen(80, '0.0.0.0', function() {
console.log('Listening to port: ' + 80);
});
console.log("Server running...");
Short Answer
Bypassing using HTML altogether, you can also simply serve the audio file instead of Picture.html:
fs.readFile("./audiofile.mp3", function(error, data) {
if (error) {
response.writeHead(404);
} else {
response.writeHead(200, { "Content-Type": "audio/mpeg"});
response.end(data, 'utf-8');
}
});
Note:
You will have to replace the filename audiofile.mp3 and the content type audio/mpeg to their appropriate values for the file you want to send.
Check Mozilla's Complete List of MIME Types for a full list of file extensions and their associated content types.
Better Answer:
The http module is fairly low-level and is unnecessarily complicated if you're learning.
You can install express.js to your project using the command npm install express --save.
With express your code simplifies to:
const express = require('express');
const app = express();
const port = 80;
app.get('/', (request, response) => {
response.sendFile(__dirname + '/Picture.html');
});
// Anything put in the public folder is available to the world!
app.use(express.static(__dirname + '/public'));
app.listen(port, () => {
console.log(`Listening on port: ${port}`)
});
Then you just have to place all your files into a folder called "public" under your project directory and you can call them from HTML!
I have an express.js app, and I am using drywall in order to manage the user system.
When a user signs up, I want a directory to be generated for that user, and I want that user to be able to upload files to that directory and view those files through his or her account.
I am not entirely sure, but I think that most likely for the directory generation I will have to do that inside views/signup/index.js, and that the user can only upload files to his or her directory if logged in.
However, I'm a bit stuck when it comes to saving and displaying the files. I have little experience with server side code, so implementing actions such as accessing files is slightly beyond my scope.
Thanks in advance to those who help.
So first you should create a folder for each user by using fs.mkdir:
http://nodejs.org/api/fs.html#fs_fs_mkdir_path_mode_callback
Let's say you want to create these folders into your app root / images:
Example:
var fs = require('fs');
fs.mkdir(__dirname + '/images/' + userId, function(err) {
if (err) {
/* log err etc */
} else {
/* folder creation succeeded */
}
});
You should probably use the userId for the folder name (since it's easier than trying to strip out the bad characters from the username itself, and this will also work in the future if the user changes his username).
The second thing you need to do is to allow the user to upload files (but only if he is logged in and into the right folder). It's better to not include the bodyParser middleware for all routes, but instead include the json && urlencoded middleware for all routes (http://www.senchalabs.org/connect/json.html && http://www.senchalabs.org/connect/urlencoded.html) and the multipart middleware only for the upload url ( http://www.senchalabs.org/connect/multipart.html && example: https://github.com/visionmedia/express/blob/master/examples/multipart/index.js ).
An example:
app.post('/images', express.multipart({ uploadDir: '/tmp/uploads' }), function(req, res, next) {
// at this point the file has been saved to the tmp path and we need to move
// it to the user's folder
fs.rename(req.files.image.path, __dirname + '/images/' + req.userId + '/' + req.files.image.name, function(err) {
if (err) return next(err);
res.send('Upload successful');
});
});
Note: in the example above I've taken into consideration that req.userId is populated with the id of the user by an auth middleware.
Showing the images to the user if he has the rights to see them (the auth middleware should be applied for this path as well):
app.get('/images/:user/:file', function(req, res, next) {
var filePath = __dirname + '/images/' + req.userId + '/' + req.params.file;
fs.exists(filePath, function(exists) {
if (!exists) { return res.status(404).send('Not Found'); }
// didn't attach 'error' handler here, but you should do that with streams always
fs.createReadStream(filePath).pipe(res);
});
});
Note: in production you might want to use send instead, that example was just demoing the flow ( https://github.com/visionmedia/send ).