Context:
This is my first time working with files using Nodejs.
I am making a Youtube video downloader for personal use.
In the frontend I have multiple buttons representing a video quality, each button has attached to it a URL where the video can be found for the specified quality.
When a specific button is pressed, the function 'download' from 'client.js' is being called and gets passed the URL representing the button and a filename.
My first try was to create a write stream to the public folder of my app, and after the download was finished, get the video from the path in the frontend, but it was taking too long and it was really inefficient for big files.
The current way it works is mostly the same as the other one, but this method is even slower than the other method I used.
How can I make the download more efficient?
For example when the user presses a button, the download to right away.
client.js
const download = async (url, filename) => {
const error = document.querySelector(".error")
const status = document.querySelector(".status")
try {
status.textContent = "Downloading ..."
const response = await axios.get("/download", {params: {url, filename}})
window.location.href = response.request.responseURL
error.textContent = ""
status.textContent = "Download Complete"
}
catch(e) {
error.textContent = "Cannot Download The Data!"
status.textContent = ""
}
}
server.js
app.get('/download', async (request, response) => {
try {
const URL = request.query.url
const filename = request.query.filename
response.attachment(filename)
const { data } = await axios.get(URL, { responseType: 'stream' })
data.pipe(response)
}
catch (e) {
return response.status(404).send({error: "Url Not Found!"})
}
})
I recently started developing desktop application using electron.
I want to send form details to main.js from the index.html on button click event. I have added a listener to the button in index.js. Searched online and found that I have to use ipcMain in main.js and ipcRenderer in index.js but the data is not being sent to ipcMain .
How do I get the form data in main.js ?
In index.html
<div class="btn-div fld">
<button id="loginButton" class="btn btn-primary st-btn" type="submit" name="button">Login</button>
</div>
<script src="index.js" charset="utf-8"></script>
In index.js
document.querySelector("#loginButton").addEventListener('click', function(){
userDetails = document.getElementById('login');
username = userDetails.username.value;
password = userDetails.password.value;
console.log("Hello");
const {ipcRenderer} = require('electron');
ipcRenderer.send('asynchronous-message', username);
})
In main.js
const { app, BrowserWindow , ipcMain } = require('electron')
ipcMain.on('asynchronous-message', (event, arg) => {
console.log( arg );
});
While creating a browser window in electron using new BrowserWindow(options) where options is an object. Define the object as:
options = {
webPreferences: {
preload: preload.js, //You need to create a file named preload.js (or any name) in your code
nodeIntegration: true,
contextIsolation: false,
}
}
Now in a new file called preload.js:
window.ipcRenderer = require('electron').ipcRenderer;
In your snippet you added const { app } ... which should be done this way to inject the javascript using a preload property in the object.
Now in the main app.js file (whatever you named maybe index.js) where you created the browser window:
const ipc = require('electron').ipcMain; //Add to your pre-existing code
ipc.on("close-app", (event, message) => { //"close-app" can be anything but, you need to use the same key in the send message side (later in this answer)
browserWindow.close(); //If you named the browserwindow as browserWindow
});
Now in your HTML (i.e., send message side)
...
<script>
window.ipcRenderer("close-app", ""); //Second parameter is used if you want to send some extra message. The extra message can be viewed in the server side from the message parameter in the app.js code (just above this paragraph)
</script>
This is a bit difficult if you are doing it for the first time.
I've added more articles which will help you clear your confusions:
A related answer at StackOverflow
Relation with socket.io communication in NodeJS
While I have seen that the other answer to this question may have worked for others, it did not work for me... I was using webpack and, for the life of me, could not get it to work even when adding ExternalsPlugin commonjs and electron. The following worked instead:
main.js
ipcMain.on("download", (event, info) => {
info.properties.onProgress = status => win.webContents.send("downloadProgress", status);
});
preload.js
contextBridge.exposeInMainWorld('electron', {
api: {
//receiving message from main.js
responseProgress: (channel, func) => {
let validChannels = ["downloadProgress"];
if (validChannels.includes(channel)) {
ipcRenderer.on(channel, (event, ...args) => func(...args));
}
},
process: (channel) => {
//example for process that is called by button in reactjs etc
}
});
ReactComponent.js
function ReactComponent() {
useEffect(() => {
//retrieving information from preload.js originally sent from main.js
window.electron.api.responseProgress("downloadProgress", (progress) => {
console.log(progress);
console.log(progress.percent);
});
}, []);
return (
//example of calling api on button click
<button onClick={() => {
window.electron.api.process("toMain");
}}>Download</button>
)
}
I am trying to add functionality to my Electron app that will allow users to open a file in the app, specifically plain text files. After looking at the Electron documentation, I found this page. I added this code to my app.js file, which I linked to in my index.html.
var fs = require('fs');
var dialog = require('electron');
$openFile = $('#openBtn');
$editor = $('#editor');
$openFile.click(function(){
dialog.showOpenDialog(function(fileNames) {
if (fileNames === undefined) return;
var fileName = fileNames[0];
fs.readFile(fileName, 'utf-8', function (err, data) {
$editor.val(data);
});
});
});
However, when I run this, this error shows up in the console: Uncaught TypeError: dialog.showOpenDialog is not a function I have tried using remote, but to no avail.
Has anyone know how to fix this problem?
Thanks in advance
const {dialog} = require('electron').remote;
document.querySelector('#selectBtn').addEventListener('click', function (event) {
dialog.showOpenDialog({
properties: ['openFile', 'multiSelections']
}, function (files) {
if (files !== undefined) {
// handle files
}
});
});
On the main process you can use
const {dialog} = require('electron');
dialog.showOpenDialog({properties: ['openFile'] }).then(function (response) {
if (!response.canceled) {
// handle fully qualified file name
console.log(response.filePaths[0]);
} else {
console.log("no file selected");
}
});
response looks like:
{
canceled: false,
filePaths: [
'<fullpath>/<filename>'
]
}
I found this "Rendering PDFs with React components" tutorial on themeteorchef about creating PDF files on Meteor serverside and then sending them back to client. I had no really need for PDF files, but docx files instead and thought that maybe I could follow similar approach when creating docx files with officegen
I created very similar server side module that generates a docx file from inputs on clientside and then tries to transform them into base64 string that is then supposed to be sent to the client. However, the base64 string is never created.
Here's the module:
let myModule;
const getBase64String = (loc) => {
try {
const file = fs.readFileSync(loc);
return new Buffer(file).toString('base64');
} catch (exception) {
myModule.reject(exception);
}
}
const generateBase64Docx = (path, fileName) => {
try {
myModule.resolve({fileName, base64: getBase64String(path+fileName)});
fs.unlink(loc);
} catch (exception) {
myModule.reject(exception);
}
}
const formatComponentAsDocx = (props, fileName) => {
try {
var docxFile = officegen({
'type': 'docx',
'orientation': 'portrait',
'title': props.title,
});
var pObj = docxFile.createP();
pObj.addText(props.body);
var path = './';
output = fs.createWriteStream(path+fileName);
docxFile.generate(output);
return path;
} catch (exception) {
myModule.reject(exception);
}
}
const handler = ({props, fileName}, promise) => {
myModule = promise;
const path = formatComponentAsDocx(props, fileName);
if (path) {
generateBase64Docx(path, fileName);
}
}
export const generateComponentAsDocx = (options) => {
return new Promise((resolve, reject) => {
return handler(options, { resolve, reject });
});
};
The problem here is the fs.readFileSync part. It always returns empty buffer and that's why the file is never transformed into base64 string and also never sent back to client. Why's that? The file itself is always created on the server and can always be found.
If I change the const file = fs.readFileSync(loc); part to for example this
fs.readFile(loc, (err, data) => {
if(err) myModule.reject(err);
console.log(JSON.stringify(data));
}
I can see some data in data, but not enough for the whole file.
What am I doing wrong here? Am I missing something?
You need to wait until the file generated by officegen is complete before you try to get the base64 out of it. That's the minimal change you need to make. I don't recommend waiting on the finalize event generated by officegen as this event is buggy. I recommend waiting on the finish event of the output stream. However, there are additional issues with the code you show:
Since you have code to unlink the file immediately after you use it, then I infer you do not need a file. So you can just create the data in memory and get a base64 string from that.
The whole rigmarole with myModule is awful awful design. If one of my colleagues presented such code, strong words would be exchanged. Yes, it is that bad. It is much better to convert the entire code base to work with promises.
The whole module can be reduced to the following. I've done a modicum of testing on this code but I don't claim that it deals with every eventuality.
import * as stream from "stream";
import officegen from "officegen";
function formatComponentAsDocx(props) {
return new Promise((resolve, reject) => {
// There's no need to wrap this in try...catch only to call reject. If any
// exception is raised in this function, the promise is automatically
// rejected.
const docxFile = officegen({
'type': 'docx',
'orientation': 'portrait',
'title': props.title,
});
const pObj = docxFile.createP();
pObj.addText(props.body);
// We record the output in our own buffer instead of writing to disk,
// and reading later.
let buf = Buffer.alloc(0);
const output = new stream.Writable({
write(chunk, encoding, callback) {
buf = Buffer.concat([buf, chunk]);
callback();
},
});
docxFile.generate(output, {
// Do propagate errors from officegen.
error: reject,
});
// We don't use the "finalize" event that docxFile.generate would emit
// because it is buggy. Instead, we wait for the output stream to emit
// the "finish" event.
output.on('finish', () => {
resolve(buf);
});
});
}
export function generateComponentAsDocx({ props }) {
return formatComponentAsDocx(props).then((data) => {
return { base64: data.toString("base64") };
});
};
Your problem is that docxFile.generate(output); is not synchronous. Thus, while your local path exists (it was created by fs.createWriteStream() call), it is empty and your synchronous fs.readFileSync is catching just that, empty file.
You should subscribe to docxFile's finalize event to catch file generation end:
docxFile.on('finalize, function (writtenBytes) {
// do you work with generated file here
});
Thus, rewriting your code:
const handler = ({props, fileName}, promise) => {
myModule = promise;
formatComponentAsDocx(props, fileName);
}
const formatComponentAsDocx = (props, fileName) => {
try {
var docxFile = officegen({
'type': 'docx',
'orientation': 'portrait',
'title': props.title,
});
var pObj = docxFile.createP();
pObj.addText(props.body);
var path = './';
output = fs.createWriteStream(path+fileName);
docxFile.on('error', function (err) {
myModule.reject(err);
});
docxFile.on('finalize', function () {
generateBase64Docx(path, fileName);
});
docxFile.generate(output);
} catch (exception) {
myModule.reject(exception);
}
}
readFileSync is synchronous, so it doesn't deal in promises.
https://nodejs.org/api/fs.html#fs_fs_readfilesync_file_options
Synchronous version of fs.readFile. Returns the contents of the file.
You probably want to use fs.readFile.
https://nodejs.org/api/fs.html#fs_fs_readfile_file_options_callback
The callback is passed two arguments (err, data), where data is the contents of the file.
If no encoding is specified, then the raw buffer is returned.
I'm looking to create a printable pdf version of my website webpages. Something like express.render() only render the page as pdf
Does anyone know a node module that does that ?
If not, how would you go about implementing one ? I've seen some methods talk about using headless browser like phantom.js, but not sure whats the flow.
Extending upon Mustafa's answer.
A) Install http://phantomjs.org/ and then
B) install the phantom node module https://github.com/amir20/phantomjs-node
C) Here is an example of rendering a pdf
var phantom = require('phantom');
phantom.create().then(function(ph) {
ph.createPage().then(function(page) {
page.open("http://www.google.com").then(function(status) {
page.render('google.pdf').then(function() {
console.log('Page Rendered');
ph.exit();
});
});
});
});
Output of the PDF:
EDIT: Silent printing that PDF
java -jar pdfbox-app-2.0.2.jar PrintPDF -silentPrint C:\print_mypdf.pdf
Phantom.js is an headless webkit server and it will load any web page and render it in memory, although you might not be able to see it, there is a Screen Capture feature, in which you can export the current view as PNG, PDF, JPEG and GIF. Have a look at this example from phantom.js documentation
If you want to export HTML to PDF. You have many options. without node even
Option 1: Have a button on your html page that calls window.print() function. use the browsers native html to pdf. use media queries to make your html page look good on a pdf. and you also have the print before and after events that you can use to make changes to your page before print.
Option 2. htmltocanvas or rasterizeHTML. convert your html to canvas , then call toDataURL() on the canvas object to get the image . and use a JavaScript library like jsPDF to add that image to a PDF file. Disadvantage of this approach is that the pdf doesnt become editable. If you want data extracted from PDF, there is different ways for that.
Option 3. #Jozzhard answer
Try to use Puppeteer to create PDF from HTML
Example from here https://github.com/chuongtrh/html_to_pdf
Or https://github.com/GoogleChrome/puppeteer
The best solution I found is html-pdf. It's simple and work with big html.
https://www.npmjs.com/package/html-pdf
Its as simple as that:
pdf.create(htm, options).toFile('./pdfname.pdf', function(err, res) {
if (err) {
console.log(err);
}
});
NOTE:
This package has been deprecated
Author message: Please migrate your projects to a newer library like puppeteer
Package
I used html-pdf
Easy to use and allows not only to save pdf as file, but also pipe pdf content to a WriteStream (so I could stream it directly to Google Storage to save there my reports).
Using css + images
It takes css into account. The only problem I faced - it ignored my images. The solution I found was to replace url in src attrribute value by base64, e.g.
<img src="data:image/png;base64,iVBOR...kSuQmCC">
You can do it with your code or to use one of online converters, e.g. https://www.base64-image.de/
Compile valid html code from html fragment + css
I had to get a fragment of my html document (I just appiled .html() method on jQuery selector).
Then I've read the content of the relevant css file.
Using this two values (stored in variables html and css accordingly) I've compiled a valid html code using Template string
var htmlContent = `
<!DOCTYPE html>
<html>
<head>
<style>
${css}
</style>
</head>
<body id=direct-sellers-bill>
${html}
</body>
</html>`
and passed it to create method of html-pdf.
Create PDF from External URL
Here's an adaptation of the previous answers which utilizes html-pdf, but also combines it with requestify so it works with an external URL:
Install your dependencies
npm i -S html-pdf requestify
Then, create the script:
//MakePDF.js
var pdf = require('html-pdf');
var requestify = require('requestify');
var externalURL= 'http://www.google.com';
requestify.get(externalURL).then(function (response) {
// Get the raw HTML response body
var html = response.body;
var config = {format: 'A4'}; // or format: 'letter' - see https://github.com/marcbachmann/node-html-pdf#options
// Create the PDF
pdf.create(html, config).toFile('pathtooutput/generated.pdf', function (err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/pathtooutput/generated.pdf' }
});
});
Then you just run from the command line:
node MakePDF.js
Watch your beautify pixel perfect PDF be created for you (for free!)
For those who don't want to install PhantomJS along with an instance of Chrome/Firefox on their server - or because the PhantomJS project is currently suspended, here's an alternative.
You can externalize the conversions to APIs to do the job. Many exists and varies but what you'll get is a reliable service with up-to-date features (I'm thinking CSS3, Web fonts, SVG, Canvas compatible).
For instance, with PDFShift (disclaimer, I'm the founder), you can do this simply by using the request package:
const request = require('request')
request.post(
'https://api.pdfshift.io/v2/convert/',
{
'auth': {'user': 'your_api_key'},
'json': {'source': 'https://www.google.com'},
'encoding': null
},
(error, response, body) => {
if (response === undefined) {
return reject({'message': 'Invalid response from the server.', 'code': 0, 'response': response})
}
if (response.statusCode == 200) {
// Do what you want with `body`, that contains the binary PDF
// Like returning it to the client - or saving it as a file locally or on AWS S3
return True
}
// Handle any errors that might have occured
}
);
Use html-pdf
var fs = require('fs');
var pdf = require('html-pdf');
var html = fs.readFileSync('./test/businesscard.html', 'utf8');
var options = { format: 'Letter' };
pdf.create(html, options).toFile('./businesscard.pdf', function(err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/app/businesscard.pdf' }
});
const fs = require('fs')
const path = require('path')
const utils = require('util')
const puppeteer = require('puppeteer')
const hb = require('handlebars')
const readFile = utils.promisify(fs.readFile)
async function getTemplateHtml() {
console.log("Loading template file in memory")
try {
const invoicePath = path.resolve("./invoice.html");
return await readFile(invoicePath, 'utf8');
} catch (err) {
return Promise.reject("Could not load html template");
}
}
async function generatePdf() {
let data = {};
getTemplateHtml()
.then(async (res) => {
// Now we have the html code of our template in res object
// you can check by logging it on console
// console.log(res)
console.log("Compiing the template with handlebars")
const template = hb.compile(res, { strict: true });
// we have compile our code with handlebars
const result = template(data);
// We can use this to add dyamic data to our handlebas template at run time from database or API as per need. you can read the official doc to learn more https://handlebarsjs.com/
const html = result;
// we are using headless mode
const browser = await puppeteer.launch();
const page = await browser.newPage()
// We set the page content as the generated html by handlebars
await page.setContent(html)
// we Use pdf function to generate the pdf in the same folder as this file.
await page.pdf({ path: 'invoice.pdf', format: 'A4' })
await browser.close();
console.log("PDF Generated")
})
.catch(err => {
console.error(err)
});
}
generatePdf();
In case you arrive here looking for a way to make PDF from view templates in Express, a colleague and I made express-template-to-pdf
which allows you to generate PDF from whatever templates you're using in Express - Pug, Nunjucks, whatever.
It depends on html-pdf and is written to use in your routes just like you use res.render:
const pdfRenderer = require('#ministryofjustice/express-template-to-pdf')
app.set('views', path.join(__dirname, 'views'))
app.set('view engine', 'pug')
app.use(pdfRenderer())
If you've used res.render then using it should look obvious:
app.use('/pdf', (req, res) => {
res.renderPDF('helloWorld', { message: 'Hello World!' });
})
You can pass options through to html-pdf to control the PDF document page size etc
Merely building on the excellent work of others.
In my view, the best way to do this is via an API so that you do not add a large and complex dependency into your app that runs unmanaged code, that needs to be frequently updated.
Here is a simple way to do this, which is free for 800 requests/month:
var CloudmersiveConvertApiClient = require('cloudmersive-convert-api-client');
var defaultClient = CloudmersiveConvertApiClient.ApiClient.instance;
// Configure API key authorization: Apikey
var Apikey = defaultClient.authentications['Apikey'];
Apikey.apiKey = 'YOUR API KEY';
var apiInstance = new CloudmersiveConvertApiClient.ConvertWebApi();
var input = new CloudmersiveConvertApiClient.HtmlToPdfRequest(); // HtmlToPdfRequest | HTML to PDF request parameters
input.Html = "<b>Hello, world!</b>";
var callback = function(error, data, response) {
if (error) {
console.error(error);
} else {
console.log('API called successfully. Returned data: ' + data);
}
};
apiInstance.convertWebHtmlToPdf(input, callback);
With the above approach you can also install the API on-premises or on your own infrastructure if you prefer.
In addition to #Jozzhart Answer, you can make a local html; serve it with express; and use phantom to make PDF from it; something like this:
const exp = require('express');
const app = exp();
const pth = require("path");
const phantom = require('phantom');
const ip = require("ip");
const PORT = 3000;
const PDF_SOURCE = "index"; //index.html
const PDF_OUTPUT = "out"; //out.pdf
const source = pth.join(__dirname, "", `${PDF_SOURCE}.html`);
const output = pth.join(__dirname, "", `${PDF_OUTPUT}.pdf`);
app.use("/" + PDF_SOURCE, exp.static(source));
app.use("/" + PDF_OUTPUT, exp.static(output));
app.listen(PORT);
let makePDF = async (fn) => {
let local = `http://${ip.address()}:${PORT}/${PDF_SOURCE}`;
phantom.create().then((ph) => {
ph.createPage().then((page) => {
page.open(local).then(() =>
page.render(output).then(() => { ph.exit(); fn() })
);
});
});
}
makePDF(() => {
console.log("PDF Created From Local File");
console.log("PDF is downloadable from link:");
console.log(`http://${ip.address()}:${PORT}/${PDF_OUTPUT}`);
});
and index.html can be anything:
<h1>PDF HEAD</h1>
LINK
result:
https://www.npmjs.com/package/dynamic-html-pdf
I use dynamic-html-pdf, this is simple and also able to pass dynamic variable to html.
var html = fs.readFileSync('./uploads/your-html-tpl.html', 'utf8');
var options = {
format: "A4",
orientation: "portrait"
// border: "10mm"
};
var document = {
type: 'file', // 'file' or 'buffer'
template: html,
context: {
'your_key':'your_values'
},
path: '/pdf/1.pdf' // pdf save path
};
pdf.create(document, options)
.then(res => {
console.log(res)
}).catch(error => {
console.error(error)
});
On html you can use {{your_key}}
I've written hpdf lib for generating PDF from HTLM or URL.
It supports configurable pool of headless browsers (as resources) in the background.
import fs from 'fs';
import { PdfGenerator } from './src';
const start = async () => {
const generator = new PdfGenerator({
min: 3,
max: 10,
});
const helloWorld = await generator.generatePDF('<html lang="html">Hello World!</html>');
const github = await generator.generatePDF(new URL('https://github.com/frimuchkov/hpdf'));
await fs.promises.writeFile('./helloWorld.pdf', helloWorld);
await fs.promises.writeFile('./github.pdf', github);
await generator.stop();
}
I wanted to add to this since I did not see the option to created pdfs from liquid templates yet, but the solution also works with normal html or urls as well.
Lets say this is our html template. Which could be anything really but see that the code include double curly braces. The key inside the braces will be looked up in the liquid_data parameter of the request and replaced by the value.
<html>
<body>
<h1>{{heading}}</h1>
<img src="{{img_url}}"/>
</body>
</html>
The corresponding liquid_data object looks like this:
{
"heading":"Hi Stackoverflow!",
"img_url":"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg"
}
This is the example I want to create a PDF for. Using pdfEndpoint and the Playground creating a pdf from that template from above is very simple.
const axios = require("axios");
const options = {
method: "POST",
url: "https://api.pdfendpoint.com/v1/convert",
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer SIGN-UP-FOR-KEY"
},
data: {
"delivery_mode": "json",
"page_size": "A4",
"margin_top": "1cm",
"margin_bottom": "1cm",
"margin_left": "1cm",
"margin_right": "1cm",
"orientation": "vertical",
"html": "<html><body> <h1>{{heading}}</h1> <img src=\"{{img_url}}\"/> </body>\</html>",
"parse_liquid": true,
"liquid_data": "{ \"heading\":\"Hi Stackoverflow!\", \"img_url\":\"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg\"}"
}
};
axios.request(options).then(function (response) {
console.log(response.data);
}).catch(function (error) {
console.error(error);
});
The service will the return a rendered pdf like this:
You can also use pdf node creator package
Package URL -
https://www.npmjs.com/package/pdf-creator-node