im trying to make a command that connects to the database, i created a little CLI script that loops through files in specific folders to get command class modules
my problem is that in one of my commands, i'm trying to connect to sequelize, and it just doesn't seem to be doing anything. i get no output to the console, nor does it even seem to try to connect
this is probably because i'm still kind of struggling to figure out how to properly do sync / async / await stuff...notice how i use glob.sync cause i want to loop through the files sychronously, but then in my command i need to connect to the database using await
cli.js:
#! /usr/bin/env node
const patterns = [
'./node_modules/unity/src/commands/**/*.js',
'./src/commands/**/*.js',
]
const glob = require('glob')
const path = require('path')
const yargs = require('yargs')
const signature = yargs.argv._[0]
const process = require('process')
patterns.forEach(pattern => {
glob.sync(pattern).forEach(file => {
const commandPath = path.resolve(file)
const command = require(commandPath)
if (command.signature == signature) {
command.argv = yargs.argv
command.run()
process.exit()
}
})
})
console.log('Command not found.')
here is an example command in one of the commands folders:
const { Sequelize } = require('sequelize')
class MigrateCommand {
static signature = 'migrate'
static argv = {}
static run() {
const sequelize = new Sequelize({
dialect: 'mysql',
host: 'localhost',
port: 3306,
database: 'dio_unity2',
username: 'root',
password: '',
})
const connect = async () => {
try {
await sequelize.authenticate()
console.log('Connection successful.')
}
catch (error) {
console.error('Unable to connect.', error)
}
}
connect()
console.log('migrate run complete')
}
}
module.exports = MigrateCommand
i've set npx up so i can just run npx unity migrate and it will call this command based on the migrate signature
now my console should say connection successful or unable to connect and then migrate run complete, but all i see in the console is migrate run complete. it's like it isn't even trying to connect at all...
i have no idea what i'm doing wrong here.
You can't make an asynchronous process synchronous without spawning a new thread and synchronously waiting on it. (There's an npm package that does that, via execSync.)
But the good news here is there's no need to in your code. You want to do things in series, but doing things in series isn't quite the same as doing them synchronously. Here's how:
First, run can't make the async process it's starting via authenticate synchronous. So instead, run should just be async (and we don't need connect):
static async run() {
const sequelize = new Sequelize({
dialect: 'mysql',
host: 'localhost',
port: 3306,
database: 'dio_unity2',
username: 'root',
password: '',
})
try {
await sequelize.authenticate()
console.log('Connection successful.')
} catch (error) {
console.error('Unable to connect.', error)
}
console.log('migrate run complete')
}
Next, in the script, we loop through your patterns and files using an async function:
(async () => {
for (const pattern of patterns) {
for (const file of glob.sync(pattern)) {
const commandPath = path.resolve(file)
const command = require(commandPath)
if (command.signature == signature) {
command.argv = yargs.argv
await command.run() // *** Note the `await`
process.exit()
}
}
}
console.log("Command not found.")
})();
That does the work asynchronously, but one after another (in series).
If it were me, I wouldn't use process.exit() to terminate the process, it's a very aggressive form of process termination which may prevent some things from finishing (details here), and may not be as clear to people doing code maintenance later as alternatives.. Now that we've put the code in a wrapper function, I'd just return out of the function to break the loops:
await command.run();
return;
I am trying to export database properties stored in properties file from Javascript module. By the time I read database properties file, Javascript file is already exported and data properties appear undefined wherever I use in other modules.
const Pool = require('pg').Pool;
const fs = require('fs')
const path = require('path');
class DbConfig {
constructor(dbData) {
this.pool = new Pool({
user: dbData['user'],
host: dbData['host'],
database: dbData['database'],
password: dbData['password'],
max: 20,
port: 5432
});
}
}
function getdbconf() {
const dbData = {};
fs.readFile("../../db_properties.txt"), 'utf8', (err, data) => {
if (err) {
console.error(err)
return
}
// dbData = {"user":"postgres", "password": "1234"...};
return dbData;
});
}
let db = new DbConfig(getdbconf());
let dbPool = db.pool;
console.log("dbpool : -> : ",dbPool); // username and password appear undefined
module.exports = { dbPool };
Is there a way to read data before exporting data from Javascript module?
Usually database config or any other sensitive info is read from a .env file using dotenv .
Or
you could also provide env from command line itself like
DB_HOST=127.0.0.1 node index.js
inside your index.js
console.log(process.env.DB_HOST)
Please create a new file (connection-pool.js) and paste this code:
const { Pool } = require('pg');
const poolConnection = new Pool({
user: 'postgresUserName',
host: 'yourHost',
database: 'someNameDataBase',
password: 'postgresUserPassword',
port: 5432,
});
console.log('connectionOptions', poolConnection.options);
module.exports = poolConnection;
For use it, create a new file (demo-connection.js) and paste this code:
const pool = require('./connection-pool');
pool.query('SELECT NOW();', (err, res) => {
if (err) {
// throw err;
console.log('connection error');
return;
}
if (res) {
console.log(res.rows);
pool.end();
}
});
This is an alternative option 🙂
Exporting the result of async calls
To export values which have been obtained asynchronously, export a Promise.
const fs = require('fs/promises'); // `/promise` means no callbacks, Promise returned
const dbDataPromise = fs.readFile('fileToRead')); //`readFile` returns Promise now
module.exports = dbDataPromise;
Importing
When you need to use the value,
const dbDataPromise = require('./dbdata');
async init() {
const dbData = await dbDataPromise;
}
//or without async, using Promise callbacks
init() {
dbDataPromise
.then(dbData => the rest of your code that depends on dbData here);
}
Current code broken
Please note that your current code, as pasted above, is broken:
function getdbconf() {
const dbData = {};
fs.readFile("../../db_properties.txt"), 'utf8', (err, data) => {
//[...] snipped for brevity
return dbData;
});
}
fs.readFile "returns" dbData, but there is nothing to return to, since you are in a callback which you did not call yourself. Function getdbconf returns nothing.
The line that says let db = new DbConfig(getdbconf()); will NOT work. It needs to be inside the callback.
The only way to avoid putting all of your code inside the callback (and "flatten" it) is to use await, or to use readFileSync
Avoiding the issue
Using environment variables
Suhas Nama's suggestion is a good one, and is common practice. Try putting the values you need in environment variables.
Using synchronous readFile
While using synchronous calls does block the event loop, it's ok to do during initialization, before your app is up and running.
This avoids the problem of having everything in a callback or having to export Promises, and is often the best solution.
So im trying to make an update checker that doesnt actually download the update but nvm and all im trying to is check if the version on the package.json on github is the same as the one in the app (im making it in electron)
And with this code i get a "path must be a string error" (heres an image https://gyazo.com/7b55a1cbe96b2719bb588a6591855839 )
also i did look it up many times and in fact the code to get the package.json from github with the http module was from how to require from URL in Node.js
function checkforupdate() {
var http = require('http')
const file_url = "https://raw.githubusercontent.com/FloffahDevelopments/FloffahsHub/master/package.json";
const oldpackagejson = require("package.json");
document.getElementById("checkupdate").innerHTML = 'Checking for updates</br>Please wait</br><img src="./assets/icons/gif/loading.gif" alt="loading" style="width: 10%; left: 45%;">'
http.get(file_url).then(res => res.json()).then(pack => {
if (pack.version !== oldpackagejson.version) {
document.getElementById("checkupdate").innerHTML = 'Update available!'
} else {
document.getElementById("checkupdate").innerHTML = 'No update available!'
}
});
}
This will make the request you want:
var https = require('https')
const file_url = "https://raw.githubusercontent.com/FloffahDevelopments/FloffahsHub/master/package.json"
const oldpackagejson = require("./package.json");
https.get(file_url, (res) => {
res.on('data', (d) => {
process.stdout.write(d)
})
}).on('error', (e) => {
console.log(e)
})
Some mistakes you made were: Treating http.get as a promise when it's not. It could by with the inclusion of a module like bluebird, though. You used the http module to make an https request. You did not give http.get it's parameters correctly, that is, your syntax was incorrect. You're trying to update the DOM on server-side code, you should separate client and server logic. res => res.json() does not change res to json, you'll need JSON.parse.
I'm looking to create a printable pdf version of my website webpages. Something like express.render() only render the page as pdf
Does anyone know a node module that does that ?
If not, how would you go about implementing one ? I've seen some methods talk about using headless browser like phantom.js, but not sure whats the flow.
Extending upon Mustafa's answer.
A) Install http://phantomjs.org/ and then
B) install the phantom node module https://github.com/amir20/phantomjs-node
C) Here is an example of rendering a pdf
var phantom = require('phantom');
phantom.create().then(function(ph) {
ph.createPage().then(function(page) {
page.open("http://www.google.com").then(function(status) {
page.render('google.pdf').then(function() {
console.log('Page Rendered');
ph.exit();
});
});
});
});
Output of the PDF:
EDIT: Silent printing that PDF
java -jar pdfbox-app-2.0.2.jar PrintPDF -silentPrint C:\print_mypdf.pdf
Phantom.js is an headless webkit server and it will load any web page and render it in memory, although you might not be able to see it, there is a Screen Capture feature, in which you can export the current view as PNG, PDF, JPEG and GIF. Have a look at this example from phantom.js documentation
If you want to export HTML to PDF. You have many options. without node even
Option 1: Have a button on your html page that calls window.print() function. use the browsers native html to pdf. use media queries to make your html page look good on a pdf. and you also have the print before and after events that you can use to make changes to your page before print.
Option 2. htmltocanvas or rasterizeHTML. convert your html to canvas , then call toDataURL() on the canvas object to get the image . and use a JavaScript library like jsPDF to add that image to a PDF file. Disadvantage of this approach is that the pdf doesnt become editable. If you want data extracted from PDF, there is different ways for that.
Option 3. #Jozzhard answer
Try to use Puppeteer to create PDF from HTML
Example from here https://github.com/chuongtrh/html_to_pdf
Or https://github.com/GoogleChrome/puppeteer
The best solution I found is html-pdf. It's simple and work with big html.
https://www.npmjs.com/package/html-pdf
Its as simple as that:
pdf.create(htm, options).toFile('./pdfname.pdf', function(err, res) {
if (err) {
console.log(err);
}
});
NOTE:
This package has been deprecated
Author message: Please migrate your projects to a newer library like puppeteer
Package
I used html-pdf
Easy to use and allows not only to save pdf as file, but also pipe pdf content to a WriteStream (so I could stream it directly to Google Storage to save there my reports).
Using css + images
It takes css into account. The only problem I faced - it ignored my images. The solution I found was to replace url in src attrribute value by base64, e.g.
<img src="data:image/png;base64,iVBOR...kSuQmCC">
You can do it with your code or to use one of online converters, e.g. https://www.base64-image.de/
Compile valid html code from html fragment + css
I had to get a fragment of my html document (I just appiled .html() method on jQuery selector).
Then I've read the content of the relevant css file.
Using this two values (stored in variables html and css accordingly) I've compiled a valid html code using Template string
var htmlContent = `
<!DOCTYPE html>
<html>
<head>
<style>
${css}
</style>
</head>
<body id=direct-sellers-bill>
${html}
</body>
</html>`
and passed it to create method of html-pdf.
Create PDF from External URL
Here's an adaptation of the previous answers which utilizes html-pdf, but also combines it with requestify so it works with an external URL:
Install your dependencies
npm i -S html-pdf requestify
Then, create the script:
//MakePDF.js
var pdf = require('html-pdf');
var requestify = require('requestify');
var externalURL= 'http://www.google.com';
requestify.get(externalURL).then(function (response) {
// Get the raw HTML response body
var html = response.body;
var config = {format: 'A4'}; // or format: 'letter' - see https://github.com/marcbachmann/node-html-pdf#options
// Create the PDF
pdf.create(html, config).toFile('pathtooutput/generated.pdf', function (err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/pathtooutput/generated.pdf' }
});
});
Then you just run from the command line:
node MakePDF.js
Watch your beautify pixel perfect PDF be created for you (for free!)
For those who don't want to install PhantomJS along with an instance of Chrome/Firefox on their server - or because the PhantomJS project is currently suspended, here's an alternative.
You can externalize the conversions to APIs to do the job. Many exists and varies but what you'll get is a reliable service with up-to-date features (I'm thinking CSS3, Web fonts, SVG, Canvas compatible).
For instance, with PDFShift (disclaimer, I'm the founder), you can do this simply by using the request package:
const request = require('request')
request.post(
'https://api.pdfshift.io/v2/convert/',
{
'auth': {'user': 'your_api_key'},
'json': {'source': 'https://www.google.com'},
'encoding': null
},
(error, response, body) => {
if (response === undefined) {
return reject({'message': 'Invalid response from the server.', 'code': 0, 'response': response})
}
if (response.statusCode == 200) {
// Do what you want with `body`, that contains the binary PDF
// Like returning it to the client - or saving it as a file locally or on AWS S3
return True
}
// Handle any errors that might have occured
}
);
Use html-pdf
var fs = require('fs');
var pdf = require('html-pdf');
var html = fs.readFileSync('./test/businesscard.html', 'utf8');
var options = { format: 'Letter' };
pdf.create(html, options).toFile('./businesscard.pdf', function(err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/app/businesscard.pdf' }
});
const fs = require('fs')
const path = require('path')
const utils = require('util')
const puppeteer = require('puppeteer')
const hb = require('handlebars')
const readFile = utils.promisify(fs.readFile)
async function getTemplateHtml() {
console.log("Loading template file in memory")
try {
const invoicePath = path.resolve("./invoice.html");
return await readFile(invoicePath, 'utf8');
} catch (err) {
return Promise.reject("Could not load html template");
}
}
async function generatePdf() {
let data = {};
getTemplateHtml()
.then(async (res) => {
// Now we have the html code of our template in res object
// you can check by logging it on console
// console.log(res)
console.log("Compiing the template with handlebars")
const template = hb.compile(res, { strict: true });
// we have compile our code with handlebars
const result = template(data);
// We can use this to add dyamic data to our handlebas template at run time from database or API as per need. you can read the official doc to learn more https://handlebarsjs.com/
const html = result;
// we are using headless mode
const browser = await puppeteer.launch();
const page = await browser.newPage()
// We set the page content as the generated html by handlebars
await page.setContent(html)
// we Use pdf function to generate the pdf in the same folder as this file.
await page.pdf({ path: 'invoice.pdf', format: 'A4' })
await browser.close();
console.log("PDF Generated")
})
.catch(err => {
console.error(err)
});
}
generatePdf();
In case you arrive here looking for a way to make PDF from view templates in Express, a colleague and I made express-template-to-pdf
which allows you to generate PDF from whatever templates you're using in Express - Pug, Nunjucks, whatever.
It depends on html-pdf and is written to use in your routes just like you use res.render:
const pdfRenderer = require('#ministryofjustice/express-template-to-pdf')
app.set('views', path.join(__dirname, 'views'))
app.set('view engine', 'pug')
app.use(pdfRenderer())
If you've used res.render then using it should look obvious:
app.use('/pdf', (req, res) => {
res.renderPDF('helloWorld', { message: 'Hello World!' });
})
You can pass options through to html-pdf to control the PDF document page size etc
Merely building on the excellent work of others.
In my view, the best way to do this is via an API so that you do not add a large and complex dependency into your app that runs unmanaged code, that needs to be frequently updated.
Here is a simple way to do this, which is free for 800 requests/month:
var CloudmersiveConvertApiClient = require('cloudmersive-convert-api-client');
var defaultClient = CloudmersiveConvertApiClient.ApiClient.instance;
// Configure API key authorization: Apikey
var Apikey = defaultClient.authentications['Apikey'];
Apikey.apiKey = 'YOUR API KEY';
var apiInstance = new CloudmersiveConvertApiClient.ConvertWebApi();
var input = new CloudmersiveConvertApiClient.HtmlToPdfRequest(); // HtmlToPdfRequest | HTML to PDF request parameters
input.Html = "<b>Hello, world!</b>";
var callback = function(error, data, response) {
if (error) {
console.error(error);
} else {
console.log('API called successfully. Returned data: ' + data);
}
};
apiInstance.convertWebHtmlToPdf(input, callback);
With the above approach you can also install the API on-premises or on your own infrastructure if you prefer.
In addition to #Jozzhart Answer, you can make a local html; serve it with express; and use phantom to make PDF from it; something like this:
const exp = require('express');
const app = exp();
const pth = require("path");
const phantom = require('phantom');
const ip = require("ip");
const PORT = 3000;
const PDF_SOURCE = "index"; //index.html
const PDF_OUTPUT = "out"; //out.pdf
const source = pth.join(__dirname, "", `${PDF_SOURCE}.html`);
const output = pth.join(__dirname, "", `${PDF_OUTPUT}.pdf`);
app.use("/" + PDF_SOURCE, exp.static(source));
app.use("/" + PDF_OUTPUT, exp.static(output));
app.listen(PORT);
let makePDF = async (fn) => {
let local = `http://${ip.address()}:${PORT}/${PDF_SOURCE}`;
phantom.create().then((ph) => {
ph.createPage().then((page) => {
page.open(local).then(() =>
page.render(output).then(() => { ph.exit(); fn() })
);
});
});
}
makePDF(() => {
console.log("PDF Created From Local File");
console.log("PDF is downloadable from link:");
console.log(`http://${ip.address()}:${PORT}/${PDF_OUTPUT}`);
});
and index.html can be anything:
<h1>PDF HEAD</h1>
LINK
result:
https://www.npmjs.com/package/dynamic-html-pdf
I use dynamic-html-pdf, this is simple and also able to pass dynamic variable to html.
var html = fs.readFileSync('./uploads/your-html-tpl.html', 'utf8');
var options = {
format: "A4",
orientation: "portrait"
// border: "10mm"
};
var document = {
type: 'file', // 'file' or 'buffer'
template: html,
context: {
'your_key':'your_values'
},
path: '/pdf/1.pdf' // pdf save path
};
pdf.create(document, options)
.then(res => {
console.log(res)
}).catch(error => {
console.error(error)
});
On html you can use {{your_key}}
I've written hpdf lib for generating PDF from HTLM or URL.
It supports configurable pool of headless browsers (as resources) in the background.
import fs from 'fs';
import { PdfGenerator } from './src';
const start = async () => {
const generator = new PdfGenerator({
min: 3,
max: 10,
});
const helloWorld = await generator.generatePDF('<html lang="html">Hello World!</html>');
const github = await generator.generatePDF(new URL('https://github.com/frimuchkov/hpdf'));
await fs.promises.writeFile('./helloWorld.pdf', helloWorld);
await fs.promises.writeFile('./github.pdf', github);
await generator.stop();
}
I wanted to add to this since I did not see the option to created pdfs from liquid templates yet, but the solution also works with normal html or urls as well.
Lets say this is our html template. Which could be anything really but see that the code include double curly braces. The key inside the braces will be looked up in the liquid_data parameter of the request and replaced by the value.
<html>
<body>
<h1>{{heading}}</h1>
<img src="{{img_url}}"/>
</body>
</html>
The corresponding liquid_data object looks like this:
{
"heading":"Hi Stackoverflow!",
"img_url":"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg"
}
This is the example I want to create a PDF for. Using pdfEndpoint and the Playground creating a pdf from that template from above is very simple.
const axios = require("axios");
const options = {
method: "POST",
url: "https://api.pdfendpoint.com/v1/convert",
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer SIGN-UP-FOR-KEY"
},
data: {
"delivery_mode": "json",
"page_size": "A4",
"margin_top": "1cm",
"margin_bottom": "1cm",
"margin_left": "1cm",
"margin_right": "1cm",
"orientation": "vertical",
"html": "<html><body> <h1>{{heading}}</h1> <img src=\"{{img_url}}\"/> </body>\</html>",
"parse_liquid": true,
"liquid_data": "{ \"heading\":\"Hi Stackoverflow!\", \"img_url\":\"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg\"}"
}
};
axios.request(options).then(function (response) {
console.log(response.data);
}).catch(function (error) {
console.error(error);
});
The service will the return a rendered pdf like this:
You can also use pdf node creator package
Package URL -
https://www.npmjs.com/package/pdf-creator-node