I have one scenario where I have to verify the downloaded text file's data against an API response.
Below is the code that I have tried.
Test:
const path = require('path')
const downloadsFolder = Cypress.config('downloadsFolder')
cy.task('deleteFolder', downloadsFolder)
const downloadedFilename = path.join(downloadsFolder, 'ABCDEF.txt')//'*.txt'
....
cy.get('#portmemo').its('response.body')
.then((response) => {
var json = JSON.parse(response);
const resCon = json[0].content.replaceAll(/[\n\r]/g, '');
cy.readFile(downloadedFilename).then((fc) => {
const formatedfc = fc.replaceAll(/[\n\r]/g, '');
cy.wrap(formatedfc).should('contains', resCon)
})
})
Task in /cypress/plugins/index.js
const { rmdir } = require('fs')
module.exports = (on, config) => {
console.log("cucumber started")
on('task', {
deleteFolder(folderName) {
return new Promise((resolve, reject) => {
rmdir(folderName, { maxRetries: 5, recursive: true }, (err) => {
if (err) {
console.error(err);
return reject(err)
}
resolve(null)
})
})
},
})
When I have the downloadedFilename as 'ABCDEF.txt', it works fine [I have hard coded here]. But I need some help to get the (dynamic) file name as it changes every time [eg.: AUADLFA.txt, CIABJPT.txt, SVACJTM.txt, PKPQ1TM.txt & etc.,].
I tried to use '.text' but I get 'Timed out retrying after 4000ms: cy.readFile("C:\Repositories\xyz-testautomation\cypress\downloads/.txt") failed because the file does not exist error.
I referred to this doc as well but no luck yet.
What is the right way to use regex to achieve the same? Also wondering is there a way to get the recently downloaded file name?
You can make use of the task shown in this question How can I verify if the downloaded file contains name that is dynamic
/cypress/plugins/index.js
const fs = require('fs');
on('task', {
downloads: (downloadspath) => {
return fs.readdirSync(downloadspath)
}
})
This returns a list of the files in the downloads folder.
Ideally you'd make it easy on yourself, and set the trashAssetsBeforeRuns configuration. That way, the array will only contain the one file and there's no need to compare arrays before and after the download.
(Just noticed you have a task for it).
I've just started using NextJs getStaticProps, and static files generated at build time is neat. But my contents just don't stay unchanged, I need static files to be updated but rebuilding the app everytime there's a modification is costly. Is there a way to generate new static files only. getServerSideProps turned out to be taking a big amount of time til first byte.
Depending on type of you content Incremental Statatic Regeneration could be a good solution. But in my experience it introduced other problems when rendering catalog/category pages. As nextjs has no idea if one part of you data is dependent on some other it may render an old catalog page with a link to post or product which no longer exists. This usually happens in combination with a 'fallback' feature for dynamic routes.
It also won't refresh you page right after you made changes so you will see a result only after some time.
Possible workaround is to load posts/product on category page dynamically via ajax. You will lose a part of UX and SEO but on the other hand this solution is relatively easy to maintain.
There is also an option or rather hack to rebuild parts of saved content by directly accessing the cache in custom server. Add 'purge=1' to the page address you want to refreshed.
const { ServerResponse, createServer } = require('http')
const next = require('next')
const { promises } = require('fs')
const path = require('path')
const dev = process.env.NODE_ENV !== 'production'
const app = next({ dev })
const handle = app.getRequestHandler()
const purgeByPath = async url => {
const pathname = url == '/' ? '/index' : url
const fullPathname = `.next/server/pages${pathname}`
const fullPathHTML = `${fullPathname}.html`
const fullPathJSON = `${fullPathname}.json`
try {
await promises.unlink(fullPathHTML)
await promises.unlink(fullPathJSON)
} catch (err) {
console.log(`Could not unlink cache files: ${err}`)
}
await app.incrementalCache.cache.del(pathname)
let req = new Request(process.env.BASE_HOST + url)
let res = new ServerResponse(req)
await app.renderToHTML(req, res, url, { _nextDataReq: true })
}
app.prepare().then(() => {
createServer(async (req, res) => {
const url = new URL(req.url, "http://localhost:3000/")
if (req.method == 'POST' && req.url == '/purge-cache') {
let raw = ''
req.on('data', chunk => raw += chunk)
req.on('end', async () => {
const data = JSON.parse(raw)
if (!data || !data.urls) {
res.statusCode = 400
res.end()
return
}
for (let url of data.urls) {
console.log(`Cleaning cache on: ${url}`)
await purgeByPath(url)
}
res.end()
})
} else if (url.searchParams.get('purge') == '1') {
await purgeByPath(req.url)
res.end()
} else {
handle(req, res)
}
}).listen(3000, (err) => {
if (err) throw err
console.log(`> Ready on http://localhost:3000/`)
})
})
The static cache consists of two parts:
Static cache files located at .next/server/pages where each route will probably have two files html and json. Most likely you'll need to delete both.
In-memory cache. Once you app cached a page and saved it to memory it won't go to files on hard drive but instead will load content from memory. So you need to delete it as well.
Cache implementation is not documented an might be swapped or removed in future versions.
This won't work on vercel and has some issues with scaling. You should also add some kind of security token to purge routes.
If I understand it correctly, you are looking for Incremental Statatic Regeneration.
To enable it, you need to add a revalidate time in getStaticProps. As your content is changed and after the revalidation time is reached, a new static page will be generated and served by the server. Depends on how often your content changes, you can change the revalidation time accordingly.
export async function getStaticProps() {
const res = await fetch('https://.../posts')
const posts = await res.json()
return {
props: {
posts,
},
// Next.js will attempt to re-generate the page:
// - When a request comes in
// - At most once every 10 seconds
revalidate: 10, // In seconds
}
}
Reference
https://nextjs.org/docs/basic-features/data-fetching#incremental-static-regeneration
I am a bit out of my comfort zone here, so looking for a bit of guidance. I am trying to access an api to display live metrics, using phonic-elixir (https://www.npmjs.com/package/phoenix-elixir) - am just sort of trying to get it running first, so have loaded up their example code and connecting to an api (forgive me if the terminology is all wrong, I am new at this!)
This is my code:
import {Socket} from 'phoenix-elixir';
let socket = new Socket('ws://API_URL_HERE', {params: {'auth-token': 'AUTH_TOKEN'}})
socket.connect()
let channel = socket.channel('updates:new', {})
channel.join()
.receive('ok', resp => { console.log('Joined successfully', resp) })
.receive('error', resp => { console.log('Unable to join', resp) })
channel.on('update', payload => {
console.log('Received: ' + payload);
console.log(payload);
})
export default socket
When I run babel index.js | node I am getting the error: this.transport = opts.transport || window.WebSocket || LongPoll; and ReferenceError: window is not defined
Just some advice to point me in the right direction would be fantastic. Is window not defined because it needs a dom? Do I need a server to run this in?
Thank you :)
I just ported the client to be compatible with node.JS.
Here is the link https://github.com/mcampa/phoenix-channels
The difference with the original client is that this does not use long-polling and you need to pass the absolute url instead of the relative url.
To install it run:
npm install --save phoenix-channels
Same API as the original:
const { Socket } = require('phoenix-channels')
let socket = new Socket("ws://example.com/socket")
socket.connect()
// Now that you are connected, you can join channels with a topic:
let channel = socket.channel("room:lobby", {})
channel.join()
.receive("ok", resp => { console.log("Joined successfully", resp) })
.receive("error", resp => { console.log("Unable to join", resp) })
phoenix-elixir is client-side library that is supposed to be used in browsers not in node environment. You should create html page with your code and open it in browser to test it out.
I'm looking to create a printable pdf version of my website webpages. Something like express.render() only render the page as pdf
Does anyone know a node module that does that ?
If not, how would you go about implementing one ? I've seen some methods talk about using headless browser like phantom.js, but not sure whats the flow.
Extending upon Mustafa's answer.
A) Install http://phantomjs.org/ and then
B) install the phantom node module https://github.com/amir20/phantomjs-node
C) Here is an example of rendering a pdf
var phantom = require('phantom');
phantom.create().then(function(ph) {
ph.createPage().then(function(page) {
page.open("http://www.google.com").then(function(status) {
page.render('google.pdf').then(function() {
console.log('Page Rendered');
ph.exit();
});
});
});
});
Output of the PDF:
EDIT: Silent printing that PDF
java -jar pdfbox-app-2.0.2.jar PrintPDF -silentPrint C:\print_mypdf.pdf
Phantom.js is an headless webkit server and it will load any web page and render it in memory, although you might not be able to see it, there is a Screen Capture feature, in which you can export the current view as PNG, PDF, JPEG and GIF. Have a look at this example from phantom.js documentation
If you want to export HTML to PDF. You have many options. without node even
Option 1: Have a button on your html page that calls window.print() function. use the browsers native html to pdf. use media queries to make your html page look good on a pdf. and you also have the print before and after events that you can use to make changes to your page before print.
Option 2. htmltocanvas or rasterizeHTML. convert your html to canvas , then call toDataURL() on the canvas object to get the image . and use a JavaScript library like jsPDF to add that image to a PDF file. Disadvantage of this approach is that the pdf doesnt become editable. If you want data extracted from PDF, there is different ways for that.
Option 3. #Jozzhard answer
Try to use Puppeteer to create PDF from HTML
Example from here https://github.com/chuongtrh/html_to_pdf
Or https://github.com/GoogleChrome/puppeteer
The best solution I found is html-pdf. It's simple and work with big html.
https://www.npmjs.com/package/html-pdf
Its as simple as that:
pdf.create(htm, options).toFile('./pdfname.pdf', function(err, res) {
if (err) {
console.log(err);
}
});
NOTE:
This package has been deprecated
Author message: Please migrate your projects to a newer library like puppeteer
Package
I used html-pdf
Easy to use and allows not only to save pdf as file, but also pipe pdf content to a WriteStream (so I could stream it directly to Google Storage to save there my reports).
Using css + images
It takes css into account. The only problem I faced - it ignored my images. The solution I found was to replace url in src attrribute value by base64, e.g.
<img src="data:image/png;base64,iVBOR...kSuQmCC">
You can do it with your code or to use one of online converters, e.g. https://www.base64-image.de/
Compile valid html code from html fragment + css
I had to get a fragment of my html document (I just appiled .html() method on jQuery selector).
Then I've read the content of the relevant css file.
Using this two values (stored in variables html and css accordingly) I've compiled a valid html code using Template string
var htmlContent = `
<!DOCTYPE html>
<html>
<head>
<style>
${css}
</style>
</head>
<body id=direct-sellers-bill>
${html}
</body>
</html>`
and passed it to create method of html-pdf.
Create PDF from External URL
Here's an adaptation of the previous answers which utilizes html-pdf, but also combines it with requestify so it works with an external URL:
Install your dependencies
npm i -S html-pdf requestify
Then, create the script:
//MakePDF.js
var pdf = require('html-pdf');
var requestify = require('requestify');
var externalURL= 'http://www.google.com';
requestify.get(externalURL).then(function (response) {
// Get the raw HTML response body
var html = response.body;
var config = {format: 'A4'}; // or format: 'letter' - see https://github.com/marcbachmann/node-html-pdf#options
// Create the PDF
pdf.create(html, config).toFile('pathtooutput/generated.pdf', function (err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/pathtooutput/generated.pdf' }
});
});
Then you just run from the command line:
node MakePDF.js
Watch your beautify pixel perfect PDF be created for you (for free!)
For those who don't want to install PhantomJS along with an instance of Chrome/Firefox on their server - or because the PhantomJS project is currently suspended, here's an alternative.
You can externalize the conversions to APIs to do the job. Many exists and varies but what you'll get is a reliable service with up-to-date features (I'm thinking CSS3, Web fonts, SVG, Canvas compatible).
For instance, with PDFShift (disclaimer, I'm the founder), you can do this simply by using the request package:
const request = require('request')
request.post(
'https://api.pdfshift.io/v2/convert/',
{
'auth': {'user': 'your_api_key'},
'json': {'source': 'https://www.google.com'},
'encoding': null
},
(error, response, body) => {
if (response === undefined) {
return reject({'message': 'Invalid response from the server.', 'code': 0, 'response': response})
}
if (response.statusCode == 200) {
// Do what you want with `body`, that contains the binary PDF
// Like returning it to the client - or saving it as a file locally or on AWS S3
return True
}
// Handle any errors that might have occured
}
);
Use html-pdf
var fs = require('fs');
var pdf = require('html-pdf');
var html = fs.readFileSync('./test/businesscard.html', 'utf8');
var options = { format: 'Letter' };
pdf.create(html, options).toFile('./businesscard.pdf', function(err, res) {
if (err) return console.log(err);
console.log(res); // { filename: '/app/businesscard.pdf' }
});
const fs = require('fs')
const path = require('path')
const utils = require('util')
const puppeteer = require('puppeteer')
const hb = require('handlebars')
const readFile = utils.promisify(fs.readFile)
async function getTemplateHtml() {
console.log("Loading template file in memory")
try {
const invoicePath = path.resolve("./invoice.html");
return await readFile(invoicePath, 'utf8');
} catch (err) {
return Promise.reject("Could not load html template");
}
}
async function generatePdf() {
let data = {};
getTemplateHtml()
.then(async (res) => {
// Now we have the html code of our template in res object
// you can check by logging it on console
// console.log(res)
console.log("Compiing the template with handlebars")
const template = hb.compile(res, { strict: true });
// we have compile our code with handlebars
const result = template(data);
// We can use this to add dyamic data to our handlebas template at run time from database or API as per need. you can read the official doc to learn more https://handlebarsjs.com/
const html = result;
// we are using headless mode
const browser = await puppeteer.launch();
const page = await browser.newPage()
// We set the page content as the generated html by handlebars
await page.setContent(html)
// we Use pdf function to generate the pdf in the same folder as this file.
await page.pdf({ path: 'invoice.pdf', format: 'A4' })
await browser.close();
console.log("PDF Generated")
})
.catch(err => {
console.error(err)
});
}
generatePdf();
In case you arrive here looking for a way to make PDF from view templates in Express, a colleague and I made express-template-to-pdf
which allows you to generate PDF from whatever templates you're using in Express - Pug, Nunjucks, whatever.
It depends on html-pdf and is written to use in your routes just like you use res.render:
const pdfRenderer = require('#ministryofjustice/express-template-to-pdf')
app.set('views', path.join(__dirname, 'views'))
app.set('view engine', 'pug')
app.use(pdfRenderer())
If you've used res.render then using it should look obvious:
app.use('/pdf', (req, res) => {
res.renderPDF('helloWorld', { message: 'Hello World!' });
})
You can pass options through to html-pdf to control the PDF document page size etc
Merely building on the excellent work of others.
In my view, the best way to do this is via an API so that you do not add a large and complex dependency into your app that runs unmanaged code, that needs to be frequently updated.
Here is a simple way to do this, which is free for 800 requests/month:
var CloudmersiveConvertApiClient = require('cloudmersive-convert-api-client');
var defaultClient = CloudmersiveConvertApiClient.ApiClient.instance;
// Configure API key authorization: Apikey
var Apikey = defaultClient.authentications['Apikey'];
Apikey.apiKey = 'YOUR API KEY';
var apiInstance = new CloudmersiveConvertApiClient.ConvertWebApi();
var input = new CloudmersiveConvertApiClient.HtmlToPdfRequest(); // HtmlToPdfRequest | HTML to PDF request parameters
input.Html = "<b>Hello, world!</b>";
var callback = function(error, data, response) {
if (error) {
console.error(error);
} else {
console.log('API called successfully. Returned data: ' + data);
}
};
apiInstance.convertWebHtmlToPdf(input, callback);
With the above approach you can also install the API on-premises or on your own infrastructure if you prefer.
In addition to #Jozzhart Answer, you can make a local html; serve it with express; and use phantom to make PDF from it; something like this:
const exp = require('express');
const app = exp();
const pth = require("path");
const phantom = require('phantom');
const ip = require("ip");
const PORT = 3000;
const PDF_SOURCE = "index"; //index.html
const PDF_OUTPUT = "out"; //out.pdf
const source = pth.join(__dirname, "", `${PDF_SOURCE}.html`);
const output = pth.join(__dirname, "", `${PDF_OUTPUT}.pdf`);
app.use("/" + PDF_SOURCE, exp.static(source));
app.use("/" + PDF_OUTPUT, exp.static(output));
app.listen(PORT);
let makePDF = async (fn) => {
let local = `http://${ip.address()}:${PORT}/${PDF_SOURCE}`;
phantom.create().then((ph) => {
ph.createPage().then((page) => {
page.open(local).then(() =>
page.render(output).then(() => { ph.exit(); fn() })
);
});
});
}
makePDF(() => {
console.log("PDF Created From Local File");
console.log("PDF is downloadable from link:");
console.log(`http://${ip.address()}:${PORT}/${PDF_OUTPUT}`);
});
and index.html can be anything:
<h1>PDF HEAD</h1>
LINK
result:
https://www.npmjs.com/package/dynamic-html-pdf
I use dynamic-html-pdf, this is simple and also able to pass dynamic variable to html.
var html = fs.readFileSync('./uploads/your-html-tpl.html', 'utf8');
var options = {
format: "A4",
orientation: "portrait"
// border: "10mm"
};
var document = {
type: 'file', // 'file' or 'buffer'
template: html,
context: {
'your_key':'your_values'
},
path: '/pdf/1.pdf' // pdf save path
};
pdf.create(document, options)
.then(res => {
console.log(res)
}).catch(error => {
console.error(error)
});
On html you can use {{your_key}}
I've written hpdf lib for generating PDF from HTLM or URL.
It supports configurable pool of headless browsers (as resources) in the background.
import fs from 'fs';
import { PdfGenerator } from './src';
const start = async () => {
const generator = new PdfGenerator({
min: 3,
max: 10,
});
const helloWorld = await generator.generatePDF('<html lang="html">Hello World!</html>');
const github = await generator.generatePDF(new URL('https://github.com/frimuchkov/hpdf'));
await fs.promises.writeFile('./helloWorld.pdf', helloWorld);
await fs.promises.writeFile('./github.pdf', github);
await generator.stop();
}
I wanted to add to this since I did not see the option to created pdfs from liquid templates yet, but the solution also works with normal html or urls as well.
Lets say this is our html template. Which could be anything really but see that the code include double curly braces. The key inside the braces will be looked up in the liquid_data parameter of the request and replaced by the value.
<html>
<body>
<h1>{{heading}}</h1>
<img src="{{img_url}}"/>
</body>
</html>
The corresponding liquid_data object looks like this:
{
"heading":"Hi Stackoverflow!",
"img_url":"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg"
}
This is the example I want to create a PDF for. Using pdfEndpoint and the Playground creating a pdf from that template from above is very simple.
const axios = require("axios");
const options = {
method: "POST",
url: "https://api.pdfendpoint.com/v1/convert",
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer SIGN-UP-FOR-KEY"
},
data: {
"delivery_mode": "json",
"page_size": "A4",
"margin_top": "1cm",
"margin_bottom": "1cm",
"margin_left": "1cm",
"margin_right": "1cm",
"orientation": "vertical",
"html": "<html><body> <h1>{{heading}}</h1> <img src=\"{{img_url}}\"/> </body>\</html>",
"parse_liquid": true,
"liquid_data": "{ \"heading\":\"Hi Stackoverflow!\", \"img_url\":\"https://stackoverflow.design/assets/img/logos/so/logo-stackoverflow.svg\"}"
}
};
axios.request(options).then(function (response) {
console.log(response.data);
}).catch(function (error) {
console.error(error);
});
The service will the return a rendered pdf like this:
You can also use pdf node creator package
Package URL -
https://www.npmjs.com/package/pdf-creator-node