How to monitor HTTP calls using browsermob-proxy and nightwatch.js? - javascript

I am writing testcases using Nightwatch.js framework for SPA application. A requirement came in here we have to monitor HTTP calls and get the performance results for the site. As this could be easily achieved using JMeter.
Using automation testing tool, we can do it by using browsermob-proxy and selenium.
Is it possible to do the same using Nightwatch.js and browsermob-proxy?
Also what are the steps to do to the same.

For using Nightwatchjs and browsermob-proxy together, check out this repo, which includes info on the NodeJS bindings for browsermob-proxy and programmatically generating HAR (HTTP Archive) files.
If you're content with just using Nightwatchjs, this repo has code in the tests directory for the following:
Custom command to get the requests made so far
Custom assertion for checking if a request, given a filter and query string params, exists.
You might have to brush up on how to add custom commands and assertions to your Nightwatch project, but after that you should be set to go!

You can use browsermob-proxy-api
just simply download browsermob-proxy server then
install by npm command: npm install browsermob-proxy-api --save-dev
configure you night watch like this in desiredCapabilites:
'test_settings': {
'default': {
'launch_url': 'http://localhost:3000',
'screenshots': {
'enabled': true, // if you want to keep screenshots
'path': './screenshots' // save screenshots here
},
'globals': {
'waitForConditionTimeout': 30000 // sometimes internet is slow so wait.
},
'desiredCapabilities': { // use Chrome as the default browser for tests
'browserName': 'chrome',
'proxy': {
'proxyType': 'manual',
'httpProxy': 'localhost:10800'
},
'acceptSslCerts': true,
'javascriptEnabled': true, // turn off to test progressive enhancement
}
},
then download index.js from here:
https://github.com/jmangs/node-browsermob-proxy-api
and add code from example to your step_definitions if you use gherkin or describe step

Bit late into dance. I managed to integrate browsermob to nightwatch. Here are the detailed steps
Download browsermob proxy https://bmp.lightbody.net/
Open your cmd and go to bin folder and then start browsermob using "browsermob-proxy".
I am assuming you have basic nightwatch setup. You also need mobproxy. Install it from "npm i browsermob-proxy-api"
Create a global hook in nightwatch. Say 'globalmodule.js' and give this file path in globals_path in nightwatch.json
In globalmodule, create global hooks as described in http://nightwatchjs.org/guide#external-globals
In beforeEach hook, add below code: //if you are not under corporate proxy and you dont need to chain to upstream proxy
var MobProxy = require('browsermob-proxy-api');
var proxyObj = new MobProxy({'host': 'localhost', 'port': '8080'});
//assuming you started browsermob in 8080 port. That is in step 2.
//if you are working under corporate proxy, you might have to chain your request. This needs editing in browsermob-proxy-api package. Follow steps given at end of this section.
Start proxy on new port
proxyObj.startPort(port, function (err, data) {
if (err) {
console.log(err);
} else {
console.log('New port started')
}
})
Once we have new port, we have to start our chrome browser in above port so that all browser request are proxied through browsermob.
proxyObj.startPort(port, function (err, data) {
if (err) {
console.log(err);
} else {
console.log('New port started')
var dataInJson = JSON.parse(data);
//Step 8:
this.test_settings.desiredCapabilities = {
"browserName": "chrome",
"proxyObj": proxyObj, //for future use
"proxyport": dataInJson.port, //for future use
"proxy": {
"proxyType": "manual",
"httpProxy": "127.0.0.1:" + dataInJson.port,
"sslProxy": "127.0.0.1:" + dataInJson.port //important is you have https site
},
"javascriptEnabled": true,
"acceptSslCerts": true,
"loggingPrefs": {
"browser": "ALL"
}
}
}
})
Try to run with above setting, you can check if cmd [created in step2 to confirm request are going via above port. There will be some activiy]
For creating HAR and getting created HAR, browsermob-proxy-api gives excellent api.
add createHAR.js in any path and mention that path in nightwatch.json[custom_commands section]
exports.command = function (callback) {
var self = this;
if (!self.options.desiredCapabilities.proxyObj) {
console.error('No proxy setup - did you call setupProxy() ?');
}
this.options.desiredCapabilities.proxyObj.createHAR(this.options.desiredCapabilities.proxyport, {
'captureHeaders': 'true',
'captureContent': 'true',
'captureBinaryContent': 'true',
'initialPageRef': 'homepage'
}, function (err, result){
if(err){
console.log(err)
}else{
console.log(result)
if (typeof callback === "function") {
console.log(this.options.desiredCapabilities.proxyObj);
console.log(this.options.desiredCapabilities.proxyport);
// console.log(result);
callback.call(self, result);
}
}
});
return this;
};
then to getHAR, add getHAR.js, add below code.
var parsedData;
exports.command = function(callback) {
var self = this;
if (!self.options.desiredCapabilities.proxy) {
console.error('No proxy setup - did you call setupProxy() ?');
}
self.options.desiredCapabilities.proxyObj.getHAR(self.options.desiredCapabilities.proxyport, function (err, data) {
console.log(self.options.desiredCapabilities.proxyObj);
console.log(self.options.desiredCapabilities.proxyport);
//console.log(result);
if(err){
console.log(err)
}else{
parsedData = JSON.parse(data)
console.log(parsedData.log.entries)
}
if (typeof callback === "function") {
console.log(self.options.desiredCapabilities.proxyObj);
console.log(self.options.desiredCapabilities.proxyport);
callback.call(self, parsedData);
}
});
return this;
};
At start of test, createHAR will not have proxyObj, So this step should be executed sync. Wrap that step with browser.perform()
browser.perform(function(){
browser.createHAR()
})
////some navigation
browser.perform(function(){
browser.getHAR()
})
Note: If you are working behind corporate proxy, You might have to use chain proxy piece which browsermob offers.
According to browsermob proxy documentation, get down to api section, -> /proxy can have request parameters "proxyUsername" and "proxyPassword"
In node_modules->browsermob-proxy-api->index.js
add below line after line 22:
this.proxyUsername = cfg.proxyUsername || '';
this.proxyPassword = cfg.proxyPassword || '';
this.queryString = cfg.queryString || 'httpProxy=yourupstreamProxy:8080'; //you will get this from pac file
then at line 177, where package is making request '/proxy' to browser.
replace
path: url
to
path: url + '?proxyUsername=' +this.proxyUsername + '&proxyPassword=' + this.proxyPassword + '&' + this.queryString

Related

API client in Javascript

I need a little help to solve a problem in my project.
Scenario:
First: I have a SPA web site that is being developed in Vue.js.
Second: I also have a Web API spec in Swagger that I want to use to generate my client code in Javascript.
Lastly: I'm using swagger-codegen-cli.jar for that.
What I've done until now
1 - Download the last swagger-codegen-cli.jar stable version with javascript support:
curl http://central.maven.org/maven2/io/swagger/swagger-codegen-cli/2.4.7/swagger-codegen-cli-2.4.7.jar -o swagger-codegen-cli.jar
2 - Generate the client code using:
java -jar swagger-codegen-cli.jar generate -i http://192.168.0.85:32839/api/swagger/v1/swagger.json -l javascript -o ./web_api_client/
3 - Add the generated module to my project:
"dependencies": {
// ...
"vue": "^2.6.10",
"vue-router": "^3.0.3",
"web_api_client": "file:./web_api_client"
},
4 - Execute npm install. Apparently, it's working fine.
5 - At this moment I faced the problem. For some reason, the module generated isn't loaded completely.
export default {
name: 'home',
components: {
HelloWorld
},
mounted() {
var WebApiClient = require("web_api_client");
var defaultClient = WebApiClient.ApiClient.instance;
var oauth2 = defaultClient.authentications["oauth2"];
oauth2.accessToken = "YOUR ACCESS TOKEN";
var apiInstance = new WebApiClient.VersaoApi();
var callback = function(error, data, response) {
if (error) {
console.error(error);
} else {
console.log('API called successfully. Returned data: ' + data);
}
};
apiInstance.apiVersaoGet(callback);
}
}
6 - The line var WebApiClient = require("web_api_client"); is working without any error, however, not working 100%. The instance of the module has been created but empty. For instance, WebApiClient.ApiClient is always undefined.
7 - I took a look at the generated code and I think the problem is related with the way the module is being loaded.
(function(factory) {
if (typeof define === 'function' && define.amd) {
// AMD. Register as an anonymous module.
define(['ApiClient', 'api/VersaoApi'], factory);
} else if (typeof module === 'object' && module.exports) {
// CommonJS-like environments that support module.exports, like Node.
module.exports = factory(require('./ApiClient'), require('./api/VersaoApi'));
}
}(function(ApiClient, VersaoApi) {
'use strict';
// ...
In this code, neither of ifs blocks are executed.
Has someone faced a problem like that?
Some advice?
Many thanks, folks.
Solution
After a while trying to fix the problem with require("web_api_client"); I decided to use ES6 instead ES5.
I found an option in swagger-codegen-cli.jar to generate the client code using ES6 as shown below:
java -jar swagger-codegen-cli.jar generate -i http://192.168.0.85:32839/api/swagger/v1/swagger.json -l javascript --additional-properties useES6=true -o ./web_api_client/
Using ES6 I was able to import the javascript module direct from the generated source as shown in the code below.
import WebApiClient from "./web_api_client/src/index";
let defaultClient = WebApiClient.ApiClient.instance;
defaultClient.basePath = 'http://192.168.0.85:32839';
// Configure OAuth2 access token for authorization: oauth2
let oauth2 = defaultClient.authentications["oauth2"];
oauth2.accessToken = "YOUR ACCESS TOKEN";
let apiInstance = new WebApiClient.VersaoApi();
apiInstance.apiVersaoGet((error, data, response) => {
if (error) {
console.error(error);
} else {
console.log("API called successfully. Returned data: " + data + response);
}
});
When I first ran the code I got an error because the module WebApiClient generated didn't have the keyword default in the export block.
Original generated code
export {
/**
* The ApiClient constructor.
* #property {module:ApiClient}
*/
ApiClient,
// ...
Alter changed
export default {
/**
* The ApiClient constructor.
* #property {module:ApiClient}
*/
ApiClient,
// ...
Now everything is working fine.

How to login to gmail account and read a mail in protractor not using the browser

I have a scenario which I need to login to a gmail account and read the content or the message. Then need to get a URL from that message. I can do this using a browser in protractor. But the issue is that gmail account enabled the 2FA. I have achieved this using the core Selenium which has jar files to log in to the gmail account using IMAP protocol.
Can someone please give me a good solution?
You can read emails from Gmail inside protractor tests using mail-listener2 npm package. Check the below example code.
mailListener.ts
const MailListenerClient = require("mail-listener2");
const cheerio = require('cheerio');
const simpleParser = require('mailparser').simpleParser;
export class MailListener {
public mailListener:any;
constructor() {
this.mailListener = new MailListenerClient({
username: "username#gmail.com",
password: "password",
host: "imap.gmail.com",
port: 993,
tls: true,
mailbox: "INBOX",
searchFilter: ["UNSEEN", ["FROM", "fromemail#gmail.com"],["SUBJECT","subject of the email"]],
/*it will search for are "unseen" mail send from "fromemail#gmail.com" with subject "fromemail#gmail.com"*/
connTimeout: 10000,
authTimeout: 5000,
markSeen: true,
mailParserOptions: {streamAttachments: true}, // options to be passed to mailParser lib.
attachments: true, // download attachments as they are encountered to the project directory
attachmentOptions: {directory: "attachments/"},
debug : console.log
});
}
init() {
this.mailListener.start();
}
close() {
this.mailListener.stop();
}
getLinkFromEmail() {
var self = this;
return new Promise((resolve, reject) => {
self.mailListener.on("mail", function (mail) {
/*simpleParser is used to convert string to HTML format*/
simpleParser(mail.eml).then(function (parsedEmail) {
var html = parsedEmail.html;
/* cheerio is used to write query on parsed HTML content
* refer https://www.npmjs.com/package/cheerio
*/
resolve(cheerio.load(html)("a").attr("href"));
});
});
self.mailListener.on("error", function (err) {
reject(err);
});
});
}
}
test.ts
import {MailListener} from "mailListner";
describe("Read email from gmail using imap", function () {
let mailListener = new MailListener();
beforeAll(function(){
mailListener.init();
});
afterAll(function(){
mailListener.close();
})
it("Test email recieved",function(){
let urlFromEmail = mailListener.getLinkFromEmail();
/*Perform some action on UI that triggers email.(Just for example im doing it)*/
element(by.id("email")).sendKeys("email#gmail.com");
element(by.buttonText("Send Email")).click();
expect(urlFromEmail).toEqual("some link");
})
});
I have written the code in typescript and hope you can rewrite the same in javascript. Let me know if this is clear or do I need to add more details to the code.
One of the best practice is to use Gmail API. Take a look at official documentation

Kibana Customized Visualization with ES and Angular Doesn't Work

First, I try to make a custom visualization in Kibana with learning here.
Then, I want my custom visualization to display like the clock how many hits my elasticsearch index has dynamically .
So, I changed some codes in above tutorial but they don't work.
Chrome Devtools tells says Error: The elasticsearch npm module is not designed for use in the browser. Please use elasticsearch-browser
I know I had better use elasticsearch-browser perhaps.
However, I want to understand what is wrong or why.
public/myclock.js
define(function(require) {
require('plugins/<my-plugin>/mycss.css');
var module = require('ui/modules').get('<my-plugin>');
module.controller('MyController', function($scope, $timeout) {
var setTime = function() {
$scope.time = Date.now();
$timeout(setTime, 1000);
};
setTime();
var es = function(){
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host: 'localhost:9200',
log: 'trace'
});
client.search({
index: 'myindex',
}).then(function (resp) {
$scope.tot = resp.hits.total;
}, function (err) {
console.trace(err.message);
});
};
es();
});
function MyProvider(Private) {
...
}
require('ui/registry/vis_types').register(MyProvider);
return MyProvider;
});
public/clock.html
<div class="clockVis" ng-controller="MyController">
{{ time | date:vis.params.format }}
{{tot}}
</div>
Thank you for reading.
Looks like the controller in angularjs treats the elasticsearch javascript client as if it was accessing from the browser.
To elude this, one choice will be by building Server API in index.js and then make kibana access to elasticsearch by executing http request.
Example
index.js
// Server API (init func) will call search api of javascript
export default function (kibana) {
return new kibana.Plugin({
require: ['elasticsearch'],
uiExports: {
visTypes: ['plugins/sample/plugin']
},
init( server, options ) {
// API for executing search query to elasticsearch
server.route({
path: '/api/es/search/{index}/{body}',
method: 'GET',
handler(req, reply) {
// Below is the handler which talks to elasticsearch
server.plugins.elasticsearch.callWithRequest(req, 'search', {
index: req.params.index,
body: req.params.body
}).then(function (error, response) {
reply(response);
});
}
});
}
});
}
controller.js
In the controller, you will need to call GET request for above example.
$http.get( url ).then(function(response) {
$scope.data = response.data;
}, function (response){
$scope.err = "request failed";
});
In my case, I used url instead of absolute or relative path since path of dashboard app was deep.
http://[serverip]:5601/iza/app/kibana#/dashboard/[Dashboard Name]
*
Your here
http://[serverip]:5601/iza/[api path]
*
api path will start here
I used this reference as an example.

How can you use SailsJs as a library?

I'm trying to use sailsjs as a library for a project with specific node files that use models.
# /testfile.js
process.chdir(__dirname);
// Ensure a "sails" can be located:
(function() {
var sails;
try {
sails = require('sails');
} catch (e) {
console.error('To run an app using `node app.js`, you usually need to have a version of `sails` installed in the same directory as your app.');
console.error('To do that, run `npm install sails`');
console.error('');
console.error('Alternatively, if you have sails installed globally (i.e. you did `npm install -g sails`), you can use `sails lift`.');
console.error('When you run `sails lift`, your app will still use a local `./node_modules/sails` dependency if it exists,');
console.error('but if it doesn\'t, the app will run with the global sails instead!');
return;
}
// Try to get `rc` dependency
var rc;
try {
rc = require('rc');
} catch (e0) {
try {
rc = require('sails/node_modules/rc');
} catch (e1) {
console.error('Could not find dependency: `rc`.');
console.error('Your `.sailsrc` file(s) will be ignored.');
console.error('To resolve this, run:');
console.error('npm install rc --save');
rc = function () { return {}; };
}
}
// Start server
sails.lift(rc('sails'));
var req = require('sails/lib/router/req');
var res = require('sails/lib/router/res');
while(!sails.isLifted)
{
}
console.log(sails);
sails.models.user.find({where: {email: 'test#test.com'}}, function (err, result) {
if(err) return res().send('Not found');
return res().send('Found')
});
})();
I don't want a server running but I want all the configuration options to be used. I've edited the globals.js file to expose models.
models: true
After a bit of wrestling, found that sails takes a bit to 'lift'. I'm using a timeout at the moment but will probably clean that bit up after I get more working.

hapi.js best way to handle errors

I'm creating my first node.js REST web service using hapi.js. I'm curious as to the best way to handle errors let's say from my dao layer. Do i throw them in my dao layer and then just try/catch blocks to handle them and send back errors in my controller, or is there a better way that the cool kids are handling this?
routes/task.js
var taskController = require('../controllers/task');
//var taskValidate = require('../validate/task');
module.exports = function() {
return [
{
method: 'POST',
path: '/tasks/{id}',
config : {
handler: taskController.createTask//,
//validate : taskValidate.blah
}
}
]
}();
controllers/task.js
var taskDao = require('../dao/task');
module.exports = function() {
return {
/**
* Creates a task
*
* #param req
* #param reply
*/
createTask: function createTask(req, reply) {
taskDao.createTask(req.payload, function (err, data) {
// TODO: Properly handle errors in hapi
if (err) {
console.log(err);
}
reply(data);
});
}
}();
dao/task.js
module.exports = function() {
return {
createTask: function createTask(payload, callback) {
... Something here which creates the err variable...
if (err) {
console.log(err); // How to properly handle this bad boy
}
}
}();
Generic Solution w/ Fully Customisable Error Template/Messages
We wrote a Hapi Plugin that handles all errors seamlessly: npmjs.com/package/hapi-error
It lets you define your own custom error pages in 3 easy steps.
1. Install the plugin from npm:
npm install hapi-error --save
2. Include the plugin in your Hapi project
Include the plugin when you register your server:
server.register([require('hapi-error'), require('vision')], function (err) {
// your server code here ...
});
See: /example/server_example.js for simple example
3. Ensure that you have a View called error_template
Note: hapi-error plugin expects you are using Vision (the standard view rendering library for Hapi apps)
which allows you to use Handlebars, Jade, React, etc. for your templates.
Your error_template.html (or error_template.ext error_template.jsx) should make use of the 3 variables it will be passed:
errorTitle - the error tile generated by Hapi
statusCode - *HTTP statusCode sent to the client e.g: 404 (not found)
errorMessage - the human-friendly error message
for an example see: /example/error_template.html
That's it! Now your Hapi App handles all types of errors and you can throw your own custom ones too!
Note: hapi-error works for REST/APIs too. if the content type header (headers.acceps) is set to application/json then your app will return a JSON error to the client, otherwise an HTML page will be served.
In doing more research along with Ricardo Barros' comment on using Boom, here's what I ended up with.
controllers/task.js
var taskDao = require('../dao/task');
module.exports = function() {
return {
/**
* Creates a task
*
* #param req
* #param reply
*/
createTask: function createTask(req, reply) {
taskDao.createTask(req.payload, function (err, data) {
if (err) {
return reply(Boom.badImplementation(err));
}
return reply(data);
});
}
}();
dao/task.js
module.exports = function() {
return {
createTask: function createTask(payload, callback) {
//.. Something here which creates the variables err and myData ...
if (err) {
return callback(err);
}
//... If successful ...
callback(null, myData);
}
}();
I think the cool kids now use a package to caught unhandled errors with Hapi, I present to you, Poop.
The only thing Poop is missing is some rich documentation, but check it out, and you'll see that Poop is great.
Some of my friends went to a node.js event in Lisbon, on of the hosts was a guy in charge of web technology stack at Wallmart, they use Hapi.js, Poop and some other cool things.
So if they use poop it must be pretty awesome.
PS: The name is suppa awesome

Categories

Resources