I am trying to capture query string parameters for analytics purpose using javascript. I did some searching and found that BMP can be used to do it but i am unable to find ample examples to implement. Could anyone point me in the right direction.
EDIT 1:
I used below code using browsermob-proxy to get har file but i get ERROR: browsermob-proxy returned error when i run it . I use selenium with it.
getHarFile() {
const proxy = browsermb.Proxy;
const pr = new proxy({host:"0.0.0.0",port:4444});
pr.doHAR("http://www.cnn.com/", (err,data) => {
if (err) {
logger.debug('ERROR: ' + err);
} else {
fs.writeFileSync('ua.com.har', data, 'utf8');
logger.debug("#HAR CREATED#");
}
})
}
So since I´m not quite sure of your scope I will throw you some ideas:
1. Fixing browsermob-proxy
You should change the host and proxy of the browsermob-proxy. Change the host to 127.0.0.1 and the port with any random number (4444 its ok). Then, make sure your browser run in that host and proxy by changing the browser settings.
2. Using plain javascript
2.1 Get current page query string
You can get the query string using location.search. If you are using some BDD framework with selenium, it is possible to execute javascript code and retrieve the result. You should always add a return to your code in order to recieve the response in your BDD test.
2.2 Using Performance API
You can access to all the network information within performance api. If you need to get the current page url you can use the following code:
performance.getEntriesByType("navigation")
This will return all the current navigation events and information.
If you want to get some information of the calls that the page made you can access it using:
performance.getEntriesByType("resource")
This will return all the calls made by your site. You have to loop over it searching the resource you want to find.
In all the ways, there is no way to get the value and key of query string as in the network tab. You have to separate it manually with a function, you can use the code provided here to get the value of a key.
My suggestion is to create your personal extension for Google Chrome, and developing an extension you can access few more apis that are not available by default in the console.
For example you will have this object in order to inspect the network tab:
chrome.devtools.network
Here two links you may find useful:
https://developer.chrome.com/extensions/devtools
https://developer.chrome.com/extensions/devtools_network
I hope it helps
I was finally able to do it using the s object available on chrome console. The url with encoded query string was available as s.rb object in chrome console. I just decoded it and extracted the query string parameters.
Related
I'm a relatively new Javascript programmer and I'm experimenting with the Marvel API (I need to access the images for a project) and having a little trouble wrapping my head around the requirements.
As I understand it, you need to pass a hash and a ts (timestamp, I presume), when calling the API from a server-side app. But I don't see in the documentation that this is required when using a client-side app.
I tried to do some basic endpoint testing with Insomnia and I receive the message "You must provide a hash.". Apparently I need the hash for client-side access as well?
I have seen some NodeJS examples that show you how to generate the hash (for example, https://www.raymondcamden.com/2014/02/02/Examples-of-the-Marvel-API), but nothing for the client side (that I could find). I also don't know how I would generate this within Insomnia (or Postman). Any pointers in the right direction would be appreciated.
I'd also like to ask what role the authorized domains play when accessing the Marvel API from a local machine. Do I need to add localhost to this list?
Thanks for any help!
Follow the steps:
Pick an API Endpoint. eg: https://gateway.marvel.com:443/v1/public/characters
Use a query value for ts. ts could be timestamp or any long string.
eg: ts=thesoer
Generate a MD5 hash of ts+privatekey+publickey through code or preferrably online. eg: md5(ts + privKey + pubKey)
For md5 hash: http://www.md5.cz/
Join the dots. URL?ts=val&apikey=key&hash=md5Hash.
eg. https://gateway.marvel.com:443/v1/public/characters?ts=thesoer&apikey=001ac6c73378bbfff488a36141458af2&hash=72e5ed53d1398abb831c3ceec263f18b
Add a pre-requisite script to your postman collection.
var pubkey = "your_public_key";
var pvtkey = "your_private_key";
var ts = new Date().getTime();
pm.environment.set("ts", ts)
pm.environment.set("apikey", pubkey)
var message = ts+pvtkey+pubkey;
var a = CryptoJS.MD5(message);
pm.environment.set("hash", a.toString())
And then you can make your calls like such
https://gateway.marvel.com/v1/public/characters?ts={{ts}}&apikey={{apikey}}&hash={{hash}}
See this collection.
Regarding your authorized domains, add your public IP.
What I have not been able to find is simple examples (no third party) of neo4j using javascript. I have got the desktop of neo4j working and got an example with a third party graph tool working (the example appears to put the request in the textarea of a DIV and send the request to the graph api and the graph is produced).
I am very familiar with MYSQL, other SQL interaction but having problems interacting with neo4j. Have done a lot of research but stuck.
From my SQL days there was:
connect statement (i.e. get a handle and I have got this to work with neo4j)
send an SQL statement to the database, (in this case it would be cypher)
get the cursor and process the results (I assume process the Jason)
I would like the example to:
Connect to the database (local and remote)
Show sample cypher commands to fetch data (movie dtabase)
How to store returned results in the javascript program
if possible provide a short explanation of Node, HTML, Javascript ie the javascript goes into app.js and there is index.htnl that refers to app.js. Do I have to use Node can I access neo4j with Javascript only?
Thanks
Marty
Take a look at the official Neo4j Driver for Javascript. The driver can be used with node.js and there is also a version that runs in a browser.
The repo's README contains links to full documentation and sample projects.
AS #cybersam told you, you should use the neo4j-javascript-driver.
You can find an example application here : https://github.com/neo4j-examples/movies-javascript-bolt
And this is snippet on how to perform the connection, a query and to parse the result :
// Create a driver instance, for the user neo4j with password neo4j.
// It should be enough to have a single driver per database per application.
var driver = neo4j.driver("bolt://localhost", neo4j.auth.basic("neo4j", "neo4j"));
// Create a session to run Cypher statements in.
// Note: Always make sure to close sessions when you are done using them!
var session = driver.session();
// the Promise way, where the complete result is collected before we act on it:
session
.run('MERGE (james:Person {name : {nameParam} }) RETURN james.name AS name', {nameParam: 'James'})
.then(function (result) {
result.records.forEach(function (record) {
console.log(record.get('name'));
});
session.close();
})
.catch(function (error) {
console.log(error);
});
// Close the driver when application exits.
// This closes all used network connections.
driver.close();
Moreover, you can also take a look at the GRAND stack : http://grandstack.io/
It's stack to build a web application based on React, Neo4j and GraphQl (with Apollo).
I need to retrieve data from a web service (via SOAP) during a nightly maintenance process on a LAMP server. This data then gets applied to a database. My research has returned many options and I think I have lost sight of the forest for the trees; partially because of the mix of client and server terms and perspectives of the articles I have read.
Initially I installed node.js and node-soap. I wrote a simple script to test functionality:
var soap = require('/usr/local/lib/node_modules/npm/node_modules/soap');
var url = "https://api.authorize.net/soap/v1/Service.asmx?WSDL";
soap.createClient(url, function(err, client)
{
if(typeof client == 'undefined')
{
console.log(err);
return;
}
console.log('created');
});
This uses a demo SOAP source and it works just fine. But when I use the actual URL I get a 5023 error:
[Error: Invalid WSDL URL: https://*****.*****.com:999/SeniorSystemsWS/DataExportService.asmx?WSDL
Code: 503
Response Body: <html><body><b>Http/1.1 Service Unavailable</b></body> </html>]
Accessing this URL from a browser returns a proper WSDL definition. I am told by the provider that the 503 is due to a same-origin policy violation. Next, I researched adding CORS to node.js. This triggered my stepping back and asking the question: Am I in the right forest? I'm not sure. So, I am looking for a command-line, SOAP capable, CORS app (or equivalent) configuration. I am a web developer primarily using PHP and Javascript, so Javascript is where I turned first, but that is not a requirement. Ideas? Or, is there a solution to the current script error (the best I think I have found is using jQuery in node.js which includes CORS)
Most likely, this error belongs to your website server.
Please go through this link, it might be helpful.
http://pcsupport.about.com/od/findbyerrormessage/a/503error.htm
Also you can open your wsdl in web browser, search for soap:address location tag under services. And figure out correct url, you are trying to invoke from your script. Directly access this url in browser and see what are you getting.
I think I have a better approach to the task. I found over the weekend that PHP has a full SOAP client. I wrote the same basic login script in PHP and it runs just fine. I get a valid authentication code in the response to loginExt (which is required in further requests), so it looks like things are working. I will comment here after verifying that I can actually use the web service.
I am using the google auth but keep getting an origin mismatch. The project I am working has sub domains that are generated by the user. So for example there can be:
john.example.com
henry.example.com
larry.example.com
In my app settings I have one of my origins being http://*.example.com but I get an origin mismatch. Is there a way to solve this? Btw my code looks like this:
gapi.auth.authorize({
client_id : 'xxxxx.apps.googleusercontent.com',
scope : ['https://www.googleapis.com/auth/plus.me',
state: 'http://henry.example.com',
'https://www.googleapis.com/auth/userinfo.email', 'https://www.googleapis.com/auth/userinfo.profile'],
immediate : false
}, function(result) {
if (result != null) {
gapi.client.load('oath2', 'v2', function() {
console.log(gapi.client);
gapi.client.oauth2.userinfo.get().execute(function(resp) {
console.log(resp);
});
});
}
});
Hooray for useful yet unnecessary workarounds (thanks for complicating yourself into a corner Google)....
I was using Google Drive using the javascript api to open up the file picker, retrieve the file info/url and then download it using curl to my server. Once I finally realized that all my wildcard domains would have to be registered, I about had a stroke.
What I do now is the following (this is my use case, cater it to yours as you need to)
On the page that you are on, create an onclick event to open up a new window in a specific domain (https://googledrive.example.com/oauth/index.php?unique_token={some unique token}).
On the new popup I did all my google drive authentication, had a button to click which opened the file picker, then retrieved at least the metadata that I needed from the file. Then I stored the token (primary key), access_token, downloadurl and filename in my database (MySQL).
Back on step one's page, I created a setTimeout() loop that would run an ajax call every second with that same unique_token to check when it had been entered in the database. Once it finds it, I kill the loop and then retrieve the contents and do with them as I will (in this case I uploaded them through a separate upload script that uses curl to fetch the file).
This is obviously not the best method for handling this, but it's better than entering each and every subdomain into googles cloud console. I bet you can probably do this with googles server side oauth libraries they use, but my use case was a little complicated and I was cranky cause I was frustrated at the past 4 days I've spent on a silly little integration with google.
Wildcard origins are not supported, same for redirect URIs.
The fact that you can register a wildcard origin is a bug.
You can use the state parameter, but be very careful with that, make sure you don't create an open redirector (an endpoint that can redirect to any arbitrary URL).
I'm trying to connect to my CouchDB on Cloudant using jQuery and jQuery.couch.sj
However, I can't even get the most basic info about my database. For example the following code prints nothing to the console.
Code
<script>
$.couch.urlPrefix ="https://acharya.cloudant.com";
$.couch.info({
success: function(data) {
console.log(data);
}
});
</script>
I've looked at the online documentation but to no avail.
If I type
var db= $.couch.db("toxtweet");
console.debug(db);
to see something about one of my CouchDB's, I get:
Object { name="toxtweet",uri="https://acharya.cloudant.com/toxtweet/", compact=function(),
more...}
And that is the correct URI. So, how would I, for example, get the number of documents in the "toxtweet" database? Trying the example doesn't work.
Update
If I view the page in Chrome instead of Firefox I see the following error.
XMLHttpRequest cannot load https://acharya.cloudant.com/. Origin http://tox.sinaiem.org is not
allowed by Access-Control-Allow-Origin.
I thought that Cloudant was a CouchApp that bypassed the same-origin policy.
I haven't used jquery to access Cloudant, but I would expect you to have to log in somewhere first unless you have somehow made your database public.
Have you checked in Chrome or Firefox what https requests and responses jquery.couch is sending and receiving?
To get the number of documents, you would typically have a view with a reduce method like this:
// map
function(doc) {
emit(doc.id, 1);
}
// reduce
function(keys, values, rereduce) {
return sum(values);
}
see here for more info What is the CouchDB equivalent of the SQL COUNT(*) aggregate function?
I would recommend you use Futon when trying out examples before doing an equivalent request in jquery.couch
Update
Have you tried JSONP to get around cross domain issue?
see here: http://support.cloudant.com/customer/portal/articles/359321-how-do-i-read-and-write-to-my-cloudant-database-from-the-browser-
CouchApp's/Cloudant don't bypass same origin policy. If you have a CouchApp on Cloudant you can access it under your domain (e.g. https://acharya.cloudant.com/DB_NAME/_design/DESIGN/index.html), if you want that on another domain you'll need a reverse proxy as AndyD suggests.
The CouchDB wiki has two nice run throughs for using HTTPD or Nginx as a reverse proxy, both should apply when running against a database hosted in Cloudant.
HTH
Simon