We are loading yammer embedded feed on our page using an Open Graph snippet. We are attempting to load it on the window.onload event. Intermittently it seems to be throwing an HTTP 404. These are examples:
"HTTP404: NOT FOUND - The server has not found anything matching the
requested URI (Uniform Resource Identifier). POST -
https://www.yammer.com/office365 "
"HTTP404: NOT FOUND - The server has not found anything matching the requested URI (Uniform Resource Identifier). (XHR)GET - https://www.yammer.com/api/v1/open_graph_objects?url=myurl&properties%5Burl%5D=myurl&properties%5Btitle%5D=mytitle&properties%5Btype%5D=page&properties%5Bimage%5D=&properties%5Bdescription%5D=&properties%5Bsite_name%5D=&properties%5Bfetch%5D=true&properties%5Bignore_canonical_url%5D=false&properties%5Bprivate%5D=true&private=false&_=1556012959748 "
This is the snippet that we are using:
var attach = function () {
var yammerGroupId = "";
if (window.location.href.indexOf('mydomain.sharepoint.com') > 0) {
yammerGroupId = 0000000;
}
else {
yammerGroupId = 0000000;
}
myDescription = "";
yam.connect.embedFeed({
container: "#embedded-feed",
feedType: "open-graph",
config: {
use_sso: true,
header: false,
footer: false,
showOpenGraphPreview: false,
defaultToCanonical: false,
hideNetworkName: false,
defaultGroupId: yammerGroupId,
promptText: "Comment on this page..."
},
objectProperties: {
type: "page",
url:window.location.href,
fetch: true,
title:document.title,
private: true,
ignore_canonical_url: false,
description: ""
}
});
}
if (window.addEventListener) {
window.addEventListener('load', attach, false)
} else if (window.attachEvent) {
window.attachEvent('onload', attach)
}
Please note that the Window onload is getting fired as expected. It works fine 7/10 times, but gives 404 error intermittently on Edge, IE, and Chrome.
It's expected that you get a 404 from /api/v1/open_graph_objects per the documentation when the URL passed as a parameter isn't an Open Graph Object (OGO) in the Yammer network. Check the URL parameter is actually valid.
For example, you set url: myUrl, but you are probably best to set this directly with "url": "window.location.href" since the value doesn't seem to be set right from your code. Use the configuration tool to work out better JavaScript to use since your code looks invalid.
I am using the i18next script in my jQuery Mobile page in Cordova/PhoneGap app.
I have the below scripts included in the bottom of HTML page.
<script src="js/jquery-1.8.3.min.js"></script>
<script src="js/jquery.mobile-1.3.2.min.js"></script>
<script src="js/i18next-1.7.1.min.js"></script>
<script src="js/main.js"></script>
The main.js file has some logic which will be included in all pages of my app.
main.js File:
function doBootstrap() {
i18n.init({lng: "en", fallbackLng: 'en'});
var header = "some tags";
header += '<h1>' + i18n.t("app.title") + '</h1>';
// other functions
}
I will be using the script to get the translated values across the page in different sections
The above function is called in Cordova devideready function which is placed beow the above mentioned includes. .
document.addEventListener("deviceready", onDeviceReady, true);
function onDeviceReady() {
doBootstrap();
}
With all the above setup I get the below error in i18next-1.7.1.min.js file.
Uncaught TypeError: Cannot read property 'defaultValue' of undefined
The .json file is present in \locales\en\translation.json and the content of it is below. No error or warning is displayed in console.
{
"app": {
"title": "Title"
}
}
What I am missing with the plugin setup?
Finally, I have made it to work. It was a bug in the code and the developer fixed it within minutes of reporting it. Great job..
Here is the link for the issue.
https://github.com/jamuhl/i18next/issues/166
Also as per the developer of plugin, I did the following changes.
function doBootstrap() {
var opts = {
getAsync: true, lng: "en", fallbackLng: 'en'
};
i18n.init(opts).done(doBootstrapMain);
}
function doBootstrapMain() {
// my regular functions
}
I am trying to validate the output that the below script generates. The information is being parsed through a proxy which has all the information encoded correctly (for example & being &) however when I use the W3 Validator I get the following
& did not start a character reference. (& probably should have been
escaped as &.)
…://www.youtube.com/watch?v=pgALxO5r7_0&feature=youtube_gdata_player"
class="wa…
I have tried but to no success to figure out what is going wrong between the proxy and the output. Any help would be appreciated. I think the issue is around
src: escape( $(this).find('link').text()),
Full Source:
<script type="text/javascript">
$(document).ready(function() {
projekktor('#player_a', {
useYTIframeAPI: false,
width: 730,
height: 452,
plugin_display: {
logoImage: "includes/images/transIcon.png"
},
controls: true,
playlist: [{0:{src:'includes/proxy.php?url=http://gdata.youtube.com/feeds/base/users/SkiBsandDXL/uploads?alt=rss&v=2&orderby=published', type:"text/xml"}}],
reelParser: function(xmlDocument) {
var result = {};
var regMatch = new RegExp("http:[^ ,]+\.jpg");
result['playlist'] = [];
$(xmlDocument).find("item").each(function() {
try {
result['playlist'].push({
0:{
src: escape( $(this).find('link').text()),
type: 'video/youtube'
},
config: {
poster: regMatch.exec(unescape( $(this).find('description').text())),
title: $(this).find('title').text(),
desc: $(this).find('description').text()
}
});
} catch(e){}
});
return result;
}
});
});
</script>
I'm going to take a few wild guesses here:
Guess one is that you are using an XHTML doctype. Unless you know how exactly how XHTML differs from HTML then use HTML. HTML 4.01 strict or HTML5.
Again, working on my guess that your working with XHTML, the contents of your script element need to be CDATA. This is reason enough to not use XHTML.
If you must use XHTML, then either put in the CDATA wrapper, or make your script external. Having the scripts external is always a good idea anyways.
I have a JsFiddle here, and added Microsoft AJAX to be loaded through external JS/resource section. How can I tell whether or not my JS code is run after the AJAX file has finished loading?
Seems that the AJAX does not load either. :(
Here is the code in the JSFiddle:
Type.registerNamespace("Tutorial.Chapter1");
Tutorial.Chapter1.Person = function(firstName, lastName) {
this._firstName = firstName;
this._lastName = lastName;
};
Tutorial.Chapter1.Person.prototype = {
set_firstName: function(value) {
this._firstName = value;
},
get_firstName: function() {
return this._firstName;
},
set_lastName: function(value) {
this._lastName = value;
},
get_lastName: function() {
return this._lastName;
},
_firstName: "",
_lastName: "",
displayName: function() {
alert("Hi! " + this._firstName + " " + this._lastName);
}
};
Tutorial.Chapter1.Person.registerClass("Tutorial.Chapter1.Person", null);
The External Resources tab of jsFiddle is currently somewhat tricky and unstable to use.
The resources defined here are often not correctly included into the code. There seems to be an issue with the automatic recognition of JS and CSS resources. If this happens, the external resource is simply not added to the head section of the resulting code. You can check that by reviewing the source code of the Result frame of your jsFiddle. You will find that your MS AJAX resource is simply NOT mentioned in the resulting HTML code.
The correct recognition can actually be forced by adding a dummy value to the resource's URL like this (see –>jsFiddle docs for more info):
...&dummy=.js
Here is an example that shows how to add the external Google Maps API resource to a jsFiddle (mind the dummy parameter at the very end!):
https://maps.googleapis.com/maps/api/js?sensor=false&dummy=.js
Unfortunately this won't work for you as the MS AJAX URL will fail when additional parameters are appended.
A solution (and currently the safest way to load external resources) is to avoid the External Resources tab altogether and load external code manually in the first line(s) of jsFiddle's HTML window like this:
<script type='text/javascript' src="http://ajax.aspnetcdn.com/ajax/3.5/MicrosoftAjax.js"></script>
Here is your jsFiddle modified to use that method: http://jsfiddle.net/rEzW5/12/
It actually does not do a lot (I did not check what is wrong with the rest of your code), but at least it does not throw JavaScript errors anymore.
Open "Add Resources" section and add the url of your external script...
#Jpsy's approach no longer seems to work (see my comment under his answer).
For me, adding the resource under External Resources also didn't work. (According to the Firefox Debugger, it couldn't find the resource).
The only way I was able to get an external bit of JavaScript code (in my case jquery.backstretch.js) to work, was to use Google to find a Fiddle which used this resource (and worked), then Fork this Fiddle and copy/paste all my code into the HTML, CSS and JavaScript panels. Ugh!
#clayRay, You absolutely went thru a code surgery. Just resolved that by mentioning external source in plain html which in my case is
<script src="https://code.jquery.com/jquery-2.2.4.min.js"></script>
Using External Resources tab didn't help a bit...
I am working on screen scraping, and want to retrieve the source code a particular page.
How can achieve this with javascript? Please help me.
Simple way to start, try jQuery
$("#links").load("/Main_Page #jq-p-Getting-Started li");
More at jQuery Docs
Another way to do screen scraping in a much more structured way is to use YQL or Yahoo Query Language. It will return the scraped data structured as JSON or xml.
e.g.
Let's scrape stackoverflow.com
select * from html where url="http://stackoverflow.com"
will give you a JSON array (I chose that option) like this
"results": {
"body": {
"noscript": [
{
"div": {
"id": "noscript-padding"
}
},
{
"div": {
"id": "noscript-warning",
"p": "Stack Overflow works best with JavaScript enabled"
}
}
],
"div": [
{
"id": "notify-container"
},
{
"div": [
{
"id": "header",
"div": [
{
"id": "hlogo",
"a": {
"href": "/",
"img": {
"alt": "logo homepage",
"height": "70",
"src": "http://i.stackoverflow.com/Content/Img/stackoverflow-logo-250.png",
"width": "250"
}
……..
The beauty of this is that you can do projections and where clauses which ultimately gets you the scraped data structured and only the data what you need (much less bandwidth over the wire ultimately)
e.g
select * from html where url="http://stackoverflow.com" and
xpath='//div/h3/a'
will get you
"results": {
"a": [
{
"href": "/questions/414690/iphone-simulator-port-for-windows-closed",
"title": "Duplicate: Is any Windows simulator available to test iPhone application? as a hobbyist who cannot afford a mac, i set up a toolchain kit locally on cygwin to compile objecti … ",
"content": "iphone\n simulator port for windows [closed]"
},
{
"href": "/questions/680867/how-to-redirect-the-web-page-in-flex-application",
"title": "I have a button control ....i need another web page to be redirected while clicking that button .... how to do that ? Thanks ",
"content": "How\n to redirect the web page in flex application ?"
},
…..
Now to get only the questions we do a
select title from html where url="http://stackoverflow.com" and
xpath='//div/h3/a'
Note the title in projections
"results": {
"a": [
{
"title": "I don't want the function to be entered simultaneously by multiple threads, neither do I want it to be entered again when it has not returned yet. Is there any approach to achieve … "
},
{
"title": "I'm certain I'm doing something really obviously stupid, but I've been trying to figure it out for a few hours now and nothing is jumping out at me. I'm using a ModelForm so I can … "
},
{
"title": "when i am going through my project in IE only its showing errors A runtime error has occurred Do you wish to debug? Line 768 Error:Expected')' Is this is regarding any script er … "
},
{
"title": "I have a java batch file consisting of 4 execution steps written for analyzing any Java application. In one of the steps, I'm adding few libs in classpath that are needed for my co … "
},
{
……
Once you write your query it generates a url for you
http://query.yahooapis.com/v1/public/yql?q=select%20title%20from%20html%20where%20url%3D%22http%3A%2F%2Fstackoverflow.com%22%20and%0A%20%20%20%20%20%20xpath%3D'%2F%2Fdiv%2Fh3%2Fa'%0A%20%20%20%20&format=json&callback=cbfunc
in our case.
So ultimately you end up doing something like this
var titleList = $.getJSON(theAboveUrl);
and play with it.
Beautiful, isn’t it?
Javascript can be used, as long as you grab whatever page you're after via a proxy on your domain:
<html>
<head>
<script src="/js/jquery-1.3.2.js"></script>
</head>
<body>
<script>
$.get("www.mydomain.com/?url=www.google.com", function(response) {
alert(response)
});
</script>
</body>
You could simply use XmlHttp (AJAX) to hit the required URL and the HTML response from the URL will be available in the responseText property. If it's not the same domain, your users will receive a browser alert saying something like "This page is trying to access a different domain. Do you want to allow this?"
You can use fetch:
const URL = 'https://www.sap.com/belgique/index.html';
fetch(URL)
.then(res => res.text())
.then(text => {
console.log(text);
})
.catch(err => console.log(err));
As a security measure, Javascript can't read files from different domains. Though there might be some strange workaround for it, I'd consider a different language for this task.
If you absolutely need to use javascript, you could load the page source with an ajax request.
Note that with javascript, you can only retrieve pages that are located under the same domain with the requesting page.
Using jquery
<html>
<head>
<script src="http://jqueryjs.googlecode.com/files/jquery-1.3.2.js" ></script>
</head>
<body>
<script>
$.get("www.google.com", function(response) { alert(response) });
</script>
</body>
I used ImportIO. They let you request the HTML from any website if you set up an account with them (which is free). They let you make up to 50k requests per year. I didn't take them time to find an alternative, but I'm sure there are some.
In your Javascript, you'll basically just make a GET request like this:
var request = new XMLHttpRequest();
request.onreadystatechange = function() {
jsontext = request.responseText;
alert(jsontext);
}
request.open("GET", "https://extraction.import.io/query/extractor/THE_PUBLIC_LINK_THEY_GIVE_YOU?_apikey=YOUR_KEY&url=YOUR_URL", true);
request.send();
Sidenote: I found this question while researching what I felt like was the same question, so others might find my solution helpful.
UPDATE: I created a new one which they just allowed me to use for less than 48 hours before they said I had to pay for the service. It seems that they shut down your project pretty quick now if you aren't paying. I made my own similar service with NodeJS and a library called NightmareJS. You can see their tutorial here and create your own web scraping tool. It's relatively easy. I haven't tried to set it up as an API that I could make requests to or anything.
You can bypass the same-origin-policy by either creating a browser extension or even saving the file as .hta in Windows (HTML Application).
Despite many comments to the contrary I believe that it is possible to overcome the same origin requirement with simple JavaScript.
I am not claiming that the following is original because I believe I saw something similar elsewhere a while ago.
I have only tested this with Safari on a Mac.
The following demonstration fetches the page in the base tag and and moves its innerHTML to a new window. My script adds html tags but with most modern browsers this could be avoided by using outerHTML.
<html>
<head>
<base href='http://apod.nasa.gov/apod/'>
<title>test</title>
<style>
body { margin: 0 }
textarea { outline: none; padding: 2em; width: 100%; height: 100% }
</style>
</head>
<body onload="w=window.open('#'); x=document.getElementById('t'); a='<html>\n'; b='\n</html>'; setTimeout('x.innerHTML=a+w.document.documentElement.innerHTML+b; w.close()',2000)">
<textarea id=t></textarea>
</body>
</html>
javascript:alert("Inspect Element On");
javascript:document.body.contentEditable = 'true';
document.designMode='on';
void 0;
javascript:alert(document.documentElement.innerHTML);
Highlight this and drag it to your bookmarks bar and click it when you wanna edit and view the current sites source code.
You can generate a XmlHttpRequest and request the page,and then use getResponseText() to get the content.
You can use the FileReader API to get a file, and when selecting a file, put the url of your web page into the selection box.
Use this code:
function readFile() {
var f = document.getElementById("yourfileinput").files[0];
if (f) {
var r = new FileReader();
r.onload = function(e) {
alert(r.result);
}
r.readAsText(f);
} else {
alert("file could not be found")
}
}
}
jquery is not the way of doing things.
Do in purre javascript
var r = new XMLHttpRequest();
r.open('GET', 'yahoo.comm', false);
r.send(null);
if (r.status == 200) { alert(r.responseText); }
<script>
$.getJSON('http://www.whateverorigin.org/get?url=' + encodeURIComponent('hhttps://example.com/') + '&callback=?', function (data) {
alert(data.contents);
});
</script>
Include jQuery and use this code to get HTML of other website. Replace example.com with your website.
This method involves an external server fetching the sites HTML & sending it to you. :)
On linux
download slimerjs (slimerjs.org)
download firefox version 59
add this environment variable: export SLIMERJSLAUNCHER=/home/en/Letöltések/firefox59/firefox/firefox
on slimerjs download page use this .js program (./slomerjs program.js):
var page = require('webpage').create();
page.open(
'http://www.google.com/search?q=görény',
function()
{
page.render('goo2.pdf');
phantom.exit();
}
);
Use pdftotext to get text on the page.
const URL = 'https://wwww.w3schools.com';
fetch(URL)
.then(res => res.text())
.then(text => {
console.log(text);
})
.catch(err => console.log(err));
const URL = 'https://www.sap.com/belgique/index.html';
fetch(URL)
.then(res => res.text())
.then(text => {
console.log(text);
})
.catch(err => console.log(err));