I'm trying to develop a language web application. It will scrape data from various websites and ask the participant numerous questions etc. I have created the file that scrapes the web page but I'm having difficulty getting the scraped variables from the node.js file. below are some extracts from the node js file:
var pword = function() {
var request = require("request");
var cheerio = require("cheerio");
var aa = Math.floor(Math.random() * 588);
var words = ['abash', 'aberrant', .....]
var A = words[aa];
var urlcollinsdictionary = "http://www.collinsdictionary.com/dictionary/english/";
var Newurldictionary = urlcollinsdictionary + A;
request({
uri: Newurldictionary,
}, function(error, response, body) {
var $ = cheerio.load(body);
$('div.homograph-entry').each(function() {
var link = $(this);
var text1 = link.text();
console.log(A);
console.log(text1);
});
});
}
My node js code works fine. my problem arises when I try to use its "text1" variable or others of its kind in my HTML/javascript coding. I've tried "getelementbyid", "variable exports/imports", even the "%%" method. but still no luck. as you've probably get guess I'm new to programming. please please please help me by making alterations/additions to the HTML code beneath to enable access to the text1 variable and others like it
<html>
<head>
language game
<title>language game</title>
<br>
<input type = "button" onclick = "word()" value = "Professional Vocab">
</head>
<body>
<script>
function word(){
alert(text1);
}
</script>
</body>
</html>
Thanks in advance...
You can add similar code in your node js file to respond to api calls
var express = require("express");
var app = express();
app.get("sendVar", function(request, response){
response.send(text1);
}
you will need to do npm install express in your command line
In your front-end side you can use jquery and make an AJAX call inside function word()
function word(){
$.get("url/sendVar", function(data){
alert(data);
});
}
Related
I am using gnuplot-js https://github.com/chhu/gnuplot-JS when I am without my laptop at university.
Is it possible to use an external url for gnuplot.js file? I host index.html, gnuplot_api.js on https://mywebsite.com/ but I would like to use another url for gnuplot.js file (for example https://external.com/blabla/ok/no/gnuplot.js) because this file is really big!
Maybe this is the part I should edit on index.html. How? :
<script src='gnuplot_api.js'></script>
<script>
gnuplot = new Gnuplot('gnuplot.js');
gnuplot.onOutput = function(text) {
document.getElementById('output').value += text + '\n';
document.getElementById('output').scrollTop = 99999;
};
or maybe this part on gnuplot_api.js?:
var Gnuplot = function(js_filename) {
this.worker = new Worker(js_filename);
this.output = [];
this.error = [];
this.isRunning = false;
Thank you very much for your support
It appears Gnuplot use an onboard webworker, so the path provided to the constructor should be on the same domain than the script.
<script src='https://cdn.jsdelivr.net/gh/chhu/gnuplot-JS/www/gnuplot_api.js'></script>
<script>
gnuplot = new Gnuplot('https://cdn.jsdelivr.net/gh/chhu/gnuplot-JS/www/gnuplot.js');
</script>
There is a web page. In page source have script:
<script>
var important = [{....}];
</script>
How get information from this variable with use node.js???
In a similar situation, when information was in function:
$(function() {
_very.important ([{....}]);
I use code:
var cloudscraper = require("cloudscraper");
cloudscraper.get("link" , function(error, response, data) {
if (error) {
console.log('ERRRRRRROR');
} else {
var info = JSON.parse(data.split("_very.important(")[1].split(")")[0]);
But, I dont know how work with this problem.
var important = [{....}];
You can assign a id to the script like <script id="script"> and get the details through innerHTML like
document.getElementById('script').innerHTML
I'm trying to use flask with url_for. The problem is that when I try to launch an alert with the value of the javascript variable everything seems ok, but when I try to launch a alert with the url_for the content of the variable is not printed. What I'm doing wrong? or What is missing in my code?
How can I pass a JavaScript variable into the url_for function?
html code:
<a class="dissable_user_btn" data-user_id="{{user.id}}" href="#" title="Change Status"><i class="fa fa-plug"></i>
</a>
JS Code:
<script type="text/javascript">
$(document).ready(function(){
$('.dissable_user_btn').click(function( event ) {
var user_id = $(this).data("user_id")
alert(user_id) //everything ok
alert ('{{url_for('.dissable', _id=user_id)}}'); //dont print the valur of user_id
</script>
Short answer: you can't. Flask & Jinja2 render the template on the server side (e.g. Flask is translating all of the {{ }} stuff before it sends the HTML to the web browser).
For a URL like this where you're including a variable as part of the path you'd need to build this manually in javascript. If this is an XHR endpoint I'd recommend using GET/POST to transfer the values to the server as a better best practice than constructing the URL this way. This way you can use Jinja:
$(document).ready(function(){
var baseUrl = "{{ url_for('disable') }}";
$('.dissable_user_btn').click(function(event) {
var user_id = $(this).data("user_id");
// first part = url to send data
// second part = info to send as query string (url?user=user_id)
// third parameter = function to handle response from server
$.getJSON(baseUrl, {user: user_id}, function(response) {
console.log(response);
});
});
});
I found another solution for this. My problem started when I needed to pass a variable with space.
First I created a function to remove trailing and leading spaces
function strip(str) {
return str.replace(/^\s+|\s+$/g, '');}
After that, I used the function and encoded the URL
<script type="text/javascript">
$(document).ready(function(){
$('.dissable_user_btn').click(function( event ) {
var user_id = $(this).data("user_id")
alert(user_id)
user_id = strip(user_id).replace(" ","%20");
alert ('{{url_for('.dissable', _id='user_id')}}.replace('user_id',user_id);
</script>
It worked pretty nice for me!
This is how I applied to my problem
<script>
function strip(str) {
return str.replace(/^\s+|\s+$/g, '');}
$(document).ready(function() {
$('#exportcountry').click(function() {
var elemento = document.getElementById("countryexportbtn");
var country = strip(elemento.textContent).replace(" ","%20");
$('#exportevent').load("{{ url_for('get_events',country = 'pais') }}".replace('pais',country));
});
});
</script>
I am using Node.JS with Express. The following line fails, and I need help fixing it.
var routines = require("myJsRoutines.js");
When I run index.html and click MenuItem, I get the first alert, but not the second one.
I have both files in the same directory. Thanks
index.html:
<!DOCTYPE html>
<html>
<head>
<title>Test</title>
</head>
<body>
MenuItem
<script>function myMenuFunc(level) {
alert("myMenuFunc1:" + level);
var routines = require("myJsRoutines.js");
alert("myMenuFunc:2" + level);
routines.processClick(level);
alert("myMenuFunc:3" + level);
}</script>
</body>
</html>
myJsRoutines.js:
exports.processClick = function processClick (param1) {
console.log(param1)
}
Script in <script> tags only runs on the client, and script on the server never directly handles DOM events like clicks. There is no magical event wireup - you need to make them interact.
Assuming folder structure from http://expressjs.com/en/starter/generator.html
Updated module code, in /modules/myJsRoutines.js...
var myJsRoutines = (function () {
var multiplier = 2;
return {
processLevel: function (level, callback) {
console.log('processLevel:', level); // CLI or /logs/express_output.log
// validation
if (!level) {
// error is usually first param in node callback; null for success
callback('level is missing or 0');
return; // bail out
}
// processing
var result = level * multiplier;
// could return result, but need callback if code reads from file/db
callback(null, result);
}
};
}()); // function executed so myJsRoutines is an object
module.exports = myJsRoutines;
In /app.js, load your module and add a get method...
var myJsRoutines = require('./modules/myJsRoutines');
app.get('/test', function (req, res) {
var level = parseInt(req.query.level) || 0;
console.log('server level:', level);
myJsRoutines.processLevel(level, function (err, result) {
if (err) {
res.status(500);
return res.send(err);
}
res.send('result ' + (result || '') + ' from the server');
});
});
In /public/index.html, add client script to make an HTTP request to the get method...
<a class="test" href="#" data-level="1">Test Level 1</a>
<a class="test" href="#" data-level="2">Test Level 2</a>
<script src="https://code.jquery.com/jquery-2.2.4.min.js"></script>
<script>
$(function(){ // jQuery DOM ready
$('.test').click(function () { // click event for <a class="test">
var level = $(this).data('level'); // from data-level="N"
var url = '/test?level=' + escape(level);
console.log('client url:', url);
// HTTP GET http://localhost:3000/test?level=
$.get(url, function (data) {
console.log('client data:', data); // browser console
});
return false; // don't navigate to href="#"
});
});
</script>
...start the server from the command line...
npm start
...open http://localhost:3000/ in your browser, Ctrl+Shift+i to open the browser console, and click the links.
Run from a node server..var routines = require("myJsRoutines.js"); in the server.js file and Just call a javascript onclick function..and post parameters..for posting parameters..you'll be needing Ajax...and console log the data in node..or After sending the data to the node server..run the function in node server.
Code snippet for calling the function from a href..
and
`MenuItem
<script type="text/javascript">
function myMenuFunc('Level 1') {
// return true or false, depending on whether you want to allow the `href` property to follow through or not
}
`
This line:
var routines = require("myJsRoutines.js");
fails because the require statement is a nodejs function. It does not work with the browser nor does it work with javscript natively. It is defined in nodejs to load modules. To see this
go to your command line and run this
> node
> typeof require
'function'
go to your browser console; firefox - press Ctrl + K
>> typeof require
"undefined"
To achieve your aim, there are two options that come to my mind
// Assumed Express server running on localhost:80
var express = require('express');
var app = express();
app.get("/myJsRoutines", loadRoutines);
app.listen(80);
Option I: XMLHttpRequest
This is a browser API that allows you to open a connection to a server and talk with the server to collect stuff using HTTP. Here's how you do this
<script>
var request = new XMLHttpRequest(); // create an xmlhttp object
request.open("GET", "/myJsRoutines"); // means GET stuff in there
request.link = link;
// wait for the response
request.addEventListener("readystatechange", function() {
// checks if we are ready to read response
if(this.readyState === 4 && this.status === 200) {
// do something with response
}
})
//send request
request.send();
</script>
Lookup XMLHttpRequest API or the new fetch API
Option II: Pug
Pug, formerly named jade is a templating engine for nodejs. How does it work? You use it to programmatically create the html on the server before sending it.
Lookup the site -> https://pugjs.org/
I'm trying to scrape links on my school's course schedule website using Node.js, request, and cheerio. However, my code is not reaching all subject links.
Link to course schedule website here.
Below is my code:
var express = require('express');
var request = require('request');
var cheerio = require('cheerio');
var app = express();
app.get('/subjects', function(req, res) {
var URL = 'http://courseschedules.njit.edu/index.aspx?semester=2016s';
request(URL, function(error, response, body) {
if(!error) {
var $ = cheerio.load(body);
$('.courseList_section a').each(function() {
var text = $(this).text();
var link = $(this).attr('href');
console.log(text + ' --> ' + link);
});
}
else {
console.log('There was an error!');
}
});
});
app.listen('8080');
console.log('Magic happens on port 8080!');
My output can be found here.
As you can see from my output, some links are missing. More specifically, links from sections 'A', 'I (Continued)', and R '(Continued)'. These are also the first sections of each column.
Each section is contained in its own div with class name 'courseList_section' so I don't understand why '.courseList_section a' doesn't loop through all links. Am I missing something obvious? Any and all insight is very appreciated.
Thank you in advance!
The problem isn't your code, it's the site you're trying to parse that's the problem. The HTML tags are invalid. You're trying to parse everything inside the .courseList_section, but the tags looks like this.
<span> <!-- Opening tag -->
<div class='courseList_section'>
<a href='index.aspx?semester=2016s&ƒ=ACC '>ACC - Accounting/Essex CC</a>
</span> <!-- Invalid closing tag for the first span, menaing that .courseList_section will be closed instead
<!-- Suddenly this link is outside the .courseList_section tag, meaning that it will be ignored by cheerio -->
<a href='index.aspx?semester=2016s&subjectID=ACCT'>ACCT - Accounting</a>
<!-- and so on -->
The solution. Fetch all links and ignore those that arn't related to any course.
var request = require('request');
var cheerio = require('cheerio');
var URL = 'http://courseschedules.njit.edu/index.aspx?semester=2016s';
request(URL, function(error, response, body) {
if(error) { return console.error('There was an error!'); }
var $ = cheerio.load(body);
$('a').each(function() {
var text = $(this).text();
var link = $(this).attr('href');
if(link && link.match(/subjectID/)){
console.log(text + ' --> ' + link);
};
});
});
Next time, try looking directly at the HTML and see if it looks okay. If it looks like ****, pass it trough an HTML beautifier and inspect it again. Not even the beautifier could handle this markup which indicated that something was wrong with the tags.