I crawled some contents by JavaScript, and want to print it on HTML.
JavaScript code below is named 'js.js'(worked well on CMD)
var request = require('request');
var cheerio = require('cheerio');
request('...URL...', function (err, res, body) {
if (err) console.log('Err :' + err);
var $ = cheerio.load(body);
$('.class').each(function () {
var content = $(this).find('.abc').text().trim();
document.write(content);
});
});
But "error:require is not defined" was printed, so I looking for solutions.
I found this page and follow advice which said that use webpack or browseify.
new code(2MB after bundled) give me 2 new error:"fail to fetch" and "access-control-allow-origin". What should I do?
the require() keyword does not exists in browser/client JavaScript, that is why you need to use webpack to transpile nodejs code to browser compatible javascript.
For the "access-control-allow-origin", the url you are tying to connect to does not allow response to unknown origin.
If you own the API/URL you could add a response header Access-Control-Allow-Origin: *.
For reference: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
Related
I'm trying to execute this function but in the terminal with Node.JS
var WebTorrent = require('webtorrent')
var client = new WebTorrent()
var magnetURI = 'magnet: ...'
client.add(magnetURI, { path: '/path/to/folder' }, function (torrent) {
torrent.on('done', function () {
console.log('torrent download finished')
})
})
I mean, for example, create an <button> tag, and when is clicked,
that the previous function be executed in the nodejs console, not in the browser console.
EXTRA:
I'm executing this two files:
app.js
let http = require('http');
let fs = require('fs');
let handleRequest = (request, response) => {
response.writeHead(200, {
'Content-Type': 'text/html'
});
fs.readFile('./index.html', null, function (error, data) {
if (error) {
response.writeHead(404);
respone.write('Whoops! File not found!');
} else {
response.write(data);
}
response.end();
});
};
http.createServer(handleRequest).listen(8000);
And
index.html that contains the <button> tag but does nothing.
Client(browser) and server are two different entities, when client is browser the only way to communicate is through HTTP protocol, in simple terms use internet.
Now browser understand only it's own kind of javascript, more precisely ECMA but not nodejs. So the following code could not be executed in browser
var WebTorrent = require('webtorrent')
var client = new WebTorrent()
Hence I would assume it is running on server which your machine and hence console.log will print to your terminal.
To run it on browser, I assume you will have to code it differently, either you will have to use browserify and analyze the client side script OR code only in client side with below libaray :
<script src="webtorrent.min.js"></script>
For more details refer, complete web page example at https://github.com/webtorrent/webtorrent/blob/master/docs/get-started.md
I get the following error message when I try to run a local node server with a very simple application (see coding below).
Loading module from “http://localhost:8080/importing.js” was blocked because of a disallowed MIME type (“text/html”).
I am new to node and ES6 Modules so I dont really understand the details of the problem. According to this URL the mime-type 'application/javascript' has to be served explicitly for modules. But how do I achieve this in my example below?
index.html
<!DOCTYPE html>
<html>
<head>
<script src="./importing.js" type="module"></script>
<meta charset="utf-8">
</head>
<body>
</body>
</html>
server.js
var http = require('http');
var fs = require('fs');
const PORT=8080;
fs.readFile('./index.html', function (err, html) {
if (err) throw err;
http.createServer(function(request, response) {
response.writeHeader(200, {"Content-Type": "text/html"});
response.write(html);
response.end();
}).listen(PORT);
});
importing.js
import {a} from './exporting.js';
console.log(a);
exporting.js
export const a = 'Constant a';
I start the server in CMD with
node server.js
Essentially you have a server that for any given request, serves the content of your index.html file - regardless of what that request might look like. A browser therefore receives the HTML and begins to interpret it making another request for the src of your script tag and since the server only serves your index.html file, the browser receives your HTML file a second time when it expected javascript.
Typically you'd create a server first then construct responses based on the request as input. A primitive example of serving your static files how you intended might look like the following:
const http = require('http')
const fs = require('fs')
const PORT = 8080
http
.createServer((request, response) => {
fs.readFile(`.${request.url}`, (err, data) => {
if (err) {
response.writeHeader(404, {
'Content-Type': 'text/plain'
})
response.write('404 Not Found')
response.end()
return
}
if (request.url.endsWith('.html')) {
response.writeHeader(200, {
'Content-Type': 'text/html'
})
}
if (request.url.endsWith('.js')) {
response.writeHeader(200, {
'Content-Type': 'application/javascript'
})
}
response.write(data)
response.end()
})
})
.listen(PORT)
Do note that this example is too trusting of the client and you would normally want to sanitise the request in some way. I kept to vanilla javascript but once you're comfortable with how it works, it's worth checking out Express as it will simplify the routing / mime-type boilerplate etc.
I know that you're importing just the command, but I'll let you know my solution for this and see if you're interested. This error, for me, came from the import statement in your module. I was trying to import the entire file, with all functions and imports it had, while essentially using the same server and HTML.
my importing.js:
import * as Spotify from "./spotify-web-api.js";
window.basicAlert = function basicAlert() {
alert("this is a test to see if basicAlert runs properly");
}
console.log("Should print to console with no error on imports");
I don't know the logic behind the import * as, but it worked to successfully import my file without throwing a MIME type error. As for the window.basicAlert =, Javascript apparently doesn't like to give any file that imports it access to its functions or variables unless it's manually attached to the window. You don't have this error now, but after the file imports successfully it will tell you that a isn't defined. While I have it attached to my function in importing.js, you'll need to put it in exporting.js like this:
const a = 'Constant a';
windows.a = a;
I didn't test that ^ but it makes sense to me. I hope this can help you out, or get closer, cause it solved my problem.
I want to download page (https://www.csfd.cz/tvurce/65871) in NodeJS, but I get just random data.
�}Ms�F������+i"��)�Jْ;�e���7�KM0��LƩ��]��Yg��b��
Ow7U��J�#�K�9��L
I thought it is just wrong encoding, but even size is wrong (downloaded page have 44K, whereas this file have only 19K. What's more surprising is that simple downloading it by python works good.
Python code:
import requests
url = "https://www.csfd.cz/tvurce/65871"
r = requests.get(url)
with open('pyth.txt','wb') as handle:
handle.write(r.content)
JavaScript code:
const request = require('request-promise')
const fs = require('fs')
request('https://www.csfd.cz/tvurce/65871').then((html) => {
fs.writeFileSync('output.html', html)
})
I tried also additional methods like request.get with parameters and so on, but still the same result. Can you please tell me what I am doing wrong?
Use compressed option in request module, see example with request module (https://github.com/request/request).
You need also followRedirect and followAllRedirect parameters to automatically follow 301 and 302 redirection cuz your request is returning 302 :
curl -X GET https://www.csfd.cz/tvurce/65871 --compressed -v -i
Response : 302
<h1>Redirect</h1>
<p><a href="https://www.csfd.cz/tvurce/65871-kit-harington/">Please
click here to continue</a>.</p>
In addition replace your writeFileSync with standard writeFile function
const request = require('request')
const fs = require('fs')
request.get({
url:'https://www.csfd.cz/tvurce/65871',
gzip: true,
followRedirect: true,
followAllRedirect: true
}, function(err, response, body){
if(err || !response || response.statusCode != 200)
{
// error case, do stg
}
else
{
fs.writeFile('output.html', body, "utf8", function(err){
if(err)
{
// error do stg
}
else
{
// success
}
});
}
})
I tried different things, different options and encodings, some parsers, and I didn't get it to work with request and request-promise. From the docs, I would say you aren't doing anything wrong.
I tried then a different module, unirest (npm install unirest --save), and it worked out of the box.
const unirest = require('unirest');
const fs = require('fs');
var Request = unirest.get('https://www.csfd.cz/tvurce/65871')
.end(function(res) {
console.log(res.body);
fs.writeFileSync('output.html', res.body)
});
Hope this is of help.
Read the Content-Encoding header. It's most likely compressed, which would explain the size difference.
I am using the Node module Request. I have also used node-fetch in the past but request seems better. I want to be able to log something with a certain ID/Class/Span, you name it. This is my current code which logs the whole body.
const request = require('request');
request("https://discord.js.org/#/docs/main/stable/class/GuildMember", (err, response, body) => {
if (err) console.log(err);
console.log(body)
});
I want to log the contents of any Property the user gives, let's say they give bannable I want to log the contents of div#doc-for-bannable.class-prop.class-item but I'm not sure how to do this.
Edit:
I tried using Cheerio as suggested but it is not giving me any sort of response, just a blank space in my console.The code I tried was:
request("https://discord.js.org/#/docs/main/stable/class/GuildMember", (err, response, body) => {
if (err) console.log(err);
const $ = cheerio.load(body);
const g = $('docs-page'); // this doesn't work either, after trying '#doc-for-bannable`
const gText = g.text();
console.log(gText)
});
The underlying request page builds the DOM after requesting data from backend using XHR/ajax requests.
So when you use 'request', the page is loaded without XHR being fired
However, if you closely look at the Network tab, you will come across
https://raw.githubusercontent.com/hydrabolt/discord.js/docs/stable.json
This has the data that you are looking for.
You can use the nodejs 'request' for this url and consume the json
So I'm trying to make a very basic node.js server that with take in a request for a string, randomly select one from an array and return the selected string. Unfortunately I'm running into a few problems.
Here's the front end:
function newGame()
{
guessCnt=0;
guess="";
server();
displayHash();
displayGuessStr();
displayGuessCnt();
}
function server()
{
xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET","server.js", true);
xmlhttp.send();
string=xmlhttp.responseText;
}
This should send the request to server.js:
var http = require('http');
var choices=["hello world", "goodbye world"];
console.log("server initialized");
http.createServer(function(request, response)
{
console.log("request recieved");
var string = choices[Math.floor(Math.random()*choices.length)];
console.log("string '" + string + "' chosen");
response.on(string);
console.log("string sent");
}).listen(8001);
So clearly there are several things going wrong here:
I get the feeling the way I am "connecting" these two files isn't correct both in the xmlhttp.open method and in using response.on to send the string back to the front end.
I'm a little confused with how I call this page on localhost. The front end is named index.html and the sever posts to 8001. What address should I be go to on localhost in order to access the initial html page after I have initialized server.js? Should I change it to .listen(index.html) or something like that?
are there other obvious problems with how I am implementing this (using .responsetext etc.)
(sorry for the long multi-question post but the various tutorials and the node.js source all assume that the user already has an understanding of these things.)
Your request should be to the server, NOT the server.js file which instantiates it. So, the request should look something like this: xmlhttp.open("GET","http://localhost:8001/", true); Also, you are trying to serve the front-end (index.html) AND serve AJAX requests at the same URI. To accomplish this, you are going to have to introduce logic to your server.js that will differentiate between your AJAX requests and a normal http access request. To do this, you'll want to either introduce GET/POST data (i.e. call http://localhost:8001/?getstring=true) or use a different path for your AJAX requests (i.e. call http://localhost:8001/getstring). On the server end then, you'll need to examine the request object to determine what to write on the response. For the latter option, you need to use the 'url' module to parse the request.
You are correctly calling listen() but incorrectly writing the response. First of all, if you wish to serve index.html when navigating to http://localhost:8001/, you need to write the contents of the file to the response using response.write() or response.end(). First, you need to include fs=require('fs') to get access to the filesystem. Then, you need to actually serve the file.
XMLHttpRequest needs a callback function specified if you use it asynchronously (third parameter = true, as you have done) AND want to do something with the response. The way you have it now, string will be undefined (or perhaps null), because that line will execute before the AJAX request is complete (i.e. the responseText is still empty). If you use it synchronously (third parameter = false), you can write inline code as you have done. This is not recommended as it locks the browser during the request. Asynchronous operation is usually used with the onreadystatechange function, which can handle the response once it is complete. You need to learn the basics of XMLHttpRequest. Start here.
Here is a simple implementation that incorporates all of the above:
server.js:
var http = require('http'),
fs = require('fs'),
url = require('url'),
choices = ["hello world", "goodbye world"];
http.createServer(function(request, response){
var path = url.parse(request.url).pathname;
if(path=="/getstring"){
console.log("request recieved");
var string = choices[Math.floor(Math.random()*choices.length)];
console.log("string '" + string + "' chosen");
response.writeHead(200, {"Content-Type": "text/plain"});
response.end(string);
console.log("string sent");
}else{
fs.readFile('./index.html', function(err, file) {
if(err) {
// write an error response or nothing here
return;
}
response.writeHead(200, { 'Content-Type': 'text/html' });
response.end(file, "utf-8");
});
}
}).listen(8001);
console.log("server initialized");
frontend (part of index.html):
function newGame()
{
guessCnt=0;
guess="";
server();
displayHash();
displayGuessStr();
displayGuessCnt();
}
function server()
{
xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET","http://localhost:8001/getstring", true);
xmlhttp.onreadystatechange=function(){
if (xmlhttp.readyState==4 && xmlhttp.status==200){
string=xmlhttp.responseText;
}
}
xmlhttp.send();
}
You will need to be comfortable with AJAX. Use the mozilla learning center to learn about XMLHttpRequest. After you can use the basic XHR object, you will most likely want to use a good AJAX library instead of manually writing cross-browser AJAX requests (for example, in IE you'll need to use an ActiveXObject instead of XHR). The AJAX in jQuery is excellent, but if you don't need everything else jQuery offers, find a good AJAX library here: http://microjs.com/. You will also need to get comfy with the node.js docs, found here. Search http://google.com for some good node.js server and static file server tutorials. http://nodetuts.com is a good place to start.
UPDATE: I have changed response.sendHeader() to the new response.writeHead() in the code above !!!
Express makes this kind of stuff really intuitive. The syntax looks like below :
var app = require('express').createServer();
app.get("/string", function(req, res) {
var strings = ["rad", "bla", "ska"]
var n = Math.floor(Math.random() * strings.length)
res.send(strings[n])
})
app.listen(8001)
https://expressjs.com
If you're using jQuery on the client side you can do something like this:
$.get("/string", function(string) {
alert(string)
})
I was facing following error with code (nodejs 0.10.13), provided by ampersand:
origin is not allowed by access-control-allow-origin
Issue was resolved changing
response.writeHead(200, {"Content-Type": "text/plain"});
to
response.writeHead(200, {
'Content-Type': 'text/html',
'Access-Control-Allow-Origin' : '*'});
Here is a fully functional example of what you are trying to accomplish. I created the example inside of hyperdev rather than jsFiddle so that you could see the server-side and client-side code.
View Code:
https://hyperdev.com/#!/project/destiny-authorization
View Working Application: https://destiny-authorization.hyperdev.space/
This code creates a handler for a get request that returns a random string:
app.get("/string", function(req, res) {
var strings = ["string1", "string2", "string3"]
var n = Math.floor(Math.random() * strings.length)
res.send(strings[n])
});
This jQuery code then makes the ajax request and receives the random string from the server.
$.get("/string", function(string) {
$('#txtString').val(string);
});
Note that this example is based on code from Jamund Ferguson's answer so if you find this useful be sure to upvote him as well. I just thought this example would help you to see how everything fits together.