The name list is supposedly as below:
Rose : 35621548
Jack : 32658495
Lita : 63259547
Seth : 27956431
Cathy: 75821456
Given you have a variable as StudentCode that contains the list above (I think const will do! Like:
const StudentCode = {
[Jack]: [32658495],
[Rose]: [35621548],
[Lita]: [63259547],
[Seth]: [27956431],
[Cathy]:[75821456],
};
)
So here are the questions:
1st: Ho can I define them in URL below:
https://www.mylist.com/student=?StudentCode
So the link for example for Jack will be:
https://www.mylist.com/student=?32658495
The URL is imaginary. Don't click on it please.
2nd: By the way the overall list is above 800 people and I'm planning to save an external .js file to be called within the current code. So tell me about that too. Thanks a million
Given
const StudentCode = {
"Jack": "32658495",
"Rose": "35621548",
"Lita": "63259547",
"Seth": "27956431",
"Cathy": "75821456",
};
You can construct urls like:
const urls = Object.values(StudentCode).map((c) => `https://www.mylist.com?student=${c}`)
// urls: ['https://www.mylist.com?student=32658495', 'https://www.mylist.com?student=35621548', 'https://www.mylist.com?student=63259547', 'https://www.mylist.com?student=27956431', 'https://www.mylist.com?student=75821456']
To get the url for a specific student simply do:
const url = `https://www.mylist.com?student=${StudentCode["Jack"]}`
// url: 'https://www.mylist.com?student=32658495'
Not sure I understand your second question - 800 is a rather low number so will not be any performance issues with it if that is what you are asking?
The properties of the object (after the trailing comma is removed) can be looped through using a for-in loop, (see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...in)
This gives references to each key of the array and the value held in that key can be referenced using objectName[key], Thus you will loop through your object using something like:
for (key in StudentCode) {
keyString = key; // e.g = "Jack"
keyValue = StudentCode[key]; // e.g. = 32658495
// build the urls and links
}
to build the urls, string template literals will simplify the process (see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals) allowing you to substitute values in your string. e.g.:
url = `https://www.mylist.com/student=?${StudentCode[key]`}
Note the use of back ticks and ${} for the substitutions.
Lastly, to build active links, create an element and sets its innerHTML property to markup built using further string template literals:
let link = `<a href=${url}>${keyValue}</a>`
These steps are combined in the working snippet here:
const StudentCode = {
Jack: 32658495,
Rose: 35621548,
Lita: 63259547,
Seth: 27956431,
Cathy: 75821456,
};
const studentLinks = [];
for (key in StudentCode) {
let url = `https://www.mylist.com/student=?${StudentCode[key]}`;
console.log(url);
studentLinks.push(`<a href href="url">${key}</a>`)
}
let output= document.createElement('div');
output.innerHTML = studentLinks.join("<br>");
document.body.appendChild(output);
The method I use I need to put +13 and -1 inside the calculation when searching the position of each part of the text (const Before and const After), is there a more reliable and correct way?
const PositionBefore = TextScript.indexOf(Before)+13;
const PositionAfter = TextScript.indexOf(After)-1;
My fear is that for some reason the search text changes and I forget to change the numbers for the calculation and this causes an error in the retrieved text.
The part of text i'm return is date and hour:
2021-08-31 19:12:08
function Clock() {
var sheet = SpreadsheetApp.getActive().getSheetByName('Clock');
var url = 'https://int.soccerway.com/';
const contentText = UrlFetchApp.fetch(url).getContentText();
const $ = Cheerio.load(contentText);
const Before = '"timestamp":"';
const After = '});\n block.registerForCallbacks();';
var ElementSelect = $('script:contains(' + Before + ')');
var TextScript = ElementSelect.html().replace("\n","");
const PositionBefore = TextScript.indexOf(Before)+13;
const PositionAfter = TextScript.indexOf(After)-1;
sheet.getRange(1, 1).setValue(TextScript.substring(PositionBefore, PositionAfter));
}
Example full text colected in var TextScript:
(function() {
var block = new HomeMatchesBlock('block_home_matches_31', 'block_home_matches', {"block_service_id":"home_index_block_homematches","date":"2021-08-31","display":"all","timestamp":"2021-08-31 19:12:08"});
block.registerForCallbacks();
$('block_home_matches_31_1_1').observe('click', function() { block.filterContent({"display":"all"}); }.bind(block));
$('block_home_matches_31_1_2').observe('click', function() { block.filterContent({"display":"now_playing"}); }.bind(block));
block.setAttribute('colspan_left', 2);
block.setAttribute('colspan_right', 2);
TimestampFormatter.format('block_home_matches_31');
})();
There is no way to eliminate the risk of structural changes to the source content.
You can take some steps to minimize the likelihood that you forget to change your code - for example, by removing the need for hard-coded +13 and -1. But there can be other reasons for your code to fail, beyond that.
It's probably more important to make it extremely obvious when your code does fail.
Consider the following sample (which does not use Cheerio, for simplicity):
function demoHandler() {
var url = 'https://int.soccerway.com/';
const contentText = UrlFetchApp.fetch(url).getContentText();
var matchedJsonString = contentText.match(/{.*?"timestamp".*?}/)[0];
if ( matchedJsonString ) {
try {
var json = JSON.parse(matchedJsonString);
} catch(err) {
console.log( err ); // "SyntaxError..."
}
console.log(json.timestamp)
} else {
consle.log( 'Something went terribly wrong...' )
}
}
When you run the above function it prints the following to the console:
2021-08-31 23:18:46
It does this by assuming the key value of "timestamp" is part of a JSON string, starting with { and ending with }.
You can therefore extract this JSON string and convert it to a JavaScript object and then access the timestamp value directly, without needing to handle substrings.
If the JSON is not valid you will get an explicit error similar to this:
[SyntaxError: Unexpected token c in JSON at position 0]
Scraping web page data almost always has these types of risk: Your code can be brittle and break easily if the source structure changes without warning. Just try to make suc changes as noticeable as possible. In your case, write the errors to your spreadsheet and make it really obvious (red, bold, etc.).
And make good use of try...catch statements. See: try...catch
How would you convert from XML to JSON and then back to XML?
The following tools work quite well, but aren't completely consistent:
xml2json
Has anyone encountered this situation before?
I think this is the best one: Converting between XML and JSON
Be sure to read the accompanying article on the xml.com O'Reilly site, which goes into details of the problems with these conversions, which I think you will find enlightening. The fact that O'Reilly is hosting the article should indicate that Stefan's solution has merit.
https://github.com/abdmob/x2js - my own library (updated URL from http://code.google.com/p/x2js/):
This library provides XML to JSON (JavaScript Objects) and vice versa javascript conversion functions. The library is very small and doesn't require any other additional libraries.
API functions
new X2JS() - to create your instance to access all library functionality. Also you could specify optional configuration options here
X2JS.xml2json - Convert XML specified as DOM Object to JSON
X2JS.json2xml - Convert JSON to XML DOM Object
X2JS.xml_str2json - Convert XML specified as string to JSON
X2JS.json2xml_str - Convert JSON to XML string
Online Demo on http://jsfiddle.net/abdmob/gkxucxrj/1/
var x2js = new X2JS();
function convertXml2JSon() {
$("#jsonArea").val(JSON.stringify(x2js.xml_str2json($("#xmlArea").val())));
}
function convertJSon2XML() {
$("#xmlArea").val(x2js.json2xml_str($.parseJSON($("#jsonArea").val())));
}
convertXml2JSon();
convertJSon2XML();
$("#convertToJsonBtn").click(convertXml2JSon);
$("#convertToXmlBtn").click(convertJSon2XML);
These answers helped me a lot to make this function:
function xml2json(xml) {
try {
var obj = {};
if (xml.children.length > 0) {
for (var i = 0; i < xml.children.length; i++) {
var item = xml.children.item(i);
var nodeName = item.nodeName;
if (typeof (obj[nodeName]) == "undefined") {
obj[nodeName] = xml2json(item);
} else {
if (typeof (obj[nodeName].push) == "undefined") {
var old = obj[nodeName];
obj[nodeName] = [];
obj[nodeName].push(old);
}
obj[nodeName].push(xml2json(item));
}
}
} else {
obj = xml.textContent;
}
return obj;
} catch (e) {
console.log(e.message);
}
}
As long as you pass in a jquery dom/xml object: for me it was:
Jquery(this).find('content').eq(0)[0]
where content was the field I was storing my xml in.
I've created a recursive function based on regex, in case you don't want to install library and understand the logic behind what's happening:
const xmlSample = '<tag>tag content</tag><tag2>another content</tag2><tag3><insideTag>inside content</insideTag><emptyTag /></tag3>';
console.log(parseXmlToJson(xmlSample));
function parseXmlToJson(xml) {
const json = {};
for (const res of xml.matchAll(/(?:<(\w*)(?:\s[^>]*)*>)((?:(?!<\1).)*)(?:<\/\1>)|<(\w*)(?:\s*)*\/>/gm)) {
const key = res[1] || res[3];
const value = res[2] && parseXmlToJson(res[2]);
json[key] = ((value && Object.keys(value).length) ? value : res[2]) || null;
}
return json;
}
Regex explanation for each loop:
res[0] - return the xml (as is)
res[1] - return the xml tag name
res[2] - return the xml content
res[3] - return the xml tag name in case the tag closes itself. In example: <tag />
You can check how the regex works here:
https://regex101.com/r/ZJpCAL/1
Note: In case json has a key with an undefined value, it is being removed.
That's why I've inserted null at the end of line 9.
I was using xmlToJson just to get a single value of the xml.
I found doing the following is much easier (if the xml only occurs once..)
let xml =
'<person>' +
' <id>762384324</id>' +
' <firstname>Hank</firstname> ' +
' <lastname>Stone</lastname>' +
'</person>';
let getXmlValue = function(str, key) {
return str.substring(
str.lastIndexOf('<' + key + '>') + ('<' + key + '>').length,
str.lastIndexOf('</' + key + '>')
);
}
alert(getXmlValue(xml, 'firstname')); // gives back Hank
You can also use txml. It can parse into a DOM made of simple objects and stringify. In the result, the content will be trimmed. So formating of the original with whitespaces will be lost. But this could be used very good to minify HTML.
const xml = require('txml');
const data = `
<tag>tag content</tag>
<tag2>another content</tag2>
<tag3>
<insideTag>inside content</insideTag>
<emptyTag />
</tag3>`;
const dom = xml(data); // the dom can be JSON.stringified
xml.stringify(dom); // this will return the dom into an xml-string
Disclaimer: I am the author of txml, the fastest xml parser in javascript.
A while back I wrote this tool https://bitbucket.org/surenrao/xml2json for my TV Watchlist app, hope this helps too.
Synopsys: A library to not only convert xml to json, but is also easy to debug (without circular errors) and recreate json back to xml. Features :- Parse xml to json object. Print json object back to xml. Can be used to save xml in IndexedDB as X2J objects. Print json object.
Disclaimer: I've written fast-xml-parser
Fast XML Parser can help to convert XML to JSON and vice versa. Here is the example;
var options = {
attributeNamePrefix : "#_",
attrNodeName: "attr", //default is 'false'
textNodeName : "#text",
ignoreAttributes : true,
ignoreNameSpace : false,
allowBooleanAttributes : false,
parseNodeValue : true,
parseAttributeValue : false,
trimValues: true,
decodeHTMLchar: false,
cdataTagName: "__cdata", //default is 'false'
cdataPositionChar: "\\c",
};
if(parser.validate(xmlData)=== true){//optional
var jsonObj = parser.parse(xmlData,options);
}
If you want to parse JSON or JS object into XML then
//default options need not to set
var defaultOptions = {
attributeNamePrefix : "#_",
attrNodeName: "#", //default is false
textNodeName : "#text",
ignoreAttributes : true,
encodeHTMLchar: false,
cdataTagName: "__cdata", //default is false
cdataPositionChar: "\\c",
format: false,
indentBy: " ",
supressEmptyNode: false
};
var parser = new parser.j2xParser(defaultOptions);
var xml = parser.parse(json_or_js_obj);
Here' a good tool from a documented and very famous npm library that does the xml <-> js conversions very well: differently from some (maybe all) of the above proposed solutions, it converts xml comments also.
var obj = {name: "Super", Surname: "Man", age: 23};
var builder = new xml2js.Builder();
var xml = builder.buildObject(obj);
In 6 simple ES6 lines:
xml2json = xml => {
var el = xml.nodeType === 9 ? xml.documentElement : xml
var h = {name: el.nodeName}
h.content = Array.from(el.childNodes || []).filter(e => e.nodeType === 3).map(e => e.textContent).join('').trim()
h.attributes = Array.from(el.attributes || []).filter(a => a).reduce((h, a) => { h[a.name] = a.value; return h }, {})
h.children = Array.from(el.childNodes || []).filter(e => e.nodeType === 1).map(c => h[c.nodeName] = xml2json(c))
return h
}
Test with echo "xml2json_example()" | node -r xml2json.es6 with source at https://github.com/brauliobo/biochemical-db/blob/master/lib/xml2json.es6
I would personally recommend this tool. It is an XML to JSON converter.
It is very lightweight and is in pure JavaScript. It needs no dependencies. You can simply add the functions to your code and use it as you wish.
It also takes the XML attributes into considerations.
var xml = ‘<person id=”1234” age=”30”><name>John Doe</name></person>’;
var json = xml2json(xml);
console.log(json);
// prints ‘{“person”: {“id”: “1234”, “age”: “30”, “name”: “John Doe”}}’
Here's an online demo!
There is an open sourced library Xml-to-json with methods jsonToXml(json) and xmlToJson(xml).
Here's an online demo!
This function directly reads the DOM properties of the XMLDocument (or document node/element) to build the JSON completely and accurately without trying to guess or match. Pass it responseXML, not responseText from XMLHttpRequest.
xml2json(xmlDoc)
If you only have a string of XML and not an XMLDocument, jQuery will convert your text to one.
xml2json($(xmlString)[0])
Each node becomes an object. (All elements are nodes, not all nodes are elements (e.g. text within an element).)
Every object contains the node name and type.
If it has attributes, they appear as properties in an attributes object.
If it has children, they appear recursively as node->objects in a children array.
If it's a Text, CDATA, or Comment node (bare text between element tags) or a comment, it shouldn't have attributes or children but the text will be in a text property.
{
// Always present
"name": "FancyElement",
"type": "Element",
// If present
"attributes: {
"attr1": "val1",
"attr2": "val2"
},
"children": [...],
"text": "buncha fancy words"
}
Caveat: I'm not familiar with all the node types. It's probably not grabbing needed/useful info from all of them. It was tested on and behaves as expected for
Element
Text
CDATA
Comment
Document
function xml2json(xml) {
try {
const types = [null,
"Element",
"Attribute",
"Text",
"CDATA",
"EntityReference", // Deprecated
"Entity", // Deprecated
"ProcessingInstruction",
"Comment",
"Document",
"DocumentType",
"DocumentFragment",
"Notation" // Deprecated
];
var o = {};
o.name = xml.nodeName;
o.type = types[xml.nodeType];
if (xml.nodeType == 3 ||
xml.nodeType == 4 ||
xml.nodeType == 8 ) {
o.text = xml.textContent;
} else {
if (xml.attributes) {
o.attributes = {};
for (const a of xml.attributes) {
o.attributes[a.name] = a.value;
}
}
if (xml.childNodes.length) {
o.children = [];
for (const x of xml.childNodes) {
o.children.push(xml2json(x))
}
}
}
return (o);
} catch (e) {
alert('Error in xml2json. See console for details.');
console.log('Error in xml2json processing node:');
console.log(o);
console.log('Error:');
console.log(e);
}
}
var doc = document.getElementById("doc");
var out = document.getElementById("out");
out.innerText = JSON.stringify(xml2json(doc), null, 2);
/* Let's process the whole Code Snippet #document, why not?
* Yes, the JSON we just put in the document body and all
* this code is encoded in the JSON in the console.
* In that copy you can see why the XML DOM will all be one line.
* The JSON in the console has "\n" nodes all throughout.
*/
console.log(xml2json(document));
#doc,
#out {
border: 1px solid black;
}
<div id="doc"><!-- The XML DOM will all be on one line --><div personality="bubbly" relevance=42>This text is valid for HTML.<span>But it probably shouldn't be siblings to an element in XML.</span></div></div>
<pre id="out"></pre>
The best way to do it using server side as client side doesn't work well in all scenarios. I was trying to build online json to xml and xml to json converter using javascript and I felt almost impossible as it was not working in all scenarios. Ultimately I ended up doing it server side using Newtonsoft in ASP.MVC. Here is the online converter http://techfunda.com/Tools/XmlToJson
I'm trying to write a parser that supports the following type of query clauses
from: A person
at: a specific company
location: The person's location
So a sample query would be like -
from:Alpha at:Procter And Gamble location:US
How do i write this generic parser in javascript. Also, I was considering including AND operators inside queries like
from:Alpha AND at:Procter And Gamble AND location:US
However, this would conflict with the criteria value in any of the fields (Procter And Gamble)
Use a character like ";" instead of AND and then call theses functions:
var query = 'from:Alpha;at:Procter And Gamble;location:US';
var result = query.split(';').map(v => v.split(':'));
console.log(result);
And then you'll have an array of pairs, which array[0] = prop name and array[1] = prop value
var query = 'from:Alpha;at:Procter And Gamble;location:US';
var result = query.split(';').map(v => v.split(':'));
console.log(result);
Asuming your query will always look like this from: at: location:
You can do this:
const regex = /from:\s*(.*?)\s*at:\s*(.*?)\s*location:\s*(.*)\s*/
const queryToObj = query => {
const [,from,at,location] = regex.exec(query)
return {from,at,location}
}
console.log(queryToObj("from:Alpha at Betaat: Procter And Gamble location: US"))
However, adding a terminator allow you to mix order and lowering some keywords:
const regex = /(\w+):\s*(.*?)\s*;/g
const queryToObj = query => {
const obj = {}
let temp
while(temp = regex.exec(query)){
let [,key,value] = temp
obj[key] = value
}
return obj
}
console.log(queryToObj("from:Alpha at Beta;at:Procter And Gamble;location:US;"))
console.log(queryToObj("at:Procter And Gamble;location:US;from:Alpha at Beta;"))
console.log(queryToObj("from:Alpha at Beta;"))
Website that I'm making is in two different languages each data is saved in mongodb with prefix _nl or _en
With a url I need to be able to set up language like that:
http://localhost/en/This-Is-English-Head/This-Is-English-Sub
My code look like that:
var headPage = req.params.headPage;
var subPage = req.params.subPage;
var slug = 'name';
var slugSub = 'subPages.slug_en';
var myObject = {};
myObject[slugSub] = subPage;
myObject[slug] = headPage;
console.log(myObject);
Site.find(myObject,
function (err, pages) {
var Pages = {};
pages.forEach(function (page) {
Pages[page._id] = page;
});
console.log(Pages);
});
After console.log it I get following:
{ 'subPages.slug_en': 'This-Is-English-Sub',
name: 'This-Is-English-Head' }
Is you can see objectname subPages.slug_en is seen as a String insteed of object name..
I know that javascript does not support underscores(I guess?) but I'm still looking for a fix, otherwise i'll be forced to change all underscores in my db to different character...
Edit:
The final result of console.log need to be:
{ subPages.slug_en: 'This-Is-English-Sub',
name: 'This-Is-English-Head' }
Insteed of :
{ 'subPages.slug_en': 'This-Is-English-Sub',
name: 'This-Is-English-Head' }
Otherwise it does not work
The reason you are seeing 'subPages.slug_en' (with string quotes) is because of the . in the object key, not the underscore.
Underscores are definitely supported in object keys without quoting.
Using subPages.slug_en (without string quotes) would require you to have an object as follows:
{ subPages: {slug_en: 'This-Is-English-Sub'},
name: 'This-Is-English-Head' }
Which you could set with the following:
myObject['subPages']['slug_en'] = subPage;
Or simply:
myObject.subPages.slug_en = subPage;