Cant parse a dictionary retrieved from xmlhttprequest.responseText [SOLVED]

Cant parse a dictionary retrieved from xmlhttprequest.responseText [SOLVED] - javascript

In my program, I make a call to my python script to return a dictionary. I am trying to access this dictionary in JavaScript after completing the XMLHttpRequest. response text. The dictionary will appear however, I am not able to parse it.
For example: calling this,responseText['Username']; returns undefined
I have tried stringifying the responseText and then parsing it, which hasn't helped at all. What am I doing wrong? I have been stuck on this for too long
Requested function:
def getBasicInfo():
dict = {}
list = []
username = ud["username"]
displayName = ide["displayName"]
accountCreationDate = ud['creationTime']
following = fol["followingUsersCount"]
followers = fol['followerCount']
playlistCount = len(ply["playlists"])
playlists = ply["playlists"]
dict.update({"Username": username})
dict.update({"Display Name": displayName})
dict.update({"Account Creation Date": accountCreationDate})
dict.update({"Following": str(following)})
dict.update({"Followers": str(followers)})
dict.update({"Number of playlists": str(playlistCount)})
for x in range(0, len(ply["playlists"])):
list.append(playlists[x]["name"])
dict.update({"Playlist names": list})
totalNumberSongs = len(sh1) + len(sh2) + len(sh3)
firstTime = sh1[0]['endTime']
lastTime = sh3[-1]['endTime']
dict.update({"Total Number of Songs Analyzed": str(totalNumberSongs)})
timeStamp = firstTime + " to " + lastTime
dict.update({"Data Lifecycle": timeStamp})
return dict
Python Script:
#!\Users\bobs\anaconda3\python.exe #Python Interpreter
from data import *
import cgi
import sys
fs = cgi.FieldStorage()
sys.stdout.write("Content-Type: application/json")
sys.stdout.write("\n")
sys.stdout.write("\n")
if "basic" in fs.keys():
info = getBasicInfo()
print(info)
if "artist" in fs.keys():
artist = countArtistsDesc()
print(artist)
if "song" in fs.keys():
song = countSongsDesc()
print(song)
if "day" in fs.keys():
day = getHighestPlayedDays(True)
print(day)
if "playlist" in fs.keys():
playlist = getPlaylist()
print(playlist)
sys.stdout.write("\n")
sys.stdout.close()
HTML Page/XMLHTTPRequest:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Basic Info</title>
</head>
<body style="background-color:#1DB954">
<h1 align="center">User's Basic Info</h1>
<script>
var http = new XMLHttpRequest();
http.open('GET', '/cgi-bin/simpleData.py?basic=True', true);
http.setRequestHeader('content-type', 'application/x-www-form-urlencoded');
// http.responseType = 'json'
http.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
document.getElementById("responseText").innerHTML = this.responseText;
document.getElementById("problem").innerHTML = this.responseText['Username'];
}
};
http.send(null);
</script>
<p id="responseText"></p>
<p id="problem"></p>
</body>
</html>
Output:
{'Username': 'bobsmith123', 'Display Name': 'Bob Smith', 'Account Creation Date': '2016-10-10', 'Following': '34', 'Followers': '46', 'Number of playlists': '32', 'Playlist names': ['5th Dimension', 'Dumb Chilling', 'Rock v2 / Nu Metal', 'Party 🍻', 'Rap', 'Pumping Iron', 'Aggression', 'Soundcloud 4', 'Oldies but Goodies', 'Chopped and Screwed', 'Cruel Winter', 'Soundcloud', 'Halloween 2020', 'Trap Christmas', "80's Night", 'EDM', 'Life of Pablo Tour', 'Igor Tour', 'Thugger', 'Playboi Carti', 'Cactus Jack', "WAKE UP MR WEST, OH HE'S UP", 'Future', 'Denzel', 'LORDE PRETTY FLACKO JOYDE', 'AstroWorld 🎢🎡🎠', 'Daytona Tour', 'Children of the Korn', 'Rock', 'Classics', 'Floyd', 'Chill Rock'], 'Total Number of Songs Analyzed': '27334', 'Data Lifecycle': '2020-01-24 19:37 to 2021-01-25 20:52'}
undefined
Solution:
#Very end of getBasicInfo()
return json.dumps(dict, separators=(',', ':'))
#After readyState == 4 && this.status == 200 in the HTML page
parsed = JSON.parse(this.responseText);

This is because python represents strings with single quotes, and json requires double quotes.
import json;
body = {'Username': 'bobsmith123', 'Display Name': 'Bob Smith',
'Account Creation Date': '2016-10-10', 'Following': '34',
'Followers': '46', 'Number of playlists': '32',
'Playlist names': [
'5th Dimension', 'Dumb Chilling', 'Rock v2 / Nu Metal', 'Party 🍻', 'Rap',
'Pumping Iron', 'Aggression', 'Soundcloud 4', 'Oldies but Goodies',
'Chopped and Screwed', 'Cruel Winter', 'Soundcloud', 'Halloween 2020',
'Trap Christmas', "80's Night", 'EDM', 'Life of Pablo Tour', 'Igor Tour',
'Thugger', 'Playboi Carti', 'Cactus Jack', "WAKE UP MR WEST, OH HE'S UP",
'Future', 'Denzel', 'LORDE PRETTY FLACKO JOYDE', 'AstroWorld 🎢🎡🎠',
'Daytona Tour', 'Children of the Korn', 'Rock', 'Classics', 'Floyd', 'Chill Rock'],
'Total Number of Songs Analyzed': '27334', 'Data Lifecycle': '2020-01-24 19:37 to 2021-01-25 20:52'
}
print(json.dumps(body))
That produces the output in the correct format
{"Username": "bobsmith123", "Display Name": "Bob Smith", "Account Creation Date": "2016-10-10", "Following": "34", "Followers": "46", "Number of playlists": "32", "Playlist names": ["5th Dimension", "Dumb Chilling", "Rock v2 / Nu Metal", "Party \ud83c\udf7b", "Rap", "Pumping Iron", "Aggression", "Soundcloud 4", "Oldies but Goodies", "Chopped and Screwed", "Cruel Winter", "Soundcloud", "Halloween 2020", "Trap Christmas", "80's Night", "EDM", "Life of Pablo Tour", "Igor Tour", "Thugger", "Playboi Carti", "Cactus Jack", "WAKE UP MR WEST, OH HE'S UP", "Future", "Denzel", "LORDE PRETTY FLACKO JOYDE", "AstroWorld \ud83c\udfa2\ud83c\udfa1\ud83c\udfa0", "Daytona Tour", "Children of the Korn", "Rock", "Classics", "Floyd", "Chill Rock"], "Total Number of Songs Analyzed": "27334", "Data Lifecycle": "2020-01-24 19:37 to 2021-01-25 20:52"}
and as a bonus you get escaped unicode characters.

Related

How can I generate a random selection based on user's <select> option choice?

I am trying to have a user select a genre from my drop down list and get a random result from the genre they selected that is then displayed on the screen.
In other words, my idea is to have a user get a random options from the genre they choose from my list.
I am very very new to JavaScript but I want to try to do something creative for my project. I have tried this so far
function xyz() {
var x = document.getElementById("selectedId").value;
if (x == "action") {
var action = ["Attack on Titan", "Fullmetal Alchimist", "One Punch Man", "Sword Art Online", "My Hero Academia", "Demond Slayer", "Naruto", "Tokyo Ghoul", "Hunter x Hunter", "Code Geass", "Jujutsu Kaisen", "One Piece", "Noragami", "Akame ga Kill!", "Mob Psycho 100", "Assassination Classroom", "Blue Exorcist", "Bleach", "Parastyle", "Cowboy Bebop", "Fairy Tail", "Soul Eater", "Jojo's Bizarre Adventure", "Black Clover", "Fullmetal Alchemist", "That Time I got Reincarnated as a Slime", "Fire Force", "Balck butler", "Spy x Family", "The Seven Deadly Sins", "Tokyo Revengers", "Devilman: Crybaby", "Chainsaw Man"]
var result = Math.floor(Math.random() * action.length);
console.log(action);
} else if (x == "comedy ") {
var comedy = ["One Punch Man", "No Game NO Life", "Mob Psycho 100", "Assassination Classroom", "KonoSuba: God's Blessing on This Wonderful World!", "Kill la Kill", "Rascal Does Not Dream of Bunny Girl Senpai", "Dr.Stone", "Kaguya-sama: Love is War", "Soul Eater", "Gurren Lagann", "The Devil is a Part-Timer", "Black Clover", "High School DxD", "That Time I Got Reincarnated as a Slime", "Maid Sama!", "Spy x Family", "Black Butler", "Nisekoi", "Ouran High School Host Club", "My Little Monster", "JoJo's Bizarre Adventure:", "K-On!", "The Disastrous Life of Saiki K.", "My Teen Romantic Comedy SNAFU TOO!", "Golden Time", "Monthly Girls' Nozaki-kun", "Rent-a-Girlfriend", "The Quintessential Quintuplets", "Don't Toy with Me, Miss Nagatoro", "How Not to Summon a Demon Lord", "Ghost Stories"]
var result = Math.floor(Math.random() * comedy.length);
console.log(comedy);
} else if (x == "drama") {
var drama = ["Attack on Titan", "Fullmetal Alchemist", "Your Name.", "Steins:Gate", "A Silent Voice", "Code Geass", "Toradora!", "Your Lie in Apirl", "Re:Zero", "Angel Beats!", "Death Parade", "Violet Evergarden", "Rascal Does Not Dream of Bunny Girl Senpai", "Anohana: The Flower We Saw That Day", "Clannad", "Charlotte", "Kakegurui", "Made in Abyss", "The Pet Girl of Sakurasou", "Guilty Crown", "Jojo's Bizarren Adventure", "Classroom of the Elite", "Tokyo Revengers", "ReLIFE", "Monster", "5 Centimeters Per Second", "Plasic Memories", "Tower of God", "The Garden of Words", "Banana Fish", "Orange", "To Your Eternity", "Fruits Basket", "91 Days"]
var result = Math.floor(Math.random() * drama.length);
console.log(drama);
} else if (x == "sliceofLife") {
var sliceofLife = ["Violet Evergarden", "Anohana: The Flower We Saw That Day", "Hyouka", "Miss Kobayashi's Dragon Maid", "ReLIFE", "5 Centimeters Per Second", "The Melancholy of Haruhi Suzumiya", "Blue Spring Ride", "Kimi ni Todoke: From Me to You", "I Want To Eat Your Pancreas", "My Dress-Up Darling", "The Garden of Words", "Beastars", "Mushu-Shi", "Wolf Children", "Nana", "Barakamon", "Natsume's Book of Friends", "Bunny Drop", "Laid-Back Camp", "Love-Live!", "Tanaka-Kun is Always Listless", "Kimi ni Todoke", "School-Live!", "Tamako Market", "Beck", "Place to Place", "One Week Friends", "Girls' Last Tour", "Remake Our Life!", "Silver Spoon", "Flying Witch", "Doukyusei-Classmates", "Colorful"]
var result = Math.floor(Math.random() * sliceofLife.length);
console.log(sliceofLife);
} else if (x == "fantasy") {
var fantasy = ["Sword Art Online", "Demon Slayer", "Naruto", "Tokyo Ghoul", "Hunter x Hunter", "No Game No Life", "Jujutsu Kaisen", "One Piece", "Noragami", "Re:Zero", "Akame ga Kill!", "The Seven Deadly Sins", "Bleach", "Blue Exorcist", "Fairy Tale", "Violet Evergrden", "Soul Eater", "The Devil is a Part-Timer!", "Black Clover", "Overlord", "The Rising of the Shield Hero", "Is it Wrong to Try to Pick Up Girls in a Dungeon", "fate/Zero", "Noragami Aragoto", "Demond Slayer", "That Time I Got Reincarnated as a Slime", "log Horizon", "Re:Zero", "Goblin Slayer", "Tenki no Ko", "Tower of God", "The God of High School", "GATE", "Fruits Basket", "The Asterisk War"]
var result = Math.floor(Math.random() * fantasy.length);
console.log(fantasy);
} else if (x == "horror") {
var horror = ["Tokyo Ghoul", "Parasyte", "Another", "Elfen Lied", "Highschool of the Dead", "Deadman Wonderland", "Devilman: Crybaby", "Hellsling Ultimate", "Akira", "When They Cry", "Kabaneri of the Iron Fortress", "From the New World", "Beserk", "Shiki", "Ajin", "Psprika", "Dorohedoro", "Mieruko-chan", "Blood+", "Corpse Party: Tortured Souls", "Hell Girl", "Gantz", "Danganronpa", "Mononoke", "High-Rise Invasion", "Happy Sugar Life", "Ghost Hunt", "The Los Village", "Pupa", "Ghost Stories", "Lunar Ledgen Tsukihime", "Junji Ito Collection", "Zetman", "Memories"]
var result = Math.floor(Math.random() * horror.length);
console.log(horror);
} else if (x == "mystery") {
var mystery = ["Earsed", "The Promised Neverland", "Another", "Kakegutui", "Durarara", "Hyouka", "Made in Abyss", "Bungo Stray Dog", "Black Butler", "Monster", "Darker then Black", "Black Bullet", "Tower of God", "When They Cry", "K", "From the New World", "Serial Experiments Lain", "Great Pretender", "Vanpire Knight", "Eden of The East", "No.6", "Summer Time Rendering", "Wolf's Rain", "Amnesia", "High-Rise Invation", "Astra Lost in Space"]
var result = Math.floor(Math.random() * mystery.length);
console.log(mystery);
} else if (x == "psychological") {
var psychological = ["Deaht Note", "Tokyo Ghoul", "Re:Zero", "Erased", "The Future Diary", "The Promised Neverland", "Psycho-Pass", "Terror in Resonance", "Classroom of the Elite", "Btooom!", "FLCL", "Perfect Blue", "Ergo Proxy", "The Fruit of Grisaia", "Shiki", "Blast of Tempest", "Paranoia Agent", "Cyberpunk: Edgerunners", "Moriarty the Patriot", "Platinum Emd", "B: The Beginning", "Invaded", "When Marnie Was There", "Tomodachi Game", "Haibane Renmei", "Death Billards", "Flowers of Evil"];
var result = Math.floor(Math.random() * psychological.length);
console.log(psychological);
} else {
var boysLove = ["Given", "No.6", "Love Stage!", "Beyond the Boundary: I'll Be Here - Future", "Junjou Romantica", "Doukyusei -Classmates-", "Hitorijimd My Hero", "Sasaki and Miyano", "Dakaretai Otoko 1-i ni Odosarete Imasu.", "Super Lovers", "Yarchin Bitch-bu", "Gakuen Heaven", "kirepapa", "Yes, No, or Maybe?", "Ten Count", "Fake"]
var result = Math.floor(Math.random() * boysLove.length);
console.log(boysLove);
}
}
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<!--Language encoding-->
<meta name="viewport" content="width=device-width, initial-scale=1.0>" <title> Final Project
</title>
<link rel="stylesheet" type="text/css" href="FPCSS.css" />
</head>
<body class="style">
<div id="Top">
<p class="title"> Anime for All
<IMG class="IMG1" SRC="five.gif"> </p>
<p>
<br>
</p>
<div class="Directory">
<!-- Website navigation-->
<h1>Directory</h1>
<p> Homepage</p>
<p> Anime History</p>
<p> Misconceptions and Genres</p>
<p> Recommendations</p>
<p> Random Anime Generator</p>
<p>Email me!</p>
</div>
<div class="MS">
<!--MS= format for main section-->
<h1>Random Anime Generator</h1>
<label> Select your Favorite genres to generate a anime!</label>
<select id="selectedId">
<option>Choose a Genre</option>
<option value="action"> Action</option>
<option value="adventure">Adventure</option>
<option value="comedy">Comedy</option>
<option value="drama">Drama</option>
<option value="sliceOfLife">Slice Of Life</option>
<option value="fantsy">Fantsy</option>
<option value="horror">Horror</option>
<option value="mystery">Mystery</option>
<option value="psychological">Psychological</option>
</select>
<p> Your random anime based on chosen genre is....</p>
<p id="result"> </p>
<script src="Final_Project_JS.js"></script>
</div>
</div>

You could store all full list of anime titles in a variable—preferably an object of arrays. Whenever user selects a genre, you could simply generate a random integer per each sublist's length and show the result in the target p element.
See the snippet below for working example. I didn't modify the original HTML markup too much, except removing irrelevant elements to the question.
For this to work as intended, you need to make sure the list of option values match the keys of titles object.
const titles = {
action: ["Attack on Titan", "Fullmetal Alchimist", "One Punch Man", "Sword Art Online", "My Hero Academia", "Demond Slayer", "Naruto", "Tokyo Ghoul", "Hunter x Hunter", "Code Geass", "Jujutsu Kaisen", "One Piece", "Noragami", "Akame ga Kill!", "Mob Psycho 100", "Assassination Classroom", "Blue Exorcist", "Bleach", "Parastyle", "Cowboy Bebop", "Fairy Tail", "Soul Eater", "Jojo's Bizarre Adventure", "Black Clover", "Fullmetal Alchemist", "That Time I got Reincarnated as a Slime", "Fire Force", "Balck butler", "Spy x Family", "The Seven Deadly Sins", "Tokyo Revengers", "Devilman: Crybaby", "Chainsaw Man"],
comedy: ["One Punch Man", "No Game NO Life", "Mob Psycho 100", "Assassination Classroom", "KonoSuba: God's Blessing on This Wonderful World!", "Kill la Kill", "Rascal Does Not Dream of Bunny Girl Senpai", "Dr.Stone", "Kaguya-sama: Love is War", "Soul Eater", "Gurren Lagann", "The Devil is a Part-Timer", "Black Clover", "High School DxD", "That Time I Got Reincarnated as a Slime", "Maid Sama!", "Spy x Family", "Black Butler", "Nisekoi", "Ouran High School Host Club", "My Little Monster", "JoJo's Bizarre Adventure:", "K-On!", "The Disastrous Life of Saiki K.", "My Teen Romantic Comedy SNAFU TOO!", "Golden Time", "Monthly Girls' Nozaki-kun", "Rent-a-Girlfriend", "The Quintessential Quintuplets", "Don't Toy with Me, Miss Nagatoro", "How Not to Summon a Demon Lord", "Ghost Stories"],
drama: ["Attack on Titan", "Fullmetal Alchemist", "Your Name.", "Steins:Gate", "A Silent Voice", "Code Geass", "Toradora!", "Your Lie in Apirl", "Re:Zero", "Angel Beats!", "Death Parade", "Violet Evergarden", "Rascal Does Not Dream of Bunny Girl Senpai", "Anohana: The Flower We Saw That Day", "Clannad", "Charlotte", "Kakegurui", "Made in Abyss", "The Pet Girl of Sakurasou", "Guilty Crown", "Jojo's Bizarren Adventure", "Classroom of the Elite", "Tokyo Revengers", "ReLIFE", "Monster", "5 Centimeters Per Second", "Plasic Memories", "Tower of God", "The Garden of Words", "Banana Fish", "Orange", "To Your Eternity", "Fruits Basket", "91 Days"],
sliceOfLife: ["Violet Evergarden", "Anohana: The Flower We Saw That Day", "Hyouka", "Miss Kobayashi's Dragon Maid", "ReLIFE", "5 Centimeters Per Second", "The Melancholy of Haruhi Suzumiya", "Blue Spring Ride", "Kimi ni Todoke: From Me to You", "I Want To Eat Your Pancreas", "My Dress-Up Darling", "The Garden of Words", "Beastars", "Mushu-Shi", "Wolf Children", "Nana", "Barakamon", "Natsume's Book of Friends", "Bunny Drop", "Laid-Back Camp", "Love-Live!", "Tanaka-Kun is Always Listless", "Kimi ni Todoke", "School-Live!", "Tamako Market", "Beck", "Place to Place", "One Week Friends", "Girls' Last Tour", "Remake Our Life!", "Silver Spoon", "Flying Witch", "Doukyusei-Classmates", "Colorful"],
fantasy: ["Sword Art Online", "Demon Slayer", "Naruto", "Tokyo Ghoul", "Hunter x Hunter", "No Game No Life", "Jujutsu Kaisen", "One Piece", "Noragami", "Re:Zero", "Akame ga Kill!", "The Seven Deadly Sins", "Bleach", "Blue Exorcist", "Fairy Tale", "Violet Evergrden", "Soul Eater", "The Devil is a Part-Timer!", "Black Clover", "Overlord", "The Rising of the Shield Hero", "Is it Wrong to Try to Pick Up Girls in a Dungeon", "fate/Zero", "Noragami Aragoto", "Demond Slayer", "That Time I Got Reincarnated as a Slime", "log Horizon", "Re:Zero", "Goblin Slayer", "Tenki no Ko", "Tower of God", "The God of High School", "GATE", "Fruits Basket", "The Asterisk War"],
horror: ["Tokyo Ghoul", "Parasyte", "Another", "Elfen Lied", "Highschool of the Dead", "Deadman Wonderland", "Devilman: Crybaby", "Hellsling Ultimate", "Akira", "When They Cry", "Kabaneri of the Iron Fortress", "From the New World", "Beserk", "Shiki", "Ajin", "Psprika", "Dorohedoro", "Mieruko-chan", "Blood+", "Corpse Party: Tortured Souls", "Hell Girl", "Gantz", "Danganronpa", "Mononoke", "High-Rise Invasion", "Happy Sugar Life", "Ghost Hunt", "The Los Village", "Pupa", "Ghost Stories", "Lunar Ledgen Tsukihime", "Junji Ito Collection", "Zetman", "Memories"],
mystery: ["Earsed", "The Promised Neverland", "Another", "Kakegutui", "Durarara", "Hyouka", "Made in Abyss", "Bungo Stray Dog", "Black Butler", "Monster", "Darker then Black", "Black Bullet", "Tower of God", "When They Cry", "K", "From the New World", "Serial Experiments Lain", "Great Pretender", "Vanpire Knight", "Eden of The East", "No.6", "Summer Time Rendering", "Wolf's Rain", "Amnesia", "High-Rise Invation", "Astra Lost in Space"],
psychological: ["Deaht Note", "Tokyo Ghoul", "Re:Zero", "Erased", "The Future Diary", "The Promised Neverland", "Psycho-Pass", "Terror in Resonance", "Classroom of the Elite", "Btooom!", "FLCL", "Perfect Blue", "Ergo Proxy", "The Fruit of Grisaia", "Shiki", "Blast of Tempest", "Paranoia Agent", "Cyberpunk: Edgerunners", "Moriarty the Patriot", "Platinum Emd", "B: The Beginning", "Invaded", "When Marnie Was There", "Tomodachi Game", "Haibane Renmei", "Death Billards", "Flowers of Evil"],
boysLove: ["Given", "No.6", "Love Stage!", "Beyond the Boundary: I'll Be Here - Future", "Junjou Romantica", "Doukyusei -Classmates-", "Hitorijimd My Hero", "Sasaki and Miyano", "Dakaretai Otoko 1-i ni Odosarete Imasu.", "Super Lovers", "Yarchin Bitch-bu", "Gakuen Heaven", "kirepapa", "Yes, No, or Maybe?", "Ten Count", "Fake"]
}
const selectionHandler = e => {
const genre = e.target.value;
const targetList = titles[genre];
if (!targetList) {
result.innerText = "No anime found";
} else {
const randomIndex = Math.floor(Math.random() * targetList.length);
result.innerText = targetList[randomIndex];
}
}
const input = document.querySelector('#selectedId');
const result = document.querySelector('#result');
input.addEventListener('change', selectionHandler);
#result {
padding: 10px;
background: #eee;
display: inline-block;
}
<div id="Top">
<p class="title"> Anime for All</p>
<div class="MS">
<!--MS= format for main section-->
<h1>Random Anime Generator</h1>
<label> Select your Favorite genres to generate a anime!</label>
<select id="selectedId">
<option>Choose a Genre</option>
<option value="action"> Action</option>
<option value="adventure">Adventure</option>
<option value="comedy">Comedy</option>
<option value="drama">Drama</option>
<option value="sliceOfLife">Slice Of Life</option>
<option value="fantasy">Fantsy</option>
<option value="horror">Horror</option>
<option value="mystery">Mystery</option>
<option value="psychological">Psychological</option>
</select>
<p> Your random anime based on chosen genre is....</p>
<p id="result"></p>
</div>
</div>

Extracting a var from <script> tag in html

I am trying to web scrape product reviews from a page but I'm not sure how to extract a var inside the <script> tags.
Here's my python code:
import requests
from bs4 import BeautifulSoup
import csv
a_file = open("ProductReviews.csv", "a")
writer = csv.writer(a_file)
# Write the titles of the columns to the CSV file
writer.writerow(["created_at", "reviewer_name", "rating", "content", "source"])
url = 'https://www.lazada.com.my/products/iron-gym-total-upper-body-workout-bar-i467342383.html'
# Connect to the URL
response = requests.get(url)
# Parse HTML and save to BeautifulSoup object
soup = BeautifulSoup(response.content, "html.parser")
data = soup.findAll('script')[123]
if 'var __moduleData__' in data.string:
print("Yes")
Here's the page source (I removed the unnecessary code):
<html>
<head>
<title></title>
</head>
<body>
<script>
var __moduleData__ = {
"data": {
"root": {
"fields": {
"review": {
"reviews": [{
"rating": 5,
"reviewContent": "tq barang dah sampai",
"reviewTime": "24 May 2021",
"reviewer": "Jaharinbaharin",
}, {
"rating": 5,
"reviewContent": "Beautiful quality👌👌👌",
"reviewTime": "08 Sep 2021",
"reviewer": "M***.",
}, {
"rating": 5,
"reviewContent": "the box was badly dented but the item was intact...just that my door frame is shallow and slippery....I can't pull up without worrying of falling down",
"reviewTime": "25 Aug 2021",
"reviewer": "David S.",
}, {
"rating": 5,
"reviewContent": "Haven’t really opened it yet but please put some effort on the packaging for future improvement thanks it was really fast",
"reviewTime": "14 Dec 2020",
"reviewer": "Yasir A.",
}, {
"rating": 5,
"reviewContent": "Seems to be ok, good quality.. No weight restriction mentioned on the box.. I'm about 90kg, it could handle my weight so far..",
"reviewTime": "22 May 2020",
"reviewer": "Kevin",
}]
},
}
},
},
};
</script>
</body>
</html>
I just want to get the review data only so I'd like to know how to extract the value of var __moduleData__.

You can use a regex to select your variable:
json.loads(re.search(r'var __moduleData__ = ({.*})', response.text).group(1))
Example
from bs4 import BeautifulSoup
import json,re,requests
url = 'https://www.lazada.com.my/products/iron-gym-total-upper-body-workout-bar-i467342383.html'
response = requests.get(url)
d = json.loads(re.search(r'var __moduleData__ = ({.*})', response.text).group(1))
d['data']['root']['fields']['seller']
Output
{'chatResponsiveRate': {'labelText': 'Chat Response', 'value': '100%'},
'chatUrl': 'https://pages.lazada.com.my/wow/i/my/im/chat?brandId=21411',
'hideAllMetrics': False,
'imEnable': True,
'imUserId': '100285367',
'name': 'MR SIX PACK',
'newSeller': False,
'percentRate': '96%',
'positiveSellerRating': {'labelText': 'Seller Ratings', 'value': '96%'},
'rate': 0.96,
'rateLevel': 3,
'sellerId': '1000052649',
'shipOnTime': {'labelText': 'Ship On Time', 'value': '97%'},
'shopId': 255007,
'size': 5,
'time': 2,
'type': '4',
'unit': 'years',
'url': '//www.lazada.com.my/shop/mr-six-pack/?itemId=467342383&channelSource=pdp'}

Object.values filter on 'keypress'

As I'm using this technique to get dummy data from SWAPI.
const httpStarwars$ = createHttpObservables('https://swapi.dev/api/films/');
fromEvent<any>(this.sTxt.nativeElement,'keypress').pipe(
map(event => event.target.value),
debounceTime(400),
distinctUntilChanged(),
switchMap(val => httpStarwars$.pipe(
map(res => Object.values(res["results"]))
))
).subscribe(courses => console.log(courses));
Here is the expected result after key press.
0: {title: "A New Hope", episode_id: 4, opening_crawl: "It is a period of civil war.
↵Rebel spaceships, st…er
↵people and restore
↵freedom to the galaxy....", director: "George Lucas", producer: "Gary Kurtz, Rick McCallum", …}
1: {title: "The Empire Strikes Back", episode_id: 5, opening_crawl: "It is a dark time for the
↵Rebellion. Although the… remote probes into
↵the far reaches of space....", director: "Irvin Kershner", producer: "Gary Kurtz, Rick McCallum", …}
2: {title: "Return of the Jedi", episode_id: 6, opening_crawl: "Luke Skywalker has returned to
↵his home planet of…
↵struggling to restore freedom
↵to the galaxy...", director: "Richard Marquand", producer: "Howard G. Kazanjian, George Lucas, Rick McCallum", …}
3: {title: "The Phantom Menace", episode_id: 1, opening_crawl: "Turmoil has engulfed the
↵Galactic Republic. The t…ustice in the
↵galaxy, to settle the conflict....", director: "George Lucas", producer: "Rick McCallum", …}
4: {title: "Attack of the Clones", episode_id: 2, opening_crawl: "There is unrest in the Galactic
↵Senate. Several t…THE REPUBLIC
↵to assist the overwhelmed
↵Jedi....", director: "George Lucas", producer: "Rick McCallum", …}
5: {title: "Revenge of the Sith", episode_id: 3, opening_crawl: "War! The Republic is crumbling
↵under attacks by t…ate mission to rescue the
↵captive Chancel
Now I want to filter the result of the above data below using filters.
map(res => Object.values(res["results"]))
What I will filter is title. So far what I used is this technique below.
map(res => Object.values(res["results"]).filter(j => j["title"] == res))
It gave me empty array like this [].
UPDATE 1:
Update 2:

Update 1
I edit my answer to use Typescript.
I think that what you want to do is just filter your response by title, so you can just do that:
interface Movie {
title: string;
episode_id: number;
}
const res = {};
res["results"] = {
0: {title: "A New Hope", episode_id: 4},
1: {title: "The Empire Strikes Back", episode_id: 5 },
2: {title: "Return of the Jedi", episode_id: 6},
3: {title: "The Phantom Menace", episode_id: 1},
4: {title: "Attack of the Clones", episode_id: 2}
}
const result = Object.values(res["results"]).filter((movie: Movie) => movie.title === "A New Hope")
console.log(result)
Just remember to verify before if res["results"] contains the object with the movies.

javascript error SyntaxError: missing : after property id

Trying to make an array of strings inside array of objects and getting an error saying SyntaxError: missing : after property id for the code below in javascript
let movies = [
{
movie title: 'Avengers: Age of Ultron',
release year: '2015',
rating:'7.3',
"genres"=[0:"Action",1:"Adventure",2:"Sci-Fi"]
},
{
movie title: 'The Dark Knight',
release year:'2008',
rating:'9.0',
genres:["Action","Crime","Drama"]
},
{
movie title: 'Forrest Gump',
release year:'1994',
rating:'8.8',
genres:["Drama","Romance"]
},
{
movie title: 'Inception',
release year:'2010',
rating:'8.8',
genres:["Action","Adventure","Sci-Fi"]
},
{
movie title: 'The Matrix',
release year:'1999',
rating:'8.7',
genres:["Action","Sci-Fi"]
},
{
movie title: 'Border',
release year:'1997',
rating:'7.9',
genres:["Action","Drama","History"]
}
];

There are two mistakes in your code.
If JSON object key contains ' ', you should write ".
You don't need to write an index to your JSON object value.
Here's the updated code.
let movies = [
{
"movie title": 'Avengers: Age of Ultron',
"release year": '2015',
rating:'7.3',
genres:["Action", "Adventure", "Sci-Fi"]
},
{
"movie title": 'The Dark Knight',
"release year":'2008',
rating:'9.0',
genres:["Action","Crime","Drama"]
},
{
"movie title": 'Forrest Gump',
"release year":'1994',
rating:'8.8',
genres:["Drama","Romance"]
},
{
"movie title": 'Inception',
"release year":'2010',
rating:'8.8',
genres:["Action","Adventure","Sci-Fi"]
},
{
"movie title": 'The Matrix',
"release year":'1999',
rating:'8.7',
genres:["Action","Sci-Fi"]
},
{
"movie title": 'Border',
"release year":'1997',
rating:'7.9',
genres:["Action","Drama","History"]
}
];

Sort an array of objects(with multiple properties) in a specified order of Keys with JavaScript [duplicate]

This question already has answers here:
Sort JavaScript object by key
(37 answers)
Closed 2 years ago.
I have a json response with the following data in alphabetical order,
let sheetCells =
{
0: [
{
"Actual Hours": 16
"Assigned": "Jerry"
"Completion %": 48.2
"Days Scheduled": 2
"Device 1": 10
"Device 2": 43
"Device 3": 72
"Device 4": 91
"Device 5": 25
"End Date": "2018-01-03T23:00:00.000Z"
"Estimated Hours": 16
"ID": "1dbk3"
"Notes": "Note 1"
"Parent ID": "2k59f"
"Priority": ""
"Start Date": "2018-01-01T23:00:00.000Z"
"Stories": "A User"
"Tests": "Y"
},
{
//...same keys as above, different values
}
]
}
but this is the DESIRED format in which I want the resulting objects in the array be sorted,
[
{
"Stories": "A User"
"ID": "1dbk3"
"Parent ID": "2k59f"
"Completion %": 48.2
"Priority": ""
"Device 1": 10
"Device 2": 43
"Device 3": 72
"Device 4": 91
"Device 5": 25
"Assigned": "Jerry"
"Notes": "Note 1"
"Tests": "Y"
"Estimated Hours": 16
"Days Scheduled": 2
"Start Date": "2018-01-01T23:00:00.000Z"
"End Date": "2018-01-03T23:00:00.000Z"
"Actual Hours": 16
},
{
//...same keys as above, different values
}
]
Here is the solution that i've tried
let sortOrder = ['Stories', 'ID', 'Parent ID', 'Completion %', 'Priority', 'Device 1', 'Device 2', 'Device 3', 'Device 4', 'Device 5', 'Assigned', 'Notes', 'Tests', 'Estimated Hours', 'Days Scheduled', 'Start Date', 'End Date', 'Actual Hours']
let result = sheetCells[0].sort((a,b) => sortOrder.indexOf(a.Stories) > sortOrder.indexOf(b.Stories)))
but this result gets just the story key and ignores the rest, please help with a working and optimal solution.Thanks

Rather than resorting the original data, you can map it to a new sorted array by indexing it with every element in your sortOrder in order. Something like this:
let result = sortOrder.map(key => sheetCells[0][key]);
While that's a working version of your solution, based on the data structure in your example, this may be closer to what you're looking for:
let result = sheetCells[0].map(cell => sortOrder.map(key => cell[key]));

Develop Reference

JavaScript is the programming language of the Web.

Cant parse a dictionary retrieved from xmlhttprequest.responseText [SOLVED] - javascript

Related

How can I generate a random selection based on user's <select> option choice?

Extracting a var from <script> tag in html

Object.values filter on 'keypress'

javascript error SyntaxError: missing : after property id

Sort an array of objects(with multiple properties) in a specified order of Keys with JavaScript [duplicate]

Categories

Resources