Develop Reference

Extract html sourcecode from a javascript generated output

I am currently working on a project of finding empty classrooms in our school in real time. For that purpose, I need to extract substitution published on our school page (https://ssnovohradska.edupage.org/substitution/?), since there might be any additional changes.
But when I try to extract the html source code and parse it with bs4, it cannot find the divs(class: "section print-nobreak") that contain the substitution text. When I took a look at the page source code(Ctrl+U) I found that there is only a javascript that prints it all directly.
Is there any way to extract the html after the javascript output has been already rendered?
Thanks for help!

Parsing HTML is unfortunately necessary to solve your problem. But I will explain how to find ways to avoid that in your future projects (not based on this website).
You've correctly noticed that the text is created by JavaScript code running on the page. This could also indicate that the data is either loaded from another resource (XHR/fetch call getting a response from an API) or is stored as a JSON/JS inside of the website's code. (Or is generated from an algorithm, but this is unlikely to be the case in such websites.)
The website actually uses both methods (initial render gets data stored inside of the website's code, but when you switch dates on the calendar it makes AJAX requests). You can see this by searching for ReactDOM.render(React.createElement( in the code. They're providing a HTML string to the createElement call, so I would suggest looking into the AJAX way of doing things.
Now, to check where the resource is located, all you need to do is opening Developer Tools in your favorite browser (usually Control+Shift+I) and navigating to the Network tab. Now that your network tab is open, you need to cause the website to load external data, for example, by pressing a date on the "calendar bar".
Here you will notice many external requests, but we're actually looking only for XHR calls. Click on the XHR button next to the "Filter" text field. That should result in only one request being shown:
Unfortunately for us, the response only contains HTML. Also, API calls are protected - they require a PHP session ID and some sort of a token (__gsh) to not fail. So, going back to step 1 - seems like our only solution is to use regular expressions to find the text between "report_html":"<div class and </div></div></div> from the source code, if you're interested in today's date only. If you want to get contents for tomorrow or any other date - you will need to either fetch the page, save the cookies and find the token to supply to the request and then make that request, or use something like puppeteer or pyppeteer (since you've mentioned BS4) and load the webpage in that. If you aren't doing the data fetching that often, you should be fine overall.

Chrome Network Inspector: Is it possible to find the calls for a specific DOM element?

I have a table that gets filled with data received from an API call. Is it possible to somehow "lock" Chrome's network inspector on this element (= the table) and make it only show calls related to it and not every single one made by the rest of the page? Maybe this is doable with an extension?

I would say it's not possible to achieve, because:
The view and the logic are also seperated in (modern) frontend applications. So the app can request a JSON file from a server when it starts, and then passes those informations through the entire application (tables etc.)
A table itself is not calling the endpoint, and can be just generated by a function with the prefilled data.
The closest you can get is:
Record screenshots in Chrome to see when the table gets filled. There you see which calls got made during this period of time:
Afterwar you know what's the "name" of the call, you can put it in the filter/search field and reload the page. Afterwards, you will always just get shown the call you selected and is possibly responsible for filling the table.
Of course, you can also go by hand and say:
What is inside the table? Which information does it display?
Then go through the api calls and look for the information in the response bodies
Once you identified the call, filter it inside the chrome dev tools.
And if you want to go to the next level:
Copy the call als a cUrl (via right-click on the call) and insert it in Postman and also install Postman Inceptor. Afterwards you can pass website calls (the one for your table) to Postman so you can have a closer look at it and also re-call it every time you need the specific call.
To summarise:
Figure out which call has the information for your table, extract the call using Postman Inceptor and pass it to Postman with it. Then, you can analyse or do whatever you want with it.

How to retrieve info from database to display with Chrome extension

I am trying to write my first chrome extension. The workflow goes something like this -When the extension is installed and active if a user hovers over a specific product/ID displayed on the page, the extension retrieves related vendor data about the product with the ID.
This is how I thought about this:
Use jQuery attr to access the ID on mouse over.
Post this ID to a retrieve.php file with .post() method
The retrieve.php file retrieves the data from database
Display the data in a tool tip on the web page.
I have some queries for the above process:
I am able to get this working on a local XAMPP server but how will it work online as the chrome extension will not have access to server. What is the way around to retrieve data without using PHP?
I am able to get the logic working but am unable to place these in respective files - Will all my logic reside in background.js ?
Any suggestions on getting this started will be much appreciated.

You could build a very simple API on your server that responds with JSON to any request it receives after processing it. Like this:
{"firstVar":"foo","secondVar":"bar" }
Your chrome extension can then make an xmlhttp request to this server and and process the returned data.(You could also use JSONP and wrap the response in a callback function which will execute as soon as you have the reponse)
The JS extension will be able to deal with the JSON nicely as it can understand that format so you can then choose to display the data in whatever way you want.
Essentially, what you want is a server that can take an ID posted to it and return the corresponding date in a nice and readable format. And a chrome extension that can make an request to a server and then process the response. Build and test them separately (keep positing an ID to the server and see the response and for your JS side at first instead of making requests to your unfinished API just set a static response to begin with which will be the same as an expected response.

I have been given a cust_key I can use to import and display data from an API and need specific examples

I am barely getting around in HTML5, and trying to do PHP and AJAX and js all at the same time. LOL. I have been given a cust_key and an API that produces an XML output based off the criteria you send in via a url. To integrate this feature to your site you will need to be able to pull information from an XML file. I have an html table built, and the radio buttons, ... now I'd like to GET the info from the remote site (how do I attach my cust_key to results from all the radio buttons?) and display the results on my own site?

Query and process REST API using extjs

I have created a web application and deployed it in my tomcat. The API is a GET request and URL is: http://mymachine.home.net:8080/test.app-1.0/test/json which returns an output as:
{"details":[{"name":"tim","age":"13"},{"name":"jim","age":"15"}]}
I want to write a very simple extjs application that will call the URL and print the response (or write the response to a file). Can someone help me with this? I tried the examples mentioned at many places, but my URL is never hit from the extjs application.
Thanks in advance

This was resolved. A few things that I added / changed were:
I included ext-all.js in the index.html page.
The response returned from the web application was changed as:
someCallback({"summary":[{"hits":9118,"avg":13,"min":1,"max":1448,"errors":0,"date":"This period"},{"hits":1,"avg":1,"min":1,"max":1,"errors":0,"date":"Average"}]});
The callback name (someCallback) is sent as query parameter to the web application. The URI was changed to http://mymachine.home.net:8080/test.app-1.0/test/json?callback=someCallback
The response has to be wrapped around a string like ‘someCallback(…)’, else the Extjs code will not be able to parse it.