I'm running an external website that harvests data from an iframe on our internal company intranet. This part is written in Javascript and is working. However, now I need access to the data the js is harvesting to process it further in python. It is not an option to process in Javascript as well as that is technically not possible.
I would appreciate some pointers on where to go from here? How do I get the javascript data into my python scripts?
You will need to use an AJAX call to pass the javascript data to your server on the backend for python to process, The best way to do this is to encode your data in a JSON Object and pass it in an AJAX call:
json_obj = {your:[data]};
$.ajax{
url: [url of webservice/action],
type: "POST",
data: json_obj,
dataType: "json",
success: function(result) {
//Write your code here
}
}
//Note that this does run asynchronously, so the code that follows this will not wait for the ajax call to finish
Python has a nifty module, json, that you can use to decode JSON objects into Python Objects using:
import json
data_obj = json.loads(json_data_string)
This will put the json string you assembled in your javascript then passed to python with AJAX into a Python structure (dictionary or list, depending on how your object was built) for easy python processing.
You can then pass a JSON string back by using the dumps() function:
data_str = json.dumps(data_obj)
Sources:
Python JSON module documentation
AJAX Examples
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I'm currently working on a school project and I need some help. I ran into a problem when trying to put the data I read from another IP address (using AJAX) into a MySQL database. I tried using node.js but because I'm calling the function when I press the button on a webpage it doesn't work... So any suggestions or tips on how to make node.js work.
ajax function:
function AddShoots() {
$.ajax({
method: "POST",
dataType: "json",
url: "http://192.168.1.8",
success: function (html) {
for ($i = 0; $i < html.length; $i++) {
console.log(html[$i]);
}
},
});
}
The data I get and would like to insert into database:
EDIT: additional explanation
sorry for the HTML/CSS there was an error while copying. To clarify my endpoint I have a python image recognition program written on raspberry PI, the Raspberry PI server that is on the IP returns the numbers seen in the picture, I would like to insert the numbers into a MySQL database and I need help with how to do that because I only inserted data into tables with PHP before, but I can't do this in this case because I'm using AJAX (at least not with my knowledge) I hope this explains everything better.
Another edit:
A picture that might help with understanding (I'm super bad at explaining sorry English is not my primary language that's why I have bad expressions and a hard time explaining some stuff).
Sounds like you're getting your technologies mixed up a bit.
Lets clarify a few terms first:
NodeJS
Is a JavaScript runtime for server-side or backend code.
Runtime
An environment for code to execute like NodeJS or a web browser.
Server-Side
This refers to code running on a server. Could be PHP, Java, Python etc... or even JavaScript if you're using Node.
Backend
Usually means the same thing as "server-side" when people say "backend".
Frontend
In this context, "frontend" is referring to code being executed in a web browser.
AJAX
A style of HTTP request that a browser can use to send and fetch data without reloading the current page.
Now that we have that out of the way...
Node is a server-side runtime for JavaScript, so your JS code running in the browser has no way to talk to your Node code directly.
If you want to click a button in the browser and see data get written to your database you have to make an AJAX call to a url that your backend is listening to (known as an endpoint or a route).
It's a bit difficult to interpret your intentions from your example but I think you mean to send a POST to http://192.168.1.8. So you'll need a route configured in your Node app that can handle the AJAX request, and from there you can write the data to your database.
Of course you'll also need to pass the data with the request. That gets passed along with the options parameter in your $.ajax(/* ... */) call.
At a high level this is what I believe you're trying to achieve:
So basically your app should have at least two files:
index.html This will have your button as well as the JS code that fetches data from your Pi. This "front-end" JS will also have to send an AJAX request to your Node app.
app.js This will be your Node app. It will have to expose an endpoint that your front-end code can send data to. Inside the function of that endpoint, you'll handle writing the data to your database.
There's a library called Express that will help you make your endpoint. I recommend following their Getting Started guide: https://expressjs.com/en/starter/installing.html
Update
I see you updated your answer with a diagram. I assumed your mental model was probably exactly like that, which is why you're having a hard time.
When a user visits a url such as https://stackoverflow.com for example, the request doesn't go to the web browser. The request has to go through a server first. When you use Node, your app essentially is the server, and is responsible for returning the correct response.
In your case the response would be an HTML file like index.html for example.
That index.html is the webpage and it is going to contain your "front-end" JavaScript code, which can communicate to your Raspberry Pi server over HTTP to get that image data you're talking about.
JavaScript running in the browser has no way to communicate to a SQL database directly. So you have to send a request over the network to your Node service. Inside your backend application you can write the data to a MySQL database with the help of packages you can get from NPM. Here's an example of one: https://www.npmjs.com/package/mysql
Update 2
It sounds like you're more comfortable using PHP. You can swap NodeJS out with PHP and my diagram would still represent what you want to achieve at a high level. The key difference is you won't have to figure out how to set up a "route".
With PHP you can just have a file named something like SaveMyData.php:
<?php
// Get the body of the POST request
$data = file_get_contents('php://input');
// Decode the JSON string so you can work with it in PHP
$decodedData = json_decode($data);
// $decodedData is now an array of the data you sent from the browser
foreach($decodedData as $row) {
// write the row to your database here
}
Then the "endpoint" (making some assumptions about how your serving your app) is just http://localhost:8080/SaveMyData.php and that becomes the URL you pass to your AJAX call from the browser.
Your AJAX request would look something like:
function AddShoots() {
$.ajax({
method: "POST", // the post to your Pi I'm assuming?
dataType: "json",
url: "http://192.168.1.8",
success: function (data) {
$.ajax({
method: "POST", // this is going out to your PHP backend
dataType: "json",
url: "http://localhost:8080/SaveMyData.php",
data: data,
success: function (response) {
// do stuff with the response if you'd like
})
});
},
});
}
Tons of "best practices" are being violated here, but for the purposes of a school assignment this should get you pointed in the right direction.
I have seen at several places developers are using JSON.stringify(data) while making an Ajax call to the server for the serialization of post data in JSON string, but why it is needed ?
$.ajax({
url: url,
type: "POST",
data: JSON.stringify(data),
contentType: "application/json",
complete: callback
});
You have to encode the data using some method in order to send it over HTTP.
JSON is a standard format that supports common data structures like arrays. This lets you describe most kinds of data that you would want to send.
Several modern frameworks are capable to directly bind JSON data structures to their model businesses, this allows for a incredibly fast and easy relationship between client and server data models.
This way you can work feely on your client side with js objects, on the moment that you send data to the server via AJAX, you just stringify these objects to allow the server end to understand them, and in an automatic way your server will be able to translate that info to your server data classes, without further interaction needed (of course, you will need defined classes that are compatible with your client model data structures).
I've a simple MVC model. I'm doing an Ajax request where I send some data to be processed by PHP and retrieve database records as JSON. As this object could be quite large, is there some way I could compress/encrypt it on the PHP (server side) and decrypt it on the Javascript side (client)
$.ajax({
url: "/php/function/link/",
dataType: 'json',
data: {
"date": date,
},
type: "POST",
success: function(_data){
// load encrypted data here and decrypt it.
},
error: function() {
alert("Some error fetching!");
}
I tried using the following methods, but they didn't seem to work (I was getting error while decompressing them on the javascript end):
JSONC
https://stackoverflow.com/a/11901649/1443702
Are there any other better ways?
I simply need to :
compress data on javascript to be passed from client->send it to server (PHP) -> decompress it and compute database queries -> compress it -> pass it to javascript(client side) -> decompress it
The best way is just to enable HTTP traffic compression in web server. All modern browsers and servers support it. Read more about HTTP_compression. And you will have additional bonus: all your traffic will be compressed, not AJAX traffic only.
For php - You can try this: http://rosettacode.org/wiki/LZW_compression#PHP
For JavaScript - http://rosettacode.org/wiki/LZW_compression#JavaScript
Ideally you should avoid sending large data set unless the use case is unavoidable. I would suggest you to reconsider your design. But recently I ran into similar use-case as part of product requirement where I needed to compress handle 5MB of JSON data (partially) in JavaScript. I tried the above and was able achieve 50% compression.
actually i have webservice running and i want to retrieve the response or output of web service (xml output) and want to show it on some web page. i am trying to give some parameters as input and sending to webservice by AJAX POST and getting some dummy response.. i have problem while sending the parameters with URL. WOULD YOU TELL ME ABOUT THE FORMAT OF AJAX POST PARAMTERS?
var params="text=text1&target=target1";
It returns some error value in response, but with the same data i am able to access with the terminal.
text=text1&target=target1 these paramters are passing as one string not different paramters
I have tried in other way also
var params='text='+text1+'&target='+target1;
but it returns nothing in response
What should I set for params value?
You can't send POST data via url query string.
You will need to either use cURL from the command line, or use a POST tool like POSTMAN to pass form data and the appropriate POST headers.
Edit: your tags suggest you're using javascript/ajax.
You can do this natively with javascript, but you'd have a significantly easier time using jQuery's $.ajax or $.post:
$.ajax({
type: "POST",
url: "http://myapp.com/someendpoint",
data: {
text: "text1",
target: "target1"
}
});
I want a code for how to parse xml data , which is comming from restfull webservices.
Please send code for how to call restfull webservices URL from javascript and this URL contains XML data . I want to read this xml data from javascript.
Thank U.
You can't grab data from another server using JavaScript, it's a security issue.
You can however create a server-side script which returns the XML required. Once you've done that, check out this very helpful article that walks through step by step of using jQuery to parse the XML
http://think2loud.com/224-reading-xml-with-jquery/
You can pass/get xml type through Jquery Ajax call. Two important things to be considered while passing the data as xml
Specifying the datatype as "xml"
Specifying the contentType as "text/xml; charset=\"utf-8\""
You can check out this article for calling your web service through java script and manipulating the data.
http://sharepoint-snippets.com/ajax-calls-sharepoint-web-services-using-jquery/
you can also check for various parameters used in Ajax call
http://api.jquery.com/jQuery.ajax/