I currently use a local Tomcat/JSP service/webpage to get data with sql and display it on the webbrowser with plotly (locally rendered with this javascript library). I want to do more data wrangling by the user like filtering and cannot do it with sql since the sql query takes several seconds and several sql queries for one webpage would take too much time. Within JSP data filtering is quite complicated since you have to write you own filter functions like https://blog.pavelsklenar.com/using-filteredrowset-simple-example/ or http://www.java2s.com/Code/Java/Database-SQL-JDBC/FilteredRowSetDemo.htm . If you know Pythons Pandas or sql this seems too complicated.
I wonder if Zeppelin is a good solution with these steps:
Get the data directly via sql (already works) but the integrated display possibilities are nice but not sufficient for me.
A user can adjust several filters (e.g. column1 between 1 and 3 and so) using the AngularJS view (https://zeppelin.apache.org/docs/0.7.3/displaysystem/front-end-angular.html).
Read the filter ranges from the last paragraph and do data wrangling with python’s pandas with some filters and save it a new dataframe.
Use this filtered dataframe and plot the data with a local plotly.js file (I have the pure plotly JS code). In general, it could work with an %angular paragraph as Using d3.js with Apache Zeppelin (even without the code from “else”, so d3 seem to be useable within zeppelin out of the box). But I wonder
how to pass the dataframe from Python to the Angular node. According to https://zeppelin.apache.org/docs/0.7.3/displaysystem/back-end-angular.html I have to use Spark to use the variables; so it seems that I have to integrate a spark wrapping paragraph. After that do I have to write code that converts the dataframe to JSON or similar?
how to use plotly or other local javascript files? How can I specify the location?
What do you think or do you have other/better ideas?
I solved the problem by using a non interavtive (that means no javascript library like plotly or d3) by using pythons matplotlib for the plotting task.
Related
I want to do a linear regression in Data Studio. I don't know if there is something already existing (I've looked ans it doesn't seem like there is).
What I want is not plotting a linear Regression in a graph but to obtain the Linear regression's coefficients to then make predictions on some user-defined input parameters.
I more or less know how to code the linear regression function in JS. Is there a way to create a field which would take some parameters as an Input and a returns the output JS custom process (involves looping)? I'm not particularly attached to the whole JS process solution if someone has an answer that solves my problem I'm open to it.
You can create a community viz in Data Studio to do this.
Easiest way to do it would be to first assume:
you have a blank HTML page.
your dependent and independent variables are in JS arrays.
you are creating a single page JS application independent of Data Studio.
Starting from there,
write the entire code in JS to calculate the parameters
surface the parameters using JS created DOM elements
use JS to create input elements to take user input of dependent variables
use user input and parameters to make prediction and surface that
So essentially you will end you with a single page JS application. Once this is ready, you can wrap this in the Community Viz framework, link the arrays to the data form Data Studio, and then use the viz in your reports.
I have a bokeh server application that displays data in a graph.
I want to be able to get a value from JavaScript and pass it to the bokeh server script and run a function with that new information that will manipulate the graph.
For example;
I have a series of buttons in JavaScript, when the button is pushed, I want to change the data from 'The number of Icecreams sold.' to 'the number of pizzas' by manipulating a variable.
I know that we can achieve this using bokeh's built-in buttons, but I'm really looking for a way for a way to achieve that from an 'on click' from the java side.
From my research, it seems there are plenty of ways to get python to do something to java, but not the other way around.
Is there any advice out there that might set me down the path?
You can define your own Javascript callbacks in the Python code. See the bokeh docs describing this here.
I am trying to build an application that takes data from an excel sheet and transforms it into a graph. The library I intend to use is chartist.js.
If the excel sheet is manipulated that should recreate a graph without refreshing the page. I want to ask the following:
Is this a good tool for the intended purpose? if not, what are good alternatives?
How would I take the data directly from excel using javascript and use it to feed the chart? Can it be done only through the use of JSON or other technologies such as angular.js? if the data changes every minute or so, wouldn't that be very expensive to do?
If it's possible in anyway, could you please provide an example for me using any chart shown in the documentation?
Thanks in advance.
There is a useful tool that exists that is pretty similar to what you want to build. It's an Excel add-in called Funfun, that also hosts an online editor with an embedded spreadsheet.
Here is a chart I made with chartist.js. As you asked, it is a chart shown on the first page of the chartist.js website under RESPONSIVE CHARTS CONFIGURATION:
https://www.funfun.io/1/#/edit/5a69b5ddfa7bd43614770df9
To make the transition from an excel spreadsheet to javascript I indeed use a JSON file, it is the short file under Settings, I select the colunms I want from the spreadsheet like so:
{
"data": "=A1:B7"
}
And I store it in local variable in the script.js file, like so:
var label = [];
var values = [];
for (var i = 1; i < $internal.data.length; i++)
{
label.push($internal.data[i][0]);
values.push($internal.data[i][1]);
}
This is optionnal, you can directly add the data to the chart library but it is cleaner.
You can of course change the value of the selected data and the chart will change instantly.
You can then load your chart in Excel by pasting the URL in the Funfun Excel add-in. Here is how it looks like with this example:
To answer your question: is this a good tool for the intended purpose ?
I think Javascript is a great language for data processing and data visualization, and for people wanting to build dynamic and interactive charts in excel there are powerful libraries with javascript such as plotly.js(for 3D charts) or Highcharts.
And this tool is the bridge between excel and other languages like javascript. This could be an alternative to VBA.
Disclosure : I’m a developer of Funfun.
I am wondering and sadly I don't know where else to ask the question.
I want to make a interactive chart using the top 5 downloaded movies.
And the current box office top 5. How i'm going to make this interactive is beyond me yet.
What I would like to know first is, if there is any way to manipulate or change the given chart from Mojo.
Right now I use:
http://boxofficemojo.com/about/data.htm
With the code given by Mojo:
<script type="text/javascript" language="javascript" src="http://www.boxofficemojo.com/data/js/wknd5.php?h=myclass1&r=myclass2"></script>
This just shows me the top 5 (see example at the previously provided URL).
Is there any way that I can use that chart and make a, pie chart or any other chart or graph out of it?
And if this is possible will it still update with money and new movies every time the site does, like the given chart with the javascript code does right now?
Hopefully someone can help me or maybe has a different way of making a chart/graph out of this data (the box office top 5).
So to be more clear. I would like to create my own chart/graph with the box office top 5 data. And I would also like it to be "live" and update itself when ever the top 5 changes or the numbers change.
There for it doesn't seem like the best idea to create my own Json with data since it won't update when the data updates without me making changes into the Json file.
The interactive part doesn't matter yet.
You want to create a chart with live data.
First thing is getting the data.
The script you are including basically creates the chart for you. So you don't have direct access to the data itself, so it would be a little tricky to use that.
You should google around for a,preferrably REST API, that provides you with data on-demand.
1. Find a suitable REST API and perform calls to fetch data
You perform HTTP requests to the API and it provides data - so on page-load for example you will request data from this API. So each time a user loads your website a call is made and it fetches the most recent data on movies.
2. Parse the data, perform calculations and built your own custom JSON
Then you need to parse that data - Take from this raw feed the data you actually need - perform calculations if needed(from what I take from your question, it's gonna be simple elementary school maths), then build you own custom JSON data structure that can be used to display the data.
3. Visualize the JSON
Include a JS library that renders Charts. Chart.js, is an excellent example and it's dead-simple to provide a JSON and render a really nice looking chart.
That's how it ,usually, goes.
The title phrases it badly so here's a longer description :
I have an application that exports data in html format. ( 500 rows, 20 columns)
It looks terrible with lots of useless columns.
I want to use something like datatables to make a more usable table, i.e. paging/sorting/filtering/hiding columns
The option I'm trying first is to insert the table from the exported html file using the .load() function from jquery. Then I loop through the table deleting/modifying columns.
This seems very slow (I suspect my looping and searching) so I'm looking for improvements.
One idea is to pre-convert my exported html file to json (using notepad++ macros or something like that) and then build the table that I want from that json file.
Any opinions on whether I can expect a large performace boost, or potential problems to look out for ?
Many thanks / Colm
JSON should be faster, when its loaded its ready to go without all of the text parsing you would need to do with a text file. Lots of other jquery addons available to make it easy for you once it is in JSON.
I think this is not about which loads data faster but which solution is better for your problem. Datatables is very flexible and you can load from different sources. Take a look at the "Data Sources" and "Server side processing" in the examples: http://datatables.net/examples/
Datatables uses mostly JSON format. To process your data need to find the best approach; convert your exported html file, process the file with javascript to convert data (jquery can help you here), etc..
This page gives some real world examples of loading data in json vs data in a html table. Fairly conclusive, see the post from sd_zuo on July 2010, a fourfold increase in speed loading from json and then just building the table that you want to display.
Granted the page deals specifically with the slowness of the innerHtml function in IE8 but I think I'll give it a go in json and see how it compares across a couple of browsers.
P.S. This page gives good advice on fast creation of html using raw javascript and then only using jquery to insert one full row at a time