I have an application in which we help our clients visualize data via graphs rendered with highcharts. The data processing and visualization is the primary value our application provides, in other words its important and should be tested.
I am using cucumber to write integration tests for the application, I would like to include some basic tests of our highcharts implementation. I am especially keen to test that the correct data is being plotted as well as that custom ajax interactions are working correctly. An example feature might look something like this:
Feature: Plot Related Alarm Data
As an project owner
I want to see antecedent data plotted around the time of the alarm
So that I can diagnose the issue and save gobs of money.
Background:
Given a bunch of stuff is set up
And I am logged in
# ...
Scenario: Alarm data is plotted
Given I have an alarm
# ...
When I visit the alarm show page
Then I should see 5 days of data in my graph
Scenario: Alarm data is clicked
Given # ...
When # ...
And I click on a data point
Then I should be on the data page
Any experience testing things like this and which tool Selenium, Watir, etc was used would be appreciated.
A strategy that I've used:
render the data in an HTML table
generate the chart in JavaScript
using the data from that table (e.g. http://www.highcharts.com/demo/column-parsed)
use Cucumber just to
verify that the table contains the
expected data using table diffs via tableish
Related
I want to do a linear regression in Data Studio. I don't know if there is something already existing (I've looked ans it doesn't seem like there is).
What I want is not plotting a linear Regression in a graph but to obtain the Linear regression's coefficients to then make predictions on some user-defined input parameters.
I more or less know how to code the linear regression function in JS. Is there a way to create a field which would take some parameters as an Input and a returns the output JS custom process (involves looping)? I'm not particularly attached to the whole JS process solution if someone has an answer that solves my problem I'm open to it.
You can create a community viz in Data Studio to do this.
Easiest way to do it would be to first assume:
you have a blank HTML page.
your dependent and independent variables are in JS arrays.
you are creating a single page JS application independent of Data Studio.
Starting from there,
write the entire code in JS to calculate the parameters
surface the parameters using JS created DOM elements
use JS to create input elements to take user input of dependent variables
use user input and parameters to make prediction and surface that
So essentially you will end you with a single page JS application. Once this is ready, you can wrap this in the Community Viz framework, link the arrays to the data form Data Studio, and then use the viz in your reports.
I currently use a local Tomcat/JSP service/webpage to get data with sql and display it on the webbrowser with plotly (locally rendered with this javascript library). I want to do more data wrangling by the user like filtering and cannot do it with sql since the sql query takes several seconds and several sql queries for one webpage would take too much time. Within JSP data filtering is quite complicated since you have to write you own filter functions like https://blog.pavelsklenar.com/using-filteredrowset-simple-example/ or http://www.java2s.com/Code/Java/Database-SQL-JDBC/FilteredRowSetDemo.htm . If you know Pythons Pandas or sql this seems too complicated.
I wonder if Zeppelin is a good solution with these steps:
Get the data directly via sql (already works) but the integrated display possibilities are nice but not sufficient for me.
A user can adjust several filters (e.g. column1 between 1 and 3 and so) using the AngularJS view (https://zeppelin.apache.org/docs/0.7.3/displaysystem/front-end-angular.html).
Read the filter ranges from the last paragraph and do data wrangling with python’s pandas with some filters and save it a new dataframe.
Use this filtered dataframe and plot the data with a local plotly.js file (I have the pure plotly JS code). In general, it could work with an %angular paragraph as Using d3.js with Apache Zeppelin (even without the code from “else”, so d3 seem to be useable within zeppelin out of the box). But I wonder
how to pass the dataframe from Python to the Angular node. According to https://zeppelin.apache.org/docs/0.7.3/displaysystem/back-end-angular.html I have to use Spark to use the variables; so it seems that I have to integrate a spark wrapping paragraph. After that do I have to write code that converts the dataframe to JSON or similar?
how to use plotly or other local javascript files? How can I specify the location?
What do you think or do you have other/better ideas?
I solved the problem by using a non interavtive (that means no javascript library like plotly or d3) by using pythons matplotlib for the plotting task.
I am trying to build an application that takes data from an excel sheet and transforms it into a graph. The library I intend to use is chartist.js.
If the excel sheet is manipulated that should recreate a graph without refreshing the page. I want to ask the following:
Is this a good tool for the intended purpose? if not, what are good alternatives?
How would I take the data directly from excel using javascript and use it to feed the chart? Can it be done only through the use of JSON or other technologies such as angular.js? if the data changes every minute or so, wouldn't that be very expensive to do?
If it's possible in anyway, could you please provide an example for me using any chart shown in the documentation?
Thanks in advance.
There is a useful tool that exists that is pretty similar to what you want to build. It's an Excel add-in called Funfun, that also hosts an online editor with an embedded spreadsheet.
Here is a chart I made with chartist.js. As you asked, it is a chart shown on the first page of the chartist.js website under RESPONSIVE CHARTS CONFIGURATION:
https://www.funfun.io/1/#/edit/5a69b5ddfa7bd43614770df9
To make the transition from an excel spreadsheet to javascript I indeed use a JSON file, it is the short file under Settings, I select the colunms I want from the spreadsheet like so:
{
"data": "=A1:B7"
}
And I store it in local variable in the script.js file, like so:
var label = [];
var values = [];
for (var i = 1; i < $internal.data.length; i++)
{
label.push($internal.data[i][0]);
values.push($internal.data[i][1]);
}
This is optionnal, you can directly add the data to the chart library but it is cleaner.
You can of course change the value of the selected data and the chart will change instantly.
You can then load your chart in Excel by pasting the URL in the Funfun Excel add-in. Here is how it looks like with this example:
To answer your question: is this a good tool for the intended purpose ?
I think Javascript is a great language for data processing and data visualization, and for people wanting to build dynamic and interactive charts in excel there are powerful libraries with javascript such as plotly.js(for 3D charts) or Highcharts.
And this tool is the bridge between excel and other languages like javascript. This could be an alternative to VBA.
Disclosure : I’m a developer of Funfun.
I am wondering and sadly I don't know where else to ask the question.
I want to make a interactive chart using the top 5 downloaded movies.
And the current box office top 5. How i'm going to make this interactive is beyond me yet.
What I would like to know first is, if there is any way to manipulate or change the given chart from Mojo.
Right now I use:
http://boxofficemojo.com/about/data.htm
With the code given by Mojo:
<script type="text/javascript" language="javascript" src="http://www.boxofficemojo.com/data/js/wknd5.php?h=myclass1&r=myclass2"></script>
This just shows me the top 5 (see example at the previously provided URL).
Is there any way that I can use that chart and make a, pie chart or any other chart or graph out of it?
And if this is possible will it still update with money and new movies every time the site does, like the given chart with the javascript code does right now?
Hopefully someone can help me or maybe has a different way of making a chart/graph out of this data (the box office top 5).
So to be more clear. I would like to create my own chart/graph with the box office top 5 data. And I would also like it to be "live" and update itself when ever the top 5 changes or the numbers change.
There for it doesn't seem like the best idea to create my own Json with data since it won't update when the data updates without me making changes into the Json file.
The interactive part doesn't matter yet.
You want to create a chart with live data.
First thing is getting the data.
The script you are including basically creates the chart for you. So you don't have direct access to the data itself, so it would be a little tricky to use that.
You should google around for a,preferrably REST API, that provides you with data on-demand.
1. Find a suitable REST API and perform calls to fetch data
You perform HTTP requests to the API and it provides data - so on page-load for example you will request data from this API. So each time a user loads your website a call is made and it fetches the most recent data on movies.
2. Parse the data, perform calculations and built your own custom JSON
Then you need to parse that data - Take from this raw feed the data you actually need - perform calculations if needed(from what I take from your question, it's gonna be simple elementary school maths), then build you own custom JSON data structure that can be used to display the data.
3. Visualize the JSON
Include a JS library that renders Charts. Chart.js, is an excellent example and it's dead-simple to provide a JSON and render a really nice looking chart.
That's how it ,usually, goes.
I have been working on dc and crossfilter js and I currently have a large dataset with 550,000 rows and size 60mb csv and am facing a lot of issues with it like browser crashes etc
So , I'm trying to understand how dc and crossfilter deals with large datasets.
http://dc-js.github.io/dc.js/
The example on their main site runs very smoothly and after seeing timelines->memory (in console) it goes to a max of 34 mb and slowly reduces with time
My project is taking up memory in the range of 300-500mb per dropdown selection, when it loads a json file and renders the entire visualization
So, 2 questions
What is the backend for the dc site example? Is it possible to find out the exact backend file?
How can I reduce the data overload on my RAM from my application, which is running very slowly and eventually crashing?
Hi you can try running loading the data, and filtering it on the server. I faced a similar problem when the size of my dataset was being too big for the browser to handle.
I posted a question a few weeks back as to implementing the same. Using dc.js on the clientside with crossfilter on the server
Here is an overview of going about it.
On the client side, you'd want to create fake dimensions and fake groups that have basic functionality that dc.js expects(https://github.com/dc-js/dc.js/wiki/FAQ#filter-the-data-before-its-charted). You create your dc.js charts on the client side and plug in the fake dimensions and groups wherever required.
Now on the server side you have crossfilter running(https://www.npmjs.org/package/crossfilter). You create your actual dimensions and groups here.
The fakedimensions have a .filter() function that basically sends an ajax request to the server to perform the actual filtering. The filtering information could be encoded in the form of a query string. You'd also need a .all() function on your fake group to return the results of the filtering.