Develop Reference

d3/js data manipulation dropping columns

I have a dataset in csv and it looks like below.
country,col1,col2,col3
Germany,19979188,11233906,43.7719591
UK,3839766,1884423,50.92349378
France,1363608,796271,41.60557873
Italy,957516,557967,41.72765781
I'd like to drop col1, col2 off while keeping country and col3. If possible, I'd like to wrap it into a function where I can pass column list that I'd like to drop/keep.
Using pandas, which I'm familiar with, I can easily do it. e.g. data.drop(['col1', 'col2'], axis = 1). But I found d3 way or js way in general is based on each row so couldn't come up with an idea to drop columns.
I was thinking of d3.map() taking desirable columns only. But I was stuck to build a general function that the column list can be passed in.
Could anyone have thoughts?

D3 fetch methods, like d3.csv, will retrieve the whole CSV and will create an array of objects based on that CSV. Because of that, filtering out some columns is useless. Actually, it's worse than useless: you'll spend time and resources with an unnecessary operation.
Therefore, the only useful solution is, if you have access and own that CSV, creating a new CSV without those columns. That way you'll have a smaller file, faster to load. Otherwise, if you cannot change the CSV itself, don't bother: just load the whole thing and use the columns you want (which will be properties in the objects), ignoring the others.
Finally, if you have a lot of data manipulation it might be interesting reducing the size of the objects in the data array. If that's your case, use a row function to return only the properties you want. For instance:
d3.csv(url, function (d){
return {country: d.country, col3: d.col3}
}).then(etc...)

How to parse value out of column in OpenRefine

I have the results from Google place details. I would like to parse the phone number out of it. How could I do so? I am using OpenRefine and using fetching column on the basis of another column.

Here is an example of Google Place JSON.
As you can see, "formated_phone_number" is a direct child of "result". In Open Refine, you can extract this element using the following GREL formula:
value.parseJson().result.formatted_phone_number
If there are more than one phone number, use a for loop to iterate over the array. The syntax in Open refine is:
forEach(value.parseJson().result, x, x.formatted_phone_number)
(x is a variable name, you can use it or what you want instead.)
The result will be an array. Since an Open Refine column can only display strings, numbers and dates, you must convert the array to string using the join function (and a delimiter of your choice).
forEach(value.parseJson().result, x, x.formatted_phone_number).join('::')

Reading columns in their order of the csv file

When data is loaded from a csv file, is it possible to get the order of the columns ?
E.g. the typical way to load a csv file is by calling the d3.csv function:
d3.csv("data.csv", function(error, data) {
var ageNames = d3.keys(data[0]).filter(function(key) { return key !== "State"; });
Typically, columns are stored as properties of an object, so the order cannot be retrieved anymore. Also, d3.keys() returns an array with a undefined order.
I am asking this because I want to sort the columns by the order in the csv file. Any help would be greatly appreciated.

You don't need to use the name of the column to access the data, you can use the index of the keys. The index is determined by the order in the file, so exactly what you're looking for.
To get the keys of an object, use Object.keys():
var keys = Object.keys(data[0]);
data[0][keys[0]]; // datum in the first row, first column

Coming from relational views, it seems surprising that the d3.js way of reading from a file has the opposite properties:
The relational view:
rows don't have any guaranteed order
columns have a well defined order and can be addressed by refering to their position
The d3.js view (of a csv file):
rows have a well defined order and can be addressed by refering to their row number. (they are stored in an array)
columns end up as a set of properties in an Object, thus they don't have a well defined order
If you want to access the order of columns as in the csv file, you can safely assume the "for...in.." loop over the properties or the d3.keys() returns the columns in the given order.
This order can just be destroyed if columns are added or removed. This is for most practical applications not the case.
See also https://javascriptweblog.wordpress.com/2011/01/04/exploring-javascript-for-in-loops/. The one exception described is: When column names are sole numbers, then the Chrome browser orders them according to their number order but their column order.

How to sort a a table with Javascript without library?

There is two main aims:
Sort table of data using javascript without libraries such as jquery, so pure javascript.
Sort/group rows together based on class/id (like css).
I thought maybe getElementById could be useful, but don't know enough javascript to investigate properly.

A table's rows can be found in the rows[] array of said table.
You can create an array containing those rows.
You can then .sort() the array based on a callback function that looks up certain info, in the form
.sort(function(a,b) { /* return -1 if a comes before b or 1 if b comes before a */ })
If you iterate over the array and appendChild the rows to the table they are in, the result is a sorted table.
If you have problems with the actual code, please post what you have so far and we can help more specifically ;)

JavaScript sort with numbers

The following program (taken from a tutorial) prints the numbers in an array in order from lowest to highest. In this case, the result will be 2,4,5,13,31
My question relates to the paramaters "a" and "b" for the function compareNumbers. When the function is called in numArray.sort(compareNumbers) what numbers will be the parameters a and b for the function. Does it just move along the array. For example, start with a=13 and b=2? After that, does the function run again comparing a=2 and b=31? or would it next compare a=31 and b=4?
Can someone please explain how that part works and also how it manages to sort them from lowest to highest? I don`t see how the function manages to do the necessary calculations on the numbers in the array.
function compareNumbers(a,b) {
return a - b;
}
var numArray = [13,2,31,4,5];
alert(numArray.sort(compareNumbers));

The particular pairs that get passed in depend on the sorting algorithm being used. As the algorithm tries to go about sorting the range, it needs to be able to compare pairs of values to determine their ordering. Whenever this happens, it will call your function to get that comparison.
Because of this, without inside knowledge about how the sorting algorithm works, you cannot predict what pairs will get compared. The choice of algorithm will directly determine what elements get compared and in what order.
Interestingly, though, you can actually use the comparison function to visualize how the sort works or to reverse-engineer the sorting algorithm! The website sortviz.org has many visualizations of sorting algorithms generated by passing custom comparators into sorting functions that track the positions of each element. If you take a look, you can see how differently each algorithm moves its elements around.
Even more interestingly, you can use comparison functions as offensive weapons! Some sorting algorithms, namely quicksort, have particular inputs that can cause them to run much more slowly than usual. In "A Killer Adversary for Quicksort," the author details how to use a custom comparator to deliberate construct a bad input for a sorting algorithm.
Hope this helps!

The two parameters will be elements of your array. The system will compare enough pairs to be able to sort them correctly. Nothing else is guaranteed.
There are lots of things the sort method could be doing under the hood; see, e.g., http://en.wikipedia.org/wiki/Sorting_algorithm for some of them. Most Javascript implementations probably use some variant of either quicksort or mergesort.
(Here are super-brief descriptions of those. Quicksort is: pick an element in the array, rearrange the array to put everything smaller than that in front of everything larger, then sort the "smaller" and "larger" bits. Mergesort is: sort the first half of the array, sort the second half of the array, and then merge the two sorted halves. In both cases you need to sort smaller arrays, which you do with the same algorithm until you get to arrays so small that sorting them is trivial. In both cases, good practical implementations do all sorts of clever things I haven't mentioned.)

It will be called for all pairs of a,b that sorting algorithm need to get all array sorted. Check out http://en.wikipedia.org/wiki/Sorting_algorithm for brief list of sorting algorithms.

When you pass a function to Array.sort(), it expects two parameters and returns a numerical value.
If you return a negative value, the first parameter will be placed before the second parameter in the array.
If you return a positive value, the first parameter will be placed after the second parameter in the array.
If you return 0, they will stay in their current position.
By doing return a - b;, you are returning a negative number if a is less than b (2 - 13 = -11), a positive number if b is less than a (13 - 2 = 11), and zero if they are even (13 - 13 = 0).
As far as which numbers are compared in what order, I believe that is up to the javascript engine to determine.
Check out the documentation on javascript array sorting at the MDC Doc Center for more detailed information.
https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Array/sort
(BTW, I always check the MDC Doc Center for any questions about how javascript works, they have the best information on the language AFAIK.)