I have a simple csv file like below. I want to read that file and find lat and long value from a webservice than write it as new columns in to the arrays objects.
I can read csv, import it, I can get lat and long values. But when i try to update array it does not work. Sometimes it update only one row, sometimes 2 rows (it is meaningless) but i can not update all off the row, what i need.
I am working on a react habitat. My webservice and code is below. I will be happy if you help me to find my bug. Thanks.
orderid,postcode,weight
1,BB9 7BP,5000
2,E4 9JB,3500
3,WR1 2RX,10000
4,WR5 2AD,5000
function getCoordinatesForAddresses(event) {
orders.map((e) => {
let url = `https://nominatim.openstreetmap.org/search.php?q=${e.postcode}&polygon_geojson=1&format=jsonv2`;
axios.get(url).then((res) => {
orders.find((ord) => ord.orderid === e.orderid)["latitude"] =
res.data[0].lat;
orders.find((ord) => ord.orderid === e.orderid)["longitude"] =
res.data[0].lon;
});
});
}
when i want to see all array i can see i have added new colums
enter image description here
but when i want to see one by one i could not see my new colums but when i expand object i see the values in the object.
enter image description here
it is very interesting for me. I tried with filter, find and nearly everything.
Related
I want to copy three cells, which are non-adjacent (AA9, AA12, and AA15), and paste them on another sheet in a single row, like a log sheet. (A2,B2,C2) I'm using "getLastRow+1" to make this happen. Currently, I'm only able to use getValue and setValue for a single cell (AA9) and after searching for similar examples online, I'm still struggling to make this work.
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("Main")
var destSheet = ss.getSheetByName('Log');
// Get the Cell Values from Main Sheet
var streak = sheet.getRange('AA9').getValue();
// Set the Values on Log Sheet
destSheet.getRange(destSheet.getLastRow()+1,1).setValue(streak);
I'm struggling to understand how to correctly implement what I want. I essentially want to copy the values in AA9, AA12, and AA15, and paste them (or use setValue) in A2,B2, and C2 on the other sheet.
I'm slowly learning and getting a better understanding of everything, and examples from google searches have gotten me far, but this is a headscratcher for me.
I've tried using Named Ranges, but I didn't understand how to find the last EMPTY row of that named range.
I've tried creating an Array for those cells, but didn't understand how to use SetValues correctly (got an error about the number of rows not being equal to the data rows, something like that)
// THIS DIDN"T WORK
var results = ['AA9','AA12','AA15'];
var streak = sheet.getRangeList(results).getRanges().map(range => range.getValues());
destSheet.getRange(destSheet.getLastRow()+1,1).setValues(streak);
I've also searched google quite a bit, and while the results have helped me script a random number generator (not shown here), I still struggle with more basic tasks.
You can make your latter code example work by using Range.getValue() instead of Range.getValues().
To easily append the values to the destination sheet, use Sheet.appendRow(), like this:
const sheet = ss.getSheetByName('Main');
const destSheet = ss.getSheetByName('Log');
const data = sheet.getRangeList(['AA9', 'AA12', 'AA15'])
.getRanges()
.map(range => range.getValue());
destSheet.appendRow(data);
Try it this way:
function myfunk() {
const ss = SpreadsheetApp.getActive();
const sh = ss.getSheetByName('Sheet0');
const dsh = ss.getSheetByName('Sheet1');
const [a,,,b,,,c] = sh.getRange("AA9:AA15").getValues().flat();
dsh.appendRow([a,b,c]);
}
When I use getValues, it works perfectly! But it does throw an error, saying "Exception: The parameters (number[]) don't match the method signature for SpreadsheetApp.Range.setValues.". But again, it does actually work on my spreadsheet.
When I use getValue, it strangely works, and doesn't throw an error, it sets the values appropriately in an appended row, but then loops back around again and repeatedly sets the value in the first cell again, albeit one row down (because it just filled the previous row so it moves down). It's so strange. I can see it do it slowly, it sets the values in the row, a split second later it loops back to the first value, but one row down, and sets that value again. But none of the other ones.
If you can visualize this, it'll post 3 numbers, correctly, appended in the row, then one row down it'll post only 1 number, but leave the other two blank.
Screenshot Example
function log_data() {
const data = sheet.getRangeList(['AA9','AA12','AA15'])
.getRanges()
.map(range => range.getValue());
destSheet.appendRow(data);
destSheet.getRange(destSheet.getLastRow()+1,1).setValue(data);
}
I'm trying to get the table data from some columns and I've tried a couple of variations I could think of, but none worked:
function updateAllTasks() {
var table = document.getElementById("dtable");
var [, ...tr] = table.querySelectorAll("tr");
var tableData = [...tr].map(r => {
var td = r.querySelectorAll("td");
console.log([...td].map(c => c[0].innerHTML, c[1].innerHTML, c[8].querySelectorAll('input[name="rowcheckbox"]')[0].value)); //This is the one giving me error ReferenceError: c is not defined
})
console.log(JSON.stringify(tableData));
}
Thanks for your help!
As mentioned by Andy, following data structures logic. The best approach for this case will be to call the data inside of an Array, and after that, build the table with the data.
This will provide more flexibility the moment you process the information.
Reference
Javascript Array
Arrays in Apps Script
So I have a table of 250 of rows, and I want to just get all the values from one column and check if they meet the required criteria:
const rows = browser.elements(selector..);
const numbers = [];
rows.value.forEach(cellData => {
const value = browser.elementIdText(cellData.value.ELEMENT).value;
// some logic to check if the value is ok
numbers.push(value);
});
// check if all numbers are sorted correctly
, but it most of the time it fails on the line (it says stale element reference: element is not attached to the page document):
const value = browser.elementIdText(cellData.value.ELEMENT).value;
I tried doing cellDate.getText(), but there was a Java socket error, could someone help? I assume the selector is not attached to the page as indicated, but I can't figure my head out how to just loop through them all.
I had a solution similar to your method before and while it seems to work, I think there might just be some slight adjustments to your code to get what you want. I never had much luck chaining from the end of the elementIdText call.
Step 1: Grab all the Data (browser.elements or browser.$$):
let cellData = browser.$$('selector that matches desired Column Data')
The above returns an array of JSON WebElements. And as you know you can correctly loop through the array looking at the "values". If you use the selector that matches the Column Values you're looking for you should have all similar data stored in the element.value.ELEMENT.
Step 2: Loop through the cellData array and pluck out the text values of the ELEMENT using browser.elementIdText()
cellData.forEach((elem) => {
let number = browser.elementIdText(elem.value.ELEMENT)
//elementIdText also returns a JSON WebElement so it's number.value
if(number.value === <condition>) {
console.log('number looks good')
//perform other on value logic
}
})
//perform other logic still in loop EX: array.push()
})
I hope this helps! Let me know if you hit any snags!
I'm stuck with a quite simple problem and need help.
I have a big CSV file with 50 columns which i absolutely can't modifie.
Now i want to make a chart where i only need 5-6 columns out of it.
My idea was now to make a new "data2" which contains only these 5-6 columns (with key and evertything) and work with this data2.
But i'm not able to create this data2.
To filter which columns i need i wanted to work with regex. Something like this:
d3.keys(data[0]).filter(function(d) { return d.match(/.../); })
But how do i create the new data2 then? I'm sure i need to work with d3.map but even with the api i'm not able to understand how it works correctly.
Can someone help me out?
Firstly, your question's title is misleading: you're not asking about making a smaller CSV file, since the file itself is not changed. You're asking about changing the data array created by D3 when that CSV was parsed.
That brings us to the second point: you don't need to do that. Since you already lost some time/resources loading the CSV and parsing that CSV, the best idea is just keeping it the way it is, and using only those 5 columns you want. If you try to filter some columns out (which means deleting some properties from each object in the array) you will only add more unnecessary tasks for the browser to execute. A way better idea is changing the CSV itself.
However, if you really want to do this, you can use the array property that d3.csv creates when it loads an CSV, called columns, and a for...in loop to delete some properties from each object.
For instance, here...
var myColumns = data.columns.splice(0, 4);
... I'm getting the first 4 columns in the CSV. Then, I use this array to delete, in each object, the properties regarding all other columns:
var filteredData = data.map(function(d) {
for (var key in d) {
if (myColumns.indexOf(key) === -1) delete d[key];
}
return d;
})
Here is a demo. I'm using a <pre> element because I cannot use a real CSV in the Stack snippet. My "CSV" has 12 columns, but my filtered array keeps only the first 4:
var data = d3.csvParse(d3.select("#csv").text());
var myColumns = data.columns.splice(0, 4);
var filteredData = data.map(function(d) {
for (var key in d) {
if (myColumns.indexOf(key) === -1) delete d[key];
}
return d;
})
console.log(filteredData)
pre {
display: none;
}
<script src="https://d3js.org/d3.v4.min.js"></script>
<pre id="csv">foo,bar,baz,foofoo,foobar,foobaz,barfoo,barbar,barbaz,bazfoo,bazbar,bazbaz
1,2,5,4,3,5,6,5,7,3,4,3
3,4,2,8,7,6,5,6,4,3,5,4
8,7,9,6,5,6,4,3,4,2,9,8</pre>
I'm looking for some advice on how to effectively use large amounts of data with d3.js. Lets say for instance, I have this data set taken from a raw .csv file (converted from excel);
EA
,Jan_2016,Feb_2016,Mar_2016
Netherlands,11.7999,15.0526,13.2411
Belgium,25.7713,24.1374
France,27.6033,23.6186,20.2142
EB
,Jan_2016,Feb_2016,Mar_2016
Netherlands,1.9024,2.9456,4.0728
Belgium,-,6.5699,7.8894
France,5.3284,4.8213,1.471
EC
,Jan_2016,Feb_2016,Mar_2016
Netherlands,3.1499,3.1139,3.3284
Belgium,3.0781,4.8349,5.1596
France,16.3458,12.6975,11.6196
Using csv I guess the best way to represent this data would be something like;
Org,Country,Month,Score
EA,Netherlands,Jan,11.7999
EA,Belgium,Jan,27.6033
EA,France,Jan,20.2142
EA,Netherlands,Feb,15.0526
EA,Belgium,Feb,25.9374
EA,France,Feb,23.6186
EA,Netherlands,Mar,13.2411
EA,Belgium,Mar,24.1374
EA,France,Mar,20.2142
This seems very long winded to me, and would use up a lot of time. I was wondering if there was an easier way to do this?
From what I can think of, I assume that JSON may be the more logical choice?
And for context of what kind of chart this data would go into, I would be looking to create a pie chart which can update the data depending on the country/month selected and comparing the three organisations scores each time.
(plnk to visualise)
http://plnkr.co/edit/P3loEGu4jMRpsvTOgCMM?p=preview
Thanks for any advice, I'm a bit lost here.
I would say the intermediary step you propose is a good one for keeping everything organized in memory. You don't have to go through a csv file though, you can just load your original csv file and turn it into an array of objects. Here is a parser:
d3.text("data.csv", function(error, dataTxt) { //import data file as text first
var dataCsv=d3.csv.parseRows(dataTxt); //parseRows gives a 2D array
var group=""; // the current group header ("organization")
var times=[]; //the current month headers
var data=[]; //the final data object, will be filled up progressively
for (var i=0;i<dataCsv.length;i++) {
if (dataCsv[i].length==1 ) { //group name
if ( dataCsv[i][0] == "")
i++; //remove empty line
group = dataCsv[i][0]; //get group name
i++;
times = dataCsv[i];//get list of time headings for this group
times.shift(); // (shift out first empty element)
} else {
country=dataCsv[i].shift(); //regular row: get country name
dataCsv[i].forEach(function(x,j){ //enumerate values
data.push({ //create new data item
Org: group,
Country: country,
Month: times[j],
Score: x
})
})
}
}
This gives the following data array:
data= [{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},
{"Org":"EA","Country":"Netherlands","Month":"Feb_2016","Score":"15.0526"}, ...]
This is IMO the most versatile structure you can have. Not the best for memory usage though.
A simple way to nest this is the following:
d3.nest()
.key(function(d) { return d.Month+"-"+d.Country; })
.map(data);
It will give a map with key-values such as:
"Jan_2016-Netherlands":[{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},{"Org":"EB","Country":"Netherlands","Month":"Jan_2016","Score":"1.9024"},{"Org":"EC","Country":"Netherlands","Month":"Jan_2016","Score":"3.1499"}]
Use entries instead of mapto have an array instead of a map, and use a rollup function if you want to simplify the data by keeping only the array of scores. At this point it is rather straightforward to plug it into any d3 drawing tool.
PS: a Plunker with the running code of this script. Everything is shown in the console.