I'm loading in a big CSV to work with in javascript -- to create SVG elements from.
There is a property in the variable called code. In the CSV, there is a column called SPECIES. "Code" and "SPECIES" contain the same letter codes.
I'd like to be able to compare the data in the CSV to the var trees, and if for a particular record in the CSV, the "SPECIES" field is the same as the "code" value in the variable, I want to return the "common" value from the variable.
The CSV file has 150,000 records, which is why I don't want to add another column with common name to it.
Here is part of the variable (there are 54 objects in all):
var trees = [
{ code: 'ACPL',
common: 'Norway Maple',
genus: 'Acer',
species: 'platanoides',
family: 'Sapindaceae',
color: '#00567e'
},
{ code: 'ACRU',
common: 'Red Maple',
genus: 'Acer',
species: 'rubrum',
family: 'Sapindaceae',
color: '#0084ff'
},
{ code: 'ACSA1',
common: 'Silver Maple',
genus: 'Acer',
species: 'saccharinum',
family: 'Sapindaceae',
color: '#c3c3c3'
}
];
Here is part of the data.csv (150,000 records total here):
DIAMETER,SPECIES,longitude,latitude
15,ACPL,-73.935944,40.668076
22,ACPL,-73.934644,40.667189
28,ACSA1,-73.941676,40.667593
27,ACPL,-73.935095,40.666322
9,ACRU,-73.941297,40.667574
27,ACPL,-73.935149,40.666324
19,ACRU,-73.933664,40.666244
Here is what I've tried, which isn't working:
var treeCode = trees.forEach(function(each) { return each.code; }); // returns the value of "code" for each object in the trees variable
var treeCommon = trees.forEach(function(each) { return each.common; }); // returns the value of "color" for each object in the trees variable
var tip = d3.tip() // d3.tip is a tooltip library for D3.js
.attr('class', 'd3-tip')
.offset([-10, 0])
.html(function(d){if (d.SPECIES == treeCode){ // the data from the csv was loaded in earlier, and d.SPECIES returns the "SPECIES" field
return treeCommon}
})
Any thoughts about what needs to be done would be greatly appreciated! Let me know if I can clarify anything.
The full code is here: https://github.com/jhubley/street-trees/blob/master/index.html
You can see in that code that there's a looong if/else statement with hexadecimal colors. That's another example of where I want to use this approach.
Thank you for looking!
There's some great documentation for underscore at http://underscorejs.org/. D3 is a fairly robust lib so I bet some of these functions are already built in to D3 as well.
So to clarify, you want to run through the records in the CSV and pull the corresponding data from the trees object. Here's how I would do this with underscore:
// Information about each tree type
var treeInfo = [
{
code: 'ACPL',
common: 'Norway Maple',
genus: 'Acer',
species: 'platanoides',
family: 'Sapindaceae',
color: '#00567e'
},
...
];
// imported from CSV
var treeList = [
{
DIAMETER: 15,
SPECIES: "ACPL",
longitude: -73.935944,
latitude: 40.668076
}
...
]
// loop through the imported list of trees
_.each(treeList, function(){
// _.each() makes `this`` refer to the object in the list as it loops through
// find the info for this tree species
var info = _.findWhere(treeData, {"code": this.SPECIES});
// insert the keys and values from the info object into the original object from the treeList
_.extend(this, info);
})
//now treeList contains all the corresponding info (by species) from treeInfo for each tree
Related
I'm trying to store some data in an associative array in php for access in javascript later. I want to process the data in such a way that I can access it in multiple ways, say both by name and by type?
//e.g. in cpp i would do
struct SortedFruits{
std::unordered_map<std::string, Fruits> byName;
std::unordered_map<std::string, std::unordered_map<std::string, Fruits&>> byType;
}
SortedFruit fruit();
//then add them like
fruit.byName["apple"] = new Apple();
fruit.byType["red"]["apple"] = &fruit.byName["apple"];
I basically just want fruits.byType['red']['apple'] === fruits.byName['apple'] somehow. Ideally I want this to be done in php so i can store it somwhere and not recomputed, but if references/pointers can't survive inside a json I'm happy doing the sorting in javascript as well.
May I ask what's the best way to do this?
I think formatting the datas in php that way to be parsed by js later from a json is not the best way to do.
Basically you have objects Fruit that have two properties name and color. I'd just encode a json with an array of Fruit and in js map it the way I want to use those objects.
I don't think mapping the objects is the responsability of the server, it's responsability is to give the client the datas.
Edit (in response of the comment)
In JS i would even not store them in multiple maps.
const fruits = [
{color: 'red', name: 'apple'},
{color: 'red', name: 'strawberry'},
{color: 'yellow', name: 'banana'}
];
const redOnes = fruits.filter(fruit => fruit.color === 'red');
// will return [{color: 'red', name: 'apple'}, {color: 'red', name: 'strawberry'}]
const apple = fruits.find(fruit => fruit.name === 'apple');
// will return {color: 'red', name: 'apple'}
I have a problem with organizing my mongoDB data to send to my page in my res and cant figure out how to do the correct js. Here is a simplified version of my schema
var productSchema = new mongoose.Schema({
medium: String,
brand: String,
group: String
});
Here is what a typical entry looks like
medium :"Acrylic",
brand :"liquitex",
group :"heavy body"
there are many more entries in the schema, but these are the only ones I need to be able to sort and organize the returned results with. The problem is I have a route that returns all colors in my database and I want to be able to display them in sections on my page that are grouped under Brand, and then has the individual colors listed under the correct group.
The problem is there are paints from other brands that fall into the heavy body group and so when I use a filter function to sort my data by group, some brands get mixed together. I cant filter by brand, because some brands have acrylic and watercolor so then those get lumped together.
I need some way to filter the returned results of a
mongoose.find({})
that can use the group data as a filter, but then filter those results by the brands so they get separated into the correct brand categories.
I have this so far:
this is all a stripped down version of my app.js file:
//finds all colors in the DB
Color.find({}).lean().exec(function( err, colors)
var groups = [];
// find all groups in the databse
colors.forEach( function(color){
groups.push(color["group"]);
});
//returns only unique names to filter out duplicates
var groupTypes = Array.from(new Set(groups));
var tempVariableBrands = [];
// this sorts all returned paints into their respective group, but we get paints from multiple brands under the same group and that is not good
groupTypes.forEach( function(group){
var name = group;
var result = colors.filter(obj => { return obj.group === group });
tempVariable.push( {group : name, result } );
});
// the tempVariable gets sent to my page like so
res.render("landing", {colorEntry:tempVariable} );
and this works fine to allow me to display each paint by its grouping, but that fails when there is more than one paint from a different manufacturer that is considered the same group like a "heavy body". This is my ejs on my page that works fine:
<% colorEntry.forEach( function(entry){ %>
<div class="brandBlock">
<div class="brandTitle">
<span><%=entry.result[0].brand%> - <%=entry.result[0].group%></span>
I for the life of me cant seem to figure out the combination of filter() and maybe map() that would allow this kind of processing to be done.
My database has like 600 documents, colors from a number of different manufacturers and I don't know how to get this as a returned structure: lets say this is a few colors in the DB that get returned from a mongoose find:
[{ medium: "Oil",
brand: "Gamblin",
group: "Artists oil colors"},
{ medium: "Acrylic",
brand: "Liquitex",
group: "Heavy Body"},
{ medium: "Acrylic",
brand: "Golden",
group: "Heavy Body"}
]
i need to organize it like this or something similar. It can be anything that just sorts this data into a basic structure like this, I am not confined to any set standard or anything, this is just for personal use and a site I am trying to build to learn more.
returnedColors = [ { brand: "Gamblin", group: "Artists oil colors", { 50 paints colors returned} },
{ brand: "liquitex" , group: "heavy body", { 20 paint colors returned } },
{ brand: "golden" , group: "heavy body",{ 60 paint colors returned} }
];
I am not a web developer and only write some web code every 6 months or so and have been trying how to figure this out for the last 2 days. I can't wrap my head around some of the awesome filter and map combo's i have seen and cant get this to work.
Any help or advice would be great. I am sure there are many areas for improvement in this code, but everything was working up until I entered paints that were from different brands that had the same group type and i had to try to rewrite this sorting code to deal with it.
It boils down to needing to be able to iterate over the entire set of returned documents from the DB and then sort them based off 2 values.
UPDATE:
I was able to get something that works and returns the data in the format that I need to be able to send it to my ejs file and display it properly. The code is rather ugly and probably very redundant, but it technically works. It starts off by using the group value to run over paints since each set of paints will have a group name, but can sometimes share a group name with a paint from another brand like "heavy body".
groupTypes.forEach( function(group){
var name = group;
var result = colors.filter(obj => { return obj.group === group });
// this gets brand names per iteration of this loop so that we will know if more than one brand of paint
// has the same group identity.
var brands = [];
result.forEach( function(color){
brands.push(color["brand"]);
});
// This filters the brand names down to a unique list of brands
var brandNames = Array.from(new Set(brands));
// if there is more than one brand, we need to filter this into two separate groups
if( brandNames.length > 1){
//console.log("You have duplicates");
brandNames.forEach( x => {
var tmpResult = [...result];
var resultTmp = result.filter(obj => { return obj.brand === x });
result = resultTmp;
//console.log("FILTERED RESULT IS: ", result);
tempVariable.push( {brand: x ,group : name, result } );
result = [...tmpResult];
});
}else{
tempVariable.push( {brand: result[0].brand ,group : name, result } );
}
});
if anyone can reduce this to something more efficient, I would love to see the "better" way or "right" way of doing something like this.
UPDATE2
Thanks to the answer below, I was put on the right track and was able to rewrite a bunch of that long code with this:
Color.aggregate([
{
$sort: { name: 1}
},
{
$group: {
_id: { brand: '$brand', group: '$group' },
result: { $push: '$$ROOT' }
}
},
{ $sort: { '_id.brand': 1 } }
], function( err, colors){
if(err){
console.log(err);
}else{
res.render("landing", {colorEntry:colors, isSearch:1, codes: userCodes, currentUser: req.user, ads: vs.randomAds()} );
}
});
Much cleaner and appears to achieve the same result.
Since you're using MongoDB, "right" way is to utilize an Aggregation framework, precisely, $group stage.
Product.aggregate([{
$group: {
_id: { group: '$group', brand: '$brand' },
products: { $push: '$$ROOT' }
}
}])
This will output array of objects containing every combination of brand and group, and push all relevant products to corresponding subarray.
Combine it with $project and $sort stages to shape your data further.
So I'm using the Topojson functionality of D3.js to create an svg world map. I've managed to figure out the scale and everything perfectly so far despite being relatively new to d3. The issue I'm having is attempting to merge or filter the data from the topojson file with some sets that I have defined. I'm going to leave out the functions used for projection. Here's what I have so far:
var svg = d3.select("#map svg");
var regions = [
{
region: 'low',
set: d3.set([ "USA", "CAN" ])
},
{
region: 'med',
set: d3.set([ "AUS" ])
},
{
region: 'high',
set: d3.set([ "RUS" ])
}
];
d3.json("site/js/topo.json", function(error, topology) {
// INDIVIDUAL COUNTRIES
svg.selectAll("path")
.data(topojson.feature(topology, topology.objects.worldByIso).features)
.enter()
.append("path")
.attr("data-iso", function(d){
return d.properties.iso;
})
.attr("d", path);
});
I'm able to return the d.properties.iso value from the topojson file to a data attribute, but what I'd really like to do is check that iso property against the 'regions' array to instead return the region (low, med, or high) if it matches the set.
Ideally the solution wouldn't need to loop through 'regions' each time to check so it can be processed quickly. Hopefully this all makes sense, I'm writing this as I run out the door from overtime!
Array.reduce
I think you want Array.reduce(). This will let you collapse an array to a single value. I changed your d3.sets to simple arrays so I could use the contains() method but a quick glance at the d3 documentation looks like their sets have has() instead.
var regions = [{
region: 'low',
set: ["USA", "CAN"]
},
{
region: 'med',
set: ["AUS"]
},
{
region: 'high',
set: ["RUS"]
}
];
let getRegion = r => regions.reduce((a, c) => c.set.includes(r) ? c.region : a, null);
console.log(getRegion('AUS'));
console.log(getRegion('USA'));
console.log(getRegion('EUR'));
I am using Dexie IndexedDB wrapper and I am trying to add an object to an existing array which in inside a nested object. The structure looks similar to below
{
Name : 'John',
age : 33,
tags : {
skill: [{
first: '.NET',
second: 'JAVA',
third: [{special1:'sleep'},{special2:'eat'}]
}]
}
}
I have tried many way to push object special3:'run' to skill.third but without success. My last attempt looked something like this
const pathObject = {};
const fullPath = 'result.tags.skill[3].third';
pathObject[fullPath] = {special3:'run'};
db.inspections.update(id, pathObject);
The object is added outside and not inside the array 'third' something like below
{
Name : 'John',
age : 33,
tags : {
skill: [{
first: '.NET',
second: 'JAVA',
third: [{special1:'sleep'},{special2:'eat'}]
}]
skill[3]: {
third: {special3:'run'}
}
}
}
I wish to know if there a way to add to arrays in nested object using Dexie if not is there a way to achieve this using indexeddb. Help is appreciated as problem is been holding back progress
The easiest is to use Collection.modify() with a callback function to mutate your model:
db.inspections.where('id').equals(id).modify(x =>
x.tags.skill[0].third.push({special3:'run'}) );
If you want to use a keypath containing array items, it is also possible, as arrays can be looked at as objects with numeric keys:
db.inspections.update(id, {"tags.skill.0.third.3": {special3:'run'}});
I'm essentially using the top answer here ( from sdespont) to try to destroy some tables.
I have one table that shows me the status of a .csv file being uploaded.
FileuploadTable:
FileName FileType FileSize AvailableActions
I have a second table that is the table displaying data from the .csv file.
I need provide the user the ability to reset the form, i.e. get rid of the .csv, and get rid of all of the data, destroy() the two tables separately, and empty() them of all the data that was there initially.
Here is the issue I'm running into.
I can't seem to set the column titles of FileUploadTable after destroy() and empty(). When I attempt to upload a new file, the elements are still on the page, just empty, though the same initialization is being called
I can't seem to get rid of the column titles in CSVTable after destroy() and empty(). When I attempt to upload a different csv, it tries to match column headers to the ones that should have been destroyed, but they don't match because, though CSVTable was destroyed and emptied, the column titles are still there...?
Not sure what I'm missing. They're being set properly on initial create.
$(elem).DataTable()
Can anyone show me a basic working implementation of destroying/emptying datatables, then re initializing with different data, so I can try to mimic it. My brain is mush from looking at their docs for the last 3 days, making no progress.
Example of my data object
[
{
//key = column title
//"val" = data in row
//object = row
key: "val",
//i.e.
FirstName: "Bob",
LastName: "Barker",
Age: 800,
//etc
},
//etc
]
OK. You can make a simple iteration over your data using Object.keys() that produces a column object on the fly holding corresponding data and title values :
var columns = [], keys = Object.keys(data[0]);
for (var i=0;i<keys.length;i++) {
columns.push({ data: keys[i], title: keys[i] });
}
Use that inside a general function that initialises the table and take care of destroying and emptying if already initialized :
var table = null;
function initTable(data) {
var columns = [], keys = Object.keys(data[0]);
for (var i=0;i<keys.length;i++) {
columns.push({ data: keys[i], title: keys[i] });
}
if (table) {
table.destroy();
$('#example').empty();
}
table = $('#example').DataTable({
data: data,
columns : columns
})
}
Now imagine the following is the success handlers of your AJAX calls, or however you get the new data that should be populated to the table :
$('#insert1').on('click', function() {
var data = [
{ FirstName: "Bob", LastName: "Barker", Age: 800 },
{ FirstName: "John", LastName: "Doe", Age: 'N/A' }
]
initTable(data);
})
$('#insert2').on('click', function() {
var data = [
{ Animal : "Lion", Taxon : 'Panthera leo' },
{ Animal : "Cheetah", Taxon : 'Acinonyx jubatus' }
]
initTable(data);
})
demo -> http://jsfiddle.net/d5pb3kto/