Converting large number of database results into array of objects in javascript - javascript

I am pulling a database query that has the following info:
id, name, roleId, roleTitle
In the query, I am pulling for users and their roles. Each user can have 0 to N number of roles. I want to in the end have an object like this:
{
id
name
roles: [{
id
title
}]
}
What would the most efficient way of doing this be? Currently I am doing something like this:
const data = [];
arr.forEach((u) => {
const index = data.findIndex(x => x.id === u.id);
if (index >= 0) {
data[index].roles.push({ id: u.roleId, title: u.roleTitle });
} else {
data.push({
id: u.id,
name: u.name,
roles: u.roleId ? [{
id: u.roleId,
title: u.roleTitle,
}] : [],
});
}
}
This solution works correctly but wasn't sure if this was the fastest way to get this done if we scale the user numbers to 10k with an average role per user of 3 or 50k and 5 roles per user

Your best bet is actually to do this all in SQL, since you are using PostgreSQL for your database (as mentioned in comments). I don't know the exact names of your tables and columns, so you may need to tweak this, but this will get you what you want:
SELECT json_agg(t)
FROM (
SELECT
u.id,
u.name,
ro.roles
FROM "user" u
LEFT JOIN (
SELECT
ur.user_id,
json_agg(
json_build_object(
'id', r.id,
'title', r.title
)
) AS roles
FROM user_role ur
LEFT JOIN "role" r ON r.id = ur.role_id
GROUP BY ur.user_id
) ro ON ro.user_id = u.id
) t;
SQL Fiddle: http://www.sqlfiddle.com/#!17/5f6ca/11
Explanation
json_build_object will create an object using the name / value pairs specified, so:
json_build_object(
'id', r.id,
'title', r.title
)
Combines the role id and title into a JSON object like this:
{id: 1, title: "Title 1"}
json_agg aggregates multiple rows into a JSON array, so it converts the role objects above into a single column that is an array of role objects per user (thanks to the GROUP BY u.id part of the inner subquery). The inner subquery gives us a result set like this (one row per user)
| user_id | roles |
|---------|------------------------------------------------------|
| 1 | [{id: 1, title: "Role 1"}, {id: 2, title: "Role 2"}] |
Then the subquery is joined to the user table, and all of that is wrapped in another subquery so json_agg can be used on the entire result and return a single json object that is an array of users with roles.

This almost certainly isn't the most efficient possible version but is faster than what you're doing now:
const data = Object.values(arr.reduce((obj, {id, name, roleId, roleTitle}) => {
if (!(id in obj)) {
obj[id] = {
id,
name,
roles: {},
};
}
if (!obj[id].roles[roleId]) {
obj[id].roles[roleId] = {
id: roleId,
title: roleTitle,
};
}
return obj;
}, {}));
By using objects (hashes) instead of arrays, determining if the user is already there or if the user already has a role is a constant-time O(1) operation (the cost of the hashing function). But searching an array, depending on the search method used, is linear in the worst case O(n) and even the best case is O(log n).
You could go down the rabbit hole of micro-optimizations that will change with the wind, but choosing the correct data structures and algorithms will usually get you the most bang for your optimization buck.
I've used Object.values to convert back to an array at the end, if you omit this and just stick with objects it could be even faster.

Hope this helps.
var modified_array = function(xs, key) {
return xs.reduce(function(rv, x) {
obj = (rv[x[key]] = rv[x[key]] || {});
obj.id = x.id;
obj.name = x.name;
obj.roles = obj.roles || []
obj.roles.push({ id: x.roleId, title: x.roleTitle})
return rv;
}, {});
};
arr = [{id:1,name:"abd",roleId: 10,roleTitle: "hello"},
{id:1, name: "abd", roleId: 15,roleTitle: "fello"}]
console.log( Object.values(modified_array(arr, 'id')));

Related

How to improve nested object arrays lookup

Is there a common known way to chain .map or .filter or .find expressions to accomplish this kind of lookup?
Given and array of objects within an array of objects
customerGroups :
[
{
id: 1,
customers: [{
id: 1, // The same customer may appear in multiple groups
name: 'Jhon'
}],
},
{
id: 2,
customers: [{
id: 2,
name: 'Jhon'
}],
},
{
id: 3,
customers: [{
id: 2,
name: 'Doe'
}],
},
]
In the use case where you have the customer.id and want to find out the customer.name I would like to extract the customers array to use the Array.Find method
const idSearch = 1
const customerName = customers.find(({id})=>id==idSearch).name
So far I been trying with
const customers = customerGroup.find(({ customer }) =>
customer.find(({ id }) =>idSearch === id),
)?.customers
const customerName = customers.find(({id})=>id==idSearch).name
I believe there is a better way to do this but I'm too burnout to figure it out.
I've also tried some shenanigans with the .map to make a new array with all the customers in it but no good results so far.
I could also fetch that array from my Backend but I already have all the customers in memory so that would be an overheat.
There is not one native method that does this, but you could first combine the customer arrays into one with flatMap, and then use find:
const customerGroups = [{id:1,customers:[{id:1,name:'Jhon'}]},{id:2,customers:[{id:2,name:'Jhon'}]},{id:3,customers:[{id:2,name: 'Doe'}]}];
const idSearch = 1;
const allCustomers = customerGroups.flatMap(({customers}) => customers);
const name = allCustomers.find(({id}) => id === idSearch)?.name;
console.log(name);
This approach works because as soon as the inside find loop discovers a result, both the inside and outside loop will terminate, leaving name set as the match which caused the loops to terminate (or as undefined if no match was found).
const d = [{id:1,customers:[{id:1,name:'Jhon'}]},{id:2,customers:[{id:2,name:'Jhon'}]},{id:3,customers:[{id:3,name: 'Doe'}]}]
const idSearch = 1
let name
d.find(j=>j.customers.find(i=>i.id===idSearch && ({name}=i)))
console.log(name)

Sort array in Javascript (Example)

I have some data in the database used for a SEARCH BAR.
In this table the field is called searchPeople (all lowercase) and contains data like:
Name##CityName
john##rome
romeu##napoli
romeu2##milan
So the user types on the SEARCH BAR some thing Rome, people that contain Rome in either their name or city. The search works well but I would like to "PRIORITIZE" the exact match String on top of the array. currently the data comes in random by the database order
{
name: 'John',
city: 'Rome'
}
Should be on top because the city matches === the string given by the user. THis can either be the name or city, I just gave an example using city match
const people = [{
name: 'Romeu',
city: 'Napoli'
},
{
name: 'John',
city: 'Rome' // this object should be first because there is a matching result
},
{
name: 'Romeu2',
city: 'Milan'
}
];
console.log(people);
// How can I sort people array with most relevant results on top?
Is there a way to sort my array to put the more correct search results on top?
You can do it like this:
sort - to sort original array
includes - to check if user input is exact match of either name or city property.
const data = [
{ name: 'Romeu', city: 'Napoli' },
{ name: 'John', city: 'Rome' },
{ name: 'Romeu2', city: 'Milan' },
];
const result = data.sort((a, b) => [a.name, a.city].includes('Rome') ? -1 : 0);
console.log('Result: ', result);
people.sort((a, b) => {
if(a.city.toLowerCase() === '*user-input*'.toLowerCase() || a.name.toLowerCase() === '*user-input*'.toLowerCase()) {
return -1;
}
return 0;
})
you can try this.
What you're speaking about is called 'relevance'. And you need somehow calculate it. And first you need to define it somehow. What is considered as more relevant in this particular search? Is name match more relevant than city match? Strict match is more relevant than partial match, that's clear.
So, strict match contributes to relevance value more that a partial match. For instance, strict match on field 'city' might give a value of 10 to a relevance score. While partial match on the beginning of a name can give value of 5. And maybe some variations of partial match contribute more than others.
For instance, search term me might contribute 1 for name Romeu and contribute 2 for city Rome (because matching the end might be defined as a more relevant than matching some center part of the word). And so on.
Does this do what you want?
Function reorder accepts the array to sort, the field to sort on, and the query string. It constructs a regular expression that looks for the exact word, delineated by word boundaries; this is then used to sort the array on the basis of matches.
If you want to sort on a second field (eg 'name'), then you could run this function again on that field.
const people = [{
name: 'Romeu',
city: 'Napoli'
}, {
name: 'John',
city: 'Rome'
}, {
name: 'Romeu2',
city: 'Milan'
}]
const reorder = ({ arr, field, q }) => {
let r = new RegExp(`\\b${q}\\b`, 'u')
return arr.sort(({ [field]: aField }, { [field]: bField }) => {
if(r.test(aField) && !r.test(bField)) return -1
if(!r.test(aField) && r.test(bField)) return 1
return 0
})
}
console.log(reorder({ arr: people, field: 'city', q: 'Rome' }))

Process multiple request queries

I have an API that serves JSON data. Currently if you do api/weapons for example it gives you all the weapons available, api/weapons/weaponName gives information about that specific weapon. What I want to do is be able to api/weapons?type=sword&rarity=5 for example. I managed to pull of api/weapons?type=sword and api/weapons?rarity=5 on their own but not together.
Here's what I'm currently doing:
let filtered = [];
if (query.type) {
filtered = filtered.concat((await weapons).filter(w => formatName(w.weaponType) === formatName(query.type)));
}
if (query.rarity) {
filtered = filtered.concat((await weapons).filter(w => w.rarity == query.rarity));
}
if (!filtered.length) filtered = [await weapons]
res.status(HttpStatusCodes.ACCEPTED).send(filtered);
formatName is just a function that makes the string all lowercase and trims it and removes all spaces.
If we take api/weapons?type=sword&rarity=5
I think what's happening right now is:
It is getting all the weapons with the type "sword"
It is getting all the weapons with the rarity "5"
It is joining all the results together, so all the weapons with the type sword (regardless of rarity) and al the weapons with the rarity 5 (regardless of type).
I want it to filter weapons with ONLY that rarity AND ONLY that type. So only 5 rarity swords for example. What is the most beneficial way of handling this
I'd suggest retrieving "weapons" once and then running any filters on them without concatenating the results:
let filtered = [ ...(await weapons) ];
if (query.type) {
filtered = filtered.filter(w => w => formatName(w.weaponType) === formatName(query.type));
}
if (query.rarity) {
filtered = filtered.filter(w => w.rarity == query.rarity);
}
res.status(HttpStatusCodes.ACCEPTED).send(filtered);
Your current logic is testing whether one constraint OR another matches, what you actually need to do is to do an AND, which means you must perform the test in a single pass of filter.
I would slightly modify your code so that you compare all constraints that you're sending...you could further modify the logic below to accept a logical operator to test whether the rarity is >= or <= to a certain number for example.
const weapons = [{
type: 'sword',
name: 'swift blade of zek',
rarity: 5
},
{
type: 'mace',
name: 'hammer of kromzek kings',
rarity: 1
},
{
type: 'sword',
name: 'split blade of thunder',
rarity: 2
},
{
type: 'sword',
name: 'blade of carnage',
rarity: 5
},
]
const getWeapons = (query = {}) => {
let filtered = [];
let constraints = [];
// We could build this object dynamically but I just wanted
// to demonstrate it using your current approach
if (query.hasOwnProperty('type')) {
constraints.push({
name: 'type',
value: query.type
})
}
if (query.hasOwnProperty('rarity')) {
constraints.push({
name: 'rarity',
value: query.rarity
})
}
// Compare all of the conditions and only return weapons
// that match all of the conditions passed.
filtered = weapons.filter(w => {
let matches = 0
constraints.forEach(c => {
if (w[c.name] === c.value) {
matches += 1
}
})
// ensures we only return complete matches
return matches === constraints.length
});
return filtered
}
console.log(getWeapons({
type: 'sword',
rarity: 5
}))
Create an object which has the same property keys as the filters you want to use. Assign a function to each property where the evaluation for that specific filter is specified.
const filters = {
type: (weapon, type) => formatName(weapon.weaponType) === formatName(type),
rarity: (weapon, rarity) => weapon.rarity === rarity,
};
Then loop over the weapons with filter. Inside the filter loop, loop over the keys of the query variable with the every method. This method will return true or false based on if every evaluation is true or not.
In the every loop, use the keys of the query to select the filter from the filters list. Pass the weapon and the values of the query object to these filter functions and return result.
By doing this you can use one, two or no filters at all. And any new filters can be added in the filters object.
const filteredWeapons = weapons.filter((weapon) =>
Object.keys(query).every((filterKey) => {
if (!(filterKey in filters)) {
return false;
}
const filter = filters[filterKey]
const value = query[filterKey]
return filter(weapon, value);
})
);
res.status(HttpStatusCodes.ACCEPTED).send(filteredWeapons);

From single array convert to an array of object with keys coming from a JSON response -JAVASCRIPT-

I am receiving a json response from an API call. I need to store its keys, and create an array of an object. I am intending to this array of an object is created dynamically no matter the keys of the response.
I've already got the keys like this:
const json_getAllKeys = data => {
const keys = data.reduce((keys, obj) => (
keys.concat(Object.keys(obj).filter(key => (
keys.indexOf(key) === -1))
)
), [])
return keys
}
That returned an array (using a sample json):
['name','username', 'email']
But I am trying to use that array to create an array of object that looks like this one
[
{
name: "name",
username: "username",
email: "Email",
}
];
I've been trying mapping the array, but got multiple objects because of the loop, and I need a single one to make it work.
keys.map(i=>({i:i}))
[
{ i: 'id' },
{ i: 'name' },
{ i: 'username' },
{ i: 'email' }
]
Any hint would be useful!
Thanks in advance :D
What you're looking for is Object.fromEntries, which is ECMA2019, I believe, so available in Node >=14 and will be provided as a polyfill if you employ babel.
I can't quite discern what your reduce should produce, but given the sample input, I would write
const input = ['name','username', 'email'];
const result = Object.fromEntries(input.map(name => ([name, name])));
// result == { name: 'name', username: 'username', email: 'email' }
You're definitely on the right track. One thing to remember is the map function will return the SAME number of output as input. So in your example, an array of 3 returns an array of 3 items.
For this reason, map alone is not going to give you what you want. You may be able to map => reduce it. However, here is a way using forEach instead. This isn't a strictly functional programming style solution, but is pretty straight forward and depending on use case, probably good enough.
let keys = ['name','username', 'email'] //you have this array
const obj = {}; // empty object to hold result
keys.forEach(i => {
obj[i] = i; // set the object as you want
})
console.log(obj); // log out the mutated object
// { name: 'name', username: 'username', email: 'email' }

Fastest way to filter object by multiple properties

I have an array of objects that I want to filter for a string. So I want to check multiple properties if they contain the filter string (case insensitive).
Here's the array:
[{
id: "01234",
name: "My Object 01234",
short_name: "MO01234"
}, ...]
So all of the following filter strings should match that object: 0123, obj, mO01 etc.
Here's what I have right now:
const filterString = this.filterString.toLowerCase();
return myObjects.filter(
entry => {
return
entry.id.toLowerCase().indexOf(filterString) >= 0 ||
entry.name.toLowerCase().indexOf(filterString) >= 0 ||
entry.short_name.toLowerCase().indexOf(filterString) >= 0;
}
);
Can you think of a faster/cleaner way to do that?
I don't think that you can do it faster, but cleaner may be something like that
const filterString = this.filterString.toLowerCase();
return myObjects.filter((entry) => {
return Object.values(entry).some((value) => {
return value.toLowerCase().includes(filterString)
})
});
If you are allowed to put additional properties in your object, perhaps you could concatenate id, name and short_name (already in lowercase) into a single string and store it in the object as e.g. search_key; then you'd only have to check that.
{
id: "01234",
name: "My Object 01234",
short_name: "MO01234",
search_key: "01234:my object 01234:mo01234"
}
return myObjects.filter(
entry => entry.search_key.indexOf(filterString) >= 0
);
One thing you have to be mindful of in this case is to prevent unintended matches that may arise because e.g. the last few characters of id and the first few characters of name together produce a match. This is why I used a : delimiter here, assuming that's a character that can't appear in an ID or a short name.
let objects = [{
id: "01234",
name: "My Object 01234",
short_name: "MO01234"
},
{
id: "test",
name: "test",
short_name: "test"
}];
const filter = (collection, searchFor) => {
return collection.filter(obj => Object.values(obj).reduce((a,b) => a || String(b).toLowerCase().indexOf(searchFor.toLowerCase()) > -1, false))
}
console.log(filter(objects, "0123"));
console.log(filter(objects, "obj"));
console.log(filter(objects, "mO01"));
You could also extend this function to take a set of columns as parameter to filter on.
Another version using Regex:
const filterRegex = (collection, searchFor) => {
return collection.filter(obj => Object.values(obj).reduce((a,b) => a || String(b).match(new RegExp(searchFor, 'gi')), false))
}
console.log(filterRegex(objects, "0123"));
console.log(filterRegex(objects, "obj"));
console.log(filterRegex(objects, "mO01"));

Categories

Resources