Chunk update using knex.js and MySQL - javascript

I am trying to learn how to use knex.js and as part of my learning I want to make some migration on my db, I want to update one of my columns.
I am trying to do this with chunks (I know it is not the best solution, it's only for learning purposes). I am using MySQL.
In my db I have a table that contains "email" column some of the rows end with ".com".
I am trying to update this column by omitting the ".com".
In my code, I select the relevant columns and I am using dash lib to chunk all my data.
My chunked data looks like this:
my db : (https://ibb.co/c8kKtcb)
chunk no.1
[
RowDataPacket {id: 1, email: dan#gmail.com},
RowDataPacket {id: 2, email: john#gmail},
RowDataPacket {id: 3, email: steph#gmail},
]
chunk no.2
[
RowDataPacket {id: 4, email: eric#gmail},
RowDataPacket {id: 5, email: tommy#gmail.com},
RowDataPacket {id: 6, email: bill#gmail.com},
]
chunk no.3
[
RowDataPacket {id: 7, email: loe#gmail},
RowDataPacket {id: 8, email: helen#gmail.com},
RowDataPacket {id: 9, email: niky#gmail.com},
]
…
This is my code, I tried to do the update but I figure that I am doing something wrong since its list end .update() method work with key value. Can I manipulate my data?
exports.up = async knex => {
const users = await knex("usersWeb")
.select("id", "email")
.where("email", "like", "%.com");
const userChunks = _.chunk(users, 3);
let chunckNumber = 1;
for (const chunk of userChunks) {
console.log("Chunck number: ", chunckNumber);
const k = await knex("usersWeb")
.whereIn(columnId, chunk.map(item => item.id))
.update("email": ); // this line :(
}
chunckNumber++;
}
exports.down = async knex => {};

I solve it after some time and I forgot to share my solution, so here is it:
exports.up = async knex => {
const users = await knex("usersWeb")
.select("id")
.where("email", "like", "%.com");
const userChunks = _.chunk(users, 3);
let chunckNumber = 1;
for (const chunk of userChunks) {
console.log("Chunck number: ", chunckNumber);
const tupleOfAllEmailIds = "(" + chunk.map(item => `${item.id}`).join(", ") + ")";
const query = `UPDATE "usersWeb"
SET "email" = REPLACE("email", ".com", "")
WHERE id IN ${tupleOfAllEmailIds}
`;
await knex.raw(query);
}
chunckNumber++;
}
exports.down = async knex => {};

Related

Sequelize: Special methods don't update the database? - Problem updating instance

I've spent a decent chunk of the day confused about the behaviour of one of my methods that should create an instance of a Model, Company, that has a Contact.
I feel like it should be simple, but it's just not outputting the contact field as I'd expect, even though it is saving it to the database correctly:
await sequelize.transaction(async (t) => {
const company = await Company.create({ name: req.body.companyName }, { transaction: t });
const contact = await Contact.create({position: req.body.position}, { transaction: t });
await company.addContact(contact, {transaction: t}); // This method does exist
await company.save(); // Thought this might help in case `addContact` didn't update the db
console.log(company.dataValues); // {id:5, name: 'test'}
console.log(contact.dataValues); // {id: 7, position: 'Legend'}
const test = await Company.findOne({where: { name: req.body.companyName }, transaction: t});
console.log(test.dataValues); // {id: 5, name: 'test'}, no contact property
console.log(company.contacts); // is this not posssible instead?
//...
I was expecting this as the output:
{
id: 5,
name: 'test',
contacts: [{ id: 7, position: 'legend' }]
}
Really can't understand why it's not working. Any help would be really appreciated at this point!
** Associations:
Company.hasMany(Contact);
Contact.belongsTo(Company);

Get consolidated difference between arrays - from the need to sync arrays in this situation

I'm not sure if I've described the problem best in the title but I'll elaborate here.
My overall goal is to keep lists in sync, I'm currently trying to get a specific output so that I can later correct list symmetry.
I've figured it out to this point:
Code:
let list2 = [
{user: 001, log: [1,2,3,4,5,6,7,8,9,10]},
{user: 002, log: [2,3,4,5,6,7,8,9, 44]},
{user: 003, log: [1,2,3,4,6,7,8]},
{user: 004, log: [1,2,3,4,5,6,7,8]}
];
for (let comparator = 0; comparator < list2.length; comparator++) {
for (let index = 0; index < list2.length; index++) {
if (comparator !== index) {
let currentDiff = list2[comparator].log.filter(x => !list2[index].log.includes(x));
console.log("User: " + list2[index].user + " needs " + currentDiff + " from user: " + list2[comparator].user);
}
}
}
Output:
User: 2 needs 1,10 from user: 1
User: 3 needs 5,9,10 from user: 1
User: 4 needs 9,10 from user: 1
User: 1 needs 44 from user: 2
User: 3 needs 5,9,44 from user: 2
User: 4 needs 9,44 from user: 2
User: 1 needs from user: 3
User: 2 needs 1 from user: 3
User: 4 needs from user: 3
User: 1 needs from user: 4
User: 2 needs 1 from user: 4
User: 3 needs 5 from user: 4
This outputs way too much data & I want to condense it
Desired output is all data is condensed so that none of the "needs" repeats, so if for example, user #2 can get 1 & 10 from user #1, then there is no need to output user #2 needs 1 from user #3... Do you follow me? I think this could be made simple but I just don't know any actions that can get this done easily.
Here is what I'm trying to achieve as output model (ideally):
[
{"user": 1,
"symmetriseLogs": [
{user: 2, missingLogs: [1, 10]},
{user: 3, missingLogs: [5, 9, 10]},
{user: 4, missingLogs: [9, 10]},
]},
{"user": 2,
"symmetriseLogs": [
{user: 1, missingLogs: [44]},
{user: 3, missingLogs: [44]},
{user: 4, missingLogs: [44]},
]},
]
Output should be whats required to symmetrize all the logs, so in the example output everything missing with user #1 & #2 can be gotten from each other, thus user #3 & #4 do not get output. Also user #2 only requires outputting 44 as that is the only log item 44 has that others are missing and which can't be gotten from user #1.
Bit of a looping logic nightmare, I would appreciate any assistance solving this. I've only gotten more confusing outputs trying to achieve this.
One approach is, before starting the iteration, you can make a mirrored structure that maps each user to the logs it has so far. Inside the loop, look up the user's existing logs to check to see which numbers need to be added.
It's not as pure as .map is intended to be, but it does the job, and I can't think of a nicer-looking approach.
const list2 = [
{user: 001, log: [1,2,3,4,5,6,7,8,9,10]},
{user: 002, log: [2,3,4,5,6,7,8,9, 44]},
{user: 003, log: [1,2,3,4,6,7,8]},
{user: 004, log: [1,2,3,4,5,6,7,8]}
];
const haveLogsByUserId = new Map(list2.map(({ user, log }) => [user, new Set(log)]));
const result = list2.map((source, i) => ({
user: source.user,
symmetriseLogs: list2
.filter((_, j) => i !== j)
.map(dest => {
const thisUserLogs = haveLogsByUserId.get(dest.user);
const missingLogs = source.log.filter(num => !thisUserLogs.has(num));
for (const num of missingLogs) thisUserLogs.add(num);
return { user: dest.user, missingLogs };
})
.filter(missingObj => missingObj.missingLogs.length)
}));
console.log(result);
Could you use something like the following code?
let list2 = [
{user: 001, log: [1,2,3,4,5,6,7,8,9,10]},
{user: 002, log: [2,3,4,5,6,7,8,9, 44]},
{user: 003, log: [1,2,3,4,6,7,8]},
{user: 004, log: [1,2,3,4,5,6,7,8]}
];
const result = []
for (let comparator = 0; comparator < list2.length; comparator++) {
const withAdd = [...list2[comparator].log]
result[comparator] = {user:list2[comparator].user,symmetriseLogs:[]}
for (let index = 0; index < list2.length; index++) {
if (comparator !== index) {
const currentDiff = list2[index].log.filter(x => !withAdd.includes(x));
if (currentDiff.length) {
console.log("User: " + list2[comparator].user + " needs " + currentDiff + " from user: " + list2[index].user);
result[comparator].symmetriseLogs.push({user:list2[index].user, missingLogs:currentDiff})
}
withAdd.push(...currentDiff)
}
}
}
console.log(result)
With your input I obtain the following output:
User: 1 needs 44 from user: 2
User: 2 needs 1,10 from user: 1
User: 3 needs 5,9,10 from user: 1
User: 3 needs 44 from user: 2
User: 4 needs 9,10 from user: 1
User: 4 needs 44 from user: 2
And:
[
{
"user": 1,
"symmetriseLogs": [
{
"user": 2,
"missingLogs": [
44
]
}
]
},
{
"user": 2,
"symmetriseLogs": [
{
"user": 1,
"missingLogs": [
1,
10
]
}
]
},
{
"user": 3,
"symmetriseLogs": [
{
"user": 1,
"missingLogs": [
5,
9,
10
]
},
{
"user": 2,
"missingLogs": [
44
]
}
]
},
{
"user": 4,
"symmetriseLogs": [
{
"user": 1,
"missingLogs": [
9,
10
]
},
{
"user": 2,
"missingLogs": [
44
]
}
]
}
]

JavaScript Get Indexes based from the Selected Array of Object Id

Today, I'm trying to get the list of javascript index based from the selected data id that I have.
I'm following this guide from https://buefy.org/documentation/table/#checkable where it needs something like this: checkedRows: [data[1], data[3]] to able to check the specific row in the table.
What I need to do is to check the table based from my web API response.
I have this sample response data.
response.data.checkedRows // value is [{id: 1234}, {id: 83412}]
and I have the sample data from the table.
const data = [{
id: 1234,
name: 'Jojo'
},{
id: 43221,
name: 'Jeff'
},{
id: 83412,
name: 'Kacey'
}]
So basically, I need to have something, dynamically, like this: checkedRows: [data[0], data[2]] because it matches the data from the response.data.checkedRows
So far, I tried using forEach
let selectedIndex = [];
response.data.checkedRows.forEach((d) => {
this.data.forEach((e) => {
if (d.id=== e.id) {
// need the result to be dynamic depending on the response.data.checkedRows
this.checkedRows = [data[0], data[2]]
}
});
});
I'm stuck here because I'm not sure how can I get the index that matches the selected checkedRows from response.
Any help?
Map the response checkedRows, and in the callback, .find the matching object in the array:
const checkedRows = [{id: 1234}, {id: 83412}];
const data = [{
id: 1234,
name: 'Jojo'
},{
id: 43221,
name: 'Jeff'
},{
id: 83412,
name: 'Kacey'
}];
const objs = checkedRows.map(({ id }) => (
data.find(obj => obj.id === id)
));
console.log(objs);
If there are a lot of elements, you can use a Set of the IDs to find instead to decrease the computational complexity:
const checkedRows = [{id: 1234}, {id: 83412}];
const data = [{
id: 1234,
name: 'Jojo'
},{
id: 43221,
name: 'Jeff'
},{
id: 83412,
name: 'Kacey'
}];
const ids = new Set(checkedRows.map(({ id }) => id));
const objs = data.filter(obj => ids.has(obj.id));
console.log(objs);

Replace each string in array with different object?

I have an array that looks like this:
[
{
"children": [
"5efa29058a3e8a3efc45c11a",
"5efa29158a3e8a3efc45c11b"
],
"_id": "5efa29b88a3e8a3efc45c11f",
"tabText": "foo",
"headerText": "foobar",
"__v": 0
},
{
"children": [
"5efa29228a3e8a3efc45c11c",
"5efa292c8a3e8a3efc45c11d",
"5efa29338a3e8a3efc45c11e"
],
"_id": "5efa29ea8a3e8a3efc45c120",
"tabText": "Foo2",
"headerText": "foobar2",
"__v": 0
}
]
For each "children" array on each object I need to take the childs id, look it up in the database, and replace the id string with an object containing the info from the database.
Mongoose Models:
Model1:
const { Schema, model } = require('mongoose')
const Tab = model('tab', new Schema({
tabText: String,
headerText: String,
children: Array
}))
module.exports = Tab
Model2:
const { Schema, model } = require('mongoose')
const Child = model('child', new Schema({
icon: String,
name: String,
description: String
}))
module.exports = Child
The models should be defined like this to form a relationship.
Model1:
const { Schema, model } = require('mongoose')
const Tab = model('Tab', new Schema({
tabText: String,
headerText: String,
children: [{ type: Schema.Types.ObjectId, ref: 'Child' }],
}))
module.exports = Tab
Model2:
const { Schema, model } = require('mongoose')
const Child = model('Child', new Schema({
icon: String,
name: String,
description: String
}))
module.exports = Child
Now the children will reference the ids of Child documents.
And when you want to populate them you can use:
await Tab.findOne().populate('children')
You just need to map over the array twice:
// First: map over each element in the array.
const newArray = array.map(arrElement => {
// Now arrElement is just one of our array elements.
// Then we can map over each element of the arrElement.children
// to get each id separately.
return arrElement.children.map(id => {
// Do stuff with the id...
// Make sure to return your new object from this function.
})
})
This is also a good case for Promise.all so you can run the queries in parallel
await Promise.all(
input.map(async (item) => {
return {
...item,
children: await Promise.all(
item.children.map(async (childId) => {
return await getItemFromDatabase(childId);
})
),
};
})
);
In order to replace each string in a Array. Here is a standard way of doing this.
'https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map'
const array1 = [1, 4, 9, 16];
// pass a function to map
const map1 = array1.map(x => x * 2);
console.log(map1);
// expected output: Array [2, 8, 18, 32]

Comparing two Arrays.map values dynamically

I have two arrays first array Deals contains deal list where it has dealId, buyerId, sellerId and, the second array Customer contains customers list where it has customerId, name. I just want to compare Deal.sellerId to Customer.customerId and want to show the Customer.name.
Don't confuse with the <DealDetail /> its a component which has attribute sellerName, for this component I want customer name value.
Deals Array:
this.state.updatedDeals.map(deal => <DealDetail key={deal.id} dealID={deal.id} sellerName={deal.sellerId} buyerName={deal.buyerId} />)
Customers Array:
this.state.updatedCustomers.map(customer => <li key={customer.id} > {customer.name} </li>)>
What I exactly wanting:
<DealDetail sellerName={deal.sellerId===customer.customerId ? customer.name} : "Unknown" />)
You can try something like below.
this.state.updatedDeals.map(
(deal) =>
{
const c = this.state.updatedCustomers.filter(customer=>deal.sellerId===customer.id);
<DealDetail
key={deal.id}
dealID={deal.id}
sellerName={c.length ==1 ? c[0].name:'unknown'}
buyerName={deal.buyerId}
/>
}
)
let updatedDeals = [
{dealId: 10, buyerId: 1, sellerId: 26},
{dealId: 11, buyerId: 1, sellerId: 26},
{dealId: 12, buyerId: 1, sellerId: 27},
];
let updatedCustomers = [
{customerId: 26, customerName: 'Isaac'},
{customerId: 28, customerName: 'Jane'}
];
let DealDisplay = [];
updatedCustomers.forEach(cust => {
if(updatedDeals.some(deal => deal.sellerId === cust.customerId)){
DealDisplay.push(cust);
}
});
Why not separate the logic in order to promote maintainability and readability. You can extract the records into a separate array as temporary variable. Then render the view using the variable
Fetched data from firestore:
updatedCustomers[doc.id] = doc.data();
And solved above problem with this:
this.state.updatedDeals.map(deal => (<DealDetail key={deal.id}
sellerName={this.state.updatedCustomers[deal.sellerId].name}
buyerName={this.state.updatedCustomers[deal.buyerId].name}
/>))

Categories

Resources