BookshelfJS belongsToMany doesn't return duplicates - javascript

I have two tables that are have the relationship belongsToMany. The pivot table also contains a column called state that could have 3 different values. Say my tables are Table1 and Table2.
There can be multiple entries for the same Table1-Table2 relation with different states.
I want to get all the pivot entries for Table1 including any multiple entries for the same Table2.
Unfortunately, the code
return this.belongsToMany(Table2, 'pivot_table1_table2').withPivot(['state'])
only returns the first entry for each Table2.
Help is appreciated.

That's how bookshelf works! It's part of the feature: remove the dupes. I found a workaround; explicitly select an attribute from the junction table that's unique. If you don't have one, create a model for the junction table. That's sadly the only solution then.
UPDATE:
Perhaps something like that. routes.code is unique in my case and it is part of the junction table. If this won't do the trick, create a model for the junction table and you're set (this is probably more preferred).
new Station().where({
id: req.params.id
}).fetch({
withRelated: [{
'routes': function(qb) {
qb.select('routes.id', 'routes.code');
}
}]
}).then(function(result) {
res.json(result.toJSON());
});

Related

how should i store lists of lists in MYSQL

I want to store this list of lists but I don't know how to store it in MySQL
list[x][y] the items in this list contains {li:[{x:x,y:y}] , pos:{x:y}}
list[x][y].li[z].x
list[x][y].li[z].y
list[x][y].pos.x
list[x][y].pos.y
for better undersing, please have a look at this
edited:
is this right? so this means i will only have 2 tables?
You should use a separate table with sub-lists that have a column parent_id, and then a third table with actual list items of low level lists.
The query for this will look like this:
SELECT li.x, li.y, sl.id
FROM li_items li
JOIN sub_lists sl on li.list_id = sl.id
JOIN lists l on sl.parent_id = l.id;
The process of converting the result rows depends on if you use some ORM or plain mysql client.
You could also store it as a JSON, as deleted answer has suggested, but than you wan't be able to query specific items without selecting and parsing all the lists. You could also use MySQL's JSON column, but In your case having separate tables seems to be better

ext js store, manipulation of data (linq type queries)

I am working on my first Ext JS project and have my source data downloaded from a web server held in a store. Now I want to have various grids all show some variation of that source data. For example, grid 1 would be the source data grouped by 1 particular column. Grid 2 would be grouped by a particular column, and where column X = 'variable'. So on and so on.
I am coming from a c# background working in WPF. So normally I would use observable collections, and run LINQ on the observable collection to form a new observable collection, then bind that new collection to my grid.
My question is, how can I run LINQ type queries on stores in EXT JS?
You can run LINQ type queries on stores in ExtJS: You can easily get a Collection from a store using the query or queryBy method.
But you can't work like this for your grids, because grids can't run solely on collections, they need a store each. You wouldn't want to use LINQ type queries for these stores.
If you are using ExtJS6, you can look into ChainedStore. You can bind each grid to a ChainedStore which has your data store as the source. Each ChainedStore can be filtered, sorted and grouped independently from the source and from other stores, by using the sort(), group() and addFilter() functions on the respective store.
If you are using an earlier version, you have to use multiple stores and copy all data from the main store manually. Please be aware that store.add() takes an array of records, not a collection. Instead of store.add(collection), use store.add(collection.getRange()). The getRange() function will return an array that contains all items in the collection.
To answer your question from the comment:
what if I need to do something as simple as create a new store, from store1, group everything by column1, and sum column2?
In ExtJS6, if you want to show the grouping and the sum in a grid, that would be something along:
var store2 = Ext.create('Ext.data.ChainedStore',{
source:store1, // use the data from the other store.
grouper:{
property:'someTestField' // grouping is done inside this store.
}
});
Ext.create('Ext.grid.Panel',{
store:store2,
features:[{
ftype:'groupingsummary' // a summary row for each group is provided by this
}],
columns:[{
xtype:'numbercolumn'
dataIndex: 'someTestField'
summaryType: 'sum' // this column should have the sum in the summary row.
}]
})
Untested and without warranty. If you want to do it without the grid, just calculate the sum, that would have to be done manually like this:
var sums = {};
store1.getGroups().each(function(group,i) {
sums[i] = 0;
group.each(function(record) {
sums[i] += record.get(propertyName);
});
});

Best practice when inserting a many-to-many relationship in a transaction using ON DUPLICATE KEY UPDATE with bookshelfjs (knex)

Alright so I'm working on a personal project with which you can make "to watch" lists. A user can look up a movie they've heard is good using a public API (TheMovieDB) and add it to their watchlist. Whenever a movie is added I save the data to a database. I use the following tables (excluding columns that aren't relevant to the question):
users
ID
movies
ID (PK)
tmdbID (Unique)
movies_users
ID (PK)
movie_ID (FK references ID in movies table)
user_ID (FK references ID in users table)
genres
ID (PK)
Name (Unique)
genres_movies
ID (PK)
genre_ID (FK references ID in genres table)
movie_ID (FK references ID in movies table)
I went with MariaDB because I wanted the ON DUPLICATE KEY UPDATE functionality, because I don't have any original data in this database, so for example when someone wants to add a movie to their database, but I already have this movie in the database, I just want to add a new row in the movies_users table. At the same time I also use that functionality when it comes to genres.
Language wise I use NodeJS with BookshelfJS.
Now I'm not in the slightest an expert on SQL so I'm not sure if my way of working is the most optimal one. Furthermore, I'm not that familiar with Promises. This is the code I use when adding a movie to the database: http://codeshare.io/BPWDV.
I use a transaction because I assume it's the safest way? I also had to use knex.raw (http://knexjs.org/#Raw-Queries) so I could use ON DUPLICATE KEY UPDATE. Basically this is what's going on:
I try to INSERT a movie into the movies table, depending on any existing rows
If the movie was already in the table I look for the ID of the movie so I can use it in the movies_users table. I don't have to look for the ID if the movie wasn't in there originally because I get back the ID of the inserted row.
I INSERT a row into the movies_users table
I map over an array of the movies' genres. For every genre I try the same as I did with the movie, it will either insert or do nothing depending on any possible duplicates. If it doesn't insert I look for the ID of the genre that's already in the table.
I then use the ID of the genre and the ID of the movie to insert a row into the genres_movies table
Personally, this looks like a horrible way to accomplish such a thing, so I feel I must be doing it wrong. The whole chaining of promises looks and feels awkward aswell. It works, but I assume there's a better way?
I'm planning on also using an actors table and an actors_movies tables, but I have no idea how I'd fit that in there as well (with the whole Promise.map thing).
So ultimately, my question is: if the code is right, how do I clean up the code OR if the code is not optimal at all, what would be a better way of accomplishing what I set out to do?

Generating a pivot table from a sql query

I have a project that consist of a function that generate a table based on some filter. Then, I found a problem that when some combination of some filter are return nothing so, when it rendered in browser as a table is less than number of combination.
To achieve it, I want to give "-" mark to the loss record that not saved in the table.
My first approach is check one-by-one the array with the filter and if the condition is match to 'loss rows', it will push array untill all missing record in result is filled with '-'. But, it'll dealing with time because, the last must be sorted again and then to be constructed into the wanted table.
Sample schema of main table "A":
amount_of_product, id_product, id_people, id_place.
The primary key columns are:
id_product, id_people, id_place.
The content of the main table is:
23456, book-a, 1, aa
5678, book-b, 1, cc
2587, book-b, 1, aa
The source query will be something like:
select * from A
where
id_product in ('book-a', 'book-b')
and id_people in ('1') and id_place in ('aa', 'bb', 'cc')
order by id_product, id_place
Then, we want to show in a table view (html) all combinations of all selected filters. 6 rows should be rendered, with empty cells shown as "-".
Sample result table:
Can I do it efficiently or are there any solution to this?
Thanks in advance.
Thanks for some advice above.
I've try to use crosstab function in postgresql to produce the pivot table but I can't produce for more complex table.
So, I use another approach to retrieve all datas based on combination row-coloumn gradually (example query from id_place 'aa', id_people '1', id-product 'aa' etc) and contruct each table head row and coloumn then contruct the data.
Regard

Most Efficient way of Filtering an Html Table?

I have an ajax function which call a servlet to get list of products from various webservices, the number of products can go up to 100,000. I need to show this list in a html table.
I am trying to provide users an interface to filter this list based on several criteria. Currently I am using a simple jQuery plugin to achieve this, but I found it to hog memory and time.
The Javascript that I use basically uses regex to search and filter rows matching the filtering criteria.
I was thinking of an alternate solution wherein I filter the JSON array returned by my servlet and bind the html table to it. Is there a way to achieve this, if there is, then is it more efficient than the regex approach.
Going through up to 100,000 items and checking if they meet your criteria is going to take a while, especially if the criteria might be complex (must be CONDO with 2 OR 3 bedrooms NOT in zip code 12345 and FIREPLACE but not JACUZZI).
Perhaps your servlet could cache the data for the 100,000 items and it could do the filtering, based on criteria posted by the user's browser. It could return, say, "items 1-50 of 12,456 selected from 100,000" and let the user page forward to the next 50 or so, and even select how many items to get back (25, 50, all).
If they select "all" before narrowing down the number very far, then a halfway observant user will expect it to take a while to load.
In other words, don't even TRY to manage the 100,000 items in the browser, let the server do it.
User enters filter and hits
search.
Ajax call to database, database has indexes on appropriate
columns and the database does the filtering.
Database returns result
Show result in table. (Probably want it to be paged to
only show 100-1000 rows at a time
because 100,000 rows in a table can
really slow down your browser.
Edit: Since you don't have a database, the best you're going to be able to do is run the regex over the JSON dataset and add results that match to the table. You'll want to save the JSON dataset in a variable in case they change the search. (I'm assuming that right now you're adding everything to the table and then using the jquery table plugin to filter it)
I'm assuming that by filtering you mean only displaying a subset of the data; and not sorting.
As you are populating the data into the table add classes to each row for everything in that row you want to filter by. e.g.:
<tr class="filter1 filter2 filter3">....
<tr class="filter1 filter3">....
<tr class="filter2">....
<tr class="filter3">....
Then when you want to apply a filter you can do something like:
$('TR:not(.filter1)').hide();
I agree with Berry that 100000 rows in the browser is bit of a stretch, but if there's anything that comes close to handling something of that magnitude then it's jOrder. http://github.com/danstocker/jorder
Create a jOrder table based on your JSON, and add the most necessary indexes. I mean the ones that you must at all cost filter by.
E.g. you have a "Name" field with people's names.
var table = jOrder(json)
.index('name', ['Name'], { sorted: true, ordered: true });
Then, for instance, this is how you select the records where the Name field starts with "John":
var filtered = table.where([{ Name: 'John' }], { mode: jOrder.startof, renumber: true });
Later, if you need paging in your table, just feed the table builder a filtered.slice(...).
If you're getting back xml, you could just use jQuery selection
$('.class', context) where context is your xml response.
From this selection, you could just write the xml to the page and use CSS to style it. That's where I'd start at first, at least. I'm doing something similar in one of my applications, but my dataset is smaller.
I don't know what you mean by "bind"? You can parse JSON and then use for loop (or $.each()) to populate ether straight HTML or by using grid plugin's insert/add

Categories

Resources