Efficiently working with large data sets in Vue applications with Vuex - javascript

In my Vue application, I have Vuex store modules with large arrays of resource objects in their state. To easily access individual resources in those arrays, I make Vuex getter functions that map resources or lists of resources to various keys (e.g. 'id' or 'tags'). This leads to sluggish performance and a huge memory memory footprint. How do I get the same functionality and reactivity without so much duplicated data?
Store Module Example
export default {
state: () => ({
all: [
{ id: 1, tags: ['tag1', 'tag2'] },
...
],
...
}),
...
getters: {
byId: (state) => {
return state.all.reduce((map, item) => {
map[item.id] = item
return map
}, {})
},
byTag: (state) => {
return state.all.reduce((map, item, index) => {
for (let i = 0; i < item.tags.length; i++) {
map[item.tags[i]] = map[item.tags[i]] || []
map[item.tags[i]].push(item)
}
return map
}, {})
},
}
}
Component Example
export default {
...,
data () {
return {
itemId: 1
}
},
computed: {
item () {
return this.$store.getters['path/to/byId'][this.itemId]
},
relatedItems () {
return this.item && this.item.tags.length
? this.$store.getters['path/to/byTag'][this.item.tags[0]]
: []
}
}
}

To fix this problem, look to an old, standard practice in programming: indexing. Instead of storing a map with the full item values duplicated in the getter, you can store a map to the index of the item in state.all. Then, you can create a new getter that returns a function to access a single item. In my experience, the indexing getter functions always run faster than the old getter functions, and their output takes up a lot less space in memory (on average 80% less in my app).
New Store Module Example
export default {
state: () => ({
all: [
{ id: 1, tags: ['tag1', 'tag2'] },
...
],
...
}),
...
getters: {
indexById: (state) => {
return state.all.reduce((map, item, index) => {
// Store the `index` instead of the `item`
map[item.id] = index
return map
}, {})
},
byId: (state, getters) => (id) => {
return state.all[getters.indexById[id]]
},
indexByTags: (state) => {
return state.all.reduce((map, item, index) => {
for (let i = 0; i < item.tags.length; i++) {
map[item.tags[i]] = map[item.tags[i]] || []
// Again, store the `index` not the `item`
map[item.tags[i]].push(index)
}
return map
}, {})
},
byTag: (state, getters) => (tag) => {
return (getters.indexByTags[tag] || []).map(index => state.all[index])
}
}
}
New Component Example
export default {
...,
data () {
return {
itemId: 1
}
},
computed: {
item () {
return this.$store.getters['path/to/byId'](this.itemId)
},
relatedItems () {
return this.item && this.item.tags.length
? this.$store.getters['path/to/byTag'](this.item.tags[0])
: []
}
}
}
The change seems small, but it makes a huge difference in terms of performance and memory efficiency. It is still fully reactive, just as before, but you're no longer duplicating all of the resource objects in memory. In my implementation, I abstracted out the various indexing methodologies and index expansion methodologies to make the code very maintainable.
You can check out a full proof of concept on github, here: https://github.com/aidangarza/vuex-indexed-getters

While I agree with #aidangarza, I think your biggest issue is the reactivity. Specifically the computed property. This adds a lot of bloated logic and slow code that listens for everything - something you don't need.
Finding the related items will always lead you to looping through the whole list - there's no easy way around it. BUT it will be much faster if you call this by yourself.
What I mean is that computed properties are about something that is going to be computed. You are actually filtering your results. Put a watcher on your variables, and then call the getters by yourself. Something along the lines (semi-code):
watch: {
itemId() {
this.item = this.$store.getters['path/to/byId'][this.itemId]
}
}
You can test with item first and if it works better (which I believe it will) - add watcher for the more complex tags.
Good luck!

While only storing select fields is a good intermediate option (per #aidangarza), it's still not viable when you end up with really huge sets of data. E.g. actively working with 2 million records of "just 2 fields" will still eat your memory and ruin browser performance.
In general, when working with large (or unpredictable) data sets in Vue (using VueX), simply skip the get and commit mechanisms altogether. Keep using VueX to centralize your CRUD operations (via Actions), but do not try to "cache" the results, rather let each component cache what they need for as long as they're using it (e.g. the current working page of the results, some projection thereof, etc.).
In my experience VueX caching is intended for reasonably bounded data, or bounded subsets of data in the current usage context (i.e. for a currently logged in user). When you have data where you have no idea about its scale, then keep its access on an "as needed" basis by your Vue components via Actions only; no getters or mutations for those potentially huge data sets.

Related

Vue select all checkbox bad performance

I have a list objects displayed, each having a checkbox to select it. I also have a checkbox at the top to select the checkbox for every object, kinda like this (assume [ ] is a checkbox):
[ ] Select all
[ ] Object 1
[ ] Object 2
[ ] Object 3
The problem I have is when I have about 100 objects and clicking "Select all", the web page freezes for a good few seconds. There is also a search bar to filter the object, but I tested this by removing it and the performance is just as slow. Each object has a property selected so we know which object is selected. Below are some snippets of my code:
HTML:
<checkbox-input
id="documentSelectAll"
:model-value="operatingRows.length === 0 ? false : allSelected"
#update:modelValue="allSelectPressed" // Calls vuex function below
/>
---
<tr
v-for="(row, index) in operatingRows"
:key="index"
>
<document-table-row
:row-idx="index"
:row-data="row"
:fields="row.fields"
:hidden-column-indices="hiddenColumnIndices"
#toggleSelectedOnRow="toggleSelectedOnRow(row.id)" // Calls vuex to select individual row
/>
</tr>
Computed properties:
operatingRows() {
const showRow = (r) => {
// Some search code, irrelevant here
};
return this.sorted.filter(showRow); // 'sorted' comes from vuex
},
selectedRows() {
return this.operatingRows.filter((r) => r.selected);
},
numSelected() {
return this.selectedRows.reduce((prev, cur) => (cur.selected ? prev + 1 : prev), 0);
},
allSelected() {
return this.numSelected === this.operatingRows.length;
},
Vuex store:
getters: {
...storeStateGetters,
sorted: (state) => state.sorted,
},
---
mutations: {
...storeStateMutations,
SET_ALL_SELECTED_ON_SORTED(state, isSelected) {
state.sorted.map((r) => {
const rn = r;
rn.selected = isSelected;
return rn;
});
},
},
I think it might be to do with the fact that there are too many computed properties? I tried removing them individually (and the associated code) but the performance still seems bad, thus I am not able to pin point the issue to any particular piece of code, rather I think it's to do with the architecture as a whole.
Any help appreciated.
Turns out it was because of the mutation. To fix this, the mutation code was moved to an action which calls a mutation to set the state of the sorted array.
selectOnSorted: ({ commit, rootGetters }, isSelected) => {
const selectedSorted = rootGetters['documents/sorted'].map((doc) => ({
...doc,
selected: isSelected,
}));
commit('SET_SORTED', selectedSorted);
},

Iterate list of strings in mithril and create drop down

I tried searching a lot of internet but could not find answer to a simple question. I am very new to mithril (do not know why people chose mithril for project :( ). I want to iterate through a list of strings and use its value in drop down with a checkbox.
const DefaultListView = {
view(ctrl, args) {
const parentCtrl = args.parentCtrl;
const attr = args.attr;
const cssClass = args.cssClass;
const filterOptions = ['Pending', 'Paid', 'Rejected'];
// as of now, all are isMultipleSelection filter
const selectedValue =
parentCtrl.filterModel.getSelectedFilterValue(attr);
function isOptionSelected(value) {
return selectedValue.indexOf(value) > -1;
}
return m('.filter-dialog__default-attr-listing', {
class: cssClass
}, [
m('.attributes', {
onscroll: () => {
m.redraw(true);
}
}, [
filterOptions.list.map(filter => [
m('.dropdown-item', {
onclick() {
// Todo: Add click metrics.
// To be done at time of backend integration.
document.body.click();
}
}, [
m('input.form-check-input', {
type: 'checkbox',
checked: isOptionSelected(filter)
}),
m('.dropdown-text', 'Pending')
])
])
])
]);
}
};
Not sure. How to do it. This is what I have tried so far but no luck. Can someone help me this?
At the beginning of the view function you define an array:
const filterOptions = ['Pending', 'Paid', 'Rejected'];
But later on in the view code where you perform the list iteration, filterOptions is expected to be an object with a list property:
filterOptions.list.map(filter =>
That should be filterOptions.map(filter =>.
There may be other issues with your code but it's impossible to tell without seeing the containing component which passes down the args. You might find it more helpful to ask the Mithril chatroom, where myself and plenty of others are available to discuss & assist with tricky situations.

Update Non-Vue-Object when Vuex Store changes

Working on a music app in Vue & ToneJS, and I'm housing my ToneJS objects (ie: new Tone.Synth() etc.) on a module outside of Vue instances/components.
So my store looks something like this:
state: {
tracks: [
{
synthType: 'triangle',
gain: 0.5,
tune: ["beep", "boop", "bleep"]
},
{
synthType: 'sine',
gain: 0.9,
tune: ["boop", "boop", "beep"]
},
]
}
When the tracks object on the store changes in some way (for example, the user changes the gain on track 1), I then need to update the settings on the ToneJS synths.
How can I do this? I have been researching the Store.watch feature of Vuex, and I am frankly quite confused about both how to use it and if that is actually a good approach.
EDIT:
The "external" module (AudioManager, imported as AM) is currently just an object literal with two arrays, synths and gains. The parent component of the app runs this dispatch on created:
initializeAudioManager: context => {
context.state.tracks.forEach( (track, index) => {
let trackSynth = new Tone.Synth()
trackSynth.oscillator.type = track.synthType
AM.synths.push(trackSynth)
let trackGain = new Tone.Gain(track.gain)
AM.gains.push(trackGain)
})
AM.synths.forEach( (synth, i) => synth.connect(AM.gains[i]) );
AM.gains.forEach( (gain, i) => gain.toMaster() );
},

How to deal with relational data in Redux?

The app I'm creating has a lot of entities and relationships (database is relational). To get an idea, there're 25+ entities, with any type of relations between them (one-to-many, many-to-many).
The app is React + Redux based. For getting data from the Store, we're using Reselect library.
The problem I'm facing is when I try to get an entity with its relations from the Store.
In order to explain the problem better, I've created a simple demo app, that has similar architecture. I'll highlight the most important code base. In the end I'll include a snippet (fiddle) in order to play with it.
Demo app
Business logic
We have Books and Authors. One Book has one Author. One Author has many Books. As simple as possible.
const authors = [{
id: 1,
name: 'Jordan Enev',
books: [1]
}];
const books = [{
id: 1,
name: 'Book 1',
category: 'Programming',
authorId: 1
}];
Redux Store
Store is organized in flat structure, compliant with Redux best practices - Normalizing State Shape.
Here is the initial state for both Books and Authors Stores:
const initialState = {
// Keep entities, by id:
// { 1: { name: '' } }
byIds: {},
// Keep entities ids
allIds:[]
};
Components
The components are organized as Containers and Presentations.
<App /> component act as Container (gets all needed data):
const mapStateToProps = state => ({
books: getBooksSelector(state),
authors: getAuthorsSelector(state),
healthAuthors: getHealthAuthorsSelector(state),
healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
});
const mapDispatchToProps = {
addBooks, addAuthors
}
const App = connect(mapStateToProps, mapDispatchToProps)(View);
<View /> component is just for the demo. It pushes dummy data to the Store and renders all Presentation components as <Author />, <Book />.
Selectors
For the simple selectors, it looks straightforward:
/**
* Get Books Store entity
*/
const getBooks = ({books}) => books;
/**
* Get all Books
*/
const getBooksSelector = createSelector(getBooks,
(books => books.allIds.map(id => books.byIds[id]) ));
/**
* Get Authors Store entity
*/
const getAuthors = ({authors}) => authors;
/**
* Get all Authors
*/
const getAuthorsSelector = createSelector(getAuthors,
(authors => authors.allIds.map(id => authors.byIds[id]) ));
It gets messy, when you have a selector, that computes / queries relational data.
The demo app includes the following examples:
Getting all Authors, which have at least one Book in specific category.
Getting the same Authors, but together with their Books.
Here are the nasty selectors:
/**
* Get array of Authors ids,
* which have books in 'Health' category
*/
const getHealthAuthorsIdsSelector = createSelector([getAuthors, getBooks],
(authors, books) => (
authors.allIds.filter(id => {
const author = authors.byIds[id];
const filteredBooks = author.books.filter(id => (
books.byIds[id].category === 'Health'
));
return filteredBooks.length;
})
));
/**
* Get array of Authors,
* which have books in 'Health' category
*/
const getHealthAuthorsSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors],
(filteredIds, authors) => (
filteredIds.map(id => authors.byIds[id])
));
/**
* Get array of Authors, together with their Books,
* which have books in 'Health' category
*/
const getHealthAuthorsWithBooksSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors, getBooks],
(filteredIds, authors, books) => (
filteredIds.map(id => ({
...authors.byIds[id],
books: authors.byIds[id].books.map(id => books.byIds[id])
}))
));
Summing up
As you can see, computing / querying relational data in selectors gets too complicated.
Loading child relations (Author->Books).
Filtering by child entities (getHealthAuthorsWithBooksSelector()).
There will be too many selector parameters, if an entity has a lot of child relations. Checkout getHealthAuthorsWithBooksSelector() and imagine if the Author has a lot of more relations.
So how do you deal with relations in Redux?
It looks like a common use case, but surprisingly there aren't any good practices round.
*I checked redux-orm library and it looks promising, but its API is still unstable and I'm not sure is it production ready.
const { Component } = React
const { combineReducers, createStore } = Redux
const { connect, Provider } = ReactRedux
const { createSelector } = Reselect
/**
* Initial state for Books and Authors stores
*/
const initialState = {
byIds: {},
allIds:[]
}
/**
* Book Action creator and Reducer
*/
const addBooks = payload => ({
type: 'ADD_BOOKS',
payload
})
const booksReducer = (state = initialState, action) => {
switch (action.type) {
case 'ADD_BOOKS':
let byIds = {}
let allIds = []
action.payload.map(entity => {
byIds[entity.id] = entity
allIds.push(entity.id)
})
return { byIds, allIds }
default:
return state
}
}
/**
* Author Action creator and Reducer
*/
const addAuthors = payload => ({
type: 'ADD_AUTHORS',
payload
})
const authorsReducer = (state = initialState, action) => {
switch (action.type) {
case 'ADD_AUTHORS':
let byIds = {}
let allIds = []
action.payload.map(entity => {
byIds[entity.id] = entity
allIds.push(entity.id)
})
return { byIds, allIds }
default:
return state
}
}
/**
* Presentational components
*/
const Book = ({ book }) => <div>{`Name: ${book.name}`}</div>
const Author = ({ author }) => <div>{`Name: ${author.name}`}</div>
/**
* Container components
*/
class View extends Component {
componentWillMount () {
this.addBooks()
this.addAuthors()
}
/**
* Add dummy Books to the Store
*/
addBooks () {
const books = [{
id: 1,
name: 'Programming book',
category: 'Programming',
authorId: 1
}, {
id: 2,
name: 'Healthy book',
category: 'Health',
authorId: 2
}]
this.props.addBooks(books)
}
/**
* Add dummy Authors to the Store
*/
addAuthors () {
const authors = [{
id: 1,
name: 'Jordan Enev',
books: [1]
}, {
id: 2,
name: 'Nadezhda Serafimova',
books: [2]
}]
this.props.addAuthors(authors)
}
renderBooks () {
const { books } = this.props
return books.map(book => <div key={book.id}>
{`Name: ${book.name}`}
</div>)
}
renderAuthors () {
const { authors } = this.props
return authors.map(author => <Author author={author} key={author.id} />)
}
renderHealthAuthors () {
const { healthAuthors } = this.props
return healthAuthors.map(author => <Author author={author} key={author.id} />)
}
renderHealthAuthorsWithBooks () {
const { healthAuthorsWithBooks } = this.props
return healthAuthorsWithBooks.map(author => <div key={author.id}>
<Author author={author} />
Books:
{author.books.map(book => <Book book={book} key={book.id} />)}
</div>)
}
render () {
return <div>
<h1>Books:</h1> {this.renderBooks()}
<hr />
<h1>Authors:</h1> {this.renderAuthors()}
<hr />
<h2>Health Authors:</h2> {this.renderHealthAuthors()}
<hr />
<h2>Health Authors with loaded Books:</h2> {this.renderHealthAuthorsWithBooks()}
</div>
}
};
const mapStateToProps = state => ({
books: getBooksSelector(state),
authors: getAuthorsSelector(state),
healthAuthors: getHealthAuthorsSelector(state),
healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
})
const mapDispatchToProps = {
addBooks, addAuthors
}
const App = connect(mapStateToProps, mapDispatchToProps)(View)
/**
* Books selectors
*/
/**
* Get Books Store entity
*/
const getBooks = ({ books }) => books
/**
* Get all Books
*/
const getBooksSelector = createSelector(getBooks,
books => books.allIds.map(id => books.byIds[id]))
/**
* Authors selectors
*/
/**
* Get Authors Store entity
*/
const getAuthors = ({ authors }) => authors
/**
* Get all Authors
*/
const getAuthorsSelector = createSelector(getAuthors,
authors => authors.allIds.map(id => authors.byIds[id]))
/**
* Get array of Authors ids,
* which have books in 'Health' category
*/
const getHealthAuthorsIdsSelector = createSelector([getAuthors, getBooks],
(authors, books) => (
authors.allIds.filter(id => {
const author = authors.byIds[id]
const filteredBooks = author.books.filter(id => (
books.byIds[id].category === 'Health'
))
return filteredBooks.length
})
))
/**
* Get array of Authors,
* which have books in 'Health' category
*/
const getHealthAuthorsSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors],
(filteredIds, authors) => (
filteredIds.map(id => authors.byIds[id])
))
/**
* Get array of Authors, together with their Books,
* which have books in 'Health' category
*/
const getHealthAuthorsWithBooksSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors, getBooks],
(filteredIds, authors, books) => (
filteredIds.map(id => ({
...authors.byIds[id],
books: authors.byIds[id].books.map(id => books.byIds[id])
}))
))
// Combined Reducer
const reducers = combineReducers({
books: booksReducer,
authors: authorsReducer
})
// Store
const store = createStore(reducers)
const render = () => {
ReactDOM.render(<Provider store={store}>
<App />
</Provider>, document.getElementById('root'))
}
render()
<div id="root"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.1.0/react.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.1.0/react-dom.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-core/5.8.24/browser.js"></script>
<script src="https://npmcdn.com/reselect#3.0.1/dist/reselect.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/redux/3.3.1/redux.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-redux/4.4.6/react-redux.min.js"></script>
JSFiddle.
This reminds me of how I started one of my projects where the data was highly relational. You think too much still about the backend way of doing things, but you gotta start thinking of more of the JS way of doing things (a scary thought for some, to be sure).
1) Normalized Data in State
You've done a good job of normalizing your data, but really, it's only somewhat normalized. Why do I say that?
...
books: [1]
...
...
authorId: 1
...
You have the same conceptual data stored in two places. This can easily become out of sync. For example, let's say you receive new books from the server. If they all have authorId of 1, you also have to modify the book itself and add those ids to it! That's a lot of extra work that doesn't need to be done. And if it isn't done, the data will be out of sync.
One general rule of thumb with a redux style architecture is never store (in the state) what you can compute. That includes this relation, it is easily computed by authorId.
2) Denormalized Data in Selectors
We mentioned having normalized data in the state was not good. But denormalizing it in selectors is ok right? Well, it is. But the question is, is it needed? I did the same thing you are doing now, getting the selector to basically act like a backend ORM. "I just want to be able to call author.books and get all the books!" you may be thinking. It would be so easy to just be able to loop through author.books in your React component, and render each book, right?
But, do you really want to normalize every piece of data in your state? React doesn't need that. In fact, it will also increase your memory usage. Why is that?
Because now you will have two copies of the same author, for instance:
const authors = [{
id: 1,
name: 'Jordan Enev',
books: [1]
}];
and
const authors = [{
id: 1,
name: 'Jordan Enev',
books: [{
id: 1,
name: 'Book 1',
category: 'Programming',
authorId: 1
}]
}];
So getHealthAuthorsWithBooksSelector now creates a new object for each author, which will not be === to the one in the state.
This is not bad. But I would say it's not ideal. On top of the redundant (<-
keyword) memory usage, it's better to have one single authoritative reference to each entity in your store. Right now, there are two entities for each author that are the same conceptually, but your program views them as totally different objects.
So now when we look at your mapStateToProps:
const mapStateToProps = state => ({
books: getBooksSelector(state),
authors: getAuthorsSelector(state),
healthAuthors: getHealthAuthorsSelector(state),
healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
});
You are basically providing the component with 3-4 different copies of all the same data.
Thinking About Solutions
First, before we get to making new selectors and make it all fast and fancy, let's just make up a naive solution.
const mapStateToProps = state => ({
books: getBooksSelector(state),
authors: getAuthors(state),
});
Ahh, the only data this component really needs! The books, and the authors. Using the data therein, it can compute anything it needs.
Notice that I changed it from getAuthorsSelector to just getAuthors? This is because all the data we need for computing is in the books array, and we can just pull the authors by id one we have them!
Remember, we're not worrying about using selectors yet, let's just think about the problem in simple terms. So, inside the component, let's build an "index" of books by their author.
const { books, authors } = this.props;
const healthBooksByAuthor = books.reduce((indexedBooks, book) => {
if (book.category === 'Health') {
if (!(book.authorId in indexedBooks)) {
indexedBooks[book.authorId] = [];
}
indexedBooks[book.authorId].push(book);
}
return indexedBooks;
}, {});
And how do we use it?
const healthyAuthorIds = Object.keys(healthBooksByAuthor);
...
healthyAuthorIds.map(authorId => {
const author = authors.byIds[authorId];
return (<li>{ author.name }
<ul>
{ healthBooksByAuthor[authorId].map(book => <li>{ book.name }</li> }
</ul>
</li>);
})
...
Etc etc.
But but but you mentioned memory earlier, that's why we didn't denormalize stuff with getHealthAuthorsWithBooksSelector, right?
Correct! But in this case we aren't taking up memory with redundant information. In fact, every single entity, the books and the authors, are just reference to the original objects in the store! This means that the only new memory being taken up is by the container arrays/objects themselves, not by the actual items in them.
I've found this kind of solution ideal for many use cases. Of course, I don't keep it in the component like above, I extract it into a reusable function which creates selectors based on certain criteria.
Although, I'll admit I haven't had a problem with the same complexity as yours, in that you have to filter a specific entity, through another entity. Yikes! But still doable.
Let's extract our indexer function into a reusable function:
const indexList = fieldsBy => list => {
// so we don't have to create property keys inside the loop
const indexedBase = fieldsBy.reduce((obj, field) => {
obj[field] = {};
return obj;
}, {});
return list.reduce(
(indexedData, item) => {
fieldsBy.forEach((field) => {
const value = item[field];
if (!(value in indexedData[field])) {
indexedData[field][value] = [];
}
indexedData[field][value].push(item);
});
return indexedData;
},
indexedBase,
);
};
Now this looks like kind of a monstrosity. But we must make certain parts of our code complex, so we can make many more parts clean. Clean how?
const getBooksIndexed = createSelector([getBooksSelector], indexList(['category', 'authorId']));
const getBooksIndexedInCategory = category => createSelector([getBooksIndexed],
booksIndexedBy => {
return indexList(['authorId'])(booksIndexedBy.category[category])
});
// you can actually abstract this even more!
...
later that day
...
const mapStateToProps = state => ({
booksIndexedBy: getBooksIndexedInCategory('Health')(state),
authors: getAuthors(state),
});
...
const { booksIndexedBy, authors } = this.props;
const healthyAuthorIds = Object.keys(booksIndexedBy.authorId);
healthyAuthorIds.map(authorId => {
const author = authors.byIds[authorId];
return (<li>{ author.name }
<ul>
{ healthBooksByAuthor[authorId].map(book => <li>{ book.name }</li> }
</ul>
</li>);
})
...
This is not as easy to understand of course, because it relies primarily on composing these functions and selectors to build representations of data, instead of renormalizing it.
The point is: We're not looking to recreate copies of the state with normalized data. We're trying to *create indexed representations (read: references) of that state which are easily digested by components.
The indexing I've presented here is very reusable, but not without certain problems (I'll let everyone else figure those out). I don't expect you to use it, but I do expect you to learn this from it: rather than trying to coerce your selectors to give you backend-like, ORM-like nested versions of your data, use the inherent ability to link your data using the tools you already have: ids and object references.
These principles can even be applied to your current selectors. Rather than create a bunch of highly specialized selectors for every conceivable combination of data...
1) Create functions that create selectors for you based on certain parameters
2) Create functions that can be used as the resultFunc of many different selectors
Indexing isn't for everyone, I'll let others suggest other methods.
Author of the question's here!
One year later, now I'm going to summarize my experience and thoughts here.
I was looking into two possible approaches for handling the relational data:
1. Indexing
aaronofleonard, already gave us a great and very detailed answer here, where his main concept is as follows:
We're not looking to recreate copies of the state with normalized
data. We're trying to *create indexed representations (read:
references) of that state which are easily digested by components.
It perfectly fits to the examples, he mentions. But it's important to highlight that his examples create indexes only for one-to-many relations (one Book has many Authors). So I started to think about how this approach will fit to all my possible requirements:
Handing many-to-many cases. Example: One Book has many Authors, through BookStore.
Handling Deep filtration. Example: Get all the Books from the Healthy Category, where at least on Author is from a specific Country. Now just imagine if we have many more nested levels of entities.
Of course it's doable, but as you can see the things can get serious very soon.
If you feel comfortable with managing such complexity with Indexing, then make sure you have enough design time for creating your selectors and composing indexing utilities.
I continued searching for a solution, because creating such an Indexing utility looks totally out-of-scope for the project. It's more like creating a third-party library.
So I decided to give a try to Redux-ORM library.
2. Redux-ORM
A small, simple and immutable ORM to manage relational data in your Redux store.
Without being verbose, here's how I managed all the requirements, just using the library:
// Handing many-to-many case.
const getBooks = createSelector({ Book } => {
return Books.all().toModelArray()
.map( book => ({
book: book.ref,
authors: book.authors.toRefArray()
})
})
// Handling Deep filtration.
// Keep in mind here you can pass parameters, instead of hardcoding the filtration criteria.
const getFilteredBooks = createSelector({ Book } => {
return Books.all().toModelArray()
.filter( book => {
const authors = book.authors.toModelArray()
const hasAuthorInCountry = authors.filter(a => a.country.name === 'Bulgaria').length
return book.category.type === 'Health' && hasAuthorInCountry
})
.map( book => ({
book: book.ref,
authors: book.authors.toRefArray()
})
})
As you can see - the library handles all the relations for us and we can easily access all the child entities and perform complex computation.
Also using .ref we return the entity Store's reference, instead of creating a new object copy (you're worried about the memory).
So having this type of selectors my flow is as follows:
Container components fetches the data via API.
Selectors get only the needed slice of data.
Render the Presentation components.
However, nothing is perfect as it sounds as. Redux-ORM deals with relational operations as querying, filtering, etc. in a very easy of use way. Cool!
But when we talk about selectors reusability, composition, extending and so on - it's kind of tricky and awkward task. It's not a Redux-ORM problem, than to the reselect library itself and the way it works. Here we discussed the topic.
Conclusion (personal)
For simpler relational projects I would give a try to the Indexing approach.
Otherwise, I would stick with Redux-ORM, as I used it in the App, for which one I asked the question. There I have 70+ entities and still counting!
When you start "overloading" your selectors (like getHealthAuthorsSelector) with other named selectors (like getHealthAuthorsWithBooksSelector, ...) you might end up with something like getHealthAuthorsWithBooksWithRelatedBooksSelector etc etc.
That is not sustainable. I suggest you stick to the high level ones (ie getHealthAuthorsSelector) and use a mechanism so that their books and the related books of those books etc are always available.
You can use TypeScript and turn the author.books into a getter, or just work with covenience functions to get the books from the store whenever they are needed. With an action you can combine a get from store with a fetch from db to display (possibly) stale data directly and have Redux/React take care of the visual update once the data is retrieved from the database.
I hadn't heard of this Reselect but it seems like it might be a good way to have all sorts of filters in one place to avoid duplicating code in components.
Simple as they are, they are also easily testable. Business/Domain logic testing is usually a (very?) good idea, especially when you are not a domain expert yourself.
Also keep in mind that a joining of multiple entities into something new is useful from time to time, for example flattening entities so they can be bound easily to a grid control.
There is a library that solves relational selects: ngrx-entity-relationship.
Its similar demo is here on codesandbox
For the case with books and authors, it would be like that:
The next code we need to define once.
// first we need proper state selectors, because of custom field names
const bookState = stateKeys(getBooks, 'byIds', 'allIds');
const authorState = stateKeys(getAuthors, 'byIds', 'allIds');
// now let's define root and relationship selector factories
const book = rootEntitySelector(bookState);
const bookAuthor = relatedEntitySelector(
authorState,
'authorId',
'author'
);
// the same for authors
const author = rootEntitySelector(authorState);
const authorBooks = relatedEntitySelector(
bookState,
'books', // I would rename it to `booksId`
'booksEntities', // and would use here `books`
);
Now we can build selectors and reuse them if it is needed.
// now we can build a selector
const getBooksWithAuthors = rootEntities(
book(
bookAuthor(
authorBooks(), // if we want to go crazy
),
),
);
// and connect it
const mapStateToProps = state => ({
books: getBooksWithAuthors(state, [1, 2, 3]), // or a selector for ids
// ...
});
The result would be
this.props.books = [
{
id: 1,
name: 'Book 1',
category: 'Programming',
authorId: 1
author: {
id: 1,
name: 'Jordan Enev',
books: [1],
booksEntities: [
{
id: 1,
name: 'Book 1',
category: 'Programming',
authorId: 1,
},
],
},
},
];

Updating nested data in redux store

What's the best/correct way to update a nested array of data in a store using redux?
My store looks like this:
{
items:{
1: {
id: 1,
key: "value",
links: [
{
id: 10001
data: "some more stuff"
},
...
]
},
...
}
}
I have a pair of asynchronous actions that updates the complete items object but I have another pair of actions that I want to update a specific links array.
My reducer currently looks like this but I'm not sure if this is the correct approach:
switch (action.type) {
case RESOURCE_TYPE_LINK_ADD_SUCCESS:
// TODO: check whether the following is acceptable or should we create a new one?
state.items[action.resourceTypeId].isSourceOf.push(action.resourceTypeLink);
return Object.assign({}, state, {
items: state.items,
});
}
Jonny's answer is correct (never mutate the state given to you!) but I wanted to add another point to it. If all your objects have IDs, it's generally a bad idea to keep the state shape nested.
This:
{
items: {
1: {
id: 1,
links: [{
id: 10001
}]
}
}
}
is a shape that is hard to update.
It doesn't have to be this way! You can instead store it like this:
{
items: {
1: {
id: 1,
links: [10001]
}
},
links: {
10001: {
id: 10001
}
}
}
This is much easier for update because there is just one canonical copy of any entity. If you need to let user “edit a link”, there is just one place where it needs to be updated—and it's completely independent of items or anything other referring to links.
To get your API responses into such a shape, you can use normalizr. Once your entities inside the server actions are normalized, you can write a simple reducer that merges them into the current state:
import merge from 'lodash/object/merge';
function entities(state = { items: {}, links: {} }, action) {
if (action.response && action.response.entities) {
return merge({}, state, action.response.entities);
}
return state;
}
Please see Redux real-world example for a demo of such approach.
React's update() immutability helper is a convenient way to create an updated version of a plain old JavaScript object without mutating it.
You give it the source object to be updated and an object describing paths to the pieces which need to be updated and changes that need to be made.
e.g., if an action had id and link properties and you wanted to push the link to an array of links in an item keyed with the id:
var update = require('react/lib/update')
// ...
return update(state, {
items: {
[action.id]: {
links: {$push: action.link}
}
}
})
(Example uses an ES6 computed property name for action.id)

Categories

Resources