I have two tables holding game data for two different games. For the sake of simplicity, let's say they only share a column called Timestamp (in particular they have a different number of columns). I want to render a list holding information from both tables, simultaneously ordered by Timestamp.
What I'm currently doing works, but I'd take almost any bet that there is a much better way to do this. I'm mostly concerned about performance at some point (mobile app). This is a stub representing the structure – believe me, I know how horrible this looks right now. I just wanted to make it work first, now I'm looking for improvements. ;)
var readyA,
readyB = false;
var dataA,
dataB;
function doLoop () {
setTimeout(renderData, 100);
}
function renderData () {
if (!readyA || !readyB) {
doLoop();
return;
}
var dataAll = dataA.concat(dataB);
dataAll.sort(function (a,b) {
return a['Timestamp'] <= b['Timestamp'];
});
// pass data into a template depending on from which game it is and render it
// ...
}
// wait for both queries to finish
doLoop();
// select data from game A
myDatabaseClass.query('SELECT ... FROM GameA', function (results) {
dataA = new Array(results.rows.length);
for (var i=0; i<results.rows.length; i++) {
dataA[i] = results.rows.item(i);
}
readyA = true;
});
// select data from game B
myDatabaseClass.query('SELECT ... FROM GameB', function (results) {
dataB = new Array(results.rows.length);
for (var i=0; i<results.rows.length; i++) {
dataB[i] = results.rows.item(i);
}
readyB = true;
});
The question would now be if I can somehow simplify this by some kind of UNION or JOIN in the query. Obviously, the Timeout construction is horrible, but that will automatically collapse to a simple callback function if the querying can be done in one query (or at least one transaction – the database class can handle that).
Edit: I did found this ( Pull from two different tables and order ) but this whole NULL AS some_column feels dirty. Is there really no better alternative?
The result of a query always is a single table with a fixed number of columns, so all the SELECTs must have the same number of columns:
SELECT a1, a2, a3, Timestamp FROM GameA
UNION ALL
SELECT b1, b2, NULL, Timestamp FROM GameB
ORDER BY Timestamp
(UNION ALL is faster than UNION because it doesn't try to remove duplicates.)
Your code is pretty good. From the point of view of a SQL hacker like me you're doing the UNION and the ORDER BY on the client side. There's nothing wrong with that. You seem to be doing it almost right. Your "concat" is the client-side equivalent of UNION, and your "sort' is the equivalent of ORDER BY.
You say that the NULL as missing-column construction feels somehow dirty if you use server-side UNION operations. But, obviously to treat two different result sets as the same so you can sort them in order you have to make them conform to each other somehow. Your a['Timestamp'] <= b['Timestamp'] sort-ordering criterion in your sort function is also a scheme for conforming two result sets to each other. It may be lower-performance than using a UNION.
Don't be afraid of using NULL as missing-column to make two result sets in a UNION conform to each other. It's not dirty, and it's not expensive.
Do consider limiting your SELECT operation somehow, perhaps by a range of timestamps. That will allow your system to scale up, especially if you put an index on the column you use to limit the SELECT.
(By the way, your sort function has a mistake in it. sort functions need to return -1, 0, or +1 depending on whether the first item is less than, equal to, or greater than the second one. You're returning a true/false value. That doesn't work properly.)
(You're parallelizing the two queries to the same MySQL instance. That's clever, but probably in truth is a formula for overloading MySQL as your game scales up. Keep in mind that each user of your game has her own machine running Javascript but they all share your MySQL.)
Related
Basically, I know how to sort date in array, so I know that it is possible to compare it. But my question is more complex. I have two array and in some cases it can be that one cross each other, and in some it doesn't
My question is how to write dynamic analytic info under this chart in case it cross each other, and in case it does not.
useMemo(() => {
for(var i = 0; i<25; i++){
if(datasetone[I]>datasetwo[i]){return setText(true)}
if(datasetone[i]< datasetwo[i]) {return setText(false);
setRoznica(datasetone[24] - datasetwo[24]);
}
}
}, [datasetone, datasetwo]);
because I know that situation on chart that one is higher then another can happen just one, I want loop through all element of dataset and in case one is higher write statement.
and second quastion is WHEN it happen, you know .flirt or .find which index of dataset is moment of being higher then second one.
This is important because people in my country do not know how to analytic a chart so I can write statement if something happen to tell them.
I want to search a sheet for a value in a sheet, and then return the row and the column where the value was found.
I am well-versed in VBA and have used the .Find function to accomplish this very easily. However, after searching for the last 30 minutes online, I have been stunned to discover that is is so hard to find the code for this extremely simple function. I feel like I am in the twilight zone. Does Javascript really not have anything analogous to .Find? If so, why is this language used so much when VBA appears to be able to accomplish the same tasks in a much more simple manner? Please advise.
I assume you are calling something like mysheet.getDataRange().getValues(), which returns the contents of a google sheet as an array of arrays, eg, [[row1A, row1B], [row2A, row2B]].
With JS, you can get the index of a value in an array using indexOf, which returns the index of the found item or -1 if the item is not in the array. I don't think you can directly search across the two nested arrays. Instead, you could try iterating over the outer array and searching the inner array. Something like this:
// get data from google sheet, doing something like this
var data = mysheet.getDataRange().getValues()
// define our own function
function findItem(data) {
// loop over outer array
for (var i=0; i < data.length; i++) {
var row = data[i]; // get the inner array
var idx = row.indexOf(searchValue); // search for value
// if the value is found, return [row, column]
if (idx > -1) {
return [i, idx];
}
}
// call the function
var res = findItem(data);
var row = res[0];
var col = res[1];
You're comparing apples and oranges. JavaScript and VBA are different languages with different goals. VBA was built to allow it to integrate seamlessly with the MSSQLServer. JavaScript in its native form has no relational-database functionality at all. It's meant more for manipulating web pages through the DOM. (It can do more than that, but that's its primary function.) While VBA can do some of the things Javascript can do, it's a rather clunky way (IMHO) of doing so and narrowly focused on a rather specific set of problems that are tied to very specific software and hardware infrastructures. While that functionality might be nice to have in some cases, most of the JavaScript you see on the web today isn't interested in databases at all.
It's not clear to me what source of data you're trying to connect to but if you are specifically looking for a JavaScript data solution you might want to look into something like MongoDB, and the code libraries that have been developed specifically for that. And there are tons of other JS libraries that are relational-data or database-specific, and you can search places like npm for those. Or you can combine JavaScript with languages that inherently include database functionality, and PHP is an excellent example of that.
I've been trying to teach myself FRP (and bacon.js specifically) by diving in head first on a new project. I've gotten pretty far on my own but recently ran into a problem that I can't seem to fight my way through:
I have an interface with a set of clickable objects. When an object is clicked, detailed information for that object is loaded in a panel to the right.
What I need is the ability to select multiple, to accumulate those objects into an array and show a "bulk actions" panel when more than one is selected.
So far I have:
a SelectMultiple boolean property that represents the current UI mode
a CurrentObject stream that holds the currently selected object
I've gotten somewhat close with this:
var SelectedObjects = CurrentObject.filter(SelectMultiple).skipDuplicates().scan([], function(a,b){
return a.concat([b]);
};
There are a few problems:
The value of SelectedObjects represents the objects selected over
all time, it doesn't reset when SelectMultiple state changes.
The value of SelectObjects does not include the original
CurrentObject (of course because the scan accumulator seed is an
empty array, not the current value of CurrentObject).
The fact that I'm looking to use the current value of a property directly seems to be a hint that there's a fundamental issue here. I have a notion that the answer involves flapMapLatest and spawning a new stream every time SelectMultiple changes, funneling selected orders into this new stream and accumulating, but I can't quite work out what that should look like.
Of course there is an additional problem that skipDuplicates only skips consecutive duplicates. I can probably work this one out on my own but a solution that addresses that issue would be ideal.
Any suggestions would be greatly appreciated!
This might work (coffeescript):
var selectMultiple # Property[Boolean] - whether in multiselect mode
var selectedObject # Property[Object] - latest selected object
var selectedObjects = selectMultiple.flatMapLatest((multiple) ->
if !multiple
selectedObject.map((obj) -> [obj])
else
selectedObject.scan([], (xs, x) ->
xs.concat(x)
)
).toProperty()
On each value of selectMultiple flag we start a new stream that'll either just track the current single selection or start accumulating from the single selection, adding items as they're selected. It doesn't support de-selection by toggling, but that's straightforward to add into the scan part.
Ok I figured out a solution. I realized that I could use a dynamically-sized slidingWindow combinator. I found the basis for the answer in the Implementing Snake in Bacon.js tutorial.
I got an error when I tried adding directly to the Bacon prototype (as described in the tutorial) so I just made a function that takes the stream to observe and a boolean that determines if it should capture values:
slidingWindowWhile = function(sourceStream, toTakeOrNotToTake) {
return new Bacon.EventStream(function(sink){
var buf = [];
var take = false;
sourceStream.onValue(function(x){
if (! take) {
buf = [];
}
buf.push(x);
sink(new Bacon.Next(buf));
});
toTakeOrNotToTake.onValue(function(v){
take = v;
});
});
};
It still seems like there should be a way to do this without using local variables to track state but at least this solution is pretty well encapsulated.
I am using Knockout JS, as business requirements dictate that most, if not all logic is processed in the browser due to low bandwidth users. It's working out awesome so far except for one issue.
I am using a number of multiselect dropdown lists that all contain cascading logic. I have, say 8 lists that process hierarchical data and alter selectable options in child lists.
This is all good until I get to the bottom 2 lists which could potentially contain 3000 items depending on parent list selections (especially when 'select all' is clicked).
the problem is, in IE, I'm getting long running script warning messages, which I need to get rid of. Here's some code:
viewModel.BottomLevelList= ko.dependentObservable(function () {
if (this.ParentList().length === 0) { //nothing selected
return [];
}
var result = [];
var i = self.longMasterList.length;
var currentId = 0;
while (i--) {
//psuodo code:
//this.ParentList().Contains(loop-item) then
//put in return list based on some further logic
//else continue
}
return result;
}, viewModel);
I have tried using various setTimeout techniques from SO to break the large array up and return control momentarily to the browser, but with no success. The result is never returned and / or the observable seems to detach itself leaving an empty list in the UI.
If I need to use AJAX I will, but this is a very last resort and would prefer to keep it in the client.
So my question boils down to:
How can I stop long running script warnings as a result of processing large data sets (in the context of Knockout JS dependant observables and cascading lists)
Is there some idiomatic JavaScript technique I could / should be using in this scenario
Am I not seeing the wood for the trees here?!
Thanks muchly for any help
I would first suggest that you optimize your dependentObservable.
When you read any observable, Knockout registers a dependency to it in Dependency Manager. It contains pretty simple code like this:
function registerDependency(observable) {
if (ko.utils.arrayIndexOf(dependencies, observable)) {
dependencies.push(observable);
}
}
I can see in your pseudo-code that you are accessing this.ParentList() in the while loop. It means registerDependency will be called 3000 times and the dependencies array will be scanned 3000 times, which is bad for IE (since it has no built-in Array.indexOf method).
So my number one suggestion would be: Read all observables before loops.
If it doesn't help, I suggest that you proceed with setTimeout(). It is a bit tricky. Please check out this example: http://jsfiddle.net/romanych/4KGAv/3/
I have defined asyncObservable. You should pass an array with all dependencies of your dependentObservable. When ko.toJS is called, all observables are unwrapped. Then we call the passed callback function with arguments enumerated in the dependencies array. This function will be evaluated async.
I have wrapped this code into ko.dependentObservable to re-evaluate loader callback on any change of passed elements passed in dependencies
UPDATE:
My code was overcomplicated for this issue. throttle extender will do the trick. Please checkout this sample: http://jsfiddle.net/romanych/JNwhb/1/
Here's an example of what I need in sql:
SELECT name FROM employ WHERE name LIKE %bro%
How do I create view like that in CouchDB?
The simple answer is that CouchDB views aren't ideal for this.
The more complicated answer is that this type of query tends to be very inefficient in typical SQL engines too, and so if you grant that there will be tradeoffs with any solution then CouchDB actually has the benefit of letting you choose your tradeoff.
1. The SQL Ways
When you do SELECT ... WHERE name LIKE %bro%, all the SQL engines I'm familiar with must do what's called a "full table scan". This means the server reads every row in the relevant table, and brute force scans the field to see if it matches.
You can do this in CouchDB 2.x with a Mango query using the $regex operator. The query would look something like this for the basic case:
{"selector":{
"name": {
"$regex": "bro"
}
}}
There do not appear to be any options exposed for case-sensitivity, etc. but you could extend it to match only at the beginning/end or more complicated patterns. If you can also restrict your query via some other (indexable) field operator, that would likely help performance. As the documentation warns:
Regular expressions do not work with indexes, so they should not be used to filter large data sets. […]
You can do a full scan in CouchDB 1.x too, using a temporary view:
POST /some_database/_temp_view
{"map": "function (doc) { if (doc.name && doc.name.indexOf('bro') !== -1) emit(null); }"}
This will look through every single document in the database and give you a list of matching documents. You can tweak the map function to also match on a document type, or to emit with a certain key for ordering — emit(doc.timestamp) — or some data value useful to your purpose — emit(null, doc.name).
2. The "tons of disk space available" way
Depending on your source data size you could create an index that emits every possible "interior string" as its permanent (on-disk) view key. That is to say for a name like "Dobros" you would emit("dobros"); emit("obros"); emit("bros"); emit("ros"); emit("os"); emit("s");. Then for a term like '%bro%' you could query your view with startkey="bro"&endkey="bro\uFFFF" to get all occurrences of the lookup term. Your index will be approximately the size of your text content squared, but if you need to do an arbitrary "find in string" faster than the full DB scan above and have the space this might work. You'd be better served by a data structure designed for substring searching though.
Which brings us too...
3. The Full Text Search way
You could use a CouchDB plugin (couchdb-lucene now via Dreyfus/Clouseau for 2.x, ElasticSearch, SQLite's FTS) to generate an auxiliary text-oriented index into your documents.
Note that most full text search indexes don't naturally support arbitrary wildcard prefixes either, likely for similar reasons of space efficiency as we saw above. Usually full text search doesn't imply "brute force binary search", but "word search". YMMV though, take a look around at the options available in your full text engine.
If you don't really need to find "bro" anywhere in a field, you can implement basic "find a word starting with X" search with regular CouchDB views by just splitting on various locale-specific word separators and omitting these "words" as your view keys. This will be more efficient than above, scaling proportionally to the amount of data indexed.
Unfortunately, doing searches using LIKE %...% aren't really how CouchDB Views work, but you can accomplish a great deal of search capability by installing couchdb-lucene, it's a fulltext search engine that creates indexes on your database that you can do more sophisticated searches with.
The typical way to "search" a database for a given key, without any 3rd party tools, is to create a view that emits the value you are looking for as the key. In your example:
function (doc) {
emit(doc.name, doc);
}
This outputs a list of all the names in your database.
Now, you would "search" based on the first letters of your key. For example, if you are searching for names that start with "bro".
/db/_design/test/_view/names?startkey="bro"&endkey="brp"
Notice I took the last letter of the search parameter, and "incremented" the last letter in it. Again, if you want to perform searches, rather than aggregating statistics, you should use a fulltext search engine like lucene. (see above)
You can use regular expressions. As per this table you can write something like this to return any id that contains "SMS".
{
"selector": {
"_id": {
"$regex": "sms"
}
}
}
Basic regex you can use on that includes
"sms$" roughly to LIKE "%sms"
"^sms" roughly to LIKE "sms%"
You can read more on regular expressions here
i found a simple view code for my problem...
{"getavailableproduct": {
"map": "function(doc) { var prefix = doc['productid'].match(/[A-Za-z0-9]+/g); if(prefix) for(var pre in prefix) { emit(prefix[pre],null); } }"
}
}
from this view code if i split a key sentence into a key word...
and i can call
?key="[search_keyword]"
but i need more complex code because if i run this code i can only find word wich i type (ex: eat, food, etc)...
but if i want to type not a complete word (ex: ea from eat, or foo from food) that code does not work..
I know it is an old question, but: What about using a "list" function? You can have all your normal views, andthen add a "list" function to the design document to process the view's results:
{
"_id": "_design/...",
"views": {
"employees": "..."
},
"lists": {
"by_name": "..."
}
}
And the function attached to "by_name" function, should be something like:
function (head, req) {
provides('json', function() {
var filtered = [];
while (row = getRow()) {
// We can retrive all row information from the view
var key = row.key;
var value = row.value;
var doc = req.query.include_docs ? row.doc : {};
if (value.name.indexOf(req.query.name) == 0) {
if (req.query.include_docs) {
filtered.push({ key: key, value: value, doc: doc});
} else {
filtered.push({ key: key, value: value});
}
}
}
return toJSON({ total_rows: filtered.length, rows: filtered });
});
}
You can, of course, use regular expressions too. It's not a perfect solution, but it works to me.
You could emit your documents like normal. emit(doc.name, null); I would throw a toLowerCase() on that name to remove case sensitivity.
and then query the view with a slew of keys to see if something "like" the query shows up.
keys = differentVersions("bro"); // returns ["bro", "br", "bo", "ro", "cro", "dro", ..., "zro"]
$.couch("db").view("employeesByName", { keys: keys, success: dealWithIt } )
Some considerations
Obviously that array can get really big really fast depending on what differentVersions returns. You might hit a post data limit at some point or conceivably get slow lookups.
The results are only as good as differentVersions is at giving you guesses for what the person meant to spell. Obviously this function can be as simple or complex as you like. In this example I tried two strategies, a) removed a letter and pushed that, and b) replaced the letter at position n with all other letters. So if someone had been looking for "bro" but typed in "gro" or "bri" or even "bgro", differentVersions would have permuted that to "bro" at some point.
While not ideal, it's still pretty fast since a look up in Couch's b-trees is fast.
why cann't we just use indexOf() in view?