Fuzzy string match + amount match in node.js - javascript

Hi i need to order the data according to the fuzzy matching of 2 variables
Consider i have a string :"pet" and Amount 50
I have an object array as like below:
[{"des":"permanent","amount":100}, {"des":"petrol","amount":1000}]
I need an array as below
[{"des":"petrol","amount":100}, {"des":"permanent","amount":1000}] if suppose petrol is highest matching also its nearer to the value 50.
I used fuzzy npm package as follows:
var options = {
extract: function(el) { return el.description; }
};
var results = fuzzy.filter(description, res, options);
But here i can check for only string, but how can i do also for amount?? Thanks in advance

Your specification isn't entirely clear, but it seems like you want sorting (changing the order of elements based on some critieria) rather than filtering (removing elements based on some criteria).
There are two possibilities:
You want to sort by fuzzy score on the des string value, then break ties by the amount proximity.
You want to sort by some aggregate weight between the fuzzy des score and by the amount proximity.
Here's the first option:
const search = {des: "pet", amount: 10};
const data = [
{des: "pet", amount: 1000},
{des: "petrol", amount: 38},
{des: "petrol", amount: -17},
{des: "pets", amount: 9},
{des: "pet", amount: 999},
];
data.sort((a, b) => {
const {score: desA} = fuzzy.match(search.des, a.des);
const {score: desB} = fuzzy.match(search.des, b.des);
if (desA !== desB) {
return desB - desA;
}
return Math.abs(search.amount - a.amount) -
Math.abs(search.amount - b.amount);
});
console.log(data);
<script src="https://unpkg.com/fuzzy#0.1.3/lib/fuzzy.js"></script>
The second option is more involved, so I'll describe it on a high level. Use fuzzy.match to figure out the score for each des string, figure out how close amount is to the search target, normalize these to the same scale (probably 0 through 1), then weigh the scores by some amount to give a final score. Sort on the final score for each element.
Be careful: fuzzy.match().score returns Infinity for perfect matches, so you might want to convert that to 0 for certain operations.

Related

ML5: Error: You are passing a target array of shape 11342,1 while using a loss 'categorical_crossentropy'. 'categorical_crossentropy'expects targets

I am attempting to use the ML5 library for classification in a React app that I am building.
I am getting the following error in my browser
Error: You are passing a target array of shape 11342,1 while using a loss 'categorical_crossentropy'. 'categorical_crossentropy'expects targets to be binary matrices (1s and 0s) of shape [samples, classes].
In several Github issues where this error is raised, the explanation is that That error indicates you have just 1 type of objects in your dataset. You must have 2 or more different object classes in your dataset. That is the explanation in the links here and here.
I don't know what this means. I have 6 inputs and 2 outputs in my data. My input will look something like this
let inputs = {
male: 1,
female: 0,
dob: 641710800000,
// have more, but keeping it simple for this example...
}
and my output will look something like this
let output = {
job: 1 // or 0, if they have a job or not, for example. i.e., two possible outputs
}
However, I'm still getting the error. Can someone help me understand why and how to fix it?
Here's my code below:
people_arr = json.voters_arr;
keys = ["male", "female", "dob"];
let model_options = {
inputs: keys,
outputs: ["job"],
task: "classification"
};
let model = ml5.neuralNetwork(model_options);
for (let person of people_arr) {
let inputs = {
male: person.male,
female: person.female,
dob: person.dob
};
let output = {};
output.job = person.job; // either 0 or 1
model.addData(inputs, output);
}
model.normalizeData();
let train_options = { epochs: 100 }
model.train(train_options, whileTraining); // <-- error happening here
.then(() => {
console.log("pre classify");
return model.classify(new_person_arr);
})
.then((err, results) => {
if (err) { console.log("error") }
else {
let new_arr = results.splice(100);
console.log("results : ", new_arr);
setValues({...values, results: new_arr })
}
})
.catch((err) => { console.log("err : ", err) });
Categorical cross-entropy expects a one-hot vector as a label, not a single number. For instance, let's say there are three people: Michael, Jim, and Dwight. Michael and Jim have jobs, Dwight doesn't. Let's say that not having a job puts you in category 0, and having one puts you in category 1. Labels in this case would look like this:
[[0,1], # Michael's label
[0,1], # Jim's label
[1,0]] # Dwight's label
Michael and Jim are in category 1, so they have a 1 at index 1 and a 0 at all other indices. Dwight is in category 0, so he has a 1 at index 0 and a 0 at all other indices.
If you want to use a single number as a label (i.e. either a 0 or a 1), you should use sparse categorical cross-entropy instead. Sparse categorical cross-entropy takes an integer as a label for each sample and assumes that there are categories from 0 to the highest value integer it sees. So it would work perfectly well with what you already have.

Can Repeater Model execute JAVASCRIPT?

I have an XmlListModel in QML
XmlListModel {
id: model
source: "qrc:/Config/myConfig.xml"
query: "/levels/level"
XmlRole { name: "levName"; query: "#levName/string()" }
XmlRole { name: "from"; query: "from/number()" }
XmlRole { name: "to"; query: "to/number()" }
}
that reads this XML file
<levels parId = "3">
<level levelName = "level1">
<from>0</from>
<to>1</to
</level>
<level levelName = "level2">
<from>1</from>
<to>2</to>
</level>
</levels>
I also have a text element:
Text {
id: myText
x: 0; y:0
text: ""
}
I need to iterate through the XmlListModel in order to assign to myText.text the right level on the basis of what I found in myList.get(3).value, where myList is a ListModel.
Example:
if myList.get(3).value is between 0 (included) and 1 (excluded) I have to set myText.text = "level1", if it is between 1 (included) and 2 (excluded) I have to set myText.text = "level2", and so on...
Any suggestion?
Unfortunately you can't query your XmlListModel in O(1) like give me the value, where x is between role from and role to.
Good for you, you have an ordered list, so you can perform a binary search on your XmlListModel. The algorithm basically goes like this:
You first check whether your search value is by coincidence the one in the middle. If it is smaller, you search in the middle of the lower half, if it is larger, you search in the upper half... and so on.
With this you can find your value in O(log n) where n is the number of entries in your XmlListModel.
https://en.wikipedia.org/wiki/Binary_search_algorithm
If you have this implemented, to work on your model - either in JavaScript or in C++ or Python... you can have it like this:
Text {
text: binarySearch(model, myList.get(3).value).levName
}
When you implement this algorithm, make sure to deal with the gaps.

Determine the key of a song by its chords

How can I programmatically find the key of a song just by knowing the chord sequence of the song?
I asked some people how they would determine the key of a song and they all said they do it 'by ear' or by 'trial and error' and by telling if a chord resolves a song or not... For the average musician that is probably fine, but as a programmer that really isn't the answer that I was looking for.
So I started looking for music related libraries to see if anyone else has written an algorithm for that yet. But although I found a really big library called 'tonal' on GitHub: https://danigb.github.io/tonal/api/index.html I couldn't find a method that would accept an array of chords and return the key.
My language of choice will be JavaScript (NodeJs), but I'm not necessarily looking for a JavaScript answer. Pseudo code or an explanation that can be translated into code without too much trouble would be totally fine.
As some of you mentioned correctly, the key in a song can change. I'm not sure if a change in key could be detected reliably enough. So, for now let's just say, I'm looking for an algorithm that makes a good approximation on the key of a given chord sequence.
...
After looking into the circle of fifths, I think I found a pattern to find all chords that belong to each key. I wrote a function getChordsFromKey(key) for that. And by checking the chords of a chord sequence against every key, I can create an array containing probabilities of how likely it is that the key matches the given chord sequence: calculateKeyProbabilities(chordSequence). And then I added another function estimateKey(chordSequence), which takes the keys with the highest probability-score and then checks if the last chord of the chord sequence is one of them. If that is the case, it returns an array containing only that chord, otherwise it returns an array of all chords with the highest probability-score.
This does an OK job, but it still doesn't find the correct key for a lot of songs or returns multiple keys with equal probabililty. The main problem being chords like A5, Asus2, A+, A°, A7sus4, Am7b5, Aadd9, Adim, C/G etc. that are not in the circle of fifths. And the fact that for instance the key C contains the exact same chords as the key Am, and G the same as Em and so on...
Here is my code:
'use strict'
const normalizeMap = {
"Cb":"B", "Db":"C#", "Eb":"D#", "Fb":"E", "Gb":"F#", "Ab":"G#", "Bb":"A#", "E#":"F", "B#":"C",
"Cbm":"Bm","Dbm":"C#m","Eb":"D#m","Fbm":"Em","Gb":"F#m","Ab":"G#m","Bbm":"A#m","E#m":"Fm","B#m":"Cm"
}
const circleOfFifths = {
majors: ['C', 'G', 'D', 'A', 'E', 'B', 'F#', 'C#', 'G#','D#','A#','F'],
minors: ['Am','Em','Bm','F#m','C#m','G#m','D#m','A#m','Fm','Cm','Gm','Dm']
}
function estimateKey(chordSequence) {
let keyProbabilities = calculateKeyProbabilities(chordSequence)
let maxProbability = Math.max(...Object.keys(keyProbabilities).map(k=>keyProbabilities[k]))
let mostLikelyKeys = Object.keys(keyProbabilities).filter(k=>keyProbabilities[k]===maxProbability)
let lastChord = chordSequence[chordSequence.length-1]
if (mostLikelyKeys.includes(lastChord))
mostLikelyKeys = [lastChord]
return mostLikelyKeys
}
function calculateKeyProbabilities(chordSequence) {
const usedChords = [ ...new Set(chordSequence) ] // filter out duplicates
let keyProbabilities = []
const keyList = circleOfFifths.majors.concat(circleOfFifths.minors)
keyList.forEach(key=>{
const chords = getChordsFromKey(key)
let matchCount = 0
//usedChords.forEach(usedChord=>{
// if (chords.includes(usedChord))
// matchCount++
//})
chords.forEach(chord=>{
if (usedChords.includes(chord))
matchCount++
})
keyProbabilities[key] = matchCount / usedChords.length
})
return keyProbabilities
}
function getChordsFromKey(key) {
key = normalizeMap[key] || key
const keyPos = circleOfFifths.majors.includes(key) ? circleOfFifths.majors.indexOf(key) : circleOfFifths.minors.indexOf(key)
let chordPositions = [keyPos, keyPos-1, keyPos+1]
// since it's the CIRCLE of fifths we have to remap the positions if they are outside of the array
chordPositions = chordPositions.map(pos=>{
if (pos > 11)
return pos-12
else if (pos < 0)
return pos+12
else
return pos
})
let chords = []
chordPositions.forEach(pos=>{
chords.push(circleOfFifths.majors[pos])
chords.push(circleOfFifths.minors[pos])
})
return chords
}
// TEST
//console.log(getChordsFromKey('C'))
const chordSequence = ['Em','G','D','C','Em','G','D','Am','Em','G','D','C','Am','Bm','C','Am','Bm','C','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Am','Am','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em','Em','C','D','Em']
const key = estimateKey(chordSequence)
console.log('Example chord sequence:',JSON.stringify(chordSequence))
console.log('Estimated key:',JSON.stringify(key)) // Output: [ 'Em' ]
The chords in a song of a particular key are predominantly members of the key's scale. I imagine you could get a good approximation statistically (if there is enough data) by comparing the predominant accidentals in the chords listed to the key signatures of the keys.
See https://en.wikipedia.org/wiki/Circle_of_fifths
Of course, a song in any key can/will have accidentals not in the keys scale, so it would likely be a statistical approximation. But over several bars, if you add up the accidentals and filter out all but the ones that occur most often, you may be able to match to a key signature.
Addendum: as Jonas w correctly points out, you may be able to get the signature, but you won't likely be able to determine if it is a major or minor key.
Here's what I came up with. Still new with modern JS so apologies for messiness and bad use of map().
I looked around the internals of the tonal library, it has a function scales.detect(), but it was no good since it required every note present. Instead I used it as inspiration and flattened the progression into a simple note list and checked this in all transpositions as a subset of all the possible scales.
const _ = require('lodash');
const chord = require('tonal-chord');
const note = require('tonal-note');
const pcset = require('tonal-pcset');
const dictionary = require('tonal-dictionary');
const SCALES = require('tonal-scale/scales.json');
const dict = dictionary.dictionary(SCALES, function (str) { return str.split(' '); });
//dict is a dictionary of scales defined as intervals
//notes is a string of tonal notes eg 'c d eb'
//onlyMajorMinor if true restricts to the most common scales as the tonal dict has many rare ones
function keyDetect(dict, notes, onlyMajorMinor) {
//create an array of pairs of chromas (see tonal docs) and scale names
var chromaArray = dict.keys(false).map(function(e) { return [pcset.chroma(dict.get(e)), e]; });
//filter only Major/Minor if requested
if (onlyMajorMinor) { chromaArray = chromaArray.filter(function (e) { return e[1] === 'major' || e[1] === 'harmonic minor'; }); }
//sets is an array of pitch classes transposed into every possibility with equivalent intervals
var sets = pcset.modes(notes, false);
//this block, for each scale, checks if any of 'sets' is a subset of any scale
return chromaArray.reduce(function(acc, keyChroma) {
sets.map(function(set, i) {
if (pcset.isSubset(keyChroma[0], set)) {
//the midi bit is a bit of a hack, i couldnt find how to turn an int from 0-11 into the repective note name. so i used the midi number where 60 is middle c
//since the index corresponds to the transposition from 0-11 where c=0, it gives the tonic note of the key
acc.push(note.pc(note.fromMidi(60+i)) + ' ' + keyChroma[1]);
}
});
return acc;
}, []);
}
const p1 = [ chord.get('m','Bb'), chord.get('m', 'C'), chord.get('M', 'Eb') ];
const p2 = [ chord.get('M','F#'), chord.get('dim', 'B#'), chord.get('M', 'G#') ];
const p3 = [ chord.get('M','C'), chord.get('M','F') ];
const progressions = [ p1, p2, p3 ];
//turn the progression into a flat string of notes seperated by spaces
const notes = progressions.map(function(e) { return _.chain(e).flatten().uniq().value(); });
const possibleKeys = notes.map(function(e) { return keyDetect(dict, e, true); });
console.log(possibleKeys);
//[ [ 'Ab major' ], [ 'Db major' ], [ 'C major', 'F major' ] ]
Some drawbacks:
- doesn't give the enharmonic note you want necessarily. In p2, the more correct response is C# major, but this could be fixed by checking somehow with the original progression.
-‎ won't deal with 'decorations' to chords that are out of the key, which might occur in pop songs, eg. CMaj7 FMaj7 GMaj7 instead of C F G. Not sure how common this is, not too much I think.
Given an array of tones like this:
var tones = ["G","Fis","D"];
We can firstly generate a unique Set of tones:
tones = [...new Set(tones)];
Then we could check for the appearence of # and bs :
var sharps = ["C","G","D","A","E","H","Fis"][["Fis","Cis","Gis","Dis","Ais","Eis"].filter(tone=>tones.includes(tone)).length];
Then do the same with bs and get the result with:
var key = sharps === "C" ? bs:sharps;
However, you still dont know if its major or minor, and many componists do not care of the upper rules (and changed the key inbetween )...
One approach would be to find all the notes being played, and compare to the signature of different scales and see which is the best match.
Normally a scale signature is pretty unique. A natural minor scale will have the same notes as a major scale (that is true for all the modes), but generally when we say minor scale we mean the harmonic minor scale, which has a specific signature.
So comparing what notes are in the chords with your different scales should give you a good estimate. And you could refine by adding some weight to different notes (for example the ones that come up the most, or the first and last chords, the tonic of each chord, etc.)
This seems to handle most basic cases with some accuracy:
'use strict'
const allnotes = [
"C", "C#", "D", "Eb", "E", "F", "F#", "G", "Ab", "A", "Bb", "B"
]
// you define the scales you want to validate for, with name and intervals
const scales = [{
name: 'major',
int: [2, 4, 5, 7, 9, 11]
}, {
name: 'minor',
int: [2, 3, 5, 7, 8, 11]
}];
// you define which chord you accept. This is easily extensible,
// only limitation is you need to have a unique regexp, so
// there's not confusion.
const chordsDef = {
major: {
intervals: [4, 7],
reg: /^[A-G]$|[A-G](?=[#b])/
},
minor: {
intervals: [3, 7],
reg: /^[A-G][#b]?[m]/
},
dom7: {
intervals: [4, 7, 10],
reg: /^[A-G][#b]?[7]/
}
}
var notesArray = [];
// just a helper function to handle looping all notes array
function convertIndex(index) {
return index < 12 ? index : index - 12;
}
// here you find the type of chord from your
// chord string, based on each regexp signature
function getNotesFromChords(chordString) {
var curChord, noteIndex;
for (let chord in chordsDef) {
if (chordsDef[chord].reg.test(chordString)) {
var chordType = chordsDef[chord];
break;
}
}
noteIndex = allnotes.indexOf(chordString.match(/^[A-G][#b]?/)[0]);
addNotesFromChord(notesArray, noteIndex, chordType)
}
// then you add the notes from the chord to your array
// this is based on the interval signature of each chord.
// By adding definitions to chordsDef, you can handle as
// many chords as you want, as long as they have a unique regexp signature
function addNotesFromChord(arr, noteIndex, chordType) {
if (notesArray.indexOf(allnotes[convertIndex(noteIndex)]) == -1) {
notesArray.push(allnotes[convertIndex(noteIndex)])
}
chordType.intervals.forEach(function(int) {
if (notesArray.indexOf(allnotes[noteIndex + int]) == -1) {
notesArray.push(allnotes[convertIndex(noteIndex + int)])
}
});
}
// once your array is populated you check each scale
// and match the notes in your array to each,
// giving scores depending on the number of matches.
// This one doesn't penalize for notes in the array that are
// not in the scale, this could maybe improve a bit.
// Also there's no weight, no a note appearing only once
// will have the same weight as a note that is recurrent.
// This could easily be tweaked to get more accuracy.
function compareScalesAndNotes(notesArray) {
var bestGuess = [{
score: 0
}];
allnotes.forEach(function(note, i) {
scales.forEach(function(scale) {
var score = 0;
score += notesArray.indexOf(note) != -1 ? 1 : 0;
scale.int.forEach(function(noteInt) {
// console.log(allnotes[convertIndex(noteInt + i)], scale)
score += notesArray.indexOf(allnotes[convertIndex(noteInt + i)]) != -1 ? 1 : 0;
});
// you always keep the highest score (or scores)
if (bestGuess[0].score < score) {
bestGuess = [{
score: score,
key: note,
type: scale.name
}];
} else if (bestGuess[0].score == score) {
bestGuess.push({
score: score,
key: note,
type: scale.name
})
}
})
})
return bestGuess;
}
document.getElementById('showguess').addEventListener('click', function(e) {
notesArray = [];
var chords = document.getElementById('chodseq').value.replace(/ /g,'').replace(/["']/g,'').split(',');
chords.forEach(function(chord) {
getNotesFromChords(chord)
});
var guesses = compareScalesAndNotes(notesArray);
var alertText = "Probable key is:";
guesses.forEach(function(guess, i) {
alertText += (i > 0 ? " or " : " ") + guess.key + ' ' + guess.type;
});
alert(alertText)
})
<input type="text" id="chodseq" />
<button id="showguess">
Click to guess the key
</button>
For your example, it gives G major, that's because with a harmonic minor scale, there are no D major or Bm chords.
You can try easy ones: C, F, G or Eb, Fm, Gm
Or some with accidents: C, D7, G7 (this one will give you 2 guesses, because there's a real ambiguity, without giving more information, it could be both)
One with accidents but accurate: C, Dm, G, A
You might be able too keep an structure with keys for every "supported" scale, with as value an array with chords matching that scale.
Given a chord progression you can then start by making a shortlist of keys based on your structure.
With multiple matches you can try to make an educated guess. For example, add other "weight" to any scale that matches the root note.
You can use the spiral array, a 3D model for tonality created by Elaine Chew, which has a key detection algorithm.
Chuan, Ching-Hua, and Elaine Chew. "Polyphonic audio key finding using the spiral array CEG algorithm." Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on. IEEE, 2005.
My recent tension model, which is available in a .jar file here, also outputs the key (in addition to the tension measures) based on the spiral array. It can either take a musicXML file or text file as input that just takes a list of pitch names for each 'time window' in your piece.
Herremans D., Chew E.. 2016. Tension ribbons: Quantifying and visualising tonal tension. Second International Conference on Technologies for Music Notation and Representation (TENOR). 2:8-18.
If you're not opposed to switching languages, music21 (my library, disclaimer) in Python would do this:
from music21 import stream, harmony
chordSymbols = ['Cm', 'Dsus2', 'E-/C', 'G7', 'Fm', 'Cm']
s = stream.Stream()
for cs in chordSymbols:
s.append(harmony.ChordSymbol(cs))
s.analyze('key')
Returns: <music21.key.Key of c minor>
The system will know the difference between, say C# major and Db major. It has a full vocabulary of chord names so things like "Dsus2" won't confuse it. The only thing that might bite a newcomer is that flats are written with minus signs so "E-/C" instead of "Eb/C"
There is an online free tool (MazMazika Songs Chord Analyzer), which analyzes and detects the chords of any song very fast. You can process the song through file upload (MP3/WAV) or by pasting YouTube / SoundCloud links. After processing the file, you can play the song while seeing all the chords playing along in-real time, as well as a table containing all the chords, each chord is assigned to a time-position & a number ID, which you can click to go directly to the corresponding chord and it`s time-position.
https://www.mazmazika.com/chordanalyzer

Complex array ordering

Suppose I have the following array:
var articles = [
{
id: 7989546,
tags: [
"98#6",
"107#6",
"558234#7"
]
},
//...
]
Each element of this array represents (partially) some kind of content in our website. It has an id and is tagged with people (#6) and/or topics (#7).
The user is going to be provided a cookie containing the suggested or recommended tags, like this:
var suggestions = [
"46#6",
"107#6",
"48793#7"
]
Consider these tags like suggestions that will be shown to the end user, like "Maybe you are interesed in reading..."
The suggestions array is already ordered by tag prioritiy. This means, that the first tag is more relevant to the user than the second tag.
Now, what I want to do is to order my articles array in the same way, that is, by tag priority.
No filters should be applied as the articles array is guaranteed to have elements that have at least one tag from the suggestions array.
If I have an article with tags: [ "98#6", "107#6", 558234#7" ] and another one with tags: [ "46#6", "36987#7" ], I want the latter to be first, because the tag 46#6 has more priority than 107#6 in the suggestions array.
How can I achieve this kind of ordering (using two arrays)?
Note: jQuery solutions are gladly accepted.
jsFiddle Demo
Just make your own sort function and then use .indexOf in order to check for tag existence. The issue that you are going to have to decide to handle on your own is what makes the most sense for collisions. If an article is tagged with a priority 1 tag, but another article is tagged with 3 lower priority tags, who gets precedence? There is some logic involved there and in my suggested solution I simply just take a total of the priority by using the length of suggestions and summing the priorities. This can be adapted to give a different type of collision detection if you wish, but the approach will be basically the same.
Step 1: Create the compare function
This is going to order the array descending base on the result from tagCount. Which is to say that if tagCount returns a value of 6 for right, and a value of 3 for left, then 6 is ordered first.
var compareFn = function(left,right){
return tagCount(right.tags) - tagCount(left.tags);
};
Step 2: Create the tagCount "algorithm" for determining priority
This simply gives precedence to the earliest occurring match, but will also give some weight to multiple later occurring matches. It does this by taking the matched index subtracted from the length of the match array (suggestions). So if there are 5 suggestions, and the first suggestion is matched, then that is going to end up being a value of 5 (length=5 - index=0).
var tagCount = function(tags){
var count = 0;
for(var i = 0; i < tags.length; i++){
var weight = suggestions.indexOf(tags[i]);
if(weight > -1)
count += tags.length - weight;
}
return count;
}
Stack Snippet
var articles = [
{
id: 7989546,
tags: [
"107#6",
"558234#7"
]
},
{
id: 756,
tags: [
"98#6",
"558234#7"
]
},
{
id: 79876,
tags: [
"98#6",
"107#6",
"558234#7"
]
},
{
id: 7984576,
tags: [
"98#6",
"107#6",
"46#6"
]
}
];
var suggestions = [
"46#6",
"107#6",
"48793#7"
];
var compareFn = function(left,right){
return tagCount(right.tags) - tagCount(left.tags);
};
var tagCount = function(tags){
var count = 0;
for(var i = 0; i < tags.length; i++){
var weight = suggestions.indexOf(tags[i]);
if(weight > -1)
count += tags.length - weight;
}
return count;
}
var a = articles.sort(compareFn);
console.log(a);
document.querySelector("#d").innerHTML = JSON.stringify(a);
<div id="d"></div>
My approach: Sort by sum of relevance score
Give you have:
var articles = [
{
id: 7989546,
tags: [
"98#6",
"107#6",
"558234#7"
]
},
{
id: 8000000,
tags: [
"107#6",
"107#10",
"558234#7",
"5555#1"
]
},
{
id: 8333000,
tags: [
"46#6",
"107#6",
"666234#7",
"107#6"
]
}
];
var suggestions = [
"46#6",
"107#6",
"48793#7"
];
And you want to sort articles by tags whereas tag ranks are defined in suggestions. One simple approach would be:
Step 1) For each article, get index of each tag exists in the suggestion. If it doesn't exist, discard.
Given suggestions ["a","b","c"]
Article tags ["a","b","zzzz","yyyy"]
Will be mapped to index [0,1] (last two tags are discarded because they do not exist in suggestion list)
Step 2) Calculate degree of relevance. Higher-ranked tag (smaller index) yields greater value (see function degreeOfRelevance() below).
Step 3) Sum the total degree of relevance and sort by this value. Thus, the article which contains higher ranked tags (based on suggestions) will yield higher total score.
Quick example:
article <100> with tags: [a,b,c]
article <200> with tags: [b,c]
article <300> with tags: [c,d,e,f]
Given suggestions: [a,b,c]
The articles will be mapped to scores:
article <100> index : [0,1] ===> sum score: 3+2 = 5
article <200> index : [1] ===> sum score: 2
article <300> index : [2] ===> sum score: 1
Therefore, the article <100> is ranked the most relevant document when sorted by score
And below is the working code for this approach:
function suggest(articles, suggestions){
function degreeOfRelavance(t){
return suggestions.length - suggestions.indexOf(t);
}
function weight(tags){
return (tags.map(degreeOfRelavance)).reduce(function(a,b){
return a+b
},0);
}
function relatedTags(a){
return a.tags.filter(function(t){
return suggestions.indexOf(t)>=0
});
}
articles.sort(function(a,b){
return weight(relatedTags(a)) < weight(relatedTags(b))
});
return articles;
}
// See the output
console.log(suggest(articles,suggestions));

MongoDB: Get count of Array

Given these documents:
db.orders.insert( {OrderId:1, OrderItems: [{OrderItemId:1, Qty:1}, {OrderItemId:2, Qty:1} ]} );
db.orders.insert( {OrderId:2, OrderItems: [{OrderItemId:1, Qty:1}, {OrderItemId:2, Qty:2} ]} );
I'd like to get the count of all OrderItems where the Qty = 1 (expected result 3). This is how I think to write the query, but it returns 2 (1 for each Order document):
db.orders.find({"OrderItems.Qty":1}).count();
How can I query to find the count of all OrderItems where the Qty = 1?
Just to be clear for others reading this thread.
The OP's command db.orders.find({"OrderItems.Qty":1}).count(); basically counts the number of Documents where any order item has a quantity of 1.
However, the OP wants a count of all OrderItems where the quantity is one. The problem here is that we're not counting Documents. We're counting items within an array within a Document. Hence the need for javascript and some form of special reduce operation.
You could use JavaScript:
db.orders.group(
{
key: null,
cond: {'OrderItems.Qty':1},
initial: {count: 0},
reduce: function(el, running)
{
running.count += el.OrderItems.filter(function(el)
{
return el.Qty == 1;
}).length;
}
});
This should do it for you in the shell without JavaScript (so it'll be a lot quicker);
db.orders.aggregate([
{$unwind:'$OrderItems'},
{$match: {'OrderItems.Qty':1}},
{$group : {
_id : "Qty1",
sum: {$sum:1}
}}
]);
Although it's unfortunate your data is structured like that if this is a common query. Having to do an $unwind is relatively expensive. It's a shame your order items aren't laid out as separate documents tagged with the order ID instead of orderID documents containing arrays of order items...in other words, the reverse of what you have. That would be much easier and more efficient to process.
db.orders.aggregate([
{$unwind: '#OrderItems'},
{$match : {'OrderItems.Qty':1}},
{$group : { _id : null, 'countOfQty1':{$sum:'$OrderItems.Qty'} }}
]);

Categories

Resources