I have some sample data of vibrations analysis from sensors installed on electrical motors. The sampling is made once or, at most, 3 times per day. The values can be expressed in g, gE or mm/s.
I’m developing a personal algorithm in JavaScript to process some samples and perform a DFT. It’s a simple code that uses brute force to process my results. I compared the results (real and imaginary parts) from JavaScript and from MATLAB results and they matched perfectly.
However, my sampling rate is very slow. Because of this, I have a lot of questions which I couldn’t find the answers on my searches:
Is it possible to apply a DFT analysis on a slow sampling data as this?
How can I determine the correct frequency scale for the X axis? It’s complicated for me because I don’t have an explicit Fs (sampling rate) value.
In my case, would it be interesting to apply some window function like Hanning Window (suitable for vibrations analyses)?
JavaScriptCode:
//Signal is a pure one-dimensional of real data (vibration values)
const fft = (signal) => {
const pi2 = 6.2832 //pi const
let inputLength = signal.length;
let Xre = new Array(inputLength); //DFT real part
let Xim = new Array(inputLength); //DFT imaginary part
let P = new Array(inputLength); //Power of spectrum
let M = new Array(inputLength); //Magnitude of spectrum
let angle = 2 * Math.PI / inputLength;
//Hann Window
signal = signal.map((x, index) => {
return x * 0.5 * (1 - Math.cos((2 * Math.PI * index) / (inputLength - 1)));
});
for (let k = 0; k < inputLength; ++k) { // For each output element
Xre[k] = 0; Xim[k] = 0;
for (let n = 0; n < inputLength; ++n) { // For each input element
Xre[k] += signal[n] * Math.cos(angle * k * n);
Xim[k] -= signal[n] * Math.sin(angle * k * n);
}
P[k] = Math.pow(Xre[k], 2) + Math.pow(Xim[k], 2);
M[k] = Math.sqrt(Math.pow(Xre[k], 2) + Math.pow(Xim[k], 2));
}
return { Xre: Xre, Xim: Xim, P: P, M: M.slice(0, Math.round((inputLength / 2) + 1)) };
}
The first figure shows the charts results (time domain on the left side and frequency domain on the right side).
The second figure shows a little bit of my data samples:
Obs.: I'm sorry for the writing. I'm still a beginner English student.
The frequency doesn't matter. A frequency as low as 1/day is just as fine as any other frequency. But consider the Nyquist-Shannon theorem.
This is problematic. You need a fix sampling frequency for a DFT. You could do interpolation as preprocessing. But better would be to do the sampling at fix times.
Related
Im trying to recreate Tradingviews pine script RSI code into Javascript code. But having a hard time figuring out how it works. I made the basic RSI using a normal moving average calculation. But the pine script uses exponential weighted moving average. And there documentation is really hard to follow to me. This is the pine script.
//#version=4
study(title="Relative Strength Index", shorttitle="RSI", format=format.price, precision=2, resolution="")
len = input(14, minval=1, title="Length")
src = input(close, "Source", type = input.source)
up = rma(max(change(src), 0), len)
down = rma(-min(change(src), 0), len)
rsi = down == 0 ? 100 : up == 0 ? 0 : 100 - (100 / (1 + up / down))
plot(rsi, "RSI", color=#7E57C2)
band1 = hline(70, "Upper Band", color=#787B86)
bandm = hline(50, "Middle Band", color=color.new(#787B86, 50))
band0 = hline(30, "Lower Band", color=#787B86)
fill(band1, band0, color=color.rgb(126, 87, 194, 90), title="Background")
This is what I oould make of it in Javascript:
// Period = 200
// Close variable is 200 closed values. Where [0] in array = oldest, [199] in array = newest value.
/**
* Relative strength index. Based on closed periods.
*
* #param {Array} close
* #param {Integer} period
* #returns
*/
function calculateRSI(close, period) {
// Only calculate if it is worth it. First {period - 1} amount of calculations aren't correct anyway.
if (close.length < period) {
return 50;
}
let averageGain = 0;
let averageLoss = 0;
const alpha = 1 / period;
// Exponential weighted moving average.
for (let i = 1; i < period; i++)
{
let change = close[i] - close[i - 1];
if (change >= 0) {
averageGain = alpha * change + (1 - alpha) * averageGain;
} else {
averageLoss = alpha * -change + (1 - alpha) * averageLoss;
}
}
// Tried this too, but seems to not really matter.
// To get an actual average.
// averageGain /= period;
// averageLoss /= period;
// Calculate relative strength index. Where it can only be between 0 and 100.
var rsi = 100 - (100 / (1 + (averageGain / averageLoss)));
return rsi;
}
The results this function gives on my chart is not too bad, but it just isn't the same as I have it in Tradingview. I belive im missing something that the pine script does and I don't.
Things I dont understand of the pine script:
When does it do a for loop? I don't see it in there functions. If they don't, how do they calculate the average for a period of longer than 2? You have to loop for that right?
How does the rma function work? This is their docs.
I might have too many questions on this, but I think if you show a somewhat working example in Javascript of the RSI calculation like they do. Then I can probably make sense of it.
Is my calculation in Javascript correct to the one in the pine script?
I am trying to create a simple feed-forward neural network in JavaScript using a tutorial found here. I believe that I followed the tutorial correctly, as when I trained it with an input matrix of [[0,0,1],[0,1,1],[1,0,1],[1,1,1]] and a solution matrix of [[0],[1],[1],[0]] the network performed as expected. However, when I tried to train the network using the MNIST handwritten number database and passing in larger matrices, all of the elements in the output array approached zero. I suspect that this has to do with the dot product of the input array and the weights returning an array filled with large numbers, but my attempts to scale down these number have caused the outputs to approach 1. Would anybody be able to figure out what is going wrong? My neural network has only one hidden layer with 300 neurons. The code snippet below shows the methods for my neural network, because I believe that that is where I am going wrong, but if you want to see my entire messy, undocumented program, it can be found here. I am unfamiliar with the math library that I am using, which means that I made some of my own methods to go along with the math methods.
multMatrices(a, b) returns the product of matrices a and b.
math.multiply(a, b) returns the dot product of the two matrices.
math.add(a, b) and math.subtract(a, b) perform matrix addition and subtraction, and
transposeMatrix(a) returns the transpose of matrix a.
setAll(a, b) performs an operation on every element of matrix a, be it plugging the element into the sigmoid function (1 / (1 + a^-e)) in the case of "sigmoid" or the sigmoid derivative function (a * (1-a)) in the case of "sigmoidDerivitive", or setting it equal to a random value between 0 and 0.05 in the case of "randomlow".
I found that setting weights to a value between 0 and 1 kept the loss at 0.9, so I set them now using "randomlow".
function NeuralNetwork(x, y){
//Initializing the neural network
this.input = x;
this.y = y;
this.sizes = [this.input._size[1], 300, this.y._size[1]];
this.layers = this.sizes.length - 1;
this.lyrs = [this.input];
this.weights = [];
this.dweights = [];
for(var i = 0; i < this.layers; i ++){
this.weights.push(new math.matrix());
this.weights[i].resize([this.sizes[i], this.sizes[i + 1]]);
this.weights[i] = setAll(this.weights[i], "randomlow");
}
this.output = new math.matrix();
this.output.resize(this.y._size);
};
NeuralNetwork.prototype.set = function(x, y){
//I train the network by looping through values from the database and passing them into this function
this.input = x;
this.lyrs = [this.input];
this.y = y;
};
NeuralNetwork.prototype.feedforward = function(){
//Looping through the layers and multiplying them by their respective weights
for(var i = 0; i < this.weights.length; i ++){
this.lyrs[i + 1] = math.multiply(this.lyrs[i], this.weights[i]);
this.lyrs[i + 1] = setAll(this.lyrs[i + 1], "sigmoid");
}
this.output = this.lyrs[this.lyrs.length - 1];
};
NeuralNetwork.prototype.backpropogate = function(){
//Backpropogating the network. I don't fully understand this part
this.antis = [
function(a, b, c){
return(
math.multiply(transposeMatrix(a[0]), multMatrices(math.multiply(multMatrices(math.multiply(math.subtract(b.y, b.output), 2), setAll(b.output, "sigmoidDerivitive")), transposeMatrix(c)), setAll(a[1], "sigmoidDerivitive")))
);
},
function(a, b, c){
return(
math.multiply(transposeMatrix(a[0]), multMatrices(math.multiply(math.subtract(b.y, b.output), 2), setAll(b.output, "sigmoidDerivitive")))
);
}];
this.input = [];
this.weightInput = 0;
for(var i = this.weights.length - 1; i >= 0; --i){
this.input.unshift(this.lyrs[i]);
this.weightInput = (i === this.weights.length - 1 ? 0 : this.weights[i + 1]);
this.dweights[i] = this.antis[i](this.input, this, this.weightInput);
}
for(var i = 0; i < this.dweights.length; i ++){
this.weights[i] = math.add(this.weights[i], this.dweights[i]);
}
};
As always, I appreciate any time spent trying to solve my problem. If my code is unintelligible, don't bother with it. JavaScript probably isn't the best language for this purpose, but I didn't want to follow a tutorial with the same language.
EDIT: This is a potential duplicate of this post, which was answered. If anybody is facing this problem, they should see if the answer over there is of help. As of now I have not tested it with my program.
I need to convert z-scores to percentile. I found reference to a function in the jStat library that I could use (jstat.ztest), but the jStat documentation seems to be ahead of the available library because there is no such function in the currently available version of the library.
I think there is a more recent version of the library on GitHub, which may include the ztest function, but I am a linux novice and could not figure out how to build the library from the instructions. I spent most of a day learning about git bash and cygwin trying to build the library; I finally decided I'd be better off asking here.
So, could anyone point me toward a javascript function that would do what I need?
Alternatively, could anyone point me toward a built version of the jStat library with ztest function included?
I found this in a forum online and it works like a charm.
function GetZPercent(z)
{
//z == number of standard deviations from the mean
//if z is greater than 6.5 standard deviations from the mean
//the number of significant digits will be outside of a reasonable
//range
if ( z < -6.5)
return 0.0;
if( z > 6.5)
return 1.0;
var factK = 1;
var sum = 0;
var term = 1;
var k = 0;
var loopStop = Math.exp(-23);
while(Math.abs(term) > loopStop)
{
term = .3989422804 * Math.pow(-1,k) * Math.pow(z,k) / (2 * k + 1) / Math.pow(2,k) * Math.pow(z,k+1) / factK;
sum += term;
k++;
factK *= k;
}
sum += 0.5;
return sum;
}
And I don't need to include a large library just for the one function.
Just editing the code from Paul's answer for a two-sided t-test
function GetZPercent(z)
{
//z == number of standard deviations from the mean
//if z is greater than 6.5 standard deviations from the mean
//the number of significant digits will be outside of a reasonable
//range
if ( z < -6.5)
return 0.0;
if( z > 6.5)
return 1.0;
if (z > 0) { z = -z;}
var factK = 1;
var sum = 0;
var term = 1;
var k = 0;
var loopStop = Math.exp(-23);
while(Math.abs(term) > loopStop)
{
term = .3989422804 * Math.pow(-1,k) * Math.pow(z,k) / (2 * k + 1) / Math.pow(2,k) * Math.pow(z,k+1) / factK;
sum += term;
k++;
factK *= k;
}
sum += 0.5;
return (2*sum);
}
This seems like such a simple ask but I had a hard time tracking down a library that does this instead of copying some random code snippet. Best I can tell this will calculate z-score from a percentage using the simple-statistics library.
I took their documentation about cumulativestdnormalprobability and backed into the following algorithm. Feels like there should be an easier way but who knows.
https://simplestatistics.org/docs/#cumulativestdnormalprobability
const z_score = inverseErrorFunction((percentile_value - 0.5) / 0.5) * Math.sqrt(2);
As already correctly stated by Shane, the equation is an implementation of the Taylor Expansion of the normal cdf. The sum value iterates above and below the "real" value with increasing precision. If the value is close to 1 or 0 there is a very low, but existing, probability that sum will be >1 or <0, because of the (relatively) early break by loopstop.
The deviation is further strengthened by rounding 1/Math.sqrt(2*Math.Pi) to 0.3989422804 and the precision issues of javascript float numbers. Additionally, the provided solution will not work for z-scores >7 or <-7
I updated the code to be more accurate using the decimal.js npm library and to directly return the p-value:
function GetpValueFromZ(_z, type = "twosided")
{
if(_z < -14)
{
_z = -14
}
else if(_z > 14)
{
_z = 14
}
Decimal.set({precision: 100});
let z = new Decimal(_z);
var sum = new Decimal(0);
var term = new Decimal(1);
var k = new Decimal(0);
var loopstop = new Decimal("10E-50");
var minusone = new Decimal(-1);
var two = new Decimal(2);
let pi = new Decimal("3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117067982148086513282306647")
while(term.abs().greaterThan(loopstop))
{
term = new Decimal(1)
for (let i = 1; i <= k; i++) {
term = term.times(z).times(z.dividedBy(two.times(i)))
}
term = term.times(minusone.toPower(k)).dividedBy(k.times(2).plus(1))
sum = sum.plus(term);
k = k.plus(1);
}
sum = sum.times(z).dividedBy(two.times(pi).sqrt()).plus(0.5);
if(sum.lessThan(0))
sum = sum.abs();
else if(sum.greaterThan(1))
sum = two.minus(sum);
switch (type) {
case "left":
return parseFloat(sum.toExponential(40));
case "right":
return parseFloat((new Decimal(1).minus(sum)).toExponential(40));
case "twosided":
return sum.lessThan(0.5)? parseFloat(sum.times(two).toExponential(40)) : parseFloat((new Decimal(1).minus(sum).times(two)).toExponential(40))
}
}
By increasing the Decimal.js precision value and decreasing the loopstop value you can get accurate p-values for very small (or very high) z-scores for the cost of calculation time.
I've been trying to learn about generating noise and find that I understand most of it but I'm having a bit of trouble with a script.
I used this page as a guide to write this script in JavaScript with the ultimate purpose of creating some noise on canvas.
It's definitely creating something but it's tucked all the way over on the left. Also, refreshing the page seems to create the same pattern over and over again.
What have I done wrong that the "noisy" part of the image is smushed on the left? How can I make it look more like the cloudy perlin noise?
I don't really understand why it doesn't produce a new pattern each time. What would I need to change in order to receive a random pattern each time the script is run?
Thank you for your help!
/* NOISE—Tie it all together
*/
function perlin2d(x,y){
var total = 0;
var p = persistence;
var n = octaves - 1;
for(var i = 0; i <= n; i++) {
var frequency = Math.pow(2, i);
var amplitude = Math.pow(p, i);
total = total + interpolatenoise(x * frequency, y * frequency) * amplitude;
}
return total;
}
I've forked your fiddle and fixed a couple things to make it work: http://jsfiddle.net/KkDVr/2/
The main problem was the flawed pseudorandom generator "noise", that always returned 1 for large enough values of x and y. I've replaced it with a random values table that is queried with integer coordinates:
var values = [];
for(var i = 0; i < height; i++) {
values[i] = [];
for(var j = 0; j < width; j++) {
values[i][j] = Math.random() * 2 - 1;
}
}
function noise(x, y) {
x = parseInt(Math.min(width - 1, Math.max(0, x)));
y = parseInt(Math.min(height - 1, Math.max(0, y)));
return values[x][y];
}
However, the implementation provided in the tutorial you followed uses simplified algorithms that are really poorly optimized. I suggest you the excellent real-world noise tutorial at http://scratchapixel.com/lessons/3d-advanced-lessons/noise-part-1.
Finally, maybe you could be interested in a project of mine: http://lencinhaus.github.com/canvas-noise.
It's a javascript app that renders perlin noise on an html5 canvas and allows to tweak almost any parameter visually. I've ported the original noise algorithm implementation by Ken Perlin to javascript, so that may be useful for you. You can find the source code here: https://github.com/lencinhaus/canvas-noise/tree/gh-pages.
Hope that helps, bye!
I'm looking for the best way of implementing random number generator, that will allow me to have control over probability from what range the generated number will be returned. To visualize what I'm trying to achieve I have a picture :
So to summarize :
Let's say that my range is 400. At the beginning I'd like to have 5% probability of getting number 0-20. But at some moment in time I'd like to have this probability increased up to 50%. Hope you get the idea.
Hmm, working on your original I had a pretty simple algorithm to generate ranges in an array in the appropriate proportion, then randomly select a range and generate a random number within that range. No doubt it can be optimised if necessary, but it works for me.
It looks like a lot of code, but 3/4 of it is comments, test data and function, the actual randomRange function is only 17 lines of code.
<script type="text/javascript">
function randomRange(dataArray) {
// Helper function
function getRandomInRange(s, f) {
return (Math.random() * (f-s+1) | 0) + s
}
// Generate new data array based on probability
var i, j = dataArray.length;
var oArray = [];
var o;
while (j--) {
o = dataArray[j];
// Make sure probability is an integer
for (i=0, iLen=o.probability|0; i<iLen; i++) {
oArray.push([o.rangeStart, o.rangeEnd]);
}
}
// Randomly select a range from new data array and
// generate a random number in that range
var oEnd = oArray.length;
var range = oArray[getRandomInRange(0, oArray.length - 1)];
return getRandomInRange(range[0], range[1]);
}
// Test data set. Probability just has to be
// representative, so 50/50 === 1/1
var dataArray = [
{
rangeStart: 0,
rangeEnd : 20,
probability: 1
},
{
rangeStart: 21,
rangeEnd : 400,
probability: 1
}
];
// Test function to show range and number is randomly
// selected for given probability
function testIt() {
var el0 = document.getElementById('div0');
var el1 = document.getElementById('div1');
function run() {
var n = randomRange(dataArray);
if (n <= 20) {
el0.innerHTML += '*';
} else {
el1.innerHTML += '*';
}
}
setInterval(run, 500);
}
</script>
<button onclick="testIt();">Generate random number</button>
<div>Numbers 0 - 20</div>
<div id="div0"></div>
<div>Numbers 21 - 400</div>
<div id="div1"></div>
It sounds to me like what you're looking for is a way to generate numbers on a normal (or Gaussian) distribution (take a look at the Wikipedia page if you don't know what that means).
The Box-Muller transformation can be used to generate pairs of normally distributed numbers.
Here is a c++ implementation of the polar form of the Box-Muller transformation that shouldn't be hard to translate to javascript.
// Return a real number from a normal (Gaussian) distribution with given
// mean and standard deviation by polar form of Box-Muller transformation
double x, y, r;
do
{
x = 2.0 * rand() - 1.0;
y = 2.0 * rand() - 1.0;
r = x * x + y * y;
}
while ( r >= 1.0 || r == 0.0 );
double s = sqrt( -2.0 * log(r) / r );
return mean + x * s * stddev;
Where mean is the mean of the normal distribution and stddev is the Standard Deviation of the distribution. This code is from a MersesenneTwister C++ class that I've been using recently that you can find on Rick Wagner's page. You can find some more useful information about the Box-Muller transformation on this page.