Getting metric has already been registered when trying to publish metrics from service. To avoid that I used register.removeSingleMetric("newMetric"); but problem is that it clear register and past records every time a new call comes in.
Wondering what to address this problem:
export function workController(req: Request, res: Response) {
resolveFeaturePromises(featurePromises).then(featureAggregate => {
return rulesService.runWorkingRulesForHook(featureAggregate, hook.name, headers)
.then(([shouldLog, result]) => {
...
// publish metric
publishHookMetrics(result);
// publish metric
sendResponse(res, new CoreResponse(result.isWorking, result.responseData), req, featureAggregate, hook);
});
}).catch(err => {
sendResponse(res, new CoreResponse(Action.ALLOW, null), req, null, hook);
console.error(err);
});
}
function publishCustomMetrics(customInfoObject: CustomInfoObject) {
const counter = new promClient.Counter({
name: "newMetric",
help: "metric for custom detail",
labelNames: ["name", "isWorking"]
});
counter.inc({
name: customInfoObject.hook,
isWorking: customInfoObject.isWorking
});
}
stack trace
[Node] [2020-07-30T17:40:09+0500] [ERROR] Error: A metric with the name newMetric has already been registered.
application.ts
export async function startWebServer(): Promise<Server> {
if (!isGlobalsInitilized) {
throw new Error("Globals are noit initilized. Run initGlobals() first.");
}
// Setup prom express middleware
const metricsMiddleware = promBundle({
includeMethod: true,
includePath: true,
metricsPath: "/prometheus",
promClient: {
collectDefaultMetrics: {
}
}
});
// start http server
const app = require("express")();
app.use(bodyParser.json());
app.use(metricsMiddleware);
const routeConfig = require("./config/route-config");
routeConfig.configure(app);
const port = process.env.PORT || 3000;
return app.listen(port, function () {
console.log("service listening on port", port);
});
}
versions:
express-prom-bundle: 6.0.0
prom-client: 12.0.0
The counter should be initialize only once, then used for each call. Simplest modification from code you gave would be something like this.
const counter = new promClient.Counter({
name: "newMetric",
help: "metric for custom detail",
labelNames: ["name", "isWorking"]
});
export function publishCustomMetrics(customInfoObject: CustomInfoObject) {
counter.inc({
name: customInfoObject.hook,
isWorking: customInfoObject.isWorking
});
}
BTW, I found this thread while searching a way to avoid the "already been registered" error in unit tests. Many tests was instantiating the same class (from a library) that was initializing a counter in the constructor.
As it is in tests, and I didn't need consistency over the metrics, I found an easy solution is to clear metrics registers at the beginning of each test.
import { register } from "prom-client";
// ...
register.clear();
Related
I'm new to aws and my task is to rebuild the app (trigger the codepipeline) when we receive an sns message.
looking for something similar to the code below but not on a schedule instead using sns but i dont think i can use an sns event:
// A pipeline being used as a target for a CloudWatch event rule.
import * as targets from '#aws-cdk/aws-events-targets';
import * as events from '#aws-cdk/aws-events';
// kick off the pipeline every day
const rule = new events.Rule(this, 'Daily', {
schedule: events.Schedule.rate(Duration.days(1)),
});
declare const pipeline: codepipeline.Pipeline;
rule.addTarget(new targets.CodePipeline(pipeline));
these are the code fragments i collected but i dont think i can do what i want to do using a lambda function either.
const consumerTopic = sns.Topic.fromTopicArn(
this,
"myTopicId",
"arn:aws:sns:*******");
const fn = new Function(this, 'aFunction', {
runtime: Runtime.NODEJS_16_X,
handler: 'snsHandler.handler',
code: Code.fromAsset(__dirname),
});
consumerTopic.addSubscription(new LambdaSubscription(fn))
You can trigger a pipeline when you receive a sns message like this:
// create sns topic
const topic = new sns.Topic(this, 'MyTopic');
// or subscribe to an existing sns topic
const consumerTopic = sns.Topic.fromTopicArn(
this,
"myTopicId",
"arn:aws:sns:***YOUR ARN***");
const fn = new Function(this, 'aFunction', {
description: "lambda function to trigger a code pipeline build from a sns message",
runtime: Runtime.NODEJS_16_X,
handler: 'snsHandler.handler',
code: Code.fromAsset(join(__dirname, '../lambda')),
});
consumerTopic.addSubscription(new LambdaSubscription(fn))
// 👇 create a policy statement
const policy = new iam.PolicyStatement({
actions: ['codepipeline:StartPipelineExecution'],
resources: ['*'],
});
// 👇 add the policy to the Function's role
fn.role?.attachInlinePolicy(
new iam.Policy(this, 'run-pipeline-policy', {
statements: [policy],
}),
);
// ***** set a message:
// aws sns publish \
// --subject "Just testing 🚀" \
// --message "Hello world 🐊" \
// --topic-arn "***YOUR ARN***"
and the lambda function:
import { SNSMessage } from 'aws-lambda';
import {AWSError} from "aws-sdk";
import {StartPipelineExecutionOutput} from "aws-sdk/clients/codepipeline";
const AWS = require('aws-sdk');
exports.handler = async function(event:SNSMessage) {
const codePipeline = new AWS.CodePipeline({region: "ap-southeast-2" });
const params = {
name: '***NAME OF CODE PIPELINE**', /* required */
// clientRequestToken: 'STRING_VALUE'
};
return new Promise ((resolve, reject) => {
codePipeline.startPipelineExecution(params, function(err:AWSError, data:StartPipelineExecutionOutput) {
if (err) {
console.log(err, err.stack);
reject(err)
} // an error occurred
else {
console.log(data);
resolve(data)
} // successful response
});
})
};
one more example here:
AccessDeniedException while starting pipeline
I want to create a CLI application and I think this question is not about a specific technology but for the sake of reproduction purposes I'm using Node with command-line-commands ( but I know there are plenty others, e.g. commander ).
Given the following sample code
#!/usr/bin/env node
'use strict';
const commandLineArgs = require('command-line-args');
const commandLineCommands = require('command-line-commands');
const commandLineUsage = require('command-line-usage');
let isRunning = false; // global state
let commandResult;
try {
commandResult = commandLineCommands([ 'start', 'info', 'help' ]);
} catch (error) {
console.error('Invalid command.');
process.exit(1);
}
if (commandResult.command === null || commandResult.command === 'help') {
const commandInfo = commandLineUsage([
{ header: 'start', content: 'Sets the value to true' },
{ header: 'info', content: 'Gets the current value' },
]);
console.log(commandInfo);
process.exit(0);
}
let options;
try {
options = commandLineArgs([], { argv: commandResult.argv });
} catch (error) {
console.error('Invalid argument.');
process.exit(1);
}
if (commandResult.command === 'start') {
isRunning = true;
} else if (commandResult.command === 'info') {
console.info({ isRunning });
}
The boolean isRunning indicates a shared state. Calling the start command sets its value to true. But calling the info command obviously starts a new process and prints a new variable isRunning with its initial falsy value.
What is the prefered technology to keep such state? Must the CLI use an external database ( e.g. local filesystem) or are there some ways to keep the information in memory until shutdown?
Generating my own file on the system and storing this variable to it feels like an overkill to me.
An old cross-platform hack is to open a known TCP port. The first process able to open the port will get the port. All other processes trying to open the port will get an EADDRINUSE error:
const net = require('net');
const s = net.createServer();
s.on('error',() => {
console.log('Program is already running!');
// handle what to do here
});
s.listen(5123,'127.0.0.1',() => {
console.log('OK');
// run your main function here
});
This works in any language on any OS. There is only one thing you need to be careful of - some other program may be accidentally using the port you are using.
I originally came across this technique on the Tcl wiki: https://wiki.tcl-lang.org/page/singleton+application.
Another old hack for this is to try and create a symlink.
Creating symlinks are generally guaranteed to be atomic by most Unix and Unix-like OSes. Therefore there is no issue with potential race conditions using this technique (unlike creating a regular file). I presume it is also atomic on Windows (as per POSIX spec) but I'm not entirely sure:
const fs = require('fs');
const scriptName = process.argv[1];
const lockFile = '/tmp/my-program.lock';
try {
fs.symlinkSync(lockFile, scriptName);
// run your main function here
fs.unlinkSync(lockFile);
}
catch (err) {
console.log('Program already running');
// handle what to do here
}
Note: While creating symlinks are atomic, other operations on symlinks are not guaranteed to be atomic. Specifically be very careful of assuming that updating a symlink is atomic - it is NOT. Updating symlinks involve two operations: deleting the link and then creating the link. A second process may execute its delete operation after your process creates a symlink causing two processes to think that they're the only ones running. In the example above we delete the link after creating it, not before.
One way would be to use a local web server.
index.js
const commandLineArgs = require('command-line-args');
const commandLineCommands = require('command-line-commands');
const commandLineUsage = require('command-line-usage');
var http = require('http');
let globalState = {
isRunning: false
}
let commandResult;
try {
commandResult = commandLineCommands([ 'start', 'info', 'help' ]);
} catch (error) {
console.error('Invalid command.');
process.exit(1);
}
if (commandResult.command === null || commandResult.command === 'help') {
const commandInfo = commandLineUsage([
{ header: 'start', content: 'Sets the value to true' },
{ header: 'info', content: 'Gets the current value' },
]);
console.log(commandInfo);
process.exit(0);
}
let options;
try {
options = commandLineArgs([], { argv: commandResult.argv });
} catch (error) {
console.error('Invalid argument.');
process.exit(1);
}
if (commandResult.command === 'start') {
globalState.isRunning = true;
http.createServer(function (req, res) {
res.write(JSON.stringify(globalState));
res.end();
}).listen(9615);
} else if (commandResult.command === 'info') {
console.info({ globalState });
}
index2.js
var http = require('http');
var req = http.request({ host: "localhost", port: 9615, path: "/" }, (response) => {
var responseData = "";
response.on("data", (chunk) => {
responseData += chunk;
});
response.on("end", () => {
console.log(JSON.parse(responseData));
});
});
req.end();
req.on("error", (e) => {
console.error(e);
});
Here the index.js is a program that holds the "shared / global state" as well as creates a web server to communicate with. Other programs such as index2.js here can make a http request and ask for the global state. You could also let other programs change the state by having index.js listen to some specific request and act accordingly.
This doesn't have to be done with http like this, you could also use something like node-rpc or node-ipc. I thought the easiest working example would be to do it with a local http client and server.
Either way, I think the word for what you are looking for is Inter Process Communication (IPC) or Remote Procedure Call (RPC). I don't see why one couldn't also utilize websockets as well. Child processes probably won't work here, even if you could implement some kind of parent-child process communication, because only the child processes spawned by the main process could use that.
EDIT
After reading your question more carefully, I think that this is just a matter of "keeping" the "console session" after start command and setting the isRunning variable.
Check this out:
const commandLineArgs = require('command-line-args');
const commandLineCommands = require('command-line-commands');
const commandLineUsage = require('command-line-usage');
const prompt = require('prompt-sync')();
let globalState = {
isRunning: false
}
let commandResult;
try {
commandResult = commandLineCommands([ 'start', 'info', 'help' ]);
} catch (error) {
console.error('Invalid command.');
process.exit(1);
}
if (commandResult.command === null || commandResult.command === 'help') {
const commandInfo = commandLineUsage([
{ header: 'start', content: 'Sets the value to true' },
{ header: 'info', content: 'Gets the current value' },
]);
console.log(commandInfo);
process.exit(0);
}
let options;
try {
options = commandLineArgs([], { argv: commandResult.argv });
} catch (error) {
console.error('Invalid argument.');
process.exit(1);
}
if (commandResult.command === 'start') {
globalState.isRunning = true;
while(globalState.isRunning)
{
let cmd = prompt(">");
if(cmd === "exit")
process.exit(0);
if(cmd === "info")
console.info({ globalState });
}
} else if (commandResult.command === 'info') {
console.info({ globalState });
}
Here I am using prompt-sync library inside a loop when the program is called with a start command. The "console session" is kept indefinitely until the user types exit. I also added and example for in case the user types info.
Example:
I have a controller which I want to test
// user model
const userModel = require('../models/user')
// get user model from handler
const User = userModel.User
function index(req, res) {
User.find(function(err, users) {
if (err) console.log(err)
console.log('debug')
res.render('../views/user/index', {
title: 'User index',
users: users
})
})
}
module.exports = {
index
}
using the following code
// dependencies
const sinon = require('sinon')
// controller to be tested
const controller = require('../controllers/user')
// get user test
describe('test user index', function() {
it ('should return users', function() {
var req = {}
var res = {
render: function() {
sinon.spy()
}
}
controller.index(req, res)
})
})
When I run the test it doesn't execute the console.log('debug') which seems to me indicates that the test can't run the User.find() function. Any help would be appreciated.
As a temporary solution, you can include DB connection string at the top of the file.
Explanation: I guess your controller is part of an app, so it connects to DB on init. However, here you are testing an isolated controller, therefore DB connection isn't initialized yet. That's quick solution, for proper way of doing that refer to official docs
I am using feathers js 4 with objection ORM and trying to create a service that returns objection js model data that includes an eager loaded BleongsToOneRelation but I keep getting the response:
Cannot read property 'get' of undefined
I have used the Feathers CLI to generate my service (and model) and then modified to add a relationship as follows:
devices.model.js
const { Model } = require('objection');
class devices extends Model {
static get tableName() {
return 'devices';
}
static get jsonSchema() {
return {
type: 'object',
required: ['macAddress'],
properties: {
macAddress: { type: 'string' },
circuitId: { type: 'integer' },
}
};
}
static get relationMappings() {
const Circuit = require('./circuits.model')();
const DeviceType = require('./device-types.model')();
return {
circuit: {
relation: Model.BelongsToOneRelation,
modelClass: Circuit,
join: {
from: 'devices.circuitId',
to: 'circuits.id'
}
}
};
}
$beforeInsert() {
this.createdAt = this.updatedAt = new Date().toISOString();
}
$beforeUpdate() {
this.updatedAt = new Date().toISOString();
}
}
module.exports = function (app) {
const db = app.get('knex');
db.schema.hasTable('devices').then(exists => {
if (!exists) {
db.schema.createTable('devices', table => {
table.increments('id');
table.string('macAddress');
table.integer('circuitId');
table.timestamp('createdAt');
table.timestamp('updatedAt');
})
.then(() => console.log('Created devices table')) // eslint-disable-line no-console
.catch(e => console.error('Error creating devices table', e)); // eslint-disable-line no-console
}
})
.catch(e => console.error('Error creating devices table', e)); // eslint-disable-line no-console
return devices;
};
device.class.js
const { Devices } = require('./devices.class');
const createModel = require('../../models/devices.model');
const hooks = require('./devices.hooks');
module.exports = function (app) {
const options = {
Model: createModel(app),
paginate: app.get('paginate'),
whitelist: ['$eager', '$joinRelation', '$modifyEager'],
allowedEager: ['circuit']
};
// Initialize our service with any options it requires
app.use('/devices', new Devices(options, app));
// Get our initialized service so that we can register hooks
const service = app.service('devices');
service.hooks(hooks);
};
Then I am calling the /devices GET request using http://localhost:3030/devices?$eager=circuit
and the result is:
{
"name": "GeneralError",
"message": "Cannot read property 'get' of undefined",
"code": 500,
"className": "general-error",
"data": {},
"errors": {}
}
Have tried adding a custom find() method to devices.class.js but this doesn't seem to help.
I've searched through similar questions here and read through the feathers-objection docs but just can't see what I'm missing. Any assistance would be greatly appreciated.
Sounds like your app variable is undefined. Debugger could help you a lot if you add a breakpoint and check call stack to see why... Maybe you are not passing it everywhere correctly to your modules.
This is the code that throws your error:
https://runkit.com/embed/kipjj807i90l
It happens because in your relationMappings you are initializing models without the app argument which is expected. In the feathers-objection examples it is shown how to modify your models in order to use them in the relation mappings: you have to explicitly handle the missing app argument.
I know it's much later, but I just found the same problem.
The solution is to modify the feathers-objection models with the following, based on the Models section here
module.exports = function(app) {
if (app) {
const db = app.get('knex');
.... etc
}
return User;
};
module.exports = User;
I'm using Visual studio with typescript. Everything in this code compiles. According to the mongoDB docs I'm using toArray correctly. I'm a little new to typescript, so I don't know if this is a typescript error or mongodb. The tests variable seems to have a method toArray but when I call it nothing returns. The console.log call isn't even ran. According to the docs and the typescript samples this is the correct way to do it. Can anyone share with me any errors in my code, or the "correct" way to do this?
///<reference path="c:\DefinitelyTyped\mongodb\mongodb.d.ts"/>
import mongodb = require("mongodb")
var server = new mongodb.Server('localhost',27017, { auto_reconnect: true})
var db = new mongodb.Db('test', server, { w: 1 });
export interface Test {
_id: mongodb.ObjectID;
a: number;
}
db.open(function () { });
export function getTest(callback: (test: any) => void): void {
db.collection('test', function (err, test_collection) {
// test_collection.find().toArray -- this doesn't work either
test_collection.find(function (err, tests) {
console.log(tests, 'from getTest') // log's an object with `toArray` method
tests.toArray(function (err, docs) { // nothing returned. Seems like the callback isn't ran
if (err) { console.log(err) }
console.log(docs, 'from toArray')
callback(docs)
})
})
})
}
You problem seems to be not placing your function within the db.open method's callback in general:
var mongodb = require("mongodb");
var server = new mongodb.Server('localhost', 27017, { auto_reconnect: true });
var db = new mongodb.Db('test', server, { w: 1 });
db.open(function() {
db.createCollection('test', function(err, collection) {
collection.find().toArray(function(err,docs) {
console.log( docs );
});
});
});
You generally need to make sure a connection is open before doing anything