Short form question:
We are trying to access the express-http-context in a dependent logger module of our main NodeJS service. However, we are getting it as undefined.
Context:
We are trying to tag all our logs with a requestId for each unique request in NodeJS. We have a logger module that wraps winston logger. This logger module is a separate package and is used across all our different NodeJs (Express) services.
Our winston logger wrapper file in our cf-logger-module is as follows:
import { omit } from 'lodash';
import { Logger, createLogger, format, transports, LeveledLogMethod } from 'winston';
import httpcontext from 'express-http-context';
import Sentry from 'winston-sentry-log';
import { ILoggerConfig } from '../interfaces/ILoggerConfig';
import { LevelCodes } from '../constants/LoggingLevels';
import ILoggerAdapter from '../interfaces/ILoggerAdapter';
import LoggerConfiguration from '../LoggerConfiguration';
export default class WinstonLogger implements ILoggerAdapter {
private readonly _logger: Logger;
private _config: ILoggerConfig;
private _moduleName: string;
private _loggingMethods: { [prop: number]: LeveledLogMethod };
public static createAdapter(moduleName: string, config: LoggerConfiguration): ILoggerAdapter {
return new WinstonLogger(moduleName, config);
}
public constructor(moduleName: string, loggerConfiguration: LoggerConfiguration) {
this._config = loggerConfiguration.config;
this._moduleName = moduleName;
const sentryOptions = {
config: {
dsn: 'https://b71e24c577604eab977dcf18720f1d5a#o1185085.ingest.sentry.io/6468703',
},
level: 'info',
};
this._logger = createLogger({
levels: LevelCodes,
transports: [new transports.Console({ level: this._config.logging.level }), new Sentry(sentryOptions)],
format: format.combine(
format.timestamp({ format: this._config.logging.timestamp.format }),
format.errors({ stack: this._config.logging.errors.stack }),
format.simple(),
format.colorize(),
format.printf(({ level, message, timestamp, ...metadata }) => {
const module = metadata.module;
const method = `${metadata.methodName}()`;
metadata = omit(metadata, 'module');
metadata = omit(metadata, 'methodName');
// console.log('Logger http context', httpcontext.get('reqId'));
return `${timestamp} [${module}:${method}] [${level}]: ${message} ${JSON.stringify(metadata)}`;
}),
),
});
/* istanbul ignore next */
this._loggingMethods = {
[LevelCodes.debug]: this._logger?.debug.bind(this._logger),
[LevelCodes.error]: this._logger?.error.bind(this._logger),
[LevelCodes.http]: this._logger?.http.bind(this._logger),
[LevelCodes.info]: this._logger?.info.bind(this._logger),
[LevelCodes.silly]: this._logger?.silly.bind(this._logger),
[LevelCodes.verbose]: this._logger?.verbose.bind(this._logger),
[LevelCodes.warn]: this._logger?.warn.bind(this._logger),
};
}
public logMessage(level: number, message: string, metadata?: object, error?: Error): void {
this._loggingMethods[level](message, this.process(metadata, error));
}
private process(metadata?: object, error?: Error): object {
/** Extract Service Name */
console.log('Logger http context', httpcontext.get('reqId'));
Object.assign(metadata, {
module: `${this._config.service.name}.${this._moduleName}`,
});
/** Extract Error */
if (error !== undefined) {
Object.assign(error, metadata);
}
/** TODO: Sanitize and metadata */
return error || metadata;
}
}
As you can see we are trying to access the reqID set using express-http-context.
Our nodeJs apps package.json dependencies look like:
"dependencies": {
"#coverforce-platform/cf-logger-module": "^1.0.26",
....
"express-http-context": "^1.2.4",
"express-ruid": "^1.1.4",
},
The app.ts looks like:
private initializeMiddlewares() {
this.app.use(cors({ origin: AppConfig.getValues().SERVICE.ORIGIN }));
this.app.use(hpp());
this.app.use(helmet());
this.app.use(compression());
this.app.use(express.json());
this.app.use(express.urlencoded({ extended: true }));
this.app.use(cookieParser());
this.app.use(httpcontext.middleware);
this.app.use(ruid({ setInContext: true, attribute: 'reqId' }));
this.app.use(Sentry.Handlers.requestHandler() as express.RequestHandler);
this.app.use(Sentry.Handlers.errorHandler());
this.app.use(Sentry.Handlers.tracingHandler());
}
and the server.ts is:
import 'reflect-metadata';
import { Container } from 'typedi';
import { LoggerFactory, ILogger } from '#coverforce-platform/cf-logger-module';
import { AppConfig } from '#coverforce-platform/cf-config-module';
import { ErrorResponseMiddleware } from '#coverforce-platform/cf-error-module';
import { SchemaValidationMiddleware } from '#coverforce-platform/cf-schema-validation-module';
import { AccountsServiceSchema } from '#coverforce-platform/cf-common-api-model';
import { accountsDatabaseLoader } from './dependencyInjection/accountsDatabase.dependency';
import { App } from './app';
import { ERROR_RESPONSE_MIDDLEWARE, SCHEMA_VALIDATION_MIDDLEWARE } from './constants/dependencyInjection.constants';
main();
async function main() {
const logger: ILogger = initLogger();
await initAppConfig(logger);
initializeDependencyInjection();
const app: App = Container.get(App);
app.listen();
}
However, in the logger the console.log('Http Context', httpcontext.get('reqId') is always returning undefined for the reqId field in the httpcontext. If we do the same within any class in the main service we are able to retrieve the httpcontext's reqId. However, we are not able to access it in the dependent package/module.
Related
I am trying to manage some logic by using .env varibles but I dont how what I am doing wrong and a can get the variables loaded inside my services.
Here is my App.module.ts:
...
import { ConfigModule } from '#nestjs/config';
import configuration from './config/configuration';
...
#Module({
imports: [
ConfigModule.forRoot({
load: [configuration],
validationSchema: JoiValidationSchema,
isGlobal: true,
}),
...
],
...
})
export class AppModule {}
Here is the configuration file:
export default () => ({
environment: process.env.NODE_ENV || 'dev',
port: process.env.PORT || 5000,
postgresPassword: process.env.POSTGRES_PASSWORD,
postgresUsername: process.env.POSTGRES_USERNAME,
postgresDbName: process.env.POSTGRES_NAME,
postgresHost: process.env.POSTGRES_HOST,
postgresPort: parseInt(process.env.POSTGRES_PORT, 10),
defaultLimit: parseInt(process.env.DEFAULT_LIMIT, 10) || 10,
});
When I print the variables in my ProductServices contructor it works good
export class ProductsService {
private readonly logger = new Logger('ProductsServices');
constructor(
...
private readonly configService: ConfigService,
) {
console.log(this.defaultLimit); // 10
console.log(this.environment); // 'dev'
}
defaultLimit = this.configService.get<number>('defaultLimit');
environment = this.configService.get<string>('environment');
But when I trying to using in one of my ProductService methods it is undefined:
async removeAll() {
console.log(this.environment); // undefined
try {
if (this.environment !== 'dev') {
throw new UnauthorizedException();
}
...
} catch (error) {
...
}
}
Any help will be appreciated
I need get the enviroment to manage some seed logic
I am trying to integrate Opentelemetry (Otl) in my Angular application to trace the frontend calls. Everything works fine and I am able to see the calls in the Zipkin.
But the only problem is that it is showing it as "unknown_service" in the Zipkin interface.
Below is my entire Angular code and Zipkin screenshot as well. This is just a sample application. But my requirement is that I am going to integrate the Opentelemetry code in the http interceptor so that it will be easy to maintain at one place instead of every service call. Also service.name should be passed dynamically so that it will be traced in Zipkin.
How can I add a service name before it gets called?
import { Component, OnInit } from '#angular/core';
import {ZipkinServicesService} from './zipkin-services.service';
// Opentelemetry components
import { context, trace } from '#opentelemetry/api';
import { ConsoleSpanExporter, SimpleSpanProcessor } from '#opentelemetry/tracing';
import { WebTracerProvider } from '#opentelemetry/web';
import { XMLHttpRequestInstrumentation } from '#opentelemetry/instrumentation-xml-http-request';
import { ZoneContextManager } from '#opentelemetry/context-zone';
import { CollectorTraceExporter } from '#opentelemetry/exporter-collector';
import { B3Propagator } from '#opentelemetry/propagator-b3';
import { registerInstrumentations } from '#opentelemetry/instrumentation';
import { ZipkinExporter } from '#opentelemetry/exporter-zipkin';
#Component({
selector: 'app-zipkin-integration',
templateUrl: './zipkin-integration.component.html',
styleUrls: ['./zipkin-integration.component.scss']
})
export class ZipkinIntegrationComponent implements OnInit {
respData: string;
webTracerWithZone;
constructor(
public zipkinService: ZipkinServicesService,
) {
const providerWithZone = new WebTracerProvider();
const options = {
url: 'http://localhost:9411/api/v2/spans',
serviceName: 'interceptor-example',// This is NOT working.
}
const exporter = new ZipkinExporter(options);
const zipKinProcessor = new SimpleSpanProcessor(exporter);
providerWithZone.addSpanProcessor(zipKinProcessor);
providerWithZone.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
providerWithZone.addSpanProcessor(new SimpleSpanProcessor(new CollectorTraceExporter()));
providerWithZone.register({
contextManager: new ZoneContextManager(),
propagator: new B3Propagator(),
});
registerInstrumentations({
instrumentations: [
new XMLHttpRequestInstrumentation({
ignoreUrls: [/localhost:8090\/sockjs-node/],
propagateTraceHeaderCorsUrls: [
'https://httpbin.org/post',
],
}),
],
});
this.webTracerWithZone = providerWithZone.getTracer('example-tracer-web');
}
ngOnInit(): void {
}
zipGet (){
let i = 10;
const span1 = this.webTracerWithZone.startSpan(`files-series-info-${i}`);
let postData = [{
no : 2,
emp : 3
}];
context.with(trace.setSpan(context.active(), span1), () => {
this.zipkinService.httpGet(postData).subscribe( (data: any) => {
this.respData = data;
// Opentelemetry after response.
trace.getSpan(context.active()).addEvent('fetching-span1-completed');
span1.end();
});
});
}
zipPost (){
let postData = [{
no : 1,
emp : 2
}];
let i = 10;
const span1 = this.webTracerWithZone.startSpan(`files-series-info-${i}`);
context.with(trace.setSpan(context.active(), span1), () => {
this.zipkinService.httpPost(postData).subscribe( (data: any) => {
this.respData = data;
// Opentelemetry after response.
trace.getSpan(context.active()).addEvent('fetching-span1-completed');
span1.end();
});
});
}
}
Service name must be set via resource as per the specification. I am not sure which version of js libs you are using. This should get you the service name.
import { Resource } from '#opentelemetry/resources';
import { ResourceAttributes } from '#opentelemetry/semantic-conventions'
...
...
const provider = new WebTracerProvider({
resource: new Resource({
[ResourceAttributes.SERVICE_NAME]: "interceptor-example"
}),
});
use providerConfig to set service name. follow code set service name to "SPA Test".
import { Resource } from '#opentelemetry/resources';
import { SemanticResourceAttributes } from '#opentelemetry/semantic-conventions'
import { BatchSpanProcessor } from '#opentelemetry/sdk-trace-base';
import { WebTracerProvider } from '#opentelemetry/sdk-trace-web';
import { ZipkinExporter, ExporterConfig } from '#opentelemetry/exporter-zipkin';
const providerConfig = {
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: "SPA Test"
}),
};
const provider = new WebTracerProvider(providerConfig);
const zipkinOptions: ExporterConfig = {
url: "http://localhost:9411/api/v2/spans"
};
const exporter = new ZipkinExporter(zipkinOptions);
const zipkinProcessor = new BatchSpanProcessor(exporter);
provider.addSpanProcessor(zipkinProcessor);
provider.register();
var tracer = provider.getTracer(CustomersComponent.name, "0.1.0");
var span = tracer.startSpan(CustomersComponent.name);
console.info(span);
span.end();
In the following example, the lambda code works (the lambda factory class), but the api code does not. Any idea why that might be? The lambda code instantiates the expected outputs. But the API code creaates nothing. It doesn't fail. It just doesn't do anything... As a quick sanity check, I instantiated the same chunk of code in the main file, and that does work fine...
top level
import { LambdaFactory } from './factory-dev/lambda/lambda-factory';
import { ApiGatewayFactory2 } from './factory-dev/api-gateway/apiGW';
export class SnDevStack extends cdk.Stack {
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// 1. Lambda
const lambdaFactory = new LambdaFactory(this, 'lambdaFactoryId');
const lambdaSigfoxSync = lambdaFactory.sigfoxSync('testing options');
// 2. Dev API
const apiGatewayFactory = new ApiGatewayFactory2(this, 'ApiGatewayFactoryId');
const api = apiGatewayFactory.getApi('testing options');
}
lambda factory class
import * as cdk from '#aws-cdk/core';
import * as lambda from '#aws-cdk/aws-lambda'; // need for Runtime
import * as lambda_nodejs from '#aws-cdk/aws-lambda-nodejs';
export class LambdaFactory extends cdk.Construct {
constructor (scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id);
}
private TAG:string = "sn-dev-";
public sigfoxSync(options: string): lambda_nodejs.NodejsFunction {
const testLambda = new lambda_nodejs.NodejsFunction(this, 'sigfox-sync', {
runtime: lambda.Runtime.NODEJS_12_X,
entry: 'lambda/sn-dev/sigfox-sync.js',
handler: 'handler',
functionName: this.TAG + 'sigfox-sync',
description: 'Get all devices from Sigfox API & Sync into dynamoDB.',
memorySize: 256,
timeout: cdk.Duration.seconds(720)
});
return testLambda;
}
}
api factory class
import * as cdk from '#aws-cdk/core';
import * as apiGateway from '#aws-cdk/aws-apigateway';
export class ApiGatewayFactory2 extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id);
}
public getApi(options: string):apiGateway.RestApi {
const api = new apiGateway.RestApi(this, 'sn-dev',
{
restApiName: 'sn-dev',
deploy: true,
defaultCorsPreflightOptions: {
allowOrigins: apiGateway.Cors.ALL_ORIGINS,
allowMethods: apiGateway.Cors.ALL_METHODS // this is also the default
},
});
return api;
}
}
To double check, the following works perfectly if I write it directly in the main file. Why on earth does it not work in the factory class????
const api = new apiGateway.RestApi(this, 'sn-dev',
{
restApiName: 'sn-dev',
deploy: true,
deployOptions: {
// documentationVersion: documentVersion
},
defaultCorsPreflightOptions: {
allowOrigins: apiGateway.Cors.ALL_ORIGINS,
allowMethods: apiGateway.Cors.ALL_METHODS // this is also the default
},
});
Looks like your ApiGatewayFactory2 is extending Stack it should extend Construct.
I think if you want to create a child stack you need to extend NestedStack.
So basically I have a controller which is using a decorator 'UseInterceptor' in that I am passing a class which is calling a method but I feel I am not doing the right way it should be. I need someone who could help me in defining a better way to maintain the singularity of the controller class.Here is the code
icons.controller.ts
var uploadOptions = new InterceptorClass(); ----> This bugs me and ---doesn't feel right way of doing.
const filesize = 512 * 512;
const driverFilesize = 512 * 512;
#Controller('icons')
export class IconsController {
constructor(
private iconsService: IconsService) { }
#Post('/marker/')
#UseGuards(CustomAccessGuard)
#UseRoles({
resource: ModuleNames.ICONS,
action: "create",
possession: "own"
})
#UseInterceptors(uploadOptions.uploadInterceptor(filesize)) ----> This is where I am using it.
async uploadMarkerFile(
#UploadedFile() file,
#GetUser() user,
#FileValidator() validator
) {
let result = await this.iconsService.uploadFile(file, IconEntityType.MARKER, user);
return new SuccessResponse(true, "Successfully Uploaded", result);
}
}
file.service.ts
For Reference..
import { FileInterceptor } from "#nestjs/platform-express"
import { memoryStorage } from 'multer';
import { editFileName, imageFileFilter } from "../icons/utils/file-upload.utils";
export class InterceptorClass {
constructor() { }
uploadInterceptor(fileSize: number) {
const UploadInterceptor = FileInterceptor('file', {
storage: memoryStorage({
// destination: './src/assets/marker/icons',
filename: editFileName
}),
fileFilter: imageFileFilter,
limits: {
files: 1,
fileSize: fileSize
}
});
return UploadInterceptor;
}
}
You could simply wrap the FileInterceptor decorator in your own decorator where you initialize it with the given config:
// in file custom-file.interceptor.ts
import { editFileName, imageFileFilter } from "../icons/utils/file-upload.utils";
export function CustomFileInterceptor(fileSize) {
const UploadInterceptor = FileInterceptor('file', {
storage: memoryStorage({
// destination: './src/assets/marker/icons',
filename: editFileName
}),
fileFilter: imageFileFilter,
limits: {
files: 1,
fileSize
}
});
return UploadInterceptor;
}
You can then use it like this:
import {CustomFileInterceptor} from "./custom-file.interceptor";
...
#UseInterceptors(CustomFileInterceptor(512))
async uploadMarkerFile(...)
I want to create a Node REST API with Typescript and created a basic class managing the Express application
import express from 'express';
import { Server } from 'http';
import { injectable } from 'inversify';
import { IWebServer } from './IWebServer';
import { RoutesLoader } from './routes/RoutesLoader';
import * as webServerConfig from '../../config/webServerConfig';
import { IPlugin } from './plugins/IPlugin';
import { LoggerPlugin } from './plugins/LoggerPlugin';
import { CorsPlugin } from './plugins/CorsPlugin';
import { BodyParserPlugin } from './plugins/BodyParserPlugin';
#injectable()
export class WebServer implements IWebServer {
public app: express.Application;
public httpServer: Server;
private port: any;
constructor () {
this.app = express();
this.httpServer = null;
this.port = webServerConfig.port;
}
public startListening(): void
{
const plugins: IPlugin[] = [
new LoggerPlugin(),
new CorsPlugin(),
new BodyParserPlugin()
];
for (const plugin of plugins) { // load all the middleware plugins
plugin.register();
}
new RoutesLoader(); // load all the routes
try {
this.httpServer = this.app.listen(this.port);
} catch (error) {
throw error;
}
}
public stopListening(): void
{
this.httpServer.close();
}
}
This code looks fine to me but the problem is that I have to assign a value to httpServer in the class constructor. As you can see I assign a value to it later in startListening. But I can't assign null to it in the constructor. undefined neither. This type is not nullable. How can I declare this variable without assigning a value to it when creating an instance of this class?
As mentioned in the comments, your httpServer field can be null and also is null before a call to startListening.
Therefor you have to specify that in the type declaration like this:
public httpServer: Server | null;
and then handle the null cases in further methods:
public stopListening(): void
{
if (this.httpServer === null) {
throw "Not listening, call "startListening()" first";
}
this.httpServer.close();
}