I'm trying to run some code which creates tracing spans for the various phases in the lifecycle of an http request (socket, dns lookup, connect or secureConnect, ttfb, end). As of now it looks more or less like this:
function tracedRequest(
options: HttpRequestOptions | HttpsRequestOptions,
callback: ResponseCallback
): ClientRequest {
const isHttps = options.protocol === 'https' || options.agent instanceof HttpsAgent;
const transport = isHttps ? https.request : http.request;
const requestSpan = tracer.createChildSpan({ name: 'request' });
if (!tracer.isRealSpan(requestSpan)) {
return transport.call(null, options, callback);
}
let socketSpan: ISpan | undefined;
let dnsSpan: ISpan | undefined;
let tcpSpan: ISpan | undefined;
let tlsSpan: ISpan | undefined;
let ttfbSpan: ISpan | undefined;
const onLookup = () => {
dnsSpan?.endSpan();
tcpSpan = tracer.createChildSpan({ name: 'http_tcp_handshake' });
};
const onConnect = () => {
tcpSpan?.endSpan();
if (isHttps) {
tlsSpan = tracer.createChildSpan({ name: 'http_tls_handshake' });
} else {
ttfbSpan = tracer.createChildSpan({ name: 'http_ttfb' });
}
}
const onSecureConnect = () => {
tlsSpan?.endSpan();
// just in case secureConnect is emmited not only for https transports
if (isHttps) {
ttfbSpan = tracer.createChildSpan({ name: 'http_ttfb' });
}
}
const onResponse = (response: IncomingMessage) => {
ttfbSpan?.endSpan();
response.prependOnceListener('end', () => {
requestSpan.endSpan();
});
}
const onSocket = (socket: Socket | TLSSocket) => {
socketSpan.endSpan();
socket.prependOnceListener('lookup', onLookup);
deferToConnect(socket, {
connect: onConnect,
secureConnect: onSecureConnect
});
}
socketSpan = tracer.createChildSpan({ name: 'http_establish_socket' });
const request: ClientRequest = transport.call(null, options, callback);
if (request.socket) {
onSocket(request.socket as any);
} else {
request.prependOnceListener('socket', onSocket);
}
request.prependOnceListener('response', onResponse);
return request;
}
The problem with this approach arises when you use an agent with keepalive enable. In this situation the socket may be reused, thus the socket already has established a connection to the remote host and neither the socket nor the lookup events will be emitted (notice that for the socket event this is handled, we can know that the socket event will not be emitted if the request.socket property is set).
How can I do the same thing for the lookup event ? Which property of the socket object can I check to be sure that the host has already been resolved and the lookup event will not be emitted ? Should I use the localAddress/localPort, remoteAddress/remotePort properties or the socket.address() method ?
So, I did some testing and apparently you can do:
if (Object.keys(socket.address()).length) {
onLookup();
} else {
socket.prependOnceListener('lookup', onLookup);
}
socket.address() returns an empty object if an address has not been resolved yet, otherwise it returns an object with the properties address, port and family.
So far it has worked for me
Related
I'm trying to create a singleton that has a single amqp connection and when createChannel method is called, it must return a new channel from the same connection:
export interface IBroker {
createChannel(): Promise<IChannel>;
}
export default class Broker implements IBroker {
private static instance: Broker;
private conn: IConnection | undefined;
private constructor(public config: IRabbitMQConfig = new RabbitMQConfig()) {}
/**
* singleton
*/
public static getInstance(): Broker {
if (!this.instance) {
this.instance = new Broker();
}
return this.instance;
}
/**
* initiates configuration on infra service
*/
async createChannel(): Promise<IChannel> {
try {
if (!this.conn) {
this.conn = await this.config.init();
await this.createExchanges();
await this.createQueues();
await this.createBinds();
logger.info('Broker started successfully');
}
if (!this.conn) {
throw new InternalError('Error starting broker. Missing connection!');
}
return await this.conn.createChannel();
} catch (err) {
logger.error('Error trying to start broker', err);
throw new InternalError('Error trying to start broker', 500);
}
}
// code...
the call config.init() returns the amqp connection.
when I test the class like below, every time I call createChannel it creates a new connection!
const a = Broker.getInstance();
const b = Broker.getInstance();
console.log(a === b); // return true
a.createChannel(); // create a new connection
b.createChannel(); // creates another connection
this.conn of Broker class is always undefined when createChannel is called!
I think the issue is that the two synchronous calls to createChannel mean that the first one hasn't initialized the connection by the time the second is called, which leads to 2 connections being created.
If you want to make your createChannel "thread-safe" in terms of creating the connection, you could do something like this (untested):
interface IConnection {
connect: () => void
}
const initConnection = () : Promise<IConnection> => {
return Promise.resolve({
connect: () => {}
});
};
class Broker {
private connection : IConnection | undefined;
private pendingConnection : Promise<IConnection> | undefined;
async createChannel() : Promise<IConnection> {
if (this.connection) {
return this.connection;
}
if (this.pendingConnection) {
return this.pendingConnection;
}
this.pendingConnection = initConnection();
const conn = await this.pendingConnection;
// Do other setup stuff
this.connection = conn;
this.pendingConnection = undefined;
return conn;
}
}
I have a small web application listening for incoming messages from a Websocket server. I receive them like so
const webSocket = new WebSocket("wss://echo.websocket.org");
webSocket.onopen = event => webSocket.send("test");
webSocket.onmessage = event => console.log(event.data);
but the sending server is more complex. There are multiple types of messages that could come e.g. "UserConnected", "TaskDeleted", "ChannelMoved"
How to detect which type of message was sent? For now I modified the code to
const webSocket = new WebSocket("wss://echo.websocket.org");
webSocket.onopen = event => {
const objectToSend = JSON.stringify({
message: "test-message",
data: "test"
});
webSocket.send(objectToSend);
};
webSocket.onmessage = event => {
const objectToRead = JSON.parse(event.data);
if (objectToRead.message === "test-message") {
console.log(objectToRead.data);
}
};
So do I have to send an object from the server containing the "method name" / "message type" e.g. "TaskDeleted" to identify the correct method to execute at the client? That would result in a big switch case statement, no?
Are there any better ways?
You can avoid the big switch-case statement by mapping the methods directly:
// List of white-listed methods to avoid any funny business
let allowedMethods = ["test", "taskDeleted"];
function methodHandlers(){
this.test = function(data)
{
console.log('test was called', data);
}
this.taskDeleted = function(data)
{
console.log('taskDeleted was called', data);
}
}
webSocket.onmessage = event => {
const objectToRead = JSON.parse(event.data);
let methodName = objectToRead.message;
if (allowerMethods.indexOf(methodName)>=0)
{
let handler = new methodHandlers();
handler[methodName](data);
}
else
{
console.error("Method not allowed: ", methodName)
}
};
As you have requested in one of your comments to have a fluent interface for the websockets like socket.io.
You can make it fluent by using a simple PubSub (Publish Subscribe) design pattern so you can subscribe to specific message types. Node offers the EventEmitter class so you can inherit the on and emit events, however, in this example is a quick mockup using a similar API.
In a production environment I would suggest using the native EventEmitter in a node.js environment, and a browser compatible npm package in the front end.
Check the comments for a description of each piece.
The subscribers are saved in a simple object with a Set of callbacks, you can add unsubscribe if you need it.
note: if you are using node.js you can just extend EventEmitter
// This uses a similar API to node's EventEmitter, you could get it from a node or a number of browser compatible npm packages.
class EventEmitter {
// { [event: string]: Set<(data: any) => void> }
__subscribers = {}
// subscribe to specific message types
on(type, cb) {
if (!this.__subscribers[type]) {
this.__subscribers[type] = new Set
}
this.__subscribers[type].add(cb)
}
// emit a subscribed callback
emit(type, data) {
if (typeof this.__subscribers[type] !== 'undefined') {
const callbacks = [...this.__subscribers[type]]
callbacks.forEach(cb => cb(data))
}
}
}
class SocketYO extends EventEmitter {
constructor({ host }) {
super()
// initialize the socket
this.webSocket = new WebSocket(host);
this.webSocket.onopen = () => {
this.connected = true
this.emit('connect', this)
}
this.webSocket.onerror = console.error.bind(console, 'SockyError')
this.webSocket.onmessage = this.__onmessage
}
// send a json message to the socket
send(type, data) {
this.webSocket.send(JSON.stringify({
type,
data
}))
}
on(type, cb) {
// if the socket is already connected immediately call the callback
if (type === 'connect' && this.connected) {
return cb(this)
}
// proxy EventEmitters `on` method
return super.on(type, cb)
}
// catch any message from the socket and call the appropriate callback
__onmessage = e => {
const { type, data } = JSON.parse(e.data)
this.emit(type, data)
}
}
// create your SocketYO instance
const socket = new SocketYO({
host: 'wss://echo.websocket.org'
})
socket.on('connect', (socket) => {
// you can only send messages once the socket has been connected
socket.send('myEvent', {
message: 'hello'
})
})
// you can subscribe without the socket being connected
socket.on('myEvent', (data) => {
console.log('myEvent', data)
})
I am trying to modify a Node.js function called 'splunk-logger'. The problem is that when the SNS Message comes into the function, the events from the Anti-Virus (Trend Micro DeepSecurity) console are grouped together. I already contacted their support and they said this is just the way events are sent and they can't help.
Example: {Message {Event_1} {Event_2} {Event_3}}
Now the JavaScript function works great and the events are forwarded to Splunk. However, since they are grouped together BEFORE they even hit the Lambda function, Splunk sees them as 1 single event instead of 3.
My thought is to take the 'event' variable (since it contains the sns 'message') and parse through that to separate each event (probably using regex or something). Then, I can either create another function to send each event immediately or simply call the "logger.flushAsync" function to send them.
Link to splunk-dev explaining the funciton: http://dev.splunk.com/view/event-collector/SP-CAAAE6Y#create.
Here is the code from the index.js:
const loggerConfig = {
url: process.env.SPLUNK_HEC_URL,
token: process.env.SPLUNK_HEC_TOKEN,
};
const SplunkLogger = require('./lib/mysplunklogger');
const logger = new SplunkLogger(loggerConfig);
exports.handler = (event, context, callback) => {
console.log('Received event:', JSON.stringify(event, null, 2));
// Log JSON objects to Splunk
logger.log(event);
// Send all the events in a single batch to Splunk
logger.flushAsync((error, response) => {
if (error) {
callback(error);
} else {
console.log(`Response from Splunk:\n${response}`);
callback(null, event.key1); // Echo back the first key value
}
});
};
Here is the code from the mysplunklogger.js file.
'use strict';
const url = require('url');
const Logger = function Logger(config) {
this.url = config.url;
this.token = config.token;
this.addMetadata = true;
this.setSource = true;
this.parsedUrl = url.parse(this.url);
// eslint-disable-next-line import/no-dynamic-require
this.requester = require(this.parsedUrl.protocol.substring(0, this.parsedUrl.protocol.length - 1));
// Initialize request options which can be overridden & extended by consumer as needed
this.requestOptions = {
hostname: this.parsedUrl.hostname,
path: this.parsedUrl.path,
port: this.parsedUrl.port,
method: 'POST',
headers: {
Authorization: `Splunk ${this.token}`,
},
rejectUnauthorized: false,
};
this.payloads = [];
};
// Simple logging API for Lambda functions
Logger.prototype.log = function log(message, context) {
this.logWithTime(Date.now(), message, context);
};
Logger.prototype.logWithTime = function logWithTime(time, message, context) {
const payload = {};
if (Object.prototype.toString.call(message) === '[object Array]') {
throw new Error('message argument must be a string or a JSON object.');
}
payload.event = message;
// Add Lambda metadata
if (typeof context !== 'undefined') {
if (this.addMetadata) {
// Enrich event only if it is an object
if (message === Object(message)) {
payload.event = JSON.parse(JSON.stringify(message)); // deep copy
payload.event.awsRequestId = context.awsRequestId;
}
}
if (this.setSource) {
payload.source = `lambda:${context.functionName}`;
}
}
payload.time = new Date(time).getTime() / 1000;
this.logEvent(payload);
};
Logger.prototype.logEvent = function logEvent(payload) {
this.payloads.push(JSON.stringify(payload));
};
Logger.prototype.flushAsync = function flushAsync(callback) {
callback = callback || (() => {}); // eslint-disable-line no-param-reassign
console.log('Sending event(s)');
const req = this.requester.request(this.requestOptions, (res) => {
res.setEncoding('utf8');
console.log('Response received');
res.on('data', (data) => {
let error = null;
if (res.statusCode !== 200) {
error = new Error(`error: statusCode=${res.statusCode}\n\n${data}`);
console.error(error);
}
this.payloads.length = 0;
callback(error, data);
});
});
req.on('error', (error) => {
callback(error);
});
req.end(this.payloads.join(''), 'utf8');
};
module.exports = Logger;
import requests
import re
import json
import os
def lambda_handler(event, context):
data = json.dumps(event)
EventIds = re.findall(r'{\\\".+?\\\"}', data)
EventLength = len(EventIds)
headers = {'Authorization': 'Splunk ' + os.environ['SPLUNK_HEC_TOKEN']}
i = 0
while i < EventLength:
response = requests.post(os.environ['SPLUNK_HEC_URL'], headers=headers, json={"event":EventIds[i]}, verify=True)
i+=1
Arrays are the data type used when Deep Security 10.0 or newer sends events to Amazon SNS. But Splunk wants one event per message. So don't send the array directly.
Instead, use the Splunk logger or Lambda to iterate through the array, sending each item as an individual message. You can modify this sample Lambda script for Node.js:
https://github.com/deep-security/amazon-sns/blob/master/lambda-save-ds-event-to-s3.js
It sends events to S3 individually (which is what you need). Just change it to send to Splunk instead.
Disclosure: I work for Trend Micro.
I'm creating React application that use JsSIP library to answer calls made via VoIP SIP provider.
I've already created a page that have two buttons (Accept and Reject). It successfully register SIP client on SIP-server. It also successfully receive call and I can answer it. But I don't hear anything while answering call.
Registering JsSIP client (in willReceiveProps because I have information for connection after props changing):
const socketHost = 'wss://' + contactCenter.host + ':' + contactCenter.port
const socket = new JsSIP.WebSocketInterface(socketHost)
const configuration = {
sockets: [socket],
uri: 'sip:' + contactCenter.login + '#' + contactCenter.host,
password: contactCenter.password,
socketHost: socketHost,
}
const coolPhone = new JsSIP.UA(configuration)
coolPhone.on('connected', (e: any) => {
const messages = ServiceContainer.get<MessageManagerInterface>(ServiceTypes.Messages)
messages.addSuccess('SIP connected')
})
coolPhone.on('newRTCSession', (e: any) => {
const messages = ServiceContainer.get<MessageManagerInterface>(ServiceTypes.Messages)
messages.addAlert('New call')
const session = e.session
session.on('failed', this.resetLocalState)
session.on('ended', this.resetLocalState)
const numberRegexp = /\"(\d+)\"/
const fromNumber = (numberRegexp.exec(e.request.headers.From[0].raw))[1]
const toNumber = (numberRegexp.exec(e.request.headers.Contact[0].raw))[1].slice(1)
this.setState({
callReceived: true,
callSession: session,
fromNumber: fromNumber,
toNumber: toNumber,
})
})
coolPhone.start()
Method that handles answer button click:
private answerCall = () => {
const messages = ServiceContainer.get<MessageManagerInterface>(ServiceTypes.Messages)
messages.addSuccess('Call answered')
const callOptions = {
mediaConstraints: {
audio: true, // only audio calls
video: false
},
pcConfig: {
iceServers: [
{ urls: ["stun:stun.l.google.com:19302"] }
],
iceTransportPolicy: "all",
rtcpMuxPolicy: "negotiate"
}
}
this.state.callSession.answer(callOptions)
this.state.callSession.connection.addEventListener('addstream', (event: any) => {
console.log(event)
this.audioElement.srcObject = event.stream
})
this.audioElement.play()
this.setState({
callAnswered: true,
callReceived: false,
})
}
What did I do wrong?
I solved the problem.
The problem was in the position of this.audioElement.play() line.
I moved it to the callback on addstream event:
this.state.callSession.connection.addEventListener('addstream', (event: any) => {
console.log(event)
this.audioElement.srcObject = event.stream
this.audioElement.play()
})
Now it works fine. Hope you also find it useful.
You can use react-sip npm library which simplifies usage of jssip inside React apps:
https://www.npmjs.com/package/react-sip
You will just need to pass your connection settings as props to <SipProvider/>, which will be somewhere near the top of your react tree.
This will allow you to perform basic start/stop/answer operations and watch the status of your call in the context!
How can I check if port is busy for localhost?
Is there any standard algorithm? I am thinking at making a http request to that url and check if response status code is not 404.
You could attempt to start a server, either TCP or HTTP, it doesn't matter. Then you could try to start listening on a port, and if it fails, check if the error code is EADDRINUSE.
var net = require('net');
var server = net.createServer();
server.once('error', function(err) {
if (err.code === 'EADDRINUSE') {
// port is currently in use
}
});
server.once('listening', function() {
// close the server if listening doesn't fail
server.close();
});
server.listen(/* put the port to check here */);
With the single-use event handlers, you could wrap this into an asynchronous check function.
Check out the amazing tcp-port-used node module!
//Check if a port is open
tcpPortUsed.check(port [, host])
//Wait until a port is no longer being used
tcpPortUsed.waitUntilFree(port [, retryTimeMs] [, timeOutMs])
//Wait until a port is accepting connections
tcpPortUsed.waitUntilUsed(port [, retryTimeMs] [, timeOutMs])
//and a few others!
I've used these to great effect with my gulp watch tasks for detecting when my Express server has been safely terminated and when it has spun up again.
This will accurately report whether a port is bound or not (regardless of SO_REUSEADDR and SO_REUSEPORT, as mentioned by #StevenVachon).
The portscanner NPM module will find free and used ports for you within ranges and is more useful if you're trying to find an open port to bind.
Thank to Steven Vachon link, I made a simple example:
const net = require("net");
const Socket = net.Socket;
const getNextPort = async (port) =>
{
return new Promise((resolve, reject) =>
{
const socket = new Socket();
const timeout = () =>
{
resolve(port);
socket.destroy();
};
const next = () =>
{
socket.destroy();
resolve(getNextPort(++port));
};
setTimeout(timeout, 10);
socket.on("timeout", timeout);
socket.on("connect", () => next());
socket.on("error", error =>
{
if (error.code !== "ECONNREFUSED")
reject(error);
else
resolve(port);
});
socket.connect(port, "0.0.0.0");
});
};
getNextPort(8080).then(port => {
console.log("port", port);
});
this is what im doing, i hope it help someone
const isPortOpen = async (port: number): Promise<boolean> => {
return new Promise((resolve, reject) => {
let s = net.createServer();
s.once('error', (err) => {
s.close();
if (err["code"] == "EADDRINUSE") {
resolve(false);
} else {
resolve(false); // or throw error!!
// reject(err);
}
});
s.once('listening', () => {
resolve(true);
s.close();
});
s.listen(port);
});
}
const getNextOpenPort = async(startFrom: number = 2222) => {
let openPort: number = null;
while (startFrom < 65535 || !!openPort) {
if (await isPortOpen(startFrom)) {
openPort = startFrom;
break;
}
startFrom++;
}
return openPort;
};
you can use isPortOpen if you just need to check if a port is busy or not.
and the getNextOpenPort finds next open port after startFrom. for example :
let startSearchingFrom = 1024;
let port = await getNextOpenPort(startSearchingFrom);
console.log(port);