I am consuming a webservice in node js and it returns me the response in this strange format. I have to extract certain values by passing the key names and I would like to convert the response to a JSON for ease of parsing. How should I go about it ?
This is what I get from the webservice
Data Source=*******;Initial Catalog=****;User ID=*******;Password=
*******;MultipleActiveResultSets=True;Min Pool Size=5;Max Pool Size=5000;Conn
ect Timeout=180;Application Name=*****
I want to extract DataSource , UserID, Password.
Dario's answer basically does the trick, but it's worth knowing that Node's querystring module does this sort of stuff "for free".
The code
const qs = require('querystring');
const s = `Data Source=*******;Initial Catalog=****;User ID=*******;Password=*******;MultipleActiveResultSets=True;Min Pool Size=5;Max Pool Size=5000;Connect Timeout=180;Application Name=*****`;
console.log(qs.parse(s, ';', '='));
outputs:
{ 'Data Source': '*******',
'Initial Catalog': '****',
'User ID': '*******',
Password: '*******',
MultipleActiveResultSets: 'True',
'Min Pool Size': '5',
'Max Pool Size': '5000',
'Connect Timeout': '180',
'Application Name': '*****' }
You can easily parse it with String split:
const obj = response.split(';', 2).reduce((json, pair) => {
const tokens = pair.split('=', 2);
json[tokens[0]] = tokens[1];
return json;
}, {});
You can manipulate it, deleting or extracting values, and finally encoding it in JSON with JSON.stringify(obj).
Note: as Oleg V. Volkov comments, if the source string contains delimeters (';' and '=') in key or value, it will don't work.
If the returned pattern is constant, it can be achieved using regexp:
var result = str.match(/Source=(.*);Initial.*ID=(.*);Password=(.*);Multiple/)
And then take your desired values from matched groups.
DataSource = result[1];
UserId = result[2];
Password = result[3];
Related
I have an node-express endpoint that accepts several url parameters as filters. For example
{{baseUrl}}/projects?page=1&name=myProject
The request is then translated to a mongoDB query.
Model:
const projectSchema = mongoose.Schema({
name: {
type: String,
required: true,
},
users: [{ type: mongoose.Schema.Types.ObjectId, ref: 'User' }],
});
const Project = mongoose.model('Project', projectSchema);
const filter = pick(req.query, ['name']);
const options = pick(req.query, ['sortBy', 'limit', 'page']);
options.populate = 'users';
const result = await projectService.queryProjects(filter, options, req.user);
This works fine, but when i use mongoDB filter expressions like "/MyProject/" the filter is passed as a string and not as an expression.
What do i need to change to get the expected result?
#current result
console.log(filter);
#Output: { name: '/MyProject/' }
#expected result
const test = { name: /MyProject/ };
console.log(test);
#Output { name: /MyProject/ }
/MyProject/ (without quotes is a regular expression):
> typeof "/MyProject/"
'string'
> typeof /MyProject/
'object'
> /MyProject/ instanceof String
false
> /MyProject/ instanceof RegExp
true
Now if you want to construct a RegEx from a string you may want to consider using RegEx constructor:
new RegExp("MyProject")
/MyProject/
Or, you can read your filter as a string from your query parameters then use the other way of passing filters to MongoDB with $regex:
const filter = pick(req.query, ['name']);
const result = await projectService.queryProjects({"name": {"$regex": filter, "$options": "i"}, options, req.user);
WARNING
Either way please be careful when using such approaches as they may lead to ReDOS and/or NoSQL injection.
From my understanding, the hash that is encrypted in the Signature by the private key, should match the hash of the payload. I have written a small node program to test out that concept but the hash fails to match even though the verification passed.
Below is what I have tried.
import crypto from 'crypto';
// generate some stub keys for the test program
const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', {
modulusLength: 2048,
publicKeyEncoding: {
type: 'spki',
format: 'pem',
},
privateKeyEncoding: {
type: 'pkcs8',
format: 'pem',
},
});
// the payload to be signed
const payload = 'helloWorld';
// create the signature
const signatureFunction = crypto.createSign('SHA256');
signatureFunction.write(payload);
signatureFunction.end();
const signature = signatureFunction.sign(privateKey, 'base64');
// verify the signature
const verifyFunction = crypto.createVerify('SHA256');
verifyFunction.write(payload);
verifyFunction.end();
// the result is "true" which means my public key can verify the encrypted hash
console.log(verifyFunction.verify(publicKey, signature, 'base64'));
// decrypt and check what is the hash
const decryptedHash = crypto.publicDecrypt(publicKey, Buffer.from(signature, 'base64'));
console.log(`decryptedHash: ${decryptedHash.toString('base64')}`);
// manually generate the hash
const manualHash = crypto
.createHash('SHA256')
.update(payload)
.digest('base64');
console.log(`manualHash: ${manualHash}`);
Below is the output:
true
decryptedHash: MDEwDQYJYIZIAWUDBAIBBQAEIBHU3cNX4IIpaNv9ImtuHCqsAY0HalTaT2Xh3IGAaErD
manualHash: EdTdw1fggilo2/0ia24cKqwBjQdqVNpPZeHcgYBoSsM=
My question is why the decryptedHash does not match the manualHash ?
Ok I manage to find out why.. apparently decryptedHash is not really the actual hash value. I need to go to https://lapo.it/asn1js/ to convert it. I am not very well-verse in that area but I manage to find some clue by looking at this post https://crypto.stackexchange.com/questions/80768/openssl-extract-hash-algorithm-from-signature-data-pkcs1-v-1-5
It seems that the decryptedHash is in DER ANS1 format.
Added 3 more lines to my program and I am able to match the manualHash
const valueGotFromLapoSite = '11D4DDC357E0822968DBFD226B6E1C2AAC018D076A54DA4F65E1DC8180684AC3';
const myHash = Buffer.from(valueGotFromLapoSite, 'hex').toString('base64');
console.log(`myHash: ${myHash}`);
Output:
true
decryptedHash: MDEwDQYJYIZIAWUDBAIBBQAEIBHU3cNX4IIpaNv9ImtuHCqsAY0HalTaT2Xh3IGAaErD
manualHash: EdTdw1fggilo2/0ia24cKqwBjQdqVNpPZeHcgYBoSsM=
myHash: EdTdw1fggilo2/0ia24cKqwBjQdqVNpPZeHcgYBoSsM=
It is a typescript
Can anybody help with the followin:
I read data from CSV file
Transform this data on flight (remove some extra columns)
Then I want updated csv in stream get back to variable in the code.
Console.log(updatedCsv) // in stream - displays what I need
BUT!
When I try to push it into array nothing happens and then variable (in which I pushed data from stream) is considered undefined:
import * as fs from "fs";
import * as csv from "csv";
udateCsv(){
fs.createReadStream('allure-report/data/suites.csv')
.pipe(csv.parse({ delimiter: ',', columns: true }))
.pipe(csv.transform((input) => {
console.log(input) // <----- it shows in console data I needed
/* like this:
{
Status: 'passed',
'Start Time': 'Wed Nov 11 17:37:33 EET 2020',
'Stop Time': 'Wed Nov 11 17:37:33 EET 2020',
'Duration in ms': '1',
'Parent Suite': '',
Suite: 'The Internet Guinea Pig Website: As a user, I can log into the secure area',
'Sub Suite': '',
'Test Class': 'The Internet Guinea Pig Website: As a user, I can log into the secure area',
'Test Method': 'Hook',
Name: 'Hook',
Description: ''
}
*/
skipHeaders.forEach((header) => delete input[header]);
this.rowsArray = input // NOTHING HAPPENS, rowsArray: string[] = new Array(); input - I don't know what is the type or if I use push. I can't get this data out of pipe
return input;
}))
.pipe(csv.stringify({ header: true }))
.pipe(fs.createWriteStream( this.path))
AND ALSO
as a workaround I wanted to read the newly generated csv but it is also unseccesfful, looks like I need to use promises. I tried some example from internet but was fail. PLEASE HELP
For those who wondering - I was able to resolve my goal using the following approach:
BUT!! I still wonder how to handle this problem via Promises, async/await approaches.
class CsvFormatter{
pathToNotUpdatedCsv: string
readline: any
readStream: any
headers: any
fieldSchema: string[] = new Array()
rowsArray: string[] = new Array()
constructor(pathToCsv: string, encoding: string) {
this.pathToNotUpdatedCsv = pathToCsv
this.readStream = fs.createReadStream(this.pathToNotUpdatedCsv, encoding = 'utf8');
}
async updateCsv(){
//read all csv lines of not updated file
this.readline = readline.createInterface({
input: this.readStream,
crlfDelay: Infinity
});
//save them to array
for await (const line of this.readline) {
this.rowsArray.push(line)
}
//remove columns in csv and return updated csv array
this.rowsArray = this.getUpdatedRows()
//separating headers and other rows in csv
this.headers = this.rowsArray.shift()
}
getUpdatedRows(){
let headersBeforeUpdate = this.removeExtraQuotes(this.rowsArray[0])
let rowsAfterUpdate = []
let indexesOfColumnToDelete = []
let partOfUpdatedArray = []
//get indexes which will be used for deletion of headers and content rows
skipHeaders.forEach((header) => {
indexesOfColumnToDelete.push(headersBeforeUpdate.indexOf(header))
})
//delete rows by index
this.rowsArray.forEach(row => {
partOfUpdatedArray = this.removeExtraQuotes(row)
indexesOfColumnToDelete.forEach(index=>{
partOfUpdatedArray.splice(index)
})
rowsAfterUpdate.push(partOfUpdatedArray)
})
return rowsAfterUpdate
}
Let's assume we have a query builder service B that spits out mongo db query when called. This query is received by service A and it executes it as is with mongo db official nodejs driver.
How do I send something like :
[{
_id: new mongo.ObjectID("5f3258cfbaaccedaa5dd2d96"),
phone: "666"
}, {
_id: new mongo.ObjectID("5f3258cfbaaccedaa5dd2da2"),
phone: "555"
}]
from service B to service A?
EDIT:
The following works perfectly fine:
var q = { _id: new mongo.ObjectID("5f3258cfbaaccedaa5dd2d96") };
const result = await this.db.collection("persons").find(q).toArray();
The following doesn't work:
var q = { _id: { $oid: "5f3258cfbaaccedaa5dd2d96" } }
const result = await this.db.collection("persons").find(q).toArray();
Now,
var q = { _id: new mongo.ObjectID("5f3258cfbaaccedaa5dd2d96") };
JSON.stringify(q)
gives you : {"_id":"5f3258cfbaaccedaa5dd2d96"} and if you pass this to service A. You can not use it in service A as follows:
const result = await this.db.collection("persons").find(qStr).toArray();
Or as,
const result = await this.db.collection("persons").find(JSON.parse(qStr)).toArray();
You need to:
Serialize your documents to extended json on one end
Deserialize your documents from extended json to language-native data structures on the other end
See https://github.com/mongodb/js-bson#node-no-bundling for how to serialize and deserialize.
You cannot feed extended json-type annotated hashes to driver functions that expect native types (which are all of them, basically, other than the one that specifically parses extended json), like you tried to do.
var q = { _id: new mongo.ObjectID("5f3258cfbaaccedaa5dd2d96") };
const serializedQ = BSON.serialize(q);
const deserializedQ = BSON.deserialize(serializedQ);
const result = await this.db.collection("persons").find(deserializedQ).toArray();
There is a standard that MongoDB calls "Extended JSON" that defines how you can encode all BSON data types in regular JSON.
It will become something like
{ _id : {$oid: "5f3258cfbaaccedaa5dd2d96"} }
Most MongoDB tools will be able to convert to and from that format.
With this line of code in python I created JSON containing multiple messages:
notification = [{
'message': 'This is the homepage',
'username': 'User',
'room': 'Home',
'timestamp': '11:59'
},
{
'message': 'Hello World!',
'username': 'User',
'room': 'Home',
'timestamp': '12:00'
}]
return render_template("chat.html", messages = notification)
Now I would like to print all the messages on the webpage, so I sent all the information to javascript. But when the output of the following script returns an empty string.
const message = `{{messages}}`;
console.log(message);
Why is the string empty?
What you are passing in the notification variable isn't actually JSON, it's a Python list. You need to use json.dumps to convert it to an actual JSON string.
Then in your JS you will need to wrap the string in quotes (single quotes best as JSON uses double quotes) so that your JS is syntactically valid. You can then even use Javascript's JSON.parse to convert it to a JS array:
const message = '{{messages}}' ;
console.log(JSON.parse(message) );
Try passing the notification as json to the template
Flask
return render_template("chat.html", messages = json.dumps(notification))
Template
const message = JSON.parse('{{messages | tojson | safe}}');
console.log(message)