HMAC-SHA256 in Python is producing different output to Nodejs - javascript

I have a python script which uses the HMAC-hashlib.sha256 algorithm to generate a signature to authenticate over API.
# ZenEmu
import base64
import hashlib
import hmac
import json
from datetime import datetime
import requests
data = json.dumps(
{
"version": "1",
"id": "284512",
"created": "2023-01-17T02:26:29Z",
"updated": "2023-01-17T02:26:29Z",
"url": "https://hitchhikerio.zendesk.com/agent/tickets/284512",
"type": "Question",
"status": "Closed",
"priority": "Low",
"title": "[TEST4] Please Ignore!",
"description": "first comment",
"tags": "test",
"requester_email": "cedric.damian#gmail.com",
"assignee_email": "admin.castle#gmail.com",
"latest_comment": {
"id": "6586310212693",
"is_public": "true",
"author_email": "admin.castle#gmail.com",
"body": "second reply!",
},
}
).encode('utf-8')
sig = base64.b64encode(
hmac.new(
"secret".encode("utf-8"),
"2022-10-10T04:42:00Z".encode("utf-8") + data,
hashlib.sha256,
).digest()
)
print(sig)
This python file pirnts this result as the signature (sig) = b'DJXG/w1ARbM2ZOQ3pamMEkNY7qSJXpjbQMsARyWCr0Y='
I have tried converting this to Javascript with the help of my colleagues and referring to similar questions and resources online to no avail
This is my Javascript file
const crypto = require('crypto');
const utf8 = require('utf8')
const data = utf8.encode(JSON.stringify({
version: "1",
id: "284512",
created: "2023-01-17T02:26:29Z",
updated: "2023-01-17T02:26:29Z",
url: "https://hitchhikerio.zendesk.com/agent/tickets/284512",
type: "Question",
status: "Closed",
priority: "Low",
title: "[TEST4] Please Ignore!",
description: "first comment",
tags: "test",
requester_email: "cedric.muuo#tradecore.com",
assignee_email: "admin.castle#tradecore.com",
latest_comment: {
id: "6586310212693",
is_public: "true",
author_email: "admin.castle#tradecore.com",
body: "second reply!",
},
}));
console.log(data);
const sig = crypto
.createHmac('sha256', utf8.encode('secret'))
.update(utf8.encode("2022-10-10T04:42:00Z") + data)
.digest('base64')
console.log(sig);
which prints the output Yqi7WvMPxdSlo5Vb9YlcbX2zu5aQpungXhuCYx3bc+4=
How do I get the Javascript code to generate the same output as the python script? (b'DJXG/w1ARbM2ZOQ3pamMEkNY7qSJXpjbQMsARyWCr0Y=')
EDIT: THE PYTHON FILE SHOULD STAY UNCHANGED, IT PRODUCES THE CORRECT OUTPUT.

Your JSON strings aren't identical. Python outputs the following:
'{"version": "1", "id": "284512", "created": "2023-01-17T02:26:29Z", "updated": "2023-01-17T02:26:29Z", "url": "https://hitchhikerio.zendesk.com/agent/tickets/284512", "type": "Question", "status": "Closed", "priority": "Low", "title": "[TEST4] Please Ignore!", "description": "first comment", "tags": "test", "requester_email": "cedric.damian#gmail.com", "assignee_email": "admin.castle#gmail.com", "latest_comment": {"id": "6586310212693", "is_public": "true", "author_email": "admin.castle#gmail.com", "body": "second reply!"}}'
Node outputs the following:
'{"version":"1","id":"284512","created":"2023-01-17T02:26:29Z","updated":"2023-01-17T02:26:29Z","url":"https://hitchhikerio.zendesk.com/agent/tickets/284512","type":"Question","status":"Closed","priority":"Low","title":"[TEST4] Please Ignore!","description":"first comment","tags":"test","requester_email":"cedric.muuo#tradecore.com","assignee_email":"admin.castle#tradecore.com","latest_comment":{"id":"6586310212693","is_public":"true","author_email":"admin.castle#tradecore.com","body":"second reply!"}}'
Notice how the Node version has no whitespace.
EDIT:
To fix it, you can use the Python function as follows:
json.dumps(your_data_object_here, separators=(",",":"))
Edit again:
To insert spaces into the Node version you can try the following:
JSON.stringify(obj).replace(/("[^"]+"[:,])/g, "$1 ");
That should insert spaces after all the : and ,, like in Python. Note that it does not handle escaped quote marks inside the strings. You'll need to update the Regex to deal with that.
Final edit:
Here's an alternate JS approach:
const data = JSON.stringify(obj, null, 1).match(/(\S.*)/gm).join("");
This constructs a "pretty-printed" JSON string with newlines and everything, then grabs each line starting at the first non-whitespace and joins them all up. That strips newlines and indentation, but preserves all whitespace between separators and inside strings. Should be sufficient.

Related

ERROR: invalid character ' ' in literal true (expecting 'e') - POST API JSON

I am trying to use the POST method to insert some data from a person with JSON. I am using the code from JS to construct, but when i start the transformation, it sends me "ERROR: invalid character ' ' in literal true (expecting 'e')". Does anyone know how to solve it?
const obj = {
"num_matricula": num_matricula,
"limit_date": "2022-05-20",
"admission_date": admission_date,
"cost_center": cost_center,
"pos_number": pos_number,
"role": role,
"department": department,
"pagamento": {
"vinculo": vinculo,
"valor": valor,
"recorrencia": recorrencia,
"contaBancaria": {
"banco": "001",
"carta": "c9160763-db6c-4e8c-a1ad-ad8709c99be2"
}
},
"deficiencia": deficiencia,
"jornada": jornada,
"profile": {
"name": name,
"email": email,
"mobile": mobile
},
"exame": {
"clinica": "6dc84ce4-7d9f-48ec-b9b1-a8a895a21fd4",
"data": "2022-05-15",
"hora": "14:00",
"obs": "Comparecer de manhΓ£",
"guia": "e37dab24-c7a4-4b92-b9d1-32ed538b8300",
},
"docs": ["c9e26093-5e0c-4bd2-bea3-ac5182a6179f"],
"send_sms": true,
"send_email": true
};
const myJSON = JSON.stringify(obj);
Some columns are already provided with data from previous step (you can see in the images below), that is why i just repeated the column name in the JS code. Just to let you know, the boolean types of data are the columns: send_email, send_sms and deficiencia.
The problem is that JSON is a string. So in your first line you see this is not valid json: "num_matricula": num_matricula,
only numbers can be without double quotes: "num_matricula": 1234,

Javascript parse JSON error, but works fine in validator

When I try to parse this JSON (Discord webhook):
{
"content": "this `supports` __a__ **subset** *of* ~~markdown~~ πŸ˜ƒ ```js\nfunction foo(bar) {\n console.log(bar);\n}\n\nfoo(1);```",
"embed": {
"title": "title ~~(did you know you can have markdown here too?)~~",
"description": "this supports [named links](https://discordapp.com) on top of the previously shown subset of markdown. ```\nyes, even code blocks```",
"url": "https://discordapp.com",
"color": 16324973,
"timestamp": "2018-12-18T09:22:12.841Z",
"footer": {
"icon_url": "https://cdn.discordapp.com/embed/avatars/0.png",
"text": "footer text"
},
"thumbnail": {
"url": "https://cdn.discordapp.com/embed/avatars/0.png"
},
"image": {
"url": "https://cdn.discordapp.com/embed/avatars/0.png"
},
"author": {
"name": "author name",
"url": "https://discordapp.com",
"icon_url": "https://cdn.discordapp.com/embed/avatars/0.png"
},
"fields": [
{
"name": "πŸ€”",
"value": "some of these properties have certain limits..."
},
{
"name": "😱",
"value": "try exceeding some of them!"
},
{
"name": "πŸ™„",
"value": "an informative error should show up, and this view will remain as-is until all issues are fixed"
},
{
"name": "<:thonkang:219069250692841473>",
"value": "these last two",
"inline": true
},
{
"name": "<:thonkang:219069250692841473>",
"value": "are inline fields",
"inline": true
}
]
}
}
Using this code:
var parsed = JSON.parse(req.body)
I get this error:
SyntaxError: Unexpected token o in JSON at position 1
But if I use a website such as
https://jsonformatter.curiousconcept.com
To validate the JSON, it says the JSON is valid.
What is wrong here?
UPDATE
I'm using an express server to simulate discord server, so it sends web hooks to the express server instead, I get the JSON using req.body.
This happens because JSON is a global object (it's the same object where you read the method parse!), so when you invoke JSON.parse(JSON) javascript thinks you want to parse it.
The same thing doesn't happen when you pass the variable to the validator, because it will be assigned to a local variable:
let JSON = "{}";
validate(JSON);
function(x) {
JSON.parse(x); // here JSON is again your global object!
}
EDIT
According to your updated question, maybe it happens because you already use bodyParser.json() as middleware, and when you use it, req.body is already an object and you don't need to parse it again.
Trying to parsing an already parsed object will throw an error.
It would be something like without using JSONStream:
http.request(options, function(res) {
var buffers = []
res
.on('data', function(chunk) {
buffers.push(chunk)
})
.on('end', function() {
JSON.parse(Buffer.concat(buffers).toString())
})
})
For using it with JSONStream, it would be something like:
http.request(options, function(res) {
var stream = JSONStream.parse('*')
res.pipe(stream)
stream.on('data', console.log.bind(console, 'an item'))
})
(OR)
Here is the Some Steps for this issue..
You Can use lodash for resolving this.
import the lodash and call unescape().
const _ = require('lodash');
let htmlDecoder = function(encodedStr){
return _.unescape(encodedStr);
}
htmlDecoder(JSON);

How to get out of this parse error in json?

var bio = {
"name": "Belphy Baby",
"role": "student",
"contacts": {
"mobile": "9567166100",
"email": "belphy16#gmail.com",
"github": "belphy16",
"location": "kottayam"
},
"welcome message": "and miles to go before i sleep",
"skills": ["leadership", "enthusiastic", "learning"],
"bioPic": "images/fry.jpg"
}
$("#main").append(bio);
Results in error:
Parse error on line 1:
var bio = { "name":
^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
Input should be just a JSON at JSONLint.com
{
"name": "Belphy Baby",
"role": "student",
"contacts": {
"mobile": "9567166100",
"email": "belphy16#gmail.com",
"github": "belphy16",
"location": "kottayam"
},
"welcome message": "and miles to go before i sleep",
"skills": ["leadership", "enthusiastic", "learning"],
"bioPic": "images/fry.jpg"
}
EDIT:
Answer is based on OP's first revision. Note the image he provided.
The error he's getting is coming from the misuse of the site's JSON validation service.
As the service's name suggests, it validates a JSON file's syntax.
Our OP's code contains hints of JavaScript. If inputted at JSONLint, it will give us his error; stating that the should-be JSON input has an error.

How do I replace json property value in multiple places using JavaScript in Apigee

I need to replace multiple occurrence of property value in a json request using JavaScript. I have tried this in JSFiddle and it worked but the same code in an Apigee JavaScript policy is not replacing the value.
I have json data as follows:
[
{
"Name": "app1",
"groups": [
{
"desc": "this is a test group",
"id": "test1",
"name": "test grp45"
},
{
"desc": "this is another test group",
"id": "test2",
"name": "test group 2"
}
],
"id": "1"
},
{
"Name": "app2",
"groups": [
{
"desc": "this is a test group",
"id": "test3",
"name": "test group 4"
},
{
"desc": "this is another test group",
"id": "test4",
"name": "test group 4"
}
],
"id": "2"
},
{
"Name": "app3",
"groups": [
{
"desc": "this is a test group",
"id": "test5",
"name": "test group 5"
},
{
"desc": "this is another test group",
"id": "test6",
"name": "test group 6"
}
],
"id": "3"
}
]
Here's what I have tried:
var val = context.getVariabl("request.content");
context.setVariable("val", val);
function findAndReplace(val1, value, replacevalue) {
for (var x in val1) {
if (typeof val1[x] == typeof {}) {
findAndReplace(val1[x], value, replacevalue);
}
if (val1[x] == value) {
val1["name"] = replacevalue;
//break; // uncomment to stop after first replacement
}
}
}
findAndReplace(val, "test1", "img");
var result = JSON.stringify(val);
var obj = JSON.parse(result);
context.setVariable("response.content", obj);
I want to replace the value of "test1" to "img".
First, you're setting response.content with the parsed obj. You'd want:
context.setVariable("response.content", result);
...instead, because the flow variable needs to be a string, not a JavaScript object.
Second, you're getting request.content and then setting response.content. You probably only want one or the other, especially considering that likely this policy is attached to the request OR the response flow, not both (you can't set the request in the response flow, and content.response would be overwritten by the target response).
Use the Apigee trace tool to see where in the flow your policy is executing, and to inspect the variables being set -- this will help you figure out what you need to fix.
As suggested above you have a couple of things to do to get this to work.
First I would suggest you use two policies. The javascript sample above would be called in the request flow - the one above would store a stringified version of the object into an intermediate flow variable say "foo" as below:
var val = JSON.parse(context.getVariable("request.content"));
findAndReplace(val, "test1", "img");
context.setVariable("foo",JSON.stringify(val));
In the response flow you can then use an assign message policy to insert "foo" into the response. As below:
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<AssignMessage async="false" continueOnError="false" enabled="true" name="assignFoo">
<DisplayName>assignFoo</DisplayName>
<FaultRules/>
<Properties/>
<Set>
<Headers/>
<Payload contentType="application/json; charset=utf-8">{foo}</Payload>
<StatusCode>200</StatusCode>
<ReasonPhrase>OK</ReasonPhrase>
</Set>
<AssignTo createNew="false" transport="http" type="response" />
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</AssignMessage>
Use string replace()
This converts the JSON into string
JSON.stringify(json).replace("test1", "img")
and then you could convert it back to JSON with replaced values
JSON.parse(json)

Trouble parsing json with node.js

I wrote a quick script to parse two fairly large json file (~17k records) to do a comparison of the two. I have confirmed they are both valid json (via jsonlintpro) and the same format. (The source is the same so this should be a given. But, I always assume the mistake is mine. And I still do. Just somewhere else.) However, the parsed file just outputs [object, Object]. I'm wondering what the cause could possibly be?
The json format is like this small snippet (anonymized of course):
[
{
"id": "1234",
"name": "Name1",
"url": "https://localhost/Name1",
"date_created": "2013-07-05T18:47:05Z",
"date_cancelled": "",
"props": [
{
"id": "54321",
"type": "Client",
"value": "General Store"
},
{
"id": "65432",
"type": "Contact_Name",
"value": "Joe Smith"
}
]
},
{
"id": "23456",
"name": "Name2",
"url": "https://localhost/Name2",
"date_created": "2014-02-27T17:46:43Z",
"date_cancelled": "",
"props": [
{
"id": "34567",
"type": "Client",
"value": "Bait Shop"
}
]
}]
And here is the pertinent code:
var _ = require('underscore');
var recs = require('./prod.json');
printArr(recs);
console.log(recs.length);
function printArr(arr) {
arr.forEach(function(item) {
console.log(item + ", ");
});
}
Any guidance would be greatly appreciated.
UPDATE:
Ok, so apparently the issue is with my printArr function. I'm not sure what I'm doing wrong there. I'd like to figure it out because I want to expand upon that so I can print selectively.
the parsed file just outputs [object, Object].
This is the expected behavior BECAUSE you are concatenating an object with a string.
Try console.log(item) instead
console.log(item); should indeed print [object, Object], did you try to output its properties instead?
function printArr(arr) {
arr.forEach(function(item) {
console.log( item.id, item.name, item.url, item.date_created, item.date_cancelled, item.props, ';');
});
}
Just export the value from the prod.json file.
prod.json file
module.exports = [
{
"id": "1234",
"name": "Name1"
},
{
"id": "1234",
"name": "Name1"
}]
elsewhere
var recs = require('./prod.json')
console.log(recs)

Categories

Resources