I have node red successfully running and am feeding it from a couple of different PLC systems,
I have a payload like this.
{"d":779,"dt":3,"ts":"2018-04-05T19:54:12.930758Z","q":18}
The fields would break down to.
d=779 dt=3 ts=2018-04-05T19:54:12.930758Z q=18
I have been attempting to parse the string and break it down (even just one field "d").
I have been trying to write a function to parse the data using combinations of splice and split.
Does anyone have a simple solution to this? Am I even using the correct node process?
I know this is completely not working but it was an attempt the read the first field. It is not padded and therefore needs to start where the colon is and end with the comma.
var str = msg.payload;
var th = str.slice(5,-1);
th = th.split(",");
msg.payload[0] = parseFloat(th[0]);
return [ msg.payload[0]];
Have a source MQTT Buffer of:
[123,34,100,34,58,51,48,54,56,44,34,100,116,34,58,50,44,34,116,115,34,58,34,50,48,49,56,45,48,52,45,49,50,84,48,56,58,53,48,58,49,56,46,49,54,50,48,48,48,90,34,44,34,113,34,58,49,57,50,125,0]
The Zero at the end is obviously the offending value:
I am attempting to remove it with function
var data = msg.payload;
console.log("payload data: length = " + data.length);
data = data.trim();
console.log("trimmed data: length = " + data.length);
msg.payload = JSON.parse(data);
return msg;
Now am getting TypeError: data.trim is not a function
Flow looks like:
enter image description here
Now getting
Thanks,
Kevin
By default most nodes will be outputing a JSON object already so there will be no need to parse it as a string just access the objects values directly e.g.
var d = msg.payload.d;
If the nodes are output String representations of JSON then you should use the built in JSON node to convert the string in msg.payload to a JSON object before accessing the values.
Also you need to return a whole JSON object from a function node, not just a string (or an array of strings)
So if you just want to the pass the d value on it should be:
var d = msg.payload.d;
msg.payload = d;
return d;
Related
I have had a read through of some of the questions but they are either written for a different language or written to be solved in a webpage,
Im using node-red to recieve BLE data but the data comes in 1 long string im recieving the following:
string
[{"timestamp":"2020-02-
24T13:44:57Z","type":"Gateway","mac":"AC233FC02D95","gatewayFree":96,"gatewayLoad":0.08}
{"timestamp":"2020-02-24T13:44:59Z","type":"iBeacon","mac":"DB1265E96B6F","bleName":"","ibeaconUuid":"ACFD065EC3C011E39BBE1A514 932AC01","ibeaconMajor":1,"ibeaconMinor":2,"rssi":-51,"ibeaconTxPower":-54,"battery":0}
{"timestamp":"2020-02-24T13:44:59Z","type":"iBeacon","mac":"F0728DEB0B9B","bleName":"","ibeaconUuid":"ACFD065EC3C011E39BBE1A514932AC01","ibeaconMajor":5,"ibeaconMinor":1,"rssi":-30,"ibeaconTxPower":-62,"battery":0}]
my goal is to input the data into node-red in the following format
Gateway-Mac,IbeaconMinor,mac,rssi
i did previously use i different collector to gather data and it gave me the string in the following format
$GPRP,AC233FC02D95,FE2279DCB92B,-70,0201061AFF4C000215ACFD065EC3C011E39BBE1A514932AC0100050003C2
which inputted collected BLE device DATA seperatly rather than in one long JSON string so it was easier to convert and input.
so my question is how can i either change it from
string
[{"timestamp":"2020-02-
24T13:44:57Z","type":"Gateway","mac":"AC233FC02D95","gatewayFree":96,"gatewayLoad":0.08}
{"timestamp":"2020-02-24T13:44:59Z","type":"iBeacon","mac":"DB1265E96B6F","bleName":"","ibeaconUuid":"ACFD065EC3C011E39BBE1A514 932AC01","ibeaconMajor":1,"ibeaconMinor":2,"rssi":-51,"ibeaconTxPower":-54,"battery":0}
{"timestamp":"2020-02-24T13:44:59Z","type":"iBeacon","mac":"F0728DEB0B9B","bleName":"","ibeaconUuid":"ACFD065EC3C011E39BBE1A514932AC01","ibeaconMajor":5,"ibeaconMinor":1,"rssi":-30,"ibeaconTxPower":-62,"battery":0}]
to
string
[{"timestamp":"2020-02-
24T13:44:57Z","type":"Gateway","mac":"AC233FC02D95","gatewayFree":96,"gatewayLoad":0.08}
{2020-02-24T13:44:59Z,DB1265E96B6F,ACFD065EC3C011E39BBE1A514932AC01",1,2,-51,-54,0}
{2020-02-24T13:44:59Z,F0728DEB0B9B,ACFD065EC3C011E39BBE1A514932AC01,5,1,-30,-62,0}]
or how can i input it into mysql in the following format
as mysql string
AC233FC02D95,F0728DEB0B9B51,1,-30
and
AC233FC02D95,DB1265E96B6F,2,-51
before changing hardware i inputted the string like so but with the new json format im not sure what to change in the code
GPRP,AC233FC02D95,FE2279DCB92B,-70,0201061AFF4C000215ACFD065EC3C011E39BBE1A514932AC0100050003C2
var raw = msg.payload;
msg.length = raw.length;
msg.raw = raw;
var data = {};
data.major = raw.slice(-10,-4);
data.mac = raw.substring(5,17);
data.hostname = raw.substring(18,30);
data.minor = raw.slice(-8,-4);
data.rssi = raw.substring(31,34)
var str = data.hostname;
var location = str.replace(`AC233FC02D95`, `PURPLE`)
var beacon = parseInt(data.minor ,16);
var msg = {
topic : "INSERT INTO `test`.`track` (`location`,`beacon`, `mac`,`rssi`)
VALUES ('"+location+"','"+beacon+"', '"+data.mac+"','"+data.rssi+"');"
}
return msg;
Just pass the msg with the string payload through the JSON node, it will parse it to an JSON object, you can then access the fields as normal.
Though from what you've posted the array entries look to be missing , separators.
From there you can build up what ever strings you want or insert the fields directly into the database.
The server returns me a JSON string like this:
{"payload":{"data":"{\"notification_type\":\"{\"type1\":\"{\"type2\":\"type2 value\"}\"}\"}"}}
This string, as I understand, cannot be parsed with JSON.parse() API because the nested JSON strings within the string should be properly escaped. If the string is not properly escaped I get the below error:
Uncaught SyntaxError: Unexpected token n in JSON at position 22
So, the string should be properly escaped respecting the nested nature like below so that JSON.parse() can process it:
var properString = "{\"payload\":{\"data\":\"{\\\"notification_type\\\":\\\"{\\\\\\\"InternalKey\\\\\\\":\\\\\\\"InternalValue\\\\\\\"}\\\"}\"}}";
console.log("Proper String = ");
console.log(properString);
var firstLevelObject = JSON.parse(properString);
console.log("First Level Object = ");
console.log(firstLevelObject);
var secondLevelObject = JSON.parse (firstLevelObject.payload.data);
console.log("Second Level Object = ");
console.log(secondLevelObject);
var thirdLevelObject = JSON.parse(secondLevelObject.notification_type);
console.log("Third Level Object = ");
console.log(thirdLevelObject);
This is how google chrome's console outputs the same:
I however am unable to convert the improper string from server to properly escaped string as defined in the variable properString so that the JSON is properly constructed which can be traversable. How can the string be converted with proper escape characters?
Reference sources:
I have referred to this answer to understand how nested escape characters should be added, but the answer does not state how conversion can be done.
Actually, the JSON string what you get is not exactly right. It's not a right JSON syntax in the nested JSON String. But you can rebuilt it to a correct JSON syntax string via regular expression or native JS.
var obj = '{"payload":{"data":"{\"notification_type\":\"{\"type1\":\"{\"type2\":\"type2 value\"}\"}\"}"}}';
var payloadString = obj.substring(0, 20).concat('"}}');
obj = obj.substring(20, obj.length - 3).replace("\"{", "{").replace("}\"}", "}}").replace("}\"}", "}}").replace("\"{", "{");
var data = JSON.parse(obj);
var obj2 = JSON.parse(payloadString);
//console.log(payload);
//console.log(data);
obj2.payload.data = data;
console.log(obj2);
I have the following json object samples, i.e.:
{"id":"value1" , "time":"valuetime1"}
{"id":"value2" , "time":"valuetime2"}
{"id":"value3" , "time":"valuetime3"}
{"id":"value4" , "time":"valuetime4"}
{"id":"value5" , "time":"valuetime5"}
{"id":"value6" , "time":"valuetime6"}
{"id":"value7" , "time":"valuetime7"}
{"id":"value8" , "time":"valuetime8"}
Based on the above, I would like to add all these json objects to an array, where I can then process and access the array for each json object and extract id1 value together with time1 value and so forth.
I believe I have to use JSON.parse but unsure of how to firstly add these objects to an array and then be able to also parse and access each object's data.
Think of each row above as a separate JSON object which I would like added to an array that is parsed.
Read the file line by line (each line is a separate json)
Parse the line json text to JavaScript object
Put/push/append JavaScript object into a JavaScript array
This is not valid JSON, so JSON.parse would not work, but assuming you have the text in a string variable, you can convert it to JSON with something like:
data = "[" + data.replace(/\}/g, "},").replace(/,$/,"") + "]";
and then parse it.
UPDATE:
var array = [];
// for each input line
var obj = JSON.parse(line);
array.push(obj);
var id = obj.id;
var time = obj.time;
...
Appreciate the responses but I believe I have managed to solve my own question.
Furthermore, as #Mr. White mentioned above, I did have my property names in error which I have now sorted - thanks.
Using the data above, all I was after was the following:
UPDATED
var s = '{"id":"value1","time":"valuetime1"},{"id":"value2","time":"valuetime2"},{"id":"value3","time":"valuetime3"},{"id":"value4","time":"valuetime4"},{"id":"value5","time":"valuetime5"}, {"id":"value6","time":"valuetime6"},{"id":"value7","time":"valuetime7"},{"id":"value8","time":"valuetime8"}';
var a = JSON.parse('[' + s + ']');
a.forEach(function(item,i){
console.log(item.id + ' - '+ item.time);
});
Thanks for the heads up #torazaburo - have updated my answer which I should've done much earlier.
I'm using Socket.IO to move data to the browser. The data sent is a stream of JSON objects, and when it arrives at the browser, it becomes one large string of JSON. The problem is, this JSON can't be parsed by JSON.parse() because it's not "real" JSON.
The data structure can be arbitrary so a RegEx might not do the trick. And this current setup is only temporary. Eventually this stream of JSON will be pre-processed server-side so a stream will not need to be sent to the browser, so I'd like to keep the AJAX/Socket.IO setup I have right now instead of switching over to a JSON stream parser like OboeJS.
What can I do to parse this string of concatenated JSON?
For clarity, the JSON will look like this:
{"a":"A"}{"b":"B"}{"c":"C"}
And I'm trying to parse it in such a way that I can access them like:
console.log(Object.a) //A
console.log(Object.b) //B
console.log(Object.c) //C
In your particular case, you could use Array.prototype.reduce to merge all JSON objects into one:
var unstructuredJson = '{"a":"A"}{"b":"B"}{"c":"C"}';
var jsonArray = "[" + unstructuredJson.split("}{").join("},{") + "]";
var objectArray = JSON.parse(jsonArray);
var result = objectArray.reduce(function(result, item) {
Object.keys(item).forEach(function(propertyName) {
result[propertyName] = item[propertyName];
});
return result;
}, {});
document.body.textContent = JSON.stringify(result);
OP said in some comment:
[...] Each JSON might have nested data like {{}{}}{}{}{}
Then, above approach is broken on this case.
My two cents is that you should put some separator character when you stream these JSON objects and life will be easier: you'll be able to split parent objects easily and you'll just need to change split("}{") with the whole separator.
I suggest you that you use some character that will never happen as part of any property value. You can find a control character on this Wikipedia article: Control character
If each substring is valid JSON but the concatenated part isn't, you can turn the string into a valid array literal and use JSON.parse, then you can process each object using Array methods, e.g. forEach:
var s = '{"a":"A"}{"b":"B"}{"c":"C"}';
var obj = JSON.parse('[' + s.replace(/}{/g,'},{') + ']').forEach(function (obj) {
document.write(JSON.stringify(obj) + '<br>');
});
Or if you want it as a single object:
var s = '{"a":"A"}{"b":"B"}{"c":"C"}';
var obj = JSON.parse(s.replace(/}{/g,','));
document.write(JSON.stringify(obj));
You can use JsonParser to parse concatenated json objects:
ObjectMapper mapper = new ObjectMapper();
JsonFactory factory = new JsonFactory(mapper);
JsonParser parser = factory.createParser(YourJsonString);
List<YourObject> YourObjectList = new ArrayList<>();
Iterator<YourObject> iterator = parser.readValuesAs(YourObject.class);
while(iterator.hasNext()) {
YourObject yourObject = iterator.next();
loginSignalsList.add(yourObject);
}
Split the large string {a:'A'}{b:'B'}{c:'C'} and parse them individually
I have a string: items[0].name that I want to apply to a JSON object: {"items":[{"name":"test"}]} which is contained in the variable test. I want to apply that string to the object in order to search it (test.items[0].name). I can only think of one way to do this: parse the square brackets and dots using my own function. Is there another way I can do this? Perhaps using eval? (Even though I'd LOVE to avoid that...)
For clarity:
I have a JSON object, what is inside of it is really irrelevant. I need to be able to query the object like so: theobject.items[0], this is normal behaviour of a JSON object obviously. The issue is, that query string (ie. items[0]) is unknown - call it user input if you like - it is literally a string (var thisIsAString = "items[0]"). So, I need a way to append that query string to theobject in order for it to return the value at theobject.items[0]
function locate(obj, path) {
path = path.split('.');
var arrayPattern = /(.+)\[(\d+)\]/;
for (var i = 0; i < path.length; i++) {
var match = arrayPattern.exec(path[i]);
if (match) {
obj = obj[match[1]][parseInt(match[2])];
} else {
obj = obj[path[i]];
}
}
return obj;
}
var name = locate(test, 'items[0].name');
...JSON doesn't have objects, it's just a string.
If you're dealing with an object (ie: you can reference it using dot/bracket notation) then it's just a JavaScript object/array...
So depending on what the deal is, if you're dealing with a 100% string:
'{"name":"string","array":[0,1,2]}'
Then you need to send it through JSON.parse;
var json_string = '{"name":"string","array":[0,1,2]}',
js_obj = JSON.parse(json_string);
js_obj.name; // "string"
js_obj.array; // [0,1,2]
js_obj.array[1]; // 1
If it's not a string, and is indeed an object/array, with other objects/arrays inside, then you just need to go:
myObj.items[0].name = items[0].name;
If it IS a string, then .parse it, and use the parsed object to do exactly what I just did.
If it needs to be a string again, to send to the server, then use JSON.stringify like:
var json_string = JSON.stringify(js_obj);
Now you've got your modified JSON string back.
If you need to support GhettoIE (IE < 8), then download json2.js from Douglas Crockford, and add that script on the page conditionally, if you can't find window.JSON.