xml2js valueProcessor removing \t and \n - javascript

I have a problem with parsing an XML file.
I want to remove strings with characters like \t\n.
XML File: http://ftp.thinkimmo.com/home/immoanzeigen24/immo.xml
{
trim: true,
normalize: true,
attrValueProcessors: [cleanValue, name => name],
valueProcessors: [cleanValue, name => name]
}
cleanValue:
const cleanValue = value => {
return value.toString().trim().replace("\t","atest");
};
I tried cleaning it with a lot of regex I've found online - but value always stays like following:
"verwaltung_objekt": {
"objektadresse_freigeben": "0",
"verfuegbar_ab": "nachaasjkdhkjshadjkashdAbsprache",
"bisdatum": "2016-01-15",
"min_mietdauer": "\n\t\t\t\t",
"max_mietdauer": "\n\t\t\t\t",
}

This is a difficult one!
I'd suggest following a simple strategy and pre-processing the xml data before you parse it.
This should resolve your issue at least.
If you just do something like:
function trimXml(xml) {
return xml.replace(/>\s+</g, "><");
}
xml = trimXml(xml);
Then parse the trimmed xml data. You should see the output now looks like so:
"verwaltung_objekt": [
{
"objektadresse_freigeben": [
"1"
],
"abdatum": [
"2017-03-01"
],
"min_mietdauer": [
""
],
"max_mietdauer": [
""
]
}
],
Which is a bit more like what you want!

Related

Javascript export excel fill top column

I created a code that can export a file via excel the code is working fine without error but my problem is the excel file has a lot of spaces can.
as you can see in the image row 123 data is put in column 7 not in column
heres my code for exporting the data
export function download() {
var header = [];
var finalData = []
var group = [
{ "group_name": "123" },
{ "group_name": "123b" },
{ "group_name": "123ef" },
{ "group_name": "Accounts Payable" },
{ "group_name": "ADG JET TEAM" },
{ "group_name": "001 Approval" }
]
var member = [
{"001 Approval": "083817 - Ranjeet Kumar (ranjeet.kumar3#concentrix.com)" },
{ "001 Approval": "C01747 - Abid Shaikh (abid.shaikh1#concentrix.com)"},
{ "001 Approval": "C01747 - Abid Shaikh (abid.shaikh1#concentrix.com)"},
{ "123b": "C01747 - Abid Shaikh (abid.shaikh1#concentrix.com)"},
{
"123ef": "C01747 - Abid Shaikh (abid.shaikh1#concentrix.com)"
}
]
group.forEach(data=>{
header.push(data.group_name)
})
finalData.push(header)
header.forEach(headerData=>{
var temp = []
member.forEach(memberData=>{
if (headerData === Object.keys(memberData)[0]){
temp.push(memberData[Object.keys(memberData)[0]])
}else{
temp.push("")
}
})
finalData.push(temp)
})
exportToCsv('export.csv', finalData)}
the exportToexcel code is from here https://jsfiddle.net/jossef/m3rrLzk0/
Open your CSV in a text editor and and check the output, what are you using as separator and delimiter. This can be a CSV bad format (fields not surrounded by quotes, for ex), or a problem with excel configuration (csv delimiters and separators).
If you have data like that
field1, field2, field 3 supercool, this is a phrase, ops
It can be a problem, it should be something similar to
"field1", "field2", "field 3 supercool", "this is a phrase, ops"
In addition, try to open your csv with Google Sheets (docs), which will try to recognize the delimiters and separators automatically. See if it works.
A common problem for that is that your CSV is saparated by spaces or commas, but phrase can have a space and a comma which will be interpreted as a separator, and the whole document will break.
It can be useful to take a look at that link: Write a string containing commas and double quotes to CSV

Stop babel parse and generate reformatting output code

I'm using #babel/parser to read in javascript, modify it and regenerate it with babel-generator. I need to do this while retaining the javascript's formatting but it's re-formatting it. I can't see anything else in the options for the parser or generator or docs that would stop it from doing this so I'm a bit lost. Here's what I have so far.
import { parse } from '#babel/parser';
import generate from 'babel-generator';
var file = parse(script, {
sourceType: 'module',
ranges: true,
tokens: true,
plugins: ['nullishCoalescingOperator', 'typescript']
});
var script = generate(file, {
retainLines: true,
comments: true
});
console.log(script.code);
The test script I am passing it is below;
import This from 'that'
import That from './this'
export default (() => {
return Object.assign(
{},
This
{
That
}
)
})()
However generate is returning the script below;
import This from 'that'
import That from './this'
export default (() => {
return Object.assign(
{},
This
{
That });
})();
As you can see it's changing the number of spaces in a return, collapsing the end }) of the Object.assign and adding semi-colons. It will be reading in script from lots of different formats and styles and I can't change it and ideally don't want to have to provide it with an es-lint file, is there anyway to get it to print the code back out with the exact same formatting?
EDIT: the ast nodes contain the loc and range vales but it seems to be ignoring it. I'm not even sure if it's parsing it incorrectly or if the generate is ignoring it.
"loc": {
"start": {
"line": 10,
"column": 8
},
"end": {
"line": 10,
"column": 10
}
},
"range": [
319,
321
],
you need use jscodeshift, instead of babel.This library is more suitable for this scenario.
https://github.com/facebook/jscodeshift

Remove unwanted columns from CSV file using Papaparse

I have a situation where a user can upload a csv file. This CSV file contains a lot of data, but I am only interested in 2 columns (ID and Date). At the moment, I am parsing the CSV using Papaparse
Papa.parse(ev.data, {
delimiter: "",
newline: "",
quoteChar: '"',
header: true,
error: function(err, file, inputElem, reason) { },
complete: function (results) {
this.parsed_csv = results.data;
}
});
When this is run this.parsed_csv represents objects of data keyed by the field name. So if I JSON.stringify the output is something like this
[
{
"ID": 123456,
"Date": "2012-01-01",
"Irrelevant_Column_1": 123,
"Irrelevant_Column_2": 234,
"Irrelevant_Column_3": 345,
"Irrelevant_Column_4": 456
},
...
]
So my main question is how can I get rid of the columns I dont need, and just produce a new csv containing the columns ID and Date?
Thanks
One thing I realised, is there a way to add dynamic variables. For instance I am letting users select the columns I want to map. Now I need to do something like this
let ID = this.selectedIdCol;
this.parsed_csv = results.data.map(element => ({ID: element.ID, Date: element.Date}));
It is saying that ID is unused however. Thanks
let data = [
{
"ID": 123456,
"Date": "2012-01-01",
"Irrelevant_Column_1": 123,
"Irrelevant_Column_2": 234,
"Irrelevant_Column_3": 345,
"Irrelevant_Column_4": 456
},
...
]
just produce results by using the following code:
data = data.map(element => ({ID: element.ID, Date: element.Date}))
Now you have desired column, please generate a new CSV on these columns
As Serrurier pointed out above, You should use the step/chunk function to alter the data rather than after parse map as in memory data is already available.
PapaParse.parse(file, { skipEmptyLines: true, header: true, step: (results, parser) => {
results.data = _.pick(results.data , [ 'column1' 'column2']);
return results;
}});
Note that if you are loading a huge file, you will have the whole file in memory right after the parsing. Moreover it may freeze the browser due to the heavy workload. You can avoid that by reading and discarding columns :
row by row
chunk by chunk.
You should read Papaparse's FAQ before implementing that. To sum up, you will store required columns by extracting them from the step or chunk callbacks.

How can I build a compound query with SearchBox in searchkit?

I'm using searchkit to try to build a basic text search. I think the query I want to build is fairly simple. It needs to be structured like this:
{
"query":{
"bool":{
"must":[
{
"multi_match":{
"query":"test search",
"type":"phrase_prefix",
"fields":[
"field_1^5",
"field_2^4",
"field_3"
]
}
},
{
"term":
{
"field_id": "3"
}
}
],
"must_not":[
{
"term":
{
"status": "archived"
}
}
]
}
},
"size":6,
"highlight":{
"fields":{
"field_1":{},
"field_2":{},
"field_3":{}
}
}
}
I've tried using the prefixQueryFields attribute, which gave me something fairly close to what I wanted except it was using a BoolShould rather than a BoolMust, plus it was always including the default SimpleQueryString query. It looked something like this:
const prefixQueryFields = [
'field_1^5',
'field_2^4',
'field_3',
];
...
<SearchBox
searchOnChange={true}
prefixQueryFields={prefixQueryFields}
/>
I couldn't figure out the issues there easily and decided to go with the queryBuilder attribute in SearchBox. This is what I came up with:
_queryBuilder(queryString) {
const prefixQueryFields = [
'field_1^5',
'field_2^4',
'field_3',
];
return new ImmutableQuery()
.addQuery(BoolMust([
MultiMatchQuery(queryString, {
type: 'phase_prefix',
fields: prefixQueryFields,
})
]))
.addQuery(BoolMustNot([
TermQuery('status', 'archived'),
]));
}
...
<SearchBox
searchOnChange={true}
queryBuilder={this.queryBuilder}
/>
This query came out even more messed up, and I have no idea what to try next after checking the documentation and a cursory look at the source code.
(For the sake of brevity, I will not bother including the incorrect queries these two attempts created unless someone thinks that info will be useful.)
Figured it out. Using the QueryDSL structures wasn't working out very well, but apparently you can create the query with pure JSON, which worked great. Basically updated my query builder to return as so:
return {
bool: {
must: [
{
multi_match:{
query: queryString,
type: 'phrase_prefix',
fields: prefixQueryFields,
}
}
],
must_not: [
{
term: {
status: 'archived',
}
}
]
}
};

How to get data from json in order to save them in database?

I am beginner in Javascript/jQuery and I am working on an interface made with KnockoutJS, so I have several models. I would like to save all the data in the database but I don't know how to do it.
I started with :
self.save = function() {
var data = ko.toJS(self);
var test = ko.toJSON(self);
console.log(test);
}
$.ajax({
url: "myURL",
data: {'carrier': data.carrier},
type: "POST",
});
and this is the result of the console.log :
{"id":1,"carrier":"1","Settings":[{"id":1,"price":{"id":1,"DeliveryStandard":"3.00","DeliveryExpress":"6.00","Details":{"id":1,"Standard":[{"id":1,"fromPrice":0,"maxPrice":"45.000"}],"Express"[{"id":1,"fromPrice":0,"maxPrice":"66.000"}]}}}}]}
I can get the value of carrier by using data.carrier but I don't know how to get the other data like DeiveryStandard, DeliveryExpress, fromPrice, maxPrice ...
Have you got an idea?
Thanks you in advance, and sorry if my question is silly!
If you format your JSON into a more readable format, with indenting, it makes it a lot easier to understand:
(though it should be noted that it is only technically JSON while in a string format, outside of that it is just a standard javascript object)
{
"id":1,
"carrier":"1",
"Settings":[
{
"id":1,
"price": { "id":1,
"DeliveryStandard":"3.00",
"DeliveryExpress":"6.00",
"Details": { "id":1,
"Standard": [{"id":1,
"fromPrice":0,
"maxPrice":"45.000"
}],
"Express" //Missing semi-colon
[{"id":1,
"fromPrice":0,
"maxPrice":"66.000"
}]
}
}
}}//One too many closing braces
]
}
First thing to note is you have 2 syntax errors, highlighted above with comments. So fix them first! (Though I wonder if they are typos as you seem to have it working at your end)
Then we can look at the structure tree to work out where the values you want are...
DeiveryStandard and DeliveryExpress are both properties of an object assigned to price, which it a property of the first item in the Settings array. So you can access them like so:
var DeliveryStandard = data.Settings[0].price.DeliveryStandard;
var DeliveryExpress= data.Settings[0].price.DeliveryExpress;
fromPrice and maxPrice are found multiple times, in both Standard and Express items. So you need to decide what version you need. If you want Standard then you can get the first item of the Standard array like so:
var standardObject = data.Settings[0].price.Details.Standard[0];
Which you can then access the properties of like:
var fromPrice = standardObject.fromPrice;
var maxPrice = standardObject.maxPrice;
I am sure you can work out how to get the Express version of the same data!
From what you seem to have been able to work out on your own, I think your problem is not knowing how to deal with the arrays. Note that arrays are defined with square brackets [], and elements within an array should be accessed with a zero-based index, for example: array[0] for the first element, and array[1] for the second element.
This should work.
console.log(data.Settings[0].price.DeliveryStandard);
Fixed some errors in your JSON.
var j = {
"id" : 1,
"carrier" : "1",
"Settings" : [{
"id" : 1,
"price" : {
"id" : 1,
"DeliveryStandard" : "3.00",
"DeliveryExpress" : "6.00",
"Details" : {
"id" : 1,
"Standard" : [
{
"id" : 1,
"fromPrice" : 0,
"maxPrice" : "45.000"
}
],
"Express": [
{
"id" : 1,
"fromPrice" : 0,
"maxPrice" : "66.000"
}
]
}
}
}
]
};
alert(j.Settings[0].price.DeliveryStandard);
alert(j.Settings[0].price.DeliveryExpress);
alert(j.Settings[0].price.Details.Standard[0].fromPrice);
alert(j.Settings[0].price.Details.Standard[0].maxPrice);

Categories

Resources