Data-import to MySQL -> values vs no-values - javascript

i have following problem:
I parse a CSV file and then the data should automatically get uploaded to a MySQL database. Now the problem is that i have the datatypes "varchar" or "decimal". Here at stackoverflow I have already gotten help that if the data type is "decimal" then there stand only the number in the records.
The records (rec) look like this:
[ { examplename1: 'example1',
examplename2: 'example2',
examplename3: 'example3',
examplename4: 'example4',
examplename5: 'example5',
examplename6: 'example6',
examplename7: 'example7',
examplename8: 'example8',
examplename9: 'example9',
examplename10: 'example10',
examplename11: 'example11',
examplename12: 'example12',
Diff_In_Hauswaehrung: -103600,
examplename14: 'example14',
Marktwert_NPV: -111146.16,
examplename16: 'example16' },
{ examplename1: 'example1',
examplename2: 'example2',
examplename3: 'example3',
examplename4: 'example4',
examplename5: 'example5',
examplename6: 'example6',
examplename7: 'example7',
examplename8: 'example8',
examplename9: 'example9',
examplename10: 'example10',
examplename11: 'example11',
examplename12: 'example12',
Diff_In_Hauswaehrung: 53851.33,
examplename14: 'example14',
Marktwert_NPV: 47328.16,
examplename16: 'example16' } ]
You can see that in "Diff_In_Hauswaehrung" and "Marktwert_NPV" there stand a number without ''.
These are also displayed in yellow in the console.
Now i want to check if there is such a number or only a string with '' and if there is a number then i want no "" around this value in the SQL Query.
Now i have a forEach loop where all values are uploaded with ""... but no distinction takes place... keysY are only the keys of the rec.
rec.forEach(entry => {
values = `"${entry[keysY[0]]}"`
for (var i = 1; i < keysY.length; i++) {
values += `,"${entry[keysY[i]]}"`
}
//console.log(values)
var sql = `INSERT INTO ${richtigername} (${namen}) VALUES (${values})`;
connection.query(sql, [values], (error, response) => {
console.log(error || response);
I hope you can help me :)
Best regards,
Frederic

To check if a value contains a valid number cast it to a Number. If the value is not a number the result of that cast is false, otherwise it's number. Next, check if the result is (not) a number using isNaN. If the result is false it is a number. To simplify this add a ! beforehand to negate the final result.
Here's an example:
!isNaN(Number("12.2"))
// returns true (valid number)
!isNaN(Number("asdf12"))
// returns false (no valid number)
!isNaN(Number(12))
// returns true (valid number)
So in your case this should do the trick:
for (var i = 1; i < keysY.length; i++) {
if(!isNaN(Number(${entry[keysY[i]]}) {
// it's a number, no quotes:
values += `,${entry[keysY[i]]}`;
} else {
// it's not a number (NaN) add quotes:
values += `,"${entry[keysY[i]]}"`;
}
}

thanks :) but i have empty columns and I solve it with typeof because then the empty columns are "strings".. with isNaN(Number() also empty columns are displayed as "false" but must be "true" or only with Number() empty columns are not NaN..
rec.forEach(entry => {
if (typeof entry[keysY[0]] == "string") {
values = `"${entry[keysY[0]]}"`
} else {
values = entry[keysY[0]]
}
for (var i = 1; i < keysY.length; i++) {
if (typeof entry[keysY[i]] == "string") {
values += `,"${entry[keysY[i]]}"`;
} else {
values += `,${entry[keysY[i]]}`;
}
the $ and ` are only needed if i want something like to put this in quotation marks or to put a comma in front of.

Related

In this query, what do I need to add so it never displays blanks or null values?

#Query(nativeQuery = true, value = "SELECT " +
"COUNT(DISTINCT pd.id) FILTER (WHERE c.year NOT IN ('2020', '2021', '')) AS \"notCurrent\", " +
This query is for a filter called notCurrent. The problem is, when the filter is selected, it still shows blank values, even though I added '' in the query. What do I need to do so no blanks/null values display when this filter is selected?
UI:
export function forceFilter(key, filterVal, table, dataType, title) {
var newTable = JSON.stringify(table);
newTable = JSON.parse(newTable);
var updated = false;
if (key != "disclosure" && filterVal == "Unknown") {
filterVal = "";
} else if (filterVal == "Not Current") {
filterVal = "notCurrent";
}
}
The default empty "value" may not be an empty string, but null, you may want to try adding this to your WHERE clause
AND c.year IS NOT NULL
you can check length greater than 0 after trimming that column and also check is not null.
Similar as below used sometime back
SELECT
DISTINCT email_id
FROM FCSR_USER_MST
WHERE length(trim(email_id)) >0 and email_id IS NOT NULL
See this linkIt would be helpful for you

Splitting a string for CSV formatting

I have a string that I've formed based on certain data. It contains a list of comma separated values.
I run this string through a function which allows me to separate these values so that I can show them appropriately in the HTML.
This works fine. However there's one error I'm unable to resolve. One of the returned values is from a HTML textfield which means the user can write on multiple different lines. As a result, the function I use to split up my values is splitting up the textfield values into multiple lines.
I've consoled logged my string before it hits the function. As you can see, the comments appear fine in the quotations with a space in between.
20,Order,Item,Title,,Assignee,Comments,Image Path,Timestamp,Due Date,Completed
21,0,1,"Issue 1",,"","I have some comments that are
going here on multiple lines",,"2019/11/28, 12:20:55","",""
The function
csvToJSON(csv) {
const lines: string[] = csv
// escape everything inside quotes to NOT remove the comma there
.replace(/"(.*?)"/gm, (item) => encodeURIComponent(item))
.split('\n');
lines.pop();
// separate the headers from the other lines and split them
const headers: string[] = lines.shift().split(',');
// should contain all CSV lines parsed for the html table
const data: any[] = lines.map((lineString, index) => {
const lineObj = {};
const lineValues = lineString.split(',');
headers.forEach((valueName, index) => {
// remove trailing spaces and quotes
if (lineValues[index] != undefined) {
lineObj[valueName] = lineValues[index].replace(/%22(.*?)%22/gm, (item) => decodeURIComponent(item)).trim();
}
});
return lineObj; // return lineObj for objects.
});
console.log('csvToJSON - data = ', data);
return { data, headers };
}
In the above function, you can see a console log at the end to log the final converted data.
This ends up resulting in the following:
0: {20: "21", Order: "0", Item: "1", Title: ""Issue 1"", "": "", Assignee: """", …}
1: {20: ""}
2: {20: "going here on multiple lines",,"2019/11/28", Order: "12:20:55","","""}
As you can see, my comments have been a bit butchered.
Is there a way to resolve this so that everything else is separated correctly, but my comments text (which can have multiple lines) isn't broken up?
try this method
function CSVtoArray(text) {
var re_valid = /^\s*(?:'[^'\\]*(?:\\[\S\s][^'\\]*)*'|"[^"\\]*(?:\\[\S\s][^"\\]*)*"|[^,'"\s\\]*(?:\s+[^,'"\s\\]+)*)\s*(?:,\s*(?:'[^'\\]*(?:\\[\S\s][^'\\]*)*'|"[^"\\]*(?:\\[\S\s][^"\\]*)*"|[^,'"\s\\]*(?:\s+[^,'"\s\\]+)*)\s*)*$/;
var re_value = /(?!\s*$)\s*(?:'([^'\\]*(?:\\[\S\s][^'\\]*)*)'|"([^"\\]*(?:\\[\S\s][^"\\]*)*)"|([^,'"\s\\]*(?:\s+[^,'"\s\\]+)*))\s*(?:,|$)/g;
if (!re_valid.test(text)) return null;
var a = [];
text.replace(re_value,
function(m0, m1, m2, m3) {
if (m1 !== undefined) a.push(m1.split("|");
else if (m2 !== undefined) a.push(m2.split(","));
else if (m3 !== undefined) a.push(m3);
return '';
});
if (/,\s*$/.test(text)) a.push('');
return a;
};

How do I change \" back to " in javascript?

I'm a beginner to Node.js and MySQL so bear with me. The code below is in my routes for my localhost server and the database when saved at first is fine, but when I try to retrieve it and push it back into my database by storing the old one in a new array, it changes the "" to \"\". What I want is for it to be still "".
I've tried looking around the web for ways to change the formatted string quotes from \"\" back to "" but to no avail and I also tried formatting the score from string back to a number but I realized that even if I did that, it would still end up being a string when stored in the database and when I retrieve it, it would be the same.
let { score } = req.body;
let { date } = req.body;
score = Number(score);
var score_date = [score, date];
wMissing.findOne({
raw : true
}).then((missing) => {
var sessionID = missing.testSessionID;
registerPatient.findOne({
where: {
nric : sessionID
},
raw : true
}).then((patient) => {
var height = patient.height;
height = height / 100
var weight = patient.weight;
var bmiVar = (weight / (height * height)).toFixed(2);
// *This is where my problem starts*
if (missing.wMissingScore == null){
var newArray = score_date;
} else {
var newArray = [missing.wMissingScore];
newArray.push(score_date);
}
newArray = JSON.stringify(newArray);
wMissing.update({
bmi: bmiVar,
wMissingScore: newArray,
name: patient.fullname,
nric: sessionID,
datePlayed: date
}, {
where: {
nric: sessionID
}
}).then(() => {
res.redirect('workoutDiets');
}).catch(err => console.log(err));
The expected output is supposed to be [["2","24/7/2019"],["3","24/7/2019"]]
But instead I'm getting ["[\"2\",\"24/7/2019\"]",["3","24/7/2019"]]
I'm still unaware of any method to change back the \"\" to "" instead. can anyone please help me to improve on my code?
Actually, what's happening is that the first element of the array is being stored as a string, if you watch carefully, the second element is written correctly as an array of strings.
The problem is the first element, you're having a special case.
If i understand correctly ´´´missing.wMissingScore´´´ is a pure string
What you can do is:
´´´
if (missing.wMissingScore == null){
var newArray = score_date;
} else {
var formatedMissingScore= missing.wMissingScore.replace("\"", "").replace("[", "").replace("]", "").split(",");
var newArray = [formatedMissingScore];
newArray.push(score_date);
}
´´´
That way you're formatting your first element string so its not text anymore and should match what you are trying to get.
#charlietfl mentioned about JSON.parse() and it worked! I just changed my else statement to
var newArray = [JSON.parse(missing.wMissingScore)];
and it managed to save the score back to what it originally was.

Trying to loop through JSON file

I am creating a dictionary app in React, I have loaded in the JSON dictionary which looks like this:
{
"DIPLOBLASTIC": "Characterizing the ovum when it has two primary germinallayers.",
"DEFIGURE": "To delineate. [Obs.]These two stones as they are here defigured. Weever.",
"LOMBARD": "Of or pertaining to Lombardy, or the inhabitants of Lombardy.",
"BAHAISM": "The religious tenets or practices of the Bahais.",
"FUMERELL": "See Femerell."
}
The user enters a word in the input field and the value is then passed into the following function to search for a matching key in the JSON. Matching words are then pushed into an array of results with their respective value.
handleSearch: function(term) {
var term = term;
var results = [];
for (var key in Dictionary) {
if (Dictionary.hasOwnProperty(key)) {
if (term == Dictionary[key]) {
results.push(Dictionary[key])
}
}
}
console.log(results)
},
However I am struggling to find a successful way of looping through it to get results. the console is logging an empty array.
Can anyone please suggest where I am going wrong?
You can have better matching by adding a compare function (in example below it is the compareTerm function). What I did there is comparing if the term STARTS with the dictionary key, if you want it to be any part of the string you can change it from === 0 to > -1.
// compare function which needs to be added somewhere
function compareTerm(term, compareTo) {
var shortenedCompareTo = compareTo
.split('')
.slice(0, term.length)
.join('');
return term.indexOf(shortenedCompareTo.toLowerCase()) === 0;
}
// only changed the compare function
handleSearch: function(term) {
var results = [];
for (var key in Dictionary) {
if (Dictionary.hasOwnProperty(key)) {
if (compareTerm(term, Dictionary[key])) {
results.push(Dictionary[key])
}
}
}
console.log(results);
},

Check case insentively if a value is in an array, and if the case doesn't match, replace it with the new value

I am attempting to replace the values in an array with the correct case sensitivity. This is because I am attempting to correct user inputs. I retrieve the correct casing from a page, and the user will have an array of values, some of which will be incorrect.
Ex:
userValues = ["apple321", "orange_22", "pineApple" , "Cantelope", "grASShopper_9000"];
var value1 = "Apple321";
var value2 = "orange_22";
var value3 = "Cantelope";
var value4 = "GrassHopper_9000";
Then after some function ran through all the values, the result would be:
userValues = ["Apple321", "orange_22", "pineApple" , "Cantelope", "GrassHopper_9000"];
The reason I have value1, value2, etc is because I've already created a loop to run through an object. Just not sure how to compare the resulting values. Here is what I have already however:
// When the user enters data, it's sometimes case insensitive. This normalizes the data.
function NormalizeData(downloaded_data)
{
$.each(downloaded_data, function(website,streams){
$.each(streams, function(stream_name,value){
stream_list[website] // This is the global variable array
value.displayName; // This is the value I need to check case sensitivity, and replace with the new case if different
});
});
}
Here is the requested data structure:
downloaded_data = {
twitch_tv : {
timthetatman : {
Online: "1",
Website: "twitch_tv",
displayName: "TimTheTatman"
}
}
}
streamlist = {
twitch_tv : {
["timthetatman"]
}
hitbox_tv: {
[]
}
}
I figured it out. Since I am using an array for the values I want to change, and not an object, it's actually much simpler than I thought. I ran a loop on every value, and if the lowercase of both values matched, but the non-lowercase values didn't, I replaced it in the array, using the numerical key as a reference.
function NormalizeData(downloaded_data)
{
$.each(downloaded_data, function(website,streams){
$.each(streams, function(stream_name,value){
$.each(stream_list[website], function(arrKey,arrVal){
if(arrVal.toLowerCase() == stream_name.toLowerCase() && stream_list[website][arrKey] !== value.displayName)
stream_list[website][arrKey] = value.displayName;
});
});
});
}
Here it is, simplified, if the array is called Array1 and the value Value1:
var Array1 = ["apple"];
var Value1 = ["Apple"];
$.each(Array1, function(arrKey,arrVal){
if(arrVal.toLowerCase() == Value1.toLowerCase() && arrVal !== Value1)
Array1[arrKey] = Value1;
});

Categories

Resources