Develop Reference

How do I prevent sql injection on a WHERE IN request with parameterized query? [duplicate]

I am trying to workout how to pass in a comma sepated string of values into a postgres select within in clause query using node-postgres but unfortunately can't seem to get it working and can't find any examples.
I have the following code:
function getUser() {
let inList = "'qwerty', 'qaz'";
const result = await pgPool.query("SELECT name FROM my_table WHERE info IN ($1)", [inList]);
if (result.rowCount == 1) return result.rows.map(row => row.name)
return false
}
Basically want the end result query to process:
SELECT name FROM my_table WHERE info IN ('qwerty', 'qaz')
Unsure if this is actually the correct way of doing this with pgPool?

The right way is in the FAQ on Github --> https://github.com/brianc/node-postgres/wiki/FAQ#11-how-do-i-build-a-where-foo-in--query-to-find-rows-matching-an-array-of-values
You should be able to do it that way :
pgPool.query("SELECT name FROM my_table WHERE info = ANY ($1)", [jsArray], ...);

NodeJS, sqlite3, and WHERE IN paramaters

Does the sqlite3 library for NodeJS support parameters for WHERE IN queries?
I have the following small program.
const sqlite3 = require('sqlite3');
const db = new sqlite3.Database('./data/data.db');
const sql = `
SELECT * FROM accounts
WHERE
name IN ('Business Expense - Internet Service','RCSB Checking')`;
db.all( sql,
function getAccout(error, rows) {
console.log(error);
console.log(rows);
});
The program "works". It queries the database, and the rows variable is successfully populated with data from two separate rows.
However, I don't want hard coded values. I want to parameterize my values. The following code works for that.
const sql = `
SELECT * FROM accounts
WHERE
name = ? OR name = ?`;
db.all( sql,
['Business Expense - Internet Service','RCSB Checking'],
function getAccout(error, rows) {
console.log(error);
console.log(rows);
});
However, I want this to work with any number of names. i.e. I want to parameterize the original WHERE IN query. However, the following program
const sql = `
SELECT * FROM accounts
WHERE
name IN (?)`;
db.all( sql,
[['Business Expense - Internet Service','RCSB Checking']],
function getAccout(error, rows) {
console.log(error);
console.log(rows);
});
does not work. It runs without error, but returns zero rows. I also tried using a concatenated string as the param
const sql = `
SELECT * FROM accounts
WHERE
name IN (?)`;
db.all( sql,
["'Business Expense - Internet Service','RCSB Checking'"],
function getAccout(error, rows) {
console.log(error);
console.log(rows);
});
and this also didn't work.
Does the sqlite3 library support parameterizing WHERE IN queries? If so, what's the syntax for doing so? If not, are there common work arounds to this in the NodeJS community?

I don't know if there is support for it, but in case there is not
const params = [...]
const sql = `
SELECT * FROM accounts
WHERE
name IN (${new Array(params.length).fill('?').join(',')})`;

I was going to suggest using JS string literals as well. Seems more like the 'node way' of doing things.

get records after creation using sequelize raw query

I am using sequelize for migration. here I execute an INSERT query with following options but its didnot return created records:
const res = oldUsers.map(u => sequelize.query(
`INSERT INTO ${LP_LOCATIONS_TABLE} (name, address, city)
VALUES (
'${u.email}', '${u.address}', '${u.city}');`,
{ type: DataTypes.QueryTypes.INSERT, raw: true },
))
the output is an array of array like below:
[ [[0],[1]] ]
i expect to get created recorders. specially PK. how can I fix it?

I forgot to put RETURNING * at the end of the raw SQL query.

From the doc, you may have to specify the option returning:true to make this happen. I use mySql, so can't test (the returning option is only for postgres).

Best way to escape array of data on MysQl nodeJS query

So I have the following object with is made by data submitted by the user in order to sign up:
var dataToInsert = {
userName: 'Wilson J',
userEmail: 'WilsonJ#gmail.com',
userAddress: '2020 St.',
userCellPhone: '95587412',
}
And I'm using the following query to insert it:
var insertQuery = `INSERT INTO users ( ${Object.keys(dataToInsert).toString()} ) VALUES( '${Object.values(dataToInsert).join("','")}' )`;
Which at the end is taken as:
INSERT INTO
users (
userName,
userEmail,
userAddress,
userCellPhone
)
VALUES
(
'Wilson J',
'WilsonJ#gmail.com',
'2020 St',
95587412
)
So far I'm having a hard time understanding how data escaping works. I'd really appreciate if someone could show me how a SQL Injection could take place with this code and how to prevent it.
I'm using the MysQl npm module and it has this method: mysql.escape() but I would like to find a more automated approach instead of escaping every single value manually.

In this day and age, it's actively discouraged to do anything other than bind variables to your query. See this for more information on other ways to escape data:
connection.query(`
INSERT INTO users ( ${Object.keys(dataToInsert).toString()} ) VALUES (?)`,
Object.values(dataToInsert),
function (error, results, fields) {
if (error) throw error;
// ...
}
);
Word of caution: You wont be able to bind variables to the column names, so unfortunately that part of the query is necessary. Ensure your keys of your dataToInsert are either static, or not from any user input.

Alternatively, you can use ? characters as placeholders for values you would like to have escaped like this: [...]
There is a way. https://github.com/mysqljs/mysql#user-content-escaping-query-values

How do I fix 8-bit bytestrings error?

I am working on a web application that parse multiple text files, store the data in a SQLite3 database and display the data in a browser. Then I make a HTTP request to my Node.js server to query the database. The data is converted to JSON and send back to the browser. Finally, the JSON is converted back to string and parse it for display.
My program works for small files size under 200MB, but very slowly in terms of parsing and displaying. When it gets 300MB+, I can not parse the files and got this error.
ProgrammingError: You must not use 8-bit bytestrings unless you use a
text_factory that can interpret 8-bit bytestrings (like text_factory =
str). It is highly recommended that you instead just switch your
application to Unicode strings.
I tried the text_factory, it gets rid of the error code. When I query the database, it returns nothing and the program hangs in there. I am a newbie. What am I doing wrong? What can I do to make it parse larger size files and faster?
#parsing some files and insert them into a sqlite database
#create database in python
conn = sqlite3.connect('MyDB.db')
conn.text_factory = str # didn't work, the program hang
c = conn.cursor()
c.execute('''CREATE TABLE actions (dates text, names text, methods text)''')
c.execute('''CREATE TABLE people (names text, ID text)''')
# insert in different function
dataB.execute("INSERT INTO people VALUES(?, ?)", (value[index], id))
dataB.execute("INSERT INTO action VALUES(?, ?, ?)", (date, value[index], someaction ))
//seperate file
//retrieve the data from javascript
var sqlite3 = require("sqlite3");
var logDB = new sqlite3.Database("log.db");
query = "SELECT DISTINCT id, name FROM people p, action a where p.name = a.name and p.id Not IN (SELECT DISTINCT id FROM someothertable);";
function queryTheServer(query, response) {
logDB.all(query, function(error, thing){
serveFromDatabase(error, thing, query, response);
});
}
//pass by as JSON
response.writeHead(200, {"Content-Type": "application/json"});
response.write(JSON.stringify(result));
response.end();