I'm creating a dynamic query in postgresql to filter a resultset.
I pass the parameters in the URL, an example is:
http://www.myurl.com/search?query=hello+word&sortorder=relevance&date=week&categories=cat1,cat2
The problem is, when a user selects a lot of categories (and other parameters that aren't included in the example URL), the URL will be very, very long. I'm using UUIDv4 for ID's so a lot of UUIDv4's are in the URL, separated by comma's.
Is it a better idea to do this with a POST request?
EDIT:
I'm using validator to check if the incoming parameters are valid UUIDv4 (or numbers, ... whatever I need for that parameter to be).
I'm using node-postgres to execute queries. This supports prepared statements, but I have NO CLUE how to use prepared statements with dynamic queries.
My SELECT code now looks like this (simplified):
const { featured, order } = req.body;
// featured
let featuredQuery = '';
if (featured !== undefined && validator.isBoolean(featured.toString())) {
let featuredQuery = ` AND featured = ${featured}`;
}
// order
let orderQuery = 'timestamp DESC';
if (order !== undefined) {
switch (order) {
default:
case 'date':
orderQuery = 'timestamp DESC';
break;
case 'user_id':
orderQuery = 'user_id DESC';
break;
}
}
const query = `
SELECT
name,
description
FROM
users
WHERE
active = true
${featuredQuery}
ORDER BY
${orderQuery}
`;
As you can see, I'm not passing prepared statements ($1, $2, ...) to the query but I'm using template literals. I have NO CLUE how to use prepared statements to add/delete where clauses.
Is there a safer way to do this?
EDIT 2: parameters are not containing sensitive information. I pass the user ID in the authorization header as a JWT.
If you're fetching the contents of your parameter-laden URL using a program, not a browser, long GET URLs don't matter very much. (In browsers they're ugly.) For best results, and less testing burden on various legacy proxy servers etc, do your best to keep your URL length to 2000 octets or less. (Payload size for POST requests has no such limitation.)
Beware that URLs get logged. So, if your parameters carry any sensitive information you should consider using POST rather than GET. Otherwise your users' sensitive information will land in your web server logs, presenting a juicy target for cybercreeps.
And, please please read up on SQL injection. Please please sanitize your incoming parameters before handing them off to your database server. Remember that cybercreeps can (and will) hit your URL from their malicious programs and try all sorts of combinations trying to break in.
Related
I am trying to make a request to an endpoint that expects only one user account Id, so I tried to separate the user account Ids (if they are multiple) by &. The this.account is an array with account Id strings. And this is how I am doing it but it still appends the values with a comma:
getAccountStats(callback) {
let acc = this.account.map((val) => {
if (val>1) {
return 'accountID_'+val+'&'
}
return 'accountID__'+val;
})
let url = `/${acc}/userAccount`;
axios.get(url, callback);
}
When I console.log the url, it returns /accountID_1,accountID_2/userAccount but I want it to be /accountID_1&accountID_2/userAccount. Any idea how to achieve this. TIA
I am trying to make a request to an endpoint that expects only one user account Id, so I tried to separate the user account Ids (if they are multiple) by &
The short answer is what you think you want won't ever work. If the endpoint only expects 1 account id, then trying to add more won't do what you want.
but I want it to be /accountID_1&accountID_2/userAccount
This doesn't look like a valid endpoint. So I doubt you really want this.
I want to access resources from a SQLite database table. I have one table for accounts, one for movies and one for reviews. The reviews-table is constructed like this:
CREATE TABLE IF NOT EXISTS reviews (
id INTEGER PRIMARY KEY,
authorId INTEGER,
movieId INTEGER,
review TEXT,
rating INTEGER,
FOREIGN KEY('authorId') REFERENCES 'accounts'('id'),
FOREIGN KEY('movieId') REFERENCES 'movies'('id')
)
What I want to do is that I want to be able to get all reviews, made by one author. But I also want to be able to get all reviews, made for the same movie. Below is my code for getting all reviews made by the same user/author. The code looks the same when getting the reviews based on the movie, with a few changes.
Both of them does what I want them to do. But of course only the one written first in the file are running.
const authorId = request.params.authorId;
const query = "SELECT * FROM reviews WHERE authorId = ?";
const values = [authorId];
db.all(query, values, function (error, review) {
if (error) {
response.status(500).end();
} else if (!review) {
response.status(404).end();
} else {
response.status(200).json(review);
}
});
});
The url will look the same no matter which of them I want running; http://localhost:3000/reviews/3. How can I differentiate the url so that they know which one should run?
I have tried to experiment with query strings, but I'm not sure how that works, and after hours of searching for something that worked on my code, I gave up.
I have also been thinking about using something like
app.use(function (request, response, next) {
if (request.method == "GET" && request.uri == "/reviews/:authorId") {
response.send("Hello World");
} else {
next;
}
});
This didn't work, and it didn't work if I tried to remove ":authorId" from the url either. The page just keeps loading.
So how do I solve this?
The most dynamic would be a single route /reviews and use the query string with the params like ?author=123 or ?movie=123, they can be combined like ?author=123&movie=123. As you want to return JSON the code will be used via API, so the pretty path is usually not as important as when it is a web-url. To make the implementation effective, most people use a function where you can drop the query object in and get the where-clause, or use an ORM.
In express, when you have routers like '/reviews/:authorId' and then '/reviews/:movieId', then the second one will never be called, because the first one will always match. That is something to be careful about when organizing your express routes.
I'm trying to make a MySQL query to filter data from a table. Effectively what I want to do is:
SELECT data FROM table WHERE column IN ?
The filter is coming from checkboxes in a form on a webpage, so I can pass an array or object fairly easily, but it'll be a varying number of parameters for the IN each time, so I can't us multiple ?. I tried making a for loop to make multiple queries concatenate the arrays that the queries returned, but I ran into scope issues with that. I also tried passing an array directly to the query, but that throws a syntax error. I'm sure there's a straightforward answer to this but I'm not sure how to do it.
Edit: source code added:
Here's where I'm at:
const filterShowQuery = `SELECT sl_Show.showId, sl_Band.BandName,
sl_Show.date, sl_Venue.venueName,
sl_Show.length, sl_Show.attendance, sl_Show.encore FROM sl_Show
JOIN sl_Band on sl_Show.BandID = sl_Band.BandId
JOIN sl_Venue on sl_Show.VenueId = sl_Venue.VenueId
WHERE sl_Band.BandName IN (?)
ORDER BY sl_Band.BandName;`;
Trying to get an array into the ? in WHERE sl_Band.BandName IN
const getShows = (req, res,next) =>{
var {bands, venues} = req.body;
var i = 0; //left over from previous attempt
var data = [];
for (b in bands){
mysql.pool.query(filterShowQuery, bands[b], (err, result) => {
if(err){
console.log('filter band error');
next(err);
return;
}
data = data.concat(result);
console.log(data); //data concatenates property and increases through for loop
})
// same action to be performed with venues once solved
// for (v in venues){
// conditions[i] = venues[v];
// i++;
console.log(data); //data is empty when logging from here or using in res
res.json({rows:data});
}
}
SECURITY WARNING!
I must to say: NEVER, NEVER PASS DATA DIRECTLY TO YOUR SQL!
If you don't know why, just google for SQL Injection. There are lots of examples on how it is done, how easily it can be done, and how to protect your application from this sort of attack.
You should always parametrize your queries. But in the very rare case which you really need to insert data concatenating a string into your sql, validate it before.
(E.g.) If it's a number, than use a Regex or some helper method to check if the value you are inserting into your SQL String is really and only a number and nothing else.
But aside that, you did not provide any source code, so it's really hard to give any help before you do that.
I am using firebase for data storage. The data structure is like this:
products:{
product1:{
name:"chocolate",
}
product2:{
name:"chochocho",
}
}
I want to perform an auto complete operation for this data, and normally i write the query like this:
"select name from PRODUCTS where productname LIKE '%" + keyword + "%'";
So, for my situation, for example, if user types "cho", i need to bring both "chocolate" and "chochocho" as result. I thought about bringing all data under "products" block, and then do the query at the client, but this may need a lot of memory for a big database. So, how can i perform sql LIKE operation?
Thanks
Update: With the release of Cloud Functions for Firebase, there's another elegant way to do this as well by linking Firebase to Algolia via Functions. The tradeoff here is that the Functions/Algolia is pretty much zero maintenance, but probably at increased cost over roll-your-own in Node.
There are no content searches in Firebase at present. Many of the more common search scenarios, such as searching by attribute will be baked into Firebase as the API continues to expand.
In the meantime, it's certainly possible to grow your own. However, searching is a vast topic (think creating a real-time data store vast), greatly underestimated, and a critical feature of your application--not one you want to ad hoc or even depend on someone like Firebase to provide on your behalf. So it's typically simpler to employ a scalable third party tool to handle indexing, searching, tag/pattern matching, fuzzy logic, weighted rankings, et al.
The Firebase blog features a blog post on indexing with ElasticSearch which outlines a straightforward approach to integrating a quick, but extremely powerful, search engine into your Firebase backend.
Essentially, it's done in two steps. Monitor the data and index it:
var Firebase = require('firebase');
var ElasticClient = require('elasticsearchclient')
// initialize our ElasticSearch API
var client = new ElasticClient({ host: 'localhost', port: 9200 });
// listen for changes to Firebase data
var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets');
fb.on('child_added', createOrUpdateIndex);
fb.on('child_changed', createOrUpdateIndex);
fb.on('child_removed', removeIndex);
function createOrUpdateIndex(snap) {
client.index(this.index, this.type, snap.val(), snap.name())
.on('data', function(data) { console.log('indexed ', snap.name()); })
.on('error', function(err) { /* handle errors */ });
}
function removeIndex(snap) {
client.deleteDocument(this.index, this.type, snap.name(), function(error, data) {
if( error ) console.error('failed to delete', snap.name(), error);
else console.log('deleted', snap.name());
});
}
Query the index when you want to do a search:
<script src="elastic.min.js"></script>
<script src="elastic-jquery-client.min.js"></script>
<script>
ejs.client = ejs.jQueryClient('http://localhost:9200');
client.search({
index: 'firebase',
type: 'widget',
body: ejs.Request().query(ejs.MatchQuery('title', 'foo'))
}, function (error, response) {
// handle response
});
</script>
There's an example, and a third party lib to simplify integration, here.
I believe you can do :
admin
.database()
.ref('/vals')
.orderByChild('name')
.startAt('cho')
.endAt("cho\uf8ff")
.once('value')
.then(c => res.send(c.val()));
this will find vals whose name are starting with cho.
source
The elastic search solution basically binds to add set del and offers a get by wich you can accomplish text searches.
It then saves the contents in mongodb.
While I love and reccomand elastic search for the maturity of the project, the same can be done without another server, using only the firebase database.
That's what I mean:
(https://github.com/metaschema/oxyzen)
for the indexing part basically the function:
JSON stringifies a document.
removes all the property names and JSON to leave only the data
(regex).
removes all xml tags (therefore also html) and attributes (remember
old guidance, "data should not be in xml attributes") to leave only
the pure text if xml or html was present.
removes all special chars and substitute with space (regex)
substitutes all instances of multiple spaces with one space (regex)
splits to spaces and cycles:
for each word adds refs to the document in some index structure in
your db tha basically contains childs named with words with childs
named with an escaped version of "ref/inthedatabase/dockey"
then inserts the document as a normal firebase application would do
in the oxyzen implementation, subsequent updates of the document ACTUALLY reads the index and updates it, removing the words that don't match anymore, and adding the new ones.
subsequent searches of words can directly find documents in the words child. multiple words searches are implemented using hits
SQL"LIKE" operation on firebase is possible
let node = await db.ref('yourPath').orderByChild('yourKey').startAt('!').endAt('SUBSTRING\uf8ff').once('value');
This query work for me, it look like the below statement in MySQL
select * from StoreAds where University Like %ps%;
query = database.getReference().child("StoreAds").orderByChild("University").startAt("ps").endAt("\uf8ff");
I want to extract some data from the database without refreshing a page. What is the best possible way to do this?
I am using the following XMLHTTPRequest function to get some data (shopping cart items) from cart.php file. This file performs various functions based on the option value.
For example: option=1 means get all the shopping cart items. option=2 means delete all shopping cart items and return string "Your shopping cart is empty.". option=3, 4...and so on.
My XHR function:
function getAllCartItems()
{
if(window.XMLHttpRequest)
{
allCartItems = new XMLHttpRequest();
}
else
{
allCartItems=new ActiveXObject("Microsoft.XMLHTTP");
}
allCartItems.onreadystatechange=function()
{
if (allCartItems.readyState==4 && allCartItems.status==200)
{
document.getElementById("cartmain").innerHTML=allCartItems.responseText;
}
else if(allCartItems.readyState < 4)
{
//do nothing
}
}
var linktoexecute = "cart.php?option=1";
allCartItems.open("GET",linktoexecute,true);
allCartItems.send();
}
cart.php file looks like:
$link = mysql_connect('localhost', 'user', '123456');
if (!$link)
{
die('Could not connect: ' . mysql_error());
}
mysql_select_db('projectdatabase');
if($option == 1) //get all cart items
{
$sql = "select itemid from cart where cartid=".$_COOKIE['cart'].";";
$result = mysql_query($sql);
$num = mysql_num_rows($result);
while($row = mysql_fetch_array($result))
{
echo $row['itemid'];
}
}
else if($option == 2)
{
//do something
}
else if($option == 3)
{
//do something
}
else if($option == 4)
{
//do something
}
My Questions:
Is there any other way I can get the data from database without
refreshing the page?
Are there any potential threats (hacking, server utilization,
performance etc) in my way of doing this thing? I believe a hacker
can flood my server be sending unnecessary requests using option=1,
2, 3 etc.
I don't think a Denial of Service attack would be your main concern, here. That concern would be just as valid is cart.php were to return HTML. No, exposing a public API for use via AJAX is pretty common practice.
One thing to keep in mind, though, is the ambiguity of both listing and deleting items via the same URL. It would be a good idea to (at the very least) separate those actions (or "methods") into distinct URLs (for example: /cart/list and /cart/clear).
If you're willing to go a step further, you should consider implementing a "RESTful" API. This would mean, among other things, that methods can only be called using the correct HTTP verb. You've possibly only heard of GET and POST, but there's also PUT and DELETE, amongst others. The reason behind this is to make the methods idempotent, meaning that they do the same thing again and again, no matter how many times you call them. For example, a GET call to /cart will always list the contents and a DELETE call to /cart will always delete all items in the cart.
Although it is probably not practical to write a full REST API for your shopping cart, I'm sure some of the principles may help you build a more robust system.
Some reading material: A Brief Introduction to REST.
Ajax is the best option for the purpose.
Now sending and receiving data with Ajax is done best using XML. So use of Web services is the recommended option from me. You can use a SOAP / REST web service to bring data from a database on request.
You can use this Link to understand more on Webservices.
For the tutorials enough articles are available in the Internet.
you're using a XMLHttpRequest object, so you don't refresh your page (it's AJAX), or there's something you haven't tell
if a hacker want to DDOS your website, or your database, he can use any of its page... As long as you don't transfer string between client and server that will be used in your SQL requests, that should be OK
I'd warn you about the use of raw text response display. I encourage you to format your response as XML or JSON to correctly locate objects that needs to be inserted into the DOM, and to return an tag to correctly handle errors (the die("i'm your father luke") won't help any of your user) and to display them in a special area of your web page
First, you should consider separating different parts of your application. Having a general file that performs every other tasks related to carts, violates all sorts of software design principles.
Second, the first vulnerability is SQL injection. You should NEVER just concatenate the input to your SQL.
Suppose I posted 1; TRUNCATE TABLE cart;. Then your SQL would look like:
select itemid from cart where cartid=1; TRUNCATE TABLE cart; which first selects the item in question, then ruins your database.
You should write something like this:
$item = $_COOKIE['cart'];
$item = preg_replace_all("['\"]", "\\$1", $item);
To avoid refreshing, you can put a link on your page. Something like, Refresh
In terms of security, it will always pay to introduce a database layer concerned with just your data, regardless of your business logic, then adding a service layer dependent on the database layer, which would provide facilities to perform business layer actions.
You should also take #PPvG recommendation into note, and -- using Apache's mod_rewrite or other similar facilities -- make your URLs more meaningful.
Another note: try to encapsulate your data in JSON or XML format. I'd recommend the use of json_encode(); on the server side, and JSON.parse(); on the client side. This would ensure a secure delivery.