I've searched around but didn't find if it's possible.
I've this MySQL query:
INSERT INTO table (id,a,b,c,d,e,f,g) VALUES (1,2,3,4,5,6,7,8)
Field id has a "unique index", so there can't be two of them. Now if the same id is already present in the database, I'd like to update it. But do I really have to specify all these field again, like:
INSERT INTO table (id,a,b,c,d,e,f,g) VALUES (1,2,3,4,5,6,7,8)
ON DUPLICATE KEY UPDATE a=2,b=3,c=4,d=5,e=6,f=7,g=8
Or:
INSERT INTO table (id,a,b,c,d,e,f,g) VALUES (1,2,3,4,5,6,7,8)
ON DUPLICATE KEY UPDATE a=VALUES(a),b=VALUES(b),c=VALUES(c),d=VALUES(d),e=VALUES(e),f=VALUES(f),g=VALUES(g)
I've specified everything already in the insert...
A extra note, I'd like to use the work around to get the ID to!
id=LAST_INSERT_ID(id)
I hope somebody can tell me what the most efficient way is.
The UPDATE statement is given so that older fields can be updated to new value. If your older values are the same as your new ones, why would you need to update it in any case?
For eg. if your columns a to g are already set as 2 to 8; there would be no need to re-update it.
Alternatively, you can use:
INSERT INTO table (id,a,b,c,d,e,f,g)
VALUES (1,2,3,4,5,6,7,8)
ON DUPLICATE KEY
UPDATE a=a, b=b, c=c, d=d, e=e, f=f, g=g;
To get the id from LAST_INSERT_ID; you need to specify the backend app you're using for the same.
For LuaSQL, a conn:getlastautoid() fetches the value.
There is a MySQL specific extension to SQL that may be what you want - REPLACE INTO
However it does not work quite the same as 'ON DUPLICATE UPDATE'
It deletes the old row that clashes with the new row and then inserts the new row. So long as you don't have a primary key on the table that would be fine, but if you do, then if any other table references that primary key
You can't reference the values in the old rows so you can't do an equivalent of
INSERT INTO mytable (id, a, b, c) values ( 1, 2, 3, 4)
ON DUPLICATE KEY UPDATE
id=1, a=2, b=3, c=c + 1;
I'd like to use the work around to get the ID to!
That should work — last_insert_id() should have the correct value so long as your primary key is auto-incrementing.
However as I said, if you actually use that primary key in other tables, REPLACE INTO probably won't be acceptable to you, as it deletes the old row that clashed via the unique key.
Someone else suggested before you can reduce some typing by doing:
INSERT INTO `tableName` (`a`,`b`,`c`) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE `a`=VALUES(`a`), `b`=VALUES(`b`), `c`=VALUES(`c`);
There is no other way, I have to specify everything twice. First for the insert, second in the update case.
Here is a solution to your problem:
I've tried to solve problem like yours & I want to suggest to test from simple aspect.
Follow these steps: Learn from simple solution.
Step 1: Create a table schema using this SQL Query:
CREATE TABLE IF NOT EXISTS `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(30) NOT NULL,
`password` varchar(32) NOT NULL,
`status` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `no_duplicate` (`username`,`password`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;
Step 2: Create an index of two columns to prevent duplicate data using following SQL Query:
ALTER TABLE `user` ADD INDEX no_duplicate (`username`, `password`);
or, Create an index of two column from GUI as follows:
Step 3: Update if exist, insert if not using following queries:
INSERT INTO `user`(`username`, `password`) VALUES ('ersks','Nepal') ON DUPLICATE KEY UPDATE `username`='master',`password`='Nepal';
INSERT INTO `user`(`username`, `password`) VALUES ('master','Nepal') ON DUPLICATE KEY UPDATE `username`='ersks',`password`='Nepal';
Just in case you are able to utilize a scripting language to prepare your SQL queries, you could reuse field=value pairs by using SET instead of (a,b,c) VALUES(a,b,c).
An example with PHP:
$pairs = "a=$a,b=$b,c=$c";
$query = "INSERT INTO $table SET $pairs ON DUPLICATE KEY UPDATE $pairs";
Example table:
CREATE TABLE IF NOT EXISTS `tester` (
`a` int(11) NOT NULL,
`b` varchar(50) NOT NULL,
`c` text NOT NULL,
UNIQUE KEY `a` (`a`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
I know it's late, but i hope someone will be helped of this answer
INSERT INTO t1 (a,b,c) VALUES (1,2,3),(4,5,6)
ON DUPLICATE KEY UPDATE c=VALUES(a)+VALUES(b);
You can read the tutorial below here :
https://mariadb.com/kb/en/library/insert-on-duplicate-key-update/
http://www.mysqltutorial.org/mysql-insert-or-update-on-duplicate-key-update/
You may want to consider using REPLACE INTO syntax, but be warned, upon duplicate PRIMARY / UNIQUE key, it DELETES the row and INSERTS a new one.
You won't need to re-specify all the fields. However, you should consider the possible performance reduction (depends on your table design).
Caveats:
If you have AUTO_INCREMENT primary key, it will be given a new one
Indexes will probably need to be updated
With MySQL v8.0.19 and above you can do this:
mysql doc
INSERT INTO mytable(fielda, fieldb, fieldc)
VALUES("2022-01-01", 97, "hello")
AS NEW(newfielda, newfieldb, newfieldc)
ON DUPLICATE KEY UPDATE
fielda=newfielda,
fieldb=newfieldb,
fieldc=newfieldc;
SIDENOTE: Also if you want a conditional in the on duplicate key update part there is a twist in MySQL. If you update fielda as the first argument and include it inside the IF clause for fieldb it will already be updated to the new value! Move it to the end or alike. Let's say fielda is a date like in the example and you want to update only if the date is newer than the previous:
INSERT INTO mytable(fielda, fieldb)
VALUES("2022-01-01", 97)
AS NEW(newfielda, newfieldb, newfieldc)
ON DUPLICATE KEY UPDATE
fielda=IF(fielda<STR_TO_DATE(newfielda,'%Y-%m-%d %H:%i:%s'),newfielda,fielda),
fieldb=IF(fielda<STR_TO_DATE(newfielda,'%Y-%m-%d %H:%i:%s'),newfieldb,fieldb);
in this case fieldb would never be updated because of the <! you need to move the update of fielda below it or check with <= or =...!
INSERT INTO mytable(fielda, fieldb)
VALUES("2022-01-01", 97)
AS NEW(newfielda, newfieldb, newfieldc)
ON DUPLICATE KEY UPDATE
fielda=IF(fielda<STR_TO_DATE(newfielda,'%Y-%m-%d %H:%i:%s'),newfielda,fielda),
fieldb=IF(fielda=STR_TO_DATE(newfielda,'%Y-%m-%d %H:%i:%s'),newfieldb,fieldb);
This works as expected with using = since fielda is already updated to its new value before reaching the if clause of fieldb... Personally i like <= the most in such a case if you ever rearrange the statement...
you can use insert ignore for such case, it will ignore if it gets duplicate records
INSERT IGNORE
... ; -- without ON DUPLICATE KEY
I have an update query using typeorm on a postgresql database, like the one below, which is performed on a list of 20+ items frequently (once every 30 sec). It takes approx. 12 seconds for the update, which is a lot for my limits.
for (item of items) {
await getConnection().createQueryBuilder().update(ItemEntity)
.set({status: item.status, data: item.data})
.whereInIds(item.id).execute();
}
Is it possible to perform such a bulk update in a single query, instead of iterating other the items? If so - how?
item.status and item.data are unique for each item.
There is a way to do a workaround for this through upsert
Using an array of data that is already on the db and using ON CONFLICT to update it.
const queryInsert = manager
.createQueryBuilder()
.insert()
.into(Entity)
.values(updatedEntities)
.orUpdate(["column1", "column2", "otherEntityId"], "PK_table_entity")
.execute();
will run something like:
INSERT INTO entity (
"id", "column1", "column2", "otherEntityId"
) VALUES
($1, $2, $3, $4),
($5, $6, $7, $8),
ON CONFLICT
ON CONSTRAINT "PK_table_entity"
DO UPDATE SET
"column1" = EXCLUDED."column1",
"column2" = EXCLUDED."column2",
"otherEntityId" = EXCLUDED."otherEntityId"
But you need to be aware that orUpdate does not support using Entity relations, you will need to pass the id column of a relation entity. It also doesnt do any manipulation for the naming strategy. Another problem is that it only works if you're not using #PrimaryGeneratedColumn for your pk (you can use #PrimaryColumn instead)
Using pure psql this can be done as described in the answers to: Update multiple rows in same query using PostgreSQL
However, the UpdateQueryBuilder from Typeorm does not support a from clause.
For now, I think that a raw query is the only way, i.e. getManager().query("raw sql ...").
I have the following postgreSQL query (a simple example of users and their messages linked through user id):
SELECT users.id,
COUNT(messages.id) as "user_messages",
COUNT(messages.id) FILTER (WHERE messages.status='sent') as "sent_messages"
FROM users
LEFT JOIN messages ON messages.user_id=users.id
GROUP BY users.id;
This query runs fine in psql, but I have no idea how to add the FILTER statement in my code using Typeorm's querybuilder and docs are not helping much.
Currently, my approach is something like:
queryBuilder.select('users.id', 'user_id')
.addSelect('COUNT(messages.id)', 'user_messages')
.addSelect('COUNT(messages.id)', 'sent_messages') // <- Add magic filter here?
.from('users', 'users')
.leftJoin('messages', 'messages', 'users.id = messages.users_id')
.groupBy('users.id')
I'm trying to avoid using a subquery in each select if possible as the FILTER approach in the first code seems way simpler.
One or more of these should work:
.addSelect('COUNT(messages.id) FILTER (WHERE messages.status = 'sent')', 'user_messages')
.addSelect('COUNT( (messages.status = 'sent')::int )', 'user_messages')
.addSelect('SUM(CASE WHEN messages.status = 'sent' THEN 1 ELSE 0 END)', 'user_messages')
I found a feasible workaround by adding the whole filter to the select string:
queryBuilder.select('users.id', 'user_id')
.addSelect('COUNT(messages.id)', 'user_messages')
.addSelect(`COUNT(messages.id) FILTER (WHERE messages.status='sent')`, 'sent_messages')
.from('users', 'users')
.leftJoin('messages', 'messages', 'users.id = messages.users_id')
.groupBy('users.id')
It is far from ideal as the whole filter is injected as a string and I cannot use any of the parameters utilities from TypeORM.
I'm not sure if there is a better solution.
I'm quite new with couchDB and I need a little support.
In MySQL I could run simply this query:
SELECT `name`, `id`, `desc` FROM `table`
WHERE `name`="jack" OR `cat` LIKE "%|52224|%";
And here are my two problems:
I started to create a view (still without Like option and everything):
function(doc) {
emit([doc.name, doc.cat], {
"name" : doc.name,
"desc" : doc.desc,
"id" : doc._id
});
}
1. When I use "emit([doc.name" the string must match 100% (also case sensitive).
-> How I make this option case un-sesnsitive?
That I can ask for ("Jack, jack, jAck, JAck,...) like in mysql?
2. How I create the OR option?
When I use [doc.name, doc.cat], I'm also forced to ask for both vars.
But when I have just one of them,
how I can query without creating for each option an own view?
To implement case insensitive search you just have to convert keys to lowercase:
emit([doc.name.toLowerCase(), doc.cat.toLowerCase()])
Now, if you convert your query to lowercase you'll have case insensitive matching.
The problem with this solution is that if you want both case sensitive and case insensitive search you can either emit both values:
[doc.name.toLowerCase(), doc.name, doc.cat]
and use startkey and endkey to filter results, or create a separate view.
The second question is a bit more tricky to implement.
First of all, if you need to filter by doc.name only you can send request with startkey=["jack",0] and endkey=["jack",'zzzzzz'] which will return all documents with doc.name="jack" and doc.cat between 0 and 'zzzzzz' (there probably is a better way to say "any cat", unfortunately I can't find it right now).
If you need a real OR, then you should emit two rows for each document:
emit(doc.name, doc); emit(doc.cat, doc);
This way you can POST needed keys with your request: {"keys": ["jack", "cat_name"]}
This will return every document with either "jack" OR "cat_name" key. However documents that have both will be returned twice, so you have to filter duplicates in your application code.
You can also use couchdb-lucene which will solve both of your problems and probably a lot more. It is a popular choice among couchdb users for implementing advanced queries.