I have data stored in a MongoDB database and I'm using Mongoose to query the data. I'm trying to run date queries against my data to return objects from the database that fall within the specified data-range.
My webform sends an API request to an external micro-service/api that is responsible for querying the data in the Mongo database. The API receives a single value that is representative of a number of days. eg: date: "7d". I then proceed to build the mongoose query like so:
if (data.date) {
const date = new Date();
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
date.setDate(date.getDate() - dateRange);
query.start = { $lte: date.toISOString() };
console.log(query);
}
and an example date from my database is:
start: 2016-02-10T09:09:01.000Z
The query is then executed:
Call.find(query, function (error, docs) {
if (error) {
callback(error, null);
} else {
callback(null, docs);
}
});
and finally, an example of query:
{ approved: 1, start: { '$lte': '2016-02-09T14:24:29.115Z' } }
However, no matter what date I send to my API the response is always empty...
Use the actual date object for your query, not string as you are doing presently. Because mongo stores dates wrapped with the ISODate helper and the underlying BSON (the storage data format used by mongo natively) has a dedicated date type UTC datetime which is a 64 bit (so, 8 byte) signed integer denoting milliseconds since Unix time epoch, your query doesn't return anything as it will be comparing the date fields in mongo with an ISO formatted string.
So, drop the toISOString() conversion and use the date object:
if (data.date) {
const date = new Date();
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
date.setDate(date.getDate() - dateRange);
query.start = { $lte: date };
console.log(query);
}
Call.find(query, function (error, docs) {
if (error) callback(error, null);
callback(null, docs);
});
Better yet, you can use the momentjs plugin that has a very intuitive and easy datetime manipluation API. One method you can use is the subtract() function to get the date object n number of days ago:
if (data.date) {
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
const date = moment().subtract(dateRange, "days");
query.start = { $lte: date };
console.log(query);
}
Call.find(query, function (error, docs) {
if (error) callback(error, null);
callback(null, docs);
});
Related
I am fetching Date saved in db. Then, I am doing a small date maths to substract date from today from 3, which is giving me Date in Format - (DD-MM-YYYY). Date saved in db format is also same - (DD-MM-YYYY). Can anyone help me out in validating $lte for that date. I am not getting any log for DipData.
nodeCron.schedule("* * * * *", async function () {
var DateNow = await moment().subtract(3, "days").format("DD-MM-YYYY");
console.log("Test Date Cron",DateNow);
console.log("-->",new Date(DateNow.format("DD-MM-YYYY")));
let DipData = await userModel.find({}, { LastAppOpenedTime: { $lte : new Date(DateNow.format("DD-MM-YYYY")) }})
console.log("-----DipData ------->", DipData);
});
First thing you need to identify if there is date which is stored in document of mongo collection is string or regular date format or epoch format. If it's string the query may gives not accurate result. If there is date format or epoch format, you can easily queried your result with proper result.
Therefore in case if there is string in LastAppOpenedTime document key you can have query with $toDate under find query.
If key is not in string format in stored document following code will work.
var DateNow = moment().subtract(3, "days");
const DipData = await userModel.find({ LastAppOpenedTime: { $lte: new Date(DateNow) } });
For the above two scenario would work if your query is in accurate form like removing the first empty braces.
userModel.find({}, { LastAppOpenedTime: { $lte : new Date(DateNow) }})
to
userModel.find({ LastAppOpenedTime: { $lte : new Date(DateNow) }})
Hello I got this working by making a few changes
const DateNow = await moment().subtract(3, "days");
console.log("Test Date Cron", DateNow);
console.log("-->", new Date(DateNow));
const DipData = await userModel.find({ createdAt: { $lte: new Date(DateNow) } });
console.log("-----DipData ------->", DipData);
res.status(200).json({ success: true, message: "Request was successful", DipData });
I noticed you had the .format("DD-MM-YYYY") at the end of your moment function but it returned a string that couldn't be converted with the new Date(DateNow). I removed mine when testing as the response from the moment was working fine without it.
And also I updated your userModel.find({}, { LastAppOpenedTime: { $lte : new Date(DateNow.format("DD-MM-YYYY")) }}) to remove the first empty {}. So you would have userModel.find({ createdAt: { $lte: new Date(DateNow) } });
I expect to get a date from my database in Ymd format.
This gives me values like 20200202. This is fine for php applications but I'm using JavaScript for the frontend.
In php, we could do something like
$date = DateTime::createFromFormat('Ymd', '20200202');
meaning I get a date object as long as the formats match.
Is there a way for JavaScript to do this?
If you are sure this date will always come in the format yyyymmdd, you can use RegEx to extract the date :
function getDate(inputDate)
{
// TODO : Check if the format is valid
const pattern = /(\d{4})(\d{2})(\d{2})/
const parts = inputDate.match(pattern);
// months start with 0 in js, that's why you need to substract 1
// -----------------------v----------v
return new Date(parts[1], parts[2] - 1, parts[3]);
}
console.log(getDate("20200202").toString());
console.log(getDate("20200213").toString());
console.log(getDate("20201231").toString());
You could always check if the format matches your requirements and then split the value you get from the database. Then you could either return a Boolean value false if not possible or a date.
const dbOutput = '20200202';
const createdDate = getDate(dbOutput);
function getDate(value) {
if(value.length !== 8) {
return false;
}
const year = +value.substring(0,4);
const month = +value.substring(4,6);
const day = +value.substring(6,8);
try {
return new Date(year, month-1, day);
} catch (error) {
return false;
}
}
I want to find some entries in my DB through the createdAt column, but just using the date. I am using postgres and the createdAt is a timestamptz. Here is an example of what an entry in it looks like: 2019-02-27 20:17:07.05+00
This is what the setting of my query looks like:
const dateString = momentDate.format('YYYY-MM-DD')
query.createdAt = { $iLike: `%${dateString}` }
Unfortunately this is not working and I am getting the following error:
retry-as-promised:error SequelizeDatabaseError: operator does not exist: timestamp with time zone ~~* unknown
Is the issue perhaps because I am using a string? What is the right way to query by date?
Using function DateDiff to get days from between createdAt and dateString. Need to cast dateString to datetime
var sequelize = require('sequelize');
var dateString = '01/03/2019';
yourModel.findAll({
where: sequelize.literal(`DATEDIFF(day,Cast(${dateString} as datetime), createdAt) = 0`)
})
.then(results => {
})
.catch(err => {
});
I'm not quite understanding the timestamp usage,
e.g.
User create article and they can choose PublishDate , the system also store CreateDate automatically.
a. Should I make PublishDate and CreateDate timestamp with time zone and set utc?
b. User post string and then I convert like below use momentjs to utc timestamp and store, when someone select this row , show them as user client time reverse use momentjs
c. I use CURRENT_TIMESTAMP to the CreateDate, the CURRENT_TIMESTAMP does that mean the server time? am I doing correct?
My thinking is I always insert utc timezone timestamp to the database, and wherever the place user/client read, convert the data to user/client timezone? am I doing correct?
a. my database(postgres) created by
CREATE TABLE IF NOT EXISTS "Article"(
"ArticleId" SERIAL NOT NULL,
"PublishDate" timestamp with time zone,
"Active" bit NOT NULL,
"CreateByUserId" integer,
"CreateDate" timestamp with time zone,
PRIMARY KEY ("ArticleId")
);
SET timezone = 'UTC';
b. user submit post to store (nodejs)
// publishDate: '{"y":2015,"m":8,"d":16,"h":15,"mi":46,"s":24}
var publishDate = JSON.parse(req.body.publishDate);
var leadingZeroAndDateFormat = function(publishDate) {
return new Promise(function (fulfill, reject){
if (publishDate.m < 10) { publishDate.m = '0'+publishDate.m; }
if (publishDate.d < 10) { publishDate.d = '0'+publishDate.d; }
if (publishDate.h < 10) { publishDate.h = '0'+publishDate.h; }
if (publishDate.mi < 10) { publishDate.mi = '0'+publishDate.mi; }
if (publishDate.s < 10) { publishDate.s = '0'+publishDate.s; }
var str = publishDate.y+'-'+publishDate.m+'-'+publishDate.d+' '+publishDate.h+':'+publishDate.mi+':'+publishDate.s;
var utc = moment(str).unix();
fulfill(utc);
});
};
c. insert to database the CreateDate use CURRENT_TIMESTAMP
var insertArticle = function(publishDate, active, createByUserId) {
return new Promise(function (fulfill, reject){
var query = 'INSERT INTO "Article" ("PublishDate","Active","CreateByUserId","CreateDate") VALUES ($1,$2,$3,CURRENT_TIMESTAMP) RETURNING "ArticleId"';
dbClient.query(query,[publishDate,active,createByUserId], function(error, result) {
if (error) {
reject(error);
} else {
fulfill(result);
}
});
});
};
Update
When I change all column without timezone then I execute insertArticle shows the error
{ [error: date/time field value out of range: "1439717298"]
name: 'error',
length: 158,
severity: 'ERROR',
code: '22008',
detail: undefined,
hint: 'Perhaps you need a different "datestyle" setting.',
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'datetime.c',
line: '3775',
routine: 'DateTimeParseError' }
var insertArticle = function(publishDate, active, createByUserId) {
return new Promise(function (fulfill, reject){
var query = 'INSERT INTO "Article" ("PublishDate","Active","CreateByUserId","CreateDate") VALUES ($1,$2,$3,$4) RETURNING "ArticleId"';
dbClient.query(query,[publishDate,active,createByUserId,moment.utc().unix()], function(error, result) {
if (error) {
reject(error);
} else {
fulfill(result);
}
});
});
};
The simplest way is to always store time stamps without time zone and in UTC. This way it is always easy to use them for display and calculations. A difference is just a subtraction and comparisons go directly.
If the column was a time stamp with time zone, then the input would be converted to UTC anyway, but the output would be in the currently set time zone and if it's not set properly it might show wrong values. It also makes the database less efficient.
In the presentation layer the timestamps can be shown in the proper time zone and values input can also be converted to UTC for storage.
This is the simplest, most efficient and most flexible way of handling timestamps.
Using CURRENT_TIMESTAMP will get the timestamp from the server at the time of execution.
The incoming timestamps without timezone are parsed in local time instead of UTC -- I'd call this a bug in node-postgres. Anyway, you can override it to do the right thing by adding the following code:
pg.types.setTypeParser(1114, function(stringValue) {
console.log(stringValue);
return new Date(Date.parse(stringValue + "+0000"));
})
I use Mongoose in my project and in one of the schemas I use Date field, like this:
reservationDay: {
type: Date,
default: Date.now
}
I can create document with this field without a problem. And when I console.log this date after it has been created I get:
reservationDay: Thu Nov 20 2014 04:45:00 GMT+0100 (CET)
However, there is a problem when I try tu update this document (even without changing reservationDay field). Here is the error message I get:
name: 'CastError',
message: 'Cast to date failed for value "20/11/2014 04:11" at path "reservationDay"' } CastError: Cast to date failed for value "20/11/2014 04:11" at path "reservationDay"
Here is how I update this document on server side:
Reservation.findOne({ _id: id }).exec(function(err, reservation) {
if (err) {
//handle
}
if (!reservation) {
//handle
}
reservation = extend(reservation, req.body);
reservation.save(function(err, updatedReservation) {
if (err) {
//handle
}
return res.json(reservation);
});
});
and the date is not changed in the req.body.
Any ideas what might cause this error?
Thanks!
"20/11/2014 04:11" is an invalid ISODate format. Somewhere you are converting reservationDate into a string of an invalid format. You need to convert it to a valid ISODate format (month before day):
new Date("11/20/2014 04:11") // Thu Nov 20 2014 04:11:00 GMT+0100 (CET)
new Date("2014/11/20 04:11") // Thu Nov 20 2014 04:11:00 GMT+0100 (CET)
new Date("20/11/2014 04:11") // Invalid Date
for easy manipulation of date formats, I'd recommend using moment.js
Convert date to MongoDB ISODate format in JavaScript using Moment JS
MongoDB uses ISODate as their primary date type. If you want to insert a date object into a MongoDB collection, you can use the Date() shell method.
You can specify a particular date by passing an ISO-8601 date string with a year within the inclusive range 0 through 9999 to the new Date() constructor or the ISODate() function. These functions accept the following formats:
new Date("<YYYY-mm-dd>") returns the ISODate with the specified date.
new Date("<YYYY-mm-ddTHH:MM:ss>") specifies the datetime in the client’s local timezone and returns the ISODate with the specified datetime in UTC.
new Date("<YYYY-mm-ddTHH:MM:ssZ>") specifies the datetime in UTC and returns the ISODate with the specified datetime in UTC.
new Date() specifies the datetime as milliseconds since the Unix epoch (Jan 1, 1970), and returns the resulting ISODate instance.
If you are writing code in JavaScript and if you want to pass a JavaScript date object and use it with MongoDB client, the first thing you do is convert JavaScript date to MongoDB date format (ISODate). Here’s how you do it.
var today = moment(new Date()).format('YYYY-MM-DD[T00:00:00.000Z]');
console.log("Next day -- " + (reqDate.getDate() + 1))
var d = new Date();
d.setDate(reqDate.getDate() + 1);
var tomorrow = moment(d).format('YYYY-MM-DD[T00:00:00.000Z]');
You can pass today and tomorrow object to MongoDB queries with new Date() shell method.
MongoClient.connect(con, function (err, db) {
if (err) throw err
db.collection('orders').find({ "order_id": store_id, "orderDate": {
"$gte": new Date(today), "$lt": new Date(tomorrow)}
}).toArray(function (err, result) {
console.log(result);
if (err) throw err
res.send(result);
})
})
I would like to update one point from my mistake.
check if you have missed leading zero for a single digit time.
"2017-01-01T8:00:00" is invalid and will give the mentioned error.
"2017-01-01T08:00:00" is valid