Develop Reference

Get changes based on uploaded time from firebase

I have initialized a real time database using firebase, I am detecting live changes to the databse using
const ref = firebase.database().ref("test");
ref.on('value', function(dataSnapshot){
console.log(dataSnapshot.val())
});
But this returns me value in ascending order. Whereas I want it to return based on time. I tried using time in: 00:00 (IST) format but if a data is marked 11:59 (am) and another 01:02 (pm) this will return me the second message first.
What will be the best way to fix this?
example data is =>
in my databse =>

It is not clear what you mean by time in ascending order
None of your example data mention time. They are just usernames and text.
If you want to order times correctly, best to use ISO date format
This stores 1:02 pm as 13:02, which will sort after 11:59. Its sorting characteristics are ideal.
Use an international time standard to store your times
An international time standard, UTC, has great advantages over national times. It is not subject to change with location, political decisions, or season. You can always interconvert with the user's local time, at the time of entry or display.
Example
const dateString = (new Date()).toISOString();
console.log(dateString)
// Result:
// 2021-06-22T14:40:37.985Z
// If you want to use them as Firebase keys, they must not contain a ".", so you might clean it up like this:
const cleanDateString = (new Date()).toISOString().replace(".","-")
console.log(cleanDateString)
// Result:
// 2021-06-22T14:47:44-445Z
Even better, use a Firebase PushID
The above date-and-time system will work if you are using it to sort the remarks made by a single person, but will not be good as a message identifier if a single space is shared by all people, since 2 people will eventually make a message at the same millisecond, and will get the same message ID.
To deal with that it is better practice to use a Firebase Push ID.
An explanation is given here: In Firebase when using push() How do I get the unique ID and store in my database
Or from Firebase itself, here:
https://firebase.google.com/docs/database/admin/save-data

how to get a part of a text from a column in the database using tdbnput and print in talend

I have a talend job that goes like the above one and i am using a select statement to access the data from oracle
select id,name,details from employee ;
and send the results via email.However for a particular column details it has 10 different lines and that are dynamic and has various other unwanted/irrelevant information and i would like to precisely filter few lines and print them to the mail using tjava component.
How can i do that?
for eg:details column has values like:
"hi i am john
i work at abc
i go to oxford
i reside at 1st street
goodbye"
so the lines are totally random,i would want to match based on a specific keyword lets say "reside at" alone and filter that for different records and capture that in that output?

In order to do this you need to create a custom SQL query in the tDBinput.
"select id,name,details from employee
where details like '%reside at%'"
(Where the % are any text check the SQL "LIKE" function for more details)
With this you will filter the row you want.
If you need more filter you can modify you SQL query like you want.

Laravel count the IP adressess from this month

I have a visitor counter that works. Now, I need to get the results back to display them into a chart. What I have now in my controller to display them works, but the date is wrong.
This is what I have:
$select_stats = DB::table('visitor')->where('visit_date', '>', '2015-04-01')->where('visit_date', '<', '2015-04-30')->get();
foreach($select_stats as $statics) {
//change our object to an array
$statics = array();
}
return View::make('admin.home.index')->with('stats', $select_stats);
I need to get the 'stats' from the current month, so in this case month 4, but if it is may, then it needs to select automatically the stats from the 5th month.
So that is already a problem for me.
Now, I need to loop them in my blade themplate, that works... But the date is in the format
YYY-MM-DD
I don't want it displaying like that I do prefer the format:
DD/MM/YYYY
Okay, so far, so good.
I also need to count the IP's that are stored in the database. Also from THAT month.
And also need to loop and display the count.
The database model I have:
The table name is called visitor.
My blade template for the chart:
<script type="text/javascript">
var visitors = [
#foreach($stats as $stat)
['{{ $stat->date }}', 500],
#endforeach
];
</script>
Yes, this works in the script.

If you're always dealing with the current month, try this:
$select_stats = DB::table('visitor')->whereBetween('visit_date', array(date('Y-m-01'), date('Y-m-t')))->get();
the lowercase t in the date function returns the number of days in the given month (http://php.net/manual/en/function.date.php).
Simply put, to convert that date format in your template, you could echo it as the following:
{{ date('Y/m/d', strtotime($stat->date)) }};
To get the count of the IP's you could perform the same query but add replace ->get() with ->count() like so:
DB::table('visitor')->whereBetween('visit_date', array(date('Y-m-01'), date('Y-m-t')))->count();
You already have the IP field from the first query, so just echo out {{$stat->ip}}
There's always different & better ways to achieve certain results, but without context, here's a solution that works.

Dynamic Frequency Map from MongoDB Keys

I'm using MiniMongo through Meteor, and I'm trying to create a frequency table based off of a dynamic set of queries.
I have two main fields, localHour and localDay. I expect many overlaps, and I'd like to determine where the most overlaps occur. My current method of doing this is so.
if(TempStats.findOne({
localHour: hours,
localDay: day
})){//checks if there is already some entry on the same day/hour
TempStats.update({//if so, we just increment frequency
localHour: hours,
localDay: day
},{
$inc: {freq: 1}
})
} else {//if nothing exists yet, we put in a new entry
TempStats.insert({
localHour: hours,
localDay: day,
freq: 1
});
}
Essentially, this code runs every time I have new data I want to insert. It works fine at the moment, in that, after all data is inserted, I can sort by frequency to find what set of hours & days occurs the most often (TempStats.find({}, {sort: {freq: -1}}).fetch()).
However, I'm looking more for a way to search by frequency for any key. For instance, searching for the day which everything occurs on the most often as opposed to both the date and hour. With my current way of doing this, I would need to have multiple databases and different methods of inserting for each, which is a bit ridiculous. Is there a Mongo (specifically MiniMongo) solution to do frequency maps based on keys?
Thanks!

It looks like miniMongo does not in fact support aggregation, which makes this kind of operation difficult. One way to go about it would be aggregating yourself at the end of each day and inserting that aggregate record into your db (without the hour field or with it set to something like -1). Conversely as wastefully you could also update that record at the time of each insert. This would allow you to use the same collection for both and is fairly common in other dbs.
Also you should consider #nickmilon's first suggestion since the use of an upsert statement with the $inc operator would reduce your example to a single operation per data point.

a small note on your code: the part that comes as an else statement is not really required your update will do the complete job if you combine it with the option upsert=true it will insert a new document and $inc will set the freq field to 1 as desired see: here and here
for alternative ways to count your frequencies: assuming you store the date as a datetime object I would suggest to use an aggregation (I am not sure if they added support for aggregation yet in minimongo) but there are solutions then with aggregation you can use datetime operators as
$hour, $week, etc for filtering and $count to count the frequencies without you having to keep counts in the database.

This is basically a simple map-reduce problem.
First, don't separate the derived data into 2 fields. This violates DB best practices. If the data comes to you this way, use it to create a Date object. I assume you have a bunch of collections that are being subscribed to and then you aggregate all those into this temporary local collection. This is the mapping of the map-reduce pattern. At this point, since your query in unknown, it's a waste of CPU (even though it's your client) to aggregate. Map first, reduce second. What you should have is a collection full of datetimes. call it TempMapCollection if you wish. Now, use a forEach() and pass in your reduce function (by day, by hour, etc).
You can reduce into another local collection, or into a javascript object. I like using collections, but if the objects are complex, you'll get EJSON errors all up in there. Since your objects are nothing more than a datetime, let's use collections.
so you've got something like:
TempMapCollection.find().forEach(function(doc) {
var date = doc.dateTime.getDate();
TempReduceCollection.upsert({timequery: hours}, {$inc: {freq: 1}});
})
Now query your reduce collection. This has the added benefit that you won't have to re-map if you want to do 2 unique queries.

How to read from another row in javascript step of Pentaho?

I'm working on an ETL process with Pentaho Data Integration (Spoon, before Kettle).
In the Modified Javascript step of Pentaho you can set a start, end and transform script. In the transform script you can write code that it will be executed only for each row, and from here I don't know how to access to data of the previous row (if it's possible).
I need access to the previous row because all rows are ordered by product, store and date (respectively), and the goal is to get the quantity on hand from the previous row and add the quantity sell or received on the current row (this would be the same product, same store but different date). I also need accessing to the previous row to compare the product and store of the current row with the previous row, because if someone of them changes I must to restart the field quantity_on_hand (I do it with a field of all columns named initial_stock).
On pseudocode would be something like this (if I hadn't the restriction of that the code written on the step is executed only for each row):
while(all_rows_processed()){
current_row.quantity_on_hand = current_row.initial_stock;
while(id_product_current_row == id_product_previous_row && id_store_current_row == id_store_previous_row){
current_row.quantity_on_hand = previous_row.quantity_on_hand + current_row.stock_variation;
}
}
This question related couldn't help me.
Any ideas to solve my problem would be appreciated.

May I ask you to reconsider Group By step? It seems suitable for your scenario.
If you sort the stream accordingly to your combination date/store/article, you can calculate cumulative sum for sell/received quantity. This way you can have a running total of inventory variation that would be reset on a group basis.
Also give a look both at this blog post and at the forum post it quotes.

I doubt you need to go to JavaScript for this. Check out the Analytic query step. That will allow you to bring a value from the previous row into the current.
The JavaScript step gives you tremendous flexibility, but if you can do it with the regular transform steps, it will typically be much faster.

use Analytic Query. By Using this Step u can access the previous / next record. Actually, not only prev and next record that you can read, but you can read N Rows Fordward or N Rows Back Wards.
Check the following URL for clearer expalanation :
http://wiki.pentaho.com/display/EAI/Analytic+Query
http://www.nicholasgoodman.com/bt/blog/2009/01/30/the-death-of-prevrow-rowclone/

Thanks for all, I've got the solution to my problem.
I've combined all your suggestions and I've used the Analytic Query, Modified Javascript and Group by steps.
Although the question wasn't very well formulated, the problem I had was to calculate the stock level on each row (there was one row for each product, date and store combination).
First (obviously later than sort rows by product_id, store_id and date ascending), I used the Analytic Query step to group by product_id and store_id, because with this step I've got a new field previous_date to identify the first row of each group (previous_date=null on the row of the group where date was the oldest).
Then I needed to calculate the quantity_on_hand of each group [product,store] at first row (first date of each group because it's sorted by date) because the initial_stock is different for each group. This is because of (sum(quantity_received) - sum(quantity sold)) != quantity_on_hand.
Finally (and the key was here), I used the Group by step like #andtorg suggested and do it as the next image shows.
This link that #andtorg suggested was very useful. It includes even two .ktr example files.
Thank you so much for help!