So I have an array that consists of some number of objects that have both StartTime and Duration member variables. The startTime variable is a string in the format "14:20" and the Duration variable is a number, representing a number of minutes.
Can you guys recommend the best logic for going through the array and combining overlapping elements. For example, if the array had two objects, with the first one having a startTime of "00:00" & duration of 60 and the second object having a startTime of "08:00" & duration of 120 they would be left as separate elements BUT if the second object had a start time of "00:30" and a duration of 120, the two would be combined into one object with a startTime of "00:00" and a duration of 150.
I have been stuck on the general logic of this for some time, because I can't figure how to handle the case when two blocks are combined, but the new combined block creates a new overlap that must be handled. Am I just thinking about this wrong?! Usually I'm good with this sort of thing but am seriously struggling here.
I would do this by creating two trees, one indexed by StartTime and the other indexed by EndTime (StartTime + Duration). Search both trees for elements that overlapped. Once the search was complete any found elements would be removed from both trees and the single new element (StartTime = minStartTime(searchResult), EndTime = maxEndTime(searchResult)) would be inserted into both trees.
Uses a little extra storage but I believe this makes your overlap problem trivial.
(Tree implementation left as an exercise :)
expanding on #gavgrif...
i recommend something along these lines.
create new array field ['start'] in minutes from midnight (14:20 -> 188)
sort array by 'start' field
loop through array looking for overlapping items to merge into segments. where overlapping is that start of this new item is not greater than the segment end.
php example:
$segment = array('start' => -1, 'end' => -1); // force new segment on init
$segments = array();
foreach ($items as $item) {
if ($item['start'] > $segment['end']) {
// add current segment (ignore initial fake segment) to segments
if ($segment['start'] > -1) {
// close out current segment
$segments[] = $segment;
}
// start new segment
$segment = $item;
} else {
// increment segment end if its later than current segment end
$segment['end'] = max($segment['end'], $item['end'];
}
}
// close out final segment
$segments[] = $segment;
Related
I'm working on some code where a value can change over time and I need to keep track of that value over time so I can get the most recent value for any timestamp. I'm using Moment.js for timestamp manipulation.
I'm optimizing for a case where typically the value pair will never change over hundreds of thousands of timestamps, or rarely change, but I still need to know if / when it does, and what it was most recently at any point in time. Timestamp queries before the earliest recorded pair should come back false.
class Timeline {
constructor() {
this.valuePairs = []
}
setValueAt(value, timestamp) {
this.valuePairs.push({ timestamp, value })
}
getValueAt(timestamp) {
// obviously I could iterate through this.valuePairs brute force, get
// the difference between each value-pair's timestamp and the target
// timestamp, returning the value with the smallest difference, but I
// feel bad about even writing this code, it's incredibly
// inefficient, and for big datasets, this would create a massive
// array to search through and take up memory
let earliest = Infinity
let closest = {
value: false,
difference: Infinity,
}
for (let i = 0; i < this.valuePairs.length; i++) {
let valuePair = this.valuePairs[i]
let difference = timestamp - valuePair.timestamp
if (valuePair.timestamp < earliest) earliest = valuePair.timestamp
if (difference < closest.difference) {
closest.difference = difference
closest.value = value
}
}
if (closest.value && timestamp > earliest) {
closest.value
} else {
return value
}
}
}
This is so inefficient that it basically would crash any program processing large batches of data. But I honestly couldn't think of a good way to go about it. What's a more efficient way to write this class?
I think you can use Trie Tree to improve efficiency of your program. To use Trie Tree, you should covert timestamp to year, month, date, hour, minute, second and millisecond. And every element represents a node in Trie Tree as shown in the below. When you search a timestamp to find its value, you can look up a node in the Trie Tree which have a same year as the timestamp, then the same month and so on. If there's not a node which has the same year as the timestamp, you should look up the nearest neighbor. And this algorithm has constant time complexity. In the worst case, you should compare every millisecond and find the nearest value, and this will cost O(1000) time.
As stated in the heading - we're trying to use the data out of the mosaic for a project but the data will only make sense when the mosaic is actually completed. So if the area we choose is too big, the satellite might not get over the spot in the span of a day which would make the result incomplete.
So is there a way to only get the results of 100% complete mosaics?
Here is the code we're using:
// Difference in days between start and finish
var diff = finish.difference(start, 'day');
// Make a list of all dates
var range = ee.List.sequence(0, diff.subtract(1)).map(function(day){return start.advance(day,'day')});
// Funtion for iteraton over the range of dates
var day_mosaics = function(date, newlist) {
// Cast
date = ee.Date(date);
newlist = ee.List(newlist);
// Filter collection between date and the next day
var filtered = collection.filterDate(date, date.advance(1,'day'));
// Make the mosaic and clip to aoi
var clipped = ee.Image(filtered.mosaic().clip(aoi)).setMulti({'system:time_start': date});
var footprint = ee.Feature(clipped.get('system:footprint'));
// Add the mosaic to a list only if the collection has images
return ee.List(ee.Algorithms.If(footprint.area().eq(aoi.area()), newlist.add(clipped), newlist));
};
// Iterate over the range to make a new list, and then cast the list to an imagecollection
var mosaic = ee.ImageCollection(ee.List(range.iterate(day_mosaics, ee.List([]))));
print(mosaic);
You've already got the idea of looking at the footprint, but it would be more accurate to
use the mask pixels instead of the footprint geometry, in case there are any holes in the image, and
instead of counting the valid areas, find out if there are any invalid areas.
var missingPixels = image
.mask() // Get the mask data as an image
.reduce(ee.Reducer.min()) // Take the least coverage across all bands
.expression('1 - b(0)') // Invert the value
.rename(['missing_pixels']); // Give the result a better name
// Sum up missing pixel indications; if nonzero, some are missing. Pick a scale that is good enough for your purposes.
var noMissingPixels = missingPixels.reduceRegion({
geometry: aoi,
reducer: ee.Reducer.sum(),
scale: 100
}).getNumber('missing_pixels').eq(0);
print(noMissingPixels);
https://code.earthengine.google.com/2dff320b7ab68e27e3acbd96efa23e4b
Additional advice: iterate() to collect a list result is inefficient and should be avoided whenever practical. When you want to process a collection and discard some elements, you can use map() with the dropNulls parameter set to true, and return null for any element you don't want to keep:
var day_mosaics = function(date) {
... regular code goes here, but no 'newlist' ...
return ee.Algorithms.If(noMissingPixels, clipped, null);
};
var mosaic = ee.ImageCollection(range.map(day_mosaics, true));
This will work well with very large collections whereas building a list may run out of memory.
Given an array whose elements are objects in the form:
{
"description": "foo",
"startTime": 0,
"endTime": 100
}
I need to find all elements for which startTime <= t <= endTime.
(endTime - startTime) >= 100, but there is no explicit upper bound on the time differential, nor is the differential necessarily the same for each object in the array. startTime can be negative.
arr[i].startTime <= arr[i + 1].startTime, but the same is not necessarily true for endTime.
description is not unique, not even necessarily for a given time span.
Times correspond to milliseconds in a video, which can easily be an hour or more. If a 1hr video has 100ms duration objects for its entire length, that's 36,000 array elements to filter. Depending on the video, there could easily be a dozen objects for a single 100ms timespan.
My current solution is a simple Array.prototype.filter call:
video.getCurrentTime((s) => { // library function; s=current time in seconds of the video
const ms = Math.floor(s * 1000);
const itemsAtTime = metadata.filter((o) => { // metadata is array of objects
const start = o.startTime;
const end = o.endTime;
return start <= ms && ms <= end;
});
// ...
for (let obj of itemsAtTime) {
// do stuff
}
});
As best as I can tell, filter is implemented as a linear search. Is there any better algorithm available to achieve my goal? Perhaps some variant of binary search?
On my 2 minute demo video/metadata that I'm testing with, most of the filter calls complete in about 0.7ms. However, a number of them take 1-5ms, and I've tracked some extreme outliers like 11ms or even 33ms. My "do stuff" loop usually completes in 0.1ms, with heavy load taking 4ms and outliers at 12ms. Worst of all, the getCurrentTime function is asynchronous, generally taking 1-5ms between calling it and having the callback get called, with heavy load in the neighborhood of 50-60ms and outliers upwards of 500ms. Considering this code is in a function being passed to setInterval, with the interval running while the video is playing (currently with a 250ms interval, but ultimately I think I'd like to use a 100ms interval or less), I'm worried about the performance when I start using hour-plus duration videos.
You could make a binary search to find the first item where startTime <= ms. Then you can filter linearly from that index onward. That's the best I can think of.
So, something like:
const binaryWhereMin = function(arr, ms) {
// write your function that returns the index of first element start <= ms
};
let index = binaryWhereMin(arr, ms);
let result = arr.splice(index, arr.length).filter(/* your filter */);
I can't see a better way, since ends are not sorted.
Here there is a binary search example. but you will need to alter it not to search for a specific element (it's not going to be as efficient!, but I believe it's still better than what you have).
Backstory and Context: I will do my best to explain what I am working on, where the problem is that I have identified, and my thought process for my current code. Bear with me as I will try to provide as much detail as possible. Note that I am still learning Quartz Composer and am rather unfamiliar with Javascript syntax.
I am working with Quartz Composer to modify the visual equalizer template that comes with the program. Essentially, my audio feed gets input into the program where it is processed as 16 audio frequency bands that are stored in an array. Each of these 16 bands corresponds to the magnitude of the frequency in that range, and will ultimately display on a "bar graph" with the various equalizer levels. This part is all well and good.
My task requires that I have more than 16 bars in my graph. Since I cannot split the audio any further without turning to a somewhat complicated external process, I figured I could just fake my way through by inserting fake bar and values in between the actual audio bars that would average or evenly space themselves in between the true audio bar values.
For example, suppose that my input array looked as such "A B C" where "A," "B," and "C" are some audio value. I declare in my code a certain "insertion" value that determines how many fake bars are to be added in between each of my true audio values. In the case of "A B C," suppose that the insertion value were set to 0. No bars are to be inserted, thus the output array is "A B C." Suppose that the insertion value is set to 1. 1 bar is to be inserted in between each of the true values, returning an array of "A (A+B)/2 B (B+C)/2 C." As such, if the insertion value is set to 2, two values will be placed between A and B such that they are evenly spaced in between, aka 1/3 and 2/3rds of the way between A and B respectively.
Relevant Code:
var array = new Array();
function (__structure outputStructure) main (__structure inputStructure, __number insertions, __number time) {
var result = new Object();
/* check if the input array for the sound bands is null or not */
if (inputStructure != null) {
/* keep track of the number of times that a value is inserted so that, upon the counter being reset, a value from the original array can be inserted */
var counter = 0;
/* keep track of which index location the original array value is to be pulled from */
var inputPlace = 0;
/* build a new array that inserts a number of values equal to the value of the insertions variable in between all values of the original array */
for (i=0; i<((insertions + 1) * (inputStructure.length - 1) + 1); ++i) {
/* if it is time to do so, pull a true audio bar value from the original string into the new string */
if (counter = 0) {
array[i] = inputStructure[inputPlace];
/* if it is not time to pull a value from the original array, insert the specified number of evenly spaced, calculated bar values into the new array */
} else {
/* space the insertion between the nearest two true values from the original input array */
array[i] = inputStructure[inputPlace] + ((inputStructure[inputPlace + 1] - inputStructure[inputPlace]) * counter / (insertions + 1));
}
counter = counter + 1;
/* reset the counter after each complete iteration of insertions is compelte so that an original array value can be pulled into the new array */
if (counter > insertions) {
counter = 0;
inputPlace = inputPlace + 1;
}
counter = counter + 1;
}
}
result.outputStructure = array;
return result;
}
In this particular code, I do not have any inputs manually declared, as they will all be pulled in from Quartz Composer. I have had no problems with this part. I assume for testing purposes, you could just hard code in values for the inputStructure, insertion, and time.
Problem:
The problem that I have identified seems to be with the if statement inside of my for loop. When I run the code, it sets every value in the new array to 0.
Question: I guess my question is what am I doing wrong with the if statement in the for loop that is causing it to overwrite all of my other iteration values? I was able to get the counter working such that each iteration would return the counter value "0 1 2 0 1 2" in the case of the insertion being set to 2, but I was unable to say okay, now check during each iteration, and if the counter is 0, pull a value from the original array. Otherwise, do something else, in this case, calculate and add the fake bar value into the new array.
Your problem is this line:
if (counter = 0) {
It will always return false, to compare you need to use == or ===
I'm working on a Javascript/jQuery calendar which includes a month view and a day view. Clicking the days will change the date, which will update the date variables in the day view.
The day view is split up into half hour segments from midnight to 11:00 PM. Clicking on any half hour <tr> (the day view is a table) will create an event between that time clicked and an hour in the future, as well as append a div on top of the calendar, spanning the range of time and positioned at the correct starting point (each pixel is a minute...)
There is a problem, however. If you create an "event" between a certain time span where there is already one in place, they overlap. This is the default behavior, obviously, but what I would like to happen is that if an event is created between a range of dates that is already occupied by an event, they align side by side so that they're not overlapping.
This resembles the behavior seen in the iCal app for mac:
Now my first thought to achieve such a goal was to use collision detection, but all the jQuery plugins for this are bloated or require the elements to be draggable.
Then I thought there might be a way in CSS to do this, where if two elements are overlapping, they split the width evenly.
Then I thought that's ridiculously far fetched, so I'm wondering how I can achieve this as easily as possible.
I'll post the full code in a jsFiddle, but for the most important function would be insertEvent which looks like this:
function insertEvent(start, end){
var end_minutes = new Date(end).getMinutes();
var end_border = new Date(new Date(end).setMinutes(end_minutes + 2));
//$(".day_date").html(start + "<br />" + end);
var diff = Math.abs(end_border - new Date(start));
var minutes = Math.floor((diff/1000)/60);
var start_element = $("td").find("[data-date='" + start + "']");
var offset = start_element.offset().top - $(".second").offset().top;
var this_element = $("<div class='event' style='height:" + minutes + "px;margin-top:" + offset + "px;'></div>");
$(".right").prepend(this_element);
}
This takes two parameters in the javascript new Date() format, one for the start date and one for the end date.
The fiddle: http://jsfiddle.net/charlescarver/HwdwL/
One of the the problems I see with your approach is that there isn't a structure to the storage of the data. I've built a calendar in Javascript before and it's not easy work. First, make sure you have some kind of abstraction for the calendar event. Something like:
function CalendarEvent(startDateTime, endDateTime) {
this.startDateTime = startDateTime;
this.endDateTime = endDateTime;
}
CalendarEvent.prototype.start = function() {
return this.startDateTime.getTime();
};
CalendarEvent.prototype.end = function() {
return this.endDateTime.getTime();
};
CalendarEvent.new = function(startDateTime, endDateTime) {
// This is a little factory method. It prevents calendar events
// from having end times that fall before the start time.
// USE THIS TO INSTANTIATE A NEW CALENDAR EVENT
if(endDateTime.getTime() < startDateTime.getTime()) {
throw new Error("End time falls before start time");
}
return new CalendarEvent(startDateTime, endDateTime);
};
CalendarEvent.compare = function(eventOne, eventTwo) {
// this is a class method to compare two events
// If used with sort it will sort by startDateTime
return eventOne.start() - eventTwo.start();
};
// ... add any other methods you need
Next you're going to want to sort the calendar events. I would sort by start time. Then once it is sorted you can actually re-render everything when changes are made. As long as you sort correctly, determining if a calendar event collides is as simple as this:
CalendarEvent.prototype.intersects = function(otherEvent) {
// If the other event starts after this one ends
// then they don't intersect
if(otherEvent.start() > this.end()) {
return false;
}
// If the other event ends before this one starts
// then they don't intersect
if(otherEvent.end() < this.start()) {
return false;
}
// Everything else is true
return true;
};
Because the data is sorted you know that if two or more calendar events intersect they will have to share the space. Granted, you must think about a few things when you divide the space. Do you want a naive implementation where you just share the space equally from left to right (left having the earliest start time). If so your visual representation could look like this if it had 4 events that shared a space (each block is an event):
However if your events have strange shapes they might cause your calendar to look strange. Consider the following:
In this instance event 2 takes up a lot of vertical space and all the space underneath event 1 is unused. Maybe for a better UX you don't want that kind of thing to happen. If so you should design your rendering algorithm accordingly. Just remember that it is probably easiest to re-render on every change that you encounter, but it's all about how you store the data. If you do not store the data in some kind of structure that is easily traversed then you won't be able to do this kind of thing.
But to complete the answer to your question, here is a fairly naive example. I haven't tested it so this is a pretty big assumption of it working. It is not entirely complete you will have to edit the rendering for yourself. This is merely to give you an idea of how to get it to work. It could definitely look prettier:
function renderCalendarEvents(calendarEvents) {
// Sort the calendar events (assuming calendarEvents is an array)
var sortedEvents = calendarEvents.sort(CalendarEvent.compare);
var index = 0;
// renderEvents is an anonymous function that will be called every time
// you need to render an event
// it returns it's columnDivisor.
var renderEvent = function(position) {
var currentEvent = sortedEvents[index];
var nextEvent = sortedEvents[index + 1];
// The default column divisor is determined by
// the current x-position + 1
var columnDivisor = position + 1;
// Increment before any recursion
index += 1;
// Check if nextEvent even exists
if(nextEvent) {
// If the nextEvent intersects with the current event
// then recurse
if(currentEvent.intersects(nextEvent)) {
// We need to tell the next event that it starts at the
// column position that is immediately +1 to the current event
columnDivisor = renderEvent(position + 1);
}
}
// placeEvent() is some function you can call to actually place
// the calendar event element on the page
// The position is the x-position of the current event
// The columnDivisor is a count of the amount of events sharing this column
placeEvent(currentEvent, position, columnDivisor);
return columnDivisor;
};
while(true) {
// render events until we're done
renderEvent(0);
if(index >= sortedEvents.length) {
break;
}
}
}
Essentially the idea with this particular algorithm is that if the nextEvent on the list exists and that event intersects with the currentEvent then we need to split the width of the currentEvent. It keeps on recursing until it finds no more intersections then it makes it's way back up the chain of recursive calls. I skipped the actual DOM manipulation logic because really the hard part is determining how much you need to split the actual column in order to get these events to fit. So hopefully this all makes a little bit of sense.
EDIT:
To be much more clear, in order to add this to your existing code I would replace your insertEvent function with something like this. I don't write all of the logic for you so you'll have to do some of your own writing. But that's half the fun :-).
function insertEvent(start, end) {
var newEvent = Calendar.new(start, end);
// you'll have to store the array somewhere.
// i'm just assuming some kind of global right now
eventsArray.push(newEvent);
// You'll want to destroy any event elements
destroyCurrentEventElements();
// Now run the rendering function
renderCalendarEvents(eventsArray);
}