How does AR.js uses its' tracking backend? - javascript

I want to extend AR.js by adding my own tracking backend. However, I have troubles finding any documentation on the architecture of this library or how it interacts with underlying components. Likewise, it'd be useful to have more information on how AR.js relates to ARToolkit, Tango, A-Frame, WebVR, ARCore and WebARonARCore. Since the area is quite new and striving, there are a lot of projects going on simultaneously and it's quite confusing and hard to differentiate their functionality sometimes.
The backend I need to implement is the object recognition based on YOLO. I have a prototype running - Android Unity Tango application, it offloads video captured from the device camera onto the edge node, where it is processed in real-time and the information about recognized objects is sent back to the device, where it is used to render annotations. I'd like to have these annotations to be represented as a-frame tags, in order to make content layering easy by using JavaScript.
Any ideas/pointers are welcomed.

Related

Studying alternatives: integrate SCORM and LMS without frames

I would like to know more about SCORM.
Ive beeing studying moodle module and the SCORM.com documentation but im not arriving to an aproach myself. As far i can tell steps are:
Upload to LMS a SCORM package
Develop a player that contains some controlls to browse the package
Provide a javascript API, called API for SCORM1.2 and API_1484_11 for SCORM2004. Implement then "at least" some basic calls to the API: Terminate, Commit, Intialize, etc. These functions will be called from the packet.
On the player open the package using a frame. ¿Any alternative?
Ive done this all but im not able to make it work. Im just writing this to make sure these are the correct steps for an LMS-SCORM integration.
So, basically my situation is that by the point of view of an LMS, how to show the packet without frames, and how to retrieve data from packets using PHP and javascript.
Maybe im pretending too much but better thing would be to make all types of SCORM packets to work.
Thanks all in advance!
Those are the basic steps, but the devil is in the details. It isn't really enough to only implement '"at least" some basic calls' unless you are producing all of the content that you will play. This is because you can't really know which of the runtime API calls will be made by a given piece of content. Also note that SCORM2004 has a whole sequencing implementation that is needed in order for the course to display and load the proper SCOs, etc.
Generally you can't really implement SCORM without frames (iframes do work), short of popping up new windows for all content which isn't ideal. This is because there is a pre-defined search algorithm the content will be using to locate the SCORM JS API(s) and that algorithm is designed to walk the tree of windows in the browser to locate it. Without frames (or separate windows) this algorithm won't work.
As far as the data, the "packet" (or SCO) is communicating via the JS runtime API so the data is being provided to the player via those calls. It is up to the implementer to decide what to do with that data so that the LMS can make use of it.
Writing a SCORM implementation, particularly a 2004 one, is a non-trivial pursuit.

Parot AR drone controlled by opencv in c++

I'm currently building a texture classifier in the c++ api of opencv. I was looking to use this to recognise textures and ideally help a parot ar drone 2.0 to navigate to a specific texture. I have found the documentation on node copter and it's opencv bindings. I wasn't sure about whether this would require me to re write my program in javascript?
If there is some sort of interface then is it feasible to run my program in the background, pull images from the parrot analyse them and send back control commands to the parrot?
I have been working with opencv for about 3 months and have some basic understanding of node.
Thanks in advance!
There are lots of ways to interface with a Parot AR drone. NodeCopter is one option, but there are others. ROS has good AR drone bindings I've used which would give you tons of flexibility at the expense of some complexity.
You might also consider building your C++ program into a stand-alone option and calling it from Node.js. You could also interface with the AR Drone API directly.
It's not too hard to write a program to control an AR.Drone with some sort of OpenCV-based tracking. Javascript would probably be my suggestion as the easiest way to do that, but as #abarry alluded, you could do it with any language that has bindings for the AR.Drone communications protocol and OpenCV.
The easiest thing would be to have a single program that controls the drone, and processes images with OpenCV. You don't need to run anything in the background.
copterface is a Node.js application that uses node-ar-drone and node-opencv to recognize faces and steer the drone toward them. It might be a good starting point for your application.
Just to give an example in another language, turboshrimp-tracker is a Clojure application that shows you live video from the drone, lets you select a region of the video containing an object, and then tracks that object using OpenCV. It doesn't actually steer the drone toward the tracked object, but that would be pretty easy to add.

Real time collaborative editing - how does it work?

I'm writing an application in which I'd like to have near real time collaborative editing features for documents (Very similar to Google Documents style editing).
I'm aware of how to keep track of cursor position, that's simple. Just poll the server ever half second or second with the current user id, filename, line number and row number which can be stored in a database, and the return value of this polling request is the position of other user's cursors.
What I don't know how to do is update the document in such a way that it won't throw your cursor off and force a full reload as that would be far to slow for my purposes.
This really only has to work in Google Chrome, preferably Firefox as well. I don't need to support any other browser.
The algorithm used behind the scenes for merging collaborative edits from multiple peers is called operational transformation. It's not trivial to implement though.
See also this question for useful links.
Real time collaborative editing requires several things to be effective. Most of the other answers here focus on only one aspect of the problem; namely distributed state (aka shared-mutable-state). Operational Transformation (OT), Conflict-Free Replicated Data Types (CRDT), Differential Synchronization, and other related technologies are all approaches to achieving near-real-time distributed state. Most focus on eventual consistency, which allow temporary divergences of each of the participants state, but guarantee that each participants state will eventually converge when editing stops. Other answers have mentioned several implementations of these technologies.
However, once you have shared mutable state, you need several other features to provide a reasonable user experience. Examples of these additional concepts include:
Identity: Who the people you are collaborating with are.
Presence: Who is currently "here" editing with you now.
Communication: Chat, audio, video, etc., that allow users to coordinate actions
Collaborative Cueing: Features that give indications as to what the other participants are doing and/or are about to do.
Shared cursors and selections are examples of Collaborative Cueing (a.k.a Collaboration Awareness). They help users understand the intentions and likely next actions of the other participants. The original poster was partly asking about the interplay between shared mutable state and collaborative cueing. This is important because the location of a cursor or selection in a document is typically described via locations within the document. The issue is that the location of a cursor (for example) is dependent on the context of the document. When I say my cursor is at index 37, that means character 37 in the document I am looking at. The document you may have right now may be different than mine, due to your edits or those of other users, and therefore index 37 in your document may not be correct.
So the mechanism you use to distribute cursor locations must be somehow integrated into or at least aware of the mechanism of the system that provides concurrency control over the shared mutable state. One of the challenges today is that while there are many OT / CRDT, bidirectional messaging, chat, and other libraries out there, they are isolated solutions that are not integrated. This makes it hard to build an end user system that provides a good user experience, and often results in technical challenges left to the developer to figure out.
Ultimately, to implement an effective real time collaborative editing system, you need to consider all of these aspects; and we haven't even discussed history, authorization, application level conflict resolution, and many other facets. You must build or find technologies that support each of these concepts in a way that make sense for your use case. Then you must integrate them.
The good news is that applications that support collaborative editing are becoming much more popular. Technologies that support building them are maturing and new ones are becoming available every month. Firebase was one of the first solutions that tried to wrap in many of these concepts into an easy to use API. A new-comer Convergence (full disclosure, I am a founder of Convergence Labs), provides an all-in-one API that supports the majority of these collaborative editing facets and can significantly reduce the time, cost, and complexity of building real time collaborative editing apps.
You don't need xmpp or wave for this necessarily. Most of the work on an opensource implementation called infinote already have been done with jinfinote ( https://github.com/sveith/jinfinote). Jinfinote was recently also ported to python ( https://github.com/phrearch/py-infinote) to handle concurrency and document state centrally. I currently use both within the hwios project ( https://github.com/phrearch/hwios), which relies on websockets and json transport. You don't want really want to use polling for these kind of applications. Also xmpp seems to complicate things unnecessarily imo.
After coming upon this question and doing a more careful search, I think the best standalone application to check out would be Etherpad, which runs as a JS browser app and using Node.js on the server side. The technology behind this is known as operational transformation.
Etherpad was originally a pretty heavyweight application that was bought by Google and incorporated into Google Wave, which failed. The code was released as open source and the technology was rewritten in Javascript for Etherpad Lite, now renamed just "Etherpad". Some of the Etherpad technology was probably also incorporated into Google Docs.
Since Etherpad, there have been various versions to this technology, notably some Javascript libraries that allow for integrating this directly into your web app:
ShareJS
ot.js
I am the maintainer of the meteor-sharejs package for adding realtime editors directly to a Meteor app, which IMHO is the best of both worlds :)
As Gintautas pointed out, this is done by Operational Transformation. As I understand it, the bulk of the research and development on this feature was done as part of the now-defunct Google Wave project, and is known as the Wave Protocol. Fortunately, Google Wave is open-sourced, so you can get some good code samples at http://code.google.com/p/wave-protocol/
The Google Docs team did a little bit of a case study around how the real time collaboration worked, but I can't find the blog entry.
There is some decent stuff on the wikipedia page, though:
http://en.wikipedia.org/wiki/Collaborative_real-time_editor
I've recently published a repository with a working example of what seems you're trying to achieve:
https://quill-sharedb-cursors.herokuapp.com
It's based off ShareDB (OT) working as the backend and Quill rich text editor on the frontend.
Basically just wires all these things with some more code to draw the cursors. The code should be fairly simple to understand and to copy over to any specific solution.
Hope it helps with the endeavor.

JavaScript vs AS3 for Interactive Globe

I am developing a web application that should display data points below the Earth's surface. It should also include user navigation, zoom, and rotational capabilities. It reads in the data from a XML file and should display 500-1000 spheres. Ideally, I would like to use HTML5, but it doesn't run in IE8.
This is a Flash application that is somewhat similar to what I want (minus the globe around the spheres):
http://demo.tufat.com/applets/molecule/index.html
There is also a good jQuery application that allows for 3d rotation out there.
So, the question is JavaScript/jQuery or AS3? I know Flash handles large amounts of data better but is Flash dying out? The application must run efficiently on most browsers.
Flash is not dying. I am not sure which Blogging Social Media Expert told you that.
Choose a language and just stick with it. Unless you know how to handle your memory and work with Paper Vision or Unity you should go with JavaScript for most likely you will end up with leaks if you do not do garbage collection, 500-1000 spheres is alot.

Reflective Web Application (WebIDE)

Preamble
So, this question has already been answered, but as it was my first question for this project, I'm going to continue to reference it in other questions I ask for this project.
For anyone who came from another question, here is the basic idea: Create a web app that can make it much easier to create other web applications or websites. To do this, you would basically create a modular site with "widgets" and then combine them into the final display pages. Each widget would likely have its own set of functions combined in a Class if you use Prototype or .prototype.fn otherwise.
Currently
I am working on getting the basics down: editing CSS, creating user JavaScript functions and dynamically finding their names/inputs, and other critical technical aspects of the project. Soon I will create a rough timeline of the features I wish to create. Soon after I do this, I intent to create a Blog of sorts to keep everyone informed of the project's status.
Original Question
Hello all, I am currently trying to formalize an idea I have for a personal project (which may turn into a professional one later on). The concept is a reflective web application. In other words, a web application that can build other web applications and is actively used to build and improve itself. Think of it as sort of a webapp IDE for creating webapps.
So before I start explaining it further, my question to all of you is this: What do you think would be some of the hardest challenges along the way and where would be the best place to start?
Now let me try to explain some of the aspects of this concept briefly here. I want this application to be as close to a WYSIWYG as possible, in that you have a display area which shows all or part of the website as it would appear. You should be free to browse it to get to the areas you want to work on and use a JavaScript debugger/console to ask "what would happen if...?" questions.
I intend for the webapps to be built up via components. In other words, the result would be a very modular webapp so that you can tweak things on a small or large scale with a fair amount of ease (generally it should be better than hand coding everything in <insert editor of choice>).
Once the website/webapp is done, this webapp should be able to produce all the code necessary to install and run the created website/webapp (so CSS, JavaScript, PHP, and PHP installer for the database).
Here are the few major challenges I've come up with so far:
Changing CSS on the fly
Implementing reflection in JavaScript
Accurate and brief DOM tree viewer
Allowing users to choose JavaScript libraries (i.e. Prototype, jQuery, Dojo, extJS, etc.)
Any other comments and suggestions are also welcome.
Edit 1: I really like the idea of AppJet and I will check it out in detail when I get the time this weekend. However, my only concern is that this is supposed to create code that can go onto others webservers, so while AppJet might be a great way for me to develop this app more rapidly, I still think I will have to generate PHP code for my users to put on their servers.
Also, when I feel this is ready for beta testers, I will certainly release it for free for everyone on this site. But I was thinking that out of beta I should follow a scheme similar to that of git: Free for open source apps, costs money for private/proprietary apps.
Conceptually, you would be building widgets, a widget factory, and a factory making factory.
So, you would have to find all the different types of interactions that could be possible in making a widget, between widgets, within a factory, and between multiple widget making factories to get an idea.
Something to keep on top of how far would be too far to abstract?
**I think you would need to be able to abstract a few layers completely for the application space itself. Then you'd have to build some management tool for it all. **
- Presentation, Workflow and the Data tier.
Presentation: You are either receiving feedback, or putting in input. Usually as a result of clicking, or entering something. A simple example is making dynamic web forms in a database. What would you have to store in a database about where it comes/goes from? This would probably make up the presentation layer. This would probably be the best exercise to start with to get a feel for what you may need to go with.
Workflow: it would be wise to build a simple workflow engine. I built one modeled on Windows Workflow that I had up and running in 2 days. It could set the initial event that should be run, etc. From a designer perspective, I would imagine a visio type program to link these events. The events in the workflow would then drive the presentation tier.
Data: You would have to store the data about the application as much as the data in the application. So, form, event, data structures could possibly be done by storing xml docs depending on whether you need to work with any of the data in the forms or not. The data of the application could also be stored in empty xml templates that you fill in, or in actual tables. At that point you'd have to create a table creation routine that would maintain a table for an app to the spec. Google has something like this with their google DB online.
Hope that helps. Share what you end up coming up with.
Why use PHP?
Appjet does something really similar using 100% Javascript on the client and server side with rhino.
This makes it easier for programmers to use your service, and easier for you to deploy. In fact even their data storage technique uses Javascript (simple native objects), which is a really powerful idea.

Categories

Resources