Preamble
So, this question has already been answered, but as it was my first question for this project, I'm going to continue to reference it in other questions I ask for this project.
For anyone who came from another question, here is the basic idea: Create a web app that can make it much easier to create other web applications or websites. To do this, you would basically create a modular site with "widgets" and then combine them into the final display pages. Each widget would likely have its own set of functions combined in a Class if you use Prototype or .prototype.fn otherwise.
Currently
I am working on getting the basics down: editing CSS, creating user JavaScript functions and dynamically finding their names/inputs, and other critical technical aspects of the project. Soon I will create a rough timeline of the features I wish to create. Soon after I do this, I intent to create a Blog of sorts to keep everyone informed of the project's status.
Original Question
Hello all, I am currently trying to formalize an idea I have for a personal project (which may turn into a professional one later on). The concept is a reflective web application. In other words, a web application that can build other web applications and is actively used to build and improve itself. Think of it as sort of a webapp IDE for creating webapps.
So before I start explaining it further, my question to all of you is this: What do you think would be some of the hardest challenges along the way and where would be the best place to start?
Now let me try to explain some of the aspects of this concept briefly here. I want this application to be as close to a WYSIWYG as possible, in that you have a display area which shows all or part of the website as it would appear. You should be free to browse it to get to the areas you want to work on and use a JavaScript debugger/console to ask "what would happen if...?" questions.
I intend for the webapps to be built up via components. In other words, the result would be a very modular webapp so that you can tweak things on a small or large scale with a fair amount of ease (generally it should be better than hand coding everything in <insert editor of choice>).
Once the website/webapp is done, this webapp should be able to produce all the code necessary to install and run the created website/webapp (so CSS, JavaScript, PHP, and PHP installer for the database).
Here are the few major challenges I've come up with so far:
Changing CSS on the fly
Implementing reflection in JavaScript
Accurate and brief DOM tree viewer
Allowing users to choose JavaScript libraries (i.e. Prototype, jQuery, Dojo, extJS, etc.)
Any other comments and suggestions are also welcome.
Edit 1: I really like the idea of AppJet and I will check it out in detail when I get the time this weekend. However, my only concern is that this is supposed to create code that can go onto others webservers, so while AppJet might be a great way for me to develop this app more rapidly, I still think I will have to generate PHP code for my users to put on their servers.
Also, when I feel this is ready for beta testers, I will certainly release it for free for everyone on this site. But I was thinking that out of beta I should follow a scheme similar to that of git: Free for open source apps, costs money for private/proprietary apps.
Conceptually, you would be building widgets, a widget factory, and a factory making factory.
So, you would have to find all the different types of interactions that could be possible in making a widget, between widgets, within a factory, and between multiple widget making factories to get an idea.
Something to keep on top of how far would be too far to abstract?
**I think you would need to be able to abstract a few layers completely for the application space itself. Then you'd have to build some management tool for it all. **
- Presentation, Workflow and the Data tier.
Presentation: You are either receiving feedback, or putting in input. Usually as a result of clicking, or entering something. A simple example is making dynamic web forms in a database. What would you have to store in a database about where it comes/goes from? This would probably make up the presentation layer. This would probably be the best exercise to start with to get a feel for what you may need to go with.
Workflow: it would be wise to build a simple workflow engine. I built one modeled on Windows Workflow that I had up and running in 2 days. It could set the initial event that should be run, etc. From a designer perspective, I would imagine a visio type program to link these events. The events in the workflow would then drive the presentation tier.
Data: You would have to store the data about the application as much as the data in the application. So, form, event, data structures could possibly be done by storing xml docs depending on whether you need to work with any of the data in the forms or not. The data of the application could also be stored in empty xml templates that you fill in, or in actual tables. At that point you'd have to create a table creation routine that would maintain a table for an app to the spec. Google has something like this with their google DB online.
Hope that helps. Share what you end up coming up with.
Why use PHP?
Appjet does something really similar using 100% Javascript on the client and server side with rhino.
This makes it easier for programmers to use your service, and easier for you to deploy. In fact even their data storage technique uses Javascript (simple native objects), which is a really powerful idea.
Related
I am building a massive real-time collaboration web application. It is a Web IDE that has support for HTML, CSS & JS programming and a stage area that would reflect the results a la JSFiddle, Plunker etc.
Now, the twist is that, it would support multi-user real-time collaboration, where people viewing the same instance of the web application would be able to write code together that would reflect across all the open instances. I have figured out the race conditions, session management et al
What I am having trouble with is
how to reflect the typing and/or deleting along with caret positioning
across these multiple instances giving the illusion that when one
person is typing, he is actually typing on all the instances.
The other thing is that RactiveJS says that it
updates only those parts of the page that are out of date. Tedious DOM
manipulation is a thing of the past.
Which is a very nice effect in their Demo. Imagine, you are on JSFiddle and you don't have to re-run every time. So, my question is, how do they do it? What is the concept behind it?
I don't want to use any library for this. I am pretty good in JS. I am having a hard time figuring out the algorithm.
Things I have considered:
Dirty Checking [but, its dirty right?]
Delta Differencing [but like ReactJS, it would have to update the entire application state every time]
Object.observe [the browser compatibility is not realistic yet]
So, if you have anything that can help me go in the right direction, I would be really thankful.
Realtime-collaboration tools, that allow concurrent editing/manipulation of objects/texts etc, usually use a variant of the Operational Transformation algorithm.
OT is not trivial to understand, much rather implement so I'd suggest you take a look at already-cooked libs for this such as:
ShareJS
Racer for Node
OT works, in some very basic way, similar to GIT
As an update to what you posted in the comments.
You say you are using Python. You wouldn't rebuild your whole codebase, I guess, but keep in mind that real-time collaboration tools usually benefit a lot from using an event-driven server-side language.
Since you are using Python, you could check out the Twisted Framework
I would like to write a 3D application using Golang, my favorite programming language. However, I would prefer not to use native OpenGL directly as the graphical frontend as this would entail a larger learning curve than I'm willing to tolerate. Additionally, I don't really want to use a Golang game engine like go:ngine.
After doing some research I found out about the amazing WebGL Javascript library three.js and I was so intrigued that I completed an introductory Javascript course in a few hours. I'm not really a web developer at all, so I'm wondering how practical it would be for me to write my application primarily in Go but with a three.js frontend.
Some specific questions:
Is it worthwhile / practical to use Javascript/three.js as a frontend to a Golang server like this?
If it is worthwhile, where can I look to learn how to integrate the two cleanly?
If it isn't really worthwhile, which alternatives do I have?
Thanks, any response appreciated.
Edit:
Do you plan to make operations on geometry in Go, possibly in realtime and communicate with your Javascript layer using Websockets?
Yes, this is what I'm thinking now. I'm intending my (Golang) program to generate streams of vertices and triangles based on a few parameters from the user. Each "structure" is generated all at once after the users supply their parameters (which I see as slider bars + input boxes on a Javascript frontend).
Here's an example of something the Go program might generate, plotted in GNUPlot:
(source: alexozer.com)
If this is the case you underestimate Javascript a bit. You should write most of your application in Javascript and use Go as a classical backend service layer like user accounts, persisting state, etc.
I'd be fine with doing that, except for these reasons:
The core generating process fundamentally depends on Goroutines and channels
I know close to nothing about web development, including frontend and backend
So I suppose I'm asking about the implementation details of one of these three possibilities:
Writing most of the program, including the generator, in JS, and a minimal backend in Go
Writing most of the program in Go and using JS just as a graphical frontend
Something else?
I recommend leaving out the whole Websocket aspect because it's only a possible optimization (and maybe not even that).
If you need to depend on Go's concurrency model then go for it, write your component in Go, then a Webserver in Go that takes parameters coming from an HTML request, uses them to compute the result and sends back the data in a JSON format.
On the frontend you will then only focus on sending this request when the user changed a parameter and for displaying of the JSON data you can use ThreeJS right away.
You'll still have to learn a bit of web development and Javascript though. But hey, they say the web is the future ;-)
I think it's the way to go for your case because your application operates in a quite strict request-result way.
The answer to your question depends a lot on where exactly you plan to draw the line between your Go component and the Javascript component. From the way you describe the problem I get the feeling that you want to write most of your application in Go and use JS only as a display layer? I wouldn't recommend doing that.
To rephrase my question: Do you plan to make operations on geometry in Go, possibly in realtime and communicate with your Javascript layer using Websockets? If this is the case you underestimate Javascript a bit. You should write most of your application in Javascript and use Go as a classical backend service layer like user accounts, persisting state, etc.
I might be able to give a more specific answer if I know what exactly you plan to do.
I've been asked at work to convert a popular mechanical engineering excel application to a single page web application. It requires no database so my idea is to put everything on the clientside in javascript.
First version is gonna be pretty small but more complexity will be added further along. Canvas graphics is gonna be drawn in realtime as user changes input. So performance is important. My question is: Should I use a MVC framework such as Angular JS? Or is it better to stick with Jquery? If I should use a MVC, which one is best suited for this app?
Everyone at my company uses IE9 so I dont need any support for older browsers.
Angular bills itself specifically as a way to build CRUD apps - basically database front ends. The app you're describing doesn't sound like it's doing a ton of that, so Angular would probably be more trouble than it's worth.
jQuery is a helper library - it makes dealing with the DOM easier, but doesn't do a thing to provide overall structure to your application. jQuery is a "necessary but not sufficient" in my book - you're going to want it, but it's not enough on its own.
Personally, I'd lean more towards Backbone, specifically because it's so minimal - it gives you some useful data binding stuff and some hints on how to organize your app, then gets out of your way to let you build the abstraction you want. Based on the fact that you mentioned the original is in Excel, some sort of data binding / change notification system is going to be important. This way you can emulate the spreadsheet's "change a cell and the chart redraws" system that everyone's already used to.
In the end it's hard to say without more detail on the app in question, and what kind of calculations / how often they have to happen are. Figure out what your app architecture should be, find (or build) a library that helps implement that architecture. It might be MVCish, it might not.
Well jQuery will get very messy as complexity increases. Angular JS is an amazing library but there are also a myriad of different ones out there. Depending on the Mechanical Engineering principles you are working with using AJAX may prove to be more perform-ant, due to the lack luster power of some older browsers.
It really depends on the complexity of the calculations, but in terms of dealing with events and managin the structure I would suggest using Jquery as well as Angular on top of it. then make the decision about whether or not to move some functions to the server or not, based on performance.
I'm writing a UI to my R script, which asks the user some names of organisms and the location of a folder, using javascript/html that will be local (not hosted, ever).
At the moment, I have just that: a couple of text boxes that take input and pass an executable R script. Originally this UI was being written as a very user friendly option, but slowly I've realized that some nifty tricks can be added such as a textbox that completes the word for the user (so if the user misspells the name of the organism, the UI will correct the input based on the files uploaded. And this would come from a list of organisms text file that R would generate immediately once the files have been added).
Is there a way to make this more efficient? For example, retrieving plots from R (as .pngs) and updating my local webpage and being able to share a log file between R and the UI (mind you, I am aware of the potential File I/O errors)..but for the sake of brainstorming.
I'm aware of Shiny, but what I would like is a simple local UI, as I will be dealing with big data (average ~ 1 gigabyte worth of files that my script will process).
Another way to ask my question that is more to the point:
Here's an example of integrating PHP and R: http://www.r-bloggers.com/integrating-php-and-r/
I am looking to create something similar with javascript/css/html/jquery etc.
Thanks
You could definitely use nodejs (nodejs.org) for that. Take a look at https://github.com/elijah/r-node and r-node. Confusingly enough, this is two different projects with the same name. More info on the latter here: squirelove.net/r-node/doku.php
In recent years JavaScript has become one of the fastest programming languages. In one case I know of, JavaScript is faster than C++. See: benchmarksgame.alioth.debian.org/u32/performance.php?test=regexdna
Bear in mind, though, that memory is very difficult to manage in JavaScript, so you should run some sort of memory leak detection program on your code, if you plan to create long running processes.
E.I: memwatch (npmjs.org/package/memwatch) or nodeheap (npmjs.org/package/memwatch)
Good luck with your endeavors!
PS. sorry for the lack of real links. I'm apparently not allowed to post more than 2 links.
Why wouldn't you be able to use Shiny locally? You design your app on your computer and run it locally with runApp('myapp') from an R-prompt. Unless you are experienced with javascript I would give shiny another look: http://www.rstudio.com/shiny/
The example you linked to can be very easily implemented using Shiny. See link below for a tutorial on how to write the app:
http://rstudio.github.com/shiny/tutorial/#hello-shiny
To run that example locally:
install.packages('shiny')
shiny::runExample('01_hello')
I have a similar case, and shiny looked like a good idea to me. However, after I did a few first steps, I am no longer sure about this. Note that most of the examples use shiny to display results. When you get into editing some fields and using a database, things can become messy; the reactive-ness gets in the way once fields can be change by program and by the user.
As an example see https://gist.github.com/dmenne/4721235/edit. The main problem for the current state of shiny is that you must use the dynamic UI for this type of work, which kills any separation of ui and server because you have to create the ui elements in the server.
shiny is a great idea, but for anything larger with interaction it is too early now. Knowing that the amazing RStudio team is behind it, I am sure the stress should be on now.
What else is there around to make user interfaces for R? TclTk makes me shudder. I working in c# a lot, and I had been using R(D)COM for interfacing some years ago, but gave up after installation and licensing problems. There is R.DOTNet which works better now; it is the most hazzle-free installation-wise, but it is not a very active project, and tends to crash. Interfacing via RServe/RServeCLI is stable, but is too difficult to install on Windows, for example on hospital computers with their strict security issues.
And there is Qt. With the active RInside community, it would be a good choice and the interface is great. I wish however my programming skills were at the level of the RStudio-guys. The fact that even Dirk is one the proof-of-concept level (using rinside with qt in windows) is not encouraging.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Background
I need to create a potentially very large HTML/JS mobile web app that will be delivered as a mobile web site and natively using Phonegap. I'm currently working to determine the best way to organize the app itself.
The basic plan is to have many modules that will each focus on a different subject of interest . Some of these modules will be very basic (ie, announcements / news) and some will be very complex (ie, sports: team players, schedules, video, etc). There will be a side-drawer navigation that will apply to most pages so users can quickly navigate to a different module. There needs to be the ability to deep-link within modules. These modules will be created by a variety of developers and vendors.
Single Page App
Most of the mobile solutions I see involve Single Pages, which seem like a bad idea to me in this case, since there is the potential for so much memory use. It also seems like it would be difficult to reconcile hash navigation between modules and hash navigation between section within modules. Module development would have to be done with the app framework in mind and limits how things can be done by vendors and developers. On the other hand, things aren't getting loaded as often and everything can easily communicate with each other.
Multiple Page App
Using multiple pages, it seems like each module could easily be created in whatever technology a vendor was comfortable with (and could do quickly and cheaply). It would cut down on memory use, but also remove the ability for modules to communicate (a feature that I don't know is necessary for us at this point). I could see making a javascript library every module would use for common handling of various events (like logging errors, navigation, etc). Each app navigation between modules would be a new page call, resetting the DOM. Each module could use a single page design if it wished.
Help Me Please
So, is there any common or new knowledge about how things like this should be designed? I'm eager to begin work, but don't want to be rewriting things that may already exist. Do I have any glaring flaws in my reasoning? I'd love to hear from anyone that has insight.
Honestly, if you are considering building any app that you believe will be high volume and of high complexity, you really ought to consider doing native development, or at least use something like Appcelerator where the application will be "compiled" down to native code for better performance. If you are intending to just let any number of developers build their own javascript components that may or may not do a good job of managing limited system resources, so are going to quickly run into application performance issues.
On the other hand, if you are just going for proof of concept and don't mind potentially having to greatly refactor your application architecture when and if you get a sufficient level of complexity, then you may want to simply go with the web app approach.
Really, you need to also be considering your backend service architecture as much or more than the frontend architecture. that is really where you are going to run into problems down the line in trying to integrate offerings from other developers.
I had a similar architectural problem to contend with a couple of years ago. It wasn't mobile, but it wasn't entirely web-based either. The target applications were a mix of web sites and desktop apps, with the potential to go mobile in the future.
The interesting part was that there had been two prior attempts at creating a framework that could be used in a variety of situations. The problem, and the reason both attempts failed, was that the developers saw it as a UX problem space. They approached it using several different technologies, but became mired months down the road because they had made assumptions up front and flown the project by the seat of their pants.
My approach was to eschew all the UI discussion completely and focus on a backend architecture that could be approached from any standpoint. To this end, I created a web service that had data going in both directions, and was ultimately serving a mathematical model. The service is being accessed from a variety of sources using different technologies: Flash, Unity, a Google Earth plugin, and finally, from an unrelated web architecture serving up good ol' HTML.
My advice to you, is don't concentrate on the front-end mapping so much as get your back-end right. Once you have a data structure in place, you can build outward, and several issues such as memory management, monolithic app or not, i.e. one page versus many, will almost resolve themselves. Work on creating a great API with lots of good interfaces and you won't fall into a "many chefs" hole. Give a bunch of dispersed developers enough rope, on the other hand, and you'll never find where all the knots are!
The decision whether to ultimately go native API over HTML5-based technology such as Sencha Touch, jQuery Mobile, or Phonegap is an evangelical black hole that will be played out over the coming months and years. Native apps may be more fluid and speedy in some cases, but the investment in resources is something that should be considered. On the other hand, JavaScript developers are lurking around every corner and are not in short supply.
Your first step is to nail down those requirements.
If you're doing this for yourself or for your own company, then nail down how these modules (co-)operate.
If you are doing this for your employer, then somebody there ought to have a clue what they want to see, otherwise, how are you going to build it?
A solution which supports multiple pages, opening and closing modules with no communication is going to require different things than a framework which is responsible for maintaining multiple widgets at the same time, which may or may not communicate through system-calls or services.
There's no way around that -- building services/sandboxing/etc for modules is going to take more work than treating each like a page-change (or making them be literal page-changes).
When you figure out what you want your program to do, start building out an idea of the API you'd like other people to have.
Are you going to provide them with an API for building UI components, or are you going to leave that to their own whims?
Personally, I'd avoid a situation where each module change just replaces iFrames, and then the end-user can do whatever they want in there.
Likewise, I'd avoid situations where you're allowing module-creators to run whatever they'd like in a non-sandboxed environment... It ends poorly for your end-users (or you, in UK court).
But that's not a concern, yet.
First-concern is what does your platform do.
Then figure out what your back-end communication is going to look like, and what interfaces you're going to provide to module creators, and how you're going to get data from your end to theirs (http-based API, REST or whatever else... ...but work it out WELL, if you don't already have it).
THEN, when you know what your platform is expected to do, AND you have a backend which can serve all kinds of tasks well, figure out what services you're going to provide to content-creators, to make their widgets, and to upload/download data from your service, and sandbox, and the like.