I want to implement markdown to my forums.
I research many possible approaches how I would do that and this is what I thought:
A simple approach would involve pagedown on client side and php-markdown on the server.
My approach is to save pure markdown to database and when displaying convert to HTML (with pagedown). Since I already have security layer for my server side (HTML elements whitelist) and all the necessary staff I don't see anything to lose here.
What I win in this case? well I have to modify pagedown to use custom buttons and patterns. That would be hard for me to maintain both php and JavaScript.
My question is: is this good aproach?
To break this question:
Is there any serious overhead on client side loading about 30
posts and converting it to HTML (performance)
With the Idea that I check elements whitelist, is there any
security issue I need to know about? (Security)
I wouldn't use client side markdown engines. From a few quick googles its of the opinion it's very CPU intensive. Loading 30 posts would add quite a bit of overhead.
If you stored MD in the DB, rendered to HTML on the fly, then employed some caching (memcached or redis) that could work quite well.
In regards to security theres a good read here, it would require some extra sanitising removing scripts/links/redirects etc.
Further reading
http://functionn.in/resource/remarked-js
How to use Markdown & MySQL?
Converting markdown from client side is not recommended as #Lex has stated. Instead, you can use some online services to convert the markdown top html for you.
Have a look at http://daringfireball.net/projects/markdown/dingus. You can use curl or something to post markdown to the site and then scrap the website to take the html part.
You can also have a look at here http://parsedown.org/
You have two options to suggest:
Strapdown - allows to create markdown documents without server-side processing, as you can see on there page, even without code, just by using static files
markdown-js - allows to create markdown document with client-side processing (javascript)
Here is how I do it:
Save markdown code in db and at rendering I'm caching the result in a file (file, Memcached or any cache storage engine you want). This way I keep the original in database and I`m not wasting resources to compile markdown at each page visit, instead I serve the cache file until it has expired or deleted because of a change.
Related
My PHP template looks like this:
$html=file_get_contents("/path/to/file.html");
$replace=array(
"{title}"=>"Title of my webpage",
"{other}"=>"Other information",
...
);
foreach(replace AS $search=>$replace){
$html=str_replace($search,$replace,$html);
}
echo $html;
I am considering switching to a javascript/ajax template system. The AJAX will fetch the $replace array in JSON format and then I'll use javascript to replace the HTML.
The page would then be a plain .html file and a loading screen would be shown until the ajax was complete.
Is there any real advantages to this or is the transition a waste of time?
A few of the reasons I think this will be beneficial:
Page will still load even if the Mysql or PHP services are down. If the ajax fails I can handle it with an error message.
Bot traffic (and anything else that doesnt run JS) will cause very little load to my server since the ajax will never be sent.
Please let me know what your thoughts are.
My 2cents is it is better to do the logic on the template side (javascript). If you have a high traffic site you can off load some of the processing to each computer calling the site. Maybe less servers.
With Javascript frameworks like AngularJs the template stuff is pretty simple and efficient. And the framework will do caching for you.
Yes, SEO can be an issue with certain sites. There at proxy tools you can put in place that will render the site and return the static html to the bot. Plus I think some bots render javascript these days.
Lastly, I like to template on the front-end because I like the backend to be a generic data provider (RESTful API). This way I can build a generic backend that drives web / mobile and other platforms in a generic way. The UI logic can be its separate thing in javascript.
But it comes down to the design needs of your application. I build lots of Software as a service applications so a single page application works well for me.
I've worked with similar design pattern in other projects. There are several ways to do this and the task would involve managing multiple project or application modules. I am assume you are working with a team of developers and not using either PHP or JavaScript MVC framework.
PHP Template
For many reasons, I'm against using “search and replace” method especially using server-side scripting language to parse HTML documents as a templating kit.
Why?
As you maintain business rules and project becomes larger, you will
find yourself reading through a long list of regular expressions,
parse HTML into DOM, and/or complicated algorithms for searching
nodes to replace with correct text(s).
If you had a placeholder, such as {title}, that would help the
script to have fewer search and replace expressions but the design
pattern could lead to messy sharing with multiple developers.
It is ok to parse one or two HTML files to manage the output but not
the entire template. The network response could be slower with
multiple and repetitive trips to server and that's just only for
template. There could be other scripts that is also making trips to
the server for different reason unrelated to template.
AJAX/JavaScript
Initially, AJAX with JavaScript might sound like a neat idea but I'm still not convinced.
Why?
You can't assume web browser is JavaScript-enabled in every mobile
or desktop. You might need to structure the HTML template in few
ways to manage the output for non-JavaScript browsers. You might
need to include <noscript> and/or <iframe> tags on every page. And,
managing alternative template for non-JavaScript browser can be
tedious.
Every web browser interpret JavaScript differently. Most developers
should know few differences between IE, FireFox, Chrome, Safari, and
to name few. You might need to create multiple JavaScript files to
detect then load JavaScript for that specific web browser. You
update one feature, you have to update script for all web browsers.
JavaScript is visible in page source. I wouldn't want to display
confidential JavaScript functions that might include credentials,
leak sensitive data about web services, and/or SQL queries. The idea
is to secure your page as much as possible.
I'm not saying both are impossible. You could still do either PHP or JavaScript for templating.
However, my “ideal” web structure should consist of a reliable MVC like Zend, Spring, or Magnolia. Those MVC framework include many useful features such as web services, data mapping, and templating kits. Granted, it's difficult for beginners with configuration requirements to integrate MVC into your project. But in the end, you could delegate tasks in configurations, MVC concepts, custom SQL queries, and test cases to developers. That's my two cents.
I think the most important aspects you forgot are:
SEO : What about search engine bots ? They wont be able to index your content if it is set by javascript only.
Execution and Network Latency : When your service is working, the browser will wait until the page is loaded (let's say 800ms) before making the extra Ajax calls to get your values. This might add an extra 500ms to get it (depending on network speed and geographic location...). If you have sent all the generated data by your server, you would have spent only ~1ms more to prepare the complete response. You would have a lot of waiting on a blank page.
Caching : You could cache the generated pages on your web app. That way your load will be minimized as well. And also, if you still want to deliver content while your backend services (MySQL/PHP..) are down you could even use Apache or Nginx caching.
But I guess it really depends on what you want to do.
For fast and simple pages, which seems to be your case, stick with backend enhancements.
For a dynamic/interactive app which can afford loading times, and doesn't care about SEO, you can delegate most things to the front-end. But then use an advanced framework like Angular, to handle templating, caching, etc...
I'm planning to write a web app using either Backbone or Angular. We want to push "widgets" from the server to the client (i.e. semi-complicated, dynamic but largely autonomous UI elements... something maybe like the popular TodoMVC app). So we'll need to send over a template, some javascript (controllers etc.), probably CSS, and JSON data (models).
We're debating how to send over all the information. How much can be, and what should be encapsulated in the JSON?
Is it possible to create files out of data passed over? i.e. could we pull out CSS and apply the rules to our document? I believe it would be easier to run javascript passed over this way.
I'm under the impression that being able to cache the template is important... does that require loading it as a (separate) resource rather than as part of some huge JSON object?
As for CSS, it needs to be loaded before inserting into the DOM (so we don't want just a promise). Would it make sense - or even be possible - to pass our CSS rules in the JSON and extract them somehow?
EDIT: to more fully describe what I'm working with, I'm focusing only on the front-end. The back-end can be customized to send over resources however we want - they'll optimize that depending on what the front-end needs. Our backend stack includes MongoDB, Tapestry, ActiveMQ.
The payload that needs to get sent over will be all the resources necessary to push something like a Mac Dashboard Widget or Windows Gadget into the browser. So HTML, CSS, Javascript, data will all get sent over. We want things to be snappy and minimize server requests as much as possible, as some of these payloads may get to be somewhat large.
Your questions are rather vague and the answers can change by coding on a different build platform. Your chosen application design can also impact the best practices to follow for implementing X type of application to X device(s). There's more than one way to skin a cat ya know.
What information are you sending over? Is it a lot of raw data arrays? Is it more document based where XML could be more beneficial? You'll need to work with your DBA if the data is stored in a repository and you'll be querying it. Will you need to write code to format your data or will the DBMS output it in the format you need?
What do you mean the CSS needs to be loaded before being inserted? Everything loads in the order you specify so making sure your CSS loads at the proper time shouldn't be an issue. If you're using ASP.NET, you can also use a SSM (style-sheet manager) to serve your CSS and only load whats required. SSM's are great if you have a lot of style-sheets to serve.
A lot of the questions you asked will be answered as you get further into the design phase. In order to get some solid answers and not just speculation on what is the best method, you will need to publish a lot more details than this. Any answers at this point shouldn't carry much weight in your decisions. They certainly wouldn't mine.
I want to add Excel button below the html table, that gives an Excel sheet of the table. I am using C# behind it, but I want it do on client side just using jQuery or Javascript without using PHP or .NET... Not more.
I am searching for hours however I don't find any fix solution. There it said you can use activeX object and CSV however I don't see any running examples.
ActiveX will only run on Internet Explorer so your solution will be limited to that platform.
Without ActiveX or server side scripts there is no way to achieve this due to JavaScript security designs it is not allowed to create files or push in memory files to download streams.
I would not recommend ActiveX or anything that tries to run code outside of the browser on the client system. ActiveX only works on IE not to mention the inherent security risks that brings about.
A possible method would be to use a server side script along with jquery to generate the file and return it without ever leaving the page your on. It looks good and you don't really have to jump through hoops to make it work nor will the user. (if you want an example of the jquery required to do this let me know).
For c# there is alot of tutorials on generating excel files from your data for example here.
The only reason I can think of that you would want to create the file locally is use of resources on your server. C# will be extremly efficient in generating that file not to mention there are a couple things you can do (dependant on the data) to negate any impact it has:
If the data changes are few and far between consider storing the
generated file and directing to that the next time until the data has
been changed.
Limit the data set it returns by way of a search criteria
There are plenty more optimization you could do but you get the idea.
My knowledge on web technologies (JS, JQ) are limited and I want to start learning them. As a starting point I want to do some file processing. Because it is something I have to do for my work and was planning to do it in Java. What I basically need to do is to go through a list of text files (assembly files) in a folder and search for routines and then list them. This is the first step and is a trivial task in Java.
But I wanted to take this a step further and do it in the browser, so that others in my team also can use it without installing anything (and also to impress them a little bit in the process. since I'm the new guy in the team :-)).
So when I input the folder, the script will go through the files and search and will display results in a web page. Basically first page will be a list of files in the folder, and clicking a file name will take me to another page which displays the routines in that file.
Sorry to bother you with details, but what I actually want to know are:
Is this possible with JS? (to
search for text patterns in a file)
Should I start with JS or JQ? (I
think many would recommend starting
with JS, but since this is a side
project and this is done purely in
my own time, would you suggest start
learning JQ because it's relatively
simpler to learn (from what I have
read) for a beginner?
Or should I just do the processing
in JAva and then interface the
results to a webpage
Any advice is appreciated.
Thank you very much.
Java and JavaScript have nothing to do with each other, jQuery is library written to simplify usage of JavaScript with some handy shortcuts.
I'm afraid JavaScript would not be able to parse text files as its main usage is manipulating content inside browser window and limited by different security policies.
To parse files you have to chose server side language.
maybe you can use java to deal with the file processing, and then send the result to js script , which will show these results to users.
js's ability is limited
For security reasons, JavaScript is sandboxed within the browser, and has basically no access to the local file system. From what you have described, it sounds like your best option is to use Java to process ...whatever...
This function has nothing to do with web browsing. Why is a browser the best tool for the job, anyway?
This question pertains to web applications. I have very little web app development experience, so might be missing some very obvious points/issues. Please point them out.
As I understand, in most web applications, a web server sends HTML over the wire to a client (browser). This happens every time a HTTP request is made. I feel this is very wasteful of bandwidth.
1) Since browsers can run JavaScript, why don't we just send a JavaScript program which can generate the webpage's HTML content (which the browser then renders).
2) Further a browser might cache the JavaScript program and next time the server only need send the data. The protocol might involve the browser sending the "program version" it has.
Consider an example of a relatively simple website Hacker News [http://news.ycombinator.com]. Let us separate the data (30 posts + their metadata) from its presentation. Assuming 1) above, the server can just send the data (say in JSON) + a JavaScript program to generate HTML. This gist shows the idea. The data for the 30 posts is in JSON [http://www.json.org/js.html] format. For this particular example the data transferred is cut in 1/2 (size of data+JavaScript / size of HTML). Further if browsers can do 2) above, it reduces the data transferred on each visit to 1/4 (size of data / size of HTML). [Note: this analysis is without considering compression; gzip,deflate is very successful in reducing the size of HTML. But isn't prevention better than cure?]
I see atleast the following advantages of this :-
* For most web pages, it will reduce the size of data transferred over the wire.
* Forces web apps to separate data from its presentation.
Disadvantages might include - more complex browsers, time to run the JavaScript program to generate HTML (this might get offset by the reduction in data size).
Now my question is - why are web applications not developed this way, or, why do web applications send HTML over the wire? Surely the web server (sending out HTML) doesn't care about HTML at all, so why should it, first, generate it, and then send it over the wire?
There are a few reasons, some of them historical this is by no means a complete list but just some of my experiences:
HTML predates JS, and a lot of scripts and libraries predate JS
Older browsers (think IE<=6) had rubbish, inconsistent JS engines, their rendering engines were much more consistent in how they treat HTML. So many more libraries and scripts predate consistent JS
It is a nightmare to debug applications written as you suggest if they are not constructed right (we have one at my work, it takes 30 minutes to find where a piece of html is actually generated)
It is a lot more work to do it right - why not use templates or static docs or something much simpler
Its not really a problem - HTML compresses really well
What you suggest is done - its called AJAX (OK, so ajax is more general than this but you all know what i mean)
It simply doesn't work for most plain-text user agents including those used by most search engines. If this page is serving most of your content, its generally a good idea to make it easy for Google to parse
Well the obvious reason on why this is the case is that JavaScript wasn't around when we started sending HTML around, and HTML was an improvement to sending around plaintext documents.
The reason we don't do this now: we eschew complex solutions to problems that aren't really problems.
Average internet connections download nearly 1M bytes per second, and web browsers are quite adept at parsing and starting to render this HTML before it's even all ready to be. They're also great at parallelizing the downloading of resources on the page. If we want to save a few bytes at the cost of some compute cycles, we gzip content before sending it. Problem solved.
And for the record, we do this with AJAX in complex webpages (checkout Github's source browsing for a great example of how awesome this can be).
What you suggest can, and is, done. Remember, web pages used to be static documents. Full blown web-based applications are a relatively recent idea.
I might also suggest that it isn't necessarily more efficient, especially when your pages are sent gzipped.
What you suggest is basically what a JavaScript full stack framework like ExtJS does. You can create rich, data intensive applications without writing any HTML -- well, only enough to reference the necessary .js libraries. The complex DOM needed for layouts, grids, forms etc is all created by the framework.
The simple answer is that HTML is older. Why is C99 not fully implemented with a lot of compilers? They figure 1989 is new enough for them. Also, JavaScript exercises a lot more control over people's browsers than they seem to want. Conditional statements and encoded data pose a security concern, and some people want to keep that can of worms closed to begin with. True, HTML is a very inefficient markup, but the size is insignificant compared to the images you download from the internet. That favicon takes up as much data as the page itself, and it's only 16 pixels across.
A good reason that the server-side code of a web application might do lots of HTML template work on the server side is that in many server environments it's not made easy to bundle up server-side data structures (object graphs) for easy delivery to the client. There may be information kept in server-side data structures that really shouldn't be delivered out to the client. Thus in order to send out a "pure" data-only response, the server would have to trim off sensitive data before delivering out the JSON. That's not an unsolvable problem, but I don't know of many server frameworks that facilitate a solution.
The server has direct, unfettered access to the database and to everything else that makes an application work: user preferences, history, account details, system settings, etc. To build an application that's client-centric for rendering purposes would mean concocting ways of keeping all that information intact and up-to-date on the client. For a lot of applications, that might not be terribly easy.
Finally, it's only relatively recently that it would make sense to trust a browser to provide a stable enough platform for building a long-lived "application environment" as a continually-updating web page. By building a web app such that pages are sometimes completely reloaded, there are lots of little "reboots". That's a cheap and dumb way of keeping a lid on at least some kinds of memory leaks.
Most implementations of sites with heavy Javascript use won't start executing until the DOM has fully loaded; then you'll get every page with 'loading screens' when the page wrapper has downloaded, but none of the content has.
Also, do remember that not all users have Javascript enabled, and not all browsers support high-level Javascript (think mobiles).
I would send HTML in a response if I wanted my application to work without Javascript. I would write HTML rendering code in my server-side language (most of the time not Javascript), which could then be used for two purposes: serving whole HTML pages, and serving bits of HTML in response to XHRs.
If the Javascript code is restricted to things like reporting UI events and replacing innerHTML with server-generated code, I don't have to duplicate any of my application logic across languages/frameworks. This duplication problem is one of the reasons why server-side Javascript is getting people excited.