Develop Reference

Migrating Javascript from the database

I am working on an old enterprise solution with these properties:
The solution has a MVC web application
The solution has a WCF service layer
The solution has javascript in the database, in the form of functions in a database column
The web application retrieves said javascript through the service layer and plugs it into certain pages
My team cannot modify the web application, nor the service layer
My team must write javascript by inserting functions into said database columns
This architecture leads to:
A very inefficient development loop
Very poor source control
I'd like to propose a solution for them, how to upgrade this, but here's where I fall a bit short on experience. My suggestion would be:
Migrate the javascript from the database to javascript files
Make some sort of hook in the web application for other teams' javascript files
My questions are:
Has anyone had this kind of problem and how did they solve it?
Is there an effective way to do this kind of javascript migration into files? My idea would be to write a small console program to do the migration
How would they make a hook to import our javascript files? My idea is to make a script bundle with some naming convention, so we can add scripts without them needing to change their code. Are there problems with this approach?
Any kind of input would be invaluable.
Edit:
Additional explanation:
The mechanism maps the javascript function names to a certain DOM elements' event attributes and inlines the code right after the element
The functions are standalone functions, depending only on libraries already in the web application
The functions are grouped by a common form
So I suppose it would be better to group them into files bearing the form names.

If these are just simple, static function definitions being inlined into the web page, then I suppose it might be possible to serialize/aggregate them all into a giant file and run something like prettier on it to make it readable.
That wouldn't be ideal to gain traction in your proposed migration, though. If the code has any volume to it at all, it would be nice to give some structure and order to maintain it.
It's already kind of a huge assumption that this javascript is just pure functions without any complex dependencies on each other, but it's possible that these pieces of Javascript work in isolation already if they are being pulled out of a database. It's hard to know without knowing more context. It seems unlikely that your life will be that easy.
If you managed to extract this monolithic Javascript file, the easiest thing to do would be include it in a script tag for the entire site and be done with it. This could be a bad idea if the file is getting to the ~MB size and slows your initial page load time.
Then again, the point at which you have a bunch of functions in one file, you could probably do a lot there to optimize and reduce duplication of code.
This is still all conjecture because I don't know the mechanism by which your web application imports the javascript once it retrieves it from the database.

Backbone View Templates - Storing Them As External Files

I've been tinkering around with Backbone.js and wanted to know if there is possible to load View templates from external files. I've been using Underscore.js's micro template solution and including all my templates inside my HTML document has been bothering me.
I wanted to know if there is a standard way of doing this. Is there any good practice I should follow?
Thank you all for you time and help :-)

For all the template libraries that compile strings to javascript, with a little code in the header, each template could be stored in a separate file and exported through a common windows.myTemplates object or similar. The code would just evaluate the given string (the actual template), and bind the result to the windows object to make it available to other scripts. This way each template would be it's own script and would be accesses through a global object like windows.myTemplates.someTemplate.
That does not mean this is a good idea; unless you have many and/or big templates that are loaded very seldom, the normal goal is to minimize the roundtrips to the server, which is the exact opposite of what you are doing. Server-side, things are a bit different, but I'm assuming you're talking client side (based on your question).
There are fancier ways of pulling things in only when needed (google "javascript AMD", require.js and similar), but those are a bit more challenging to get started with.

Page specific javascript files in Rails 3.1

I've been doing research on this:
Using Rails 3.1, where do you put your "page specific" javascript code?
But I have yet to see a satisfactory answer, which also makes me question whether I'm doing something wrong.
Here's my mental model: for different views, I'm going to have different
$(document).ready(....)
blocks, that obviously reference elements that are very specific to that page. I don't want to pollute things by loading that code for every single page and somehow trying to figure out how to only execute it when on specific pages; that seems pretty ugly.
My intuition, admittedly not backed up by any preliminary experiments, is that the ideal thing would be to:
Load application wide code from application.js.
Load shared controller code from something like assets/controller_name/shared.js
Load view-specific code from something like assets/controller_name/show.js
Off the top of my head. The helper would, the first time it ran, check if the file exists and, if so, do a javascript_include for it.
Perhaps this has some performance issues compared to the "let's just wrap the whole thing up in a big sticky ball and send it all" approach, but seems like a better approach to compartmentalizing code.
However, as above, I get the feeling I'm missing something. Is $(document).ready on a per-page basis a bad idea? Should that just be in the template and call a page specific bit of JS from application.js? The linked article above comes to that conclusion, but I don't like the image I'm getting in my head of one huge $(document).ready riddled with if this, if that, if the other thing.

What you propose is sound, but not the rails 3.1 way.
They say to divide the JS into many files, but serve as a whole single piece to the user. This allows better performance and scalability, so is a good thing if the final big piece of mud is not so big. Really 3 http requests give worse performance than 1 http request.
So you have already dipartimentized your code, because you have different Coffeescript files, which have different scopes.
To load in your app, just standardize a way to initialize the single piece of code, like calling a "myapp.users.init()" method-.
You could even automatize that peace of code using an helper, so it will be transparent for the controller.

Background
(Why Use The Asset Pipeline At All?)
One of the basic premises behind the Rails asset pipeline is the idea that it is preferable to load all the JS and CSS for a site up front once, and then cache them indefinitely (at least until the site is updated). The Asset Pipeline allows you to do this relatively automatically while still organizing your JS and CSS src files in a logical fashion.
This of course carries an upfront load cost, at the promise of saving time on additional roundtrips loading individual files. If that premise doesn't sit well then the asset pipeline is probably not for you.
The Nut Of The Problem
Ok, so we want to combine all our JS into one file to load it more efficiently. Just because we are going to load all of our JS doesn't mean we want to run all of our JS.
In the reality of a complex webapp you will probably have lots of page specific functionality that you won't want to spend resources executing when the user is not viewing the corresponding page. What we need is a uniform strategy for executing only the portion of our big monolithic JS file that is applicable to the current page.
Convention To The Rescue
I'm not aware of an official Rails strategy to deal with this, but there are some great solutions that establish and then leverage a good convention (which makes things feel "railsy"). The general idea is to define all your page specific JS code into an object literal, and then run only the code relevant to the current page on load.
For the specifics great strategy on how to organize and conditionally execute your JS code, see the answer by #welldan97 on this question:
Using Rails 3.1, where do you put your "page specific" javascript code?
which in turn is based on this article by Jason Garber:
http://viget.com/inspire/extending-paul-irishs-comprehensive-dom-ready-execution

Javascript isn't good for SEO, is it?

If I decided to use some javascipt in my website like
$('#body').load(URL);
or
$.get(URL, {param:value}, function(){ ... });
or
window.title = 'TEXT';
Is it good for SEO? Or am I recommended to use pure PHP for data on the page for SEO purposes?

The question of if javascript is good for SEO or not is missing the point. We should pretty much assume that any content which is only available by javascript will not be crawled by the search engines. Google at least claims to be able to crawl some javascript only content but is fairly tight lipped about what exactly they can crawl. Other search engines probably don't crawl it and it's certainly the case that not all do. So assume it doesn't get crawled.
That doesn't mean it's bad for SEO.
If the content will contribute to your SEO, then it's bad for SEO. If the content is neutral to SEO, then it's neutral for SEO. So the answer to your question really depends on the nature of your content. If the content is part of your SEO campaign, then stick with server-side HTML generation be it PHP or some other method. Otherwise the question of SEO has no bearing on the decision to to use javascript or not. Accessibility would be another thing to take into account. Javascript only content is terrible for that.

The larger search engines can/do render limited amounts of javascript. However, for SEO purposes your best bet is rendering the content via HTML rather than javascript. A good rule of thumb is to utilize HTML for content/expressing limited content structure (e.g. paragraph type text = p, lists = ul/ol, headings = h1/h2/h3, etc...), CSS for presentation, and JS for client side programming. With that being said, always ensure a good user experience first. If you can do the above while providing a great user experience, great! If you can't, users first. Its likely you can keep both users and bots happy 95% of the time if you take the time to do so.
Further reading (sorry, I can only post one link as a new user):
Matt Cutts Interview (Check out #26 on Google Javascript Rendering)
A spider's view of Web 2.0
EDIT Added that for "a new user" ;) ~ drachenstern

I think first you should consider what SEO means. It means "Search Engine Optimization" ... how does a search engine get data in the first place for it to be optimized?
It does a GET on the page and whatever data is returned in the GET is processed. No JS engine. No POST data. So you should be optimizing for whatever data is returned on a GET.
Additionally, you tagged this with PHP, but the question has nothing to do with PHP.
Have you seen any of the questions on this list?
https://stackoverflow.com/search?q=javascript+seo

No Sir, Google does not translate flash and java script properly so it may not crawl those area using java script or flash content. I suggest you should keep your website simple but if it is necessary to keep flashy/java script content then you should keep a text base backup.

The first thing you should be asking is not what is good for SEO, but what is good for users. For users, loading data with JavaScript will give them an interactive page, where they can start seeing the page immediately while it is still loading and where the page can update without having to reload it.
From Google's Webmaster Guidelines and article on Cloaking, you should not assume that crawlers can understand JavaScript. This does not mean that you should not use JavaScript on your website, but rather that you should provide the textual equivalent in noscript tags, for use both by users with JavaScript disabled
as well as for crawlers, bearing in mind that the content of these noscript tags should be roughly equivalent to what was shown with JavaScript enabled; showing different content to users and to search engines is called "cloaking" and is frowned upon to say the least.

Google doesn't (yet) execute a page's Javascript (JS). So if your JS replaces/creates content on a page then the content would normally be invisible to the crawlers (not good).
But, the Googlers have implemented a url hack that enables your server to create pages (from the server, not from JS), with all the different varients of your JS page's content.
This solves the SEO problem of Ajax powered pages. At least for Google searches...
See Crawable Ajax

Javascript or any scripts for that matter should never be used to house your sites content, ever! The entire web is driven by HTML and CSS, and in rare cases XML languages, everything else is a headache when it comes to SEO. Ask yourself this question, what exactly is SEO and what is it that search engines are indexing? Javascript and all programming/scripting languages are proprietary, this means that they are NOT standards as defined by the W3C, which means they are essentially worthless when it comes to indexing content. On the other hand, HTML, CSS, and XML are real standards developed for the web! It's ok to use scripts to add additional functionality to your pages, embed apps like social networking plugins, etc, but you should never use them to hold your websites HTML, CSS, or actual content ever, for any reason. Here's a link to a good article that will explain why you should be using HTML and CSS, and not a million scripts, optimizing webpages using proper html markup. Scripts cause other problems besides code that is hard for search engines to decipher. For one, they are harder for browsers to process, causing pages to load much slower than "static" pages made with HTML and CSS would. Pages made with PHP tend to create "dynamic" URL's that users and search engines cannot read. This is why Google recommends people who use jsp or PHP for their webpages include a sitemap, otherwise your links will never be found and might as well not exist. Stick to the conventions! Lets face it, we have standards for a reason. If every electronic component in your home required had a different type of plug that required a special socked, and all those devices had differing voltage and amperage requirements, what would happen? You would essentially burn down your house! And, you'd be spending 5 hours a day at the hardware store looking for those special adapters to fit your wall sockets with. If you plan on designing a website, use scripts for embedding apps or connecting with a database only, and use HTML and CSS to build "static" webpages. Also, use text links, as they are both human and search engine readable, and easy to index and make sense of. Never use scripts for your links. Programming and scripting can be fun, but not on the internet its not.
Search engines index HTML, CSS, and content (multi-media, graphics, videos, text, thats it!) everything else is pointless and annoying to both users and search engines alike. For best results use XML and design a custom language.

Google can crawl, index and rank javascript generated content.
But... it uses an old Chrome version (42) with an old javascript render engine.
The consequence is that your javascript code needs to work in older browsers and older chrome versions (older than 42). So no fancy ES6 functions, you need to use polyfills or use Babel for example.
Although you can do a lot with javascript (like click events or inject your mobile menu), it's recommended to still use a-href instead of a button with a javascript event and then using a function to get to a new page.
You can check the mobile testing tool from Google: https://search.google.com/test/mobile-friendly and check the errors/warnings/logs. If the rendered output looks like intended, Google will see your content.
In search console you can also ask to index the page. Sometimes the javascript crawler is first, sometimes the 'classic' crawler.
Doublecheck it some days afterward by googling a sentence or paragraph from your page.
There's no answer about if it's better or not. Content is content and Google should rank your website, SPA, PWA, AMP site, PDF document, online Doc, wikipage, and so on based on their content, not on the underlying technique.
If you are familiar with JavaScript, give it a go.
Regards, Peter

Security and JavaScript files containing a site's logic

Now that JavaScript libraries like jQuery are more popular than ever, .js files are starting to contain more and more of a site's logic. How and where it pulls data/information from, how that info is processed, etc. This isn't necessarily a bad thing, but I'm wondering to what extend this might be a security concern.
Of course the real processing of data still happens in the backend using PHP or some other language, and it is key that you make sure that nothing unwanted happens at that point. But just by looking at the .js of a site (that relies heavily on e.g. jQuery), it'll tell a person maybe more than you, as a developer, would like. Especially since every browser nowadays comes with a fairly extensive web developer environment or add-on. Even for a novice manipulating the DOM isn't that big of a deal anymore. And once you figure out what code there is, and how you might be able to influence it by editing the DOM, the 'fun' starts.
So my main concerns are:
I don't want everyone to be able to look at a .js file and see exactly (or rather: for a large part) how my site, web app or CMS works — what is there, what it does, how it does it, etc.
I'm worried that by 'unveiling' this information, people who are a lot smarter than I am figure out a way to manipulate the DOM in order to influence JavaScript functions they now know the site uses, possibly bypassing backend checks that I implemented (and thus wrongly assuming they were good enough).
I already use different .js files for different parts of e.g. a web app. But there's always stuff that has to be globally available, and sometimes this contains more than I'd like to be public. And since it's all "out there", who's to say they can't find those other files anyway.
I sometimes see a huge chuck of JavaScript without line breaks and all that. Like the compact jQuery files. I'm sure there are applications or tricks to convert your normal .js file to one long string. But if it can do that, isn't it just as easy to turn it back to something more readable (making it pointless except for saving space)?
Lastly I was thinking about whether it was possible to detect if a request for a .js file comes from the site itself (by including the script in the HTML), instead of a direct download. Maybe by blocking the latter using e.g. Apache's ModRewrite, it's possible to use a .js file in the HTML, but when someone tries to access it, it's blocked.
What are your thoughts about this? Am I overreacting? Should I split my JS as much as possible or just spend more time triple checking (backend) scripts and including more checks to prevent harm-doing? Or are there some best-practices to limit the exposure of JavaScripts and all the info they contain?

Nothing in your JavaScript should be a security risk, if you've set things up right. Attempting to access an AJAX endpoint one finds in a JavaScript file should check the user's permissions and fail if they don't have the right ones.
Having someone view your JavaScript is only a security risk if you're doing something broken like having calls to something like /ajax/secret_endpoint_that_requires_no_authentication.php, in which case your issue isn't insecure JavaScript, it's insecure code.
I sometimes see a huge chuck of JavaScript without line breaks and all that. Like the compact jQuery files. I'm sure there are applications or tricks to convert your normal .js file to one long string. But if it can do that, isn't it just as easy to turn it back to something more readable (making it pointless except for saving space)?
This is generally minification (to reduce bandwidth usage), not obfuscation. It is easily reversible. There are obfuscation techniques that'll make all variable and function names something useless like "aa", "bb", etc., but they're reversible with enough effort.
Lastly I was thinking about whether it was possible to detect if a request for a .js file comes from the site itself (by including the script in the HTML), instead of a direct download. Maybe by blocking the latter using e.g. Apache's ModRewrite, it's possible to use a .js file in the HTML, but when someone tries to access it, it's blocked.
It's possible to do this, but it's easily worked around by any half-competent attacker. Bottom line: nothing you send a non-privileged user's browser should ever be sensitive data.

Of course you should spend more time checking back-end scripts. You have to approach the security problem as if the attacker is one of the key developers on your site, somebody who knows exactly how everything works. Every single URL in your site that does something to your database has to be protected to make sure that every parameter is within allowed constraints: a user can only change their own data, can only make changes within legal ranges, can only change things in a state that allows changes, etc etc etc. None of that has anything at all to do with what your Javascript looks like or whether or not anyone can read it, and jQuery has nothing at all to do with the problem (unless you've done it all wrong).
Remember: an HTTP request to your site can come from anywhere and be initiated by any piece of software in the universe. You have no control over that, and nothing you do to place restrictions on what clients can load what pages will have any effect on that. Don't bother with "REFERER" checks because the values can be faked. Don't rely on data scrubbing routines in your Javascript because those can be bypassed.

Well, you're right to be thinking about this stuff. It's a non-trivial and much misunderstood area of web application development.
In my opinion, the answer is that yes it can create more security issues, simply because (as you point out) the vectors for attack are increased. Fundamentally not much changes from a traditional (non JS) web application and the same best practises and approaches will server you very well. Eg, watching out for SQL injection, buffer overflows, response splitting, etc... You just have more places you need to watch out for it.
In terms of the scripts themselves, the issues around cross-domain security are probably the most prevalent. Research and learn how to avoid XSS attacks in particular, and also CSRF attacks.
JavaScript obfuscation is not typically carried out for security reasons, and you're right that it can be fairly easily reverse engineered. People do it, partially to protect intellectual property, but mainly to make the code download weight smaller.
I'd recommend Christopher Wells book published by O'Reilly called 'Securing Ajax Applications'.

There is free software that does JavaScript Obfuscation. Although there is not security though obscurity. This does not prevent all attacks against your system. It does make it more difficult, but not impossible for other people to rip off your JavaScript and use it.
There is also the issue of client side trust. By having a lot of logic on the client side the client is given the power to choose what it wants to execute. For instance if you are escaping quote marks in JavaScript to protect against SQL Injection. A Hacker is going to write exploit code to build his own HTTP request bypassing the escaping routines altogether.
TamperData and FireBug are commonly used by hackers to gain a deeper understanding of a Web Application.
JavaScript code alone CAN have vulnerabilities in it. A good example is DOM Based XSS. Although I admit this is not a very common type of XSS.

Here's a book by Billy Hoffman about Ajax security:
http://www.amazon.com/Ajax-Security-Billy-Hoffman/dp/0321491939/ref=sr_1_1?ie=UTF8&s=books&qid=1266538410&sr=1-1