Using Google Closure Compiler to minify a bunch of javascripts. Now I'd like to also add source maps to those to debug out in the wild.
Thing is, I want to keep the original (and preferrably also the map files) on a completely different place, like another server. I've been looking for a solution to this, and found out about the sourceRoot parameter. But it seems as it's not supported?
Also found this --source_map_location_mapping parameter, but no documentation whatsoever. Seems as it wants a pipe-delimited argument (filesystem-path|webserver-path). Tried a couple of different approaches to this, like local filename|remote url but without prevail. That just gives me No such file or directory and java.lang.ArrayIndexOutOfBoundsException.
Has anyone succeeded to place the minified/mapped source files on a remote machine?
Or does anyone know of any documentation for --source_map_location_mapping?
Luckily Google Closure Compiler's source code is available publicly
https://gist.github.com/lydonchandra/b97b38e3ff56ba8e0ba5
REM --source_map_location_mapping is case SENSITIVE !
REM need extra escaped double quote --source_map_location_mapping="\"C:/tools/closure/^|httpsa://bla/\"" as per http://stackoverflow.com/a/29542669
java -jar compiler.jar --compilation_level=SIMPLE_OPTIMIZATIONS --create_source_map=C:\tools\closure\latest\maplayer.js.map --output_wrapper "%output%//# sourceMappingURL=maplayer.js.map" --js=C:\tools\closure\mapslayer.js --js_output_file=maplayer.min.js --source_map_location_mapping="\"C:/tools/closure/^|httpsa://bla/\""
The flag should be formatted like so:
--source_map_location_mapping=foo/|http://bar
The flag should be repeated if you need multiple locations:
--source_map_location_mapping=foo/|http://bar --source_map_location_mapping=xxx/|http://yyy
But what I expect that you are running into is that the "|" might be interpreted by your command shell. For example:
echo --source_map_location_mapping=foo/|http://bar
-bash: http://bar: No such file or directory
(The choice to use "|" was unfortunate). Make sure it is escaped appropriately. like:
--source_map_location_mapping="foo/|http://bar"
I submitted a pull request to report an error for badly formatted flag values:
https://github.com/google/closure-compiler/pull/620
which will at least you know that your flag value is incorrect (so you won't see the out of bounds exception).
John is correct functionality-wise, but I think I can clear it up a bit (as this was super confusing for me to get working).
I suspect many people have the same issue as I:
source map urls are generated relative to your current directory
they don't necessarily match up to relative urls on your website/server
Even if they did match up directly, the strangely-defined pseudo-spec found here means that Chrome/Firefox are going to try to load your paths relative to your sourcemap. i.e. the browser loads /assets/sourcemaps/main.map, sees assets/js/main.js, and loads /assets/sourcemap/assets/js/main.js (yay). (Or it might be relative to the original js file actually, I just happened to have them in the same directory).
Let's use the above example. Say we have assets/js/main.js in our sourcemap, and want to make sure that loads mywebsite.com/assets/js/main.js. To do this, you'd pass the option:
--source_map_location_mapping="assets|/assets"
Like John mentioned, quotes are important, and repeat the arg multiple times for multiple options. The prefixed / will let Firefox/Chrome know you want it relative to your website root. (If you're doing this in something like grunt-closure-tools you'll need to escape more:
config:{
source_map_location_mapping:"\"assets|/assets\"",
}
This way, we can essentially map any given sourcemap path to any given website path. It's not really a perfect replacement for some sort of closure source root, but it does let you map each section of your sources individually to their own roots, so it's not that bad a compromise, and does give some additional flexibility (i.e. you could specify some cdn paths for some of your assets but not for other).
An additional thing you might find helpful, you can automatically add the sourceMappingURL via an output_wrapper. (Though, if you want the ability to debug in production, you should probably prefer some ability to make the server return X-Sourcemap: blah.js.map headers instead, inaccessible by the public)
--output_wrapper="(function(){%output%}).call(this); //# sourceMappingURL=/assets/js/my_main_file.js.map"
Related
In CSS I know that I can reference/link to another file using either of the following:
photo.jpg or ./photo.jpg for a file in the same location as the one I am working in, and photo-folder/images/photo.jpg or ./photo-folder/images/photo.jpg to link a file in one or more directories below/inside the parent directory of the one I am working in.
I also know the same idea applies in HTML like ./[href] or [href].
But I am wondering, let's say the positions of the files in my example are reversed, that is, I want to link to a photo in a directory which is the direct/indirect parent of the file I'm working in. How would I call that file without using the full drive address such as C:/Users/Username/Code/etc/photo.jpg, while my working file is C:/Users/Username/Code/etc/code/file.css...
How is it done in these languages: HTML, CSS, JS, Python? (You are welcome to answer for other languages, I might need it later! :D)
Side question: is there any technical difference between ./file and file, and would it be preferable to stick to one format over the other (e.g. to prevent problems, for readability, etc)? Though I have never run into any problems using either, or even combining both in the same project...
To get to parent directory use ../, this is "per directory", so if you need access several levels up, you'd need use: ../../../ (for 3 levels)
As of using ./ for current directory - it's redundant
As far as I know this is pretty much applies to all languages, however in HTML/CSS there is another relative path that can be used to get to the root of the domain: / (no dots)
so for example if you open https://example/some/long/path/file.html and need access to file at http://example/file.jpg you simply put as relative path: /file.jpg
Some websites seem to have file names such as 'assets/app-02b4523sev8fsd56e.js'. I have noticed that these numbers do not change though, so I thought it has something to do with security but I am not sure. Is there any reason behind this?
This is normally to break caches stored by the browser so that the latest version of a file is loaded. Every time a file is changed this value will normally be changed also. This can be done manually by changing the filename and/or the paths in other files referencing this file, or this can be done programatically in some way. You may also see this done like the following and it may also contain a timestamp rather than a hash like the above:
assets/scripts.js?v=20150611190618
This is often referred to as a 'cache-buster' amongst other names.
ive tried everything i cud to figure this out, but i cannot track a piece of javascript in a webpage
so, just to give you some context even though my problem is not related to just this scenario. it depends on a much bigger spectrum.
Anyway, im developing on sugarCRM and im trying to edit the default onclick behavior of a slot in calendar module (you dont need to understand this to help me, so please keep reading). when i click on a slot, a modal dialog window opens that lets me log a meeting or a call.
So i tracked down the javascript behind this. ive used firebug and chrome, and they both give a list of all the JS files that are being used on a given webpage
for example i search for "SUGAR.collection" and firebug tells me its located in a file named "sugar_field_grp.js?v=FVh1Z-v5nA6bYov7-aFFqQ" i can see this piece of code resides in sugar_field_grp.js,
but the code im trying to change resides in "index.php?module=Calendar&action=index&parentTab=Activities", firebug actually tells me this is the file that has the javascript i want to change.
I can also right click view page source and i can see that piece of code inside the script tag. so considering this piece of code doesnt reside in a JS file, i cannot change it, its generated at runtime (i think) but there must be some source, there must be a file thats telling sugarCRM to generate this code
tl;dr how to track down a piece of javascript code that resides on pagesource and theres no JS file specified by firebug or chrome save for index.php (this file doesnt have that javascript either)
i know its been a long post
thanks for reading
Learn how to search for strings in files on disk on your machine.
On Linux, MacOS and most unixen the go-to tool for this is grep. This applies to any programming language you work with. For your case simply cd into the directory of your source code and do:
grep -r SUGAR.collection .
If you're using git as your source control tool then git grep is much faster.
On Windows there are various GUI tools you can use to search for text in files. Just google: grep for windows.
If you're using an IDE then just your IDE's find-in-files functionality.
To track down specific code using Chrome / Webkit go through the following two steps:
Client:
1. Search all static text sources
Open the Dev Panel using CTRL + SHIFT + I
Hit CTRL + SHIFT + F for a global search dialog to pop up
Right next to it you can set pretty printing of the JS code to on: button { }
Enter your search term or terms using regular expressions
Optional: Decide if you need a case insensitive search which has a greater searchspace and takes longer
Example:
2. Search the dynamic user-DOM contents
Go to the Tab 'Elements' hit CTRL + F.
Enter your search term (This will also search iframes, svg's etc... within the parent DOM)
3. Recommended:
Cross-reference the results of step 1. and step 2.
If a given string is present in both the DOM and the static sources, then you can assume that the content is not programmatically created on the client-side.
Server:
Many projects perform a media bundling step prior to content-delivery. They pack web-resources into the main file (e.g. index.php) to save HTTP roundtrips.
Use sourcemaps / and or search the entire codebase for a salient static string or a salient keyword near the static string to locate the original source files.
Searching files:
Locally, I generally use the rapid index, and heuristic search of JetBrain's IDE's (IDEA, PHPStorm,...) and Sublime. The grep-command tool can definitely not compete here in terms of performance. On Windows I additionally use Totalcommander and its archive/regex finding abilities.
When quickly looking up code on the server you may use something like:
grep -r -C10 --color=always 'keyword1|keyword2' htdocs/ | less -R
which will also provide you with line-context. two caveats: you may want to filter out binaries first and symlinks outside the scope will be ignored.
If I decide to use last_modified_time of a javascript or css file, and use the unix timestamp of it as a key in the name to bust cache when file is modified.
What is the difference between following two practices ?
filename is : my_script.js and timestamp is : 1321951817
1/ File gets included as :
<script type="text/javascript" src="http://example.com/js/my_script.js?v=1321951817"></script>
Hence,the query string parameter creates a new cache everytime the v is changed.
2/ File gets included as :
<script type="text/javascript" src="http://example.com/js/my_script.1321951817.js"></script>
The filename changes with every modification, a rewrite rule removes the timestamp and points the requested url to my_script.js
3/ UPDATE: ONE MORE METHOD BASED ON ANSWERS BELOW : File is renamed and get included as :
<script type="text/javascript" src="http://example.com/js/my_script.1321951817.js"></script>
The filename is changed and NO REWRITE RULE is used.
Question : Are these two techniques inherently the same, or are there any advantages/disadvantages of using query string parameters instead of direct file name.
Using an updated querystring is a bad solution. See what Steve souders has so say about it: http://www.stevesouders.com/blog/2008/08/23/revving-filenames-dont-use-querystring/
The ideal method is to rename the file itself.
Some people prefer using the time stamp of the last modification date, which i think is a problem.
In modern web development you really need to optimize your page as much as possible, which means combining css and javascript into single files which are minfied.
That means that you introduce a build step into your process, and that the last modification time of your file always will be at your last build. If you set that as your file name, you essentially bust the users cache all the time, and some times you dont need to.
I recommend renaming the files to an md5 sum of their content. That way you can do new builds all the time, but the file name only changes if the content changes. This makes your file name an identifier of the content. Using this you can set a far future expires header on all your static content and simply stop worrying about it any more.
I can recommend using a build system for this, since this workflow gets boring fast.
My company open sourced one a while ago that does this among a lot of other things that optimize your web page: https://github.com/One-com/assetgraph-builder
There are many other build tools that do the same. Take a look around and find the one that best suits your development setup.
You are saying it yourself: in the second example you are using a rewrite rule which checks with regex every page you are loading.
The first one just fooling the browser to think it's a different file. So the first one is the way to go.
I observed chunks like below sometimes on web pages. So i am curious to know what this really does? or why its written in such a way?
<script src="somefile.js?param1=one¶m2=two" />
i can only make out following few intentions behind it
Its not page URL (i mean .aspx/.php/.jsp etc.) so its not hacking kind of code where user can add code like this to pass data without getting users attention as its tag which does not render on UI OR implementing old type of AJAX alternative
This kind of URL param are useful if user do not wish the JS file (any other resource like image) to get cached. This can be quick way to manage caching
But i am unable to figure out following
Looks like page URL parameters but are these parameters anyway readable in JavaScript file and have some additional utility?
Do these parameters have any extra role to play here ?
What are the other possible practical scenarios where code like this can be/is used?
So please provide some inputs related with the same
Thanks,
Running Non-JS Code within .js Extensions
In cases like this, that source .js file might (given proper server-configurations) actually have PHP/.NET code within it, which can read those appended values.
As you said, Avoiding Cache...
Additionally, people will at times append a random string at the end of their referenced elements to avoid loading cached data.
The URL having '.js' means nothing. It could still be handled by a server-side script like an ASP or PHP.
Either the javascript file is not static (it is generated by the server based on the parameters in its querystring)
OR
In the JavaScript file itself, you can have it check its own querystring parameters (not just that of the page, but that of the javascript source url).
OR
(This doesn't exactly match your scenario, but) you can also add parameters at the end of image and script urls as a way of versioning. The version with the url="somescript.js?V=3" will be cached by the user until the page then changes and the url is not="somescript.js?V=4". The file will be replaced by the version on the server no matter what the browser setting may be.
My guess (without looking at this specific case) is that the javascript file is reading its own querystring. I have done this, and its very helpful.
Looks like page URL parameters but are these parameters anyway readable in JavaScript file and have some additional utility?
Yes you can read them in JavaScript, Scriptaculous uses that approach for loading modules, eg:
<script type="text/javascript" src="scriptaculous.js?load=effects,dragdrop">
</script>
Do these parameters have any extra role to play here ?
What are the other possible practical scenarios where code like this can be/is used?
That can be also used for server-side script joining and minifying, of course using some url rewriting technique to have the .js extension, and as you say, it's a common technique to add timestamp parameters to break the browser cache.
It can be used for three different reasons:
1) To generate the JavaScript file in the server depending on the parameters;
2) To avoid caching;
3) To pass parameters to JavaScript itself
An example of this in practice would be a server side handler for somefile.js that uses the parameters (names of other scripts) to determine which scripts are actually required and combine/minify them, returning them as a single somefile.js script file.