Login Register

ShrinkSafe

Overview

The last optional step in Dojo's build process is compression. Like many non-trivial JavaScript tools, Dojo uses a tool to reduce the size, and therefore latency, of JavaScript by browsers. This article covers only the compression system. While the rest of the Dojo package and build system are interesting in their own rights, there is a lot of pent-up demand for a stable, reliable, and non-destructive JavaScript compressor.

While many compression tools exist, several factors together make the Dojo compressor unique. We'll get to those, but first, how to make it work for you.

Example

To run this example, you'll need a working install of Java (at least 1.4 is recommended). The latest version of the tool itself can be downloaded from Dojo Subversion at:

http://svn.dojotoolkit.org/src/util/trunk/shrinksafe/custom_rhino.jar

To demo our compression system, let's build an example that we want to compress. Here's the listing for infile.js:

function MyClass(){
    this.foo = function(argument1, argument2){
        var addedArgs = parseInt(argument1)+parseInt(argument2);
        return addedArgs;
    }
    var anonymousInnerFunction = function(){
        // do stuff here!
    }
}
function MyFunc(){
    // this is a top-level function
}
// we've got multiple lines of whitespace here

We run it through the compressor with this command to generate outfile.js:

java -jar custom_rhino.jar -c infile.js > outfile.js 2>&1

And the contents of outfile.js should now look something like:

function MyClass(){
this.foo=function(_1,_2){
var _3=parseInt(_1)+parseInt(_2);
return _3;
};
var _4=function(){
};
}
function MyFunc(){
}

So that looks smaller, but by how much?:

obelisk:/tmp/compress alex$ ls -lah
...
-rw-r--r--    1 alex  users  321B Aug 12 09:21 infile.js
-rw-r--r--    1 alex  users  140B Aug 12 09:21 outfile.js

321 bytes to 140 bytes, a 56% reduction. Not bad!

Riding a Rhino

So how does this all work? And why should anyone choose this tool over the other ones that are available. The answer to both of those questions is in the design of the the Dojo compressor. Instead of brittle regular expressions, the Dojo compressor is based on Rhino, a JavaScript engine from the Mozilla project.

Being based on a real parse stream, the Dojo compressor can get a better idea for the context of a token (variable name, etc.) than the regular-expression based tools. This allows us to achieve the over-riding goal of a compressor that would be acceptable to the Dojo project: it must never mangle a public API.

API Safety

There are many "obfuscators" available in addition to size reduction tools. Over the years, many people have attempted to "encrypt" or otherwise obfuscate JavaScript sent over the wire to browsers, and it never pans out. Why not? For starters, JavaScript (as implemented in browsers) is completely interpreted. This means that any further compilation beyond source transformations will not work everywhere, and the tool provides a "decryption" tool along with the "encrypted" or obfuscated source, the unencrypted version will be available at runtime for anyone with a debugger to see. For those tools that just transform source code by mangling variable names, it's even easier to revert their changes. Therefore, obfuscation and encryption aren't useful goals. Size reduction, on the other hand, is a useful goal.

But not if your size-reduction tool breaks things. There are, of course, many increments available for the "compression" process. Potential choices available to a tool author include:

  • removing comments
  • collapsing line-beginning whitespace
  • removing line-ending whitespace
  • collapsing multiple blank lines
  • removing all new-line characters
  • removing whitespace around operators
  • removing whitespace near/around curly braces
  • replacing symbols with shorter names (this is how most "obfuscation" is done)

And the list goes on and gets ever-more esoteric as one tries to squeeze every last K out of a JavaScript file. But at some point, you can go too far. The Dojo compressor attempts to strike a balance between debuggability (not replacing all symbols and not removing all newlines) and size reduction.

Getting The Source

The source code for Rhino is available from Mozilla anonymous CVS. Instructions for Mozilla CVS are at:

http://www.mozilla.org/cvs.html

And the Rhino code lives in their repository at:

/cvsroot/mozilla/js/rhino

Our patches should apply cleanly against Rhino HEAD and are available in unified diff format from:

http://svn.dojotoolkit.org/src/util/trunk/shrinksafe/custom_rhino.diff

Unlike the rest of Dojo, the Dojo Foundation does not control the copyright of the original work, and we therefore cannot license this code under there AFL. It is made available under the tri-licensing terms of the Mozilla project.

The Future

The Dojo compression tool is by no means the last word in file-size or on-the-wire reduction. Gzipping content on the wire is the next obvious improvement for those deploying applications.

The Dojo package system and JSAN allow developers to include just those packages from a library that they require, and future work on real JS linkers will further strip down capable libraries like Dojo to only absolutely what is needed by application authors.

The Dojo project intends to continue to produce the best Open Source tools for JS and webapp developers, and we will make these transparently available in the Dojo build system, as we do today with the compression and package systems.

UTF8 support

I have created a new version of the patch that adds an option to specify the character set (eg. UTF8) used by the input file. You can download it from: http://paiq.nl/custom_rhino_charset.diff

Using the current Rhino HEAD gives some runtime problems you will want to avoid, try an older version. I used:

cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -D "20 Apr 2006 17:00 PDT" mozilla/js/rhino

Apart from that, compilation and usage is the same; except for the added optional '-E <encoding>' parameter.

--Frank

Post UTF-8 ShrinkSafe?

Any chance you can post the JAR for the UTF-8 version?

download the patch and apply

download the patch and apply it... thats why it was made available....

-Karl

That implies I want to do a

That implies I want to do a full build of something that I want to use as a tool. I don't have a java dev environment, and don't particularly relish getting one. If this is the bar required to use shrinksafe, then I'll evaluate that decision. But if somebody already did the hard work and built one, then I have a bunch of other things I could spend the 3 hours on.

JAR would be handy

I agree with the previous poster.

Dont really feel like getting CVS up and running (i'm using SVN), setting up a java build environment, ajust all classpaths and stuff to compile that tool when i see that maybe someone else has done that successfully..

So, providing a built version of the patched rhino would save me (and others?) a lot of time.

Thx in advance,

Alex

UTF8 support: JAR!

By popular demand: :)

http://paiq.nl/custom_rhino_charset.jar

--Frank

Thank you very much :)

made my life much easier.

Greetings,

Alex

re

Frank thank You for link I was looking for it for a long time.
Best Regards
Tom

illegal character

Hi,
I' ve tried to use the custom_rhino_charset.jar, but I got this error message :(

js: "objClient.js", line 1: illegal character
js: function obj_keyboardControler(svsForm){
js: .^

Visual Studio wrote this weird "" into my js files.
Any chance to work around?

Best regards.

Sweet!

Hi,

Nice work! I'll have to give ShrinkSafe a try. How does it compare to JSMin or the YUI Compressor?

- rob

compressor rater

I'm working on a tool that does a comprehensive comparison of all the various compression tools, including the 3 you mentioned.
I will post some statistics soon!

Really good

Shrinksafe is a really good tool, no doubt.
But I'm bothered by the fact, that the new Update for IE these days, still has the same problems with some java webappliactions like before.
Hope the next version will support it completely, so that everyone can admire my java features compressed with ShrinkSafe.

--------------
Kathi

Optimize String Concatenations

In my web project, I often use String concatenations.
It's because I use innerHTML a lot to change the content of the screen.
Using innerHTML is faster to run and to code than DOM methods.
To help keep the source clean, I often do things like this:

html +=
"<li class='pressable'>" +
"<a onclick='return Screen.pressAutoRefresh()' href='#'>" +
"<img width='22' height='22' alt='' src='icones/22/auto-refresh.png'/>" +
"</a>" +
"</li>";

The compressed result is something like this:
"<li class='pressable'>"+"<a onclick..."

Obviously, a lot of '"+"' can be removed to optimize concatenations so that the browser do not have to do useless computations.

By doing so manually, I'm often gaining from 1/3 to 1/2 of speed!
This is important.

Any chance ShrinkSafe can do that automatically?

And, by the way, I would want to include Shrink safe automatically in my Eclipse JSP project so that as soon as I modify a .js script, ShrinkSafe compress it and place it in the build folder. I've downloaded ShrinkSafe JAR and I'm now trying to do that. Do someone already done that? Any pointer to help me?

Combining string literals

Combining string literals into one big string is a good idea. As far as I know, none of the other free compressors do that either...
But it would be a neat feature to add!

Chris Zyp has made a tool that can run Dojo ShrinkSafe and cache the results at runtime as a Servlet Filter.
I know it's not exactly what you are looking for, but it's an alternative way to do this to achieve the same end result.
It also includes Gzip compression, which is a much larger factor in the total effective amount of compression you get in the end.

I hope this might be able to help you:

http://xucia.com/page/Resource_Accelerate?mode=html&noscript=true
--
Arthur Blake

String Concatenations

FYI, this feature was just added to the YUI compressor!
See:

http://www.julienlecomte.net/blog/2007/08/29/yui-compressor-version-21-n...
--
Arthur Blake

Explain the arguments we can use

Works very well for me, now I would like to play with the different level of compression, I've read about removing carriage return as an option and not being done by default. Which parameter is that ?

Same here

I would also like to know the different arguments what what they do.

Works Fine

Hi, it is working fine for me. What are the issue you have encounter?

Regards,
James Burt

The CompressorRater

The CompressorRater is a web application that lets you dynamically compare the compression levels of Dojo Shrinksafe, along with several other popular javascript compressors, both with and without the affects of additional gzip compression.

check it out here:

http://compressorrater.thruhere.net/
--
Arthur Blake

Exactly what i need

Thanks for the script. Helped me with my studies in java at university.

best regards

Nico

Internet Advices

Batch File for Noob Accessibility

hi, I really like ShrinkSafe. In fact I recommended it to some of my friends. And their feedback?

"I can't remember the syntax well!"
"Its too long. I'm lazy to type in a lot of letters!".

So I decided to make a simple batch file that will do the standard compression easier. Of course, this code is modifiable to also accept the other parameters for advanced compression settings. Also, this assumes a Windows XP environment with JAVA installed. I named the batch file "ShrinkSafe.bat" by the way.

@echo OFF
cls
echo Dojo ShrinkSafe - Javascript Compression System
echo.
set /p input=What script do you wish to compress? 
set /p output=What filename do you wish to output it on? 
java -jar custom_rhino.jar -c %input% > %output% 2>&1
set name=
echo.
echo Compression done!
echo.
pause
exit

Edit: Oh and a request to the developers perhaps:

When I compress my scripts, it always produces 2 blank newlines at the bottom. Is it just me? It's still 2 whitespaces right?

Web interface for Extreme Noobness

It's also extremely easy to use ShrinkSafe via the CompressorRater:

http://compressorrater.thruhere.net/

The CompressorRater is a tool for rating several different JS compressors but it also lets you get the output of those compression runs.

--
Arthur Blake

Batch file help

Hi guys,
Could someone tell me how to do the following correctly?

java -jar custom_rhino.jar -c infile1.js infile2.js infile3.js > outfile.js 2>&1

That is spcefiy more than one input file and also to strip newline characters as you can on the web interface.

Thanks!

Works very well for me, now

Works very well for me, now I would like to play with the different level of compression, I've read about removing carriage return as an option and not being done by default.

Alex Mailer

Great Source...

...great info and heads up sharing how you guys do this and what tools you use, greatly appreciate it.

Good stuff.

Nice.

What are the issue you have encounter

Hi, it is working fine for me. What are the issue you have encounter?

Regards,
James Burt

oyun

Great work ShrinkSafe Thank

Great work ShrinkSafe

Thank You
Brent

URL

It seems that URL in the article are wrong, diff and jar are in http://svn.dojotoolkit.org/util/trunk/shrinksafe/

Fixed

Thanks beuss, fixed now. :)

Mafyalemi

Thanks

MafyAlemi

Dojo compressor

Ive read that Dojo compressor doesn’t like that the variable event is redeclared or the compressed code will produce errors...

Thanks,
Paul
boxing gloves and mma uniforms
personalized dog tags
fundraising ideas

Example

Hi Paul,

If you can find some example code that fails, or can find a reference for what you read then we'd appreciate you filing a ticket about it so we can fix it.

- Rob :)

Thank you!

Hi guys,

great info. Works fine!

Klassische Musik

Error when i try to compress js with regular expression

js: "validate_fields.js", line 46: syntax error
js: var emailFilter = /^(^|[ \t\r\n<;(])((([A-Za-z0-9$_.+%=-])|%[A-Fa-f0-9]{2})+@(([A-Za-z0-9$_.+!*,;/?:%&=-])|%[A-Fa-f0-9]{2})+\.[a-zA-Z0-9]{1,4})+$/;
js: ..................................................................................................................................^
js: "validate_fields.js", line 1: Compilation produced 10 syntax errors.

Rhino-based compressor and obfuscator

FYI, if you still wish to compress names of local variables (non-destructive compression) AND change public API names too (destructive compression, or obfuscation) you can use another Rhino-based tool - JavaScript Obfuscator from Stunnix. It has a GUI too, can process code inside HTML, and can process several file at once (thus it's much faster).

thanks for sharing. It works

thanks for sharing. It works great for me!

Barton

thanks...

Hi guys,
Could someone tell me how to do the following correctly?

java -jar custom_rhino.jar -c infile1.js infile2.js infile3.js > outfile.js 2>&1

That is spcefiy more than one input file and also to strip newline characters as you can on the web interface.

Thanks! kız oyunları

good work !

Great!And size-reduction it's very important to me.
Keep it powerful, and small . dll download

That is spcefiy more than

That is spcefiy more than one input file and also to strip newline characters as you can on the web interface.

Thanks! Oyun | Thanks! Okey Thanks! Sinema