The YUI Compressor uses a slightly modified version of the parser used in the Rhino JavaScript engine. The reason for modifying the parser came from the need to support JScript conditional comments, unescaped forward slashes in regular expressions — which all browsers support and many people use — and a few micro optimizations. The problem is that many users had the original Rhino Jar file somewhere on their system, either in their classpath, or in their JRE extension directory (<JRE_HOME>/lib/ext) This caused many headaches because the wrong classes were being loaded, leading to many weird bugs.
Today, I finally decided to do something about it. This meant writing a custom class loader to load the modified classes directly from the YUI Compressor Jar file. You can download the source and binary package here:
Download version 2.4 of the YUI Compressor
The skeleton of the custom class loader is pretty straightforward:
package com.yahoo.platform.yui.compressor;
public class JarClassLoader extends ClassLoader
{
public Class loadClass(String name) throws ClassNotFoundException
{
// First check if the class is already loaded
Class c = findLoadedClass(name);
if (c == null) {
// Try to load the class ourselves
c = findClass(name);
}
if (c == null) {
// Fall back to the system class loader
c = ClassLoader.getSystemClassLoader().loadClass(name);
}
return c;
}
protected Class findClass(String name)
{
// Most of the heavy lifting takes place here
}
}
The role of the findClass method is to first locate the YUI Compressor Jar file. To do that, we look in the classpath for a Jar file that contains the com.yahoo.platform.yui.compressor.JarClassLoader class:
private static String jarPath;
private static String getJarPath()
{
if (jarPath != null) {
return jarPath;
}
String classname = JarClassLoader.class.getName().replace('.', '/') + ".class";
String classpath = System.getProperty("java.class.path");
String classpaths[] = classpath.split(System.getProperty("path.separator"));
for (int i = 0; i < classpaths.length; i++) {
String path = classpaths[i];
JarFile jarFile = new JarFile(path);
JarEntry jarEntry = findJarEntry(jarFile, classname);
if (jarEntry != null) {
jarPath = path;
break;
}
}
return jarPath;
}
private static JarEntry findJarEntry(JarFile jarFile, String entryName)
{
Enumeration entries = jarFile.entries();
while (entries.hasMoreElements()) {
JarEntry entry = (JarEntry) entries.nextElement();
if (entry.getName().equals(entryName)) {
return entry;
}
}
return null;
}
Once we know where the YUI Compressor Jar file is, we can load the appropriate class from that file. Note the need to define the package the class belongs to before calling defineClass!
protected Class findClass(String name)
{
Class c = null;
String jarPath = getJarPath();
if (jarPath != null) {
JarFile jarFile = new JarFile(jarPath);
c = loadClassData(jarFile, name);
}
return c;
}
private Class loadClassData(JarFile jarFile, String className)
{
String entryName = className.replace('.', '/') + ".class";
JarEntry jarEntry = findJarEntry(jarFile, entryName);
if (jarEntry == null) {
return null;
}
// Create the necessary package if needed...
int index = className.lastIndexOf('.');
if (index >= 0) {
String packageName = className.substring(0, index);
if (getPackage(packageName) == null) {
definePackage(packageName, "", "", "", "", "", "", null);
}
}
// Read the Jar File entry and define the class...
InputStream is = jarFile.getInputStream(jarEntry);
ByteArrayOutputStream os = new ByteArrayOutputStream();
copy(is, os);
byte[] bytes = os.toByteArray();
return defineClass(className, bytes, 0, bytes.length);
}
private void copy(InputStream in, OutputStream out)
{
byte[] buf = new byte[1024];
while (true) {
int len = in.read(buf);
if (len < 0) break;
out.write(buf, 0, len);
}
}
The last thing we need to do is bootstrap the application. In order to do that, we simply load the main class (YUICompressor) using our new custom class loader. All the classes that will be needed at runtime will use the same class loader:
package com.yahoo.platform.yui.compressor;
public class Bootstrap
{
public static void main(String args[]) throws Exception
{
ClassLoader loader = new JarClassLoader();
Thread.currentThread().setContextClassLoader(loader);
Class c = loader.loadClass(YUICompressor.class.getName());
Method main = c.getMethod("main", new Class[]{String[].class});
main.invoke(null, new Object[]{args});
}
}
As you can see, it's not terribly complicated to write a custom class loader. Note: I left out all the exception handling code and the import statements for clarity. The final code can be found in the downloadable archive. Cheers!
Why aren’t these optimizations directly injected into the Rhino project, it would be very helpful?
Wouldn’t it have been much easier to just change the modified-Rhino package name? Just like for instance the Xalan version included in recent JREs live in com.sun.org.apache.xalan.internal. GWT uses a modified Rhino parser too, which lives in the com.google.gwt.dev.js.rhino package.
@JY
As a reference implementation, Rhino must stick as close as possible to standard ECMAScript. The changes I made to the parser are to accommodate the more relaxed syntax supported by modern web browsers, and to support features that aren’t part of the standard.
@Thomas
My goal was to modify the smallest possible number of files from the Rhino distribution. That way, fixes in Rhino ca be easily backported into the YUI Compressor. Moreover, using a class loader is a slightly more elegant (not to say funner) solution!
Hello,
Do you plan to support Javascript 1.7 features in a future version?
We are using the compressor for a Firefox extension, and we use a lot of JS 1.7/1.8.
Thanks.
Hello, and thanks for all your work on YUI Compressor! Seems to be very useful and solid, and I was excited to try it on some very large scripts I’ve written.
One problem I have found, though, is that Rhino (apparently) chokes on the “debugger” keyword in JavaScript. I have some scripts that, under certain circumstances, execute the debugger keyword to force a breakpoint and activation of a JavaScript debugger (if present) — i.e. as a developer, if I’m testing my JavaScript while it’s running in an instance of IE embedded in another application, and I need to get in and inspect the DOM or the state of my code, I set things up such that I can give the code some secret input (in my case I click a certain button with Ctrl+Shift held down), which triggers execution of the debugger keyword. This causes the embedded instance of IE to invoke my JS debugger — it’s very useful in some cases.
When YUIC encounters the debugger keyword in JavaScript, though, it reports the following syntax error:
[ERROR] xxx:yy:identifier is a reserved word
…
org.mozilla.javascript.EvaluatorException: Compilation produced syntax error.
at com.yahoo.platform.yui.compressor.YUICompressor$1.runtimeError(YUICompressor.java:135)
at org.mozilla.javascript.Parser.parse(Parser.java:410)
at org.mozilla.javascript.Parser.parse(Parser.java:355)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.parse(JavaScriptCompressor.java:312)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.(JavaScriptCompressor.java:533)
at com.yahoo.platform.yui.compressor.YUICompressor.main(YUICompressor.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.yahoo.platform.yui.compressor.Bootstrap.main(Bootstrap.java:20)
Seems like “debugger” is on a list of reserved words that Rhino is aware or, but at the same time it doesn’t expect to ever find it in a script. As I said, it executes perfectly well under Internet Explorer; it’s also documented as working in Mozilla & derivatives. Any chance you’d consider (further) modifying Rhino to accept the debugger keyword?
Hi,
Is there a bug tracking site for YUICompressor? Generally, it looks like it is working fairly well, but I am seeing a JS file fail miserably even though the JS is all valid. Is there anywhere to report this issue?
The JS file in question that YUICompressor is choking over is from the jQuery Rich Array plugin.
Is there something special about this JS that the Compressor would find faulty?
Thanks,
Eric
Hi,
It may be possible to add another option to the compressor so ANYTHING that start with an underscore will be replaced like any scoped variable (much like Dean Edwards js compressor).
Also, it would be nice another option (if I ask, I ask a lot :) ) to force variable replacement, even when the compressor finds an eval? Of course that may be dangerous if you don’t know what you are doing, but I have some specifics where I needed eval and I’m sure what happens inside the eval will not break because of an invalid referenced variable (like parsing json data or handling conditional comments for explorer).
Thanks in advance,
Marcelo.
First off, I love YUI, and the compressor. Nice work!
My question:
I’m trying to automate the “minification” of my site’s files using a Java app. I basically took the code from YUICompressor.java and hacked it to fit my needs. It almost works, but when my files are minified they are truncated at 8192 bytes (8k).
As an experiment I took my input stream and wrote it directly to my output stream and I get the full file contents. It’s only when I call JavaScriptCompressor’s compress() method that the file gets truncated.
Any suggestions? It even gets truncated if I use System.out as my output stream. I’m baffled.
Thanks!
Answered my own question. I was using a ByteArrayOutputStream and accidentally calling “.flush()” for the stream itself rather than the stream writer before trying to convert the results to a string. Changed one line of code and it’s all working now.
Thanks again for a great utility!
I am having the same issues as Lowell and Eric. I am getting the exception org.mozilla.javascript.EvaluatorException while using a jQuery plugin which works on the browsers I’ve tested (IE[6,7] and FF)
Any work arounds?
I narrowed my issue down to two scripts
1. jquery-however-intent.min.js
2. jquery.blockUI.js
Anyone else have this issue and figured out a way to resolve it?
The bootstrapping works fine when executing the jar like normal, but I’m calling the compress method directly, like this:
JavaScriptCompressor compressor = new JavaScriptCompressor(in, e_reporter);
compressor.compress(out, -1, false, true, true, false);
I had to add the regular js.jar into the classpath for a new project i’m working on, and now the compressor isn’t working properly any more. Any ideas?
In WebSphere 6.0 Rhino is provided in the parent classloader so I found it was necessary to rename packages (as suggested by Thomas Broyer). It is possible to automate this operation. Anyone interested should check out the solution used by the Liferay project. I am not affiliated with Liferay, but I found their scripts very helpful.
Dean, I’ve fixed the issue with jquery.blockUI.js If no charset paramis specified for YUICompressor then it use system default encoding (which is ISO-8859-1 for me) but original jquery.blockUI.js have UTF-8 encoding. So you should converte this file to the system default encoding (e.g. ISO-8859-1) or explicitly setting YUICompressor charset param to the UTF-8 for this file.