LynxCache - Free Lynx-based Mirroring Service

Anticipating a your site getting Slashdotted, or succuming to the Digg effect? Mirror your site for free with LynxCache.

Vitamin Features » Serving JavaScript Fast

mirror of http://www.thinkvitamin.com/features/webapps/serving-javascript-fast

Vitamin Features Serving JavaScript Fast

[1]

Home [2] Features [3] Training [4] Reviews [5] Interviews [6] Advisory board [7] Subscribe [8] About [9] Contact [10] Job Board [11]

FEATURES

Print Format [12] Email to a Friend [13]

WEB APPS [14] JAVASCRIPT [15] : SERVING JAVASCRIPT FAST [16]

Sunday, May 21st, 2006 Cal Henderson [17] Flickr [18] [19] [20]

The next generation of web apps make heavy use of JavaScript and CSS. We'll show you how to make those apps responsive and quick.

With our so-called "Web 2.0" applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we're delivering the optimum experience. In practice, this means a combination of making our content as small and fast to download as possible, while avoiding unnecessarily refetching unmodified resources.

This is complicated a little by the nature of CSS and JavaScript resources. In contrast to image assets, CSS and JavaScript source code is very likely to change many times as time goes by. When these resources change, we need our clients to download them all over again, invalidating the version in their local cache (and any versions stored in other caches along the way). In this article, we'll look at ways we can make the whole experience as fast as possible for our users - the initial page load, subsequent page loads and ongoing resource loading as the application evolves and content changes.

I believe strongly in making things as simple as possible for developers, so we'll also be looking at ways we can set up our systems to automatically take care of these optimization issues for us. With a little up front work, we can get the best of both worlds - an environment that makes development easy with great end-user performance - all without changing the way we work.

MONOLITH

The old school of thought was that we could achieve optimal performance by combining multiple CSS and JavaScript files into fewer, larger blocks. Rather than having ten 5k JavaScript files, we combine them into a single 50k file. While the total size of the code is still the same, we avoid having the overhead associated with multiple HTTP requests. Each request has a setup and teardown phase on both the client and server, incurs request and response header size overhead, and resource overhead on the server side in the form of more processes or threads (and perhaps more CPU time for on-the-fly gzipped content).

The parellization aspect is also important. By default, both Internet Explorer and Mozilla/Firefox will only download two resources [21] from a single domain at once when using persistent connections (as suggested in the HTTP 1.1 spec [22], section 8.1.4). This means that while we're waiting to download those JavaScript files, 2 at a time, we're not loading image assets - the page our users see during the loading phase will be missing its images.

However, there are a couple of downsides to this approach. By bundling all of our resources together, we force the user to download everything up front. By chunking content into multiple files we can spread out the cost of loading across several pages, amortizing the speed hit across a session (or avoiding some of the cost completely, depending on the path the user chooses). If we make the first page slow to speed up subsequent pages, we might find that we have more users who never wait around to request a second page.

The big downside to the single file approach has not often, historically, been considered. In an environment where we will have to often change our resources, _any_ changes to a single-file system will require the client to re-download a copy of the entire CSS or JavaScript working set. If our application has a single monolithic 100k JavaScript source file, any tiny change to our code will force all clients to suck down the 100k all over again.

A SPLINTERED APPROACH

The alternative approach lies somewhere in the middle - we split our CSS and JavaScript resources into multiple sub-files, while at the same time keeping that number functionally low. This compromise comes at a cost - we need to be able to develop applications with our code split out into logical chunks to increase development efficiency, while delivering merged files for performance. With a few additions to our build system (the set of tools which turn your development code into production code, ready for deployment), this needn't be a compromise we have to make.

For an application environment with distinct development and production environments, you can use a few simple techniques to keep your code manageable. In your development environment, code can be split into many logical components to make separation clear. In Smarty [23] (A PHP templating language) we can create a simple function to manage the loading of our JavaScript:

SMARTY: {insert_js files="foo.js,bar.js,baz.js"} PHP: function smarty_insert_js($args){ foreach (explode(',', $args['files']) as $file){ echo "n"; } } OUTPUT:

So far, so easy. But then we instruct our build process to merge certain files together into single resources. In our example, imagine we merged foo.js and bar.js into foobar.js, since they are nearly always loaded together. We can then record this fact in our application configuration and modify our template function to use this information.

SMARTY: {insert_js files="foo.js,bar.js,baz.js"} PHP: # map of where we can find .js source files after the build process # has merged as necessary $GLOBALS['config']['js_source_map'] = array( 'foo.js' => 'foobar.js', 'bar.js' => 'foobar.js', 'baz.js' => 'baz.js', ); function smarty_insert_js($args){ if ($GLOBALS['config']['is_dev_site']){ $files = explode(',', $args['files']); }else{ $files = array(); foreach (explode(',', $args['files']) as $file){ $files[$GLOBALS['config']['js_source_map'][$file]]++; } $files = array_keys($files); } foreach ($files as $file){ echo "n"; } } OUTPUT:

The source code in our templates doesn't need to change between development and production, but allows us to keep files separated while developing and merged in production. For bonus points, we can write our merging process in PHP and use the same configuration block to perform the merge process, allowing us to keep a single configuration file and avoid having to keep anything in sync. For super-bonus points, we could analyze the occurrence of scripts and style sheets together on pages we serve, to determine which files would be best to merge (files that nearly always appear together are good candidates for merging).

For CSS, a useful model to start from is that of a master and subsection relationship. A single master style sheet controls style across your entire application, while multiple sub-sheets control various distinct feature areas. In this way, most pages will load only two sheets, one of which is cached the first time any page is requested (the master sheet).

For small CSS and JavaScript resource sets, this approach _may_ be slower for the first request than a single large resource, but if you keep the number of components low then you'll probably find it's actually faster, since the data size per page is much lower. The painful loading costs are spread out around different application areas, so the number of parallel loads is kept to a minimum while also keeping the resources-per-page size low.

COMPRESSION

When talk about asset compression, most people think immediately of mod_gzip [24]. Beware, however - mod_gzip is actually _evil_, or at the least, a resource hogging nightmare. The idea behind it is simple - browsers request resources and send along a header to show what kind of content encodings they accept. It looks something like this:

Accept-Encoding: gzip,deflate

When a server encounters this header, it can then gzip or deflate (compress) the content it's sending to the client, where the client will then decompress it. This burns CPU time on both the client and server, while reducing the amount of data transferred. All well and good. The way mod_gzip works, however, is to create a temporary file on disk in which to compress the source data, serve that file out, then delete it. For high volume systems, you very quickly become bound by disk IO. We can avoid this by using mod_deflate [25] instead (Apache 2 only), which does all the compression in memory - sensible. For Apache 1 users, you can instead create a RAM disk and have mod_gzip writes its temporary files there - not quite as fast as pure in-memory compression, but not nearly as slow as writing to disk.

Even so, we can avoid the compression overhead completely by pre-compressing the relevant static resources and using mod_gzip to serve people the compressed version where appropriate. If we add this compression into our build process, it all happens transparently to us. The number of files that need compressing is typically quite low - we don't compress images since we don't gain much, if any, size benefit (since they're already compressed) so we only need to compress our JavaScript and CSS (and any other uncompressed static content). Configuration options tell mod_gzip where to look for pre-compressed files.

mod_gzip_can_negotiate Yes mod_gzip_static_suffix .gz AddEncoding gzip .gz

Newer versions of mod_gzip (starting with version 1.3.26.1a) can pre-compress files for you automatically by adding a single extra configuration option. You'll need to make sure that Apache has the correct permissions to create and overwrite the gzipped files for this to work.

mod_gzip_update_static Yes

However, it's not that simple. Certain versions of Netscape 4 (specifically 4.06 to 4.08) identify themselves as being able to interpret gzipped content (they send a header saying they do), but they cannot correctly decompress it. Most other versions of Netscape 4 have issues with loading compressed JavaScript and CSS in different and exciting ways. We need to detect these agents on the server side and make sure they get served an uncompressed version. This is fairly easy to work around, but Internet Explorer (versions 4 through 6) has some more interesting issues. When loading gzipped JavaScript, Internet Explorer will sometimes incorrectly decompress [26] the resource, or halt compression halfway through, presenting half a file to the client. If you rely on your JavaScript working, you need to avoid sending gzipped content to Internet Explorer. In the cases where Internet Explorer _does_ receive gzipped JavaScript correctly, some older 5.x versions won't cache the file, regardless of it's e-tag headers.

Since gzip compression of content is so problematic, we can instead turn our attention to compressing content without changing its format. There are many JavaScript compression scripts available, most of which use a regular expression driven rule set to reduce the size of JavaScript source. There are several things which can be done to make the source smaller - removing comments, collapsing whitespace, shortening privately scoped variable names and removing optional syntax.

Unfortunately, most of these scripts either obtain a fairly low compression rate, or are destructive under certain circumstances (or both). Without understanding the full parse tree, it's difficult for a compressor to distinguish between a comment and what looks like a comment inside a quoted string. Adding closures to the mix, it's not easy to find which variables have a private lexical scope using regular expressions, so some variable name shortening techniques will break certain kinds of closure code.

One compressor does avoid this fate - the Dojo Compressor [27] (there's a ready-to-use version here [28]) works by using Rhino (Mozilla's JavaScript engine implemented in Java) to build a parse tree, which it then reduces before serializing it to a file. The Dojo Compressor can give pretty good savings for a low cost - a single compression at build time. By building this compression into our build process, it all happens transparently for us. We can add as much whitespace and as many comments as we like to our JavaScript in our development environment, without worrying about bloating our production code.

Compared to JavaScript, CSS is relatively simple to compress. Because of a general lack of quoted strings (typically paths and font names) we can mangle the whitespace using regular expressions. In the cases where we _do_ have quoted strings, we can nearly always collapse a whitespace sequence into a single space (since we don't tend to find multiple spaces or tabs in URL paths or font names). A simple Perl script should be all we need:

#!/usr/bin/perl my $data = ''; open F, $ARGV[0] or die "Can't open source file: $!"; $data .= $_ while ; close F; $data =~ s!/*(.*?)*/!!g; # remove comments $data =~ s!s+! !g; # collapse space $data =~ s!} !}n!g; # add line breaks $data =~ s!n$!!; # remove last break $data =~ s! { ! {!g; # trim inside brackets $data =~ s!; }!}!g; # trim inside brackets print $data;

We can then feed individual CSS files through the script to compress them like so:

perl compress.pl site.source.css > site.compress.css

With these simple plaintext optimizations we can reduce the amount of data sent over the wire by as much as 50% (depending upon your coding style - it might be much less), which can translate to a much faster experience for our users. But what we'd really like to do is avoid users having to even request files unless completely necessary - and that's where an intimate knowledge of HTTP caching comes in handy.

CACHING IS YOUR FRIEND

When a user agent requests a resource from a server for the first time, it caches the response to avoid making the same request in the future. How long it stores this response for is influenced by two factors - the agent configuration and any cache control response headers from the server. All browsers have subtly different configuration options and behaviors, but most will cache a given resource for at least the length of a session, unless explicitly told otherwise.

It's quite likely you already send out anti-caching headers for dynamic content pages to avoid the browser caching pages which constantly change. In PHP, you can achieve this with a pair of function calls:

Sounds too easy? It is - some agents will ignore this header under certain circumstances. To _really_ convince a browser not to cache a document, you'll need to be a little more forceful:

This is fine for content we don't want to be cached, but for content that doesn't change with every request we want to encourage the browser to cache it aggressively. The "If-Modified-Since" request header allows us to get part of the way there. If a client sends an "If-Modified-Since" header with its request, Apache (or your web server of choice) can respond with status code 304 ("Not Modified"), telling the browser that its cached copy of the file is already up to date. With this mechanism, we can avoid sending the contents of a file to the browser, but we still incur the overhead of an HTTP request. Hmmm.

Similar to the if-modified-since mechanism are entity tags. Under Apache, each response for a static resource is given an "ETag" header containing a checksum generated from the file's modified-time, size and inode number. A browser can then perform a HEAD request to check the e-tag for a resource without downloading it. E-tags suffer from the same problem as the if-modified-since mechanism - the client still needs to perform an HTTP request to determine the validity of the locally cached copy.

In addition, you need to be careful with if-modified-since and e-tags if you serve content from multiple servers. With two load-balanced web servers, a single resource could be requested from either server by a single agent - and could be requested from each at different times. This is great - it's why we load balance. However, if the two servers generate different e-tags or modified dates for the same files, then browsers won't be able to properly cache content. By default, e-tags are generated using the inode number of the file, which will vary from server to server. You can turn this off using a single Apache configuration option:

FileETag MTime Size

With this option, Apache will use only the modification time and file size to determine the e-tag. This, unfortunately, leads us to the other problem with e-tags, which can affect if-modified-since too (though not nearly as badly). Since the e-tag relies on the modified time of the file, we need those times to be in sync. If we're pushing files to multiple web servers, there's always a chance that the time at which the files are pushed are subtly different by a second or two. In this case, the e-tags generated by two servers will still be different. We could change the configuration to generate e-tags only from the file size, but this means that we'll generate the same e-tag if we change a file's contents without changing its size. Not ideal.

CACHING IS YOUR _BEST_ FRIEND

The problem here is that we are approaching the issue from the wrong direction. These possible caching strategies all revolve around the client asking the server if its cached copy is fresh. If we could notify the client when we change a file, it would know that its own cached copy was fresh, until we told it otherwise. But the web doesn't work that way - the client makes requests to the server.

But that's not quite true - before fetching any JavaScript or CSS files, the client makes a request to the server for the page which will be loading those files via or tags. We can use the response from the server to notify the client of any changes in those resources. This is all a little cryptic, so let's spell it out - if we change the filenames of JavaScript and CSS files when we change their contents, we can tell the client to cache every URL forever, since the content of any given URL will never change.

If we are sure that a given resource will never change, then we can send out some seriously aggressive caching headers. In PHP, we just need a couple of lines:

Here we tell the browser that the content will expire in 10 years (there are 315,360,000 seconds in 10 years, more or less) and that it can keep it around for 10 years. Of course, we're probably not serving our JavaScript and CSS via PHP - we'll address that in a few moments.

MISTAKES ABOUND

Manually changing the filenames of resources when the contents are modified is a dangerous task. What happens if you rename the file, but not the templates pointing to it? What happens if you change some templates but not others? What happens if you change the templates but don't rename the file? Most likely of all, what happens if you modify a resource but forget to rename it or change any references to it. In the best of these cases, users will not see the new content and be stuck with the old versions. In the worst case, no valid resource is found and your site stops working. This sounds like a dumb idea.

Luckily computers are really good at this sort of thing - dull repetitive tasks which need to be done exactly right, over and over again, when some kind of change occurs.

The first step in making this process as painless as possible is to realize that we don't need to rename files at all. URLs we serve content from and where the content is located on disk don't have to have anything to do with each other. Using Apache's mod_rewrite [29] we can create a simple rule to redirect certain URLs to certain files.

RewriteEngine on RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$ /$1$2 [L]

This rule matches any URL with one of the specified extensions which also contains a 'version' nugget. The rule then rewrites these URLs to a path without the version nugget. Some examples:

URL Path /images/foo.v2.gif -> /images/foo.gif /css/main.v1.27.css -> /css/main.css /javascript/md5.v6.js -> /javascript/md5.js

With this rule in-place, we can change the URL (by changing the version number) without changing where the file lives on disk. Because the URL has changed, the browser treats it as a different resource. For bonus points, you can combine this with the script grouping function from earlier to produce a list of versioned tags as needed.

At this point, you might ask why we don't just add a query string to the end of the resource - /css/main.css?v=4. According the letter of the HTTP caching specification, user agents should _never_ cache URLs with query strings. While Internet Explorer and Firefox ignore this, Opera and Safari don't - to make sure all user agents can cache your resources, we need to keep query strings out of their URLs.

Now that we can change our URLs without moving the file, it would be nice to be able to have the URLs updated automatically. In a small production environment (or a development environment, for people with large production environments), we can do this really easily using a template function. This example is for Smarty, but applies equally well to other templating engines.

SMARTY: PHP: function smarty_version($args){ $stat = stat($GLOBALS['config']['site_root'].$args['src']); $version = $stat['mtime']; echo preg_replace('!.([a-z]+?)$!', ".v$version.$1", $args['src']); } OUTPUT:

For each linked resource, we determine the file's location on disk, check its mtime (the date and time the file was last modified on disk) and insert that into the URL as the version number. This works great for low traffic sites (where stat operations are cheap) and for development environments, but it doesn't scale well to high volume deployments - each call to stat requires a disk read.

The solution is fairly simple. In a large system we already have a version number for each resource, in the form of the source control revision number (you're already using source control, right?). At the point when we go to build our site for deployment, we simply check the revision numbers of all of our resource files and write them to a static configuration file.

We can then modify our templating function to use these version numbers when we're operating in production.

In this way, we don't need to rename any files, or even remember when we modify resources - the URL will be automatically changed everywhere whenever we push out a new revision - lovely. We're almost where we want to be.

BRINGING IT ALL TOGETHER

When we talked about sending very-long-period cache headers with our static resources earlier, we noted that since this content isn't usually served through PHP, we can't easily add the cache headers. We have a couple of obvious choices for dealing with this; inserting PHP into the process or letting Apache do the work.

Getting PHP to do our work for us is fairly simple. All we need to do is change the rewrite rule for the static files to be routed through a PHP script, then have the PHP script output headers before outputting the content of the requested resource.

Apache: RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$ /redir.php?path=$1$2 [L] PHP: header("Expires: ".gmdate("D, d M Y H:i:s", time()+315360000)." GMT"); header("Cache-Control: max-age=315360000"); # ignore paths with a '..' if (preg_match('!..!', $_GET[path])){ go_404(); } # make sure our path starts with a known directory if (!preg_match('!^(javascript|css|images)!', $_GET[path])){ go_404(); } # does the file exist? if (!file_exists($_GET[path])){ go_404(); } # output a mediatype header $ext = array_pop(explode('.', $_GET[path])); switch ($ext){ case 'css': header("Content-type: text/css"); break; case 'js' : header("Content-type: text/javascript"); break; case 'gif': header("Content-type: image/gif"); break; case 'jpg': header("Content-type: image/jpeg"); break; case 'png': header("Content-type: image/png"); break; default: header("Content-type: text/plain"); } # echo the file's contents echo implode('', file($_GET[path])); function go_404(){ header("HTTP/1.0 404 File not found"); exit; }

While this works, it's not a great solution. PHP demands more memory and execution time than if we did everything in Apache. In addition, we have to be careful to protect against exploits made possible by sending us doctored values for the path query parameter. To avoid all this headache, we can have Apache add the headers directly. The RewriteRule directive allows us to set environment variables when a rule is matched, while the Header directive lets us add headers only when a given environment variable is set. Combining these two directives, we can easily chain the rewrite rule together with the header settings.

RewriteEngine on RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$ /$1$2 [L,E=VERSIONED_FILE:1] Header add "Expires" "Mon, 28 Jul 2014 23:30:00 GMT" env=VERSIONED_FILE Header add "Cache-Control" "max-age=315360000" env=VERSIONED_FILE

Because of Apache's order of execution, we need to add the RewriteRule line to the main configuration file (httpd.conf) and not a per-directory (.htaccess) configuration file, otherwise the Header lines get run first, before the environment variable gets set. The Header lines can either go in the main configuration file or in an .htaccess file - it makes no difference.

SKINNING RABBITS

By combining the above techniques, we can build a flexible development environment and a fast and performant production environment. Of course, this is far from the last word on speed. There are further techniques we could look at (separate serving of static content, multiple domain names for increased concurrency) and different ways of approaching the ones we've talked about (building an Apache filter to modify outgoing URLs in HTML source to add versioning information on the fly). Tell us about techniques and approaches that have worked well for you by leaving a comment.

Cal's new book, Building Scalable Web Sites [30], contains more tips and tricks to help you develop and manage the next generation of web applications.

Like this article? Digg it [31]!

Fuel [32] is a brand new affordable conference about powering your business with the web: London June 13

Join Jina Bolton, Jon Hicks, Larissa Meek, Daniel Burka and more for Future of Web Design [33], London April 17-18

Leave a response [34], or trackback [35] from your own site.

196 RESPONSES TO "SERVING JAVASCRIPT FAST"

alexander says (on May 22nd, 2006 at 7:54 am [36] )

I have noticed several sites using something like:

What technique is recommended for adding the unix timestamp, is it server-side reading the modification date of the file or something else? meneame.net says (on May 22nd, 2006 at 8:05 am [37] )

CóMO SERVIR JAVASCRIPT RáPIDO

Las nuevas aplicaciones web 2.0 tienden a incrementar el uso de Javascript considerablemente. Uno de los ingenieros principales que desarrollan Flickr presenta en este artículo las claves para servir Javascript rápido. Carnet Web de Pascal says (on May 22nd, 2006 at 10:08 am [38] )

SERVIR DU JAVASCRIPT RAPIDEMENT

Un intéressant article de Cal Henderson (développeur de Flickr) :

Serving JavaScript Fast

Il y explique tout un tas de techniques côté serveur pour améliorer la rapidité d'un site dépendant beaucoup de javascript/css, rien de révolutionnaire Robertas Aganauskas says (on May 22nd, 2006 at 11:21 am [39] )

Thanks Cal, that really helped a lot (especially the idea of elegant versioning files). Max says (on May 22nd, 2006 at 1:10 pm [40] )

While these are great recommendations for an Apache/PHP environment, I would've liked to see more cross platform solutions, to include .NET/IIS as well. Great nevertheless. Stefan Hayden says (on May 22nd, 2006 at 1:29 pm [41] )

According the letter of the HTTP caching specification, user agents should never cache URLs with query strings. While Internet Explorer and Firefox ignore this, Opera and Safari don’t

I'm confused about which browsers you say DO cache files with query strings? All the tests I've done show that in both IE and Firefox adding a query string does infact force the browser to re-download the css file. And that if the current query string matches the previous query string then the file stays cached.

you are saying it's Opera and Safari this does not work on? Cal Henderson says (on May 22nd, 2006 at 4:33 pm [42] )

Stefan:

No browser will treat URLs with different query strings as the same. "foo.css?v=1 and "foo.css?v=2 will always cause a refetch.

However, Safari and Opera (most, if not all versions) will not cache any URL with a query string - it will treat two URLs with the _same_ query string as if they were _different_ URLs. If a first page requests "foo.css?v=1 and a second page _also_ requests "foo.css?v=1, the first request will not be cached. Mandy Singh says (on May 22nd, 2006 at 7:00 pm [43] )

Nice Article!

Just want to know more about the gzip related problem. Is it seen mostly with .js & .css file on Internet Explorer?

Would it be any better if gzip for .js & .css files was turned off (I know half the code is going to be in them) but at least the html pages would be served faster (never mind the includes which are anyway going to be cached).

Let me know your thoughts.

Thanks, Mandy. Cal Henderson says (on May 22nd, 2006 at 8:49 pm [44] )

Mandy:

It's _mostly_ seen with JavaScript and CSS, but can happen with HTML too. In this case it's a lot more obvious to the user that something's gone wrong so there's a greater change they'll reload and clear the issue themselves. With JavaScript, if a bad version of a file gets cached then the user may never realize it's not working as expected. Pig Pen - Web Standards Compliant Web Design Blog Blog Archive Serving JavaScript Fast says (on May 22nd, 2006 at 9:22 pm [45] )

[] Serving JavaScript Fast by Cal Henderson over on Vitamin. [] Keith says (on May 22nd, 2006 at 10:47 pm [46] )

Thank you Cal, Great Article. AJAX道 web应用加速技巧 says (on May 23rd, 2006 at 12:46 am [47] )

[] http://www.thinkvitamin.com/features/webapps/serving-javascript-fast 归类于： Javascript, 编程例子, PHP, Perl by -- gaoliang @ 8:45 am [] Nicolai says (on May 23rd, 2006 at 3:48 am [48] )

- Why not use mod_negotiation for compression? That will allow you to use pre-compressed files and avoid per-request on-the-fly compression. You can automate this and use the most expensive/effective compression level (gzip -9 foo).

- Separate static content, dynamic content, and database onto separate machines.

- Serve static content from a machine with lots of RAM to allow OS- or server-level file caching. Check out a built-for-speed server like lighttpd.

- Compile a stripped-down version of your Web server using only the modules you need.

- For frequently-called, brain-dead utility work like "Bringing it all together", avoid PHP like the plague. Write it in C using FastCGI.

- For big PHP, use the Zend Optimizer. Check out the PHP5 function benchmarks at http://byster.net/?page_id=48

- All the usual stuff: don't follow symlinks, don't use .htaccess files, don't resolve client hostnames (move the logs to another machine and batch it later), etc..

- If you use SSL, buy a dedicated hardware SSL device that offloads crypto processing from the servers to it.

- When linking to directories, add the trailing slash. i.e. http://foo.com/bar/ rather than http://foo.com/bar. This avoids an HTTP redirect, which results in an unncessary HTTP transaction. Emergent Properties Blog Archive How To Speed Up Your Web App says (on May 23rd, 2006 at 4:41 am [49] )

[] read more | digg story [] Bizarro says (on May 23rd, 2006 at 6:28 am [50] )

No offense dude I know that you work on Flickr but this is the most convoluted, bizarre set of recommendations I have ever seen. Is this really workable in real life? It seems like a complicated process that could break at any stage. Thanks for sharing the insanity though Andy Kant says (on May 23rd, 2006 at 6:46 am [51] )

Great article, I will make sure to use your recommendations in the future. Thanks! ajaxDNA says (on May 23rd, 2006 at 7:09 am [52] )

[] A very good article from Cal Henderson of Flickr/Yahoo. [] Sam Minnee says (on May 23rd, 2006 at 7:16 am [53] )

Bizarro: The recommendations look like they're very tightly bound to the author's development workflow. So, while it may be byzantine to have to manually keep this stuff in check, if it fits into the way you normally go about building applications, then it's all good.

On the flip side, the author's done a good job of explaining his motivations for all the pieces, so we can all go and make use of the same ideas in our own applications, though maybe in different ways. All Dugg Blog Archive How To Speed Up Your Web App says (on May 23rd, 2006 at 9:12 am [54] )

[] The next generation of web apps make heavy use of JavaScript and CSS. This article, by the lead developer of Flickr, shows you how to make those apps responsive and quick.read more | digg story Share and Enjoy:These icons link to social bookmarking sites where readers can share and discover new web pages. [] [[ the sirens of titan ]] Blog Archive links for 2006-05-23 says (on May 23rd, 2006 at 9:21 am [55] )

[] Vitamin Features » Serving JavaScript Fast js/css caching tips (tags: javascript optimization development webdev web2.0 ajax) Posted by geekfreak Filed in Bookmarks [] Kim says (on May 23rd, 2006 at 10:04 am [56] )

Using mod_rewrite to hide version numbers is a bad thing. mod_rewrite requires a lot of ressources.

A better solution is using symlinks on the web server. Richard says (on May 23rd, 2006 at 11:39 am [57] )

I was looking at a simliar article yesterday, I guess this guy has copied a lot of content from this page.

http://vivekjishtu.blogspot.com/2006/04/speed-up-your-ajax-based-webapps.html 26 Miles Serving JavaScript Fast says (on May 23rd, 2006 at 12:52 pm [58] )

[] Read more at Thinkvitamin.com. Posted by Matt | May 23, 2006 | [] Cal Henderson says (on May 23rd, 2006 at 3:42 pm [59] )

Nicolai: Why use mod_negotiation (which gets measureably slower with Multiviews turned on) when the latest version of mod_gzip does exactly that, but with the added ability to freshen pre-compressed files as needed? As for recommending writing the braindead part in C, if you read that section carefully I suggest you use mod_rewrite and mod_headers which are already written in C and well optimized. The rest of your recommendations, while great, are all server performance, which we're not talking about here and is already very well covered elsewhere.

Bizarro: Can you suggest any part of it that would actually break? It's neither fragile nor complicated - we change URLs pointing to files. By buiilding it in at the template level it doesn't require us to think about it at all once it's been set up. It's working every day for hundreds of millions of pages - not just Flickr, but many large applications. Akamai's Edge Platform uses a very similar technique.

Kim: It's a myth that mod_rewrite is a really poor performer, but that's besides the point - there's a much greater issue with SymLinks. Ignoring the case of a development environment, imagine you had 10 JavaScript files and each of them (after some time) had been through one hundred revisions. Each time we deploy the app, we need to create 1000 symlinks. Depending on our the file system we're using, seek time can increase linearly with the number of files per directory. The time to create all of these symlinks is non-trivial too. We could rely on only symlinking the current revision and letting a 404 handler clean up the rest, but that seems messy to me, compared to the mod_rewrite option.

Alan says (on May 23rd, 2006 at 4:03 pm [60] )

Very excellent article, particularly the attention paid to the REALITIES of mod_gzip/mod_deflate and browser caching behavior when attaching query strings. Recently I've been fighting both battles at work, and it'll be nice to have a clear, well written resource to point to. Nicolai says (on May 23rd, 2006 at 4:29 pm [61] )

Sorry, but isn't this whole thing about server performance?

_Why use mod_negotiation (which gets measureably slower with Multiviews turned on) when the latest version of mod_gzip does exactly that, but with the added ability to freshen pre-compressed files as needed?_

Sorry, I guess I didn't read that carefully enough. That's excellent news! Since there's no need to use MultiViews in mod_negotiation to accomplish this, though, it may still be faster. Auto-freshen is a really nice feature from a developer standpoint, but it may slow it down a bit by doing a stat-and-compare for each request. I don't know for sure, though, I'll look at the source and see exactly what it's doing.

_As for recommending writing the braindead part in C, if you read that section carefully I suggest you use mod_rewrite and mod_headers which are already written in C and well optimized._

I agree, that's the way to go! But if you can't run Apache or an httpd that can do this natively in-process, C+FastCGI is a much more efficient alternative than PHP.

_The Header lines can either go in the main configuration file or in an .htaccess file - it makes no difference._

While this is true in terms of the resulting functionality, it's misleading in terms of performance. Using .htaccess files requires Apache to look for, open, read, close, parse, and compile it for every single request, which results in a significant speed penalty on a loaded server. (As you said, "This works great for low traffic sites (where stat operations are cheap) and for development environments, but it doesn’t scale well to high volume deployments - each call to stat requires a disk read."-then add a bunch of work on top of that.)

_It’s a myth that mod_rewrite is a really poor performer_

It doesn't _have_ to be, but it certainly can. Any use of regular expressions is going to cause a bit of a hit-pattern matching is not exactly a speedy operation. But you can make it much, much worse by writing your regular expressions badly. Use greedy operators (*) or specify more of the pattern than you need to and watch the performance plummet! Cal Henderson says (on May 23rd, 2006 at 4:40 pm [62] )

Nicolai: All excellent points. When I said this wasn't about server performance, I meant that we're looking at client performance as the main goal. Adding 10ms on the server isn't noticeable, but avoiding entire requests makes a big impact for the client. Unless you're serving A LOT of pages, server optimization can be a secondary activity. Even for small traffic applications, making some of the changes I talked about will make the user experience faster.

But yes, avoiding .htaccess files can give you a good speed boost, especially for deeply-nested webroots. Slashe says (on May 23rd, 2006 at 11:11 pm [63] )

Great article, I've just implemented some of the ideas and I'm already seeing a difference.

Note: I think your perl script is missing some slashes. jon says (on May 24th, 2006 at 12:30 am [64] )

how about setting a cache expiry date for the file, and then on that date sending/updating the file (most probably the .js) ?

if this is workable.wouldn't this be the ideal solution ? links for 2006-05-24 at leron's crib says (on May 24th, 2006 at 3:18 am [65] )

[] Vitamin Features » Serving JavaScript Fast via digg, tips for js & css performance optimization (tags: ajax compression css development performance php programming web2.0) [] Who Cares? How To Speed Up Your Web App says (on May 26th, 2006 at 1:57 am [82] )

[] The next generation of web apps make heavy use of JavaScript and CSS. This article, by the lead developer of Flickr, shows you how to make those apps responsive and quick. Link [] 94smart's Blog 接着讲Flickr的八卦 says (on May 26th, 2006 at 9:15 am [83] )

[] 前两天，顺着DBA的Flickr 的开发者的 Web 应用优化技巧，找到了Cal Henderson写的Serving JavaScript Fast，又让我窥探到Flickr的些些技术内幕，Flickr用的模板居然是SMARTY，这个之前普遍被人诟病的模板系统。 [] penk - Keep on rockin' in the free world Blog Archive 本日書籤 says (on May 30th, 2006 at 8:29 pm [90] )

[] http://www.thinkvitamin.com/features/webapps/serving-javascript-fast [] COLD CASE Blog Archive How To Speed Up Your Web App says (on June 14th, 2006 at 2:58 pm [116] )

[] read more | digg story Explore posts in the same categories: coldcase [] Yoann Boukredine says (on June 16th, 2006 at 10:12 am [117] )

Have a look at that great utilities :

http://hometown.aol.de/_ht_a/memtronic/

http://www.syntropy.se/?ct=downloads&target=jcefree

http://svn.dojotoolkit.org/dojo/trunk/buildscripts/lib/

Real solution : obfuscate and pre-compression (small name in variable etc) :)

Yoann! dandyna says (on June 18th, 2006 at 2:45 pm [118] )

This is a real useful article to me who's beginning with javascript selva says (on June 24th, 2006 at 11:36 am [119] )

hai henderson, Your tips for the java script loading is very good. but i need code exaple for jsp based examples to merge files,etc.. You have descriped everything in PHP.. Try to give in JSP also.. I realy need that.. Danny says (on July 18th, 2006 at 5:57 pm [120] )

Email to a Friend [121] is not validating webmasternewsblog.com Blog Archive says (on July 25th, 2006 at 6:19 pm [122] )

[] Speed up your JavascriptVitamin has a good article about how to optimize your Javascript for speed and efficiency, as old techniques won't fly in a "Web 2.0 world. Cal Henderson's best practices include compression, caching, and letting PHP or other server-side technologies do some of the work. [] De cómo comprimir ficheros javascript - Scriptia says (on July 30th, 2006 at 7:31 pm [123] )

[] En Serving JavaScript Fast, Carl Herlson, de Flickr, habla largo y tendido sobre diferentes métodos para acelerar la carga de scripts y hojas de estilo (combinación, compresión, cacheado). [] Suave's Blog Web Cache Tutorial says (on August 9th, 2006 at 9:40 am [124] )

[] Serving Javascript Fast [] auto says (on October 22nd, 2006 at 3:07 pm [134] )

Appreciate it men! Ed Eliot says (on October 31st, 2006 at 1:45 pm [135] )

An interesting read. I'd been thinking about the possibility of using mod rewrite to version JS and CSS files so it's great to see that thinking re-enforced here.

If memory serves me right IE actually only downloads one file at a time when it's downloading JavaScript (two for everything else) - not sure about other browsers. The adverse effect of multiple JavaScript file downloads is therefore amplified so unless your JS is really huge it I think it may still be beneficial to download one file containing everything. Martin says (on November 8th, 2006 at 11:14 pm [136] )

RFC 2616 (HTTP 1.1) says:

To mark a response as "never expires", an origin server sends an Expires date approximately one year from the time the response is sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one year in the future.

And by the way, with mod_expires it’s very easy to set the expiration time. Ferienhaus Ostsee says (on November 24th, 2006 at 10:31 am [137] )

If memory serves me right IE actually only downloads one file at a time when it’s downloading JavaScript (two for everything else) - not sure about other browsers. The adverse effect of multiple JavaScript file downloads is therefore amplified so unless your JS is really huge it I think it may still be beneficial to download one file containing everything. Ignacio says (on November 24th, 2006 at 3:43 pm [138] )

Me resulta muy interesante este articulo, pero mi ingles no llego a comprender muy bien y tengo una mezcla de que es lo que hay que hacer para poder hacer funcionar esto, nadie conoce un articulo traducido de esto? o algun ejemplo andando sobre esto? Paul Loy says (on November 29th, 2006 at 9:00 pm [139] )

What Ferienhaus says is irrelevant. If IE does only download one javascript file at a time it's will take virtually the same time to download several smaller javascripts sequentially as it would to download one big huge one.

The benefits of lots of small files is that each page might not actually need ALL the javascript so you can optimise the amount of javascript you send to the browser. XUMADESIGNS Blog Archive Think Vitamin says (on January 1st, 2007 at 7:44 am [140] )

[] Thinkvitamin , bu siteye nerden rastladığım hakkında pek fikrim yok ancak oldukça hoş makaleler bulunmakta.Yazar kadrosu oldukça geniş ve profesyonel.Aslında o kadar çok güzel bölümü varki burda yazmak yerine sizlerin gezmesini tavsiye ederim.Bu yazıyı yazarken Cal Henderson'un Serving JavaScript Fast bu yazısını okumaktaydım ve oldukça güzel bir yazı olduğunu söyleyebilirim. [] mainzBlog says (on January 8th, 2007 at 10:26 pm [141] )

AJAX PATTERNS: ON-DEMAND JAVASCRIPT

Heute hatte ich das Problem: Wie bring ich Javascript-Code möglichst effizient in einer Ajax-Anwendung - das heißt viele verschiedene Javascript-Dateien - zur Ausführung im Browser, ohne dass dieser den Hitzetod durch Überarbeitung stirbt. (Vom gl Joseph Smarr Fixing eval() to use global scope in IE says (on January 31st, 2007 at 5:23 pm [142] )

[] When we built the first version of the new site, we combined all the JavaScript into one giant file as part of our deployment process. The total codebase was huge and it had the predictable effect that initial page-load time was terrible because the user's CPU was solidly spiked for several seconds while the poor browser choked through the massive amount of code it had to parse. So we started loading a lot of our code on-demand (packaging it into several logical chunks of related files and using dojo's package/loader system to pull in the code as needed). [] Speed Up Your Javascript Load Time | BetterExplained says (on February 22nd, 2007 at 6:33 am [143] )

[] Think vitamin: Serving Javascript Fast Posted February 21, 2007, under General Tags: browser, cache, compress, javascript, load, performance, programming, speed, Web Related Posts: [] Faster Page Loading - Professional PHP says (on February 28th, 2007 at 3:58 am [144] )

[] Cal Henderson of Flickr also takes up this topic in serving JavaScript fast. He goes into more depth about the issue of à la carte external objects versus aggregated external objects as well as covering compression and caching issues with examples in PHP. [] dobby says (on March 15th, 2007 at 9:00 pm [145] )

it is a top site Bodo says (on March 16th, 2007 at 12:41 pm [146] )

The comments are very helpful.Each one hat it's own experience by optimizing homepages. It’ a neverending story to optimize websites. Bob from Germany weser Nienburg Chad's dailies Blog Archive links for 2007-03-25 says (on March 25th, 2007 at 11:29 pm [147] )

[] Serving JavaScript Fast (tags: javascript articles webdev) [] Archive Faster Page Loads With Image Concatenation says (on May 8th, 2007 at 2:19 pm [157] )

[] Here's an interesting idea. In the same way that you can automatically compile multiple CSS and JavaScript files for a speed boost you can concatenate images. The file and HTTP request savings are obvious, but this method introduces a few CSS questions. Not impossible to overcome, but you need to be careful. [] vândpupăză Blog Archive Succesul este cel mai mare inamic al lui GetLoaded.ro1 says (on May 21st, 2007 at 8:45 pm [158] )

[] Serving JavaScript Fast [] vândpupăză Blog Archive Succesul este cel mai mare inamic al lui GetLoaded.ro says (on May 21st, 2007 at 8:45 pm [159] )

[] Serving JavaScript Fast [] Vitamin Features » Serving JavaScript Fast says (on June 10th, 2007 at 4:16 pm [160] )

[] http://www.thinkvitamin.com/features/webapps/serving-javascript-fast "The next generation of web apps make heavy use of JavaScript and CSS. We’ll show you how to make those apps responsive and quick."Tags: javascript, optimization, ajax, css, performance, development, apache, architecture(del.icio.us history) [] Chirru Blog Archive get subversion revision-number in a django-project says (on June 16th, 2007 at 9:17 am [161] )

[] Udi wrote: > Mind if I ask why? > i've got the idea from here: http://www.thinkvitamin.com/features/webapps/serving-javascript-fast the idea is that you serve all your media files (js/css/png/jpg) at urls that contain for example the svn version number, like: /media/1432/js/form.js and you setup your webserver to send such headers along the file which says that the file does not need to be re-fetched for a very long time (let's say a year). and when you release a new version of your program, then, because the version-number has changed, all your media-urls change. but it's much better described in the article. btw. for the reference, i abandoned the "svnversion" approach. it produces some funny version numbers when you use svn:externals, so i went with "svn info" and extracting the version-number from there. and, for now i decided that i will not do it at the startup, but will do it at release-time (so i will extract the version-number at the release-time, save it to let's say revision.py, and import it into settings.py). gabor [] Thomas Erichson says (on June 21st, 2007 at 6:34 pm [162] )

Hi there, nice website, well done, keep on doing so , with kind regards from germany Andy says (on June 21st, 2007 at 9:26 pm [163] )

We've found that body onload is essential in IE for dealing with what we suspect are gzip issues for javascript, also, not forgetting that any DOM code must wait for DOM/page rendering to complete anyway.Onload certainly fixed issues we were having in this area.

The browser processes javascript and css synchronously, hence, this is probably the rational behind a synchronous approach to loading resources of these types.

Also, I'm thinking what is the performance of gzip when comparing CPU time, file size and http savings. Below a certain threshold gzip isn't worthwhile - and I believe is configurable to prevent on-the-fly compression below a certain file size. Above that however, I believe gzip becomes very efficient. I'll head off now and try to find some research on this.

This has prompted me to run up some stats on our platform to compare page views with hits on our assets boxes - in other words, are we getting a good saving from GET caching.

Useful stuff, thanks. Carol Ballontiere says (on July 10th, 2007 at 8:07 am [164] )

Hello, super website here, well done, and thank you for all the useful links and hints. Kilburn says (on July 11th, 2007 at 9:47 am [165] )

RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$ /$1$2 [L]

I think there's a little error in this regexp. The highlighted dot should have a backslash to escape it, otherways you're matching any character here, meaning that ie. this would be a non-desired match:

http://domain.tld/css/main.view.css Serving JavaScript Fast [REF] says (on July 13th, 2007 at 3:19 pm [166] )

[] Link [] Tims Blog Blog Archive Cant See Anything in URL Bar says (on July 13th, 2007 at 6:31 pm [167] )

[] serving javascript fast [] All in a days work says (on July 23rd, 2007 at 2:23 am [168] )

[] Serving JavaScript Fast - A splintered approach The alternative approach lies somewhere in the middle - we split our CSS and JavaScript resources into multiple sub-files, while at the same time keeping that number functionally low. [] beer2beer Article minify: Sirviendo javascript says (on July 23rd, 2007 at 8:00 pm [169] )

[] Antes de pasarme a minify, usaba una solución homebrew inspirada en el artículo de recomendada lectura "Serving Javascript fast" de Cal Henderson, desarrollador de flickr. Básicamente usaba ShrinkSafe para comprimir Javascript estáticamente (es decir, no bajo demanda), y PHP muy al estilo de minify pero más simple, para servir via mod_rewrite todo el contenido .js y .css de la aplicación, añadiéndole cache vía servidor. E iba bien, pero lo de combinar los archivos en uno me llamó la atención y la verdad es que minify va mejor! [] Patrick Whittingham says (on July 25th, 2007 at 7:17 pm [170] )

What about IIS 5,6,7 or does one do it on the application server level (ie., .Net / Java / cfmx) ? Rainbow in the AJAX Toilet says (on July 26th, 2007 at 12:00 pm [171] )

[] http://www.thinkvitamin.com/features/webapps/serving-javascript-fast [] 万用表 says (on July 28th, 2007 at 1:06 pm [172] )

nice 万用表 says (on July 28th, 2007 at 1:08 pm [173] )

http://www.yiqiei.cn Client side web site performance — onenaught.com says (on August 6th, 2007 at 8:04 am [174] )

[] Serving JavaScript fast from Vitamin. (Lots of good stuff here. See also the note about Apache's mod_gzip and mod_deflate. In short, use mod_deflate if you can.) [] Web site performance: Expires Header — onenaught.com says (on August 7th, 2007 at 8:30 pm [175] )

[] Cal Henderson, Serving JavaScript Fast, May 21, 2006 [] del.icio.us Cool Stuff On Web Developement says (on August 11th, 2007 at 1:14 am [176] )

[] Vitamin Features ? Serving JavaScript Fast - The next generation of web apps make heavy use of JavaScript and CSS. We?ll show you how to make those apps responsive and quick. [] Amit says (on August 15th, 2007 at 10:25 pm [177] )

Cal,

How well does Smarty perform on a large server load as a program like Flickr has? I've been hearing more good things about the Blitz templating system for PHP over that of Smarty. Scriptaculous Accordion with XHR Support Clown Shoes says (on September 5th, 2007 at 10:10 pm [178] )

[] // accordion.js v1.0 // // Copyright (c) 2007 stickmanlabs // Author: Kevin P Miller | http://www.stickmanlabs.com // // ListScroller is freely distributable under the terms of an MIT-style license. // // I don't care what you think about the file size // Be a pro: // http://www.thinkvitamin.com/features/webapps/serving-javascript-fast // http://rakaz.nl/item/make_your_pages_load_faster_by_combining_and_compressing_javascript_and_css_files // [] Brett says (on September 22nd, 2007 at 3:11 am [179] )

Hello,

Excellent article. Very informative.

Since you mentioned Smarty and caching of CSS/JS, I thought I'd mention an extension I've made called SmartyDocB which lets you add scripts or CSS from within any template and have it shuffled off into one site-wide CSS or JS file (or shuffle it off to specifically designated files, which might suit the approach of dividing the files slightly), with comments automatically added around the injected content to indicate the source of the template (so when you are looking at the site-wide file, you know where the content came from).

This approach, I believe, is much more intuitive and handy, as you can store and edit such information alongside the HTML/XML for which it was most relevant, while still taking advantage of browser-caching and a global view of your scripts/styles. You can also manually edit these auto-generated site-wide or almost-site-wide files as long as you avoid editing within the auto-generated blocks.

To avoid live file rewriting on each page request, the system allows one through the API or optionally through a GET request to force a rewrite of all template content.

Documentation is at http://smartydocb.sourceforge.net/docs/documentation.xml (large file) and the code is at http://sourceforge.net/projects/smartydocb

Feedback is most welcome (even though my health limits how much time I can contribute to the project). .stron Sztuka optymalizacji. Podsumowanie. says (on September 25th, 2007 at 9:52 pm [180] )

[] Analogiczny problem dotyczy liczby zewnętrznych skryptów. Również w tym przypadku można zrealizować zadanie jednym zapytaniem. Bardzo obszernie i szczegółowo opisuje tą metodę Cal Henderson w artykule Serving JavaScript Fast. Artykuł zawiera ponadto przykłady programowego keszowania i kompresowania z użyciem funkcji PHP. [] Serving JavaScript Fast at davidbisset.com says (on September 29th, 2007 at 4:03 pm [181] )

[] Tips when working with big javascript libaries. the source digg_url='http://www.davidbisset.com/3452/serving-javascript-fast/'; digg_skin = 'button'; digg_bgcolor = '#FFFFFF'; digg_title = 'Serving JavaScript Fast'; digg_bodytext = ''; digg_topic = ''; Powered by Gregarious (41) Share This [] google reklam says (on October 8th, 2007 at 10:12 am [182] )

spasiwa Geekularity Magnolia Bookmarks says (on October 14th, 2007 at 7:01 am [183] )

[] Vitamin Features » Serving JavaScript Fast [] IPSOJOBS Blog Cache and GZIP your javascripts and CSSs files to speed up your site says (on October 15th, 2007 at 8:32 am [184] )

[] Good introduction to the problem of speeding up the webpages through caching and zipping javascripts Serving Javascript Fast [] prez says (on October 16th, 2007 at 9:43 am [185] )

Katalog stron says (on October 19th, 2007 at 8:05 am [186] )

Nice article. Mariachi Mike says (on October 23rd, 2007 at 10:21 am [187] )

Anybody know if there's a way to retrieve revision numbers by file with StarTeam? We need that in order to be able to enable far-future headers and automate the changes. Tom says (on October 23rd, 2007 at 3:06 pm [188] )

it's a great article and a good Site,Thx. Bye TOM YSlow - a nice performance plugin to Firebug - Robert's talk says (on October 31st, 2007 at 10:27 am [189] )

[] Serving JavaScript Fast - Nitty gritty details [] Learning AJAX Javascript by Example - Tutorials, Source-Code and Documentation | WebWolf Blog says (on January 27th, 2008 at 11:32 pm [210] )

[] Serving JavaScript Fast "With our so-called "Web 2.0 applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we're delivering the optimum experience. In practice, this means a combination of making our content as small and fast to download as possible, while avoiding unnecessarily refetching unmodified resources." [] Jordi Hernandez says (on February 6th, 2008 at 10:02 am [211] )

There is an open source java tool which allows to easily implements all the optimization techniques described in this article while easing the development process. The library in question is Jawr [212], and basically it helps develop java web apps with heavy use of javascript. It does so by allowing you to split the .js codebase in as many files and directories as desired, without compromising page loading times. In fact, it will improve loading times since it will join the files into one or several bundles (which is easily configurable), and afterwards the bundles will be minified and gzipped. This way you can have a nicely modularized codebase that is easy to mantain instead of a couple of huge javascript files. Jawr has a development mode in which instead of the bundles, you get each member separately and with no minification, so you can use an exploded war directory and test changes on the fly. All without having to change the JSPs you are testing. Finally, you get the same advantages (modularization and compression) for CSS files. Chat Muhabbet Sohbet says (on February 6th, 2008 at 8:02 pm [213] )

Thank You ! Web Development 2.0 Blog Archive Faster Page Loading says (on February 9th, 2008 at 9:00 am [214] )

[] Serving JavaScript Fast "With our so-called "Web 2.0 applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we're delivering the optimum experience. In practice, this means a combination of making our content as small and fast to download as possible, while avoiding unnecessarily refetching unmodified resources." [] anglictina says (on February 17th, 2008 at 6:11 pm [216] )

This look very cool, thank you Samurai says (on February 29th, 2008 at 11:30 pm [217] )

When sending the headers for these javascript/css files to keep them cached for as long as possible: would I set these headers in the page that uses them (for example index.php) or in the actual javascript/css file using php? Cal Henderson says (on March 3rd, 2008 at 12:36 am [218] )

When serving the js/css files. you can do that by serving them through php and adding the headers there, or (better) use apache config settings to send those headers directly Serving JavaScript Fast : iNetVista says (on March 5th, 2008 at 6:27 pm [219] )

[] http://www.thinkvitamin.com/features/webapps/serving-javascript-fast [] Daniel says (on March 13th, 2008 at 3:08 pm [220] )

What about versioning images inside css files? When I have something like this: background-image: url(../../image/admin/list_toolbar_bg.gif); gerard says (on March 20th, 2008 at 5:23 pm [221] )

This is a great article. Just found it via highscalability.com, and is still relevant even tho it is 2 yrs. old.

I'll contribute one big piece that we do differently, and it will help @daniel's problem.

We basically do everything mentioned in this article, although when it comes to filenames, instead of messing with rewriting or filtering URL's, our build process actually _copies_ our static content to a second location, _renaming_ them with their version numbers.

Now that we know how the original files map to their renamed versioned copies, we basically do a global regexp find-and-replace in our template files and CSS files. When the end-user requests the versioned file, there is no URL rewriting, because the request is for a real file.

So in @daniel's example, "url(../../image/admin/list_toolbar_bg.gif" becomes "url(../../image/admin/list_toolbar_bg.v6.gif"

Of course, we had some gotcha's with our regexp replace, which is why I'm considering using DOM instead.

The job takes a while, but that's ok since it's offline and we build continuously.

Additional bonus: when we copy CSS and JS, we can combine and minify them at this point. Additional bonus 2: when we replace the filenames in our templates, we insert our static subdomains for for concurrencey (so we don't worry about subdomains during development . links for 2008-03-21 Bloggitation says (on March 21st, 2008 at 12:21 am [222] )

[] Serving JavaScript Fast (tags: javascript ajax web2.0 tuning sysadmin) [] Philippe says (on March 21st, 2008 at 2:34 pm [223] )

How to generate a file with subversion revision numbers for each files in php ?

"At the point when we go to build our site for deployment, we simply check the revision numbers of all of our resource files and write them to a static configuration file." Cal Henderson says (on March 21st, 2008 at 7:39 pm [224] )

Philippe:

you can use svn's ls command to get the latest revision number for a file, for example:

$ /usr/bin/svn ls -v file:///var/svn/flickr-test/trunk/index.gne 45324 mygrant 9894 Jan 23 19:13 index.gne Eric Martindale says (on April 2nd, 2008 at 4:41 pm [225] )

Great work! I'll have to see what I can do to implement some of these methods in the AJAX Chat [226] that I've recently taken over

now if only I could get sockets working correctly. ;] Adriaan Nel says (on April 2nd, 2008 at 5:44 pm [227] )

I've gotta say, I've never had a problem with sites becoming so busy that they are served slowly to the end user, even large dynamic pages, and I'm also talking about relatively high usage sites (more than 2 mil page views per month).

I always try to optimize everything, but haven't implimented anything this far reaching yetmaybe I'll give it a try and see if it really helps.

Thanks for the article though. Derrick says (on April 2nd, 2008 at 6:01 pm [228] )

It'd be much cooler if part of your deployment process was the merging of the js files, vs taking a script hit each time a page is requested. Otto says (on April 2nd, 2008 at 8:57 pm [229] )

The site worked okay for me. The content is large, so I'll summarize it: 1. Combining small JS files into one big one is not always a good thing. Keep them separate and lean, but low in number. Use PHP to merge them on the fly where needed and where it make sense to do so. 2. CSS: Same thing. 3. Compression. Use it. mod_deflate and or mod_gzip are good, but precompressing the files when you deploy is better. Insert way to make mod_gzip work with precompressed gz files here. Insert the trick to make mod_gzip compress and cache on the fly here. 4. Compression doesn't work with some old as hell browsers (seriously, Netscape 4? You're worried about Netscape 4? Oh, the article was written in 2006, I get it), so using javascript and/or CSS compressors as part of your deployment process can be helpful (ugh). 5. Send good cache and expires headers. Put some thought into this, as it helps more than anything else when done right. 6. Smart use of rewrites can make things easier to code, and by changing filenames this way, you can force cache busting. 7. Combine techniques.

In short, it's a useful primer for those developers who didn't know much about web development already. isao says (on April 3rd, 2008 at 1:10 am [230] )

nice article. the techniques apply to other content types besides javascript of course. realted: http://developer.yahoo.com/performance/ Jeremy says (on April 3rd, 2008 at 1:41 am [231] )

Very nice article. I've already sent it to my team for review and will go over it with them tomorrow.

I'd be very interested to see some benchmarks that detail the number of JS files on the x-axis vs. load time on the y-axis. As the number of JS files increased you could assume the size of each file decreased. For example, point 1 on the x-axis would be one (1) 100k file. Point 2 would be two (2) 50k files. Point 3 would be three (3) 33k files, etc.

Please let me know if someone has those benchmarks somewhere.

On a separate note, I have experienced IE 6 and 7 caching pages with the same query string but the easiest workaround I've come across is adding a random query string value: &rand=x where x is the current timestamp. The Yahoo User Interface has the ability to use this technique with their connection manager and it works like a champ. I've never had a problem with Firefox caching URLs with query strings. Rick Torres says (on April 3rd, 2008 at 7:59 am [232] )

Great ideas ! I hope to speed up my page by breaking the .js files in modules that are served on specific pages only.

This resource was a great help to me ! TechMount Archive Daily Friction #325 says (on April 3rd, 2008 at 8:58 am [233] )

[] Serving JavaScript Fast - With our so-called "Web 2.0 applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we’re delivering the optimum experience. [] Serving JavaScript Fast says (on April 3rd, 2008 at 10:33 am [234] )

[] read more | digg story Explore posts in the same categories: Uncategorized [] jive says (on April 3rd, 2008 at 4:17 pm [235] )

Yahoo recommends you put your JavaScript at the bottom of the page before the closing body tag. Serving JavaScript Fast | White Sands Digital says (on April 6th, 2008 at 12:43 pm [236] )

[] The next generation of web apps make heavy use of JavaScript and CSS. We ’ll show you how to make those apps responsive and quick.read more | digg story Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages. [] Improving Performance by Combining all Scripts and CSS | ~/my-other-beans says (on April 9th, 2008 at 1:51 pm [237] )

[] Although they do not provide the source code for the ASP.NET control, they do point to an old article on Vitamin, Serving JavaScript Fast, that served as their inspiration. []

Acronyms says (on April 10th, 2008 at 7:47 am [238] )

Bookmarked. Thanks a lot. BTW having a brief summary of principles would be really good to have. Serving JavaScript Fast Know Things says (on April 11th, 2008 at 7:39 am [239] )

[] read more | digg story []