Like Evil Dead 2, this “remake” of 2006’s Build Workshopwas far more groovy than the first, in terms of special effectsproducing concrete results. I look forward to the next one… perhaps in the fall, when the leaves are turning and I need to get out of Toronto again? I could see these workshops becoming a quarterly event, if nothing else to keep people talking about and actively working on this issue. Facetime is important, especially when we’re all otherwise swamped with Real Work ^tm.

While the last one produced ideas, plans, and documentation about best practices, it failed to materialize its one big requirement, which was a commmon build infrastructure, hosted at Eclipse. I’ve since created documentation ([1], [2]) for doing a DIY build server, which has been successfully implemented by at least two projects. But it’s still fairly labour intensive, and it’s tough to share.

This time around, we focused on something that’s been on my TODO listfor about a year: running my build system on build.eclipse.org. We’d originally planned to produce a vserver image or vserver config script, but since there’s still ample work to be done to genericize my existing image to work outside Modeling, this seemed a shorter initial path to prototype. And the fact that we can’t distribute such an image (because of all the GPL code in a Linux distro) was also a bit of a blow to the idea.

Bjorn and Denis, in trying to understand a little of the madness that is my system, have made me revisit all my original assumptions and requirements, to ensure they’re still valid, and that there’s not a better approach. I love having my assumptions challenged — it’s the only way to prove a system matches its demand, and that I’m not simply stagnating under a mantra of ‘because that’s how we’ve always done it’. (It’s sort of like my attempt to challenge the assumptionthat next year’s coordinated release should be called “Io”… but I digress.)

One thing we are still clinging to, for this first iteration at least, is that we’ll be building all-java, single-platform builds, for small projects & components who want a website with downloads, an update site, p2 metadata, jar signing, pack200 optimization… and little or no overhead in terms of infra setup. So, this solution will NOT address complex builds like the Platform, WTP, TPTP, or product builds. This is strictly (for now, anyway) designed to ease the burden on developers who don’t want to have to care about web/build infra. Of course none of this addresses the releng code that defines WHAT and HOW to build — it only enables a faster route to market for running and publishing builds. If you’re a project of the size of VE, PDT, or STP — or something smaller — this system’s for you.

Building anything more complex remains out of scope for now, and I admit freely that some of the reason for that is that Denis doesn’t do builds, Bjorn does small Technology Project builds, and I do Modeling builds — none of which motivates us to spend effort solving problems we don’t (yet) have. For 2 years my system didn’t do UI testing, because until UML2 Tools & GEF joined the party, there was no need. Now there are several projects w/ UI testing, so the system allows for that.

What is in scope is to explore the use of the Cruise Control interface to improve build scheduling and queuing, so that we can better manage disc and cpu usage. In time, the hope is that if a build queue gets too long, we’ll have statistics to back up the claim that we need more memory, cpu, or disc space, in order to better meet demand.

Clearly, I have a lot of work ahead of me, but today showed that both Bjorn and Denis are willing help out here. (That’s not meant perjoratively — only that we all have other time constraints pressing on us, but that we’ve collectively agreed to set aside cycles to focus on this.)

Here are the five pieces that must come together to make a build system work:

1. properly defined features and plugins — the responsibility of the component lead
2. a .releng project (or perhaps a Buckminster project?) to define what to build, what order to build, when to test, and HOW to package — shared responsibility between component lead and the release engineer, if your project is large enough to have one. Note that for these first two, my Summer of Codestudent is exploring the use of JET to produce wizards to guide this processand make it a little more friendly and less RTFM’ish.
3. UI to run builds on demand
4. UI to validate builds (JUnit results summary, links to build metadata like logs and config files). This could be views in Eclipse, but because publication involves putting bits on a website, it’s currently handled predominantly as PHP (with some Ant and bash scripts)
5. UI to publish acceptable builds to Eclipse.org & generate other build output (eg., an update site, Ganymede site contribution file, etc.); this could be merged into the build itself, but I split it because IMHO not all builds need to be published, and generating all the extra meta isn’t required when all you want to do is test the user’s install experience with your project. But of course this assumption can be challenged…

Then, in terms of automation (and places we can improve), there’s:

1. feeding the latest dependencies to the system so that when a new Platform (or EMF, or GEF, etc.) is available, the ad hoc and automated builds can simply use that new dependency. RSS feedsof course come to mind here, which though I was a big proponent of, haven’t really done much with (insert age-old “time constraints” excuse here)
2. scheduled builds are great, and can be set to run only if CVS has changed, but continuous building might be handy too. However, it’s important to consider how often to check CVS for changes [frequency], what’s considered a complete change (vs. part of a series of commits) [threshhold], and where to check [all the sources? or just the mapfiles]. Build too often & you’re wasting others’ cpu time. Not a big deal when there’s only 3 builds on build.eclipse.org, but if all 20+ Modeling builds move there… sharing and coordination suddenly becomes very important. And if your project consists of less than, say, 5 committers… do you really need continuous builds?
3. automated cleanup of old dependencies so the UI stays clean and disc usage is kept reasonable

And then, of course, there’s room to improve integration.

1. supporting Subversion sources
2. supporting Maven-based dependencies
3. converting bash scripts that “do work” to Ant scripts w/ custom tasks; submitting these back to PDE build or releng.basebuilder for reuse
4. converting bash scripts that “do calculations” to PHP-based web apis, so they can be called by web, shell, ant, or java
5. porting configuration parameters to the Portal

At the end of the day, we had:

1. evaluated Maven, Buckminster, PDE Build, basebuilder.releng, and the stuff I’ve done to simplify the PDE/basebuilder experience
2. successfully run the GEF build on build.eclipse.org (with some UI problems to be fixed, and at least one failing JUnit test)
3. implemented code to extract build parameters from the Portal instead of from static php config files; testing and iteration TBD (probably more things to add/remove/simplify)
4. created a way for the genie daemon to run builds launched from the web or crontab (but jar signing fails as we longer need to push bits to build.eclipse.org)
5. dumped a lot of the “common modeling build” code into a new Dash project, org.eclipse.dash.commonbuilder, which will house common web UI, server config scripts/properties (eg., paths for JDKs & build folders), and build/promote scripts
6. begun scripting the process of bootstrapping (or updating from CVS) this common builder, so it’s reproducable and hands-off; verification TBD

If that doesn’t sound like an exhausting enough day, have a look at this.

Note that this is not a project plan, and until one is drafted, nothing is set in stone. More people willing to help will of course allow more things to get implemented. So, does this project interest you? Are you willing to contribute time and effort kicking its tires by porting your build to this system, in order to make it better for all?

What is XSTM?

XSTMis made of three projects. The Java implementation is called JSTMand is the base from which the other versions are derived. An adapted version made with Luciano, the author of GWM , is available for GWT . It allowsthis library to be used in a browser. NSTM is a .NET port based on IKVM.

All implementations are compatible with each other so object replication can take place e.g. between a Java server and a .NET Smart Client.

This JavaScript client library allows you to make Facebook API calls from any web site and makes it easy to create Ajax Facebook applications. Since the library does not require any server-side code on your server, you can now create a Facebook application that can be hosted on any web site that serves static HTML. An application that uses this client library should be registered as an iframe type. This applies to either iframe Facebook apps that users access through the Facebook web site or apps that users access directly on the app’s own web sites. Almost all Facebook APIs are supported.

JSJaC is a jabber/XMPP client library written in JavaScript to ease implementation of web based jabber/XMPP clients. For communication with a jabber server it needs to support either HTTP Pollingor XMPP Over BOSH(formerly known as HTTP Binding). JSJaC has an object oriented design which should be quite easy to use. Communication is done by using the XML HTTP Request objectalso refered to as AJAX technology. Your browser must support this.

A simple command-line tool to create the folders and helper files for a new JavaScript project/library. As a bonus, you can quickly create a website to promote your project.

When you start a new JavaScript library, how do you layout the source files, the tests, the distribution files? Do you have support scripts to generate distributions from source files? Run your JavaScript unit tests? Generators to create new unit test HTMLfiles?

jQuery UI is a set of themable widgets and interactions, built on top of the jQuery JavaScript Library, that you can use to build highly interactive web applications.

Google Web Toolkit (GWT) is an open source Java software development framework that makes writing AJAX applications like Google Mapsand Gmaileasy for developers who don't speak browser quirks as a second language. Writing dynamic web applications today is a tedious and error-prone process; you spend 90% of your time working around subtle incompatibilities between web browsers and platforms, and JavaScript's lack of modularity makes sharing, testing, and reusing AJAX components difficult and fragile.

GWT lets you avoid many of these headaches while offering your users the same dynamic, standards-compliant experience. You write your front end in the Javaprogramming language, and the GWT compiler converts your Java classes to browser-compliant JavaScript and HTML.