I’m happy to announce that my book “Implementing Domain-Specific Languages with Xtext and Xtend” is now available for pre-order on Packt website! 🙂
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book
I know that Xtext 2.4 has not been released yet, but I could not resist blogging about a very cool new feature in Xbase: improved automatic import functionalities!
Actually, import functionalities were already great when using Xbase also in previous versions of Xtext, but now they provide a much better experience for the user of your DSL! Indeed, all the import functionalities you are used to with JDT (like automatic import insertion, and organize imports) are available also for your Xbase language; these features were already available in Xtend, and they have been ported to Xbase itself.
At the time of writing, you need to get the very latest updates of Xtext 2.4, using the update site http://download.eclipse.org/modeling/tmf/xtext/updates/composite/latest/ .
Before you used to do something like
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
grammar org.xtext.example.helloinferrer.HelloInferrer with org.eclipse.xtext.xbase.Xbase generate helloInferrer "http://www.xtext.org/example/helloinferrer/HelloInferrer" Model: imports += Import* greetings+=Greeting*; Import: 'import' importedNamespace = QualifiedNameWithWildcard ; ... |
Now, you can use in your grammar the new Xbase rule: XImportSection:
|
1 2 3 4 5 6 7 8 9 10 |
grammar org.xtext.example.helloinferrer.HelloInferrer with org.eclipse.xtext.xbase.Xbase generate helloInferrer "http://www.xtext.org/example/helloinferrer/HelloInferrer" Model: importSection=XImportSection? greetings+=Greeting*; ... |
In this post I’m reusing some experiments you can find here (https://github.com/LorenzoBettini/Xtext2-experiments, I had blogged about these experiments in previous posts).
If you now rerun the MWE2 generator, and make sure you merge the plugin.xml_gen with plugin.xml in the .ui project, your editor will provide some interesting features for free (if you use my examples, you can find a project wizard “New Project” => “Xtext” => “HelloInferrer Project”):
Imports with wildcards are deprecated:
You now have the context menu “Organize Imports” (Shift + Control + O); try that one in the presence of such deprecation warning and imports are organized for you:
Similarly, unused imports are reported as warnings:
Again, use “Organize Imports” to fix that!
The new feature I like most is the automatic insertion of imports! (just like in JDT and Xtend): try to get content assist for a Java type, for instance,
Accept a proposal and the import will be automatically inserted (instead of the fully qualified name):
Xtext rocks! 🙂
| Revision History | |||
|---|---|---|---|
| 9 May 2013 | Updated listings to reflect the git repository sources. Put a tip on using a mirror aggregated with b3. | ||
In this tutorial I’ll show how to use Buckminster to build an Eclipse RCP Product, both in the IDE and headlessly (with ant). The application I’m building is the standard Eclipse Mail RCP example with the addition of Self-Update functionalities.
We will build two products configured with update sites; the first one will rely on standard Eclipse repositories for required features, while the second one will rely only on our own repositories.
The sources of this example can be found at http://sourceforge.net/p/buckyexamples/bucky-mail-rcp/?branch=ref%2Fmaster.
There are some nice tutorials about building Eclipse RCP Products with Buckminster (such as, e.g., Ralf Ebert‘s, Code and Me‘s, and wiki pages).
However, I found these pages out-of-date in the sense that they use Indigo or Helios for building products; with Juno things are more complicated not due to Buckminster, but to new dependencies in Juno Eclipse features and bundles (for instance, org.eclipse.rcp internally depends on org.eclipse.emf.common and org.eclipse.emf.ecore) even if you do not use the new e4 application model; see for instance the dependencies in the screenshot
This means that you will have to deal with that in the target platform definition.
Furthermore, due to the way p2 repositories are built, you will soon get the dreaded “java returned 13” when using Buckminster to build an Eclipse product (which relies on the p2 director actually), due to the above mentioned dependencies. Even for simple products like the Eclipse RCP Mail application… you can imagine when products are bigger 😉 By the way, you get similar problems even if you try to use the standard Eclipse Product export wizard.
In this post I’ll detail my experience in dealing with these problems by using Buckminster and Eclipse standard mechanisms for dealing (automatically) with dependencies; similarly, I’m not using standard Target platform definitions (which again have problems if you want to build for multiple architectures), but I’m using the nice Buckminster materialization features for materializing the target platform. The same techniques can be used with much more complex products to build them without problems due to dependencies and required software.
The tutorial is quite long since I’ll also try to provide some explanations to the problems you have when building products (in general I guess) – although the explanations are not necessary, I think they might be useful to understand things better about features, bundles, products and p2.
First of all, you need to install Buckminster in your Eclipse, using this repository
http://download.eclipse.org/tools/buckminster/updates-4.2
you will need only the features shown in the screenshot
Then, we need to create a project which with all our releng functionalities; this will be a general Eclipse project, with a Buckminster Component Specification (CSPEC); this CSPEC basically declares the target platform features as dependencies. The reasons why I’m not using a standard Eclipse Target platform definition can be found in my other post, in the section “Why not the Target Editor?”; they can be summarized with the fact that, with this technique you can get a target platform for building for multiple platforms and with all the required software automatically.
We call this releng project org.eclipse.buckminster.examples.rcp.mail.releng and the buckminster.cspec looks like this (we will enrich this cspec later to perform headless builds):
|
1 2 3 4 5 6 7 8 9 10 |
<?xml version="1.0" encoding="UTF-8"?><cs:cspec xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0" name="org.eclipse.buckminster.examples.rcp.mail.releng" componentType="buckminster" version="1.0.0"> <cs:dependencies> <cs:dependency name="org.eclipse.equinox.executable" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.equinox.p2.user.ui" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp.source" componentType="eclipse.feature"/> </cs:dependencies> </cs:cspec> |
We basically want a target paltform with org.eclipse.rcp (and its sources, since they are useful when developing), org.eclipse.equinox.executable to build executable applications, and org.eclipse.equinox.p2.user.ui to enable the p2 update manager in our RCP application.
Then, we define a Resource Map (RMAP) which tells Buckminster where to find these dependencies; we will use of course official Eclipse p2 site repositories for these dependencies; we store this map into a file build.rmap:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0" xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0"> <?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0" xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0"> <rm:property key="projects.location" value="${workspace.root}" /> <rm:property key="eclipse.download" value="http://download.eclipse.org" /> <rm:property key="eclipse.target.platform" value="${eclipse.download}/releases/juno" /> <rm:locator pattern="^org\.eclipse\.buckminster\.examples\.rcp\.mail(?:\..+)?$" searchPathRef="localsources" /> <rm:locator searchPathRef="eclipse" /> <rm:searchPath name="eclipse"> <rm:provider componentTypes="eclipse.feature,osgi.bundle" readerType="p2" source="false" mutable="false"> <rm:property key="buckminster.source" value="false" /> <rm:property key="buckminster.mutable" value="false" /> <rm:uri format="{0}"> <bc:propertyRef key="eclipse.target.platform" /> </rm:uri> </rm:provider> </rm:searchPath> <rm:searchPath name="localsources"> <rm:provider componentTypes="eclipse.feature,osgi.bundle,buckminster" readerType="local" mutable="false"> <rm:uri format="{0}/{1}"> <bc:propertyRef key="projects.location" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> </rm:searchPath> </rm:rmap> |
This basically tells Buckminster to take
Now, we define a Component Query (CQUERY) which materializes this very component; when Buckminster resolves a component if first, transitively, resolves all its dependencies; thus resolving our releng component corresponds to materialize our target platform. The build.cquery looks like this (note the reference to the build.rmap we wrote above):
|
1 2 3 4 5 6 7 8 |
<?xml version="1.0" encoding="UTF-8"?><cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="build.rmap"> <cq:rootRequest name="org.eclipse.buckminster.examples.rcp.mail.releng" componentType="buckminster"/> <cq:property key="target.arch" value="*"/> <cq:property key="target.os" value="*"/> <cq:property key="target.ws" value="*"/> <cq:advisorNode namePattern=".*" useTargetPlatform="false"/> </cq:componentQuery> |
Before starting the materialization, it is better to start from a plain and empty target platform (just like you do with a standard target definition with the target editor); one nice way of doing this, as illustrated also here, is
With the build.cquery opened in Component Query Editor, we can start the materialization by pressing “Resolve and Materialize”
This might take some time depending on your Network connection.
(TIP: you may want to aggregate a local mirror using Eclipse b3; the aggregator file for the mirror is in aggregator/target-platform-mirror.b3aggr ; you can aggregate the mirror from Eclipse, after installing b3, or headlessly using the target “b3_aggregation” of build.ant. Then, you can use the build-local.cquery you find in the releng project in the git repository; see also the README.txt file you find in the releng project).
When the materialization finishes you find the target platform in the TP project in your workspace (if you followed the instructions above).
TIP: after a materialization of the target platform, it might be better to restart Eclipse, since sometimes Buckminster tends not to catch up correctly with the new target platform.
We now create the bundle for our RCP application, using the Eclipse wizard and the RCP Mail template; this part is standard so I will not detail it here: org.eclipse.buckminster.examples.rcp.mail.bundle contains the RCP Mail bundle created with the wizard, and appropriately modified in order to enable the same update UI functionalities used in the SDK inside our RCP app (this is illustrated in this wiki page, the modifications to ApplicationWorkbenchWindowAdvisor and ApplicationActionBarAdvisor are marked in the code with ‘XXX’ task tags). We also create org.eclipse.buckminster.examples.rcp.mail.optional.bundle which contains an optional menu (and toolbar button).
We then create the feature project org.eclipse.buckminster.examples.rcp.mail.product.feature which contains our product definition:
In the Dependencies tab, add as the only dependency the feature we are in, org.eclipse.buckminster.examples.rcp.mail.product.feature. (We also do some branding and customizations, but they’re not interesting in this context).
Now, we have to “fill” the feature.xml of org.eclipse.buckminster.examples.rcp.mail.product.feature, keeping in mind that what’s in this feature will make our final product. Thus we add:
We can create a launch configuration (Eclipse Application) by selecting our product, and then select only our feature org.eclipse.buckminster.examples.rcp.mail.product.feature and then press “Select Required”.
The mail application should like this (note the Preferences menu and the Update functionalities)
In org.eclipse.buckminster.examples.rcp.mail.product.feature we also add a touchpoint advice file p2.inf (See the online help for more details) to configure the repositories (update sites) that should initially be present in the application:
|
1 2 3 4 5 6 |
instructions.configure=org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:0,name:Buckminster Mail RCP Example Site,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:1,name:Buckminster Mail RCP Example Site,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//download.eclipse.org/releases/juno/,type:0,name:Juno,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//download.eclipse.org/releases/juno/,type:1,name:Juno,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//download.eclipse.org/tools/orbit/downloads/drops/R20120526062928/repository/,type:0,name:Orbit,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//download.eclipse.org/tools/orbit/downloads/drops/R20120526062928/repository/,type:1,name:Orbit,enabled:true); |
We add the Juno release repository, the Orbit repository (not required, but just as a demonstration), and the repository on Sourceforge, where we deploy our p2 repository for our application (see the next section).
This file will be used during product build (see later).
To build an update site (in the new terminology, “p2 repository”) for our product feature, we create a new feature project, org.eclipse.buckminster.examples.rcp.mail.product.site, and in the feature.xml we include our product feature (org.eclipse.buckminster.examples.rcp.mail.product.feature); we also create org.eclipse.buckminster.examples.rcp.mail.optional.feature (which includes org.eclipse.buckminster.examples.rcp.mail.optional.bundle) and we also include this feature in product.site; this way, once the repository is deployed, the optional functionalities can be installed in an existing mail application.
Since we’d like to have a category for our features, we add a category.xml file like the following one:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<?xml version="1.0" encoding="UTF-8"?> <site> <feature id="org.eclipse.buckminster.examples.rcp.mail.product.feature"> <category name="mail.category"/> </feature> <feature id="org.eclipse.buckminster.examples.rcp.mail.optional.feature"> <category name="mail.category"/> </feature> <category-def name="mail.category" label="Buckminster Mail RCP Example"> <description> Buckminster Mail RCP Example </description> </category-def> </site> |
Remember: when building the site.p2 on a feature project with Buckminster, you will build a p2 repository NOT for the very feature, but for the included features.
Before creating the repository we use a properties file like the following to specify additional parameters for the repository creation:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
# This can be used only to build a site.p2, not a product # since we can only build a site for multiple architectures # Where all the output should go buckminster.output.root=${user.home}/tmp/mail/buckminster.output # Where the temp files should go buckminster.temp.root=${user.home}/tmp/mail/buckminster.temp # How .qualifier in versions should be replaced # get the build timestamp instead of time of last modified resource qualifier.replacement.*=generator:lastModified generator.lastModified.format='v'yyyyMMdd-HHmm # alternative: get the build timestamp instead of time of last modified resource # qualifier.replacement.*=generator:buildTimestamp # generator.buildTimestamp.format='v'yyyyMMdd-HHmm # don't build source bundles and features cbi.include.source=false target.os=* target.ws=* target.arch=* |
these properties specify:
We can now run on this project the site.p2 Buckminster action: right click on the feature project => Buckminster => Invoke Action…, select the properties file above and select site.p2.
The p2 repository will be generated (if you used the above properties file) into <your home>/tmp/mail/buckminster.output/org.eclipse.buckminster.examples.rcp.mail.product.site_1.0.0-eclipse.feature/site.p2 ; you can test your newly created repository by using the Install New Software dialog in a running Eclipse, specifying the complete local path of the created site.p2
The generated repository can then deployed to a remote site (in our example we deploy it to Sourceforge where we host also this tutorial code: http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates (if you want to browse it, use http://sourceforge.net/projects/buckyexamples/files/bucky-mail-rcp/updates/).
Buckminster does not provide direct means to build a product, since it relies on the standard Eclipse org.eclipse.equinox.p2.director application, you just need to set up a few more files, with pretty standard contents.
I think it might be worthwhile to know a few basic things about how the Eclipse p2 director application works to understand what follows (you can also using it for doing cool things like installing features into your eclipse installations from the command line).
If you provide the director application with
the director will create/install (provision) in that destination folder the requested product.
For instance, try to run the following command, replacing the path of your eclipse executable (in Windows you should use the command line version, eclipsec.exe), the destination path and the architecture details for your system
|
1 2 3 4 5 6 7 8 9 10 11 |
</path/to/your/eclipse/executable> -noSplash -application org.eclipse.equinox.p2.director -repository http://download.eclipse.org/releases/kepler -installIU org.eclipse.sdk.ide -tag InitialState -destination <destination path> -profile SDKProfile -profileProperties org.eclipse.update.install.features=true -p2.os linux -p2.ws gtk -p2.arch x86_64 |
and you’ll get a brand new Eclipse SDK from the Kepler site.
So what we will do to build our product is
Since we have already a feature project for building the p2 repository, org.eclipse.buckminster.examples.rcp.mail.product.site, we will use it also for building the product.
Inside this project we create a folder (build) with the following product.ant file which calls the p2.director ant task (in the following we will provide also some explanations):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
<project name="Product packaging"> <target name="create.product"> <property name="product.install.directory" location="${sp:destination}" /> <delete dir="${product.install.directory}" includeemptydirs="true" failonerror="false" /> <mkdir dir="${product.install.directory}" /> <buckminster.valuepath value="${fs:repositories}" id="repositories.valuepath" /> <pathconvert pathsep="," targetos="unix" property="product.repositories" refid="repositories.valuepath"> <map from="" to="file:/" /> </pathconvert> <echoproperties /> <echo message="Repositories: ${product.repositories}" /> <echo message="Install Dir : ${product.install.directory}" /> <p2.director destination="${product.install.directory}" metadataRepository="${product.repositories}" artifactRepository="${product.repositories}" profile="${profile}" arch="${target.arch}" os="${target.os}" ws="${target.ws}" roaming="true" extraarguments="-profileProperties org.eclipse.update.install.features=true"> <iu id="${iu}" /> </p2.director> </target> </project> |
For each plugin and feature projects Buckminster automatically infers a Component Specification (you can view that by right clicking on the project => Buckminster => View CSpec…) with its standard actions; we must extend this specification with additional actions to create the product, creating inside our feature project a buckminster.cspex file (note cspex, with the final x instead of c):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
<?xml version="1.0" encoding="UTF-8"?> <cspecExtension xmlns:com="http://www.eclipse.org/buckminster/Common-1.0" xmlns="http://www.eclipse.org/buckminster/CSpec-1.0"> <actions> <public name="create.product" actor="ant"> <actorProperties> <property key="buildFile" value="build/product.ant" /> <property key="targets" value="create.product" /> </actorProperties> <properties> <property key="profile" value="BuckyRcpMailProfile" /> <property key="iu" value="org.eclipse.buckminster.examples.rcp.mail.product" /> </properties> <prerequisites alias="repositories"> <attribute name="site.p2" /> </prerequisites> <products alias="destination" base="${buckminster.output}"> <path path="BuckyRcpMail.${target.ws}.${target.os}.${target.arch}/" /> </products> </public> <public name="create.product.zip" actor="ant"> <actorProperties> <property key="buildFileId" value="buckminster.pdetasks" /> <property key="targets" value="create.zip" /> </actorProperties> <prerequisites alias="action.requirements"> <attribute name="create.product" /> </prerequisites> <products alias="action.output" base="${buckminster.output}"> <path path="BuckyRcpMail.${target.ws}.${target.os}.${target.arch}.zip" /> </products> </public> </actions> </cspecExtension> |
This cspex extends the default actions of our feature projects with the new actions create.product and create.product.zip; note that the latter depends on the former and simply creates a zip of the directory containing the generated product. The create.product action calls the ant task with the same name in product.ant passing the profile and the IU of our product configuration, i.e., the one we chose above during the creation of product configuration, org.eclipse.buckminster.examples.rcp.mail.product in our example (this is the value we inserted for the field ID in the “General Information” section of the product configuration editor).
The action create.product has as prerequisite the (standard Buckminster) action site.p2 (since the component name is not specified, the component we are in is assumed); the alias of site.p2 action, repositories, will be passed to the product.ant; indeed, what is being passed to the ant file is the path resulting from site.p2, in the shape of fs:repositories, which is a path group (you can read more in the Eclipse Buckminster, The Definitive Guide, Section “Access to prerequisites and product locations” – in product.ant, we extract the actual path and do some conversion for Windows backslashes so that we obtain a valid URL for the director). We also pass a name for the profile of our product, the operating system details and the destination.
Summarizing, the steps to create the product with the p2 director are:
Before running the create.product action in the IDE we must specify a property file (similar to the one we used for mail.site project): creating the product will not work with * for os/ws/arch, because you can create the product only for a specific platform at time. For instance, this is the one I use for my Linux system
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# Where all the output should go buckminster.output.root=${user.home}/tmp/mail/buckminster.output # Where the temp files should go buckminster.temp.root=${user.home}/tmp/mail/buckminster.temp # How .qualifier in versions should be replaced qualifier.replacement.*=generator:lastModified generator.lastModified.format='v'yyyyMMdd-HHmm # don't build source bundles and features cbi.include.source=false target.os=linux target.ws=gtk target.arch=x86_64 |
In the sources of this tutorial, you can find property files for most common configurations.
You can then right click on org.eclipse.buckminster.examples.rcp.mail.product.site, select the create.product action, and the property file for the architecture you want to build your product for
In the console view you can see that first the site.p2 for the current feature project is created and then the director is called to install the product… but… you get this error:
|
1 2 3 4 5 6 7 8 9 10 |
[ant] Cannot complete the install because one or more required items could not be found. [ant] Software being installed: Mail RCP Product 1.0.0.qualifier (org.eclipse.buckminster.examples.rcp.mail.product 1.0.0.qualifier) [ant] Missing requirement: Eclipse e4 Rich Client Platform 1.1.0.v20120521-2329-8yFTIGIbGGduEZ6-7-jLx41nXj (org.eclipse.e4.rcp.feature.group 1.1.0.v20120521-2329-8yFTIGIbGGduEZ6-7-jLx41nXj) requires 'org.eclipse.emf.common.feature.group [2.7.0,3.0.0)' but it could not be found [ant] Cannot satisfy dependency: [ant] A problem occured while invoking the director. |
To understand what is going on, let’s do another digression
When a p2 repository is generated for a feature, the repository will contain:
The required plug-ins and features can be made explicit in the “Dependencies” tab in the feature editor; even if they are not specified, the dependencies are computed automatically and they are still needed by the director when the product is installed.
Since these dependencies will not be part of the generated p2 repository, we get an error when installing the product…
In our case we get an error about the feature org.eclipse.emf.common which is required by org.eclipse.e4.rcp. The feature org.eclipse.rcp (which is included in our product.feature) INCLUDES org.eclipse.e4.rcp, but org.eclipse.e4.rcp REQUIRES (not includes) org.eclipse.emf.common (and also org.eclipse.emf.ecore):
Thus, org.eclipse.emf.common will be not present in the generated p2 repository and the installation fails. With Indigo, this problem was not experienced, since org.eclipse.rcp was self-contained (and indeed, Ralf Ebert’s tutorial did not experience this problem). You would experience similar problems if your product is more involved than the simple Mail example application and requires more features and bundles: in that case the dependencies are much more.
In the following, I’ll describe 3 possible solutions to deal with that. The third one is (in my humble opinion) the best one, and is the one I’ll use in this tutorial.
We can pass the director (in our case we must modify the product.ant) additional p2 repositories, i.e., the ones we used in the RMAP, that we used to materialize the target platform (in our example the Juno release repository and the Orbit repository).
The drawbacks of this approach is that you depend on remote sites each time you build the products (you can use local mirrors though) and most of all, you do not have control on what is taken from which repository, i.e., you do not have the same control you had when defining the target platform; thus, you risk to build a product which does not use the same things of your target platform. This is especially true when building complex products with a complex target platform taking different software from several different repositories.
You can add as included features into your product.site feature project (NOT the product.feature project) the missing features one by one; this way, when the site.p2 is built the required features will go there as well, and the director will find them when installing the product.
There will surely be more than one missing feature, and the director will issue an error only on the first missing feature; thus, it will take some time to include them all.
Furthermore, including features will not be enough: you will surely have to add also bundles (plug-ins and possible fragments) to your product.site feature. For instance, org.eclipse.equinox.p2.user.ui (the one we added to our product to handle update functionalities) includes the plug-in org.eclipse.equinox.p2.ui.importexport,
which depends (i.e., requires, not includes) the bundle org.eclipse.ui.forms,
which, again, will not be part of our site.p2 unless explicitly added as a bundle!
Thus, it will take some time to have a site.p2 that makes the director happy 😉
Finally, I think that having all the dependencies hardcoded in the feature.xml makes the project highly coupled with that specific target environment (in Indigo the additional software might not even be available, in Kepler new requirements might have to be added)… why should I deal with dependencies myself?
The idea is to have a p2 repository which contains EVERY feature and bundle of our current target platform (the one we materialized at the beginning); I had blogged about a manual technique, but I like to avoid manual solutions (for maintainability and portability reasons) and prefer automatic ones.
Indeed, all the features and bundles for your products are in your target platform (otherwise your bundles would not compile, or your product launch would not work); yes, even the ones you were not aware of, like org.eclipse.emf.common and org.eclipse.emf.ecore, since Buckminster has materialized them for you as dependencies of org.eclipse.rcp. We just need a way to create a p2 repository from the current target platform.
There is a p2 ant task for this, p2.publish.featuresAndBundles, which publishes metadata for pre-existing binary features and plug-ins. All we need to do is:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<project name="Product packaging"> <target name="create.product"> ... as before ... </target> <target name="create.target.platform.repository"> <buckminster.targetPlatformLocation property="target.platform.location" /> <property name="target.platform.repository" value="file:/${sp:output}" /> <property name="repository.name" value="Target Platform Repository" /> <p2.publish.featuresAndBundles repository="${target.platform.repository}" repositoryName="${repository.name}" source="${target.platform.location}" publishArtifacts="true" /> <buckminster.publishJRE metadataRepository="${target.platform.repository}" publishArtifacts="false" /> </target> </project> |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
<public name="create.product" actor="ant"> <actorProperties> <property key="buildFile" value="build/product.ant" /> <property key="targets" value="create.product" /> </actorProperties> <properties> <property key="profile" value="BuckyRcpMailProfile" /> <property key="iu" value="org.eclipse.buckminster.examples.rcp.mail.product" /> </properties> <prerequisites alias="repositories"> <attribute name="site.p2" /> <attribute name="site.tp" /> </prerequisites> <products alias="destination" base="${buckminster.output}"> <path path="BuckyRcpMail.${target.ws}.${target.os}.${target.arch}/" /> </products> </public> ... <public name="site.tp" actor="ant"> <actorProperties> <property key="buildFile" value="build/product.ant" /> <property key="targets" value="create.target.platform.repository" /> </actorProperties> <properties> <property key="repository.name" value="Buckminster Mail RCP Example Platform Site" /> </properties> <products alias="output" base="${buckminster.output}/site.tp/" /> </public> |
The repository for the target platform will be generated in the directory site.tp.
Now, if we run the create.product action again, the director will be provided both with the p2 repository of our product and the p2 repository of the target platform, and the director will be able to find everything it needs to install the product.
You should get no error now, and you can find your product ready to run in the output directory; if you used the properties file above, depending on the chosen architecture and OS, you will find the product in <your home>/tmp/mail/buckminster.output/org.eclipse.buckminster.examples.rcp.mail.product.site_1.0.0-eclipse.feature/BuckyRcpMail.<ws>.<os>.<arch> .
We have already seen that we can deploy our product.site on the net (in our case, it’s here); however, this will still require additional Eclipse repositories (for required software which is not part of our product repository). If this reminds you of the same issue we had when building the product you’re on the right track 😉
The site.tp was useful to build our product, but we could also deploy it on the Internet, http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates (if you want to browse it, use http://sourceforge.net/projects/buckyexamples/files/bucky-mail-rcp/cloud-updates/), together with our site.p2 to make our product independent from other repositories.
To show our “independence” we create another product, a “cloud” version, where the preconfigured update sites are only the ones we maintain: we create org.eclipse.buckminster.examples.rcp.mail.cloud.product.feature, which is basically the same as org.eclipse.buckminster.examples.rcp.mail.product.feature, with another product configuration, another product identifier and another touchpoint advice file p2.inf with only our mantained repositories (i.e., the standard update site, site.p2, and the cloud-updates, which is the one we get from the target platform, site.tp):
|
1 2 3 4 |
instructions.configure=org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:0,name:Buckminster Mail RCP Example Site,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:1,name:Buckminster Mail RCP Example Site,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates/,type:0,name:Buckminster Mail RCP Example Platform Site,enabled:true); \ org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates/,type:1,name:Buckminster Mail RCP Example Platform Site,enabled:true); |
To deal with this product we have additional dedicated actions in the .cspex file of product.site (see the original source), namely create.cloud.product and create.cloud.product.zip, which basically rely on the same product.ant’s create.product target passing a different profile name and a different product UI.
You can also try to install our product from the command line, using the deployed remote repositories, with the same technique we showed for installing the Eclipse SDK from the command line; you just need to use one of the two product identifiers org.eclipse.buckminster.examples.rcp.mail.product or org.eclipse.buckminster.examples.rcp.mail.cloud.product, specify the path of your eclipse executable (in Windows you should use the command line version, eclipsec.exe), the destination path and the architecture details for your system, here are some examples: the first one installs the mail product for Linux 64bit, the second one installs the mail cloud product for Windows 64bit
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
<your/eclipse/dir>/eclipse -noSplash \ -application org.eclipse.equinox.p2.director \ -repository \ http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates,\ http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates \ -installIU org.eclipse.buckminster.examples.rcp.mail.product \ -tag InitialState \ -destination <your/destination> \ -profile MailProfile \ -profileProperties org.eclipse.update.install.features=true \ -p2.os linux \ -p2.ws gtk \ -p2.arch x86_64 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
<your/eclipse/dir>/eclipse -noSplash \ -application org.eclipse.equinox.p2.director \ -repository \ http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates,\ http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates \ -installIU org.eclipse.buckminster.examples.rcp.mail.cloud.product \ -tag InitialState \ -destination <your/destination> \ -profile MailCloudProfile \ -profileProperties org.eclipse.update.install.features=true \ -p2.os win32 \ -p2.ws win32 \ -p2.arch x86_64 |
NOTE: there will be redundancies in the deployed repositories since some features and bundles are contained both in site.p2 and site.tp; but this is just a tutorial example (and they do not disturb the installation/updates): it is up to you to choose what to deploy and how to aggregate them, in case.
For continuous integration, but also for having an automated headless way to build the products for multiple architectures and build the p2 repositories with only one click, it is very useful to have an Ant script that does all of the above. The build.ant is stored in the project org.eclipse.buckminster.examples.rcp.mail.releng, and it relies on the common.ant file which does most of the work. The Ant script relies on Buckminster headless; I won’t detail here how to install Buckminster headless, also because the Ant script checks whether it is already installed in your computer, and if it is not, it will install it for you 🙂 (it first installs Buckminster director whose zip file is in the tools subdirectory of the releng project). In the launches directory of the releng project you also find some launch configurations to experiment with.
Buckminster headless will need a way of importing the projects into the headless workspace to build them; the build.cquery we already described can be used to materialize in the workspace the releng project, but then the buckminster.cspec needs to be modified in order to materialize also the other projects. We just need to add dependencies to projects which depend on all of the other projects (since Buckminster will automatically materialize all dependencies). The feature project org.eclipse.buckminster.examples.rcp.mail.product.site depends on all the other ones. So we add this dependency (the rmap already provides all the locators for materializing all our projects). The final buckminster.cspec looks like this:
|
1 2 3 4 5 6 7 8 9 10 |
<?xml version="1.0" encoding="UTF-8"?> <cs:cspec xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0" name="org.eclipse.buckminster.examples.rcp.mail.releng" componentType="buckminster" version="1.0.0"> <cs:dependencies> <cs:dependency name="org.eclipse.buckminster.examples.rcp.mail.product.site" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.equinox.executable" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.equinox.p2.user.ui" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp.source" componentType="eclipse.feature"/> </cs:dependencies> </cs:cspec> |
The build.ant script then invokes Buckminster headless with two text files containing the headless commands: headless-resolve-commands.txt to materialize all the projects and the target platform (by resolving build.cquery)
|
1 2 |
setpref targetPlatformPath="${target.platform}" resolve "${projects.location}/org.eclipse.buckminster.examples.rcp.mail.releng/build.cquery" |
and headless-perform-commands.txt which performs all the actions to build what we want.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
build # build the site for installing the product, i.e., mail feature and platform feature # IMPORTANT: build the site first for all architectures, so that we can create # products for multiple architectures perform -D target.os=* -D target.ws=* -D target.arch=* org.eclipse.buckminster.examples.rcp.mail.product.site#site.p2 # now build the products and zip them perform -D target.os=linux -D target.ws=gtk -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip perform -D target.os=linux -D target.ws=gtk -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip perform -D target.os=macosx -D target.ws=cocoa -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip perform -D target.os=macosx -D target.ws=cocoa -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.product.zip # now build the cloud products and zip them perform -D target.os=linux -D target.ws=gtk -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip perform -D target.os=linux -D target.ws=gtk -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip perform -D target.os=win32 -D target.ws=win32 -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip perform -D target.os=macosx -D target.ws=cocoa -D target.arch=x86 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip perform -D target.os=macosx -D target.ws=cocoa -D target.arch=x86_64 org.eclipse.buckminster.examples.rcp.mail.product.site#create.cloud.product.zip |
Before building the products with the create.product actions (and zip them) it is crucial to first build the product.site repository for all platforms; this way, we will then be able to build products for several platforms (if you don’t do that, only the first product will have executable files).
All the output will be stored in the buildroot directory in the same root of our projects; the target platform will be materialized in the subdirectory target.platform, the repositories and the products can be found in the subdirectory buckminster.output (each of them in the subdirectories we saw before).
You can find the packaged products on Sourceforge. Download the one for your system (either the standard version or the “cloud” version); you will see the preconfigured update sites. Try to update the application with Check for Updates (there should be an update available for the product) and to install the optional feature.
I hope you found this tutorial useful 🙂
With the techniques described here you should be able to build even complex products seamlessly without having to deal explicitly with dependencies which are already dealt with by Eclipse PDE and p2. Once the target platform is defined correctly (and you see that with Buckminster you can do that without having to worry about required software), everything will be built automatically and your projects (especially product definitions and feature definitions) will be clean and neat 🙂
I’m quite a newbie in Ant, but I thought it would be straightforward to do a regular expression replacement in a string contained in a property; in my case, I had to replace Windows backslashes with Unix slashes… apparently, unless you use the propertyregex task from Ant Contrib, that is not supported out-of-the-box: you have to pass through a file!
This stackoverflow post shows a possible solution, and starting from that, I created a macrodef to make it easier
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<!-- = = = = = = = = = = = = = = = = = macrodef: replace_win_slashes = = = = = = = = = = = = = = = = = --> <macrodef name="replace_win_slashes"> <attribute name="property.to.process" default="default" /> <attribute name="output.property" default="default" /> <sequential> <echo message="@{property.to.process}" file="some.tmp.file" /> <loadfile property="@{output.property}" srcFile="some.tmp.file"> <filterchain> <tokenfilter> <replaceregex pattern="\\" replace="/" flags="g" /> </tokenfilter> </filterchain> </loadfile> <delete file="some.tmp.file" /> </sequential> </macrodef> |
This macro takes the property value to process (property.to.process) and the property name where to store the result; it outputs the input value to a temporary file, reads it back with a regular expression replacement (which is supported for files) and store it in the specified property (the temporary file is then deleted).
Here’s an example of use
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
<project name="replace win slashes" > <property name="my.prop1" value="file:C:\Users\bettini/my/path1" /> <!-- = = = = = = = = = = = = = = = = = macrodef: replace_win_slashes = = = = = = = = = = = = = = = = = --> <macrodef name="replace_win_slashes"> <attribute name="property.to.process" default="default" /> <attribute name="output.property" default="default" /> <sequential> <echo message="@{property.to.process}" file="some.tmp.file" /> <loadfile property="@{output.property}" srcFile="some.tmp.file"> <filterchain> <tokenfilter> <replaceregex pattern="\\" replace="/" flags="g" /> </tokenfilter> </filterchain> </loadfile> <delete file="some.tmp.file" /> </sequential> </macrodef> <replace_win_slashes property.to.process="${my.prop1}" output.property="my-prop-1" /> <echo message="my-prop-1 = ${my-prop-1}" /> </project> |
In a previous post, I blogged about mirroring Eclipse repositories with the p2.mirror Ant task; Since this Ant task needs to be run via the Eclipse antRunner application, you need a full installation of Eclipse on the machine that will run the task and I showed how to run this task from the command line.
It might be more comfortable sometimes to run it from Eclipse; for this, you need to create a launch configuration which runs an Eclipse Application, the same you run from the command line: org.eclipse.ant.core.antRunner.
We create a new general project (say, my.eclipse.mirror) and we create a mirror.ant file with the following example content (see the previous post for more details):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
<project name="Create Mirror" default="create-mirror" basedir="."> <target name="create-mirror"> <property name="target.dir" location="${user.home}/eclipsemirror" /> <echo message="" /> <echo message="Mirroring into ${target.dir}..." /> <p2.mirror source="http://download.eclipse.org/releases/juno"> <destination location="${target.dir}/releases/juno" /> <iu id="org.eclipse.rcp.feature.group" /> <iu id="org.eclipse.rcp.source.feature.group" /> <iu id="org.eclipse.equinox.executable.feature.group" /> <iu id="org.eclipse.equinox.p2.user.ui.feature.group" /> </p2.mirror> <p2.mirror source="http://download.eclipse.org/tools/orbit/downloads/drops/R20120526062928/repository"> <destination location="${target.dir}/tools/orbit/downloads/drops/R20120526062928/repository" /> </p2.mirror> <p2.mirror source="http://download.eclipse.org/tools/buckminster/headless-4.2"> <destination location="${target.dir}/tools/buckminster/headless-4.2" /> </p2.mirror> </target> </project> |
This example will mirror
Now we select the “Run Configurations…” dialog
and we create a new Eclipse Application launch configuration, which we call run_mirror; we choose, as the Program to Run, the application org.eclipse.ant.core.antRunner. Note that this application is installed by default in your SDK, but if there is a different target platform active in your workspace, which does not include the org.apache.ant bundle, this launch will fail.
In the Arguments tab, we must specify the -buildfile argument (expected by the runner) passing our ant file mirror.ant; this file is intended to be relative to the Working directory, thus we must modify it by selecting our project relative to the workspace:
Optionally, we save this launch configuration, using the “Common” tab, into our project
By default, the mirror will be saved into the folder eclipsemirror into your home folder; you may want to change this by specifying the additional program argument -Dtarget.dir=<path>, for instance,
Choose Apply and then Run; in your console view you should see that the mirror task has started
You can ignore possible problems of dependencies which cannot be satisfied
Depending on your Network connection it might take some time, but in the end you should see something like
And you can see that the mirror directory will mimic the original repositories structure
This example project can be found at https://github.com/LorenzoBettini/eclipse-mirror-example
Happy mirroring! 🙂
I used to have many Eclipse installations in my machines; typically they were different Juno versions downloaded from ecipse.org, for instance, Eclipse for RCP developers, Eclipse for DSL developers, Eclipse Modeling Tools, etc. Moreover, most of them were customized with the same plugins (for instance, Mylyn connectors) which I had to install on all of them. I preferred to have separate installations not to have a monolithic single Eclipse instance (where some features might also interfere with each other).
Then I started to use the ability of Eclipse to deal with multiple configurations, which is really a cool feature.
The idea is that you have a single Eclipse installation with all the features you always used and that you would desire in all of your Eclipse installations; then you have different directories for each “configuration”.
You can start Eclipse with a command line like the following, which uses the command line argument -configuration:
|
1 |
eclipse-main/eclipse -configuration eclipse-other/configuration |
Assuming that the main Eclipse installation is in eclipse-main directory, and that the new separate configuration will be stored in eclipse-other/configuration (which will be automatically created if it does not yet exist). What you get is a running Eclipse instance with all the features and plugins of the main Eclipse installation, but all the new features which will be installed from this running instance will go in the new configuration, thus they won’t disturb the main installation!
If you try to install new software from this Eclipse running instance, you’ll see that the list of available software sites is empty, so you will have to fill such list with the typical Eclipse software sites, such as http://download.eclipse.org/releases/juno and http://download.eclipse.org/eclipse/updates/4.2.
And then you can install new features in this Eclipse, and they will be available only in this configuration. You can then check the plugins and features directories in eclipse-other which will contain the new installed features and bundles (which will not be stored in the same directories of eclipse-main); similarly, the plugins and features directories in eclipse-other will not contain the features and bundles which are stored in the same directories of eclipse-main, though they are available in the new Eclipse configuration.
Of course, you’ll have to use the above command line each time you want this Eclipse version (you should have shell scripts to run a specific Eclipse).
Main advantages in this approach are
If you can still use command line to install new features in the separate configurations (I blogged about that); you just need to adjust the command line with the -configuration parameter.
For instance, to have an Eclipse configuration in eclipse-texlipse/configuration (based on the main Eclipse installation stored in eclipse-main) with the addition of Texlipse and Subversion features I run these commands
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
./eclipse-main/eclipse \ -configuration ./eclipse-texlipse/configuration \ -application org.eclipse.equinox.p2.director -noSplash \ -repository http://download.eclipse.org/releases/juno,http://download.eclipse.org/eclipse/updates/4.2,\ http://texlipse.sourceforge.net/ \ -installIUs \ net.sourceforge.texlipse.feature.group,\ de.vonloesch.pdf4eclipse.feature.group \ -vmargs -Declipse.p2.mirrors=true -Djava.net.preferIPv4Stack=true ./eclipse-main/eclipse \ -configuration ./eclipse-texlipse/configuration \ -application org.eclipse.equinox.p2.director -noSplash \ -repository http://download.eclipse.org/releases/juno,http://download.eclipse.org/eclipse/updates/4.2 \ -installIUs \ org.eclipse.team.svn.feature.group \ -vmargs -Declipse.p2.mirrors=true -Djava.net.preferIPv4Stack=true |
Note that using the command line for installing new features will also store in the Eclipse configuration the specified update sites (so you will find them in the Install New Software dialog).
Although I’m not a big user of Windows, sometimes I use it to test my software and products. When switching to Windows Vista and Windows 7 I’ve always noticed a huge use of the disk.
Here are some tricks I’ve used to reduce such use
First of all make sure to disable indexing for the whole hard disk (checkbox “Allow files on this drive…”)
Then stop the Windows Search service and…
make sure it is disabled (right click and the Properties)
Finally, make sure that you have no scheduled defragmentation (you can check this by pressing “Defragment now…”
This should reduce disk access, hopefully 🙂
any comment and suggestion is more than welcome!
Inspired by this nice blog post, I decided it was time to try to mirror some Eclipse repositories to
To do this I’m using p2 ant tasks for mirroring repositories.
When mirroring eclipse repositories, my main idea is to keep the path structure of the original repository URL. For instance, if you are using something like
http://download.eclipse.org/releases/juno
the mirror should differ only for the base url, e.g., something like
/home/myhome/eclipsemirror/releases/juno
This is useful if the p2 repositories you use for materializing your target platform are parametrized with respect to the base URL. For instance, I’m using Buckminster to materialize my target platforms (see also this post), and in my RMAP files I have something like
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0"> <rm:property key="eclipse.download" value="http://download.eclipse.org"/> <rm:property key="xtext.p2.repository" value="${eclipse.download}/modeling/tmf/xtext/updates/composite/releases/"/> <rm:property key="eclipse.target.platform" value="${eclipse.download}/releases/juno"/> <rm:property key="swtbot.repository" value="${eclipse.download}/technology/swtbot/helios/dev-build/update-site"/> <rm:property key="orbit.repository" value="${eclipse.download}/tools/orbit/downloads/drops/R20120526062928/repository/"/> ... </rm:rmap> |
You see that the URLs are parametrized over the property eclipse.download (which defaults to http://download.eclipse.org). If I mirror those repositories keeping the same structure
then, switching to my local mirror for target materialization it’s just a matter of passing for the property eclipse.download the URL of my local directory, e.g., file:/home/bettini/eclipsemirror, without even changing my RMAP files.
So let’s start mirroring! We need to define an Ant script for the p2 antRunner.
For instance, for mirroring the whole orbit repository (with that particular drops version) we create this script, let’s call it mirror-orbit.xml:
|
1 2 3 4 5 6 7 8 9 |
<project name="Create Mirror" default="create-mirror" basedir="."> <target name="create-mirror"> <p2.mirror source="http://download.eclipse.org/tools/orbit/downloads/drops/R20120526062928/repository"> <destination location="${target.dir}/tools/orbit/downloads/drops/R20120526062928/repository" /> </p2.mirror> </target> </project> |
Note that we keep in the target dir the same path structure of the original repository.
Since these Ant tasks need to be run via the Eclipse antRunner application, you need a full installation of Eclipse on the machine that will run the task. And you run this task with a command line like the following
|
1 2 3 4 |
/path/to/eclipse -noSplash \ -application org.eclipse.ant.core.antRunner \ -buildfile mirror-orbit.xml \ -Dtarget.dir=$HOME/eclipsemirror |
Of course you can choose any target dir; the idea is however to always use the same target dir so that all repositories will be mirrored in that path.
Mirroring an entire repository might not always be the case, especially for Juno main release repository, which is quite huge. But you can specify in the Ant task the installable units you’re interested in; then, the p2 task will only mirror those installable units (and all its dependencies). For instance,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
<project name="Create Mirror" default="create-mirror" basedir="."> <target name="create-mirror"> <p2.mirror source="http://download.eclipse.org/releases/juno"> <destination location="${target.dir}/releases/juno" /> <iu id="org.eclipse.sdk.feature.group" /> <iu id="org.eclipse.equinox.executable.feature.group" /> <iu id="org.eclipse.emf.sdk.feature.group" /> <iu id="org.eclipse.emf.query.sdk.feature.group" /> <iu id="org.eclipse.net4j.sdk.feature.group" /> <iu id="org.eclipse.net4j.db.feature.group" /> <iu id="org.eclipse.emf.cdo.sdk.feature.group" /> <iu id="org.eclipse.draw2d.sdk.feature.group" /> </p2.mirror> </target> </project> |
This task will mirror all the features that should let you define a target platform for RCP development with EMF and CDO.
NOTE: if you try to mirror org.eclipse.platform.sdk from the releases/juno repository, you will see that it will actually mirror the whole repository! (see also this forum post).
If you get some warnings during the mirror about unsolvable dependencies, you can ignore them: basically those dependencies are in a different repositories, and probably you will mirror those repositories too later.
Of course you can use several p2.mirror elements in the same Ant task. For example, this is the one we use in Emf Components, to have a mirror for our target platform: it also mirrors Swtbot and Xtext SDKs:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
<project name="Create Mirror" default="create-mirror" basedir="."> <target name="create-mirror"> <p2.mirror source="http://download.eclipse.org/releases/juno"> <destination location="${target.dir}/releases/juno" /> <iu id="org.eclipse.sdk.feature.group" /> <iu id="org.eclipse.equinox.executable.feature.group" /> <iu id="org.eclipse.emf.sdk.feature.group" /> <iu id="org.eclipse.emf.query.sdk.feature.group" /> <iu id="org.eclipse.net4j.sdk.feature.group" /> <iu id="org.eclipse.net4j.db.feature.group" /> <iu id="org.eclipse.emf.cdo.sdk.feature.group" /> <iu id="org.eclipse.draw2d.sdk.feature.group" /> </p2.mirror> <p2.mirror source="http://download.eclipse.org/tools/orbit/downloads/drops/R20120526062928/repository"> <destination location="${target.dir}/tools/orbit/downloads/drops/R20120526062928/repository" /> </p2.mirror> <p2.mirror source="http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases"> <destination location="${target.dir}/modeling/tmf/xtext/updates/composite/releases" /> <iu id="org.eclipse.xtext.sdk.feature.group" /> <iu id="org.eclipse.xtend.sdk.feature.group" /> <iu id="org.eclipse.xpand.sdk.feature.group" /> </p2.mirror> <p2.mirror source="http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site"> <destination location="${target.dir}/technology/swtbot/helios/dev-build/update-site" /> <iu id="org.eclipse.swtbot.eclipse.feature.group" /> <iu id="org.eclipse.swtbot.ide.feature.group" /> <iu id="org.eclipse.swtbot.eclipse.test.junit4.feature.group" /> <iu id="org.eclipse.swtbot.forms.feature.group" /> <iu id="org.eclipse.swtbot.feature.group" /> </p2.mirror> </target> </project> |
Final warning: it might take some time for the mirror task to complete (usually hours depending on your connection and download.eclipse.org load) and it will also take some hard disk space (for the above mirror it takes about 2 Gb).
You may have to experiment a bit to get all the features you need in the mirror; for instance, I didn’t know about the draw2d above, but I had to add it since during target materialization that feature was requested by some other feature. If you’re lost about that, you can always mirror the whole thing 😉
There’s also a follow up post showing how to run this ant task from Eclipse!
But then your builds will be faster 🙂
One of the new cool features which came with Xtext 2.3.0 is a wizard which generates all the artifacts to build your Xtext DSL project with Buckminster; this will allow you to easily build your Xtext project p2 site and also to build your project and its tests headlessly (e.g., from the command line, and, more importantly from within a continuous integration system like Jenkins).
This new feature was not advertised much, nor documented, so I decided to write a tutorial about that, also with the permission of the developer of this feature Dennis Hübner.
In this tutorial I will show how to
In the end, I will also try to describe/explain all the Buckminster files that the wizard created for you, so that one can customize them.
The sources used in this tutorial can be found at
https://github.com/LorenzoBettini/Xtext-Buckminster-Example
So, first of all, let’s create a simple Xtext hello project; for this example I will use the following settings:
and then, of course, we generate Xtext artifacts.
The DSL is basically the standard Greetings DSL; in this case, I’m using the JvmModelInferrer to generate Java classes from “Hello” specifications. Indeed, the DSL itself is not important in this tutorial. I’ve also added two Junit tests in the corresponding org.xtext.example.hellobuck.tests project:
Note the org.xtext.example.hellobuck.tests.launch Launch configuration that Xtext created for you (which basically runs all the Junit tests in this project. We will see how this will be used later.
Now we can use the Xtext wizard “Build with Buckminster”: New => Other => Xtext => Continuous Integration => Build with Buckminster:
You will have to specify the sdk feature of your Xtext project (that has been created by the Xtext new project wizard); the releng and site projects will have predefined names. For the Buckminster installation directory you have 3 choices:
In any case, specifying the headless Buckminster installation path is only useful if you intend to build your projects headlessly with ant (it is not requested for building in the IDE, nor if you plan to build it with Jenkins by setting job using the Buckminster Jenkins plugin, as we will see later).
Before pressing Finish, we also want to specify the tests to launch during the headless build, so we press the button Add and we select the launch configuration in the .tests project which the Xtext wizard created for us.
If you want to build an existing Xtext project and you do not have that launch configuration, all you need to do is to simply create a new plain Junit launch configuration from the IDE, for instance, to run your existing Junit test suite, and save it in the .tests project yourself. You can also configure the headless build to run tests later on.
When the wizard finishes, you will end up with two additional projects
Although the .buckminster project created by the wizard is thought to be used in an headless environment, the .site project is useful also to build the p2 repository from the IDE itself! Indeed, if you created an old style update site, you could build the update site from the IDE itself. But if you switched to the new category.xml format, then building the corresponding p2 repository from the IDE is not straightforward.
If you want to run Buckminster from Eclipse, you first need to install it in the IDE of course, by using this repository: http://download.eclipse.org/tools/buckminster/updates-4.2
Before creating the p2 repository, you may want to tweak some Buckminster properties, for instance, you may want to create a buckminster.properties file in the .buckminster project (or in the .site project) like the following
|
1 2 3 4 5 6 7 8 9 10 |
# Where all the output should go buckminster.output.root=${user.home}/tmp/hellobuck/build # Where the temp files should go buckminster.temp.root=${user.home}/tmp/hellobuck # How .qualifier in versions should be replaced qualifier.replacement.*=generator:lastRevision target.os=* target.ws=* target.arch=* |
so that Buckminster will generate all its artifacts in the tmp/hellobuck directory of your home folder.
We now right click on the .site project, and select Buckminster => Invoke Action… , we select site.p2 action, and optionally refer to the buckminster.properties file
When the action finishes, you will file the p2.site in the directory $HOME/tmp/hellobuck/build/org.xtext.example.hellobuck.site_1.0.0-eclipse.feature/site.p2/
The wizard created for us, in the .buckminster project a build.ant file that we can use to build the project, run the tests, and create the p2 repository with ant
Note that this file has the location of your Buckminster headless installation path hardcoded (recall the wizard we used before for creating Buckminster projects); if that is not correct, you can still pass this information to ant when invoking it
|
1 |
ant -Dbuckminster.home=<PATH> -f build.ant |
This ant script will build your project in the directory buildroot which will be created in the directory where all your projects are located (e.g., in the workspace or in another path); in particular it will
This should be the output (where WORKSPACE is actually the location of your projects):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 |
Buildfile: WORKSPACE/org.xtext.example.hellobuck.buckminster/build.ant cleanup: buckminster: [echo] IMPORTANT: Populating an empty target platform may took over 10 minutes. [exec] INFO: setpref 'targetPlatformPath=WORKSPACE/buildroot/target.platform' [exec] WARN: Target platform directory 'WORKSPACE/buildroot/target.platform' does not exist and will be created [exec] INFO: resolve 'WORKSPACE/org.xtext.example.hellobuck.buckminster/projects-platform.cquery' [exec] INFO: Resetting target platform Directory WORKSPACE/buildroot/target.platform [exec] INFO: Import complete. [exec] INFO: resolve 'WORKSPACE/org.xtext.example.hellobuck.buckminster/project.cquery' [exec] INFO: Import complete. [exec] INFO: build [exec] INFO: junit '-l' 'org.xtext.example.hellobuck.tests/org.xtext.example.hellobuck.tests.launch' '--flatXML' '--output' 'WORKSPACE/buildroot/buckminster.output/test.results/org.xtext.example.hellobuck.tests.launch.xml' [exec] INFO: Starting test session org.xtext.example.hellobuck.tests [exec] INFO: Running test org.xtext.example.hellobuck.tests.HelloBuckCompilerTest.testGeneratedJava... [exec] INFO: ...OK [0.877s] [exec] INFO: Running test org.xtext.example.hellobuck.tests.HelloBuckCompilerTest.compareGeneratedJava... [exec] INFO: ...OK [0.189s] [exec] INFO: Running test org.xtext.example.hellobuck.tests.HelloBuckParserTest.testParsingAndValidation... [exec] INFO: ...OK [0.096s] [exec] INFO: Tests finished. [exec] INFO: Elapsed time: 1.164 seconds. [exec] INFO: Total number of tests executed: 3 [exec] INFO: Successful tests: 3 [exec] INFO: Failed tests: 0 [exec] INFO: Errors: 0 [exec] INFO: Ignored tests: 0 [exec] INFO: Overall status: OK [exec] INFO: perform 'org.xtext.example.hellobuck.site#site.p2' [exec] INFO: [start org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#eclipse.build] [exec] INFO: [end org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#eclipse.build] [exec] INFO: [start org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#manifest] [exec] INFO: [end org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#manifest] [exec] INFO: [start org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#bundle.jar] [exec] INFO: [end org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#bundle.jar] [exec] INFO: [start org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#eclipse.build] [exec] INFO: [end org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#eclipse.build] [exec] INFO: [start org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#manifest] [exec] INFO: [end org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#manifest] [exec] INFO: [start org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#bundle.jar] [exec] INFO: [end org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#bundle.jar] [exec] INFO: [start org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#manifest] [exec] INFO: [end org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#manifest] [exec] INFO: [start org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#feature.jar] [exec] INFO: [end org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#feature.jar] [exec] INFO: [start org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#source.manifest] [exec] INFO: [end org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#source.manifest] [exec] INFO: [start org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#source.bundle.jar] [exec] INFO: [end org.xtext.example.hellobuck:osgi.bundle$1.0.0.qualifier#source.bundle.jar] [exec] INFO: [start org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#source.manifest] [exec] INFO: [end org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#source.manifest] [exec] INFO: [start org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#source.bundle.jar] [exec] INFO: [end org.xtext.example.hellobuck.ui:osgi.bundle$1.0.0.qualifier#source.bundle.jar] [exec] INFO: [start org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#source.manifest] [exec] INFO: [end org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#source.manifest] [exec] INFO: [start org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#source.feature.jar] [exec] INFO: [end org.xtext.example.hellobuck.sdk:eclipse.feature$1.0.0.qualifier#source.feature.jar] [exec] INFO: [start org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#copy.subfeatures] [exec] INFO: [end org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#copy.subfeatures] [exec] INFO: [start org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#copy.plugins] [exec] INFO: [end org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#copy.plugins] [exec] INFO: [start org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#manifest] [exec] INFO: [end org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#manifest] [exec] INFO: [start org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#site.p2] [exec] INFO: [end org.xtext.example.hellobuck.site:eclipse.feature$1.0.0.qualifier#site.p2] [echo] [echo] Updatesite output in: WORKSPACE/buildroot/buckminster.output/org.xtext.example.hellobuck.site_*-eclipse.feature/site.p2/ BUILD SUCCESSFUL Total time: 3 minutes 39 seconds |
It will also tell you where you can find the generated p2 site.
IMPORTANT: there’s a bug in the currently generated build.ant file, which prevent you from building with ant in Windows; until the fixed version of the wizard is released, you need to make sure that the build.ant has the right contents, as in this example source.
To build your Xtext project in Jenkins you can either create a Job which uses ant and the build.ant script, or create a Job which uses the Buckminster Jenkins plugin (I’ve also blogged about that). I will detail both ways. Both jobs have in common the access to the git repository
This requires that you configured an ant installation in your Jenkins, and that you already have a working version of Buckminster headless in Jenkins; you need to specify them in the ant build step configuration (the screeshots, of course, refer to the paths and values for my Jenkins installation):
then, we configure post build actions to archive both the artifacts (the p2 repository) and the Junit results:
To configure a Buckminster build step, we prepare a text file in the .buckminster projects with the Buckminster commands to execute (we take inspiration from the commands.txt file that the wizard created), we call it jenkins-commands.txt:
|
1 2 3 4 5 6 7 |
resolve 'org.xtext.example.hellobuck.buckminster/projects-platform.cquery' resolve 'org.xtext.example.hellobuck.buckminster/project.cquery' clean build perform -D target.os=* -D target.ws=* -D target.arch=* org.xtext.example.hellobuck.site#buckminster.clean perform -D target.os=* -D target.ws=* -D target.arch=* org.xtext.example.hellobuck.site#site.p2 junit -l 'org.xtext.example.hellobuck.tests/org.xtext.example.hellobuck.tests.launch' --stderr -o 'tests_output/org.xtext.example.hellobuck.tests.launch.xml' |
Then we configure a build step (this relies on a Buckminster installation that we have already configured in Jenkins), note that we refer to the jenkins-commands.txt we created above (an alternative would be to copy the commands directly in the text area “Commands”):
To configure the post build actions, since we used different output paths, we have to specify them accordingly (in particular, we have not used the output directory buildroot, thus the default will be used):
It may be interesting to learn more about the files that the Xtext Buckminster wizard generated for you (at least, I personally found instructive to look at them, and learned something about Buckminster 🙂 most of which I described in another post of this blog). The details might also help you if you need to customize the generated files.
The .buckminster project contains a Component Specification (CSPEC), buckminster.cspec, which looks like
|
1 2 3 4 5 6 7 |
<?xml version="1.0" encoding="UTF-8"?> <cs:cspec xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0" name="org.xtext.example.hellobuck.buckminster" componentType="buckminster" version="1.0.0"> <cs:dependencies> <cs:dependency name="org.eclipse.platform" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.xtext.sdk" componentType="eclipse.feature"/> </cs:dependencies> </cs:cspec> |
thus, due to its dependencies, it represents the target platform for your Xtext project. If your Xtext project required some further specific dependencies to build and test, this is the place to express them!
For instance, in an Xtext project of mine, Xsemantics, I run also some SwtBot tests, which also rely on pde, thus, the dependencies in the corresponding .buckminster project’s buckminster.cspec look like
|
1 2 3 4 5 6 7 8 |
<cs:dependencies> <cs:dependency name="org.eclipse.platform" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.xtext.sdk" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.swtbot.eclipse" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.swtbot.eclipse.test.junit4" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.swtbot.ide" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.pde" componentType="eclipse.feature"/> </cs:dependencies> |
To materialize components and dependencies with Buckminster, you need a Resource Map (RMAP), and the wizard created two maps: one containing p2 repositories location (for materializing the features of your target platform), called projects-platform.rmap and one for binding your projects to the workspace (when building headlessly), called project.rmap.
projects-platform.rmap looks like
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0"> <rm:property key="eclipse.download" value="http://download.eclipse.org"/> <rm:property key="xtext.p2.repository" value="${eclipse.download}/modeling/tmf/xtext/updates/composite/releases/"/> <rm:property key="eclipse.target.platform" value="${eclipse.download}/releases/juno"/> <rm:redirect pattern="org.xtext.example.hellobuck.buckminster" href="project.rmap"/> <rm:locator pattern="^org\.eclipse\.(?:xtext|xpand|xtend|xtend2|emf\.mwe|emf\.mwe2)(?:\..+)?$" searchPathRef="xtext"/> <rm:locator pattern="^com\.google.*" searchPathRef="xtext"/> <rm:locator pattern="^org\.antlr.*" searchPathRef="xtext"/> <rm:locator pattern="javax.inject" searchPathRef="xtext"/> <rm:locator searchPathRef="eclipse"/> <rm:searchPath name="xtext"> <rm:provider componentTypes="osgi.bundle,eclipse.feature" readerType="p2" source="false"> <rm:property key="buckminster.source" value="false"/> <rm:uri format="{0}"> <bc:propertyRef key="xtext.p2.repository"/> </rm:uri> </rm:provider> </rm:searchPath> <rm:searchPath name="eclipse"> <rm:provider componentTypes="eclipse.feature,osgi.bundle" readerType="p2" source="false" mutable="false"> <rm:property key="buckminster.source" value="false"/> <rm:property key="buckminster.mutable" value="false"/> <rm:uri format="{0}"> <bc:propertyRef key="eclipse.target.platform"/> </rm:uri> </rm:provider> </rm:searchPath> </rm:rmap> |
This basically tells Buckminster to take all Xtext stuff (including additional components like Google Guice and Antlr) from the Xtext main repository, and to use the main Eclipse Juno release repository for everything else. However, it also redirects to project.rmap for resolving the main Buckminster component of our project org.xtext.example.hellobuck.buckminster.
Again, if your target platform needs additional features from different sites, this is the place where to express them; reusing the example above which also needed SwtBot the rmap would also have these additional stuff:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0"> ... <rm:property key="swtbot.repository" value="${eclipse.download}/technology/swtbot/helios/dev-build/update-site"/> <rm:property key="orbit.repository" value="${eclipse.download}/tools/orbit/downloads/drops/R20120526062928/repository/"/> ... <rm:locator pattern="^org\.hamcrest*" searchPathRef="orbit"/> <rm:locator pattern="^org\.eclipse\.(?:swtbot)(?:\..+)?$" searchPathRef="swtbot"/> ... <rm:searchPath name="swtbot"> <rm:provider componentTypes="osgi.bundle,eclipse.feature" readerType="p2" source="false"> <rm:property key="buckminster.source" value="false"/> <rm:uri format="{0}"> <bc:propertyRef key="swtbot.repository"/> </rm:uri> </rm:provider> </rm:searchPath> <rm:searchPath name="orbit"> <rm:provider componentTypes="osgi.bundle,eclipse.feature" readerType="p2" source="false"> <rm:property key="buckminster.source" value="false"/> <rm:uri format="{0}"> <bc:propertyRef key="orbit.repository"/> </rm:uri> </rm:provider> </rm:searchPath> ... </rm:rmap> |
The project.rmap tells Buckminster where to find the projects of our Xtext project (so that it can bind it to the workspace when building headlessly):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0"> <rm:property key="projects.location" value="${workspace.root}"/> <rm:locator pattern="^org.xtext.example.hellobuck(?:\..+)?$" searchPathRef="project"/> <rm:searchPath name="project"> <rm:provider componentTypes="eclipse.feature,osgi.bundle,buckminster" readerType="local" mutable="false"> <rm:property key="buckminster.mutable" value="false"/> <rm:uri format="{0}/{1}"> <bc:propertyRef key="projects.location"/> <bc:propertyRef key="buckminster.component"/> </rm:uri> </rm:provider> </rm:searchPath> </rm:rmap> |
This rmap assumes that all the Eclipse projects of your Xtext project are in the same base directory; if this is not the case, because you split them, for instance, in plugins, features, doc, etc. directories, then you need to tweak this rmap accordingly, for instance, again, taken from Xsemantics,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0"> <rm:property key="projects.location" value="${workspace.root}" /> <rm:locator pattern="^it.xsemantics(?:\..+)?$" searchPathRef="project" /> <rm:searchPath name="project"> <rm:provider componentTypes="eclipse.feature,buckminster" readerType="local" mutable="false"> <rm:uri format="{0}/releng/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="eclipse.feature" readerType="local" mutable="false"> <rm:uri format="{0}/features/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="osgi.bundle" readerType="local" mutable="false"> <rm:uri format="{0}/plugins/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="osgi.bundle" readerType="local" mutable="false"> <rm:uri format="{0}/doc/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="osgi.bundle" readerType="local" mutable="false"> <rm:uri format="{0}/examples/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="osgi.bundle" readerType="local" mutable="false"> <rm:uri format="{0}/tests/{1}"> <bc:propertyRef key="workspace.root" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> <rm:provider componentTypes="eclipse.feature,osgi.bundle,buckminster" readerType="local" mutable="false"> <rm:property key="buckminster.mutable" value="false" /> <rm:uri format="{0}/{1}"> <bc:propertyRef key="projects.location" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> </rm:searchPath> </rm:rmap> |
Then, you have to Component Query files (CQUERY): projects-platform.cquery, to materialize the target platform, and project.cquery, to materialize your projects in the workspace. The first one could also be used in the IDE to actually materialize the target platform in your workspace (instead of using a target definition file); the second one is useful when building headless.
projects-platform.cquery:
|
1 2 3 4 5 6 7 8 |
<?xml version="1.0" encoding="UTF-8"?> <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="projects-platform.rmap"> <cq:rootRequest name="org.xtext.example.hellobuck.buckminster" componentType="buckminster"/> <cq:property key="target.arch" value="*"/> <cq:property key="target.os" value="*"/> <cq:property key="target.ws" value="*"/> <cq:advisorNode namePattern="^.*\.source$" skipComponent="true"/> </cq:componentQuery> |
Note that this cquery has an advisor node to skip .source components: thus, the materialized target platform will not contain the sources for all the features. This is useful headlessly to reduce the time to materialize the target platform. But if you use it also for materializing the target platform in the IDE, then sources are useful to inspect Java classes, and you might want to remove this advisor node (or simply create another cquery only to be used in the IDE, without that advisor node). This query refers to projects-platform.rmap.
project.cquery:
|
1 2 3 4 |
<?xml version="1.0" encoding="UTF-8"?> <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="project.rmap"> <cq:rootRequest name="org.xtext.example.hellobuck.site" componentType="eclipse.feature"/> </cq:componentQuery> |
As it usually happens when using Buckminster, the cquery for materializing the projects in the (headless) workspace refers to a single feature project which, transitively refers to all the bundles and additional features of your application. Usually, this is the feature project for building the p2 repository, like in this case.
“But wait! the site feature project does not refer to test projects, since I do not want to include them in my p2 repository! So, how can Buckminster run my tests headlessly if they are not materialized?”
If you look at the feature project .site, you will note that it contains a Component Extension (CSPEX) which allows to extend the automatically inferred Component Specification for the feature project with additional specifications. In this case the buckminster.cspex contains
|
1 2 3 4 5 6 |
<?xml version="1.0" encoding="UTF-8"?> <cs:cspecExtension xmlns:com="http://www.eclipse.org/buckminster/Common-1.0" xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0"> <cs:dependencies> <cs:dependency name="org.xtext.example.hellobuck.tests" componentType="osgi.bundle" /> </cs:dependencies> </cs:cspecExtension> |
You see that this specification adds a dependency to the org.xtext.example.hellobuck.tests project, so that, when Buckminster materializes org.xtext.example.hellobuck.site, and its dependencies, it will also materialize org.xtext.example.hellobuck.tests. You can use this technique to materialize additional projects; for instance, in from Xsemantics, I have a main feature for tests, and a tests project for each example, thus buckminster.cspex reads as follows
|
1 2 3 4 5 6 7 8 9 |
<?xml version="1.0" encoding="UTF-8"?> <cs:cspecExtension xmlns:com="http://www.eclipse.org/buckminster/Common-1.0" xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0"> <cs:dependencies> <cs:dependency name="it.xsemantics.example.fj.tests" componentType="osgi.bundle" /> <cs:dependency name="it.xsemantics.example.expressions.tests" componentType="osgi.bundle" /> <cs:dependency name="it.xsemantics.example.lambda.tests" componentType="osgi.bundle" /> <cs:dependency name="it.xsemantics.tests.feature" componentType="eclipse.feature" /> </cs:dependencies> </cs:cspecExtension> |
Finally, in the .site project, you will also find the feature.xml
|
1 2 3 4 5 6 7 8 |
<?xml version="1.0" encoding="UTF-8"?> <feature id="org.xtext.example.hellobuck.site" label="org.xtext.example.hellobuck.site Feature" version="1.0.0.qualifier"> <includes id="org.xtext.example.hellobuck.sdk" version="0.0.0"/> </feature> |
which includes the .sdk feature of your project (Remember: when building the site.p2 on a feature project with Buckminster, you will build a p2 repository NOT for the very feature, but for the included features), and the category.xml to give categories to your features in the p2 repository:
|
1 2 3 4 5 6 7 8 9 10 11 |
<?xml version="1.0" encoding="UTF-8"?> <site> <feature id="org.xtext.example.hellobuck.sdk" version="0.0.0"> <category name="main"/> </feature> <feature id="org.xtext.example.hellobuck.sdk.source" version="0.0.0"> <category name="main.source"/> </feature> <category-def name="main" label="Hellobuck"/> <category-def name="main.source" label="Source for Hellobuck"/> </site> |
Note that the .sdk.source feature is not a “real” feature project in your workspace, but, by default, Buckminster will automatically build a source jar for all your features and bundles.
That’s all! Many thanks to the author of this wizard, Dennis Hübner, for making it and for many suggestions!
Have fun with Bucky and Xtext 🙂
Acrobat Reader used to be available from Ubuntu Partner repository, but it is not available anymore in Ubuntu 12.10 Quantal Quetzal!
So you have to download the .deb package from adobe.com and install it:
http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.1/enu/AdbeRdr9.5.1-1_i386linux_enu.deb
However, if you have a 64bit system, do not forget to install also these packages:
|
1 |
sudo apt-get install libxml2:i386 ia32-libs |
Otherwise, acroread will fail
acroread: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory
This post was inspired by another blog post I had found on the web when trying to build a p2 site corresponding to a target platform defined for my projects. For an year I’ve been using Buckminster for building my Eclipse projects (especially when I had problems building my Xtext projects with Maven/Tycho, so that I decided to switch to Buckminster).
Having a p2 site for the current platform has some advantages, I guess:
In this post/tutorial I’ll describe my experiences
In this tutorial we will create a target platform consisting of the following features
The code is available at: https://github.com/LorenzoBettini/Target_to_p2_with_Buckminster,
This tutorial will show how to materialize the target platform and build the p2 sites
For the IDE you will need to install Buckminster main features in Eclipse, using the update site http://download.eclipse.org/tools/buckminster/updates-4.2.
NOTE: sometimes Buckminster’s cache in the IDE becomes stale, and you might experience errors about missing components; you might want to restart Eclipse and usually this makes the problem go away.
I experienced many problems when using Eclipse Target Editor (based on p2 sites); I found it quite unreliable, especially for modifying a target definition, for updating a target definition (the Update button works 1 time out of 10); moreover, it takes some time for the target to be resolved (every time you reopen a target definition with the editor).
Furthermore, when you define a target for building RCP products, then you usually want to be able to build for several platforms, not only the current one; in these cases you have a checkbox to select (“Include all environments“):
But in that case, you must give up on “Include required software”, and this might get you into troubles about missing bundles, when building your product, if it needs to include more involved features (besides the standard “Eclipse RCP SDK” and “Eclipse Platform Launcher Executables”); for instance, if your product needs also EMF Eclipse Modeling Framework SDK (which is required when using Eclipse RCP 4.2).
Defining a target platform the “Buckminster way” does not have this limitation (and, honestly, once you get familiar with that, it’s much faster than using the target editor).
Although Buckminster is able to materialize a target platform starting from a standard target definition, it also provides an alternative way:
In our example, this project is called bucky.example.rcpemf.target (it is a generic project without any nature). This is the project where we will define all Buckminster files (CSPEC, CQUERY and RMAP). In this project we define the component specification: buckminster.cspec
|
1 2 3 4 5 6 7 8 9 |
<?xml version="1.0" encoding="UTF-8"?> <cs:cspec xmlns:cs="http://www.eclipse.org/buckminster/CSpec-1.0" name="bucky.example.rcpemf.target" componentType="buckminster" version="1.0.0"> <cs:dependencies> <cs:dependency name="org.eclipse.emf.sdk" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.equinox.executable" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp" componentType="eclipse.feature"/> <cs:dependency name="org.eclipse.rcp.source" componentType="eclipse.feature"/> </cs:dependencies> </cs:cspec> |
Here we use the name of the features for our target platform. (NOTE: org.eclipse.rcp and org.eclipse.rcp.source, together, correspond to Eclipse RCP SDK.)
The rmap (which we call target-platform.rmap) is defined as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
<?xml version="1.0" encoding="UTF-8"?> <rm:rmap xmlns:rm="http://www.eclipse.org/buckminster/RMap-1.0" xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0"> <rm:property key="projects.location" value="${workspace.root}" /> <rm:searchPath name="juno-releases"> <rm:provider componentTypes="osgi.bundle,eclipse.feature" readerType="p2" source="false" mutable="false"> <rm:uri format="http://download.eclipse.org//releases/juno" /> </rm:provider> </rm:searchPath> <rm:searchPath name="juno-updates"> <rm:provider componentTypes="osgi.bundle,eclipse.feature" readerType="p2" source="false" mutable="false"> <rm:uri format="http://download.eclipse.org/eclipse/updates/4.2" /> </rm:provider> </rm:searchPath> <rm:searchPath name="local-source"> <rm:provider componentTypes="eclipse.feature,osgi.bundle,buckminster" readerType="local" mutable="false"> <rm:property key="buckminster.mutable" value="false" /> <rm:uri format="{0}/{1}"> <bc:propertyRef key="projects.location" /> <bc:propertyRef key="buckminster.component" /> </rm:uri> </rm:provider> </rm:searchPath> <rm:locator searchPathRef="local-source" failOnError="false" /> <rm:locator searchPathRef="juno-releases" failOnError="false" /> <rm:locator searchPathRef="juno-updates" failOnError="true" /> </rm:rmap> |
In our case we use the two main Eclipse p2 repositories (if the features you need are available from other p2 repositories, you will need to provide p2 reader types for these repositories as well). We will not use regular expressions for specifying the source p2 repositories, since there are no ambiguities in this example; instead, we simply specify the locators in the order we desire (preferring juno release site to juno updates). Note that we also have a search path for local sources. This is useful for building headlessly, so that Buckminster is able to find, resolve and bind to the workspace our own projects.
Finally, the component query (which we call target-platform.cquery) is defined as
|
1 2 3 4 5 6 7 |
<?xml version="1.0" encoding="UTF-8"?> <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="target-platform.rmap"> <cq:rootRequest name="bucky.example.rcpemf.target" componentType="buckminster"/> <cq:property key="target.arch" value="*"/> <cq:property key="target.os" value="*"/> <cq:property key="target.ws" value="*"/> </cq:componentQuery> |
Thus we request the component corresponding to our project bucky.example.rcpemf.target. Note that we specify that we are not interested in any specific architecture, so that we will have a target platform for all platforms/environments/architectures. Note also that in the cquery we reference the above defined rmap.
We can now materialize the target platform so defined by opening the target-platform.cquery with Buckminster corresponding editor:
WAIT: If you pressed the “Resolve and Materialize” now, Buckminster would materialize the target platform in the directory tp of a hidden project in your workspace (.buckminster). Moreover, by default, the created target platform is based on the configuration of the currently running Eclipse instance. If you want to start from a plain and empty platform (just like you do with a standard target definition with the target editor), there are a few more steps to perform:
You can specify any directory of your filesystem, in any case, make sure the directory you choose already exists.
One possible way of doing this, as illustrated also here, is
This procedure is shown by the following screenshots.
NOTE: these additional steps are not required when building headlessly
Once you pressed Finish and set this new target as the active target platform, you’re ready to materialize the cquery from the cquery editor. Of course, it might take some time for downloading all the features. Once the materialization is finished, if you followed the above procedure and created a general project TP in your workspace, you can see the materialized target platform
Once the target platform is resolved (you can also have a look at the current new target platform from the Eclipse preferences), we are ready to create a feature that we use to create the p2 site corresponding to our target platform.
We thus create a new feature project, that we call bucky.example.rcpemf.target.site, and we initialize it with all the bundles (plugins and fragments) of the current target platform (if you have other bundle projects in your workspace, make sure to unselect such additional workspace projects)
Before creating the p2 site, you may want to create a buckminster.properties (e.g., in the main project bucky.example.rcpemf.target) where we set some properties for Buckminster (e.g., that we want to build a site for all environments, and that we want the site to be created in a specific folder, in this example /tmp/bucky):
|
1 2 3 4 5 6 7 8 9 10 |
# Where all the output should go buckminster.output.root=/tmp/bucky/build # Where the temp files should go buckminster.temp.root=/tmp/bucky # How .qualifier in versions should be replaced qualifier.replacement.*=generator:lastRevision target.os=* target.ws=* target.arch=* |
We are now ready to build a p2 site for this feature with Buckminster:
If you used the same property file, you can find the site.p2 in /tmp/bucky/build/bucky.example.rcpemf.target.site_1.0.0-eclipse.feature/site.p2/ .
The p2 site you’ve just created could already be used, for instance, for building your RCP product with Buckminster through the p2 director. However, as it is, it contains only bundles, no (installable) features, thus it cannot be used, for instance, as a mirror for your target platform… see the next section
We create a new feature project bucky.example.rcpemf.target.mirror.site, and, as before, we initialize it with all the bundles (plugins and fragments) of the current target platform (if you have other bundle projects in your workspace, make sure to unselect such additional workspace projects); but this time, once the feature project is created, we also add to the feature.xml all the features that we used to create our target platform:
Let’s run the action site.p2 (using the same buckminster.properties file) on this new feature project. This time, we will end up with a p2 site with all the installable features of our target platform! You can verify this by using the Eclipse install new features menu and using as repository the one just created (if you used the same output directory it will be located at /tmp/bucky/build/bucky.example.rcpemf.target.mirror.site_1.0.0-eclipse.feature/site.p2/).
Even more, you might create a target definition file, open it with the Eclipse target editor, and create a target definition using your local mirror!
If you want to materialize the target platform and build the p2 site from the command line, you need to install the headless version of Buckminster.
For instance, you download the director from http://www.eclipse.org/buckminster/downloads.html and then you run the director as follows
|
1 2 3 4 5 6 |
director -r \ http://download.eclipse.org/tools/buckminster/headless-4.2/ \ -d <WHERE_TO_INSTALL> -p Buckminster \ -i org.eclipse.buckminster.cmdline.product \ -i org.eclipse.buckminster.core.headless.feature.feature.group \ -i org.eclipse.buckminster.pde.headless.feature.feature.group |
Headlessly, we will need to
|
1 2 3 4 5 6 7 |
<?xml version="1.0" encoding="UTF-8"?> <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="target-platform.rmap"> <cq:rootRequest name="bucky.example.rcpemf.target.mirror.site" componentType="eclipse.feature"/> <cq:property key="target.arch" value="*"/> <cq:property key="target.os" value="*"/> <cq:property key="target.ws" value="*"/> </cq:componentQuery> |
We then write an ANT file, build.ant, in the project bucky.example.rcpemf.target, as follows (this was inspired by the one generated by the Xtext wizard, developed by Dennis Hübner – I’ve blogged about that, and from the one that can be found here):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
<?xml version="1.0" encoding="UTF-8"?> <!-- Buckminster Headless - build buckminster.home must be specified on the command line, e.g., ant -Dbuckminster.home=/home/bettini/buckminster -f build.ant Properties: WORKSPACE Eclipse workspace location, or hudson job workspace build.root Where to build? WARNING: This folder will be cleaned up, so do not point to user.home or something important Default: ${WORKSPACE}/buildroot buckminster.home Buckminster headless to use. See http://www.eclipse.org/buckminster/downloads.html projects.location Where to find projects to build? Default: ${WORKSPACE} --> <project name="Buckminster Headless" default="buckminster"> <property name="WORKSPACE" location="${ant.file}/../../" /> <property name="build.root" location="${WORKSPACE}/buildroot" /> <property name="projects.location" location="${WORKSPACE}" /> <property name="resolve.commands.file" location="${projects.location}/bucky.example.rcpemf.target/headless-resolve-commands.txt" /> <property name="perform.commands.file" location="${projects.location}/bucky.example.rcpemf.target/headless-perform-commands.txt" /> <property name="target.platform" value="target.platform" /> <target name="buckminster" depends="cleanup" description="description"> <fail unless="buckminster.home" message="buckminster.home must be specified." /> <echo message="IMPORTANT: Populating an empty target platform may took over 10 minutes." /> <java fork="true" dir="${buckminster.home}" logError="true" classname="org.eclipse.core.launcher.Main" failonerror="true"> <classpath> <fileset dir="${buckminster.home}/plugins"> <include name="org.eclipse.equinox.launcher_*.jar" /> </fileset> </classpath> <arg line='-update' /> <arg line='-data "${build.root}/buckminster.workspace"' /> <arg line='-configuration "${build.root}/configuration"' /> <arg line='--script "${resolve.commands.file}"' /> <sysproperty key="projects.location" value="${projects.location}" /> <sysproperty key="buckminster.output.root" value="${build.root}/buckminster.output" /> <sysproperty key="buckminster.temp.root" value="${build.root}/buckminster.temp" /> <sysproperty key="target.platform" value="${build.root}/${target.platform}" /> <jvmarg line=" -Xms256m -Xmx512m" /> </java> <java fork="true" dir="${buckminster.home}" logError="true" classname="org.eclipse.core.launcher.Main" failonerror="true"> <classpath> <fileset dir="${buckminster.home}/plugins"> <include name="org.eclipse.equinox.launcher_*.jar" /> </fileset> </classpath> <arg line='-update' /> <arg line='-data "${build.root}/buckminster.workspace"' /> <arg line='-configuration "${build.root}/configuration"' /> <arg line='--script "${perform.commands.file}"' /> <sysproperty key="projects.location" value="${projects.location}" /> <sysproperty key="buckminster.output.root" value="${build.root}/buckminster.output" /> <sysproperty key="buckminster.temp.root" value="${build.root}/buckminster.temp" /> <sysproperty key="target.platform" value="${build.root}/${target.platform}" /> <jvmarg line=" -Xms256m -Xmx512m" /> </java> <echo message=" " /> <echo message="Updatesite output in: ${build.root}/buckminster.output/bucky.example.rcpemf.target.mirror.site_*-eclipse.feature/site.p2/" /> </target> <target name="cleanup"> <delete failonerror="false" includeemptydirs="true"> <fileset dir="${build.root}" defaultexcludes="false"> <exclude name="**/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/" /> <exclude name="**/${target.platform}/" /> </fileset> </delete> </target> <target name="reset.target-platform"> <delete includeemptydirs="true"> <fileset dir="${build.root}" defaultexcludes="false"> <include name="**/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/" /> <include name="**/${target.platform}/" /> </fileset> </delete> </target> </project> |
You will need to run ant passing this file and also specifying the path where your headless Buckminster is installed, in my case I run
|
1 |
ant -Dbuckminster.home=/home/bettini/buckminster -f build.ant |
This ant script runs Buckminster twice:
|
1 2 3 |
setpref targetPlatformPath="${target.platform}" resolve "${projects.location}/bucky.example.rcpemf.target/target-platform.cquery" resolve "${projects.location}/bucky.example.rcpemf.target/mirror-project.cquery" |
|
1 2 3 4 |
clean build perform -D target.os=* -D target.ws=* -D target.arch=* "bucky.example.rcpemf.target.mirror.site#buckminster.clean" perform -D target.os=* -D target.ws=* -D target.arch=* "bucky.example.rcpemf.target.mirror.site#site.p2" |
You need to execute Buckminster twice because of some conflicting bundles (see the discussion here) which cause problems if you attempt to run everything in one shot; if you did not separate the steps, you would get something like
|
1 2 |
INFO: perform '-D' 'target.os=*' '-D' 'target.ws=*' '-D' 'target.arch=*' 'bucky.example.rcpemf.target.site#site.p2' CSpec org.apache.lucene.core:osgi.bundle$3.5.0.v20120319-2345 has no action, group, or local artifact named bundle.and.fragments |
Once the script is finished you will get the p2 site in the subdirectory buildroot/buckminster.output/bucky.example.rcpemf.target.mirror.site_*-eclipse.feature/site.p2.
The last task is to build headlessly in Jenkins; of course, this assumes you have a working Jenkins installation with the Buckminster plugin (I’ve blogged about that also).
We write the commands to execute in two text files (similar to the ones shown in the previous section):
|
1 2 |
resolve 'bucky.example.rcpemf.target/target-platform.cquery' resolve 'bucky.example.rcpemf.target/project.cquery' |
|
1 2 3 4 |
clean build perform -D target.os=* -D target.ws=* -D target.arch=* bucky.example.rcpemf.target.mirror.site#buckminster.clean perform -D target.os=* -D target.ws=* -D target.arch=* bucky.example.rcpemf.target.mirror.site#site.p2 |
Then we can create a Jenkins job, which fetches the sources from the git repository
specify two Buckminster execution steps (as noted in the previous section, this is mandatory for the whole process to succeed):
and archive the artifacts
When the job finishes, you can see the archived p2 site
That’s all…
I hope you enjoyed this post, and comments are more than welcome 🙂
If you want to access your remote Ubuntu machine with VNC, in particular by tunnelling through ssh, there is already some documentation which can be found here. However, at least for me, the procedure explained there does not work out of the box. So here’s what I had to do to make it work.
First of all you need to install in the machines the following packages:
Then, the script to run on your client machine to access the server has to be slightly modified as follows
|
1 2 3 4 5 6 7 |
#!/bin/sh ssh -C -f -L 5900:localhost:5900 USER@REMOTEIP \ x11vnc -safer -localhost -nopw -once \ -auth /home/USER/.Xauthority -display :0 -create \ && sleep 5 \ && vncviewer localhost:0 |
where you will have to replace USER with your user on the remote machine, and REMOTEIP with the address of your remote machine.
Basically, the changes I had to make to the original script were to add the -auth command line option specifying the path to the .Xauthority, and the command line option -create to actually start an instance of the X server on the remote machine.
The bugs described in this post are now fixed (since I became the new maintainer of this plugin 🙂
If you use Buckminster for your builds, and have a personal Jenkins build system, you might want to rely on the Buckminster Plugin, which can be installed directly from Jenkins:
Unfortunately this plugin has some bugs (at least two) which somehow prevent you from using it, especially for new Eclipse versions. But it is quite easy to fix them once you know how; I found the solutions on the web, but I thought I could summarize them here to have them in one place.
First, if you’re not using Internet Explorer, then you’ll not be able to add build steps (which makes the plugin useless 🙂
As documented in the bug report (not yet fixed): you need to edit this file (all paths are intended to be prefixed with your Jenkins path)
plugins/buckminster/WEB-INF/classes/hudson/plugins/buckminster/EclipseBuckminsterBuilder/config.jelly
and “remove the <f:section></f:section> tags which surround the <f:advanced> entries”. This is expressed by this diff with the original file
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
*** config.jelly 2012-05-11 22:13:59.000000000 +0200 --- config.jelly~ 2011-04-01 00:30:56.000000000 +0200 *************** *** 46,52 **** <f:entry title="${%Script File}" field="userCommand" help="${rootURL}/plugin/buckminster/help-customCommand.html"> <f:textbox name="userCommand" value="${instance.getUserCommand()}" /> </f:entry> ! <f:advanced> <f:entry title="${%Change Workspace}" help="${rootURL}/plugin/buckminster/help-customWorkspace.html"> <f:textbox name="userWorkspace" value="${instance.getUserWorkspace()}" /> --- 46,52 ---- <f:entry title="${%Script File}" field="userCommand" help="${rootURL}/plugin/buckminster/help-customCommand.html"> <f:textbox name="userCommand" value="${instance.getUserCommand()}" /> </f:entry> ! <f:section title="Advanced Options"> <f:advanced> <f:entry title="${%Change Workspace}" help="${rootURL}/plugin/buckminster/help-customWorkspace.html"> <f:textbox name="userWorkspace" value="${instance.getUserWorkspace()}" /> *************** *** 67,71 **** <f:textarea name="equinoxLauncherArgs" value="${instance.getEquinoxLauncherArgs()}" /> </f:entry> </f:advanced> ! </j:jelly> \ No newline at end of file --- 67,71 ---- <f:textarea name="equinoxLauncherArgs" value="${instance.getEquinoxLauncherArgs()}" /> </f:entry> </f:advanced> ! </f:section> </j:jelly> |
Then, with the default configuration of the plugin, you won’t be able to install a recent version of Buckminster, thus you need to provide a custom .json (to put in userContent/buckminster/buckminster.json) file to allow new versions of Buckminster to be installed (an older example is found here); this is the json file I use:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
{"buckminsters":[ {"id":"3.7", "name":"Buckminster 3.7", "url":"http://ftp.halifax.rwth-aachen.de/eclipse/tools/buckminster/products/director_latest.zip", "iu":"org.eclipse.buckminster.cmdline.product", "repositoryURL":"http://download.eclipse.org/tools/buckminster/headless-3.7", "repositories":[ { "url":"http://download.eclipse.org/tools/buckminster/headless-3.7", "features":[ {"id":"org.eclipse.buckminster.core.headless.feature"}, {"id":"org.eclipse.buckminster.cvs.headless.feature"}, {"id":"org.eclipse.buckminster.emma.headless.feature"}, {"id":"org.eclipse.buckminster.git.headless.feature"}, {"id":"org.eclipse.buckminster.maven.headless.feature"}, {"id":"org.eclipse.buckminster.pde.headless.feature"} ]}, {"url":"http://download.cloudsmith.com/buckminster/external-3.7", "features":[ {"id":"org.eclipse.buckminster.subclipse.headless.feature"} ] } ] }, {"id":"4.2", "name":"Buckminster 4.2", "url":"http://ftp.halifax.rwth-aachen.de/eclipse/tools/buckminster/products/director_latest.zip", "iu":"org.eclipse.buckminster.cmdline.product", "repositoryURL":"http://download.eclipse.org/tools/buckminster/headless-4.2", "repositories":[ { "url":"http://download.eclipse.org/tools/buckminster/headless-4.2", "features":[ {"id":"org.eclipse.buckminster.core.headless.feature"}, {"id":"org.eclipse.buckminster.cvs.headless.feature"}, {"id":"org.eclipse.buckminster.git.headless.feature"}, {"id":"org.eclipse.buckminster.maven.headless.feature"}, {"id":"org.eclipse.buckminster.pde.headless.feature"} ]}, {"url":"http://download.cloudsmith.com/buckminster/external-4.2", "features":[ {"id":"org.eclipse.buckminster.subclipse.headless.feature"} ] } ] } ] } |
Hope this helps 🙂
I’m really sad that EclipseCon Europe has already finished, it has been a wonderful edition with lots of very interesting presentations!
My favourite ones were:
And last but not least… “be nice to nerds…” 🙂
If you have many eclipse installations (with different features/plugins) and you want to have such installations in several computers (possibly with different operating systems or with different architectures), then being able to install eclipse features from the command line might be quite helpful (at least, it is for me 🙂
You can find some related posts, like Lars Vogel’s or Paul Webster’s. These are just my two cents 🙂
What you need to do is to run eclipse from the command line (if you’re using Windows, you need to run eclipsec.exe, note the final ‘c’, instead of eclipse.exe), with these parameters
|
1 2 3 4 5 6 |
./eclipse \ -clean -purgeHistory \ -application org.eclipse.equinox.p2.director \ -noSplash \ -repository <repo1>,<repo2>,<repo3> \ -installIUs <feature1>,<feature2>,<feature3>,... |
Where the repo URLs are just the same as the ones you use as update sites or p2 repositories when installing a feature in Eclipse (you may want to always put the eclipse distribution main update site, e.g., http://download.eclipse.org/releases/juno), while the feature IDs are the actual identifiers of features you want to install; knowing the correct feature ID might not be immediate to discover, if you’re only used to the names you see in the update manager.
For instance, say that you want to install Xtext SDK, from the site http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ , then in Eclipse you would do something like in the following screenshot
but instead of “Xtext SDK“, in the command line, you should specify org.eclipse.xtext.sdk.feature.group. While in this case it was easy to infer the feature ID, but… at least for me, it was not immediate to know that “Eclipse Java EE Developer Tools” feature is indeed org.eclipse.jst.enterprise_ui.feature.feature.group !!! 🙂
Fortunately, you can get to know that by clicking that “More…” link in the above screenshot, which leads you an information dialog where you can easily find the identifier of the selected feature:
Of course you can also have the list of all the contents of an update site, by using the option -list:
|
1 2 3 4 5 |
./eclipse \ -clean -purgeHistory \ -application org.eclipse.equinox.p2.director \ -noSplash \ -repository <repo1> -list |
For instance, this is the command line I use to install in the Xtext eclipse distribution (http://www.eclipse.org/Xtext/download.html) additional stuff like the Xpand SDK, some Mylyn connectors, SwtBot and EMF CDO:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
./eclipse \ -clean -purgeHistory \ -application org.eclipse.equinox.p2.director \ -noSplash \ -repository \ http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/,\ http://download.eclipse.org/releases/juno,\ http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site,\ http://update.atlassian.com/atlassian-eclipse-plugin/e3.8 \ -installIUs \ org.eclipse.xpand.sdk.feature.group,\ org.eclipse.swtbot.eclipse.feature.group,\ org.eclipse.swtbot.eclipse.test.junit4.feature.group,\ org.eclipse.swtbot.feature.group,\ org.eclipse.swtbot.forms.feature.group,\ org.eclipse.swtbot.ide.feature.group,\ org.eclipse.egit.mylyn.feature.group,\ org.eclipse.mylyn.bugzilla_feature.feature.group,\ org.eclipse.mylyn.builds.feature.group,\ org.eclipse.mylyn.context_feature.feature.group,\ org.eclipse.mylyn.discovery.feature.group,\ org.eclipse.mylyn.gerrit.feature.feature.group,\ org.eclipse.mylyn.git.feature.group,\ org.eclipse.mylyn.github.feature.feature.group,\ org.eclipse.mylyn.hudson.feature.group,\ org.eclipse.mylyn.ide_feature.feature.group,\ org.eclipse.mylyn.java_feature.feature.group,\ org.eclipse.mylyn.monitor.feature.group,\ org.eclipse.mylyn.pde_feature.feature.group,\ org.eclipse.mylyn.tasks.ide.feature.group,\ org.eclipse.mylyn.team_feature.feature.group,\ org.eclipse.mylyn.trac_feature.feature.group,\ org.eclipse.mylyn.wikitext_feature.feature.group,\ org.eclipse.mylyn_feature.feature.group,\ org.eclipse.emf.cdo.sdk.feature.group,\ org.eclipse.emf.query.sdk.feature.group,\ com.atlassian.connector.eclipse.jira.feature.group \ -vmargs -Declipse.p2.mirrors=true -Djava.net.preferIPv4Stack=true |
The final -vmargs are just some additional arguments which you may want to skip.
Hope this helps 🙂
OK, I’ve been using PmWiki for some years now for my webpage… it was time for a fresh look 🙂
Let’s try WordPress then!
I’ll also take the chance to host my main programming blog on this very site.
Goodbye old home page 🙂
