Lorenzo Bettini is an Associate Professor in Computer Science at the Dipartimento di Statistica, Informatica, Applicazioni "Giuseppe Parenti", Università di Firenze, Italy. Previously, he was a researcher in Computer Science at Dipartimento di Informatica, Università di Torino, Italy.
He has a Masters Degree summa cum laude in Computer Science (Università di Firenze) and a PhD in "Logics and Theoretical Computer Science" (Università di Siena).
His research interests cover design, theory, and the implementation of statically typed programming languages and Domain Specific Languages.
He is also the author of about 90 research papers published in international conferences and international journals.
The goal of composite repositories is to make this task easier by allowing you to have a parent repository which refers to multiple children. Users are then able to reference the parent repository and the children’s content will transparently be available to them.
The nice thing of composite repositories is that they can be nested at any level. Thus, I like to have nested composite repositories according to the major.minor, major.minor.service.qualifier.
Thus the layout of the p2 composite repository should be similar to the following screenshot
Note that the name of the directories that contain a standard p2 repository have the same name of the contained feature.
The key points of a p2 composite repository are the two files compositeArtifacts.xml and compositeContent.xml. Their structure is simple, e.g.,
XHTML
1
2
3
4
5
6
7
8
9
10
11
12
13
<?xml version='1.0'encoding='UTF-8'?>
<?compositeMetadataRepository version='1.0.0'?>
<repository name='Composite Site Example All Versions'type='org.eclipse.equinox.internal.p2.metadata.repository.CompositeMetadataRepository'version='1.0.0'>
Note that a child location is intended relative to the path of these files; you can also specify absolute paths, not to mention http urls to other remote p2 sites.
The structure is not that complex, so you can also create it by hand; but keeping it up to date might not be that trivial. With that respect, p2 provides some ant tasks for managing composite repositories (creating, adding an entry, removing an entry), and that’s my favorite way to deal with composite repositories. I’ll detail what I usually do in this blog post, in particular, how to create (or update) a p2 composite repository with a new entry during the build.
The ant file is completely reusable and customizable by passing properties; you can reuse it as it is, after you setup your pom.xml as detailed below.
In this blog post I’ll show how to do that with Maven/Tycho, but the same procedure can be done in a Buckminster build (as I’ll hint at the end).
I’ll use a simple example, https://github.com/LorenzoBettini/p2composite-example, consisting of a plug-in project, a feature project, a project for the site, and a releng project (a Maven/Tycho parent project). The plug-in and feature project are not interesting in this context: the most interesting one is the site project (a Tycho eclipse-repository packaging type).
Of course, in order to run such ant tasks, you must run them using the org.eclipse.ant.core.antRunner application. Buckminster, as an Eclipse product, already contains that application. With Tycho, you can use the tycho-eclipserun-plugin, to run an Eclipse application from Maven.
We use this technique for releasing a new version of our EMF-Parsley Eclipse project. We do that directly from our Hudson HIPP instance; the idea is that the location of the final main composite site is the one that will be served through HTTP from the download.eclipse.org. We have a dedicated Hudson job that will release a new version and put it in the composite repository.
The ant file
The internal details of this ant files are not necessary to reuse it, so you can skip the first part of this section (you only need to know the main properties to pass). Of course, if you read it and you have suggestions for improve it, I’d be very grateful 🙂
The ant file consists of some targets and macro definitions.
The main macro definition is the one invoking the p2 ant task:
Note that we’ll also create a p2.index file. I prefer not to compress the compositeArtifacts.xml and compositeContent.xml files for easier inspection or manual modification, but you can compress them setting the “compressed” to “true” property above.
First of all, this task will copy the p2 repository created during the build in the correct place inside the nested p2 composite repository.
Then, it will create or update the composite site for the nested repository major.minor, and then it will create or update the composite site for the main site (the one storing all the versions). The good thing about these ant tasks is that if you add a child location that already exists they won’t complain (though you can set a property to make them fail in such situations); this is crucial for updating the main repository, since most of the time you will not release a new major.minor.
This target calls (i.e., depends on) another target to compute the properties to pass to the macrodef, according to the information passed from the pom.xml
XHTML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<!--
site.label The name/title/label of the created composite site
unqualifiedVersion The version without any qualifier replacement
Default properties (that can be modified by passing a value from the pom.xml file):
software.download.area: the absolute path of the parent folder for the composite p2 site (default is “p2.repositories” in your home directory)
updates.dir: the relative path of the composite p2 site (default is “updates”); this is relative to software.download.area
Thus, by default, the main p2 composite update site will end in ${user.home}/p2.repositories/updates. As hinted in the beginning, this can be any absolute local file system path; in EMF-Parsley Eclipse, since we release from Hudson, it will be the path served by the Eclipse we server download.eclipse.org. So we specify the two above properties accordingly.
These are the properties that must be passed from the pom.xml file
site.label: the main label that will appear in the composite site (and that will be recorded in the “Eclipse available sites”). The final label will be “${site.label} All Versions” for the main site and “${site.label} <major.minor>” for the nested composite sites.
project.build.directory: the location of the p2 repository created during the build (usually of the shape <project.id>/target/repository)
unqualifiedVersion: the version without qualifier (e.g., 1.1.0)
buildQualifier: the replaced qualifier in the built version
Note that except for the first property, the other ones have exactly the same name as the ones in Tycho (and are set by Tycho directly during the build, so we’ll reuse them).
The ant file will use an additional target (not shown here, but you’ll find it in the sources of the example) to extract the major.minor part of the passed version.
Calling the ant task from pom.xml
Now, we only need to execute the above ant task from the pom.xml file of the eclipse-repository project,
ATTENTION: in the following snipped, for the sake of readability, I split the <appArgLine> into several lines, but in your pom.xml it must be exactly in one (long) line.
As I said, you should pass site.label as you see fit (for the other properties you can use the default).
You may want to put this plugin specification inside a Maven profile, that you activate only when you are actually doing a release (see, e.g., what we do in this pom.xml, taken from our EMF-Parsley Eclipse project).
[echo]Composite name:Composite Site Example All Versions
[echo]Adding childrepository:1.0
BUILD SUCCESSFUL
And here’s the directory layout of your ${user.home}/p2.repositories
Run the command again, and you’ll get another child in the nested composite repository 1.0 (the qualifier has been replaced automatically with the new timestamp):
Let’s increase the service number, i.e., 1.0.1, (using the tycho-versions-plugin) and rebuild:
As I hinted before, with Buckminster you can directly call the p2 ant tasks, since they are included in the Buckminster headless product. You will only need to add custom actions in the .cspec (or in the .cspex if you’re inside a plugin or feature project) that call the ant task passing the right properties. An example can be found here. This refers to a slightly different ant file from the one shown in this blog post, but the idea is still the same.
Possible Improvements
You may want to add another nesting level, e.g., major -> major.minor etc… This should be straightforward: you just need to call the macrodef another time, and compute the main update site directory differently.
I recently started to play with Sonarqube to reduce “technical debt” and hopefully improve code quality (see my previous post). I’d like to report on my experiences about using Sonarqube to analyze Xtend code.
Xtend compiles into Java source code, so it looks like it is trivial to analyze it with Sonarqube; of course, Sonarqube will analyze the generated Java code, but it’s rather easy to refer to the original Xtend code, since Xtend generates clean Java code 🙂
However, we Sonarqube 4.4 it looks like it’s harder than I thought due to some facts:
Xtend automatically adds @SuppressWarnings(“all”) annotations to all generated Java classes.
In the parent project we specify the actual project with sources to be analyzed, and the project containing tests (in this example I also use jacoco for code coverage, but that’s not crucial for this example):
The plugin and the plugin.tests projects intentionally contain Xtend and Java files with some Findbugs issues, e.g.,
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
classXtendGreeting{
def greeting(){
"Hello, world."
}
def useEquals(){
// 2 findbugs issues:
// findbugs:DM_NUMBER_CTOR
// findbugs:EC_UNRELATED_TYPES
newInteger(0).equals("foo");
}
}
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
packageexample
importorg.junit.Test
importorg.junit.Assert
classXtendGreetingTest{
@Testdef voidtestGreeting(){
val greeting=newXtendGreeting().greeting
println(greeting)
Assert.assertEquals("Hello, world.",greeting)
}
def useEquals2(){
// 2 findbugs issues:
// findbugs:DM_NUMBER_CTOR
// findbugs:EC_UNRELATED_TYPES
newInteger(0).equals("foo");
}
}
Now, assuming you have Sonarqube 4.4 running on your machine, you can run the typical maven commands to analyze your code (make sure you set the MaxPermSize in the MAVEN_OPTS otherwise the Xtend compiler will run out of memory):
Shell
1
2
3
export MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=256m"
mvn clean install
mvn sonar:sonar
If you go to Sonarqube web interface you see
So you see that Sonarqube correctly detected Findbugs issues in all the Java files, but for the Java code generated by Xtend, it only detected the issues in the plugin.tests project, not on the plugin project (as explained here http://sonarqube.15.x6.nabble.com/sonarqube-findbugs-and-generated-sources-td5028237.html, Sonarqube does “not take into consideration this suppress warnings annotation in test files”).
To deal with this problem, I created an ant file which basically removes all the @SuppressWarnings(“all”) annotations in all the generated Java files in the xtend-gen folder:
and I created a Maven profile in the parent pom that, when activated, invokes the ant target, in the process-sources phase (recall that this phase is executed after generate-sources phase, when the Xtend files are compiled into Java code)
I recently started to play with Sonarqube to reduce “technical debt” and hopefully improve code quality. I’d like to report on my experiences about using Sonarqube to analyze Xsemantics, a DSL for writing rule systems (e.g., type systems) for Xtext languages.
I was already using the Jenkins Continuous Integration server, and while building I was already using Findbugs and Jacoco, thus, I was already analyzing such software, but Sonarqube brings new analysis rules for Java programs and it also integrates results from Findbugs and Jacoco, aggregating all the code quality results in a web site.
In spite of the Jenkins builds Sonarqube detected some issues when I started
First of all, I had to exclude the src-gen and emf-gen directories (the former is where Xtext generates all its artifacts, and the latter is where Xcore generates the EMF model files); since these are generated files and I did not want to make them part of the analysis. I’ve done such exclusion with a property in the main pom.xml (for readability I split it into lines):
XHTML
1
2
3
4
5
6
7
8
9
10
11
12
<sonar.exclusions>
file:**/src-gen/**,
file:**/emf-gen/**,
file:**/example/**,
file:**/tests/**,
**/*RuntimeModule.java,
**/*UiModule.java,
**/*XcoreReader.java,
**/*UiExamples.java,
**/*TypeSystemGen*.java,
**/*StandaloneSetup*.java
</sonar.exclusions>
Note that for the moment I’m also excluding tests from the analysis… it is considered best practice to analyse tests as well (and I have many of them), but I wanted to concentrate on the code first. I also excluded other Java files for which issues are reported, like the Xtext Guice modules, due to the wildcards in the method signatures… I have to live with them anyway 🙂
After that the number of issues reduced a little bit, but there were still some issues to fix; most of them were easy, basically due to Java conventions I hadn’t use (e.g., name of fields and methods or even names of type parameters).
Another thing that I had never considered was dependency cycles among Java packages and files. Sonarqube reports them. Luckily there were only few of them in Xsemantics, and the hardest part was to read the Dependency Structure Matrix, but in the end I managed to remove them (there must be nothing in the upper triangle to have no cycle):
Then came the last major issue: Duplicated Code!!! All by itself this issue was estimated with 13 days of technical debt! And most of the duplicated code was in the model inferrer (a concept from Xbase). Moreover, such inferrer is written in Xtend, a cleaner Java, and the Xtend compiler then generates Java code. Thus, Sonarqube analyses the generated Java code, and the detected duplicate code blocks are on the Java code. This means that it takes some time to understand the corresponding original Xtend code. That’s not impossible since Xtend generates clean Java code, but it surely adds some work 🙂
Before starting to remove duplicated code (around 80 blocks in the generated Java code) the Xtend inferrer was around 1090 lines long (many parts are related to string templates for code generation) corresponding to around 2500 lines of generated Java code! After the refactoring the Xtend inferrer was around 1045 lines long, and the generated Java code reduced to around 2000 lines.
That explains also the reduction of lines of code and complexity:
But now technical debt is 0 🙂
And it’s nice to look at this dashboard 🙂
By the way, I also had to disable some issues I did not agree on (tabulation characters) and avoid reported issues on method name conventions on a specific file (because methods that start with the underline characters _ have a specific meaning in Xtext/Xtend). Instead of disabling them on the Sonarqube web interface, I preferred to disable them using properties in the pom file so that it works across different Sonarqube installations (e.g., I also have a local Sonarqube instance on my machine to do some quick experiments). Such multi properties are not officially supported in the Sonar invocation (e.g., through the sonar runner or via Maven), but I found a workaround: http://stackoverflow.com/questions/21825469/configure-sonar-sonar-issue-ignore-multicriteria-through-maven (but, be careful, it is considered a hack as reported in the mailing list: http://sonarqube.15.x6.nabble.com/sonar-issue-ignore-multicriteria-td5021722.html):
XHTML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
<!-- see http://sonarqube.15.x6.nabble.com/sonar-issue-ignore-multicriteria-td5021722.html and
I recently had to install Linux Kubuntu 13.10 Saucy Salamander (at the time of writing I’ve already upgraded it to 14.04 Trusty Tahr) on a Dell Precision M3800 (a really cool and powerful laptop, see the details here).
The installation went really smooth, and I’m enjoying a very fast and stable Linux OS on this laptop.
In this blog post I’ll detail only a few tips and further tweaks after the installation.
This laptop comes with the “crazy” resolution of 3200×1800! Unfortunately, this is barely usable at least in my experience: everything is so small that I can’t read almost anything… adjusting the DPI as suggested here really did not help: the fonts, window border become readable and usable, but the system looks ugly… (by the way, the same problem holds in Windows 8, at least for my everyday program, i.e., Eclipse: most fonts and icons are not readable)… until these resolution problems are fixed in Kubuntu (and in some applications as Eclipse), I reverted the resolution to something smaller (and still the resolution is high :), that is 1920×1080.
Enable Hibernate
First check that hibernate actually works by running (remember that your swap partition is at least as large as your available RAM):
1
sudo pm-hibernate
After you computer turns off, try and switch it back on. If your open applications re-open you can re-enable hibernate: run below command to edit the config file:
At first, I thought that Function keys were not working at all… then I discovered that on new laptops like this one F-keys are default to their media mode (!). You can change the default behavior of the F keys in the BIOS, but I prefer the F-Lock icon on the Esc button: this will take them back to their standard behavior.
Xcore is an extended concrete syntax for Ecore that, in combination with Xbase, transforms it into a fully fledged programming language with high quality tools reminiscent of the Java Development Tools. You can use it not only to specify the structure of your model, but also the behavior of your operations and derived features as well as the conversion logic of your data types. It eliminates the dividing line between modeling and programming, combining the advantages of each.
I took inspiration from Jan Köhnlein’s blog post; after switching to a manually maintained Ecore in Xsemantics, I felt the need to further switch to Xcore, since I had started to write many operation implementations in the metamodel, and while you can do that in Ecore, using Xcore is much easier 🙂 Thus in my case I was starting from an existing language, not to mention the use of Xbase (not covered in Jan’s post). Things were not easy, but once the procedure works, it is easily reproducible, and I’ll detail this for a smaller example.
So first of all, let’s create an Xtext project, org.xtext.example.hellocustomxcore, (you can find the sources of this example online at https://github.com/LorenzoBettini/Xtext2-experiments); the grammar of the DSL is not important: this is just an example. We will first start developing the DSL using the automatic Ecore model inference and later we will switch to Xcore.
(the language is basically the same of the previous post).
The grammar of this example is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
grammar org.xtext.example.helloxcore.HelloXcore with
With this DSL we can write programs of the shape (nothing interesting, this is just an example)
1
2
3
4
5
Hellofoo!
Greetingbar{
sayFoo()+"bar"
}
Now, let’s say we want to check in the validator that there are no elements with the same name; since both “Hello” and “Greeting” have the feature name, we can introduce in the metamodel a common interface with the method getName(). OK, we could achieve this also by introducing a fake rule in the Xtext grammar, but let’s do that with Xcore.
Switching to Xcore
Of course, first of all, you need to install Xcore in your Eclipse.
Before we use the export wizard, we must make sure we can open the generated .genmodel with the “EMF Generator” editor (otherwise the export will fail). If you get an error opening such editor about resolving proxy to JavaJVMTypes.ecore like in the following screenshot…
..then we must tweak the generated .genmodel and add a reference to JavaVMTypes.genmodel: open HelloXcore.genmodel with the text editor, and search for the part (only the relevant part of the line is shown)
Since we’re editing the .genmodel file, we also take the chance to modify the output folder for the model files to emf-gen (see also later in this section for adding emf-gen as a source folder):
Now save the edited file, refresh the file in the workspace by selecting it and pressing F5 (yes, also this operation seems to be necessary), and this time you should be able to open it with the “EMF Generator” editor. We can go on exporting the Xcore file.
We want the files generated by Xcore to be put into the emf-gen source folder; so we add a new source folder to our project, say emf-gen, where all the EMF classes will be generated; we also make sure to include such folder in the build.properties file.
First, we create an .xcore file starting from the generated .genmodel file:
navigate to the HelloXcore.genmodel file (it is in the directory model/generated)
right click on it and select “Export Model…”
in the dialog select “Xcore”
The next page should already present you with the right directory URI
In the next page select the package corresponding to our DSL, org.xtext.example.helloxcore.helloxcore (and choose the file name for the exported .xcore file corresponding Helloxcore.xcore file)
Then press Finish
If you get an error about a missing EObjectDescription, remove the generated (empty) Helloxcore.xcore file, and just repeat the Export procedure from the start, and the second time it should hopefully work
The second time, the procedure should terminate successfully with the following result:
The xcore file, Helloxcore.xcore has been generated in the same directory of the .genmodel file (and the xcore file is also opened in the Xcore editor)
A dependency on org.eclipse.emf.ecore.xcore.lib has been added to the MANIFEST.MF
The new source folder emf-gen is full of compilation errors
Remember that the model files will be automatically generated when you modify the .xcore file (one of the nice things of Xcore is indeed the automatic building).
Fixing the Compilation Errors
These compilation errors are expected since Java files for the model are both in the src-gen and in the emf-gen folder. So let’s remove the ones in the src-gen folders (we simply delete the corresponding packages):
After that, everything compile fines!
Now, you can move the Helloxcore.xcore file in the “model” directory, and remove the “model/generated” directory.
Modifying the mwe2 workflow
In the Xtext grammar, HelloXcore.xtext, we replace the generate statement with an import:
The DirectoryCleaner fragment related the “model” directory should be removed (otherwise it will remove our Helloxcore.xcore file as well); and we don’t need it anymore after we manually removed the generated folder with the generated .ecore and .genmodel files.
Then, in the language part, you need to loadResource the XcoreLang.xcore, the Xbase and Ecore ecore and genmodel, and finally the xcore file you have just exported, Helloxcore.xcore.
We can comment the ecore.EMFGeneratorFragment (since we manually maintain the metamodel from now on).
The MWE2 files is now as follows (I highlighted the modifications):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
...
Workflow{
bean=StandaloneSetup{
scanClassPath=true
platformUri="${runtimeProject}/.."
// The following two lines can be removed, if Xbase is not used.
// fragment = ecore.EMFGeneratorFragment auto-inject {}
...
Before running the workflow, you also need to add org.eclipse.emf.ecore.xcore as a dependency in your MANIFEST.MF.
We can now run the mwe2 workflow, which should terminate successfully.
We must now modify the plugin.xml (note that there’s no plugin.xml_gen anymore), so that the org.eclipse.emf.ecore.generated_package extension point contains the reference to the our Xcore file:
As we saw in the previous post, Junit tests do not work anymore with errors of the shape
1
org.eclipse.xtext.parser.ParseException:java.lang.IllegalStateException:Unresolved proxy http://www.xtext.org/example/helloxcore/HelloXcore#//Hello. Make sure the EPackage has been registered.
All we need to do is to modify the StandaloneSetup in the src folder (NOT the generated one, since it will be overwritten by subsequent MWE2 workflow runs) and override the register method so that it performs the registration of the EPackage (as it used to do before):
We can now customize our metamodel, using the Xcore editor.
For example, we add the interface Element, with the method getName() and we make both Hello and Greeting implement this interface (they both have getName() thus the implementation of the interface is automatic).
1
2
3
4
5
6
7
8
9
10
11
12
interfaceElement{
op StringgetName()
}
classHelloextendsElement{
Stringname
}
classGreetingextendsElement{
Stringname
contains XExpression expression
}
Using the Xcore editor is easy, and you have content assist; as soon as you press save, the Java files will be automatically regenerated:
We also add a method getElements() to the Model class returning an Iterable<Element>(containing both the Hello and the Greeting objects). This time, with Xcore, it is really easy to do so (compare that with the procedure of the previous post, requiring the use of EAnnotation in the Ecore file), since Xcore uses Xbase expression syntax for defining the body of the operations (with full content assist, not to mention automatic import statement insertions). See also the generated Java code on the right:
And now we can implement the validator method checking duplicates, using the new getElements() method and the fact that now both Hello and Greeting implement Element:
When you use Xtext for developing your language the Ecore model for the AST is automatically derived/inferred from the grammar. If your DSL is simple, this automatic meta-model inference is usually enough. However, there might be cases where you need more control on the meta-model and in such cases you will want to switch from an inferred Ecore model to a an imported one, which you will manually maintain. This is documented in the Xtext documentation, and in some blog posts. When I needed to switch to an imported Ecore model for Xsemantics, things have not been that easy, so I thought to document the steps to perform some switching in this tutorial, using a simple example. (I should have talked about that in my Xtext book, but at that time I ran out of pages so there was no space left for this subject 🙂
So first of all, let’s create an Xtext project, org.xtext.example.hellocustomecore, (you can find the sources of this example online at https://github.com/LorenzoBettini/Xtext2-experiments); the grammar of the DSL is not important: this is just an example. We will first start developing the DSL using the automatic Ecore model inference and later we will switch to an imported Ecore.
The grammar of this example is as follows (to make things more interesting, we will also use Xbase):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
grammar org.xtext.example.hellocustomecore.HelloCustomEcore with
With this DSL we can write programs of the shape (nothing interesting, this is just an example)
1
2
3
4
5
Hellofoo!
Greetingbar{
sayFoo()+"bar"
}
Now, let’s say we want to check in the validator that there are no elements with the same name; since both “Hello” and “Greeting” have the feature name, we can introduce in the Ecore model a common interface with the method getName(). OK, we could achieve this also by introducing a fake rule in the Xtext grammar, but let’s switch to an imported Ecore model so that we can manually modify that.
Switching to an imported Ecore model
First of all, we add a new source folder to our project (you must create it with File -> New -> Source Folder, or if you create it as a normal folder, you then must add it as a source folder with Project -> Properties -> Lava Build Path: Source tab), say emf-gen, where all the EMF classes will be generated; we also make sure to include such folder in the build.properties file:
1
2
3
4
5
6
7
8
source..=src/,\
src-gen/,\
xtend-gen/,\
emf-gen/
bin.includes=model/,\
META-INF/,\
.,\
plugin.xml
Remember that, at the moment, the EMF classes are generated into the src-gen folder, together with other Xtext artifacts (e.g., the ANTLR parser):
Xtext generates the inferred Ecore model file and the GenModel file into the folder model/generated
This is the new behavior introduced in Xtext 2.4.3 by the fragment ecore.EMFGeneratorFragment that replaces the now deprecatedecore.EcoreGeneratorFragment; if you still have the deprecated fragment in your MWE2 files, then the Ecore and the GenModel are generated in the src-gen folder.
Let’s rename the “generated” folder into “custom” (if in the future for any reason we want to re-enable Xtext Ecore inference, our custom files will not be overwritten):
NOTE: if you simply move the .ecore and .genmodel file into the directory model, you will not be able to open the .ecore file with the Ecore editor: this is due to the fact that this Ecore file refers to Xbase Ecore models with a relative path; in that case you need to manually adjust such references by opening the .ecore file with the text editor.
From now on, remember, we will manually manage the Ecore file.
Now we change the GenModel file, so that the EMF model classes are generated into emf-gen instead of src-gen:
We need to change the MWE2 file as follows:
Enable the org.eclipse.emf.mwe2.ecore.EcoreGenerator fragment that will generate the EMF classes using our custom Ecore file and GenModel file; indeed, you must refer to the custom GenModel file; before that we also run the DirectoryCleaner on the emf-gen folder (this way, each time the EMF classes are generated, the previous classes are wiped out); enable these two parts right after the StandaloneSetup section;
Comment or remove the DirectoryCleaner element for the model directory (otherwise the workflow will remove our custom Ecore and GenModel files);
In the language section we load our custom Ecore file,
and we disable ecore.EMFGeneratorFragment (we don’t need that anymore, since we don’t want the Ecore model inference)
The MWE2 files is now as follows (I highlighted the modifications):
Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
...
Workflow{
bean=StandaloneSetup{
scanClassPath=true
platformUri="${runtimeProject}/.."
// The following two lines can be removed, if Xbase is not used.
Now we’re ready to run the MWE2 workflow, and you should get no error (if you followed all the above instructions); you can see that now the EMF model classes are generated into the emf-gen folder (the corresponding packages in the src-gen folders are now empty and you can remove them):
We must now modify the plugin.xml (note that there’s no plugin.xml_gen anymore), so that the org.eclipse.emf.ecore.generated_package extension point contains the reference to the new GenModel file:
That’s because the generated StandaloneSetup does not register the EPackage anymore, see the diff:
All we need to do is to modify the StandaloneSetup in the src folder (NOT the generated one, since it will be overwritten by subsequent MWE2 workflow runs) and override the register method so that it performs the registration of the EPackage:
We can now customize our Ecore model, using the Ecore editor and the Properties view.
For example, we add the interface Element, with the method getName() and we make both Hello and Greeting implement this interface (they both have getName() thus the implementation of the interface is automatic).
We also add a method getElements() to the Model class returning an Iterable<Element> (containing both the Hello and the Greeting objects)
and we implement that method using an EAnnotation, using the source “http://www.eclipse.org/emf/2002/GenModel” and providing a body
Let’s run the MWE2 workflow so that it will regenerate the EMF classes.
And now we can implement the validator method checking duplicates, using the new getElements() method and the fact that now both Hello and Greeting implement Element:
Modified the SWTBot test so that it can be reused also in a test suite (see the comments to this post).
I happened to give a lecture at the University of Florence on Test Driven Development; besides the standard Junit tests I wanted to show the students also some functional tests with SWTBot. However, I did not want to introduce Eclipse views or dialogs, I just wanted to test a plain SWT application with SWTBot.
In the beginning, it took me some time to understand how to do that (I had always used SWTBot in the context of an Eclipse application); thanks to Mickael Istria, who assisted me via Skype, it ended up being rather easy.
assertResultGivenInput("foo","Not a valid input");
}
@Test
publicvoidtestNonValidInput(){
assertResultGivenInput("-1","Not a valid input");
}
}
There are a few things to note in the abstract base class:
You need to spawn the application in a new thread (the bot will run in a different thread)
You must start the application before creating the bot (otherwise the Display will be null)
after that you can simply use SWTBot API as you’re used to.
Note that the thread will create our window and then it will enter the event loop; this thread synchronizes with the @Before method (executed before each test), which creates the SWTBot (using the shell created by the thread). The @After method (executed after each test), will close our window, so that each test is independent from each other. The thread executes in an infinite loop, thus as soon as the shell is closed it will create a new one, etc.
Of course, this must be executed as a “Junit test”, NOT as a “Plug-in Junit test”, neither as a “SWTBot Test”, since we do not want any Eclipse application while running the test:
In the sources of the example you can find also the files to run the tests headlessly with Buckminster or with Maven/Tycho. Just enter the directory mathutils.build and
During the headless run, first the Junit tests for the implementation of the factorial will be executed (these are not interesting in the context of SWTBot) and then the SWTBot tests will be executed.
Up to now, I was always putting the Xtend generated Java files in my git repositories (for my Xtext projects), since I still hadn’t succeeded in invoking the Xtend standalone compiler in a Buckminster build. Dennis Hübner published a post with some hints on how to achieve that, but that never worked for me (and apparently it did not work for other users).
After some experiments, it seems I finally managed to trigger Xtend compilation in Buckminster builds, and in this post I’ll show the steps to achieve that (I’m using an example you can find on Github).
The main problems I had to solve were:
how to pass the classpath to the Xtend compiler
how to deal with chicken-and-egg problems (dependencies among Java and Xtend classes).
IMPORTANT: the build process described here uses a new flag for the Buckminster’s build command, which has been recently added; thus, you must make sure you have an updated version of Buckminster headless (from 4.3 repository).
The steps to perform can be applied to your projects as well; they are simple and easy to reproduce. In this blog post I’ll try to explain them in details.
This blog post assumes that you are already familiar with setting up a Buckminster build.
The example
The example I’m using is an Xtext DSL (just the Greeting example using Xbase), with many .xtend files and with the standard structure:
org.xtext.example.hellobuck, the runtime plugin,
org.xtext.example.hellobuck.ui, the ui plugin, which uses Xtend classes defined in the runtime plugin,
org.xtext.example.hellobuck.tests, the tests plugin, which uses Xtend classes defined in the runtime and in the ui plugin,
org.xtext.example.hellobuck.sdk, the SDK feature for the DSL.
Furthermore, we have two additional projects created by the Xtext Buckminster Wizard:
org.xtext.example.hellobuck.buckminster, the releng project,
org.xtext.example.hellobuck.site, the feature project for creating the p2 repository,
Creating a launch configuration for the Xtend compiler
The first step consists in creating a Java launch configuration in the runtime plugin project that invokes the Xtend standalone compiler. This was shown in Dennis’ original post, but you need to change a few things. Here’s the XtendCompiler.launch file to put in the org.xtext.example.hellobuck runtime plugin project (of course you can call the launch file whateven you want):
This launch configuration can be reused in other projects, provided the highlighted lines are changed accordingly, since they refer to the containing project.
An important part of this launch configuration is the PROGRAM_ARGUMENTS that are passed to the Xtend compiler, in particular the -classpath argument. This was the main problem I experienced in the past (and that I saw in all the other posts in the forum): the Xtend compiler needs to find the Java classes your Xtend files depend upon and thus you need to pass a valid -classpath argument. But we can simply reuse the classpath of the containing project 🙂
This launch configuration calls the Java application org.eclipse.xtend.core.compiler.batch.Main thus you must add a dependency on the corresponding bundle in your MANIFEST.MF. The bundle you need to depend on is org.eclipse.xtend.standalone (the dependency can be optional):
Test the launch in your workbench
You can test this launch configuration from Eclipse, with Run As => Java Application. In the Console view you should see something like:
This will give you confidence that the launch configuration works correctly and that all dependencies for invoking the Xtend compiler are in place.
Add an XtendCompiler.launch in the other projects
You must now add an XtendCompiler.launch in all the other projects containing Xtend files. In our example we must add it to the ui and the tests projects.
You can copy the one you have already created but MAKE SURE you update the relevant 3 parts according to the containing projects! See the highlighted lines above.
NOTE: you do NOT need to add a dependency on org.eclipse.xtend.standalone in the MANIFEST.MF of the ui and tests projects: they depend on the runtime plugin project which already has that dependency.
You may want to run the XtendCompiler.launch also in these projects from the Eclipse workbench, again to get confidence that you configured the launch configurations correctly.
IMPORTANT: when the Xtend compiler compiles the files in the ui and tests project, you will see some ERROR lines, e.g.,
From what I understand, these errors do not prevent the Xtend compiler to successfully generate Java files (see the final INFO line) and the procedure terminates successfully. Thus, you can ignore these errors. If the Xtend compiler really cannot produce Java files it will terminate with a final error.
Configure the headless build
Now it’s time to configure the Buckminster headless build so that it runs the Xtend compiler. We created .launch files because one of the cool things of Buckminster is that it can seamlessly run the launch files.
The tricky part here is that since we perform a clean build, there is a chicken-and-egg scenario
no Java files have been compiled,
most Java files import Java files created by Xtend
the Xtend files import Java classes
To solve these problems we perform an initial clean build; this will run the Java compiler and such compilation will terminate with errors. We expect that, due to the chicken-and-egg situation. However, this will create enough .class files to run the Xtend compiler! It is important to run the build command with the (new) flag –continueonerror, otherwise the whole build will fail.
After running XtendCompiler.launch in the org.xtext.example.hellobuck runtime project, we run another build –continueonerror so that the Java files generated by the Xtend compiler will be compiled by Java. We then proceed similarly for the ui and the tests project:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
clean
# this first Java build will fail with compilation errors, since Xtend
# classes have not been compiled yet. However, it will compile some
# Java classes so that we will be able to compile Xtend classes
Then, your build can proceed as usual (at this point I prefer to perform a clean build): run the tests (both plain Junit and Plug-in Junit tests) and create the p2 repository:
You can now run your headless build on your machine or on Jenkins.
My favorite way of doing that is by using an ANT script. The whole ANT script can be found in the example.
This script also automatically installs Buckminster headless if not present.
Before executing the commands file, it also removes the contents of the xtend-gen folder in all the projects; this way you are sure that no stale generated Java files are there.
Of course, you should now remove the xtend-gen folder from your git repository (and put it in the .gitignore file).
In Jenkins you can configure an Invoke Ant build step as shown in the screenshot (“Start Xvfb before the build, and shut it down after” is required to execute Plug-in Junit tests; we also pass an option to install Buckminster headless in the job’s workspace).
As noted above, this will also install Buckminster headless if not found in the location specified by the property buckminster.home. This script will take some time, especially the first time, since it will materialize the target platform.
Updated the steps for entering download mode so that Odin can detect your phone.
I’ve always wanted to update my Samsung Galaxy Wonder to Android Jelly Bean; lately my cellphone became quite slow (especially after the latest upgrades to Android 2.3 from Samsung) and I always wanted to install Chrome and Google Keep that both require Android 4. Samsung does not provide any official release of Android 4, so I decided to go for a custom ROM. In particular, I’m using Android Jelly Bean, 4.2.2, CyanogenMod 10.1 ALPHA (Build 7).
I had quite a hard time to understand how to install it; basically because I had never installed a custom ROM. Moreover, the instructions to do that can be found on the web but I never found a complete tutorial that shows the procedure from the very start: they all assume that you have already done previous steps.
Then, I decided to write a complete tutorial (this is based on Windows). This assumes you have an external sdcard on the phone.
The update will wipe out all your data, so make sure you backup them first. Proceed at your own risk. You will also lose Samsung warranty.
Make sure you read the whole tutorial first, before proceeding.
Disclaimer: All the tools, mods or ROMs mentioned below belong to their respective owners/developers. I am not to be held responsible if you damage or brick your device.
USB Drivers
Make sure you can connect your Android phone with the computer. If not, install the USB drivers for Samsung Galaxy W properly. The easiest way is to install Samsung Kies.
Now you need to put your phone it into download mode, in order to upload the clockworkmod recovery file:
turn off phone,
hold Volume Down + Home + Power Button for a while till the phone turns on.
the phone will turn on and show some screen,
plug in usb cable
press Volume Up
At this point Odin should detect the connected phone
In Odin press the “Start” button, and the downloading should start (see the phone):
Wait for the download to finish (see Odin)
Now you can unplug and turn-off the phone.
Create a backup of the current image
Turn on the phone in the Recovery Mode: hold Volume Up + Home + Power Button for a while till the phone turns on (Note: this time it is “Volume Up”, not “Volume Down”). Release power button as soon as samsung logo appears and hold your volume up button + home button until clockworkmod recovery appears on the screen
To use the menus:
Volume buttons to move in the menu
Home button to select a menu
Power button to go back
Select “backup and restore” and then “backup to external sdcard”:
and wait for the backup to complete
You can now reboot the phone from the main screen. The phone will reboot as usual.
You may want to store a copy of the backup in your computer hard disk; just connect the phone with USB, and navigate to the directory where the backup was saved on the phone sdcard:
Put these files in the root folder of your external SD card (you can turn on the phone as usual and connect it via USB for that, or copy these files from the computer using an external card reader).
When the copy finished, disconnect the phone and turn it off.
Switch ON the phone in the Recovery Mode: pressing and holding Volume Up + Home + Power buttons together.
Now wipe data and cache selecting the following commands (and wait for them to complete)
Select “wipe data/factory reset”
Select “wipe cache partition”
Select “advanced” and then select “wipe dalvik cache”
Now go back to the main menu (Recall: use the power button to go back)
Select “install zip from sdcard” and choose the zip file containing the OS (in this case it is cm-10.1-20130611-EXPERIMENTAL-ancora-alpha7.zip) from the root of the SD card (where you previously copied it).
Now do the same for the google apps zip file: select “install zip from sdcard” and choose the zip file containing the OS (in this case it is gapps-jb-20130301-signed.zip) from the root of the SD card.
Reboot into the new system
Now you’re ready to reboot into the new system from the main menu!
NOTE: Your Phone will boot now and it might take about 5 minutes to boot on your first time. So, please wait.
If everything went fine, you should see the new logo
Then you should see all the menu screens for configuring the phone!
Note: as for me, this procedure started with an error message saying that the vocal synthesis engine crashed, but I simply ignored the message and went on.
After you inserted your Google account, the phone should be ready; at this point, if you selected an Internet connection, all the applications you had previously installed from Google Play should be installed automatically… it took about an hour in my case.
First Impressions
The system seems rather stable and surely more responsive than before!
Battery usage seems to have increased, especially when connected with WIFI.
All in all, I’m very happy of the new system. 🙂
External Sources
These are all the links where I found the software and information I based this tutorial on.
I know that Xtext 2.4 has not been released yet, but I could not resist blogging about a very cool new feature in Xbase: improved automatic import functionalities!
Actually, import functionalities were already great when using Xbase also in previous versions of Xtext, but now they provide a much better experience for the user of your DSL! Indeed, all the import functionalities you are used to with JDT (like automatic import insertion, and organize imports) are available also for your Xbase language; these features were already available in Xtend, and they have been ported to Xbase itself.
At the time of writing, you need to get the very latest updates of Xtext 2.4, using the update site http://download.eclipse.org/modeling/tmf/xtext/updates/composite/latest/ .
Before you used to do something like
1
2
3
4
5
6
7
8
9
10
11
12
13
14
grammar org.xtext.example.helloinferrer.HelloInferrer with
If you now rerun the MWE2 generator, and make sure you merge the plugin.xml_gen with plugin.xml in the .ui project, your editor will provide some interesting features for free (if you use my examples, you can find a project wizard “New Project” => “Xtext” => “HelloInferrer Project”):
Imports with wildcards are deprecated:
You now have the context menu “Organize Imports” (Shift + Control + O); try that one in the presence of such deprecation warning and imports are organized for you:
Similarly, unused imports are reported as warnings:
Again, use “Organize Imports” to fix that!
The new feature I like most is the automatic insertion of imports! (just like in JDT and Xtend): try to get content assist for a Java type, for instance,
Accept a proposal and the import will be automatically inserted (instead of the fully qualified name):
Updated listings to reflect the git repository sources. Put a tip on using a mirror aggregated with b3.
In this tutorial I’ll show how to use Buckminster to build an Eclipse RCP Product, both in the IDE and headlessly (with ant). The application I’m building is the standard Eclipse Mail RCP example with the addition of Self-Update functionalities.
We will build two products configured with update sites; the first one will rely on standard Eclipse repositories for required features, while the second one will rely only on our own repositories.
There are some nice tutorials about building Eclipse RCP Products with Buckminster (such as, e.g., Ralf Ebert‘s, Code and Me‘s, and wiki pages).
However, I found these pages out-of-date in the sense that they use Indigo or Helios for building products; with Juno things are more complicated not due to Buckminster, but to new dependencies in Juno Eclipse features and bundles (for instance, org.eclipse.rcp internally depends on org.eclipse.emf.common and org.eclipse.emf.ecore) even if you do not use the new e4 application model; see for instance the dependencies in the screenshot
This means that you will have to deal with that in the target platform definition.
Furthermore, due to the way p2 repositories are built, you will soon get the dreaded “java returned 13” when using Buckminster to build an Eclipse product (which relies on the p2 director actually), due to the above mentioned dependencies. Even for simple products like the Eclipse RCP Mail application… you can imagine when products are bigger 😉 By the way, you get similar problems even if you try to use the standard Eclipse Product export wizard.
In this post I’ll detail my experience in dealing with these problems by using Buckminster and Eclipse standard mechanisms for dealing (automatically) with dependencies; similarly, I’m not using standard Target platform definitions (which again have problems if you want to build for multiple architectures), but I’m using the nice Buckminster materialization features for materializing the target platform. The same techniques can be used with much more complex products to build them without problems due to dependencies and required software.
The tutorial is quite long since I’ll also try to provide some explanations to the problems you have when building products (in general I guess) – although the explanations are not necessary, I think they might be useful to understand things better about features, bundles, products and p2.
Materializing the Target Platform
First of all, you need to install Buckminster in your Eclipse, using this repository
you will need only the features shown in the screenshot
Then, we need to create a project which with all our releng functionalities; this will be a general Eclipse project, with a Buckminster Component Specification (CSPEC); this CSPEC basically declares the target platform features as dependencies. The reasons why I’m not using a standard Eclipse Target platform definition can be found in my other post, in the section “Why not the Target Editor?”; they can be summarized with the fact that, with this technique you can get a target platform for building for multiple platforms and with all the required software automatically.
We call this releng project org.eclipse.buckminster.examples.rcp.mail.releng and the buckminster.cspec looks like this (we will enrich this cspec later to perform headless builds):
We basically want a target paltform with org.eclipse.rcp (and its sources, since they are useful when developing), org.eclipse.equinox.executable to build executable applications, and org.eclipse.equinox.p2.user.ui to enable the p2 update manager in our RCP application.
Then, we define a Resource Map (RMAP) which tells Buckminster where to find these dependencies; we will use of course official Eclipse p2 site repositories for these dependencies; we store this map into a file build.rmap:
all the components with name which starts with org.eclipse.buckminster.examples.rcp.mail (which will be used for all our bundles and features in this example) from the local hard disk
and everything else from the main Juno releases repository.
Now, we define a Component Query (CQUERY) which materializes this very component; when Buckminster resolves a component if first, transitively, resolves all its dependencies; thus resolving our releng component corresponds to materialize our target platform. The build.cquery looks like this (note the reference to the build.rmap we wrote above):
Before starting the materialization, it is better to start from a plain and empty target platform (just like you do with a standard target definition with the target editor); one nice way of doing this, as illustrated also here, is
Create a new general project named TP (or some name of your preference) in the workspace
In “Window” => “Preferences” =>“Plug-in Development” =>“Target Platform” Select Add…
Start with an empty target definition
Enter TP in the Name: field (or some name of your preference)
Add a directory
Click on “Variables…” scroll down and select “workspace_loc” and then type TP in the Argument: field
Press “Ok” and “Finish” twice, and
set this as the active platform
With the build.cquery opened in Component Query Editor, we can start the materialization by pressing “Resolve and Materialize”
This might take some time depending on your Network connection.
(TIP: you may want to aggregate a local mirror using Eclipse b3; the aggregator file for the mirror is in aggregator/target-platform-mirror.b3aggr ; you can aggregate the mirror from Eclipse, after installing b3, or headlessly using the target “b3_aggregation” of build.ant. Then, you can use the build-local.cquery you find in the releng project in the git repository; see also the README.txt file you find in the releng project).
When the materialization finishes you find the target platform in the TP project in your workspace (if you followed the instructions above).
TIP: after a materialization of the target platform, it might be better to restart Eclipse, since sometimes Buckminster tends not to catch up correctly with the new target platform.
Creating our mail projects
We now create the bundle for our RCP application, using the Eclipse wizard and the RCP Mail template; this part is standard so I will not detail it here: org.eclipse.buckminster.examples.rcp.mail.bundle contains the RCP Mail bundle created with the wizard, and appropriately modified in order to enable the same update UI functionalities used in the SDK inside our RCP app (this is illustrated in this wiki page, the modifications to ApplicationWorkbenchWindowAdvisor and ApplicationActionBarAdvisor are marked in the code with ‘XXX’ task tags). We also create org.eclipse.buckminster.examples.rcp.mail.optional.bundle which contains an optional menu (and toolbar button).
We then create the feature project org.eclipse.buckminster.examples.rcp.mail.product.feature which contains our product definition:
In this project we create a new product definition, mail.product, based on features (make sure that there is no <plugins> section in your feature-based product: open mail.product with the Text editor and delete the <plugins> section if found)
Give the product an ID (which must be different from the ID of the containing feature), org.eclipse.buckminster.examples.rcp.mail.product
In the Product definition section choose the Product org.eclipse.buckminster.examples.rcp.mail.bundle.product from Application org.eclipse.buckminster.examples.rcp.mail.bundle.application (don’t get confused by the term ‘product’ which is overloaded in this context: it is used both for the org.eclipse.core.runtime.products extension point, and for the product configuration which will be used to create the final product 🙂
In the Dependencies tab, add as the only dependency the feature we are in, org.eclipse.buckminster.examples.rcp.mail.product.feature. (We also do some branding and customizations, but they’re not interesting in this context).
Now, we have to “fill” the feature.xml of org.eclipse.buckminster.examples.rcp.mail.product.feature, keeping in mind that what’s in this feature will make our final product. Thus we add:
org.eclipse.buckminster.examples.rcp.mail.bundle, in the plug-in section, for our Mail RCP application
org.eclipse.rcp, as included feature, required to build an RCP product
org.eclipse.equinox.p2.user.ui, as included feature, to enable the p2 update manager in our RCP application
Note that org.eclipse.buckminster.examples.rcp.mail.optional.bundle is NOT part of product.feature, since this is meant to represent an optional functionality that can be installed later
We can create a launch configuration (Eclipse Application) by selecting our product, and then select only our feature org.eclipse.buckminster.examples.rcp.mail.product.feature and then press “Select Required”.
The mail application should like this (note the Preferences menu and the Update functionalities)
In org.eclipse.buckminster.examples.rcp.mail.product.feature we also add a touchpoint advice file p2.inf (See the online help for more details) to configure the repositories (update sites) that should initially be present in the application:
1
2
3
4
5
6
instructions.configure=org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:0,name:Buckminster Mail RCP Example Site,enabled:true); \
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:1,name:Buckminster Mail RCP Example Site,enabled:true); \
We add the Juno release repository, the Orbit repository (not required, but just as a demonstration), and the repository on Sourceforge, where we deploy our p2 repository for our application (see the next section).
This file will be used during product build (see later).
Build a p2 Repository for our application
To build an update site (in the new terminology, “p2 repository”) for our product feature, we create a new feature project, org.eclipse.buckminster.examples.rcp.mail.product.site, and in the feature.xml we include our product feature (org.eclipse.buckminster.examples.rcp.mail.product.feature); we also create org.eclipse.buckminster.examples.rcp.mail.optional.feature (which includes org.eclipse.buckminster.examples.rcp.mail.optional.bundle) and we also include this feature in product.site; this way, once the repository is deployed, the optional functionalities can be installed in an existing mail application.
Since we’d like to have a category for our features, we add a category.xml file like the following one:
<category-def name="mail.category"label="Buckminster Mail RCP Example">
<description>
Buckminster Mail RCP Example
</description>
</category-def>
</site>
Remember: when building the site.p2 on a feature project with Buckminster, you will build a p2 repository NOT for the very feature, but for the included features.
Before creating the repository we use a properties file like the following to specify additional parameters for the repository creation:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# This can be used only to build a site.p2, not a product
# since we can only build a site for multiple architectures
how the .qualifier in bundles and features version is replaced (in this example we replace it with a timestamp of the latest changed resource, with some formatting)
by default, Buckminster will generate also source features and source bundles when creating the p2 repository, but for a product it might not make sense to make sources installable, thus we disable the generation of sources
we create a repository for all supported architectures and operating systems
We can now run on this project the site.p2 Buckminster action: right click on the feature project => Buckminster => Invoke Action…, select the properties file above and select site.p2.
The p2 repository will be generated (if you used the above properties file) into <your home>/tmp/mail/buckminster.output/org.eclipse.buckminster.examples.rcp.mail.product.site_1.0.0-eclipse.feature/site.p2 ; you can test your newly created repository by using the Install New Software dialog in a running Eclipse, specifying the complete local path of the created site.p2
The generated repository can then deployed to a remote site (in our example we deploy it to Sourceforge where we host also this tutorial code: http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates (if you want to browse it, use http://sourceforge.net/projects/buckyexamples/files/bucky-mail-rcp/updates/).
Building the product
Buckminster does not provide direct means to build a product, since it relies on the standard Eclipse org.eclipse.equinox.p2.director application, you just need to set up a few more files, with pretty standard contents.
a repository (or a list of repositories separated by commas),
the ID of an installable unit,
a profile,
the architecture details
and a destination folder,
the director will create/install (provision) in that destination folder the requested product.
For instance, try to run the following command, replacing the path of your eclipse executable (in Windows you should use the command line version, eclipsec.exe), the destination path and the architecture details for your system
and you’ll get a brand new Eclipse SDK from the Kepler site.
So what we will do to build our product is
first create a p2 repository with all the features and bundles needed by our RCP application
and then run the p2 director with appropriate arguments (relying on the p2 repository we created).
Setting up a project for building the product
Since we have already a feature project for building the p2 repository, org.eclipse.buckminster.examples.rcp.mail.product.site, we will use it also for building the product.
Inside this project we create a folder (build) with the following product.ant file which calls the p2.director ant task (in the following we will provide also some explanations):
For each plugin and feature projects Buckminster automatically infers a Component Specification (you can view that by right clicking on the project => Buckminster => View CSpec…) with its standard actions; we must extend this specification with additional actions to create the product, creating inside our feature project a buckminster.cspex file (note cspex, with the final x instead of c):
This cspex extends the default actions of our feature projects with the new actions create.product and create.product.zip; note that the latter depends on the former and simply creates a zip of the directory containing the generated product. The create.product action calls the ant task with the same name in product.ant passing the profile and the IU of our product configuration, i.e., the one we chose above during the creation of product configuration, org.eclipse.buckminster.examples.rcp.mail.product in our example (this is the value we inserted for the field ID in the “General Information” section of the product configuration editor).
The action create.product has as prerequisite the (standard Buckminster) action site.p2 (since the component name is not specified, the component we are in is assumed); the alias of site.p2 action, repositories, will be passed to the product.ant; indeed, what is being passed to the ant file is the path resulting from site.p2, in the shape of fs:repositories, which is a path group (you can read more in the Eclipse Buckminster, The Definitive Guide, Section “Access to prerequisites and product locations” – in product.ant, we extract the actual path and do some conversion for Windows backslashes so that we obtain a valid URL for the director). We also pass a name for the profile of our product, the operating system details and the destination.
Summarizing, the steps to create the product with the p2 director are:
create a p2 repository for this feature (actually, for the features and bundles included in this feature)
run the director application specifying our product identifier as the installable unit, and the above created p2 repository (together with the other parameters for the director)
Before running the create.product action in the IDE we must specify a property file (similar to the one we used for mail.site project): creating the product will not work with * for os/ws/arch, because you can create the product only for a specific platform at time. For instance, this is the one I use for my Linux system
You can then right click on org.eclipse.buckminster.examples.rcp.mail.product.site, select the create.product action, and the property file for the architecture you want to build your product for
In the console view you can see that first the site.p2 for the current feature project is created and then the director is called to install the product… but… you get this error:
1
2
3
4
5
6
7
8
9
10
[ant]Cannot complete the install because one ormore required items could notbe found.
[ant]Software being installed:Mail RCP Product1.0.0.qualifier
To understand what is going on, let’s do another digression
Small digression about the p2 publisher
When a p2 repository is generated for a feature, the repository will contain:
the included features (the tab “Included Features” in the feature editor)
the plug-ins and fragments (the tab “Plug-ins” in the feature editor)
but NOT the required software (required features and required bundles)
The required plug-ins and features can be made explicit in the “Dependencies” tab in the feature editor; even if they are not specified, the dependencies are computed automatically and they are still needed by the director when the product is installed.
Since these dependencies will not be part of the generated p2 repository, we get an error when installing the product…
Back to our product building
In our case we get an error about the feature org.eclipse.emf.common which is required by org.eclipse.e4.rcp. The feature org.eclipse.rcp (which is included in our product.feature) INCLUDES org.eclipse.e4.rcp, but org.eclipse.e4.rcp REQUIRES (not includes) org.eclipse.emf.common (and also org.eclipse.emf.ecore):
Thus, org.eclipse.emf.common will be not present in the generated p2 repository and the installation fails. With Indigo, this problem was not experienced, since org.eclipse.rcp was self-contained (and indeed, Ralf Ebert’s tutorial did not experience this problem). You would experience similar problems if your product is more involved than the simple Mail example application and requires more features and bundles: in that case the dependencies are much more.
In the following, I’ll describe 3 possible solutions to deal with that. The third one is (in my humble opinion) the best one, and is the one I’ll use in this tutorial.
Pass additional repositories to the director
We can pass the director (in our case we must modify the product.ant) additional p2 repositories, i.e., the ones we used in the RMAP, that we used to materialize the target platform (in our example the Juno release repository and the Orbit repository).
The drawbacks of this approach is that you depend on remote sites each time you build the products (you can use local mirrors though) and most of all, you do not have control on what is taken from which repository, i.e., you do not have the same control you had when defining the target platform; thus, you risk to build a product which does not use the same things of your target platform. This is especially true when building complex products with a complex target platform taking different software from several different repositories.
Fix the feature manually with missing requirements
You can add as included features into your product.site feature project (NOT the product.feature project) the missing features one by one; this way, when the site.p2 is built the required features will go there as well, and the director will find them when installing the product.
There will surely be more than one missing feature, and the director will issue an error only on the first missing feature; thus, it will take some time to include them all.
Furthermore, including features will not be enough: you will surely have to add also bundles (plug-ins and possible fragments) to your product.site feature. For instance, org.eclipse.equinox.p2.user.ui (the one we added to our product to handle update functionalities) includes the plug-in org.eclipse.equinox.p2.ui.importexport,
which depends (i.e., requires, not includes) the bundle org.eclipse.ui.forms,
which, again, will not be part of our site.p2 unless explicitly added as a bundle!
Thus, it will take some time to have a site.p2 that makes the director happy 😉
Finally, I think that having all the dependencies hardcoded in the feature.xml makes the project highly coupled with that specific target environment (in Indigo the additional software might not even be available, in Kepler new requirements might have to be added)… why should I deal with dependencies myself?
Provision your target platform as a p2 repository
The idea is to have a p2 repository which contains EVERY feature and bundle of our current target platform (the one we materialized at the beginning); I had blogged about a manual technique, but I like to avoid manual solutions (for maintainability and portability reasons) and prefer automatic ones.
Indeed, all the features and bundles for your products are in your target platform (otherwise your bundles would not compile, or your product launch would not work); yes, even the ones you were not aware of, like org.eclipse.emf.common and org.eclipse.emf.ecore, since Buckminster has materialized them for you as dependencies of org.eclipse.rcp. We just need a way to create a p2 repository from the current target platform.
There is a p2 ant task for this, p2.publish.featuresAndBundles, which publishes metadata for pre-existing binary features and plug-ins. All we need to do is:
modify product.ant with a target which invokes p2.publish.featuresAndBundles with all the arguments, in particular, the current target platform as the source to create the p2 repository (we get its location using a Buckminster property):
add an action, site.tp, in buckminster.cspex that invokes the above ant target (the action could be made private, but in case you want to run it manually to do some tests we make it public); note that we give the repository of the target platform a name since we also deploy it as we will show later. Then, the action create.product will have as a prerequisite also the site.tp action, besides site.p2:
The repository for the target platform will be generated in the directory site.tp.
Now, if we run the create.product action again, the director will be provided both with the p2 repository of our product and the p2 repository of the target platform, and the director will be able to find everything it needs to install the product.
You should get no error now, and you can find your product ready to run in the output directory; if you used the properties file above, depending on the chosen architecture and OS, you will find the product in <your home>/tmp/mail/buckminster.output/org.eclipse.buckminster.examples.rcp.mail.product.site_1.0.0-eclipse.feature/BuckyRcpMail.<ws>.<os>.<arch> .
Deploying your sites
We have already seen that we can deploy our product.site on the net (in our case, it’s here); however, this will still require additional Eclipse repositories (for required software which is not part of our product repository). If this reminds you of the same issue we had when building the product you’re on the right track 😉
The site.tp was useful to build our product, but we could also deploy it on the Internet, http://master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates (if you want to browse it, use http://sourceforge.net/projects/buckyexamples/files/bucky-mail-rcp/cloud-updates/), together with our site.p2 to make our product independent from other repositories.
To show our “independence” we create another product, a “cloud” version, where the preconfigured update sites are only the ones we maintain: we create org.eclipse.buckminster.examples.rcp.mail.cloud.product.feature, which is basically the same as org.eclipse.buckminster.examples.rcp.mail.product.feature, with another product configuration, another product identifier and another touchpoint advice file p2.inf with only our mantained repositories (i.e., the standard update site, site.p2, and the cloud-updates, which is the one we get from the target platform, site.tp):
1
2
3
4
instructions.configure=org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:0,name:Buckminster Mail RCP Example Site,enabled:true); \
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/updates/,type:1,name:Buckminster Mail RCP Example Site,enabled:true); \
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates/,type:0,name:Buckminster Mail RCP Example Platform Site,enabled:true); \
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(location:http${#58}//master.dl.sourceforge.net/project/buckyexamples/bucky-mail-rcp/cloud-updates/,type:1,name:Buckminster Mail RCP Example Platform Site,enabled:true);
To deal with this product we have additional dedicated actions in the .cspex file of product.site (see the original source), namely create.cloud.product and create.cloud.product.zip, which basically rely on the same product.ant’s create.product target passing a different profile name and a different product UI.
You can also try to install our product from the command line, using the deployed remote repositories, with the same technique we showed for installing the Eclipse SDK from the command line; you just need to use one of the two product identifiers org.eclipse.buckminster.examples.rcp.mail.product or org.eclipse.buckminster.examples.rcp.mail.cloud.product, specify the path of your eclipse executable (in Windows you should use the command line version, eclipsec.exe), the destination path and the architecture details for your system, here are some examples: the first one installs the mail product for Linux 64bit, the second one installs the mail cloud product for Windows 64bit
NOTE: there will be redundancies in the deployed repositories since some features and bundles are contained both in site.p2 and site.tp; but this is just a tutorial example (and they do not disturb the installation/updates): it is up to you to choose what to deploy and how to aggregate them, in case.
Building Headlessly
For continuous integration, but also for having an automated headless way to build the products for multiple architectures and build the p2 repositories with only one click, it is very useful to have an Ant script that does all of the above. The build.ant is stored in the project org.eclipse.buckminster.examples.rcp.mail.releng, and it relies on the common.ant file which does most of the work. The Ant script relies on Buckminster headless; I won’t detail here how to install Buckminster headless, also because the Ant script checks whether it is already installed in your computer, and if it is not, it will install it for you 🙂 (it first installs Buckminster director whose zip file is in the tools subdirectory of the releng project). In the launches directory of the releng project you also find some launch configurations to experiment with.
Buckminster headless will need a way of importing the projects into the headless workspace to build them; the build.cquery we already described can be used to materialize in the workspace the releng project, but then the buckminster.cspec needs to be modified in order to materialize also the other projects. We just need to add dependencies to projects which depend on all of the other projects (since Buckminster will automatically materialize all dependencies). The feature project org.eclipse.buckminster.examples.rcp.mail.product.site depends on all the other ones. So we add this dependency (the rmap already provides all the locators for materializing all our projects). The final buckminster.cspec looks like this:
The build.ant script then invokes Buckminster headless with two text files containing the headless commands: headless-resolve-commands.txt to materialize all the projects and the target platform (by resolving build.cquery)
Before building the products with the create.product actions (and zip them) it is crucial to first build the product.site repository for all platforms; this way, we will then be able to build products for several platforms (if you don’t do that, only the first product will have executable files).
All the output will be stored in the buildroot directory in the same root of our projects; the target platform will be materialized in the subdirectory target.platform, the repositories and the products can be found in the subdirectory buckminster.output (each of them in the subdirectories we saw before).
Try the example
You can find the packaged products on Sourceforge. Download the one for your system (either the standard version or the “cloud” version); you will see the preconfigured update sites. Try to update the application with Check for Updates (there should be an update available for the product) and to install the optional feature.
Conclusions
I hope you found this tutorial useful 🙂
With the techniques described here you should be able to build even complex products seamlessly without having to deal explicitly with dependencies which are already dealt with by Eclipse PDE and p2. Once the target platform is defined correctly (and you see that with Buckminster you can do that without having to worry about required software), everything will be built automatically and your projects (especially product definitions and feature definitions) will be clean and neat 🙂
I’m quite a newbie in Ant, but I thought it would be straightforward to do a regular expression replacement in a string contained in a property; in my case, I had to replace Windows backslashes with Unix slashes… apparently, unless you use the propertyregex task from Ant Contrib, that is not supported out-of-the-box: you have to pass through a file!
This stackoverflow post shows a possible solution, and starting from that, I created a macrodef to make it easier
This macro takes the property value to process (property.to.process) and the property name where to store the result; it outputs the input value to a temporary file, reads it back with a regular expression replacement (which is supported for files) and store it in the specified property (the temporary file is then deleted).
In a previous post, I blogged about mirroring Eclipse repositories with the p2.mirror Ant task; Since this Ant task needs to be run via the Eclipse antRunner application, you need a full installation of Eclipse on the machine that will run the task and I showed how to run this task from the command line.
It might be more comfortable sometimes to run it from Eclipse; for this, you need to create a launch configuration which runs an Eclipse Application, the same you run from the command line: org.eclipse.ant.core.antRunner.
We create a new general project (say, my.eclipse.mirror) and we create a mirror.ant file with the following example content (see the previous post for more details):
org.eclipse.rcp.sdk, the eclipse launcher and the feature for adding update site features to your RCP, from the main Juno release site,
the whole Orbit repository (that version of the repository)
the whole Buckminster headless repository
Now we select the “Run Configurations…” dialog
and we create a new Eclipse Application launch configuration, which we call run_mirror; we choose, as the Program to Run, the application org.eclipse.ant.core.antRunner. Note that this application is installed by default in your SDK, but if there is a different target platform active in your workspace, which does not include the org.apache.ant bundle, this launch will fail.
In the Arguments tab, we must specify the -buildfile argument (expected by the runner) passing our ant file mirror.ant; this file is intended to be relative to the Working directory, thus we must modify it by selecting our project relative to the workspace:
Optionally, we save this launch configuration, using the “Common” tab, into our project
By default, the mirror will be saved into the folder eclipsemirror into your home folder; you may want to change this by specifying the additional program argument -Dtarget.dir=<path>, for instance,
Choose Apply and then Run; in your console view you should see that the mirror task has started
You can ignore possible problems of dependencies which cannot be satisfied
Depending on your Network connection it might take some time, but in the end you should see something like
And you can see that the mirror directory will mimic the original repositories structure
I used to have many Eclipse installations in my machines; typically they were different Juno versions downloaded from ecipse.org, for instance, Eclipse for RCP developers, Eclipse for DSL developers, Eclipse Modeling Tools, etc. Moreover, most of them were customized with the same plugins (for instance, Mylyn connectors) which I had to install on all of them. I preferred to have separate installations not to have a monolithic single Eclipse instance (where some features might also interfere with each other).
Then I started to use the ability of Eclipse to deal with multiple configurations, which is really a cool feature.
The idea is that you have a single Eclipse installation with all the features you always used and that you would desire in all of your Eclipse installations; then you have different directories for each “configuration”.
You can start Eclipse with a command line like the following, which uses the command line argument -configuration:
Assuming that the main Eclipse installation is in eclipse-main directory, and that the new separate configuration will be stored in eclipse-other/configuration (which will be automatically created if it does not yet exist). What you get is a running Eclipse instance with all the features and plugins of the main Eclipse installation, but all the new features which will be installed from this running instance will go in the new configuration, thus they won’t disturb the main installation!
If you try to install new software from this Eclipse running instance, you’ll see that the list of available software sites is empty, so you will have to fill such list with the typical Eclipse software sites, such as http://download.eclipse.org/releases/juno and http://download.eclipse.org/eclipse/updates/4.2.
And then you can install new features in this Eclipse, and they will be available only in this configuration. You can then check the plugins and features directories in eclipse-other which will contain the new installed features and bundles (which will not be stored in the same directories of eclipse-main); similarly, the plugins and features directories in eclipse-other will not contain the features and bundles which are stored in the same directories of eclipse-main, though they are available in the new Eclipse configuration.
Of course, you’ll have to use the above command line each time you want this Eclipse version (you should have shell scripts to run a specific Eclipse).
Main advantages in this approach are
The features you want to use in all configurations are stored in only one place, and they will be maintained only in the eclipse-main installation (e.g., kept up to date)
you save some space in your hard disk (I had 4 Eclipse installations which required 2.5 Gb; with the new approach, i.e., one Eclipse main installation and 3 configurations I only need 500 Mb!)
If you can still use command line to install new features in the separate configurations (I blogged about that); you just need to adjust the command line with the -configuration parameter.
For instance, to have an Eclipse configuration in eclipse-texlipse/configuration (based on the main Eclipse installation stored in eclipse-main) with the addition of Texlipse and Subversion features I run these commands
Note that using the command line for installing new features will also store in the Eclipse configuration the specified update sites (so you will find them in the Install New Software dialog).
Although I’m not a big user of Windows, sometimes I use it to test my software and products. When switching to Windows Vista and Windows 7 I’ve always noticed a huge use of the disk.
Here are some tricks I’ve used to reduce such use
First of all make sure to disable indexing for the whole hard disk (checkbox “Allow files on this drive…”)
Then stop the Windows Search service and…
make sure it is disabled (right click and the Properties)
Finally, make sure that you have no scheduled defragmentation (you can check this by pressing “Defragment now…”
When mirroring eclipse repositories, my main idea is to keep the path structure of the original repository URL. For instance, if you are using something like
http://download.eclipse.org/releases/juno
the mirror should differ only for the base url, e.g., something like
/home/myhome/eclipsemirror/releases/juno
This is useful if the p2 repositories you use for materializing your target platform are parametrized with respect to the base URL. For instance, I’m using Buckminster to materialize my target platforms (see also this post), and in my RMAP files I have something like
You see that the URLs are parametrized over the property eclipse.download (which defaults to http://download.eclipse.org). If I mirror those repositories keeping the same structure
then, switching to my local mirror for target materialization it’s just a matter of passing for the property eclipse.download the URL of my local directory, e.g., file:/home/bettini/eclipsemirror, without even changing my RMAP files.
So let’s start mirroring! We need to define an Ant script for the p2 antRunner.
For instance, for mirroring the whole orbit repository (with that particular drops version) we create this script, let’s call it mirror-orbit.xml:
Note that we keep in the target dir the same path structure of the original repository.
Since these Ant tasks need to be run via the Eclipse antRunner application, you need a full installation of Eclipse on the machine that will run the task. And you run this task with a command line like the following
1
2
3
4
/path/to/eclipse-noSplash\
-application org.eclipse.ant.core.antRunner\
-buildfile mirror-orbit.xml\
-Dtarget.dir=$HOME/eclipsemirror
Of course you can choose any target dir; the idea is however to always use the same target dir so that all repositories will be mirrored in that path.
Mirroring an entire repository might not always be the case, especially for Juno main release repository, which is quite huge. But you can specify in the Ant task the installable units you’re interested in; then, the p2 task will only mirror those installable units (and all its dependencies). For instance,
This task will mirror all the features that should let you define a target platform for RCP development with EMF and CDO.
NOTE: if you try to mirror org.eclipse.platform.sdk from the releases/juno repository, you will see that it will actually mirror the whole repository! (see also this forum post).
If you get some warnings during the mirror about unsolvable dependencies, you can ignore them: basically those dependencies are in a different repositories, and probably you will mirror those repositories too later.
Of course you can use several p2.mirror elements in the same Ant task. For example, this is the one we use in Emf Components, to have a mirror for our target platform: it also mirrors Swtbot and Xtext SDKs:
Final warning: it might take some time for the mirror task to complete (usually hours depending on your connection and download.eclipse.org load) and it will also take some hard disk space (for the above mirror it takes about 2 Gb).
You may have to experiment a bit to get all the features you need in the mirror; for instance, I didn’t know about the draw2d above, but I had to add it since during target materialization that feature was requested by some other feature. If you’re lost about that, you can always mirror the whole thing 😉
One of the new cool features which came with Xtext 2.3.0 is a wizard which generates all the artifacts to build your Xtext DSL project with Buckminster; this will allow you to easily build your Xtext project p2 site and also to build your project and its tests headlessly (e.g., from the command line, and, more importantly from within a continuous integration system like Jenkins).
This new feature was not advertised much, nor documented, so I decided to write a tutorial about that, also with the permission of the developer of this feature Dennis Hübner.
In this tutorial I will show how to
create the p2 repository from the IDE
build your project (and run the tests) headlessly from the command line (through ant)
build your project (and run the tests) headlessly in a continuous integration system like Jenkins
In the end, I will also try to describe/explain all the Buckminster files that the wizard created for you, so that one can customize them.
So, first of all, let’s create a simple Xtext hello project; for this example I will use the following settings:
project name: org.xtext.example.hellobuck
name: org.xtext.example.hellobuck.HelloBuck
extension: greetings
check “Create SDK feature project”
and then, of course, we generate Xtext artifacts.
The DSL is basically the standard Greetings DSL; in this case, I’m using the JvmModelInferrer to generate Java classes from “Hello” specifications. Indeed, the DSL itself is not important in this tutorial. I’ve also added two Junit tests in the corresponding org.xtext.example.hellobuck.tests project:
Note the org.xtext.example.hellobuck.tests.launch Launch configuration that Xtext created for you (which basically runs all the Junit tests in this project. We will see how this will be used later.
Use the Xtext Build with Buckminster Wizard
Now we can use the Xtext wizard “Build with Buckminster”: New => Other => Xtext => Continuous Integration => Build with Buckminster:
You will have to specify the sdk feature of your Xtext project (that has been created by the Xtext new project wizard); the releng and site projects will have predefined names. For the Buckminster installation directory you have 3 choices:
if you have already an headless installation of Buckminster in your system you can specify the path;
you can install it using the link in the dialog
you can leave that path empty and specify it later in the build.ant generated file (or pass it on the command line when invoking ant)
In any case, specifying the headless Buckminster installation path is only useful if you intend to build your projects headlessly with ant (it is not requested for building in the IDE, nor if you plan to build it with Jenkins by setting job using the Buckminster Jenkins plugin, as we will see later).
Before pressing Finish, we also want to specify the tests to launch during the headless build, so we press the button Add and we select the launch configuration in the .tests project which the Xtext wizard created for us.
If you want to build an existing Xtext project and you do not have that launch configuration, all you need to do is to simply create a new plain Junit launch configuration from the IDE, for instance, to run your existing Junit test suite, and save it in the .tests project yourself. You can also configure the headless build to run tests later on.
When the wizard finishes, you will end up with two additional projects
org.xtext.example.hellobuck.buckminster with all the Buckminster (CQUERY and RMAP, build.ant, etc.) files to build your project headlessly;
org.xtext.example.hellobuck.site which is a feature project to create the p2 repository for your Xtext project.
Build the p2 repository from the IDE
Although the .buckminster project created by the wizard is thought to be used in an headless environment, the .site project is useful also to build the p2 repository from the IDE itself! Indeed, if you created an old style update site, you could build the update site from the IDE itself. But if you switched to the new category.xml format, then building the corresponding p2 repository from the IDE is not straightforward.
If you want to run Buckminster from Eclipse, you first need to install it in the IDE of course, by using this repository: http://download.eclipse.org/tools/buckminster/updates-4.2
Before creating the p2 repository, you may want to tweak some Buckminster properties, for instance, you may want to create a buckminster.properties file in the .buckminster project (or in the .site project) like the following
so that Buckminster will generate all its artifacts in the tmp/hellobuck directory of your home folder.
We now right click on the .site project, and select Buckminster => Invoke Action… , we select site.p2 action, and optionally refer to the buckminster.properties file
When the action finishes, you will file the p2.site in the directory $HOME/tmp/hellobuck/build/org.xtext.example.hellobuck.site_1.0.0-eclipse.feature/site.p2/
Build with ant
The wizard created for us, in the .buckminster project a build.ant file that we can use to build the project, run the tests, and create the p2 repository with ant
Note that this file has the location of your Buckminster headless installation path hardcoded (recall the wizard we used before for creating Buckminster projects); if that is not correct, you can still pass this information to ant when invoking it
Shell
1
ant-Dbuckminster.home=<PATH>-fbuild.ant
This ant script will build your project in the directory buildroot which will be created in the directory where all your projects are located (e.g., in the workspace or in another path); in particular it will
materialize the target platform for your project (the first time you run it), and this requires an Internet connection and might take some time
build your projects
run the Junit tests
build the p2 site repository
This should be the output (where WORKSPACE is actually the location of your projects):
It will also tell you where you can find the generated p2 site.
IMPORTANT: there’s a bug in the currently generated build.ant file, which prevent you from building with ant in Windows; until the fixed version of the wizard is released, you need to make sure that the build.ant has the right contents, as in this example source.
Build with Jenkins
To build your Xtext project in Jenkins you can either create a Job which uses ant and the build.ant script, or create a Job which uses the Buckminster Jenkins plugin (I’ve also blogged about that). I will detail both ways. Both jobs have in common the access to the git repository
A Job using ant
This requires that you configured an ant installation in your Jenkins, and that you already have a working version of Buckminster headless in Jenkins; you need to specify them in the ant build step configuration (the screeshots, of course, refer to the paths and values for my Jenkins installation):
then, we configure post build actions to archive both the artifacts (the p2 repository) and the Junit results:
A job using Buckminster plugin
To configure a Buckminster build step, we prepare a text file in the .buckminster projects with the Buckminster commands to execute (we take inspiration from the commands.txt file that the wizard created), we call it jenkins-commands.txt:
Then we configure a build step (this relies on a Buckminster installation that we have already configured in Jenkins), note that we refer to the jenkins-commands.txt we created above (an alternative would be to copy the commands directly in the text area “Commands”):
To configure the post build actions, since we used different output paths, we have to specify them accordingly (in particular, we have not used the output directory buildroot, thus the default will be used):
Details and Customization
It may be interesting to learn more about the files that the Xtext Buckminster wizard generated for you (at least, I personally found instructive to look at them, and learned something about Buckminster 🙂 most of which I described in another post of this blog). The details might also help you if you need to customize the generated files.
The .buckminster project contains a Component Specification (CSPEC), buckminster.cspec, which looks like
thus, due to its dependencies, it represents the target platform for your Xtext project. If your Xtext project required some further specific dependencies to build and test, this is the place to express them!
For instance, in an Xtext project of mine, Xsemantics, I run also some SwtBot tests, which also rely on pde, thus, the dependencies in the corresponding .buckminster project’s buckminster.cspec look like
To materialize components and dependencies with Buckminster, you need a Resource Map (RMAP), and the wizard created two maps: one containing p2 repositories location (for materializing the features of your target platform), called projects-platform.rmap and one for binding your projects to the workspace (when building headlessly), called project.rmap.
This basically tells Buckminster to take all Xtext stuff (including additional components like Google Guice and Antlr) from the Xtext main repository, and to use the main Eclipse Juno release repository for everything else. However, it also redirects to project.rmap for resolving the main Buckminster component of our project org.xtext.example.hellobuck.buckminster.
Again, if your target platform needs additional features from different sites, this is the place where to express them; reusing the example above which also needed SwtBot the rmap would also have these additional stuff:
This rmap assumes that all the Eclipse projects of your Xtext project are in the same base directory; if this is not the case, because you split them, for instance, in plugins, features, doc, etc. directories, then you need to tweak this rmap accordingly, for instance, again, taken from Xsemantics,
Then, you have to Component Query files (CQUERY): projects-platform.cquery, to materialize the target platform, and project.cquery, to materialize your projects in the workspace. The first one could also be used in the IDE to actually materialize the target platform in your workspace (instead of using a target definition file); the second one is useful when building headless.
Note that this cquery has an advisor node to skip .source components: thus, the materialized target platform will not contain the sources for all the features. This is useful headlessly to reduce the time to materialize the target platform. But if you use it also for materializing the target platform in the IDE, then sources are useful to inspect Java classes, and you might want to remove this advisor node (or simply create another cquery only to be used in the IDE, without that advisor node). This query refers to projects-platform.rmap.
As it usually happens when using Buckminster, the cquery for materializing the projects in the (headless) workspace refers to a single feature project which, transitively refers to all the bundles and additional features of your application. Usually, this is the feature project for building the p2 repository, like in this case.
“But wait! the site feature project does not refer to test projects, since I do not want to include them in my p2 repository! So, how can Buckminster run my tests headlessly if they are not materialized?”
If you look at the feature project .site, you will note that it contains a Component Extension (CSPEX) which allows to extend the automatically inferred Component Specification for the feature project with additional specifications. In this case the buckminster.cspex contains
You see that this specification adds a dependency to the org.xtext.example.hellobuck.tests project, so that, when Buckminster materializes org.xtext.example.hellobuck.site, and its dependencies, it will also materialize org.xtext.example.hellobuck.tests. You can use this technique to materialize additional projects; for instance, in from Xsemantics, I have a main feature for tests, and a tests project for each example, thus buckminster.cspex reads as follows
Finally, in the .site project, you will also find the feature.xml
XHTML
1
2
3
4
5
6
7
8
<?xml version="1.0"encoding="UTF-8"?>
<feature id="org.xtext.example.hellobuck.site"
label="org.xtext.example.hellobuck.site Feature"
version="1.0.0.qualifier">
<includes
id="org.xtext.example.hellobuck.sdk"
version="0.0.0"/>
</feature>
which includes the .sdk feature of your project (Remember: when building the site.p2 on a feature project with Buckminster, you will build a p2 repository NOT for the very feature, but for the included features), and the category.xml to give categories to your features in the p2 repository:
<category-def name="main.source"label="Source for Hellobuck"/>
</site>
Note that the .sdk.source feature is not a “real” feature project in your workspace, but, by default, Buckminster will automatically build a source jar for all your features and bundles.
That’s all! Many thanks to the author of this wizard, Dennis Hübner, for making it and for many suggestions!
However, if you have a 64bit system, do not forget to install also these packages:
1
sudo apt-get install libxml2:i386 ia32-libs
Otherwise, acroread will fail
acroread: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Run the command again, and you’ll get another child in the nested composite repository 1.0 (the qualifier has been replaced automatically with the new timestamp):
Let’s increase the service number, i.e., 1.0.1, (using the tycho-versions-plugin) and rebuild:
Let’s increase the minor number, i.e., 1.1.0 and rebuild
Let’s increase the major number, i.e., 2.0.0
and so on 🙂
