When you use Xtext for developing your language the Ecore model for the AST is automatically derived/inferred from the grammar. If your DSL is simple, this automatic meta-model inference is usually enough. However, there might be cases where you need more control on the meta-model and in such cases you will want to switch from an inferred Ecore model to a an imported one, which you will manually maintain. This is documented in the Xtext documentation, and in some blog posts. When I needed to switch to an imported Ecore model for Xsemantics, things have not been that easy, so I thought to document the steps to perform some switching in this tutorial, using a simple example. (I should have talked about that in my Xtext book, but at that time I ran out of pages so there was no space left for this subject 🙂
So first of all, let’s create an Xtext project, org.xtext.example.hellocustomecore, (you can find the sources of this example online at https://github.com/LorenzoBettini/Xtext2-experiments); the grammar of the DSL is not important: this is just an example. We will first start developing the DSL using the automatic Ecore model inference and later we will switch to an imported Ecore.
The grammar of this example is as follows (to make things more interesting, we will also use Xbase):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
grammar org.xtext.example.hellocustomecore.HelloCustomEcore with org.eclipse.xtext.xbase.Xbase generate hellocustomecore "http://www.xtext.org/example/hellocustomecore/HelloCustomEcore" Model: importSection=XImportSection? hellos+=Hello* greetings+=Greeting* ; Hello: 'Hello' name=ID '!' ; Greeting: 'Greeting' name=ID expression = XExpression ; |
and we run the MWE2 generator.
To have something working, we also write an inferrer
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
def dispatch void infer(Model element, IJvmDeclaredTypeAcceptor acceptor, boolean isPreIndexingPhase) { acceptor.accept(element.toClass("generated." + element.eResource.URI.lastSegment.split("\\.").head.toFirstUpper)) .initializeLater([ for (hello: element.hellos) { members += hello.toMethod ("say" + hello.name.toFirstUpper, hello.newTypeRef(String)) [ body = '''return "Hello «hello.name»";''' ] } for (greeting : element.greetings) { members += greeting.toMethod ("say" + greeting.name.toFirstUpper, greeting.newTypeRef(String)) [ body = greeting.expression ] } ]) } |
With this DSL we can write programs of the shape (nothing interesting, this is just an example)
1 2 3 4 5 |
Hello foo! Greeting bar { sayFoo() + "bar" } |
Now, let’s say we want to check in the validator that there are no elements with the same name; since both “Hello” and “Greeting” have the feature name, we can introduce in the Ecore model a common interface with the method getName(). OK, we could achieve this also by introducing a fake rule in the Xtext grammar, but let’s switch to an imported Ecore model so that we can manually modify that.
Switching to an imported Ecore model
First of all, we add a new source folder to our project (you must create it with File -> New -> Source Folder, or if you create it as a normal folder, you then must add it as a source folder with Project -> Properties -> Lava Build Path: Source tab), say emf-gen, where all the EMF classes will be generated; we also make sure to include such folder in the build.properties file:
1 2 3 4 5 6 7 8 |
source.. = src/,\ src-gen/,\ xtend-gen/,\ emf-gen/ bin.includes = model/,\ META-INF/,\ .,\ plugin.xml |
Remember that, at the moment, the EMF classes are generated into the src-gen folder, together with other Xtext artifacts (e.g., the ANTLR parser):
Xtext generates the inferred Ecore model file and the GenModel file into the folder model/generated
This is the new behavior introduced in Xtext 2.4.3 by the fragment ecore.EMFGeneratorFragment that replaces the now deprecated ecore.EcoreGeneratorFragment; if you still have the deprecated fragment in your MWE2 files, then the Ecore and the GenModel are generated in the src-gen folder.
Let’s rename the “generated” folder into “custom” (if in the future for any reason we want to re-enable Xtext Ecore inference, our custom files will not be overwritten):
NOTE: if you simply move the .ecore and .genmodel file into the directory model, you will not be able to open the .ecore file with the Ecore editor: this is due to the fact that this Ecore file refers to Xbase Ecore models with a relative path; in that case you need to manually adjust such references by opening the .ecore file with the text editor.
From now on, remember, we will manually manage the Ecore file.
Now we change the GenModel file, so that the EMF model classes are generated into emf-gen instead of src-gen:
We need to change the MWE2 file as follows:
- Enable the org.eclipse.emf.mwe2.ecore.EcoreGenerator fragment that will generate the EMF classes using our custom Ecore file and GenModel file; indeed, you must refer to the custom GenModel file; before that we also run the DirectoryCleaner on the emf-gen folder (this way, each time the EMF classes are generated, the previous classes are wiped out); enable these two parts right after the StandaloneSetup section;
- Comment or remove the DirectoryCleaner element for the model directory (otherwise the workflow will remove our custom Ecore and GenModel files);
- In the language section we load our custom Ecore file,
- and we disable ecore.EMFGeneratorFragment (we don’t need that anymore, since we don’t want the Ecore model inference)
The MWE2 files is now as follows (I highlighted the modifications):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
... Workflow { bean = StandaloneSetup { scanClassPath = true platformUri = "${runtimeProject}/.." // The following two lines can be removed, if Xbase is not used. registerGeneratedEPackage = "org.eclipse.xtext.xbase.XbasePackage" registerGenModelFile = "platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel" } component = DirectoryCleaner { directory = "${runtimeProject}/emf-gen" } component = org.eclipse.emf.mwe2.ecore.EcoreGenerator { genModel = "platform:/resource/${projectName}/model/custom/HelloCustomEcore.genmodel" srcPath = "platform:/resource/${projectName}/src" } component = DirectoryCleaner { directory = "${runtimeProject}/src-gen" } //component = DirectoryCleaner { // directory = "${runtimeProject}/model" //} component = DirectoryCleaner { directory = "${runtimeProject}.ui/src-gen" } component = DirectoryCleaner { directory = "${runtimeProject}.tests/src-gen" } component = Generator { pathRtProject = runtimeProject pathUiProject = "${runtimeProject}.ui" pathTestProject = "${runtimeProject}.tests" projectNameRt = projectName projectNameUi = "${projectName}.ui" encoding = encoding language = auto-inject { loadedResource = "platform:/resource/${projectName}/model/custom/HelloCustomEcore.ecore" uri = grammarURI // Java API to access grammar elements (required by several other fragments) fragment = grammarAccess.GrammarAccessFragment auto-inject {} // generates Java API for the generated EPackages // fragment = ecore.EMFGeneratorFragment auto-inject {} // the old serialization component // fragment = parseTreeConstructor.ParseTreeConstructorFragment auto-inject {} // serializer 2.0 fragment = serializer.SerializerFragment auto-inject { generateStub = false } ... |
We add the dependency org.eclipse.xtext.ecore in the MANIFEST.MF:In the Xtext grammar we replace the generate statement with an import statement:
1 2 3 4 5 |
grammar org.xtext.example.hellocustomecore.HelloCustomEcore with org.eclipse.xtext.xbase.Xbase //generate hellocustomecore "http://www.xtext.org/example/hellocustomecore/HelloCustomEcore" import "http://www.xtext.org/example/hellocustomecore/HelloCustomEcore" |
Now we’re ready to run the MWE2 workflow, and you should get no error (if you followed all the above instructions); you can see that now the EMF model classes are generated into the emf-gen folder (the corresponding packages in the src-gen folders are now empty and you can remove them):
We must now modify the plugin.xml (note that there’s no plugin.xml_gen anymore), so that the org.eclipse.emf.ecore.generated_package extension point contains the reference to the new GenModel file:
1 2 3 4 5 6 7 8 9 10 11 |
<plugin> <extension point="org.eclipse.emf.ecore.generated_package"> <package uri = "http://www.xtext.org/example/hellocustomecore/HelloCustomEcore" class = "org.xtext.example.hellocustomecore.hellocustomecore.HellocustomecorePackage" genModel = "model/custom/HelloCustomEcore.genmodel" /> </extension> </plugin> |
If you try the editor for the DSL it will still work; however, the Junit tests will fail with errors of this shape:
1 2 3 4 |
org.eclipse.xtext.parser.ParseException: java.lang.IllegalStateException: Unresolved proxy http://www.xtext.org/example/hellocustomecore/HelloCustomEcore#//Hello. Make sure the EPackage has been registered |
That’s because the generated StandaloneSetup does not register the EPackage anymore, see the diff:
All we need to do is to modify the StandaloneSetup in the src folder (NOT the generated one, since it will be overwritten by subsequent MWE2 workflow runs) and override the register method so that it performs the registration of the EPackage:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
public class HelloCustomEcoreStandaloneSetup extends HelloCustomEcoreStandaloneSetupGenerated{ public static void doSetup() { new HelloCustomEcoreStandaloneSetup().createInjectorAndDoEMFRegistration(); } @Override public void register(Injector injector) { if (!EPackage.Registry.INSTANCE.containsKey("http://www.xtext.org/example/hellocustomecore/HelloCustomEcore")) { EPackage.Registry.INSTANCE.put("http://www.xtext.org/example/hellocustomecore/HelloCustomEcore", org.xtext.example.hellocustomecore.hellocustomecore.HellocustomecorePackage.eINSTANCE); } super.register(injector); } } |
And now the Junit tests will run again.
Modifying the Ecore model
We can now customize our Ecore model, using the Ecore editor and the Properties view.
For example, we add the interface Element, with the method getName() and we make both Hello and Greeting implement this interface (they both have getName() thus the implementation of the interface is automatic).
We also add a method getElements() to the Model class returning an Iterable<Element> (containing both the Hello and the Greeting objects)
and we implement that method using an EAnnotation, using the source “http://www.eclipse.org/emf/2002/GenModel” and providing a body
With the following implementation
1 2 3 |
return com.google.common.collect.Iterables.<Element>concat (getHellos(), getGreetings()); |
Let’s run the MWE2 workflow so that it will regenerate the EMF classes.
And now we can implement the validator method checking duplicates, using the new getElements() method and the fact that now both Hello and Greeting implement Element:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
import org.eclipse.xtext.validation.Check import org.eclipse.xtext.xbase.typesystem.util.Multimaps2 import org.xtext.example.hellocustomecore.hellocustomecore.Element import org.xtext.example.hellocustomecore.hellocustomecore.Model class HelloCustomEcoreValidator extends AbstractHelloCustomEcoreValidator { public static val DUPLICATE_NAME = 'HelloCustomEcoreDuplicateName' @Check def checkDuplicateElements(Model model) { val nameMap = <String,Element>Multimaps2.newLinkedHashListMultimap for (e : model.elements) nameMap.put(e.name, e) for (entry : nameMap.asMap.entrySet) { val duplicates = entry.value if (duplicates.size > 1) { for (d : duplicates) error( "Duplicate name '" + entry.key + "' (" + d.eClass.name + ")", d, null, DUPLICATE_NAME); } } } } |
That’s all! I hope you found this tutorial useful 🙂
Pingback: Switching to Xcore in your Xtext language | Lorenzo Bettini
Thank you so much for the nice tutorial. Even though my workflow was still using the deprecated generator, I could switch to an imported model
Thanks, Lorenzo. You help me a lot.
Lorenzo, you possibly should mention — for newcomers — that emf-gen needed to be added to the Java Build Path via Menu Project -> Properties -> Lava Build Path: Source tab.
Thanks, I added the clarification; actually it’s even easier, all you have to do is to create the folder with File -> New -> Source Folder 🙂
Definitely, your way is much easier!
Also, Lorenzo, I misspeled the action needed . Not “Lava Build Path”, but “Java Build Path”! 🙁
I also found, that after switching to the imported model the model registration disappeared from generated class method
HelloCustomEcoreStandaloneSetupGenerated.register(Injector)
(in src-gen/org.xtext.example.hellocustomecore).
It is important for me, since I have standallone (“headless”) version of my application.
Possible solution is to add the method register(Injector) to the HelloCustomEcoreStandaloneSetup (in the src/org.xtext.example.hellocustomecore) and to place disappeared lines there:
public void register(Injector injector) {
if (!EPackage.Registry.INSTANCE.containsKey(“http://www.xtext.org/example/hellocustomecore/HelloCustomEcore”)) {
EPackage.Registry.INSTANCE.put(“http://www.xtext.org/example/hellocustomecore/HelloCustomEcore”, org.xtext.example.hellocustomecore.hellocustomecore.HellocustomecorePackage.eINSTANCE);
}
super.register(injector);
}
After this change all worked fine. Once more, thanks a lot!
Oh, Lorenzo, I simply have not followed your text to the very end! Everytthing already was mentiond! Is it possible to remove my previous comment?
Don’t worry: we can leave both comments, just to stress the importance of that part of the standalone setup 🙂
@Lorenzo – do you know if xtext changed something significantly since you wrote this blog? I just cannot make this run on 2.8.3 (not even with your github example)
This line just makes me feel like a loser 😉 — “Now we’re ready to run the MWE2 workflow, and you should get no error (if you followed all the above instructions); ”
On your customECore project I get:
“java.lang.RuntimeException: Problems instantiating module org.xtext.example.hellocustomecore.GenerateHelloCustomEcore: java.lang.reflect.InvocationTargetException
….
Caused by: java.lang.IllegalStateException: Problem parsing ‘classpath:/org/xtext/example/hellocustomecore/HelloCustomEcore.xtext’:
TransformationDiagnostic: null:8 Cannot find compatible feature importSection in sealed EClass Model from imported package http://www.xtext.org/example/hellocustomecore/HelloCustomEcore: The existing reference ‘importSection’ has an incompatible type ‘XImportSection’. The expected type is ‘XImportSection’ [org.eclipse.xtext.xtype.XImportSection]. (ErrorCode: CannotCreateTypeInSealedMetamodel)
TransformationDiagnostic: null:18 Cannot find compatible feature expression in sealed EClass Greeting from imported package http://www.xtext.org/example/hellocustomecore/HelloCustomEcore: The type ‘XExpression’ used in the reference ‘expression’ is inconsistent. Probably this is due to an unsupported kind of metamodel hierarchy. (ErrorCode: CannotCreateTypeInSealedMetamodel)
at org.eclipse.xtext.generator.LanguageConfig.setUri(LanguageConfig.java:247)
”
Whereas mine gives a slightly different error:
“Caused by: java.lang.IllegalStateException: Problem parsing ‘classpath:/org/eclipse/xtext/example/arithmetics/Arithmetics.xtext’:
XtextLinkingDiagnostic: null:11 Couldn’t resolve reference to EPackage ‘http://www.eclipse.org/Xtext/example/Arithmetics’.
“
Dear Oliver
Unfortunately that blog post is out of date w.r.t. the current version of Xtext. Currently, I’m not using Xcore anymore for my Xtext languages, so I did not have time to update it. I’ll try to have a look at this issue, and try to update the example. I can’t promise I’ll be able to do that in the very near future, I’m afraid.