A taxonomy of tool-related issues affecting the adoption of model-driven engineering

Although poor tool support is often blamed for the low uptake of model-driven engineering (MDE), recent studies have shown that adoption problems are as likely to be down to social and organizational factors as with tooling issues. This article discusses the impact of tools on MDE adoption and practice and does so while placing tooling within a broader organizational context. The article revisits previous data on MDE use in industry (19 in-depth interviews with MDE practitioners) and reanalyzes that data through the specific lens of MDE tools in an attempt to identify and categorize the issues that users had with the tools they adopted. In addition, the article presents new data: 20 new interviews in two specific companies—and analyzes it through the same lens. A key contribution of the paper is a loose taxonomy of tool-related considerations, based on empirical industry data, which can be used to reflect on the tooling landscape as well as inform future research on MDE tools.

The authors would like to thank all those who took part in the interviews, including those who facilitated the study at Ericsson and Volvo.

Authors and Affiliations

1. School of Computing and Communications, InfoLab21, Lancaster University, Lancaster, UK

   Jon Whittle, John Hutchinson & Mark Rouncefield

2. Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, Gothenburg, Sweden

   Håkan Burden & Rogardt Heldal

Corresponding author

Correspondence to Jon Whittle.

Communicated by Dr. Moreira and Dr. Schätz.

An earlier version of this article appeared as “Industrial Adoption of Model-Driven Engineering: Are the Tools Really the Problem?” in the 2013 International Conference on Model Driven Engineering Languages and Systems (MODELS). The major additions in this version are: (i) contextual research from the CSCW (computer-supported cooperative work) community is discussed, which provides important background knowledge for interpreting and generalizing from the findings; (ii) an appendix is included, which describes the taxonomy in more detail; (iii) additional information about study design and validity is presented.

Appendix: Extended description of sub-categories in the taxonomy

This appendix expands briefly on the meaning of the sub-categories in the taxonomy. These extended descriptions were not included in Tables 1, 2, 3, 4 and 5 due to lack of space. The appendix can therefore be used as a reference in case the titles of sub-categories in Tables 1, 2, 3, 4, and 5 are unclear. Each sub-category defines a domain of discourse which interviewees highlighted. Interviewees may have made negative or positive remarks in each case: The presence of a sub-category therefore merely marks that this domain is of key importance to interviewees when applying MDE tools in practice.

1.1 Technical factors (Table 2)

This branch of the taxonomy deals with technical issues that may affect the adoption of an MDE tool. That is, the sub-categories in Table 2 concern technical limitations of tools which may affect adoption, particular features of tools, and/or the impact of technical features of tools on the overall software development process.

1.1.1 Tool features / specific functionalities offered in tools

This category details the presence or otherwise of particular tool functionalities or features. The sub-categories should not be considered an exhaustive list of possible tool features nor a list of features that a tool is expected to have; the list merely covers the features most regularly discussed by our interviewees.

Modeling behavior (1)

Does the tool allow system behavior to be modeled? If so, how is behavior modeled and is it an effective way of modeling behavior?

Action languages (1)

Does the tool support the use of action languages? If so, what action language(s) is supported? Is the action language and tools support provided for it effective?

Support for domain-specific languages (6)

Does the tool support the definition and application of domain-specific languages? If so, in what way—what kinds of facilities are provided?

Support for architecture (3)

What facilities, if any, does the tool have for specifying and modeling architecture?

Code generation templates (6)

Does the tool support the use of code generation templates as a mechanism for allowing tool users to generated customized code? Or does the tool provide only a default format/style for generated code which cannot be changed?

UML profiles (1)

Does the tool support the definition and application of UML profiles?

Scoped code generation (2)

Does the tool have features that support the incremental and iterative generation of code; that is, when the model changes, does the entire code base need to be regenerated, or only the code that is affected by the model change?

Model analysis (5)

What facilities, if any, does the tool have for analyzing models either statically or dynamically?

Reverse engineering models (3)

Does the tool support reverse engineering—where models are created from existing code?

Sketching models (1)

Does the tool support the creation and use of informal “sketched” models which might be used to capture ideas at an early stage in the modeling process?

Refactoring models (1)

Does the tool support the refactoring of existing models?

1.1.2 Practical applicability / challenges of applying tools in practice

This category is not concerned with the features a tool has but how the tool is used in practice and whether it is possible to adapt it according to different processes and procedures.

Tool scalability (1)

Is the tool able to cope with large-scale models—of the sort that may be found in large industrial projects?

Tool versioning (1)

Are there issues associated with tool versioning? For example, frequent upgrades, maintaining compatibility with previous formats, etc.

Chaining tools together (2)

How easy/difficult is it to use multiple tools in conjunction with each other to provide end-to-end functionalities?

Industrial quality of generated code (8)

Is the generated code of the quality, efficiency and size that would be expected in an industrial setting?

Flexibility of tools (3)

To what extent is the tool flexible enough to adapt to different processes, other tools and/or ways of working? For example, does it impose strict processes on users? Or does it require other tools to be used with it?

Maturity of tools (1)

Has the tool reached a level of maturity where robustness makes it suitable for an industrial project?

Dealing with legacy (2)

What facilities, if any, does the tool have to support the use of existing artefacts—e.g., existing models, existing code, etc?

1.1.3 Complexity / challenges brought on by excessive complexity in tools

This category concerns issues of complexity brought on by modeling tools or languages.

Tool complexity (4)

How complex or otherwise is the tool itself? Is that level of complexity considered an appropriate level of complexity or otherwise?

Language complexity (5)

If the tool supports a particular modeling language, how complex or otherwise is the language?

Accidental complexity introduced by tools (1)

Does the tool—through its design—tend to introduce unnecessary complexity into either the modeling process or the resulting artifacts?

1.1.4 Human factors / consideration of tool users

This category is concerned with the effect the tool’s design and features have on its users.

Whether tools match human abstractions (4)

Do the abstractions in the tool match the way that humans think about abstraction? Or does the tool force users to recast their own internal abstractions in to a form that can be captured in the tool?

Usability (4)

Is the tool designed with usability in mind so that users find it easy or intuitive to learn or is it designed in such a way that it actually impedes its use?

1.1.5 Theory / theory underpinning tools

This category concerns theoretical underpinnings of tools, such as whether they are grounded in a formal theory or not.

Theoretical foundations of tools (1)

Does the tool actually have theoretical foundations? If so, what are they and what is their impact?

Formal semantics (2)

Does the tool provide facilities to support—or impose—formal semantics in the modeling process? If so, how does this influence/impact modeling?

1.1.6 Impact on development / impact of tools on technical success criteria

This category is concerned with the effect tools have on higher-level project outcomes rather than the specific process of modeling/development itself.

Impact on quality (2)

Are there features of the tool that affect the quality of the software developed using the tool—and are these positive effects or negative effects?

Impact on productivity (4)

Are there features of the tool that affect the productivity of users —individually or as a team—when using the tool? Are these positive effects or negative effects?

Impact on maintainability (3)

Are there features of the tool that affect the maintainability of the software produced using the tool—and are these positive effects or negative effects?

1.2 Internal organizational factors

This branch of the taxonomy is concerned with how tools relate to the way an organization is structured managerially, to any existing procedures or processes, and/or to any preexisting factors such as the culture of the organization or the skill levels available.

1.2.1 Processes / adapting tools to processes or vice-versa

This category is concerned with to what extent process change is necessitated by the introduction of a tool. For example, does the tool mean that existing organizational processes have to be changed to support the tool? Or is the tool flexible enough that it can be easily adapted to fit an existing process?

Tailoring to a company’s existing processes (5)

In a sense, this is an “applied” version of the tool flexibility issue—how possible is it to adapt the tool to existing processes? How easy is it to leverage existing expertise?

Sustainability of tools over the long term (3)

This sub-category concerns the effort needed to maintain the use of a tool within the organization over the long term. For example, does it require significant ongoing maintenance efforts that require internal resources? Or can the tool be bought once and then used at no cost indefinitely? How easily can the tool be adapted over time when the nature of the organization’s business evolves?

Appropriating tools for purposes they were not designed for (3)

It is well known that users will use any tool in ways that were unforeseen by the tool’s developers—are there examples of this happening with MDE tools and what is the impact of this appropriation on a project/organization?

Issues of integrating multiple tools (6)

Again, this is an “applied” version of the “tool chaining” technical issue—what are the consequences, from an organizational perspective, of adopting a particular tool when that tool has to be integrated with a range of other tools?

Migrating to different tool versions (3)

How does the support for migrations between tool versions affect an organization’s processes? Is migration easily supported or does it require an organization to introduce lengthy and complex internal processes?

Offsetting gains: tools bring gains in one aspect but losses in another (2)

Are there aspects of a tool that appear to bring gains in one part of an organization but incur losses in another? An example would be code generation, which could bring clear productivity gains to one team, but could bring losses to another if the code generated is inefficient and has to be adapted before deployment.

Whether maintenance is carried out at the code or model level (3)

What processes does an organization have in place to enforce good practice, such as ensuring changes are made at the model level rather than by modifying generated code? Or are new processes required because an organization makes changes to generated code which then have to be reflected back in the model?

1.2.2 Organizational culture / impact of cultural attitudes on tool application

This category concerns cultural issues related to an organization.

Tailoring to a company’s culture (4)

However flexible a tool is, it will impose new ways of working on any organization adopting its use—does the tool offer any facilities to support its tailoring to an organization’s existing culture? And if not, what are the consequences?

Inertia: reluctance to try new things (1)

Is there a culture of reluctance in the organization to try new things? If so, does the tool naturally exacerbate this problem or alleviate it?

Over-ambition: asking too much of tools (1)

What do tools promise? And are these promises realistic when they are conveyed to companies? Do companies believe that tools will deliver more than they realistically can?

Low hanging fruit: using tools on easy problems first (6)

How do organizations apply the tool in practice, and to which problems? Some companies, for example, have found success in applying a tool to easily solved and well understood problems. Other companies have tried to apply MDE tools to very complex problems. Which is the best stragegy?

1.2.3 Skills / skills needed to apply tools

This category is concerned with how adoption/use of a particular tool affects notions of “skills” in an adopting company—and how they are acquired.

Training workforce (11)

What impact does adoption of a particular tool have on the training requirements of the adopting company?

Availability of MDE skills in workforce (4)

Is the choice of tool affected by the availability of suitably experienced practitioners for recruitment? Or is there a type of MDE skill that transcends a particular tool? Alternatively, is tool choice a result of available expertise?

1.3 External organizational factors

This branch of the taxonomy deals with issues concerning external influences on the organization when they adopt MDE tools and how those influences affect application of the tools.

1.3.1 External influences /factors which an organization has no direct control over

If a company has collaborators, suppliers or standards to adhere to, how does the use of a tool affect those relationships? Do external partners impose conditions that affect tool use or does use of particular tool impose conditions on external partners?

Impact of marketing issues (1)

Does using the development method du jour influence the adoption of any particular tool? And what is that effect?

Impact of government/industry standards (4)

Some software is subject to extreme external verification. Does the use of a particular tool (and associated methods) impact on this—in a positive or negative way?

1.3.2 Commercial aspects / business considerations impacting on tool use and application

This category is concerned with how tool choice/adoption impacts how businesses make their commercial decisions—whether advantages offset costs, for example.

Business models for applying MDE (3)

Does the tool (approach) support the use of existing business models? Or do business models have to be adapted to adopt MDE tools?

Cost of tools (5)

Is the cost of a commercial MDE tool an impediment to the use of that tool—or otherwise? Does the cost of a tool override other considerations?

How to select tools (2)

How should potential users of a tool make such a decision? Are there appropriate resources available to inform decisions? For example, are there reports available that compare capabilities of tools?

1.4 Social factors

This branch of the taxonomy deals specifically with social issues such as trust and control regarding tool selection and use.

1.4.1 Control / impact of tools on whether stakeholders feel in control of their project

This category is particularly concerned with the relationship that tool users have with stakeholders such as tool vendors.

Ways of interacting with tool vendors (2)

What opportunities, if any, are there for interacting with the tool vendor and how do these relate to how open the tools are? For example, will the vendor adapt the tool for a particular organization? If so, does this incur a cost? Or is the vendor adamant that tools cannot be adapted for particular situations?

Subverting tools: workarounds needed to apply them (1)

Is it necessary to create specific work-arounds to allow tools to match more closely the working practices of the user? For example, these can often arise because lack of flexibility of the tool vendor and can lead to a lack of trust in the tool.

1.4.2 Trust / impact of trust on tool use and adoption

This category is concerned with all aspects of how trust affects tool use and attitudes toward tool adoption.

Trust of vendors (4)

How is trust in the tool vendor established (or lost)? For example, what is the vendor’s reputation when it comes to matters such as cost, support, updates, etc?

Engineers’ trust of tools (6)

Are there aspects of the tools that impact on the specific engineering aspects of their use? For example, is it possible to establish trust in the quality of the code that is generated or the robustness and accuracy of model checking/analysis?

Impact of personal career needs (1)

Does the tool meet or hinder the expectations of individual users? For example, are developers unsure about using obscure tools that do not offer them skills that might have broader appeal—and therefore career opportunities?

1.4.3 Open community

This category is concerned with the availability or otherwise of an open community of tools users that provide peer support in the use of tools.

Developer forums

Do developer forums exist to support users of the tool by providing examples, advice and assistance?

Reprints and permissions