The second contribution of the thesis lies in an improved understanding of the method engineering process. These improvements were obtained by extending method engineering to cover method evaluation and refinement. The proposed incremental approach does not cover only the selection and construction of methods for a given situation (a priori), but also the evaluation of the applicability of methods and method improvements (a posteriori). Earlier research into the ME process has so far focused on constructing methods in an “one-shot” manner, as described in Sections 3.2 and 5.2. We however regard ME as an incremental process. We believe that the method is rarely defined at once, and written down as a complete metamodel. The process of arriving at a method is fragmented, evolutionary and largely intuitive. Though we can identify some refined pieces of ISD methods, the reality of ME tends to be meandering towards a solution, as situations change and stakeholders learn more. This means that any ME approach focusing only on the initial method construction is incomplete and ME principles need to be extended to cover improvements of the methods based on their use.

The incremental approach complements ME principles by proposing steps for a posteriori ME. These steps deal with collecting, analyzing, refining, and sharing methodical knowledge. The steps are based on explicit metamodels as well as on decisions and rationale behind method development. Metamodels are needed to understand methods in use and method refinements, and method rationale is needed to describe why methods were specified as they were. Throughout these steps we applied three mechanisms of method evaluation and refinement. Like the metamodeling constructs, these mechanisms examine modeling techniques in the context of modeling tools. The first mechanism — the type-instance matching — compares differences between a modeling technique’s intended use (as seen from the metamodel) and actual use (as seen from models). The second mechanism — analysis of modeling power — examines the capability of the method to represent the desired aspects of the object system in models and to maintain the consistency of the models. The third mechanism — the analysis of the role of a method in problem solving — focuses on a method’s capability to generate alternative solutions and support subsequent decision making. The latter two mechanisms address experiences and learning of method stakeholders, such as designers, end-users, domain experts, ISD tool experts, and method engineers.

These mechanisms are suggested so that they can focus on method aspects which need refinement. As a consequence, if the analysis phase suggests a method modification, it reveals that the a priori constructed method was not fully applicable. The refinements extend, modify or remove parts of the method knowledge. The refinements can be gradual and small (in comparison with other method development strategies). This explains the title of the proposed approach. The term gradual suggests that method refinements are made to the currently used method, rather than by selecting a new method. Small changes are a consequence of gradual changes; applicability is achieved by modifying parts of the method.

The incremental approach was examined through an action research intervention in two cases. The cases covered all the major steps of ME. Our discussion focused primarily on the a posteriori view and method evaluation mechanisms. The evaluation led to several refinements of the constructed methods. The refinements added, modified, and removed parts of method knowledge. Through a tool implementation the new method was taken into use. This finding provides evidence that a priori method construction alone does not always provide adequate support. In the cases, the suggested identification principles and method improvement mechanisms were found to be useful. The metamodeling approach used also revealed some extensions to the metamodeling constructs. Moreover, the use of metamodels was found to be useful while specifying local methods and analyzing their evolution. In this sense, the metamodels supported not only the local method development, but also the action research studies of detailed method knowledge and its evolution.

During the action research interventions the method refinements were performed only once. An incremental approach to method engineering, however, would necessitate several “reflection” cycles. Method engineers must obtain data from several situations to yield a metamodel repository with information about the applicability of methods (and their parts). With effective use of this incremental approach, methods can be constructed and improved based on their demonstrated ability to support specific situational factors.

The ME principles developed can be applied in an organization which is developing its own methods and needs methodical guidelines. Moreover, the principles of incremental ME are suitable not only for local method development, but also for development of standardized methods (as shown in case B).