Despite the efforts poured into method development and research, there seems to be no universal agreement whether methods are useful in ISD at all (Lyytinen 1987, Cotterman and Senn 1992, Wynekoop and Russo 1993). For example, Wynekoop and Russo (1993) summarize several fundamental questions on ISD methods which are largely unanswered. Of these questions two are especially important to our study: “are methods actually used in practice?” and “why are local methods developed?” The importance of these questions is further emphasized because of the contradiction between the great efforts made to promote text-book methods and their surprisingly low use in practice. In short, there are thousands of methods available (Bubenko 1986) and new ones are continually developed, but at the same time empirical research reveals that many companies do not use them, and if they do then they have developed their own variants (Hardy et al. 1995, Russo et al. 1995, Fitzgerald 1995, Flynn and Goleniewska 1993).

As a result, it seems that method development is relatively easy since so many of them exist, but methods developed by others do not meet method users’ requirements. We can find reports and studies about organizations which have found their local methods applicable or even reported success stories of method use (Jaaksi 1997, Nissen et al. 1996). These observations lead us to analyze two paradoxes of methods in more detail, namely the low acceptance of methods and the popularity of local methods. These paradoxes are important to our research objective of supporting the development of methods through incremental ME.

Although the capability of methods to improve the productivity and quality of ISD has commonly been acknowledged, systematic use of methods is still surprisingly low (Chikofsky 1988, Danzinger and Haynes 1989, Necco et al. 1987, Smolander et al. 1990, Aaen et al. 1992, Fitzgerald 1995). Thus, there is a paradox here between the claimed advantages of methods, which should indicate high use, and the empirical observations revealing low acceptance of methods. This paradox is further emphasized when we consider the amount of work both industry and academics put into the development and study of methods.

The low acceptance of methods is reported by many professionals, confirmed by empirical research and recognized in many studies focusing on the use of tools. For example, Yourdon has estimated (reported in Chikofsky 1988) that only 10% of software professionals have actively used structured methods in their daily practice, and 50% of organizations have tried them at some time. Nevertheless, 90% of developers are familiar with structured methods, emphasizing the low acceptance of methods.

In addition, several empirical studies on the use of methods or tools confirm the estimations on the low use of methods. A study by Fitzgerald (1995) into 162 organizations observe that only 40% of them apply methods. Another study by Necco et al. (1987) into 97 organizations shows that 62% of companies used a structured approach. A study by Hardy et al. (1995) indicates that method use can be as high as 82%. As can be seen, these studies have different or even conflicting results. One reason for the variety lies in the selection of the sample and in the definition of ‘method use’. First, samples are not homogeneous. For example, Fitzgerald (1995) included small companies which did not have large ISD projects, companies which applied packaged software, and companies which had outsourced ISD. These companies were also found to be less favorable to the use of methods, which explains the lower use rate found. On the other hand, studies concentrating on method use normally show higher rates of method use, e.g. 82% in Hardy et al. (1995). Nevertheless, a study by Russo et al. (1995) which focused on organizations using methods still found that 7% of organizations which had claimed in an earlier survey to have a method did not use it. Hence, even if the sample organizations would be the same, respondents can have a different understanding of what methods and method use mean.

Second, distinctions between levels of method use is important, especially the borders between systematic, ad-hoc, and no use of methods. What does it actually mean when ISD professionals say that they follow some method? For example, how fully should method use be defined and documented, how completely should they be followed, and how widely spread and obligatory method use should be in an organization before we can make a judgment that methods are actually used. For example, although in the survey by Hardy et al. (1995) 82% of organizations claim to use methods, it does not mean that they always follow them. In a partial solution to this problem, Fitzgerald (1995) suggests a distinction between formalized and non-formalized methods: a formalized method denotes a commercial or a documented method, and a non-formalized a non-commercial or an undefined method. An organization’s own methods could fall into both categories. By considering only the use of formalized methods the rate of method use drops considerably: from 40% to 26% (Fitzgerald 1995). A field study by Smolander et al. (1990) partly confirms these findings by showing that the methods applied were mostly a collection of loosely coupled informal techniques. Moreover, Russo et al. (1996) characterizes method use based on frequency — used always, seldom or occasionally — to find out the adherence to methods. This categorization shows that most organizations having a method actually apply them (66%).

Thus, the diversity of the meaning of method use and the lack of knowledge regarding how methods are actually used explains differences in survey results. It seems that the use of surveys to study method use and commitment to methods and their actual usage is difficult. As a result, researchers (Wynekoop and Russo 1993, Galliers and Land 1987) have advocated diversity of research approaches. In the case of method use this would generally indicate field studies, case studies, and action research.

Empirical studies, however, reveal the major benefits and drawbacks of method use. Major benefits include enhanced documentation, systematized ISD process, meeting requirements better, and increased user involvement (Smolander et al. 1990, Hardy et al. 1995). Organizations which do not use methods consider the improvements caused by methods to be modest: methods are considered labor-intensive, difficult to use and learn, and as having poorly defined and ambiguous concepts (McClure 1989, Brinkkemper 1990). Methods are also seen as limiting and slowing down development, generating more bureaucracy and being unsuitable (Smolander et al. 1990). Hence, introduction of a method changes the prevailing practices of ISD to such an extent that the method is abandoned or at least its use is made voluntary.

To summarize, method developers have partly failed in introducing methods which would be acceptable by the ISD community at large. There is some empirical evidence which explains which aspects of methods and their use situations influence their success (or failure) (Wynekoop and Russo 1993). The research focus seems to be more on the internal properties and characteristics of methods than on their use situations (Tolvanen et al. 1996). Of course, one may state that the idea of methods is not to apply them as given. In reality, most methods are proposed as universal, i.e. to design inventory systems, automatic teller machines, or mobile phones without considering situational characteristics (Fitzgerald 1996).

A second paradox is related to the use of local methods in contrast to applying third-party methods (i.e. commercial or text-book methods). Surveys investigating method use in organizations (Pyburn 1983, Smolander et al. 1990, Flynn and Goleniewska 1993, Hardy et al. 1995, Fitzgerald 1995, Russo et al 1995) as well as case studies and descriptions of organization specific methods (Kronlöf 1993, Aalto 1993, Jaaksi 1997, Vlasblom et al. 1995, Nissen et al. 1996, Kurki 1996, Tollow 1996) reveal that organizations tend to develop their own local “variants” of methods, or adapt them (Nandhakumar and Avison 1996) to their specific needs. Hence, there is a paradox here between method developers proposing situation-independent methods and method users who have developed situation-bound methods.

Surveys indicate that local methods are more popular than their commercial counterparts (Fitzgerald 1995, Russo et al. 1995). This partly explains the low acceptance of CASE tools which normally necessitate the use of a fixed method (Wijers and van Dort 1990, Aaen et al. 1992). Among the surveys, both Russo et al. (1995) and Fitzgerald (1995) show that 65% of the organizations which use methods have developed them in-house: their own method is preferred over a third-party one. Other studies obtain similar figures: 62,5% (Flynn and Goleniewska 1993), 42% (Russo et al. 1996), 36% (CASE Research Corporation cited in Yourdon 1992), and 38% (Hardy et al. 1995) of organizations have developed their own methods. Hardy’s study, furthermore, claims that 88% of the organizations adapted the methods in-house; the same percentage was found in the study by Russo et al. (1995). Thus, although organizations develop their own methods, methods need to be adapted to different use situations in the same way as with third-party methods. This means that organizations’ own methods do not completely fit with the use situations in their projects. Some studies (Hardy et al. 1995), however, have found that organizations which have developed their own methods are more satisfied with them than users of third-party methods. This is quite obvious, since otherwise the local method would hardly have been developed and maintained. On the other hand, few would announce that they have developed a bad method. Thus, it seems natural that methods developed locally are considered better than third-party methods.

Unlike surveys of method use, surveys of local method development get surprisingly similar results, although it would be expected that the distinction between local and external methods as well as between levels of adaptation would be more difficult to make. However, since surveys do not go into details, they do not provide answers about what local method development actually means, or what aspects of method knowledge are modified.

To examine local method development more closely, other research methods such as case studies and field studies are required (Tolvanen et al. 1996). Although local methods are typical in organizations that actually use methods, their selection, development, and applicability is less studied (Wynekoop and Russo 1993). With alternative research methods the modifications of ISD methods could be inspected in detail, e.g. what the development of local method or method adaptation means, as well as how in-house methods differ from third-party ones and how extensive the modifications are. These questions are only partly answered in case studies and reports on local method development (cf. Aalto 1993, Jaaksi 1997, Vlasblom et al. 1995, Nissen et al. 1996, Kurki 1996, Tollow 1996) as they mostly focus on outcomes rather than on differences between local and text-book methods, or how the local method is developed. However, these results are important as a motivation for our aim to develop means for carrying out local method development efforts.

To sum up, many of the organizations or projects which apply methods do not use the methods proposed by others. Commercial methods are modified for example by simplifying or by combining them with other methods (e.g. Jaaksi 1997), or then organizations develop their own methods. This is noteworthy since commercial methods claim to have a well-thought out conceptual structure together with process models and guidance which have worked successfully in other ISD efforts. These methods are furthermore backed by manuals, training programs, tutorials, and tools, necessary when introducing methods. The reason for local method development can not be simply a negative attitude towards something developed outside the organization (i.e. ‘not invented here’ attitude). Development of a local method requires significant expenditure of resources which would not be needed if commercial methods were applied. The relatively high costs, need for resources and recognized ad-hoc method development practices (Smolander at al. 1990) would also discourage local method development efforts. Thus, it seems that the need for more applicable methods is so great that it leads organizations to develop their own methods, either organization specific or project specific.