Print
Deutsch

Memorandum on design-oriented information systems research


Contact:
Institute of Information Management
Universität St. Gallen
Müller-Friedberg-Strasse 8
CH-9000 St. Gallen
Phone: +41 (0) 71 224 38 00
Fax: +41 (0) 71 224 32 97
hubert.oesterle@unisg.ch




Accreditations:
http://www.efmd.org/index.php/accreditation-/equis http://www.aacsb.edu/ Akkreditierungen Universität St.Gallen

  

 

Click to open as PDF

      

European Journal of Information Systems

Memorandum on design-oriented information systems research

Osterle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., Loos, P., Mertens, P., Oberweis, A., Sinz, E. J., Memorandum on design-oriented information systems research, in: European Journal of Information Systems, 20, 2010, Nr. 1, S. 7-10, http://www.palgrave-journals.com/ejis/journal/v20/n1/pdf/ejis201055a.pdf  

Information Systems Research („Wirtschaftsinformatik“) basically follows two research approaches: the behavioristic approach and the design-oriented approach. In this memorandum, ten authors propose principles of design-oriented information systems research. Moreover, the memorandum is supported by 112 full professors from the German-speaking scientific community, who with their signature advocate the principles specified therein.Click to open as PDF

 

For the memorandum in German please refer to the following publication, which you find on the German version of this website (Klick on the link "Deutsch" in the upper right corner):

Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., Loos, P., Mertens, P., Oberweis, A., Sinz, E. J., Memorandum zur gestaltungsorientierten Wirtschaftsinformatik, in: Zeitschrift für betriebswirtschaftliche Forschung, 6, 2010, Nr. 62, S. 664 - 672

A response to the design-oriented information systems research memorandum

Baskerville, R., Lyytinen, K., Sambamurthy, V., Straub, D., A response to the design-oriented information systems research memorandum, in: European Journal of Information Systems, 20, 2010, Nr. 1, S. 11-15Click to open as PDF, http://www.palgrave-journals.com/ejis/journal/v20/n1/pdf/ejis201056a.pdf

Editorial

Junglas, I., Niehaves, B., Spiekermann, S., Stahl, B. C., Weitzel, T., Winter, R., Baskerville, R., The inflation of academic intellectual capital: the case for design science research in Europe, in: European Journal of Information Systems, 20, 2010, Nr. 1, S. 1-6Click to open as PDF, http://www.palgrave-journals.com/ejis/journal/v20/n1/pdf/ejis201057a.pdf

 

Comments

Jörg Becker, University of Münster

Design Oriented Information Systems Research

Chapter: The process of design oriented information systems research

Typically, design oriented information systems research follows an iterative process comprising four phases:

  • Analysis,
  • Design,
  • Evaluation,
  • Diffusion.                                                                            

Analysis:

The impulse to conduct research on a certain issue or problem can be set both by researchers and by practitioners. In the Analysis phase, the problem is identified and described, and the research goals are specified. As far as the definition of the problem is concerned, it must be ensured that the problem is of relevance. Relevant problems can be, for example: failure of a project dealing with information systems, a software solution not living up to expectations, information overflow (frequently occurring together with irrelevant or incorrect information made available), excessive expenses occurring in modeling projects, a model not sufficiently understood by its users, poor quality in the collaboration of producers and service providers rendering bundles of products and services in hybrid value creation settings, low acceptance of It systems, poor efficiency of information systems etc. Since the subject of information system research is information processing in companies, in public administration, and even in private households, looking at what is going on in the real life is indispensable in order to define a problem.

With regard to research goals, it is important to note that, apart from the design related goal (which is in the focus of all activities), often a descriptive and explanatory goal is pursued. Information systems research, on the one hand, is supposed to deliver in functional terms, as it has to deal with information systems in different business domains (e.g. trade industry, bank industry, insurance industry, manufacturing industry, public administration). On the other hand, information systems research has to come up with method related ideas, since procedures, methods and models need to be created that contribute to the design and evaluation of information systems.

Design oriented information systems research puts up a research plan for development and improvement of artifacts required. In the Analysis phase, the factors the problem is affected by are identified. So one aspect of the Analysis phase is to identify the determining factors and to take into account the aspect of contingency.

Information systems research deals with various research questions. To do so, it uses a broad spectrum of instruments in terms of methods and language. Research planning configures a set of research methods best suited for the project at hand. One important task of the researcher is to abstract and structure the research subject in such a way that processes and IT systems can be designed.

On all levels along the process of gaining insight, the researcher should be aware of their positioning [2]:

  • ontological aspect: ontological realism („there is a real world“) vs. ontological idealism („the world only exists in our mind“) vs. Kant´s view (“noumenon – the thing as such”,
  • epistemological aspect: epistemological realism („gaining insight is independent of the individual“) vs. constructivism (“gaining insight always depends on the individual”),
  • notion of truth: correspondence theory vs. consensus theory vs. semantic theory,
  • Where does insight come from? (empiricism, rationalism, Kant´s view),
  • Methodological aspect (induction, deduction, hermeneutics).

Design:

Artifacts are to be derived by means of widely accepted methods, justified to the fullest extent possible, created, and delineated against existing solutions in science and practice. Artifacts can be models (modeling technique or models that are instantiated but generally applicable at the same time), guidelines, prototypes, or (rather infrequently) ready-to-run IT systems designed for commercial use. In any case, they are – as mentioned above – abstractions of an object system. Since such abstractions cannot be determined as true or untrue by means of mathematical methods, mechanisms are needed that allow for congruence regarding the plausibility of an abstraction. It is helpful here to follow the Guidelines of Modeling [3]. These are six recommendations for improving the quality of models:

  • Principle of correctness:

The principle of correctness comprises two sub-principles: syntactic correctness, which can be verified by formal methods, and semantic correctness, which can not. According to Kamlah and Lorenzen [9], an abstraction can be considered correct if consensus is achieved in the discourse of the benevolent and the knowledgeable within a community. Empirical relevance (in terms of a large number for n) is not so much in the focus here, as design oriented information systems research is about designing innovation, not about observing existing phenomena.

  • Principle of relevance:

Included in the abstraction is only the part of the object system that complies with the goals of those who will use the model in the future.

  • Principle of economic efficiency:

An abstraction should be further improved and optimized only as long as the benefit brought about by additional optimization exceeds the costs of it. The use of semantic modeling languages and reference models allows to increase economic efficiency in the process of abstracting.

  • Principle of clarity:

An abstraction must be comprehensible and practicable in the eyes of the user.

  • Principle of comparability:

Even if created by means of different methods, abstractions must be compatible.

  • Principle of systematic design:

Different views on the object system (e.g. data view, process view) must be consistent to each other.

It is exactly this combination of methodical know-how and (business) domain knowledge, together with the ability to abstract and structure, that could be called an information systems researcher’s unique selling proposition – if this marketing related term is allowed here.

Evaluation:

Scientific rigor demands that artifacts be evaluated against the research goals specified in the beginning, using the research methods specified in the research plan. An important part of the evaluation is taken over by the review processes for scientific publications. Reviewing articles on design oriented information systems research is not a mechanical process (Has a statistical method be applied correctly?), but rather demands the reviewers’ ability to judge (Is the article’s rationale convincing? Is the justification plausible? Is the way results are derived plausible? Has the research design been explicated and is it adequate? Is the derivation of results guided by theory?) When judging the quality of an article, reviewers should focus more on the article’s rationale than on its empirical evidence. In this respect, evaluation of an article on design oriented information systems research has to take into account multiple perspectives. Unfortunately, this demand is often counteracted by the „socialization“ of many reviewers.

Diffusion:

Design oriented information systems research wants highest possible diffusion of its results to its stakeholder groups. Diffusion instruments mainly are:

  • scientific articles and practitioners papers,
  • conference papers and oral presentations,
  • dissertation theses and habilitation thesis,
  • textbooks (and books in general, which, unfortunately, are hardly considered anymore in ranking criteria),
  • websites, guidelines, principles, trade show appearances,
  • publicly available specifications (PAS) by the German Institute for Standardization (DIN) or by similar organizations,
  • lectures, seminars, on-the-job training,
  • applications for funding and third-party funded research, particularly joint research projects comprising several partners,
  • implemen­tations in companies and public administration, both as open-source products or as commercially used products,
  • enterprise start-ups and spin-offs.

Literature:

Becker, J.; Krcmar, H.; Niehaves, B.: Wissenschaftstheorie und gestaltungsorientierte Wirtschaftsinformatik. Heidelberg 2009.

Becker, J.; Niehaves, B.: Epistemological Perspectives on IS Research - A Framework for Analyzing and Systematizing Epistemological Assumptions. Information Systems Journal, 17 (2007) 2, S. 197-214.

Becker, J.; Rosemann, M.; Schütte, R.: Grundsätze ordnungsmäßiger Modellierung. Wirtschaftsinformatik 37 (1995) 5, S. 435-445.

Gregor, S.; Jones, D.: The Anatomy of a Design Theory. JAIS, 8 (2007),  S. 312-335.

Hardless, C.; Lindgren, R.; Schultze, U.: Technology-Mediated Learning Systems For Project Work - A Design Theory. Scandinavian Journal of Information Systems, 19 (2007) 2, S. 3-36.

Hevner, A. R.; March, S. T.; Park, J.; Ram, S.: Design Science In Information Systems Research. MIS Quarterly, 28 (2004) 1, S. 75-105.

Hooker, J. N.: Is design theory possible? Journal of Information Technology Theory and Application, 5 (2004) 2, S. 73-82.

Iivari, J.: A Paradigmatic Analysis of Information Systems as a Design Science. Scandinavian Journal of Information Systems, 19 (2007) 2, S. 39-64.

Kamlah, W.; Lorenzen, P.: Logische Propädeutik: Vorschule des vernünftigen Redens. 3. Auflage, Stuttgart 1996.

Kuechler, B.; Vaishnavi, V.: Theory Development in Design Science Research: Anatomy of a Research Project. In: V. Vaishnavi; R. Baskerville (Eds.), Proceedings of the Third International Conference on Design Science Research in Information Systems and Technology (S. 1-15). Atlanta: Georgia State University.

March, S. T.; Smith, G. F.: Design And Natural Science Research On Information Technology. Decision Support Systems, 15 (1995) 4, S. 251-266.

Markus, M. L.; Majchrzak, A.; Gasser, L.: A Design Theory for Systems That Support Emergent Knowledge Processes. MIS Quarterly, 26 (2002) 3, S. 179-212.

Niehaves, B.: On Epistemological Diversity in Design Science New Vistas for a Design-Oriented IS Research? In: Proceedings of the International Conference on Information Systems (ICIS2007). Montréal, Québec, Canada, 2007.

Peffers, K.; Tuunanen, T.; Rothenberger, M.A.; Chatterjee, S.: A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24 (2008) 3, S. 45-78.

Pries-Heje, J.; Baskerville, R.: The Design Theory Nexus. MIS Quarterly, 32 (2008) 4, S. 731-756.

Simon, H. A.: The Science of the Artificial. Cambridge, Mass 1996.

van Aken, J. E.: Management Research Based on the Paradigm of  the Design Sciences: The Quest for Field-Tested and Grounded Technological Rules. The Journal of Management Studies, 41 (2004) 2, S. 219-246.

Walls, J. G.; Widmeyer, G. R.; El Sawy, O. A.: Assessing Information System Design Theory in Perspective: How Useful Was Our 1992 Initial Rendition? JITTA : Journal of Information Technology Theory and Application, 6 (2004) 2, S. 43-58.

 

Jan vom Brocke, University of Liechtenstein

Note: The following translation slightly differs from the original German text; in particular, I had to remove some passages that related to specific elements of the German version of the memorandum. 

Dear Hubert,

I am delighted to accept your invitation to comment on the memorandum on design-oriented information systems research. First, I would like to summarize some of the points we already discussed. Second, I had another close look at the paper and I also read the latest EJIS issue in order to identify additional points to be discussed at the International Conference on Wirtschaftsinformatik 2011 in Zurich.

At first I would like to point out that, despite the current controversy over the paper, I am basically a supporter of the memorandum. However, there are some aspects in the current version which – in my view – might not fully live up to the paper’s true intention. What I do appreciate about the memorandum is its basic goals (e.g., to establish rules for design-oriented research, to define criteria for review processes, and to position design-oriented research in the IS domain). Also, I share the view that design-oriented research has received rather little attention in the top journals of our discipline so far. I further support the goal of the memorandum’s initiators to promote innovative and relevant IS research, and to encourage our young academics to conduct research of this type. This being said I nevertheless think that the memorandum’s current version might be misunderstood in parts. To further increase the acceptance of the paper, therefore, I would recommend to partially revise the memorandum text both with regard to its content and with regard to wording and phrasing.

I was asked to comment on concrete passages in the text and to come up with recommendations for their improvement, what I will do in turn. In the following, I will address wording and phrasing issues (1), the comparison of European research and Anglo-Saxon research (2), the chance of publishing design-oriented IS research (3), and the recommendations given by the memorandum (4).

(1) Remarks on the memorandum’s wording and phrasing

In my view, some passages in the introductory paragraphs of the memorandum have been written in an unnecessarily harsh tone, which may have contributed to the partly negative reactions on it. It starts as follows:

Is information systems (IS) research supposed to be beneficial for society and business? Until recently, this question was answered with a clear ‘Yes’ by the European IS research community, particularly by the German speaking countries and Scandinavia […].”

I think that this passage could be misunderstood in at least two ways. First, the statement indicates that today some IS researchers would not answer this question with “yes” anymore (or at least their “yes” would not be said so convincingly). But I am sure that none of our colleagues – regardless of whether he or she works design-oriented or not – would speak in favor of irrelevant research. Second – and I consider this point to be much more problematic – the passage could convey the impression that the initiators and signers of the memorandum hold the view that outside Europe, particularly in the “Anglo-Saxon world,” this question would not be answered with “yes”. The reason why this impression could occur may be that the memorandum describes the different research approaches in a somewhat broad-brush way, especially with regard to the geographic aspect (I will turn to this point in more detail later in my explanations).

However, leading exponents of the discipline have complained for years that research in the field has lacked relevance for the practitioners’ community, which could be surmised from the fact that very few Ph.D.s from the IS discipline have ended up working in business.“

This passage, too, might be considered somewhat provocative. What should be taken into account here is that the Ph.D. system in North America is considerably different from the ones in the German-speaking regions of Europe. Whereas in Germany many university graduates go into business, Ph.D. programs at American top universities usually are explicitly designed to pave the way for an academic career. While we certainly can be pleased that our graduates are so strongly demanded by business, I think it is at the same time pleasing for the American educational system that so many of their Ph.D.s qualify for an academic career. I think this aspect is not adequately acknowledged by the memorandum. Furthermore, apart from my criticism of what is being said here, I also have concerns as to how it is being said. In particular the German version of the memorandum (in which it says „unterkommen“ instead of “have ended up“, which might be read as “they have nowhere else to go”) might suggest that American Ph.D.s generally have little chance in industry practice. At this, we should also note that many of our Anglo-Saxon colleagues have quite close collaborative working relationships with businesses.

While the memorandum’s initiators and signers advocate the idea of design-oriented IS research, they also explicitly welcome behavioral research and the pluralism of methods in the discipline.

I appreciate that the English version of the memorandum says “explicitly welcome” instead of „akzeptieren” (i.e. “accept,” as it is said in the German version). Some of my comments on the German version, which I am leaving out here, related to this expression.

Shortly before this sentence the memorandum mentions „ […] a newly emerging branch of Anglo-Saxon IS research known as design science [that] strives for the same objective as European IS research does in order to meet the demand for more practical relevance of scientific results.“ I believe this particular refers to the 2004 MISQ article by Alan Hevner, Salvatore March, Jinsoo Park, and Sudha Ram. These authors, too, explicitly stress the role and relevance of both approaches discussed in the memorandum (i.e., behavioral and design science). However, these authors emphasize that the two approaches are distinct but complementary, because a proper theoretical basis in many cases is key to accomplish successful design. In their EJIS editorial on the memorandum Iris Junglas, Björn Niehaves, Sarah Spiekermann, Bernd Carsten Stahl, Tim Weitzel and Robert Winter write: „We all agree that we need theory building science as much as we need technology building science. They belong together because without understanding the world we cannot design the right artifacts for it.” (p. 3). In my view, however, the memorandum might create the impression that the two approaches are competing with each other:

Basically, these criteria say that publications providing statistical evidence of empirically identified characteristics of existing IS are favored over publications presenting innovative solutions that are considered highly beneficial for business.“

I think that the two approaches (not only in this passage) are not sufficiently described by the memorandum (behavioral research certainly does not only aim at „providing statistical evidence of empirically identified characteristics“; see below for more on this). Also, I fear that passages such as the above one might convey the impression that the authors of the memorandum did not always intend to describe the two paradigms in a neutral way (for example, „providing statistical evidence of empirically identified characteristics“ vs. „innovative solutions that are considered highly beneficial for business“). The following passage even speaks of a „danger“ with regard to the spread of behavioral research in Europe:

One can no longer deny that European IS research is in danger of shifting from a design-oriented discipline into a descriptive one.

And:

If we seek the cause for this quite questionable trend (both from a societal and a business perspective), the discipline itself needs to be put under scrutiny.

I think IS researchers who are interested in the generation and testing of theory could even feel discredited by such utterances. With regard to the memorandum as a whole, some statements might have better been phrased a little less offensive. As Junglas et al. (2010, p. 2) write: „Again, in the opinion of some, the Memorandum movement’s tone risks splitting the German-speaking IS community (either you are for us or against us) and also the global community (German vs non-German IS)”. Also, Junglas et al. have identified an “emotional aggressiveness” that could be detected in the memorandum (p. 3). While I am sure that this was not the intention of the authors of the memorandum, I share the concerns of these authors. My fear is that such improper wording could counteract the very promising intention of the paper.

Apart from such wording and phrasing issues, the memorandum might in parts be revised also with regard to its content, which I would like to illustrate in the following.

(2) Remarks on the comparison of European and Anglo-Saxon research

Some of the memorandum’s statements on IS research, particularly with regard to geographic and paradigmatic aspects, might have been formulated in a more differentiated manner.

In the Anglo-Saxon world, the corresponding research area to the German term ‘Wirtschaftsinformatik’ (recently translated as ‘business and information systems engineering’) is IS research. Rooted in the business school culture, it is based on a behaviorist approach. Rather than aiming at the design of innovative IS, it focuses more on observing IS characteristics and user behavior.

In my opinion, the memorandum does not adequately describe research in the Anglo-Saxon community if it is reduced to observing IS characteristics and user behavior only (which is what readers might interpret when reading this passage). In fact, the opposite is the case; theory building requires researchers to go beyond mere observation and description and provide scientific knowledge that is abstract, causal, and has empirical relevance. I think that IS research is rather characterized by a high degree of pluralism, both with regard to the methods used and with regard to the units of analysis. Also, reading this passage one might get the impression that certain countries or geographic regions are said to exclusively follow certain research approaches while neglecting others. While I share the memorandum’s view of research in the Anglo-Saxon community primarily following the – as the memorandum calls it – behaviorist approach, I would like to stress also that the methodologies most frequently cited in design-oriented research have their origins in exactly this Anglo-Saxon community. Examples are the paper by Hevner et al. already mentioned above, the work of Vijay Vaishnavi and Bill Kuechler (Auerbach Publications 2008, AIS Homepage 2004/5), or the process model by Ken Peffers, Tuure Tuunanen, Marcus Rothenberger, and Samir Chatterjee (JMIS 2007-08). Besides, a number of very important design-oriented research results also originate from the Anglo-Saxon community, for instance, the highly influential work of Ted Codd on relational database models. Other examples are given by Richard Baskerville, Kalle Lyytinen, Vallabh Sambamurthy, and Detmar Straub in their response on the memorandum. But the same is true vice versa – the German-speaking IS research community is certainly not limited to design-oriented research. This aspect, too, should be more strongly acknowledged by the memorandum.

(3) Remarks on the memorandum’s view on the publication of design-oriented research

In some passages, the memorandum might create the impression that it aims at criticizing the review processes applied in Anglo-Saxon journals (which is a point also taken up by Richard Baskerville et al. in their reply to the memorandum). I believe the rationale for this is as follows: the number of design-oriented articles published in journals from the Anglo-Saxon world is considered relatively low (because „the journals most relevant are based in the Anglo-Saxon scientific community, predominantly following behaviorism as the guiding research paradigm“), young academics from the German-speaking region need to change their attitude and principles accordingly („[…] researchers, and young researchers in particular, have no other choice but to comply with the journals’ evaluation criteria for paper submissions), and therefore there has been a growing „danger“ of turning away from design-oriented research (cf. above). I consider this rationale to be – at least – disputable. First of all, I think it is possible to publish design-oriented research articles in the top IS journals. For example, in 2008 there was a special issue on design science in MISQ, and Alan Hevner currently is, as an MISQ senior editor, responsible for the review and publication of design-oriented articles in this journal. Also, many other top IS journals have published design-oriented research as well (e.g., Peffers et al. in JMIS; March and Smith in DSS). I share the memorandum’s view that design-oriented research has been rare in the top journals, and I appreciate that the paper has stimulated a discussion on the publication of design science. The fact that the number of design-oriented articles is so low, however, cannot just be due to presumed preferences regarding the use of research methods among the members of the journals’ editorial boards (an impression the memorandum, in its current form, could however convey). If I compare design-oriented research with other established (both qualitative and quantitative) research approaches, I more and more get the impression that it often takes a lot of time to properly conduct all phases (and iterations) relevant in an entire design-oriented research process, and that it is very difficult to document all the results of such an endeavor in one single paper. Good design-oriented research needs to provide transparency with regard to many aspects, including design requirements, artifact scope, innovative potential, theoretical foundation, presentation and explanation of artifacts, exemplary use cases, evaluation etc. A full and detailed documentation of all these aspects might easily exceed the space that some journals provide for the publication of an article (i.e., the maximum number of characters, words, pages etc.). As such, I consider the conduct of design-oriented research, and an adequate publication of the results respectively, per se as a big challenge. A question that comes to my mind however is: How does the memorandum, in its current form, support young academics to cope with these challenges? A strategy currently discussed (in particular in the context of paper-based dissertation projects) is to publish design-oriented research not necessarily „en bloc“, but to publish the results of certain design science activities separately (e.g., a requirements analysis or an evaluation of existing artifacts). The difficulty with this is that design science is a highly iterative process (see also below) that might not be adequately documented this way. The memorandum, however, does not address these and related questions that young academics are currently confronted with. In the following, I will try to indicate how the memorandum could be strengthened in this regard.

(4) Remarks on the memorandum’s recommendations

I consider some of the memorandum’s recommendations as too generic and difficult to put into practice. For example, many design-oriented studies lack a thorough and comprehensible evaluation of artifacts. This may be due to the fact that the evaluation of artifacts is generally difficult to accomplish and document (because the memorandum is right: „Only in rare cases can an artifact produced be (formally) verified.“). Difficulties here mainly refer to the selection of adequate research methods, formulation and verification of adequate hypotheses and implementation and testing of artifacts in practice. As for these problems, the memorandum only presents one statement:

Scientific rigor demands validation of artifacts produced against the objectives specified, applying the methods stated in the research plan. The review process prior to scientific publications is part of the evaluation.

Later on in the memorandum text, another statement can be found on the evaluation of artifacts:

Artifact evaluation is done through laboratory experiments, pilot applications (i.e., instantiation of prototypes), simulation procedures, expert reviews, and field experiments (i.e., instantiations in a number of user organizations).“

Apart from these statements and rather unspecific recommendations, I do not agree with the view that a review process can be a substitute for or “part of” an evaluation process. Put simply, it is very unlikely that an artifact can be evaluated by reviewers. The usefulness of a new software product, for example, can be thoroughly tested only in a concrete business context. What is required here is a more detailed guideline for young academics on how they can evaluate artifacts.

When it comes to the result types of design-oriented research, I would recommend for the memorandum to also relate to design theories, such as proposed by Shirley Gregor and David Jones (JAIS 2007), or Joseph Walls et al. (1992; 2004). Design theories provide prescriptive statements on how to design artifacts and could also be taken into consideration in the memorandum’s paragraph on the objectives of design-oriented research:

Design-oriented IS research aims to develop and provide instructions for action (i.e., normative, practically applicable means-ends conclusions) that allow the design and operation of IS and innovative concepts within IS (instances).“

Furthermore, I believe that the process of design-oriented research is not sufficiently described in the paper. While the memorandum distinguishes four phases of a typical design research process (analysis, design, evaluation, and diffusion), it does not sufficiently deal with the interrelations between these four phases (except for a short remark on „iterations“). I personally see design-oriented research as a process that exhibits a circular logic in particular (many artifacts are, for example, developed and evaluated in action research studies; see Marcus et al. 2002; Jones and Gregor 2004). It is the goal of design-oriented research (as it is apparently understood in the memorandum) to solve problems identified in practice (analysis) by means of construction of innovative artifacts (design), and to assess the artifacts’ problem solving capacity in practice (evaluation), with the results being taken into account continuously for improvement of the artifacts. This may require many evaluation and adaptation cycles, and it can consume a lot of time. If a “satisfying” solution (e.g., with regard to certain (meta) requirements) has been achieved, the results need to be communicated (diffusion). The activity of feeding back evaluation results into the design phase, however, is not described by the memorandum in any detail. All in all, I thus believe that some of the memorandum’s recommendations remain too generic. In this regard, Junglas et al. (2010, S. 2) write: „Also, many just don’t think that the Memorandum offers anything new beyond already published criteria.” Of course, the paper’s intention was to espouse “academic freedom in science and teaching“ and allow researchers to be „free to decide on research objectives and research methods [...] as long as they adhere to the above mentioned principles“. This, of course, requires a certain degree of generality in the presentation. In choosing this way of presentation, however, the memorandum comes up with only few recommendations that could be put into practice. In my opinion, more detailed recommendations would strengthen the memorandum.

I hope that with these remarks and recommendations I could make a contribution to improve the memorandum. I consider the debate on the paper exciting and stimulating. For the memorandum’s further development, the remarks by Richard Baskerville et al. provide very important input, and so does the editorial by Iris Junglas et al. As much as I appreciate the debate, however, I think we should make sure it is led in an open and tolerant way. I thus appreciate that the German comments on the memorandum have recently been made available in English. I would be really happy if the paper could support young academics to carry out design-oriented research (which is what the memorandum’s authors had in mind, I think). However, in their editorial Junglas et al. (2010, S. 2) write: „The Memorandum may have created a split between those who have endorsed it and those who have not. It is growing as a major concern of doctoral students and junior faculty who are considering a career in academia. Rumors abound about fears that being identified on either side of the debate could lead to substantial disadvantages if this turns out to be ‘wrong’ side for the situation at hand.”

It is my strong belief that the memorandum has initiated an important debate. Now the German-speaking IS research community needs to come up with a good response. I personally think, however, that even more important than the discussion by itself, will it be to actually conduct good design research. In this respect, the memorandum should be seen as a call for innovative design research, which not only yields substantial benefit to the practitioners community but which is also conducted and documented rigorously and comprehensibly. I am convinced that our international colleagues too will acknowledge and appreciate such research. And while the German-speaking IS community will certainly benefit from their broad experience in design research, it should at the same take the chance and learn from other branches of the discipline.

European IS research has an excellent opportunity to build upon its strengths in terms of design orientation and at the same time demonstrate its scientific rigor through the use of generally accepted methods and techniques for acquiring knowledge.

I fully agree to this statement.

Kind regards
Jan

 

Baskerville, R., Lyytinen, K., Sambamurthy, V., Straub, D. (2010): A response to the design-oriented information systems research memorandum, in: European Journal of Information Systems, 20 (1), S. 11-15.

Gregor, S., Jones, D. (2007): The anatomy of a design theory, in: Journal of the AIS, 8 (5), S. 312-335.

Hevner, A. R., March, S. T., Park, J., Ram, S. (2004): Design science in Information Systems research, in: MIS Quarterly, 28 (1), S. 75-105.

Jones, D., Gregor, S. (2004): An information systems design theory for e-learning, in: Proceedings of the 15th Australasian Conference on Information Systems (ACIS’04), paper 23.

Junglas, I., Niehaves, B., Spiekermann, S., Stahl, B. C., Weitzel, T., Winter, R., Baskerville, R. (2010): The inflation of academic intellectual capital: the case for design science research in Europe, in: European Journal of Information Systems, 20 (1) S. 1-6.

March, S. T., Smith, G. F. (1995): Design and natural science research on information technology, in: Decision Support Systems, 15, S. 251-266.

March, S. T., Storey, V. C. (2008): Design science in the Information Systems discipline: an introduction to the special issue on design science research, in: MIS Quarterly, 32 (4), S. 725-730.

Markus, M. L., Majchrzak, A., Gasser, L. (2002): A design theory for systems that support emergent knowledge processes, in: MIS Quarterly, 26 (3), S. 179-212.

Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., Loos, P., Mertens, P., Oberweis, A., Sinz, E. J. (2010): Memorandum on design-oriented information systems research, in: European Journal of Information Systems, 20 (1), S. 7-10.  

Peffers, K., Tuunanen, T., Rothenberger, M., Chatterjee, S. (2007): A design science research methodology for Information Systems research., in: Journal of MIS, 24 (3), S. 45-77.

Vaishnavi, V., Kuechler, W. (2004/5), Design research in Information Systems, January 20, 2004, last updated August 16, 2009, desrist.org/design-research-in-information-systems/.

Vaishnavi, V., Kuechler, W. (2008): Design science research methods and patterns: innovating information and communication technology, Auerbach Publications, Taylor & Francis Group, Boca Raton, Florida, USA.

Walls, J.G., Widmeyer, G. R., El Sawy, O. A. (1992): Building an information system design theory for vigilant EIS, in: Information Systems Research, 3 (1), S. 36-59.

Walls, J.G., Widmeyer, G. R., El Sawy, O. A. (2004): Assessing information system design theory in perspective: how useful was our 1992 initial rendition?, in: Journal of Information Technology Theory and Application, 6 (2), S. 43-58.

 

Ulrich Frank, University of Duisburg-Essen

Achieving methodological rigor in information systems research

1. Establishing methodological rigor – a challenge and an opportunity

The discipline of Wirtschaftsinformatik (hereinafter referred to as “information systems research”) aims at theories and methods supporting the development and organizational implementation of business information systems. In this respect, information systems are not considered as an end in itself, but as a means for maintaining and improving the competitiveness of organizations. While there is consensus on this view on an international level (also by the Information Systems (IS) branch of the discipline), opinions regarding an appropriate scientific methodology to achieve scientific rigor in information systems research are diverging. IS aims at supporting practice by means of theories, which – if sufficiently evaluated – could serve as a basis for the design of information systems. IS emphasizes a corresponding research methodology which is oriented towards idealized processes known from natural sciences. Behavioristic empirical social sciences are based on the evaluation of hypotheses, which can be derived from theories, by empirical analysis. In doing so, theories are validated and – if need be – adapted. This approach appears to be coherent and promising, as it aims at the development of substantial theories and as it implicates the explicit use of a research method, which generally is said to be a „characteristic feature of scientific procedures, and – pars pro toto – a characteristic feature of science itself“ ([Lore84], S. 876). In contrast, information systems research mainly aims at the construction of innovative artifacts and the design of corresponding organizational contexts. For quite some time now, this type of research has been named “design oriented” or “construction oriented” information systems research. Examples of results of such research are software prototypes, conceptual (reference) models, modeling languages, methods, conceptual frameworks, models of cross-organizational value creation chains, or new business models. This approach, too, appears to be plausible, as it offers practitioners substantial orientation for the design and use of future information systems. This assumption is backed by the enormous reputation of information systems research and its graduates among practitioners. However, until a few years ago it was not common in information systems research to explicate a methodological basis for conducting research. Especially due to growing international competition, the concept of information systems research has come under pressure. Against this background, information systems research sees itself confronted with two tasks. First, it has to analyze if it is reasonable to adopt the concept of IS – and thereby adopt the research method prevailing in IS. Second, if IS is rejected as the paradigm to follow, a methodological basis has to be established that takes into account the specific requirements of information systems research.

The research concept of IS in many respects does not seem to be very promising. One indication supporting this view is the significant uncertainty and discontent in the discipline itself. For years, the discipline’s basic concept has been questioned again and again in numerous publications – partially in quite dramatic words ([BaLa92], [BeWe96], [LyKi04]. For example, the authors of a recently published article complain about research results of IS being of no significant benefit for practical purposes [GiBh09]). If analyzed more closely, a number of reasons can be found for this shortcoming of IS (see also [Fran06]). One reason is that research which is limited to the analysis of existing forms of development and use of information systems has only limited potential to trigger innovation in practice. Moreover, a lack of substantial theories can be stated, leading sometimes to the production and publication of amazingly trivial results. Also, teaching Ph.D.s complex statistical methods seems to be more important than developing a consistent terminology allowing for differentiated analysis of the research subject. In view of these substantial problems IS has to cope with, it would be a bad choice to adopt this model for being used in information systems research. As a consequence, the question arises as to what could be an appropriate methodological basis of information systems research.

2. Requirements to be met by a research methodology

The sustained criticism of behavioristic research in IS has made the discipline take up other research approaches. One of them is the hermeneutical approach. Hermeneutical methods often are denoted with the term „qualitative“, as opposed to behavioristic research generally being labeled as „quantitative“. However, if we take a closer look, such differentiation does not seem to be adequate, as quantity and quality are orthogonal dimensions. Also, hermeneutical approaches partially give up the principle of objectivity of research, assuming that access to social systems is always subjective. Unlike behavioristic methods, which aim at explaining certain phenomena by referring to substantial and proven theories, hermeneutical methods aim at understanding certain action systems: „Providing for understanding means to enable the listener or reader to mentally reconstruct a certain issue by means of adequate associations ... some form of empathy or inner reconstruction of the atmosphere.” ([Wri74], S. 20). To promote such inner reconstruction, hermeneutical methods use associations with known patterns of sensual perception and cognitive conceptualization, for example in the form of analogies or metaphors. For construction oriented research, a research method seems to be of particular importance which some years ago was introduced by the term „design science“. This approach, indeed, addresses the issue of scientific development and validation of artefacts according to the notion of designing in information systems research. However, even though the „design science“ approach has come up with a number of useful recommendations, it does have considerable shortcomings [Zele07].

Before we address the question as to how information systems research can achieve real methodological rigor, a number of basic questions need to be clarified first, referring to the differentiation of results gained through academic research versus results gained in other social sub-systems, to special characteristics of research methods, and to the significance of research methods for the work of researchers. It is true that highly sophisticated artefacts of the kind we can find on the agenda of information systems research have been developed in practice, such as architectures of complex information systems, comprehensive conceptual models, or modeling languages. It cannot be denied by all means that such artefacts also can live up to the requirements of academic research – as working according to scientific principles not necessarily is reserved for those who work in academic institutions. Still, however, we should have available some criteria allowing to distinguish between artefacts developed according to scientific principles and artefacts developed by practitioners, as such differentiation is critical for the scientific identity and legitimacy of information systems research. In order to achieve such differentiation, three postulates can be taken into consideration which apply to scientific results in general: originality, abstraction, and justification. Originality of knowledge or a result is given if this knowledge or result is new in some way, i.e. if it is capable of surprising experts. Abstraction refers to the requirement that academic research should not deal with finding a solution for one specific case, but with finding a solution applicable to many similar cases. Justification means that scientific results always demand to be justified in a comprehensive manner. Moreover, knowledge and results produced by an application oriented discipline should be relevant, i.e. it should be capable of being beneficial to the application area intended. This does not mean that this benefit needs to be rendered immediately – and should not exclude the question as to what interests determine the prospective use of certain knowledge or a certain result.

Apart from these basic characteristics of scientific knowledge, researchers should always be aware of the preconditions, limits, and goals of their work. A research method is capable of supporting this reflection by creating transparency regarding important interrelations. Like it is the case with the methods for analysis and design being research subjects of information systems research, a research method consists of a linguistic structure in terms of a specific terminology, a procedure, and a set of basic presuppositions and evaluation criteria. The terminology supports the targeted structuring of academic research. In this respect, we should think of some central terms, such as theory, hypothesis, model etc. The procedure describes idealized patterns for the process of research projects. Basic presuppositions refer to the epistemological and ontological position. And evaluation criteria refer to the justification of scientific knowledge and results.

3. Research method configuration – an approach

The approach for configuring a research method outlined in this section is based on a number of presuppositions:

  • While differentiating various forms of research (like behavioristic, hermeneutic, or construction oriented) makes sense from an analytical point of view, such clear differentiation is not always possible when it comes to assigning certain research projects with one of these labels. Many projects in information systems research are characterized by a combination of these different forms.

  • Due to what has been said under the previous bullet point, it would be inappropriate to limit oneself to using only one single research method. Rather, a method is an instrument that should be adapted to the specific requirements of each project.

  • Apart from that, sticking to a certain method bears the danger of designing a research question in such a way that it accommodates the requirements of this method. Research should be motivated primarily by the desire to gain new knowledge, though.

  • Thus, the individual researcher should have the opportunity to choose the methodological approach suiting best the respective research question and the specific abilities of the researcher.

Even though these presuppositions indicate that information systems research should not be limited to construction oriented approaches, the following paragraphs, still, focus on this type of research. This is done because of the specific epistemological problems connected with construction oriented research. Reflections on this topic need to be limited to some central aspects here. A more detailed description of the construction oriented approach can be found in [Fran06]. A special challenge with regard to construction oriented research refers to the postulate of justification. Justifying a statement basically means to ascribe this statement to other statements which are commonly accepted or evident. If this is not possible, the truth of the statement has to be verified. In order to do so, a theory of truth has to be chosen. The correspondence theory of truth says that a statement is true if it complies with the fact it describes. The correspondence theory of truth is typical of the behavioristic method. By the verification procedure the central parameters are operationalized and the corresponding manifestations are captured in a representative selection. It is obvious that verification of an artefact developed in information systems research against reality must fail, as such an artefact is not just a technical concept but always a concept of the corresponding action system as well (or, as we also could say, a concept of a possible world [Fran09]. The coherence theory of truth says that a statement can be considered true if it can be embedded coherently in existing, commonly accepted knowledge. The classical method of verification here is literature analysis. However, surveys are often conducted here too. The consensus theory of truth says that a statement is true if renowned experts, taking into account certain rules of communication, agree that the statement is true. The coherence theory and the consensus theory of truth are important elements of hermeneutical, interpretative approaches for justification. A concept clearly different from these theories is the theory of formal truth. This theory is limited to statements about the characteristics of formal systems. It thereby excludes statements of empirical quality, but has the advantage that it allows to prove the assignment of truth predicates. And so we come to another hypothesis which the approach presented here is based on: Scientific knowledge is not connected to a certain concept of truth exclusively. The choice as to which notion of truth is appropriate in each case depends on a number of characteristics of the statement to be validated (see 3). It is obvious that truth alone is not sufficient to justify the results of construction oriented research, as these constructions aim at the accomplishment of certain goals. Goals and intentions, however, cannot be characterized as being true or not. So, a construction can only be validated with regard to being appropriate to accomplish certain design goals. However, such a validation can contain statements that can be assessed in terms of being true or not (for example, if the goal G is to be achieved, then the measure M is effective).

The basis of the configuration – besides the knowledge related postulates already being mentioned – is the transparency postulate, saying that all presuppositions a construction is based on need to be explicated, unless they are evident. When doing so, some special characteristics of construction oriented research need to be taken into consideration. Presuppositions often imply justification to be done on two levels. One refers to the general appropriateness of a design feature in supporting the accomplishment of a predefined goal. The other refers to the question as to whether this general appropriateness in fact promotes successful action, i.e. whether and how a certain system characteristic is used by prospective users. Let’s look at the following example: „IT architecture models and relevant usage context models are appropriate instruments for supporting strategic IT planning.“ Regardless of the not so trivial question as to what requirements precisely should be met by such models, this statement can well be justified with regard to its general appropriateness. To do so, only a few steps are needed: The statement about the subject of strategic IT planning being complex can either be presupposed as being consensus or illustrated by a brief consideration of the subject. The statement about complexity aggravating (planning) decisions can also be presupposed as being consensus. Here, it might be helpful to refer to appropriate theories from cognitive psychology. As models by definition are goal oriented abstractions, the reduction of complexity they bring about requires no further justification. In contrast to this, the statement about models developed by means of a certain modeling language being accepted by prospective users is of empirical quality. Verifying this statement is much more difficult. An empirical analysis as suggested by the behavioristic approach requires the existence of appropriate theories from which this statement can be deduced. Such theories often do not exist, though. If surveys are conducted, or experiments involving selected representatives from the respective target group, the results gained usually cannot be generalized, as the motives of these interviewees and representatives are determined by individual patterns of preference and qualification, which can hardly be perceived as invariant. Rather, such patterns very often are contingent, i.e. they can occur in different forms and can have different manifestations.

Another special aspect of construction oriented research lies in the fact that requirements may compete with each other, or may even contradict each other. For example, flexibility competes with integratability. Whereas flexibility rather speaks for a loosely coupled architecture, high integratability rather requires tight coupling. One way to meet such a conflict of requirements could be to provide for mitigation (for example, by introducing suitable concepts of abstraction). Another could be to prefer one of the requirements over the other. However, justifying such a decision is very likely to be problematic, as a corresponding theory usually does not exist. Conducting a survey in practice might be difficult as well, as the total of effects of the design decision might be hard to communicate  – and as the assessment of the design decision by prospective users is contingent.

For illustration of the process of configuration of a research method, we take a look on an established procedure for developing artifacts, comprising four phases: motivation, requirements analysis, design, and evaluation. “Motivation” aims at demonstrating that the problem under investigation is of relevance, i.e. that it addresses a gap in research. “Requirements analysis” aims at specifying the requirements the artefact to be developed is supposed to meet as precisely as possible. “Design” refers to the development of the artefact in an iterative process, of which an important part is to name the central design decisions. “Evaluation“, finally, serves to verify the artefact against the requirements specified before. In all phases, all presuppositions which are not evident or for which obviously there is no consensus need to be made transparent. For justification an appropriate criterion is to be selected (see Table 1). In this process, it may turn out that the corresponding justification procedure cannot be executed, due to reasons such as excessive demand of resources. If this is the case, the researcher has to switch to another, weaker justification procedure. In the example mentioned, this could mean that the researcher makes a literature analysis to search for already justified statements backing the respective presupposition. It is important her to note that such an approach is only convincing if literature is not searched selectively, i.e. if the search is not limited to statements that fit the researcher’s intentions. In difficult cases the researcher should feel free to indicate a statement that appears to be reasonable but for which a convincing justification cannot be delivered yet as a working hypothesis. By doing so, the transparency postulate is met, and scientific knowledge is generated that remains to be verified. The criteria listed in Table 1 are supposed to support in the selection of a certain justification procedure. The formal theory of truth is excluded here, since the preconditions for applying it should be clear.

Correspondence theory of truth

Precondition

A substantial theory exists from which the hypothesis can be deduced. A theory can be considered substantial if it contains significant information and if it is proven and tested. If no theory exists, a survey needs to be conducted with regard to a characteristic value of interest (for example: „CIOs consider IT management dashboards an effective instrument for supporting decision-making.“). However, the result gained from such a survey can render a limited justification only.

Procedure

An empirical field study can be recommended if the hypothesis can be operationalized without any significant distortions, i.e. if the measurable parameters adequately represent the issue under investigation. At the same time, the measuring procedure should be reliable, i.e. it should reliably differentiate the different values of the parameters to be measured. Another precondition refers to the availability of a representative selection of research objects. A field experiment can be an option if it is possible to change the value of the independent variable in a controlled manner. A laboratory experiment can be an option if the issue under investigation can be reproduced in the laboratory without any significant distortions. A case study would be appropriate only to falsify a hypothesis and/or the theory from which it was deduced. However, the requirements on theories in social sciences are not so high, and so falsification of a hypothesis by a single case would not be considered to be sufficient.

Commentary

In most cases, the correspondence theory of truth cannot be applied due to a substantial theory missing. Besides, the requirements on the corresponding procedures often cannot be met in a satisfying manner. This is particularly true if we think of the effort needed to bring about a representative implementation of an artefact to be evaluated.

Coherence theory of truth

Precondition

Justifications exist in literature which can be used to back the hypothesis. The hypothesis should not refer to a single issue only, but should aim at generalization (postulate of abstraction).

Procedure

Useful sources are searched for in a literature analysis. After that, by means of adequate interpretation and analysis a rationale is developed which helps justify the hypothesis. Alternatively, it is possible to conduct an expert survey to validate if the hypothesis is coherent with the excellent knowledge generated this way.

Commentary

Both literature analyses and expert surveys bear the danger of producing distortions. When searching through literature such sources are preferably searched for which fit the researcher’s intention. And the quality of an expert survey depends largely on the quality of the experts’ statements.

Consensus theory of truth

Precondition

The hypothesis should not refer to a single issue only.

Procedure

Ideally, applying this theory requires to set up a rational discourse. Participants of such a discourse must be knowledgable, must not be opportunistic, and should be allowed to speak freely (recommendations on this can be found in [Habe81] and [Lore74]). A simplified form of such a discourse can be reconstructed from literature sources by means of juxtaposition of arguments.

Commentary

Setting up rational discourses is connected with considerable methodical challenges, since the decision as to whether the preconditions for a discourse are sufficiently met contains a large potential of distortion. Apart from that, experts – regardless of how they should be identified – in most cases might not be available. Ideally, review processes – provided they lead to consensus – could be used as a validation instance. However, reviewers usually already expect a convincing justification.

Table 1: Criteria for selecting justification procedures

4. Concluding remarks

The paper at hand outlined how a research method can be configured. An approach like this is in no way limited to construction oriented research, but can be applied to any form of information systems research (and to research on other subjects). Even though the approach includes criteria directing the configuration process, configuration decisions need to be made for each specific case, taking into account the specific research subject as well as the possibilities and interests of the researcher. Thus the researcher should design an appropriate method themselves, and he or she should critically evaluate the method by using it consciously and reflecting on it. This point really needs to be stressed here, as sometimes one gets the impression that there are „established“ research methods – and one has no other choice than to learn how to apply them. Such a view is understandable, as it addresses the need for security of many stakeholders. However, it is not convincing, as it violates fundamental principles of scientific culture – namely freedom and criticism (also with regard to research methods, of course). Documenting research results does not always require to explicate the research method used. Also, consuming more time and space for explaining the research method than for presenting the research results is improper. However, the development and presentation of the research results in any case should reveal also the research method that has been used.

Literature

Banville, C.; Landry, M.: Can the Field of MIS be Disciplined? In: Galliers, R. (Ed.): Information Systems Research: Issues Methods and Practical Guidelines. London et al.: Blackwell 1992, S. 61-88

Benbasat, I.; Weber, R.: Research Commentary:  Rethinking Diversity in Information Systems Research. In: Information Systems Research. Vol. 7, No. 4, 1996, pp. 389-399

Frank, U.: Towards a Pluralistic Conception of Methods in Information Systems Research. ICB Research Report No. 7, Universität Duisburg-Essen 2006

Frank, U.: Die Konstruktion möglicher Welten als Chance und Herausforderung der Wirtschaftsin-formatik. In: Becker, J.; Krcmar, H.; Niehaves, B. (Hrsg.) Wissenschaftstheorie und gestaltungsorientierte Wirtschaftsinformatik. Physica-Verlag: Heidelberg 2009, S. 167-180

Gill, G.; Bhattacherjee, A.: Whom Are We Informing?  Issues and Recommendations for MIS Research from an Informing Sciences Perspective. In: MISQ, Vol. 33, No. 2, 2009, S. 217-235

Habermas, J.: Theorie des kommunikativen Handelns: Vol 1: Handlungsrationalität und gesellschaftliche Rationalisierung. Suhrkamp: Frankfurt/M. 1981

Lorenzen, P.: Konstruktive Wissenschaftstheorie. Suhrkamp: Frankfurt/M. 1974

Lorenz, K.: Methode. In: Mittelstraß, J. (Hrsg.): Enzyklopädie Philosophie und Wissenschaftstheorie. 2. Bd., Bibliographisches Institut: Mannheim, Wien, Zürich 1984, S. 876-879

Lyytinen, K.; King, J.L.: Nothing At The Center?: Academic Legitimacy in the Information Systems Field, Journal of the Association for Information Systems, Volume 5, No. 6 (2004), pp. 220 – 246

Myers, M.D.; Baskerville, R.L.: Commentary on Gill and Bhattacherjee:  Is There an Informing Crisis? In: MISQ, Vol. 33, No. 2, 2009, S. 663-665

Wright, G. H. v.: Erklären und Verstehen. Athenäum: Frankfurt/M. 1974

Zelewski, S.: Kann Wissenschaftstheorie behilflich für die Publikationspraxis sein? Eine kritische Auseinandersetzung mit den "Guidelines" von Hevner et al. In: Lehner, F., Zelewski, S. (Eds.): Wissenschaftstheoretische Fundierung und wissenschaftliche Orientierung der Wirtschaftsinformatik. GITO: Berlin 2007, S. 74-123

 

Oliver Günther, Humboldt University of Berlin

Despite some slight atmospheric disturbances caused by the publication of the memorandum (disturbances that may have been avoided to some extent if the communication strategy had been a little bit more diplomatic), I welcome the debate triggered by the memorandum, since a number of questions have been raised that can be considered central to our scientific discipline:

  • Which methodological approaches are typical or acceptable in information systems research?
  • When can research in our discipline be considered successful?
  • How can the different methodological approaches be evaluated against selected success criteria?
  • Are there any regional or cultural differences when responding to the questions above?
  • Are there any sub-communities in our discipline that – consciously or unconsciously – prefer one specific research approach or research paradigm over others? And if so, why are they doing so?
  • If such sub-communities can be identified, would increased collaboration between them be desirable in order to obtain more reliable and more relevant research results?

In fact, every scientific discipline should ask itself these questions on a regular basis, since such questions are highly relevant with regard to the role a scientific discipline plays in a society, and also since the answers to these questions are always subject to an enormous dynamic. In recent years particularly the globalization of research (and of the forums research results are published in) has had a sustained impact. There has been an increasing number of complaints about scientific articles being rejected simply because the authors used approaches or methods not favoured by the respective forum.

Insisting on certain approaches and research methodologies while rejecting others is the wrong way to go, particularly in a scientific discipline that is inherently interdisciplinary and polymethodic, like information systems research. I myself have witnessed many times how researchers have achieved more reliable and more relevant scientific results by deliberately employing a diversity of methods to solve a problem. Information systems research in particular should present itself as an advocate for method openness and method diversity. This does not mean, of course, that information systems research should employ – let’s say – astrological research on ascendants. Approaches that are proven and widely accepted, however, such as the behaviouristic approach or the design oriented approach, must be able to exist alongside each other in our discipline (and also in our scientific journals!), both as competing approaches and – increasingly – as approaches used in symbiosis. Method diversity and different methodical backgrounds of information systems researchers are the foundation of our success. We should be proud of that instead of apologizing for it.

To make a long story short: Let many flowers bloom!

 

Felix Hampe, University of Koblenz-Landau


As a response to the growing interest in the Memorandum in international circles and given the upcoming EJIS publications, I have written my comments in English.

I would like to express my thanks and strong support for the initiative the authors of the Memorandum have taken, even in the face of some harsh criticism by other well-respected academic scholars.

From my point of view the Memorandum was overdue, given such provoking judgements and suggestions as found in the well-known article “Why the Old World Cannot Publish“. For me, this article expresses in many ways, what I indeed observed as a common tendency of many US-dominated publication venues: a lack of pluralism. I am happy to note that such mindset is not universal and some of the most respected scholars of our discipline support a more pluralistic outlook. For example, Bob Galliers, expressed at his ACIS 2010 keynote a strong call for pluralism of research approaches to be used in IS. The rebuttal of the Memorandum seems to support this viewpoint, too. The important thing is to translate this openness to pluralism into action and to let different forms of research evolve and flourish.

Many of the young researchers at foreign universities (at least those I have had the pleasure to meet at seminars in recent years) appreciate being offered the pluralistic perspective on research as so excellently presented in the contribution by our colleague Ulrich Frank in “Towards a Pluralistic Conception of Research Methods in Information Systems Research“. This paper and the Memorandum itself clearly belong to the group of ‘must reads’ in my PhD student colloquia on research in IS.

As a final snapshot from our neighbour discipline Informatics I cite a paragraph from a recent call for papers, which may provide an example of the way other successful and well respected communities open up for “gestaltungsorientierte Forschung”.  The 3rd and 4th paragraph of  the CFP: 2nd ACM SIGHIT International Health Informatics Symposium (IHI 2011) reads as follows:
IHI 2011 serves as a venue for the discussion of innovative technical contributions highlighting end-to-end applications, systems, and technologies, even if available only in prototype form (e.g., a system is not deployed in production mode and/or evaluation may be performed by giving examples). We strongly encourage authors to submit their original contributions describing their algorithmic contributions, methodological contributions, and well-founded conjectures based on an application-oriented context. A paper does not have to be comprehensive and can focus on a single aspect of design, development, evaluation, or deployment.

Contributions in the realm of social and behavioural issues might include empirical studies of health-related information use and needs, socio-technical studies on the implementation and use of health information technology, studies on health informatics in the context of community impact and implications, studies on public policies on leveraging health informatics infrastructure, among others.

I am not aware of many IS publication venues showing such an inclusive mindset in a cfp, although IS research does not study substantially different types of phenomena.

As a final remark let me suggest to bring the discussion even further by raising (naughty) questions about the reasons for these obviously very controversial positions on what constitutes dominant high standard IS research:

  • Which percentage of articles in the last 10 years taken from the ‘basket of eight’ has had serious impact within and outside the IS community, i.e. are well known and produced results which continued to be of significance for the real world of IS/IT?
  • Might the development of curricula in many IS schools have caused an insufficient skill set to be able to develop or implement non-trivial IS prototype solutions? Does that force their graduates and PhD candidates to conduct mainly behaviouristic type of research as they haven’t any profound IT skills to develop any artefacts of this category?
  • What about their professors and lecturers supervising such research projects and being under pressure to have a high publication output themselves to get promotion or budget (as the EJIS Editorial touches on many times)?

  • (I can’t resist to refer here to the uncountable TAM papers successfully published in the area of IS research, which are based on midsize samples from students but applying sophisticated statistical procedures leading to most unsustainable and often trivial results. For the field of mobile applications I certainly have seen many dozens over the last years.)
  • Don’t we look at a self-reproducing system here, namely the reviewers might tend to accept the type of work they are capable to conduct themselves? Has this in any way lead to the “Crisis in IS”? Again: is the current system the best of all possible designs with respect to defining the ranking lists and grant allocation mechanisms used in our community? Do we fairly treat the next generation of scholars in defining their research goals this way?

Finally, from my point of view the memorandum should be regarded as “food for thought” to stimulate a discussion within our IS community. I strongly support the authors in their initiative to challenge the ‘journal-establishment’, which obviously leads to rebuttals like the one appearing in EJIS – and, doesn’t that read a bit self-righteous?

 

Thomas Hess, University of Munich


This paper is a written version of an oral presentation held by the author on September 17th, 2009, at the University of St. Gallen, during a workshop on design oriented information systems research. The paper is not a direct comment to the memorandum, but explains a partial aspect of the memorandum in detail.

 

Subject of (design oriented) information systems research

1. Background

Design oriented research approaches traditionally are widely spread in German-speaking  information systems research (see, for example, Wilde/Hess 2007). This branch of information systems research is particularly acknowledged by practitioners. In recent years more and more criticism has been uttered with regard to the aspect of metho­d consistency in design oriented information systems research. Against this background, a movement has developed that wants to strengthen and sharpen the methodological profile of the discipline. In this respect, an important aspect seems to be the question: What is the subject of design oriented information systems research?

This short article approaches this question in three steps. In the first step, the object domain of information systems research is specified. Building upon the findings from the first step, the second step aims at identifying important, general topics of information systems research. Finally, the third step is about the refinement of the results from the previous two steps with regard to the special view of design oriented information systems research.

2. Object domain of information systems research

The object domain of information systems research are information systems in enterprises and in public administration, including the context these systems are used in (see Mertens z.B. et al. 2006, S. 1). Such information ­systems can be understood as socio-economic systems, consisting of a human component (the human being), a machine component (information technology), and the tasks to be supported (the context). Figure 1 illustrates this basic assumption.

 

Fig. 1: Information systems as socio-economic systems

This definition takes a reduced view with regard to at least three aspects. Firstly, only those infor­ma­tion ­­­­­systems are relevant for information systems research that include a machine component (mere paper based information systems, for example, are irrelevant here). To be more precise, we should speak of IT based information systems (i.e. application systems or application software).

Secondly, the object domain of information systems research is not just constituted by information systems, but by information and communication systems. So when we speak of the machine component of socio-economic systems, we must include information technology and communi­cation ­technology.

Thirdly, speaking of „enterprises and public administration“ is somewhat of too narrow a view. Of course, it is true that enterprises and public administration started implementing information systems for internal use only. With the upcoming of high-performance communication technology (the Internet being the most important one), however, these systems were increasingly opened to be used by external users  as well (customers, business partners, for example). Moreover, information systems today are used to make partially automated or even fully automated transactions (some well-known examples can be found in the finance industry, for example). Also, information systems meanwhile have become products or constitute the basis for service offerings used both by commercial and private users (the latter comprising, for example, auction platforms like eBay or music distribution systems like Apple Music Store).

Furthermore, it is important to say that the context information systems are used in is crucial for information systems research. At the beginning, the context under investigation in information systems research was the business function to be supported. The next phase, spanning many years, saw the process ­perspective as the dominant context (if we think of the intensive discussion on the interdependence of business process design and implementation of (standard) software). Meanwhile, the context is viewed in much broader terms, with the design of business models and even the establishment of enterprises having an effect on and being affected by the machine component and the human component of information systems (if we think of Google, for example).

3. General topics of information systems research

Traditionally (see Mertens et al. 2006, S. 3-4), information systems research is focusing on the functional design of information systems and on methodical aspects in information systems development and operation. The functional design refers to the design of concrete functions to be offered by information systems for concrete purposes and in concrete areas of application, traditionally in enterprises and public administration, but also, increasingly, in private households.

Regarding methodology, we can distinguish between two perspectives: one on the functional goals and one on the formal goals of information systems research. The functional goals perspective refers to methods for information systems design and development. Unlike in applied informatics, the specific characteristics of the two basic application domains (enterprises and public administration) are given special consideration.

The formal goals perspective focuses on the efficient and effective use of information systems as a re­source, taking into consideration both the provision and operation of such systems and the use of such systems. Such issues usually are dealt with under the headline „information management”. The term „IT management“ would be more precise, though, since many other information management related questions are dealt with in other fields, such as accounting or organiza­tional theory.

Two other topics can be found at the interface of information systems research with economics and informatics. The context information systems are used in has already been identified as an important aspect when looking at information systems (see Section 2 of this article). If this aspect is emphasized, also indirect effects of information systems (for example, changes in the division of labor in enterprises, shifts with regard to revenue sources) can be subsumed under information systems research.  Likewise, information systems researchers at the interface with informatics are dealing with questions on technical feasibility, even though this interface has been blurred in recent years, due to a more practice oriented approach followed in core informatics. Figure 2 gives an overview of the topics of information systems research.

 

Fig. 2: Topics of information systems research

4. Topics of design oriented information systems research

By definition, also design oriented information systems research addresses the topics around the object domain outlined above. However, design oriented information systems research focuses on the creation of constructs, the creation of models based on such constructs, and the creation of methods based on such models, including instantiation of such methods in concrete cases. To say it in other words: The research subject of design oriented information systems research is the creation of artefacts (see March/Smith 1995). Information systems already existing as well as methods already available to create such systems is interesting to design oriented information systems research only as an input for creating new solutions.

And here we can see the interface between design oriented information systems research and behavioristic information systems research. The behavioristic approach perceives the world as “something that is”. In this world, artefacts designed by design oriented information systems research can occur in the form of concrete implemen­tations, which then can serve, for example, as an impulse for evaluation of theories or other knowledge.

Literature:

Wilde, T./ Hess, T.: Forschungsmethoden der Wirtschaftsinformatik – eine empirische Untersuchung, in: Wirtschaftsinformatik, 49 (2007) 4, S. 280-287

Mertens, P./ Bodendorf, F./ König, W./ Picot, A./ Schumann, M./ Hess, T.: Grundzüge der Wirtschaftsinformatik, 9. Auflage, Heidelberg u.a. 2006

March, S.T./ Smith, G.F.: Design and natural sciences on information technology, in. Decision Support Systems, 15 (1995), 251-266

 

Wilhelm Hummeltenberg, University of Hamburg

Business Informatics vs. Information Systems  

Replica on „Memorandum on design orientend information systems research“

The memorandum (Österle, Hubert; et al. in zfbf 62(2010) 662-679) addresses the dispute on the future development and evaluation of interdisciplinary research and education at the interface between „Business, Social and Economic Sciences“ and „Computer Sciences“. Debates on this issue currently take place both in North America and in Europe:

  • Koch, Ned; Gray, Paul; Hoving, Ray; Klein, Heinz; Myers, Michael D.; and Rockart, Jack: IS Research Relevance Revisited: Subtel Accomplishment, Unfulfilled Promis, or Serial Hypocrisy? Communications of the Association for Information Systems, Vol. 8 (2002) 330-346 [http://aisel.aisnet.org/cais/vol8/iss1/23/]
  • Plinke, Wulf: Theoria cum praxi  -  Bemerkungen zur Entwicklung der Managementausbildung seit 100 Jahren. Zfbf 60(2008), 846-863.

Regarding the underlying concept of science, business informatics follows the principles of industrial engineering. With the upcoming of theoretical and applied computer sciences, business informatics liberated itself from “Engineering Management” and “Industrial Engineering”, in French “Génie industriel”. Today, business informatics as a „solution-oriented approach and integrative discipline“ is acknowledged on an international level, also in the Anglo-Saxon world [http://en.wikipedia.org/wiki/Business_Informatics; 29.01.2011]:

Business Informatics (BI) is a discipline combining information technology (IT), informatics and management concepts. The BI discipline was created in Germany, from the concept of "Wirtschaftsinformatik". It is an established, successful academic discipline including bachelor, master and diploma programs in Austria, France, Germany, Ireland, The Netherlands, Switzerland and is establishing in an increasing number of other European countries as well as Australia. BI integrates core elements from the disciplines business administration, information systems and computer science into one field.

Business Informatics as an integrative discipline

Business Informatics (BI) shows many similarities to Information Systems (IS), which is a well established discipline originating from the Northern American hemisphere. However, there are a few major differences that make Business Informatics a unique own discipline:

  1. Business Informatics includes Information Technology, like the relevant portions of Applied Computer Science, to a much larger extent than Information Systems does.
  2. Business Informatics includes significant construction and implementation oriented elements. I.e. one major focus lies in the development of solutions for business problems rather than the ex post investigation of their impact.

Information Systems strongly focuses on empirically explaining phenomena of the real world. Often, IS has been said to have an "explanation-oriented" focus in contrast to the "solution-oriented" focus that dominates BI. IS researchers make an effort to explain phenomena of acceptance and influence of IT in organizations and the society applying an empirical approach. In order to do that usually qualitative and quantitative empirical studies are conducted and evaluated. In contrast to that, BI researchers mainly focus on the creation of IT solutions for challenges they have observed or assumed.

Tight integration between research and teaching following the Humboldtian ideal is another tradition in Business Informatics. Recent insights gained in actual research projects become part of the curricula quite fast because most researchers are also lecturers at the same time. The pace of scientific and technological progress in BI is quite rapid, therefore subjects taught are under permanent reconsideration and revision. In its evolution, the BI discipline is fairly young. Therefore, significant hurdles have to be overcome in order to further establish its vision.

Further Translations of “Wirtschaftsinformatik” / “Business Informatics” in Europe

Dutch

Bedrijfskundige informatica (Nederland) / Beleidsinformatica (Vlaanderen)

Polish

Informatyka ekonomiczna

Spanish

Informática de negocios

Turkish

Işletme enformatiği

German and International Business Informatics Challenges

Responding to the Bologna Process in Germany, a couple of obstacles need to be overcome which business informatics is facing with the two-step bachelor/master system:

  • What can be the role of business informatics as an integrative discipline in bachelor programs in economics in order to obtain a first-level vocational qualification after three years of studies?
  • How can the demand for a first-level vocational qualification after three year of studies be fulfilled for an integrative discipline focusing on solutions to practical problems?
  • What can be the role of business informatics as an integrative discipline in master programs in economics?

The answers to these questions will build the foundation of the future of business informatics in Germany. In addition, Business Informatics has to be established not only in Europe and South East Asia, but in North America, too. First steps are ongoing like the “International Business Informatics Challenge and Conference 2011 (IBIC ’11), May 26 – 28, 2011, at the Ball State University, Muncie, Indiana, USA. [http://www.businessinformatics-challenge.org/]

 

Dimitris Karagiannis, University of Vienna

The role of IT in design oriented information systems research

1 Introduction

The world of IT is complex, as the multitude of standards, methods, models, architectures, frameworks, workbenches, and programming languages document. Its role in design oriented information systems research can be described in two – chronological – phases, where IT plays a different role in each phase.  

In the first phase, which can be denoted with the terms „search of concept“ and/or „proof of concept“ and which primarily deals with basic research issues, IT takes over the „enabler“ role. What specifically the role of IT as an enabler can be is determined by the specific type of basic research conducted in discrete application areas. The following quote from German weekly newspaper „DIE ZEIT“ illustrates this view by an example. „Already twenty years ago, when he was a post-doc at the University of California in Los Angeles, Erich Wanker was doing research on proteins. In those days, the young genetic engineering researcher was pretty far away from doing research of clinical relevance. ‘What I was doing at that time was arduous basic research without any practical applicability’, says Wanker. ‘I sometimes had the feeling that I was barking up the wrong tree.’ (Albrecht 2009).

In the second phase, which can be denoted as „proof of product“ and which primarily deals with the transfer of research results into innovative design solutions and with the implementation of information systems, IT takes over the „driver“ role. Regarding the role of IT as a success factor in business (the impact of new technologies on company management, for example), useful information can be found in the Harvard Business Manager issue on information technology (10/2008).

2 IT as a driver

The role of IT as an enabler is not investigated here, as multiple scientific publications are available on this topic already. The following sections deal with the role of IT as a driver in the „proof of product“ phase. This role is examined under consideration of the „research object“ aspect as it has been specified by the „Memorandum on design oriented information systems research”, and by means of an analogy with the process of developing new medical substances and products in clinical research (based on knowledge gained in various projects). The analogy is used to illustrate to the stakeholder groups specified in the Memorandum the role of IT in design oriented information systems research.

In the section headed „Research Object“, the memorandum says: „The body of knowledge of design-oriented IS research is constituted by the scientific literature produced by the discipline and – to a much larger extent – by the experiences and knowledge accumulated in business concerning IS, software products, organizational concepts, methods, and tools.” Based on findings from basic research in the fields of biology, chemistry, and physiology, the clinical research deals with the examination and testing of substances and products, and with the use of such substances and products for treatment of diseases and related symptoms. This is done by systematic search, which is characterized by a multi-stage, iterative process that finally leads to one or several solutions being identified. For these candidates, predictions about their effect in the human organism as well as instructions on the way they are to be produced are already available. Furthermore, already in this pre-clinical phase unintended reciprocal effects can be excluded. Statements about the effect of certain substances in vivo, however, cannot be yet made. For further examination of substances, special test procedures are applied in order to determine their toxicity and the effects they have on the animal organism. The next step is to examine in vivo effects in a number of clinical tests conducted with human beings. In Phase I, substances are tested with healthy volunteers (ranging from twenty to one hundred in number). Phases II and III involve patients (one hundred to five thousand in number) suffering from the disease the respective substance is to be tested for. All these studies are made under medical supervision and are subject to strict control by bodies such as the European Medicines Agency (EMA) (2010) or the Food and Drug Administration (FDA) (2010), ensuring both compliance with formal and regulatory provisions and objective verification of the studies’ design (Fill u. Reischl 2009).

If we transfer these considerations to the discipline of design oriented information systems research, the role IT plays in this discipline can be compared to the role new medical substances play in clinical research. It is, however, important to note that IT in design oriented information systems research must not be considered to be only a “substance“ or “product“ for treatment of certain „disease symptoms“, but that IT may extend a company’s abilities, or may even be the enabler of specific abilities of a company. Searching for new approaches in design oriented information systems research also follows an iterative process that may lead to several solutions. As is the case with the pre-clinical phase in clinical research, in design oriented information systems research likewise it is not possible at this stage of the process to make reliable statements on the effect of IT on the “organism” (here: business). However, certain parameters can already be configured in terms of being appropriate to be used in a certain company and preventing “unintended” reciprocal effects from happening. For example, it is possible here to address requirements with regard to data security and data privacy protection, which need to be considered in any case in the specific environment. By means of empirical surveys and „security checks“of limited scope, it would be possible already to make some basic statements on future effects in real-world environments. However, such measures cannot be a substitute for the actual implementation of a solution in business environments. The side effects and reciprocal effects occurring in such real-world business settings finally indicate how well the new „product“ is suitable for the intended purpose and if the objectives specified in the beginning could be accomplished.

If IT products are used the way described here, the question comes up as to how these products have to be designed. Like substances in clinical research can be offered in different „forms of administration“, IT products too are characterized by a number of parameters (for example, memory design, efficiency, complexity, or the types of components, operating systems, application, and interfaces used, or aspects like security or consumption of resources), which all have an influence on the effect of the product. And like in clinical research, the concrete „form of administration“ of an IT product needs to be specified at an early point, as it has direct consequences on the use of the product in real-world settings. For example, the choice in favour of a certain programming language is often determined by certain hardware and software requirements, which might be available and appropriate during the phases of research, but which in a real-world business environment might not be applicable due to technical or organizational restrictions. The same is true with regard to the quantity and quality of resources, such as databases or service level agreements, which in research environments often are not available in the same capacity or complexity as in a business environment, and which thereby present an additional obstacle when it comes to adapting products to real-world environments.

During the entire process of clinical research, and particularly during the phase of clinical testing, ensuring highest possible quality and complying with national and international regulations is of high significance. For example, applications for admission to clinical research are intensively reviewed by public authorities and ethical commissions, which closely interact with the producers in order to ensure that test procedures applied are appropriate and lead to reliable results.

A similar procedure could be chosen for design oriented information systems research, as in this discipline quality assurance and compliance with regulatory provisions needs to be considered as well. If an IT product is to be tested in a real-world business environment, certain test procedures are required to ensure functional, legal and safety requirements. In analogy to clinical research, mechanisms could be put in place here in order to validate the „governance“ of an IT product, thereby ensuring it is safely used in a business environment.

3 Summary

In all central aspects mentioned by the Memorandum, IT plays a certain role:

  • stakeholder groups, for which IT has the potential to make a central contribution with regard to user-machine interaction,
  • research object, for which the role of IT has been described in detail in the previous section,
  • result types: in which software is considered as an „open source code“, with the Open Model Initiative making a substantial contribution (Koch et al. 2006; Karagiannis et al. 2007),
  • research principles: in which the IT can be supported by the abstraction and evaluation of IT and e.g. by utilizing certain modelling approaches.

4 Literature

Albrecht, H.: Wie das Leben so faltet, DIE ZEIT vom 10.09.2009, S. 33ff.

European Medicines Agency: European Medicines Agency, www.ema.europa.eu, 27.01.2010.

Fill, H.-G.; Reischl, I.: An Approach for Managing Clinical Trial Applications Using Semantic Information Models, in: Rinderle-Ma, S. et al. (Hrsg.): 3rd International Workshop on Process-oriented information systems in healthcare (ProHealth ’09) in conjuncation with 7th Int'l Conf. on Business Process Management (BPM 2009), Springer, Berlin, Heidelberg 2009, S. 581-592.

Food and Drug Administration: U S Food and Drug Administration Home Page, www.fda.gov, 27.01.2010.

Harvard Business Manager: Schwerpunkt Informationstechnik, Hamburg 10/2008.

Karagiannis, D.; Grossmann, W.; Höfferer, P.: Open Model Initiative - A Feasibility Study, Projektstudie im Auftrag des BMVIT, Projektnummer GZ.BMVIT-604.000/0012-III/I5/2007, www.openmodel.at, 2007.

Koch, S.; Strecker, S.; Frank, U.: Conceptual Modelling as a New Entry in the Bazaar: The Open Model Approach, in: Damiani, E. et al. (Hrsg.): Proceedings of the Second International Conference on Open Source Systems, Springer, Boston 2006, S. 9-20.

 

Stefan Klein, University of Münster

[from a letter to Hubert Oesterle]

In my view, the memorandum on design oriented information systems research takes too narrow a view and legitimates (not deliberately, I assume) some practices that I would criticize.

1.    Information systems research is presented in a stereotypical manner and is incomplete.

Information systems research is much more heterogeneous and cannot be subsumed under the behavioristic approach. Education of Ph.D.s in the really good schools almost exclusively aims at paving the way to an academic career.

The criticism of research that lacks relevance neglects the quality issue. Some articles or papers which score low on relevance are simply poor in quality. However, such poor quality of writing and scientific elaboration does not automatically discredit the research paradigm or research method selected.

For some years now the Research Center Directors Meeting is organized at ICIS. . during this meeting colleagues  - primarily from the U.S. - report on their experiences regarding intensive collaboration with industry partners. As far as I know, it is just that information systems research in the U.S. makes a clearer distinction between consulting (many colleagues there have worked in industry and have a consulting mandate) and research.

2.    The way design oriented information systems research is described is stylized and idealizing.

In the German-speaking information systems research community there is a large modeling group. Modeling, however, is not per se about innovation, but primarily about description. Such description often follows a process oriented paradigm and is an important intermediate step in systems development. Business, on the other hand, has been using ethnographic methods, which belong to the behavioristic paradigm rather, in order to better understand how users use information systems and to build upon these processes when designing systems. … Reality is more diverse and more heterogeneous than the memorandum is suggesting.

The most prominent objective of European IS research basically has been to produce practically beneficial, business relevant results. Adoption of these results by business (i.e., economic payoff) has often been considered more important in terms of providing evidence of the correctness of results than transparent, well-documented scientific development of results following generally accepted criteria (i.e., scientific rigor). Inevitably, in some cases, this has led to the publication of results that fall short of rigorous scientific standards.

To my (limited) knowledge, the boundaries between academia and consulting in information systems research are often blurred. Many Ph.D.s work in consulting projects and publish articles and write their doctoral thesis based on that work. Project results are usually developed in collaboration with business partners and are considered beneficial by these partners.

What is missing here is not just „well-documented scientific development of results following generally accepted criteria (i.e., scientific rigor)“, but an interest in scientific work as well as knowledge about literature and methods that go beyond consulting. It is not understood that academia is a „game” of its own, which has its own rules that do not necessarily apply to the „game“ of business.

Taking into account that 80 - 90 per cent of Ph.D.s do not strive for an academic career, we need to think about the academic standards required for these people’s education. Some universities in the UK therefore distinguish between DBA (qualification for a business career) and Ph.D. (qualification for an academic career). Maybe such a differentiation would be helpful in information systems research too.

3.    The issue of insufficient evaluation of results, which is a central aspect of the criticism of the design oriented approach, is not given serious consideration.

Scientific rigor demands validation of artifacts produced against the objectives specified, applying the methods stated in the research plan. The review process prior to scientific publications is part of the evaluation.

When it comes to evaluating results, the review process, in my opinion, cannot fulfil the role it is supposed to play, unless systematic and method based evaluation of results has been an integral part of the research process itself. The memorandum accepts a way of immanent evaluation, with researchers evaluating the results against their own objectives and the methods they selected. In doing so, however, there are no safeguards that researchers comply with scientific standards. If an article says that the results were presented to a business partner and the business partner was satisfied with the results, one might consider this as a positive evaluation in terms of the objective of generating practical benefit. In a review process, however, such a statement cannot be validated. I am aware only of a few examples where propositions for the evaluation of results that were made in the course of the design science debate have been considered seriously and adapted accordingly.

„A clear majority of the representatives of the discipline is meant to signal to junior faculty that our discipline explicitly welcomes design oriented research, for which it has specified rules.“ From my perspective, the rules (principles) are not specified clearly enough yet. „General applicability“, for example, cannot be achieved, especially not by a design oriented approach, as recommendations given are always context specific.

4.    Research objectives

Design-oriented IS research aims to develop and provide instructions for action (i.e., normative, practically applicable means-ends conclusions) that allow the design and operation of IS and innovative concepts within IS (instances).

Again, this view appears to be much too narrow. There is not specification given for “instruction for action” such as methodologically grounded. Design oriented information systems research, too, should not limit its research objectives to instructions for action, as the process of designing actually is one way to gain scientific knowledge, following the principles of action research or experimental approaches, for example.

5.    Style of the memorandum

In my view, the memorandum is largely uncritical of practices blurring the boundaries between business and science. Due to the growing pressure to acquire third-party funding, scientists often find themselves to be managers, with scientific research following expectations and requirements of those who finance the work. Ph.D.s have to produce the results defined in the project plans and are rewarded with degrees for that. In this respect, „design science“, at least according to what I have observed, occasionally is used to disguise to justify consulting-oriented research as science. In such cases, results are likely to be legitimated by their practical applicability rather than by applying scientific methods or by relating to the scientific discourse. For neighbouring disciplines, such as business adminstration, economics, or social sciences, it is often not possible to connect to the results produced.

Outside Germany, research is conducted differently, i.e. under different institutional conditions and within different structures. In my opinion – and I might be called idealistic for saying that – research mainly should be about the quality of results, and thus about the advancement of our understanding of the world. In every scientific discipline and in every scientific community regardless of their orientation, there are excellent researchers, there are average researchers, and there are poor researchers. Together we should aim for quality, diversity, and professionalism.

To me the memorandum appears to identify the discussion about paradigms as the main problem. Saying that there are two poles (one is design science, the other is behaviorism), however, is neither hitting the nail on the head nor is it helpful. In my view, it is all about different practices of scientific work as well as about different institutional frameworks. Therefore, I promote the idea of scientific discourse and dialog, and actually this is what I have promoted for many years.

In a situation in which publication of articles in journals is considered to be the measure of all things, it seems unavoidable, regrettably so, that the debate on the publication issue has become politicized. Nevertheless, my impression is that the leading international journals and conferences do invest a lot of time and care for their reviews. This, too, is a sign of academic tradition and  culture.

Therefore, I consider other questions to be more important, like:

  • How can I and my Ph.D.s become better researchers?
  • How can we improve the conditions at our universities in order to regain the freedom and structures conducive to academic work? – This point includes the criticism of rankings in journals etc. 
  • How can we better understand and more effectively influence the economic and social impact caused by information systems? 

Peter Mertens, University of Erlangen-Nuremberg

On the current culture of publishing in Wirtschaftsinformatik

Recent developments have evoked a situation in which – when it comes to evaluating the work of scientists of Wirtschaftsinformatik – papers published in international journals are given much more credit than full-blown formats of publication, such as dissertations, conference proceedings or technical books. This culture has brought authors to try to distribute the results of their research projects in small, special fractions and in as many journals as possible, with e. g. five papers published in journals being considered more valuable than five chapters in a book consisting of the same content.

But we should not forget our customers, i.e. the readers. Let us assume a Ph.D. student is examining the effectiveness of using holograms to fight product counterfeiting and piracy in four different industries, with the aim to provide recommendations for improvement. First of all, this student will explain the physical and technical benefits of using holograms for brand protection, since he/she cannot assume that his/her readers are familiar with this new technology. Next, the Ph.D. student will analyze the problem in four industries selected, taking half a year for each. Finally, he/she will summarize the results across all four industries investigated, leading to certain conclusions and – based on these – improved concepts.

The student manages to have a first paper – dealing with the technical foundations of the research subject for the first industry investigated – be published by Journal A. If the Ph.D. student now wants to publish the results for the second industry investigated, the reviewers of Journal A will tell him: "Unfortunately, the second paper you have submitted does not differ sufficiently enough from the first one". The Ph.D. student then turns to Journal B. If he/she is lucky, the editors of Journal B are willing to present their readers a paper about product counterfeiting and piracy, and they accept the manuscript on the second industry investigated, with the technical part on holograms needing to be placed before the actual main part of the study here as well. This way the story goes on until the results for all of the four industries investigated have been published in four different journals. And probably a fifth journal needs to be found to present a summary of that all.

In order to get the big picture of the research project conducted, the deplorable reader (who may be another Ph.D. student or a practitioner) is now supposed to provide, in whatever way, each of these five journals. And, as if this was not inconvenient enough already, the reader has to cope with the problem that he must pore over the five papers in a different way, due to varying conditions for publication, reviewers’ opinions, formats, and so on.

Günter Müller, University of Freiburg

 

Unity in diversity

On both sides of the Atlantic, information systems research currently is facing stormy weather. Since we in Europe always like to take a look at what is happening in North America, and also since we tend to consider – quite rapidly – what is happening there to be the measure of all things, one motivation behind the memorandum is the concern that our successful, fifty-year-old model could be sacrificed for something unevaluated. On the one hand, there is the question why evaluation of models is said to be superior to interacting with reality. On the other hand, this has made many journals send signals to young academics and indicate what has to be considered as science and what has to be rejected, leading to results that contradict the opinion stated in the memorandum. Thus, the memorandum is not a reproduction of the memoirs of its authors but addresses issues that really are central and relevant for the future of the discipline:

  • What is the role of our discipline in society? Where is our discipline located in society?
  • Is there a method best suited for our discipline?
  • How can young academics be given the right signals? How can the flourishing of our discipline be ensured for the next fifty years?
  • Are there „parallel societies“ within the information systems research community that want to split the discipline? 
  • What needs to be done to maintain the discipline’s unity despite the diversity coming from  different methodological approaches?

Principally, a scientific discipline is either characterized by the specific subject it deals with or by specific methods it applies. The German term Wirtschaftsinformatik defines as the subject of its research the role and the benefits of information and communication technology in businesses. Regarding methodological approaches, Wirtschaftsinformatik in the past predominantly dealt with the design of artefacts. Researchers were designing reality. The creed and the claim is that researchers had applied a “design oriented” approach already before the time when „design science“ came up. University graduates are getting  jobs, and the number of students enrolling for Wirtschaftsinformatik studies is on the rise, as is the number of chairs. The U.S. branch of information systems research cannot keep up with these figures. These figures, however, are just one side of the coin. The other side says: The scientific reputation on the other side of the Atlantic appears to be higher. The relevance in the local Wirtschaftsinformatik comes with compromises on cost of rigor in research.  This opinion is caused by different understandig of science and puts the rising generation of scientists under strong pressure.

To be clear on that – such conclusions are exaggerations. They have occurred, though, in many research committees. They represent the change that is currently under way. The notion of „those who are not with me are against me“, which can be detected in the memorandum, contradicts the formula for success that has been prevalent in Wirtschaftsinformatik throughout the past fifty years, when the name of the game was adaptation (or alignment, or conformity), and when the viewpoint was to “go with the times, otherwise you will be gone“. The Wirtschaftsinformatik journal is a good example of that development. The journal is now fifty years old, it is one of the oldest journals on the subject worldwide. What Wirtschaftsinformatik primarily did in the first years, the 1960s, was classifying – with great amazement – IT novelties, reasoning on electronic brains, outlining the “fear of the formed society” combined with a Management Information Systems vision. Already in the second phase, in the 1970s, the journal dealt with more abstract concepts, such as industrial data processing or electronic data processing (initiated by Ackoff’s article on „Management Misinformation Systems“). In the third phase, the journal came of age. Aspects like systematic design, economic operation, impact of information systems on business and society, or specification and research of automated services (e.g. in the finance industry) added to the initial disciplines and made “the impossible possible”.

Automation of programmable processes was followed by formalization of anticipated processes, across company boundaries, if possible. It is this fourth phase we are currently in. Behavioristic information systems research, which is favored in the U.S. and by the most important journals, presupposes that all technical and other relevant prerequisites are now available and that now is the time to learn to understand standardized technology with regard to methods applied and with regard to its relationship with individuals and organizations. Design oriented information systems research, on the other hand, says that neither the instruments nor the integration with business processes nor the interaction with actors has been understood so far, and that there is still leeway in research to do so. We do not know today how this fourth phase will end and if the two methodological approaches will converge one day. But we all should agree that each side deals with a different aspect of the subject, and that therefore both sides have the right to exist.

The idea of „unity in diversity“ goes back to Nikolaus von Kues, also known as Cusanus, who – as an ambassador of the pope – coined this phrase in the struggle for unity of the Roman and the Greek church and who complained about the narrow-mindedness on both sides. The current debate, too, appears to be a little narrow-minded. It would be good to conduct the debate in such a way that each viewpoint can be identified and clearly understood, and that a synthesis can be made in order to accomplish unity in diversity. This task is worth being done, since it would at least allow to send helpful signals to young academics, who could be gained for our discipline amidst the competition with other disciplines.  

This diversity of methods together with the different methodological backgrounds of individuals forms the basis for information systems research to remain adaptable and for maintaining the success of the discipline. Managing this diversity is the main challenge in the future.

 

Elmar Sinz, University of Bamberg

 

 

 

 

Constructional research in information systems research:

What are the research goals of design oriented information systems research?

This paper is a written version of an oral presentation held by the author on September 17th, 2009, at the University of St. Gallen, during a workshop on constructional research in information systems research. Based on the characterization of information systems research as a scientific discipline and design oriented information systems research as a specific branch of this discipline, the author specifies research goals of design oriented information systems research. In the focus of design oriented information systems research is the solution of a constructional problem, i.e. the problem of constructing business information systems (IS). Under specific consideration of the fundamental structure of IS, the author defines categories for research goals of design oriented information systems.

Information systems research as a scientific discipline

Scientific disciplines are defined mainly by their research subject, their research goals, and the methods and techniques they use (see Sinz 2009, S. 225f). The research subject of information systems research is the same as for the scientific discipline of business studies: enterprises, including their sub-systems and super-systems, or, more generally, operational systems in business and public administration. The difference between business studies and information systems research becomes obvious if we look at the research goals each discipline pursues. Whereas the research goals of business studies primarily are oriented towards economic behavior, the research goals of information systems research refer to the issue of information processing in operational systems. Although such information processing, of course, also is supposed to follow the principles of economic behavior, information systems research is characterized by a number of specific aspects:

  • The research goals of information systems research focus on analysis, design, and control of IS.
  • The research subject of information systems research are information processing sub-systems of operational systems, i.e. operational information systems (see Ferstl and Sinz 2008, S. 1f).
  • The methods and techniques used in information systems research to a large proportion come from the field of economics, especially from business studies and computer sciences. In addition, as operational information systems are socio-technical systems, information systems research also takes up methods from scientific disciplines like systems theory, cybernetics, organizational research, labor sciences, psychology, and sociology. And, finally, information systems research has always developed own, specific methods and techniques, often by combining and developing further the methods and techniques from the neighboring disciplines.

The constructional problem of design oriented information systems research

Design oriented information systems research, evidently, primarily aims at the design of IS. Seen from an abstract perspective, the design of IS can be considered a constructional problem, which in turn represents a specific research problem (Ferstl 1979, 43ff):

  • A research problem comprises a research object, which needs to be described in terms of known system characteristics, and a goal of investigation, which relates to unknown system characteristics of the research object.
  • A constructional problem is a specific research problem, with the research object being a system that does not exist yet and the behavior of this system being postulated. The goal of investigation refers to a system structure capable of realizing the desired behavior.
  • The scope of possible solutions to the constructional problem can be limited by predefined requirements to use certain system components or sub-structures.

The constructional problem of design oriented information systems research does not only cover the phases of  planning, development, and implementation of IS, but also – as is the case in other engineering disciplines – the phases of using and operating IS.

In the center of design oriented information systems research, of course, are not rather mechanical design activities, but research activities aiming at the development of new methods and techniques for solving a constructional problem as well as at the testing of new methods and techniques by means of, for example, system prototypes. In this respect, goals of investigation are research goals in design oriented information systems research.

To sum it up: The research goals of design oriented information systems research refer to the solution of a constructional problem with regard to IS.

Structural model of operational information systems

The Wissenschaftliche Kommission Wirtschaftsinformatik (WKWI) (scientific commission for information systems research) of the German Verband der Hochschullehrer für Betriebswirtschaft (academic association for business research) stated in October 1993: „The research subject of  information systems research are information and communication systems in business and public administration (in short: information systems (IS)). IS are socio-technical systems comprising human components and machine components (sub-systems) as task performers […]. In the focus of the investigation is the support in the fulfilment of operational tasks (WKWI 1994, 80).“ This statement on the research subject of  information systems research is still valid. However, the research subject of information systems research has been extended – especially due to the Internet being used by more and more people – to include also information systems used for private purposes.

Fig. 1: Structural model of operational information systems

Looking at Figure 1, the basic structure of IS becomes obvious. It consists of two levels: a task level, relating to business process tasks (tasks with regard to production of products/services or control tasks) and business management tasks (control tasks), and a task performer level, comprising human task performers and machine (computers, IT systems) task performers. The components are all related to each other. Non-automated tasks are done by human task performers, automated tasks are done by machine task performers, and partially automated tasks are done collaboratively by humans and machines. Communication between humans and machines requires specific relations that need to be designed in such a way that synergy effects can best be exploited. The structural model of operational information systems can be found as a basis for explanations in a number of textbooks on information systems research (see Mensch-Aufgabe-Technik (Heinrich 1993, 13f), Aufgaben- und Aufgabenträgerebene (Ferstl and Sinz 2008, 1ff)).

Research goals of design oriented information systems research

Task, human being, and computer are predefined component types in the constructional problem of design oriented information systems research. Research areas of design oriented information systems research can be identified, and associated research goals can be classified, with regard to these component types and with regard to the relations between them. A third group of research areas comprises so-called holistic research areas, which include also questions relating to research methodology.

1.       Component type oriented research area

Component type

Examples

Task

 

  • Management (= design and control) of business processes and value creating networks
  • Management of the semantic heterogeneity in the terminological systems in and between domains

 

Human being

 

  • Qualification of employees
  • E-learning
  • Knowledge management in organizations

 

Computer

 

  • Application system architectures
  • Application system development
  • Use of base technology and middleware
  • Management of technological heterogeneity

 

2.       Relation type oriented research areas

Relation type

Examples

Task - Computer

 

  • Design of task automation
  • Integration of tasks and IT systems
  • Business IT alignment

 

Task - Human

 

  • Design of operational systems organization
  • Organizational research

 

Human - Computer

 

  • Design of human-machine interface (ergonomics, cognition, multi-modal communication)

 

3.       Holistic research areas

Relation type

Examples

Task - Human - Computer

 

  • Design of enterprise architectures and/or information systems architectures
  • Management of information systems complexity
  • Economic aspect of information systems design and operation (standardization, reuse etc.)
  • Quality of information systems (security, reliability, robustness etc.)

 

Research methodology

 

  • Methodology of design oriented information systems research
  • Ensuring the quality of the results of design oriented information systems research

 

The research areas described comply with empirical evidence. An analysis made by Heinzl, König and Hack (2001), for example, considers questions concerning the architecture of information systems and the interplay of information technology and organization to be the No. 2 and No. 3 issue regarding the most important research goals to be addressed in the next three years. Management of complexity in information systems, user/human-machine interfaces, architectures of information systems, new ways of division of labor, and new forms of collaboration are ranked No. 1, 3, 5 and 6 issues among the most important research goals to be addressed in the next ten years.

Summary and discussion

What has been outlined in this paper so far can be summed up by four core statements:

  1. Scientific disciplines can be characterized by their research subject, their research goals, and the methods and techniques they use.

  2. Research goals of design oriented information systems research refer to the solution of the constructional problem of IS. The starting point is the postulation of behavioral characteristics of an IS. What needs to be found is an IS structure capable of realizing the postulated behavior and taking into account predefined requirements regarding the use of certain system components and sub-structures.

  3. IS are socio-technical systems comprising three component types: operational tasks, human beings (as human task performers), and IT systems (as machine task performers).

  4. By means of the component types and the relations between them, specific research areas of design oriented information systems research can be identified and classified.

One question resulting from what has been elaborated in the previous sections is: Do these research areas of design oriented information systems research already imply the research questions and, as a consequence, the research goals of design oriented information systems research? From the author’s perspective, this question basically can be answered with ‘yes’. Despite all differences regarding momentary relevance or urgency of specific research areas, it is evident that both the requirements to be met by IS (i.e. the constructional problem with regard to postulated behavior) and the relevant technologies and infrastructures (i.e. the constructional problem with regard to predefined use of system components or sub-structures) continue to be subject to drastic change and evolution. One example that can be mentioned in this respect refers to the issue of task automation of IS. Here, new research questions are permanently generated due to the upcoming both of new technologies (e.g. sensor technology, RFID) and  new requirements (e.g. different design of human-machine interfaces because of new findings in cognitive research).

Literatur

Ferstl OK (1979) Konstruktion und Analyse von Simulationsmodellen. Hain, Königstein.

Ferstl OK, Sinz EJ (2008) Grundlagen der Wirtschaftsinformatik, 6. Aufl. Oldenbourg, München.

Heinrich LJ (1993) Wirtschaftsinformatik: Einführung und Grundlegung. Oldenbourg, München.

Heinzl A, König W, Hack J (2001) Erkenntnisziele der Wirtschaftsinformatik in den nächsten drei und zehn Jahren. WIRTSCHAFTSINFORMATIK 43 (3): 23-233

Sinz EJ (2009) Grundlagenforschung in der Wirtschaftsinformatik – Versuch einer Positionsbestimmung. WIRTSCHAFTSINFORMATIK 51 (2): 225-227.

WKWI (1994) Profil der Wirtschaftsinformatik. WIRTSCHAFTSINFORMATIK 36 (1): 80-81.

 

 

Volker Wulf, University of Siegen

Design case studies: An approach to overcome the dichotomy of behavioristic and design oriented research

Markus Rohde, Gunnar Stevens and Volker Wulf (University of Siegen)

We largely agree with the memorandum’s viewpoints. In particular, we hold the view that a clear signal should be set against the tendency towards a “behavioralization of information systems research”, which we consider to be a point that is rightly criticized by the authors of the memorandum. However, we think that there are some problems with the rationale of the memorandum:

  • The memorandum’s view on information systems research is too broadbrush. The Anglo-Saxon community is much more heterogeneous with regard to research paradigms used than stated in the memorandum. However, the memorandum is right in stating that one research paradigm (which is characterized by little practical orientation) tends to dominate the discipline – with this domination getting effective in rankings in the areas of business studies and economics. So we support the memorandum’s criticism of this empiristic paradigm aiming at the detection of quantifiable principles.
  • The counter-position taken by the memorandum, however, appears to be too unspecific and, to some extent, determined too much by positivistic ideas. In particular, what is not dealt with is the momentum of creativity when it comes to designing innovative technical artefacts and when applying them (abduction as proposed by Peirce seems to be an interesting way of thinking here, instead of deduction or induction). Also, the significance of evaluation, particularly in practice, is not sufficiently elaborated by the memorandum.

Regarding a more precise specification of our view, we would like to refer to two papers (Rohde et al. (2009) and Wulf (2009)), which can be accessed below.

 

Click to open as PDF

Wulf, V., Theorien sozialer Praktiken zur Fundierung der Wirtschaftsinformatik, in: Becker, J., Krcmar, H., Niehavs, B. (Hrsg.), Wissenschaftstheorie und gestaltungsorientierte Wirtschaftsinformatik, Physica-Verlag, Heidelberg 2009, S. 211-224, http://www.uni-siegen.de/fb5/wirtschaftsinformatik/paper/2009/2009_wulf_theorien_sozialer_praktiken.pdf

Click to open as PDF

Rohde, M., Stevens, G., Brödner, P., Wulf, V., Towards a paradigmatic shift in IS: designing for social practice, Beitrag in: (Hrsg.), Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, ACM, Philadelphia, Pennsylvania 2009, S. 1-11, http://desrist2009.ist.psu.edu/Papers/desrist2009_submission_22.pdf


General Legal Information - Privacy - (Webdesign by IWI Tech Team)