UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An empirical study of judgments on internal control Cook, Shane Padriac 1980

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1980_A4_6 C66.pdf [ 4.36MB ]
Metadata
JSON: 831-1.0094795.json
JSON-LD: 831-1.0094795-ld.json
RDF/XML (Pretty): 831-1.0094795-rdf.xml
RDF/JSON: 831-1.0094795-rdf.json
Turtle: 831-1.0094795-turtle.txt
N-Triples: 831-1.0094795-rdf-ntriples.txt
Original Record: 831-1.0094795-source.json
Full Text
831-1.0094795-fulltext.txt
Citation
831-1.0094795.ris

Full Text

AN EMPIRICAL STUDY OF JUDGMENTS ON INTERNAL CONTROL by SHANE PADRIAC COOK B.S., University of Oregon, 1970 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN BUSINESS ADMINISTRATION in The Faculty of Graduate Studies (Commerce and Business Administration) We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA March, 1980 © Shane Padriac Cook In presenting this thesis i s p a r t i a l f u l f i l l m e n t of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t freely available for reference and study. I further agree that permission for extensive copying of t h i s thesis for scholarly purposes may be granted to the Head of my Division or his representatives. It i s understood that copying or publication of th i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. Division of Accounting and Management Information Systems Faculty' of Commerce and Business Administration The University of B r i t i s h Columbia 2075 Wesbrook Place Vancouver, B.C. Canada, V6T 1W5 Dated this Abstract A review of the l i t e r a t u r e in the area of audit judgment and cognitive s t y l e research forms the basis for the research study reported in t h i s paper. T h i r t y - s i x independent auditors were required to complete a psycho-l o g i c a l test structured to measure cognitive complexity. They were also asked to judge weakness of internal control in a hypothetical company as a function of f i v e yes/no questions representing p a r t i a l judgments according to a f u l l f r a c t i o n a l f a c t o r i a l design. The a n a l y s i s - o f - v a r i a n c e version of the Brunswik lens model was used to analyze the subject responses. The results of the experiment indicated that the subjects' models of judgment i s highly l i k e l y to stem from the fact that most of the subjects were concrete (cognitive simple) in their judgments. The usefulness of f r a c t i o n a l f a c t o r i a l studies in human judgment research was also appraised, and i t was found to be adequate in the experiment performed. Suggestions for further research applying the cue u t i l i z a t i o n methodology in au d i t and management information systems research were made. In addition, suggestions for a more n a t u r a l i s t i c approach to audit research were proposed. Thesis Supervisor i i ACKLOWLEDGEMENTS I would l i k e to thank my advisers, Dr. Albert S Dexter, Dr. Michael Gibbins, and Dr. Hartmut J. W i l l (Chair man) for the numerous suggestions and constructive c r i t i c isms. I would also l i k e to thank Dr. Izak Benbasat for hi suggestions during the planning stage of thi s thesis and Mr Kenneth P. Johnson, of Coopers and Lybrand, New York, NY for his commentary on public accounting prac t i c e . Thei contributions to thi s thesis are g r a t e f u l l y acknowledged. i i i TABLE OF CONTENTS Page CHAPTER I: INTRODUCTION Objectives Of The Thesis 1 Organization Of The Thesis 1 Research In Audit Judgments 2 Internal Control Studies 3 Audit Program Planning Study 6 Cognitive Psychology 7 Cognitive Styles 9 Cognitive Complexity 14 Summary 18 CHAPTER II : METHODOLOGY OF THE EXPERIMENT Introduction 28 The Experimental Task 28 Experimental Design 30 Strengths And Weaknesses Of The Experiment 32 Brunswik Lens Model 32 Fractional F a c t o r i a l Designs 37 Paragraph Completion Test 38 Administration Of The Experiment 40 Summary 41 CHAPTER I I I : RESULTS OF THE EXPERIMENT Introduction Judgment Description Importance Of Main Effects And Interactions Effects Of Firm A f f i l i a t i o n And Level Of Experience Judgment Insight Subjective Weights Insight Index Judgment Consistency Cognitive Complexity . Summary CHAPTER IV: CONCLUSIONS Summary Of The Problem 6 7 Summary Of The Experimental Results 68 Implications Of The Experiment's Results 70 Implications for Further Research 71 BIBLIOGRAPHY 74 APPENDIX 1 CASE STUDY 79 APPENDIX 2 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS 81 APPENDIX 3 INDIVIDUAL SUBJECT CONSENSUS AND INSIGHT SCORES 90 48 48 49 50 51 51 52 53 54 i v LIST OF TABLES Page Table I Ashton Questions 19 Table II Hamilton And Wright Questions 20 Table III Ashton And Brown Questions 21 Table IV Joyce Variables 22 Table V Questions Used In Current Study 43 Table VI Example Of Internal Control Case Used In Exper-iment 44 Table. VII Relative Importance Of Main EffectsAnd Inter-actions To The Auditors ( f u l l Model) 57 Table VIII Relative Importance Of Main Effects And Two Way Interactions To The Auditors (One Half Frac-t i o n a l Model) 58 Table IX Relative Importance Of Main Effects And Inter-actions To The Auditors By Firm ( f u l l Model) 59 Table X Relative Importance Of Main Effects And Two Way Interactions To The Auditors By Level Of Exper-ience ( f u l l Model) 60 Table XI Values Of Subjective Weights By Firm And Exper-ience Level 61 Table XII Insight Index By Firm A f f i l i a t i o n And Level Of Experience 62 Table XIII Judgment Consensus Among Auditors In Same And Different Firms 63 Table XIV Judgment Consensus Among A u d i t o r s In Same Experience Levels 64 v LIST OF FIGURES Page Figure 1 General Relationship Between Environmental And Behavioral Complexity 23 Figure 2 Relationship Between Environmental And Behav-i o r a l Complexity For Different Levels Of Per sonality Structure 23 Figure 3 Brunswik Lens Model 45 vi 1 CHAPTER I INTRODUCTION Objectives Of The Thesis The objective of thi s thesis i s to study the consis-tency of audit judgments 'among auditors. A secondary objective i s to determine whether differences exist due to t r a i n i n g and experience, in the s u b j e c t s ' a b i l i t y to d i f f e r e n t i a t e and integrate information. A further objec-tive of t h i s thesis i s to be found in an evaluation of the usefulness of f r a c t i o n a l f a c t o r i a l designs in human judgment research. Organization Of The Thesis The remaining sections of t h i s chapter w i l l discuss l i t e r a t u r e relevant to the research. These sections w i l l include reviews of research in audit judgments, and research in cognitive psychology. The second chapter w i l l discuss the methodoloqy of the research experiment performed. Among the topics to be discussed are the experimental task, the s p e c i f i c experi-mental design used, strengths and l i m i t a t i o n s of the experiment, and d e t a i l s concerning the administration of the experiment. Chapter III presents of the re s u l t s of the experiment. A f i n a l chapter summarizes the experimental findings and 2 presents further implications of the findings with sugges-tions for further research. Research In Audit Judgments This section presents a review of auditing research considered as a behavioural view of auditing and management information systems (MIS) and a preview of the type of research to be conducted. A comprehensive review of two authors' works w i l l be given as the techniques used by them form the basic methodology of t h i s experiment. Auditor's judgments have seldom been the subject of systematic research. Yet the importance of auditor judg-ments i s accepted almost without auestion. "A t y p i c a l rationale for t h i s s i t u a t i o n i s the AICPA's statement that 'judgment i s the most important factor in the making of any audit, but in many situations i t i s p r a c t i c a l l y impossible to write out in s p e c i f i c language how the auditor applies judgment. 1" 1 This thesis w i l l review two areas of study on the judgmental processes of auditors. The f i r s t area deals with judgment formation i n the e v a l u a t i o n of i n t e r n a l control and the second area focuses on the audit planning stage. Both of these studies are examples of "cue u t i l i z a -t i o n " or "analysis-of-variance" techniques. Through the systematic manipulation of a predetermined set of information cues, the researcher can produce a quasi-experimental manipulation of the cues within each subject's set of judgment situ a t i o n s . The r e s u l t s for each subject 3 can then be analyzed using several s t a t i s t i c a l techniaues that provide descriptions of which cues were important to the s u b j e c t ' s judgments, how cues i n t e r a c t e d , and how 2 consistent a subject was. S p e c i f i c advantages and disad-vantages of t h i s technique w i l l be discussed in chapter II. Internal Control Studies 3 Ashton was the f i r s t to use cue u t i l i z a t i o n in the study of auditor judgments. This f i r s t experiment conducted by Ashton studied judgments on internal control over payroll systems by 63 practicing auditors in the Minneapolis/St Paul area. The auditors were presented with 32 sets of hypothet-i c a l internal control cases, each consisting of six ques-tions (cues) that had been pre-answered "yes" or "no." The six questions used are presented in Table I. The thirty-two sets were chosen according to a h a l f - f r a c t i o n a l f a c t o r i a l design. The subjects were required to evaluate the adequacy of the internal control using a six-point scale. The data was analyzed lisinq the analysis-of-variance technique. This design allowed the estimation of the signficance of main and two-factor interaction e f f e c t s . A l l other interactions were included in the error term. To estimate the importance of each question and two-factor interactions, omega-souared 4 s t a t i s t i c s were computed. This s t a t i s t i c i s a measure of the strength of association and can be used as an estimate of the r e l a t i v e proportion of variance that i s accounted for 4 by each main e f f e c t and interaction in an analysis-of-variance design. As a second portion to his study, Ashton investigated two types of judgment consistency. One type of judgment consistency i s consensus, or the consistency of the subjects at one point in time. Consensus was determined by c o r r e l a t -ing the internal control ratings given with those of the other subjects. A second type of consistency i s s t a b i l i t y , which i s the consistency over time for the same subject. Ashton determined s t a b i l i t y by having the subjects complete the experimental instrument at a l a t e r dates (6 to 13 weeks later) and correlating the two sets of internal control ratings for each auditor. In addition to this work, Ashton evaluated judgment insight by having each subject a l l o c a t e 100 points among the six questions accordina to the r e l a t i v e importance the subject gave the question. Insight was measured by co r r e l a t i n g the omega-squared s t a t i s t i c s with the subjective weights given. The results of the experiment indicated that the main e f f e c t s accounted for, on average, 80.22% of the variance, and that the f i f t e e n two-way interactions only 6.4%, leaving the error factor to account for 13.38%. Two questions on separation of duties accounted for, on average, 51.4% of the variance. This tends to support the hypothesis that the auditor judgments varied systematically with the answer to the question. There was some minor support for the hypothe-s i s that the judgments were influenced by the interaction of 5 the answers. The average c o r r e l a t i o n s f o r consensus, s t a b i l i t y , and insight were .70, .81 and .89 respectively. Ashton's experiment has been r e p l i c a t e d by three a d d i t i o n a l s t u d i e s : Hamilton and Wright^, Ashton and 6 7 Kramer , and Ashton and Brown . 5 Hamilton and Wright used a f u l l r e p l i c a t i o n of a 2 f a c t o r i a l design using the f i v e questions shown in Table IT. The average proportion of variance explained by the f i v e main ef f e c t s was 82%. The average co r r e l a t i o n for consensus and insight were .66 and .87 respectively. Hamilton and Wright divided t h e i r 17 subjects in two groups - those with more than three years of experience and those with three or less years of experience. The more experienced subjects had a higher average proportion of variance explained by the main e f f e c t s than the less experienced group (.85 versus .80), a higher correlation for insight measurement (.93 versus .84), and higher c o r r e l a t i o n for judgment consensus (.78 versus .62 ). Ashton and Kramer replicated Ashton's o r i g i n a l experi-ment using 30 students enrolled in an undergraduate auditing course at the University of Texas at Austin. The six main ef f e c t s accounted for, on average, 65.6% of the judgment variance and the two-factor interactions accounted for, on average, 8.4% of the variance compared to the o r i g i n a l experiments averages of 80.2% and 6.4%, respectively. The correlations for consensus and insight were .66 and .77 6 respectively, compared to .70 and .89 for Ashton's o r i g i n a l work. Ashton and Brown replicated the previously described work using a one-half r e p l i c a t i o n of a 2 f a c t o r i a l design using the eight questions shown in Table III. This design allowed for the estimation of a l l main e f f e c t s , and a l l two-factor and three-factor interactions. The subjects were 31 p r a c t i c i n g a u d i t o r s employed by seven i n t e r n a t i o n a l public accounting firms in the United States. The eight main e f f e c t s accounted f o r , on average, 7.1.9% of the judgment variance, and the two and three cue interactions accounted for 6.1% of the variance. The insight measurement co r r e l a t i o n was .86 and the test of s t a b i l i t y used showed a c o r r e l a t i o n of .91. The consensus correlation was .67. The four studies reviewed here have been remarkably c o n s i s t e n t . The average values f o r judgment i n s i q h t , consensus and s t a b i l i t y were found to be high. The lower amount of variance of the main e f f e c t s explained in the Ashton and Kramer study and higher amount of judgment variance of the main e f f e c t s for the more experienced auditors can be explained by the amount of training and p r a c t i c a l experience of the respective subjects. Audit Program Planning Study g Joyce perfomed a cue u t i l i z a t i o n study evaluating auditor judgments on audit program planning (audit time estimation) based on information cues about preliminary data 7 c o l l e c t i o n on accounts receivable. The independent and dependent variables are shown in Table IV. The subjects were 35 p r a c t i c i n g auditors from four large public account-ing firms in the Chicago area. Joyce conducted two experi-ments, the f i r s t using a one-half r e p l i c a t i o n of a 2^ f a c t o r i a l design and the second using a f u l l r e p l i c a t i o n of 5 a 2 f a c t o r i a l design. The f i v e main ef f e c t s accounted for 74.7% of the judgment variance. No estimates of the interactions were made. Joyce's r e s u l t s indicated a high variance among the importance of the d i f f e r e n t cues given to the subjects. This i s consistent with p r i o r studies which analyzed sample size estimations by auditors, which the author believes i s highly correlated to the amount of audit work that would be 9 required to be done. Joyce did find that there was - with one exception - high c o r r e l a t i o n of the r e s u l t s among the auditors within the firms studied. Joyce used the same method as Ashton to obtain a measurement of s e l f - i n s i g h t . His r e s u l t s showed a c o r r e l a t i o n of .449 versus .89 for Ashton's o r i g i n a l work. Cognitive Psychology Research in management information systems and cogni-tive psychology w i l l be reviewed here because auditors form s p e c i a l i z e d group of users of management information systems. Cognitive research i s important to t h i s thesis as the judgement processes of the subjects are being studied. 8 Mason and M i t r o f f l u discussed the implications of psycho-l o g i c a l type for researchers and p r a c t i t i o n e r s in MIS. The a r t i c l e has stimulated t h i s author's i n t e r e s t in the application of psychology to MIS. The research in MIS and accounting has used concepts from the f i e l d of cognitive psychology. This focus on cognitive psychology i s a t t r i -butable in part to the research in human information processing and human decision making. Cognition i s a general term covering a l l the various models of knowing — perceiving, remembering, conceiving, judging, reasoning. Researchers in the accounting and MIS f i e l d s have tended to l i m i t t h e i r interest to d e c i s i o n -making applications of cognitive psychology. Accountants' primary interest in studying d e c i s i o n -making i s the p o t e n t i a l f o r improving the q u a l i t y of d e c i s i o n s by improving the information provided. The understanding of human information processing i s thought to aid in c l a r i f y i n g the needs of decision makers.^ MIS concerns are s i m i l a r . Two c r i t i c a l research needs identifed are the development of methods for determing appropriate content of an information system, and investigating the c h a r a c t e r i s t i c s of d e c i s i o n makers which a f f e c t MIS design. Cognitive factors are only a subset of the factors influencing accounting and MIS design. Mason and M i t r o f f i d e n t i f i e d six key variables which comprise a management 9 information system (person,psychological type, organization-1 3 al context, evidence, mode of presentation). Benbasat and 1 4 Taylor referenced three studies which discussed influences on MIS design: Chervany, et al studied the functional r o l e ; Adams and Schroeder surveyed desired information charac-t e r i s t i c s such as accuracy, use and summarization f o r d i f f e r e n t organizational l e v e l s ; Ghymn and King showed that the importance of economic, legal and so c i a l information was di f f e r e n t for various managerial l e v e l s . Cognitive Styles Cognitive style i s a hypothetical construct developed to explain the mediation between stimuli and responses. The term cognitive style refers to the c h a r a c t e r i s t i c ways in which individuals organize conceptually t h e i r perceptions of the i r environment. Several psychologists have noted that cognitive style i s a process of information transformation. If i t were not for these cognitive representations which modify the one-to-one r e l a t i o n s h i p between s t i m u l i and response, an individual would be expected to respond to stim u l i in completely predictable fashions. Common to a l l theories and research in cognitive style is an emphasis on the structure, rather than the content of the thought. Structure refers to how cognition i s organ-ized; content refers to what information i s available. Numerous approaches to cognitive s t y l e have been developed and several authors have attempted to c l a s s i f y various 10 approaches. Despite the elements of common interest in the study of c o g n i t i v e s t y l e , r e s e a r c h e r s d i s a g r e e on the d e t a i l s of, methodology and the use of measuring instruments. The r e s u l t i s a large body of l i t e r a t u r e from which i t i s d i f f i c u l t , i f not impossible, to extract general p r i n c i p l e s . Two d i f f e r e n t interpretations of cognitive s t y l e are now presented, those proposed for use by accountants and those developed by MIS researchers. These demonstrate in part the variety of constructs applied in the f i e l d , and s p e c i f i c a l l y i d e n t i f y the d e f i n i t i o n s often used in account-ing and management information systems research. The Committee on Human Information Processing of the American Accounting Association stated that "Cognitive s t y l e r e f l e c t s how decision makers prefer to approach and solve various tasks." 1^ MIS researchers, on the other hand, have taken a s l i g h t l y broader view of cognitive s t y l e s . In attempting to bring order to information processing a c t i v i t i e s , i n d i v i -duals are viewed as tending to act consistently despite information v a r i a t i o n . These r e l a t i v e l y fixed patterns are called cognitive s t y l e s . 1 ^ Related to the cognitive s t y l e s issues i s the area of cognitive complexity. Cognitive complexity r e f l e c t s an individual's capacity to d i f f e r -entiate and integrate s t i m u l i . Cognitive complexity w i l l be discussed in more d e t a i l l a t e r in t h i s chapter. The c o g n i t i v e s t y l e c o n s t r u c t i m p l i e s i n d i v i d u a l differences in cognitive processes and that such differences 11 are r e l a t i v e l y s t a b l e over time and s i t u a t i o n s . The following i s a discussion of the two cognitive styles which have received the most recognition in the MIS l i t e r a t u r e , f i e l d independence-dependence and a n a l y t i c - h e u r i s t i c s t y l e s . The f i e l d independence-dependence cognitive s t y l e has been extensively studied by W i t k i n . 1 7 F i e l d independence-dependence has been defined by Witkin as " a n a l y t i c a l " as opposed to a "global" way of perceiving, and hypothesized that i t e n t a i l s a tendency to experience items as discrete from t h e i r backgrounds — i . e . , the a b i l i t y to overcome the influence of an embedding context. The Embedded Figures Test, which requires subjects to perceive a simple figure within a larger complex figure, i s often used to measure thi s cognitive s t y l e . This c l a s s i f i c a t i o n has been used by 18 19 Doktor and Hamilton and by Lusk to study the r e l a t i o n -ships between the acceptance of managerial reports and t h i s cognitive s t y l e . The a n a l y t i c - h e u r i s t i c reasoning types were developed by Huysmans for an experiment designed.to test the impact of 21 cognitive styles on managerial recommendations. Analy-t i c a l reasoning refers to a tendency to reduce problem situations to a core set of underlying causal r e l a t i o n s h i p s . H e u r i s t i c reasoning r e f e r s to an emphasis on workable solutions to t o t a l problem sit u a t i o n s . The search i s for analogies with familiar,previously solved problems, rather than a system of underlying causal r e l a t i o n s h i p s . 12 The following section w i l l review two research studies i n v o l v i n g c o g n i t i v e s t y l e s and MIS. Benbasat and 22 Schroeder reported that analytics who used decision aids (e.g., forecasting models) used fewer reports than the h e u r i s t i c s who used decision aids, but for subjects without decision aids the reverse was true. This indicates a reliance on the use of models by analytics and that the models have more information value f o r them than the h e u r i s t i c s who r e l y more on the reporting system even when they have the models available. 2 3 Benbasat and Dexter reported s e v e r a l important experimental r e s u l t s with implications of MIS design. The r e s u l t s indicated that decision making style had a strong impact in determing which of the two information generation approaches would be more suitable in the given decision setting. Their finding showed that structured reports with aggregated data are better' suited for a n a l y t i c s , while the database inquiry system i s better suited for h e u r i s t i c s . In terms of p r o f i t performance h e u r i s t i c s who used struc-tured/aggregated reports did the poorest. The i n a b i l i t y of h e u r i s t i c s to "break up" the structured report to understand the impact of the individual elements would lead to poorer performance with the structured/aggregated r e p o r t s . Analytics performed equally well in terms of p r o f i t with either system? th e i r performance l e v e l was matched by the h e u r i s t i c s who had the database inquiry system. This 24 finding i s consistent with B a r i f f and Lusk's study which 13 reported that h e u r i s t i c s preferred disaggregated reports. Although analytics did not d i f f e r as to p r o f i t performance, those using the structured/aggregated reports had a s i g n i f i -cantly lower decision making time. There was no such difference for h e u r i s t i c s . Furthermore, analytics requested l e s s h i s t o r i c a l information when given the s t r u c t u r e d reports; however, with the database system, they requested approximately twice as much data than did other groups. 25 Many additional studies have been published on the application of the cognitive style constructs to MIS because of the simularity of the re s u l t s with the papers discussed, the i r results w i l l not be included here. Cognitive s t y l e 2 6 issues have been addressed some what by Driver and Mock but t h e i r major thesis dealt with the more s p e c i f i c matter of cognitive complexity than with a general cognitive s t y l e issue. Cognitive style research in accounting has tended to focus on the acceptance of accounting reports by management. 2 7 In Lusk's study, analytics selected an investment repre-sented by a r e l a t i v e l y d i f f e r e n t i a b l e , nonglobal, a r t i c u l a t -ed annual report (a detailed annual report). Heuristics selected an investment represented by a global, u n a r t i c u l a t -ed annual r e p o r t (a very general r e p o r t with minimal f i n a n c i a l information and b r i e f discussions of operations and other matters). These re s u l t s are consistent with those of Benbast and Dexter. 14 Cognitive Complexity As stated e a r l i e r , cognitive complexity i s viewed as the processing of input stimuli as two related but separate a c t i v i t i e s : d i f f e r e n t i a t i o n and integration. D i f f e r e n t i a -tion refers to the individual's a b i l i t y to locate s t i m u l i among dimensions; integration refers to the a b i l i t y to u t i l i z e complex rules to combine t h e i r dimensions. The i n d i v i d u a l who i s low in d i f f e r e n t i a t i o n and integrative a b i l i t y i s called concrete. The individual who i s high in th i s a b i l i t i e s i s ca l l e d abstract. A l l i n d i v i -duals may be c l a s s i f i e d along a continuum from concrete to abstract. The individual's a b i l i t y to d i f f e r e n t i a t e and integrate mediates between stimuli and responses. The input s t i m u l i are f i l t e r e d or processed by the mediating structure. This allows the i n d i v i d u a l to d i f f e r e n t i a t e environmental elements and integrate then to provide behavioral outputs. The r e l a t i o n s h i p to management information systems should be obvious. Managers need data to be f i l t e r e d or manipulated to become useful as information. The mediating structures involved in cognitive complexity i f they could be adequately understood could be p o t e n t i a l l y useful in helping managers f i l t e r data i n t o the p a t t e r n s that they are accustomed to using. Schroder, Driver and Steufert developed a theory of human information processing based on cognitive complexity. They proposed three hypotheses: 15 (1) "Information processing by 'people in general 1 ( i n d i v i -dual differences disregarded) reaches a maximum l e v e l o f s t r u c t u r a l complexity at some optimal l e v e l of environmental complexity (point X i n Figure 1 ) . Increasing or decreasing environmental complexity (points Z and Y) from the optimal point (X) lowers the conceptual l e v e l , as indictated by a reduction in the le v e l of information processing involved in behavior 2 8 (from A to B in Figure 1)." This has been referred to as the "U-Curve Hypothesis." (2) The second hypothesis states that individual d i f f e r -ences w i l l be greater around the optimal point than at the extreme points of the environment. This i s shown in Figure 2. (3) The t h i r d hypothesis proposed was "that f o r more complex structures (Curve A) (a) i s always higher (generates more i n t e g r a t i v e l y complex in f o r m a t i o n -processsing behavior) over the midranges of environ-mental complexity and eoual at the extreme ranges of environmental complexity; and (b) reaches i t s optimal point at higher l e v e l s of environmental complexity. Consequently, the maximal differences between behav-i o r a l complexity for any two curves expressing d i f f e r -ences in str u c t u r a l l e v e l would occur at the optimal 2 9 l e v e l for the more abstract group." Compare E-F with E-G in Figure 2. 1 6 A l l of these hypotheses were supported in the subse-quent empirical studies performed. Driver and Mock"^ presented a review of behavioral research in accounting. Driver and Mock performed an experiment varying the information structures so that they would be optimal for the various decision styles that they had proposed. This study i s s i g n i f i c a n t in that i t attempt-ed to relate decision style theory based on human informa-t i o n processing concepts i n v o l v i n g c o g n i t i v e complexity concepts to preferences to information and amounts of decision time. However, the i r experimental results were inconclusive as were the res u l t s of two sim i l a r research 31 3 2 experiments (Savich , McGhee, Shields and Birnberg ). Complexity theory implies that individuals reauire an optimal l e v e l of information load. Individuals with low lev e l s of d i f f e r e n t i a t i o n w i l l be able to handle only a few l e v e l s of integration and individuals with low l e v e l s of integration w i l l be e a s i l y overloaded when too much informa-tion i s presented. This supports Mason and Mit r o f f ' s argument that the information system should be matched with 3 3 the psychological type of i t s users. C a r l i s l e in his Ph.D. d i s s e r t a t i o n studied the r e l a t i o n s h i p s between cognitive complexity and the complexity of user interfaces for time-sharing terminals. His res u l t s upheld Schroder, Driver and Steufert's hypotheses. What i s more important, however, are the two i n t e r r e -lated components of conceptual complexity already mentioned: 17 d i f f e r e n t i a t i o n , i . e . , the amount or number of attr i b u t e s of information used by the individual in his thinking and integration, i . e . , the number of concepts formed by the i n d i v i d u a l . Although some authors continue to d e f i n e cognitive complexity only in terms of d i f f e r e n t i a t i o n (e.g., 3 4 B i e r i ), the complexity of integration has become the primary focus of inte r e s t . The a b i l i t y to form a vari e t y of concepts on the basis of a given informational array appears to be the most adaptive facet of human i n t e l l i g e n c e . Cognitive complexity i s important to audit judgment research because auditors are required to integrate various p i e c e s of information i n making d e c i s i o n s concerning internal control systems,, substantive testing procedures, s u f f i c i e n c y of e v i d e n t i a l matter, fairness of statement presentation, and so on. Cognitive complexity and cognitive style concepts have not yet been applied to auditor judgment research. This thesis w i l l study the differences between cognitive complex and cognitive simple subjects in the context of internal control evaluations which w i l l be analysed to determine auditor consensus. Summary This chapter has presented a review of the l i t e r a t u r e on audit judgment and cognitive s t y l e s research. This l i t e r a t u r e forms the foundation for the research to be carried out. Although much has been written in the area of 18 cognitive s t y l e s , l i t t l e research has been done in applying cognitive s t y l e research to the accounting and MIS f i e l d s . To the author's knowledge, no research has been published on the a p p l i c a t i o n of c o g n i t i v e s t y l e s to a u d i t judgment research. 19 Table I — Ashton's Questions' 3 3 Are the tasks of both timekeeping and payment of employees adequately separated from the task of payroll preparation? Are the tasks of both payr o l l preparation and payment of employees adequately separated from the task of pay r o l l bank account r e c o n c i l i a t i o n ? Are the names on the pa y r o l l checked p e r i o d i c a l l y against the active employee f i l e of the personnel department? Are formal procedures established for changing names on the p a y r o l l , pay rates, and deductions? Is the p a y r o l l audited p e r i o d i c a l l y by i n t e r n a l aud i t o r s ? Was the internal control over payr o l l found to be sat i s f a c t o r y during the previous audit? 2 0 Table II - Hamilton and Wright Questions J b Is the task of timekeeping adecruately separated from the task of payroll preparation? Is the task of payro l l preparation adequately separated from the task of payro l l bank account r e c o n c i l i a t i o n ? Is the task of payment of employees adequately separat-ed from the tasks of timekeeping and payro l l bank account r e c o n c i l i a t i o n ? Are formal procedures established for changing names on the p a y r o l l , pay rates and deductions? Did internal auditors audit the p a r o l l during the audit period? 21 Table III - Ashton and Brown Questions 3 7 Are the tasks of both timekeeping and payment of employees adequately separated from th task of payr o l l preparation? Are the tasks of both payroll preparation and payment of employees adequately separated from the task of p a y r o l l bank account r e c o n c i l i a t i o n ? Are the names on the p a y r o l l checked p e r i o d i c a l l y against the active employee f i l e of the personnel department? Are the duties of theose preparing the payroll rotated? In h i r i n g new employees, i s an inquiry made as to his/her background and former employers? Are formal procedures established for changing names on the p a y r o l l , pay rates, and deductions? Is the p a y r o l l audited p e r i o d i c a l l y by i n t e r n a l aud itors? Was the internal control over payroll found to be sa t i s f a c t o r y during the previous audit? 22 Table IV - Joyce variables' 3 0  Independent Variables 1 . Sales approval 2. Bad Debt Expense/Sales 3. Write-off Approval 4. Separation of B i l l i n g Function and Subsidiary Ledger Maintenance 5. Sales/Average Accounts Receivable Dependent Variables (man-hours to complete audit procedures) 1 . Confirmation of accounts receivable 2. Review of accounts written o f f as u n c o l l e c t i b l e 3. Review of cash c o l l e c t i o n s of accoutnts receivable subsequent to balance sheet date 4. Determination of adequacy of allowance f o r uncol-l e c t i b l e accounts 5. Review of year end sales cutoff 23 Figure 1 - General Relationship between Environmental and 39 Behavioral Complexity High low y x Environmental Complexity High Figure 2 - Relationship between Environmental and Behavioral 40 Complexity for Different Levels of Personality Structure High Low j Curve A <yr i ' T I I i \ l \ I ^ - _ I low c D High Environmental Complexity 24 FOOTNOTES 1 Ashton, Robert H. "An Experimental Study of Internal C o n t r o l Judgments", Journal of Accounting Research, Spring 1974, p. 43. 2 Gibbins, Michael, "A Behavioral Approach to Auditing Research", Symposium on Auditing Research II, University of I l l i n o i s at Urbana-Champaign, 1977, pp. T41-184. 3 op. c i t . 4 Hays, William L., S t a t i s t i c s for psychologists, Holt, Rinehart and Winston, 1963, pp. 406-407. 5 Hamilton, R.E. and Wright, W.F., "The Evaluation of Internal Control over P a y r o l l " , Research Paper No. 397, Graduate School of Business, Stanford U n i v e r s i t y , December 1977. 6 Ashton, Robert H. and Kramer, Sandra S., "Students as Surrogates i n Behavioral Research: Some Evidence", unpublished manuscript, November 1978. 7 Ashton, Robert H. and Brown Paul R., " D e s c r i p t i v e Modeling of Auditors' Judgments", Working Paper 79-6, Graduate School of Business, University of Texas at Austin, 1979. 8 Joyce, Edward J . , "Expert Judgment in Audit Proqram Planning", Journal of Accounting Research, Supplement 1976, pp. 29-60. 9 Aly, H.F., and Duboff, J.I., " S t a t i s t i c a l vs. Judqmental Sampling: An E m p i r i c a l Study of A u d i t i n g Accounts Receivable of a Small Reta i l Store," The Accounting  Rev Jew, 1971, pp. 119-128, Committee on Aud i t ing Procedure, A Case Study of the Extent of Audit Samples, New York: American Institute of Accountants, 1955, and C o r l e s s , J.C., "Assessing P r i o r D i s t r i b u t i o n s f o r Applying Bayesian S t a t i s t i c s in Auditing," The Account- ing Review, 1972, pp. 556-566. 10 Mason, Richard 0. and M i t r o f f , Ian I., "A program for research on management information systems," Management  Science, January 1973, pp. 475-487. 11 Committee on Human Information Processing of the American Accounting Association, "Report of the Com-mittee," August 1977. 25 12 Dickson, G.W., "Management Information Systems: D e f i n i -tions, Problems, and Research, " SMIS Newsletter, Vol. 1, 1971, pp. 6-12. 13 op. c i t . , p. 475. 14 Benbasat, Izak and Taylor, Ronald N. , "The impact of cognitive styles on information system design", MIS  Quarterly, June 1978, pp. 43-54. 15 Report of the 1976-77 committee on human information P r o c e s s i n g , American Accounting A s s o c i a t i o n , Auqust 9//, p. 2U. 16 Schroder, Harold M. and Suedfeld, Peter, Human Informa- tion Processing, Toronto: Holt, Rinehart and Winston, Inc., 1967. 17 Witkin, H.A., Dyk, R.G., Patterson, H.F., Goodenough, D.R., and Karp, S.A., Psychological D i f f e r e n t i a t i o n , New York: John Wiley & Sons, 1962. 18 Doktor, R.H. and Hamilton, W.T., "Cognitive Style and the Acceptance of Management Science Recommendations," Management Science, A p r i l 1973, pp. 884-894. 19 Lusk, E.J., "Cognitive Style Aspects of Annual Reports: Fi e l d Independence/Dependence," Empirical Research in  Accounting: Selected Studies, 1973, pp. 191-202. 20 Report of the 1976-77 committee on human information fr o c e s s i n q , American Accounting A s s o c i a t i o n , August JTT: " 21 Huysmans, J.H.B.M.,' The Implementation of Operations  Research, Wiley-Interscience, New York, 1970. 22 Benbasat, Izak, and Schroeder, Roger G. , "An Experi-mental I n v e s t i g a t i o n of Some MIS V a r i a b l e s , " MIS Quarterly, March 1977, pp. 37-49. 23 Benbasat, Izak, and Dexter, Albert S., Value and Events  Approaches to Accounting, Working paper No. 488, University of B r i t i s h Columbia, Faculty of Commerce, A p r i l 1978. 24 B a r i f f , M.L. and Lusk, E.J., "Cognitive and personality tests for the design of management information systems", Management Science, A p r i l 1977, pp 820-829. 26 25 For example see Cervany, N.L. and Dickson, G.W., "An Experimental Evaluation of Information Overload in a Production Environment," Management Science, June 1974, pp. 1335-1344, and Dickso"n^ G.W. and Barkin, S.R., "An Investigation of Data Selection and Cognitive Style," Information and Management, November 1977, pp. 35-45. 26 D r i v e r , Michael J . , and Mock, Theodore J.,"Human Information Processing, D e c i s i o n S t y l e Theory, and Accounting Information Systems", The Accounting Review, July 1975, pp. 490-508. 27 Lusk, E.J., "Cognitive Style Aspects of Annual Reports: F i e l d Independence/Dependence, "Empirical Research in  Accounting: Selected Studies, 1973, pp. 191-202. 28 Ibid. p. 28. 29 Ibid, p. 40. 30 Driver and Mock, op. c i t . 31 Savich, Richard S., "The Use of Accounting Information The Accounting Review, July 1977, pp. 642-652. 32 McGhee , Walter; S h i e l d s , Michael D. , and B i r n b e r g , Jacob, "The E f f e c t s of P e r s o n a l i t y on a Subject's Information Processing", The Accounting Review, July 1978, pp. 681-697. 33 C a r l i s l e , J ., "Cognitive Factors in Interactive Decision Systems," Ph.D. d i s s e r t a t i o n , Yale University, 1974. 34 B i e r i , J. , " C o g n i t i v e complexity and judgment of inconsistent information", in R.P. Abelson, E. Aronson, W.J. McGuire, T.M. Newcomb, M.J. Rosenburg, and P.E. Tannenbaum (Eds.), Theories of cognitive complexity: a  sourcebook, Chicago: Rand McNally, 1968. 35 Ashton, Robert H., "An Experimental Study of Internal C o n t r o l Judqments" , J o urnal of Accounting Research, 1974, pp. 143-157. 36 Hamilton, R.E. and Wright, W.F., "The Evaluation of Internal Control over P a y r o l l " , Research Paper No 397, Graduate School of Business, Stanford U n i v e r s i t y , December 1977. 37 Ashton, Robert H. and Brown Paul R., " D e s c r i p t i v e Modeling of Auditors' Judgments", Working Paper 79-6, Graduate School of Business, University of Texas at Austin, 1979. 27 38 Joyce, Edward J . , "Judgment in Audit Proqram Planning", Journal of Accounting Research, Supplement 1976, pp. 20-60. 39 Schroder, Harold M. , Driver, Michael J.,and Steufert, Si e g f r i e d , Human Information Processing; Individuals  F u n c t i o n i n g i n Complex S o c i a l S i t u t a t i o n s , Toronto: Holt, Rinehart and Winston^ Inc, 1 967, p. 3~6~. 40 Ibid, p. 38. 2 8 CHAPTER II METHODOLOGY OF THE EXPERIMENT Introduction The primary focus of t h i s chapter i s upon the method-ology used in the experiment. In t h i s chapter, three topics are discussed. F i r s t , the experimental task i s described in d e t a i l ; second, the s p e c i f i c experimental design i s present-ed; and t h i r d , the strengths and weaknesses of the experi-ment are discussed. In addition, d e t a i l s of the administra-tion of the experiment are given. The Experimental Task The selection of an area of auditing on which judgments could be studied i s a d i f f i c u l t task. Evidence concerning the c h a r a c t e r i s t i c s and operation of the internal control system must be gathered in order to perform an audit in compliance with professional standards. 1 The elements of an internal control system are well defined and have been discussed in the professional l i t e r a t u r e for some time. The audit approach to internal control evaluation may vary from firm to firm but the basic elements are the same. Due to 2 the d i f f e r e n t approaches to audit f i e l d work used by the firms, the area of internal control was considered to be a more valuable area for research by the author than would other areas of audit work. 29 A l l factors relevant to an evaluation of internal control can not be presented in an experimental study. For t h i s experiment, f i v e factors were selected. These are l i s t e d in Table V. These questions represent f i v e basic internal control factors used by an international public accounting firm. They were selected as being representative auestions on the adequacy of control. The five questions represent the independent variables 5 3 in a 2 f a c t o r i a l design. The questions can be answered with either "yes" or "no" representing two l e v e l s of each fac t o r . Each combination of yes-no answers to the f i v e questions represents a a possible set. There are 2^ = 32 possible sets. For each possible set, each subject made a judgment on the adequacy of internal control for the given case (See appendix 1). The judgments were expressed using a scale from 1 (extremely weak) to 6 (adequate to strong) 4 developed by Ashton. Ashton in a p i l o t study for his primary experiment used an eight point scale which included various degrees of strength besides va r i o u s degrees of weaknesses in the internal control. The subjects of t h i s p i l o t study stated that the various degrees of strength were unnecessary, as they were unconcerned about the strength of internal control once i t was considered to be adequate. They were, however, concerned with the various degrees of weaknesses which could have an ef f e c t on other audit work. The reponses to the eight possible ratings were not normally d i s t r i b u t e d . In a 30 second p i l o t study, the six point scale was used and the responses obtained took on the c h a r a c t e r i s t i c s of a normal d i s t r i b u t i o n . Experimental Design In this design, a l l main ef f e c t s and interactions are estimable. The thirty-two combinations of factor l e v e l s which were used in the experiment are presented in Table VI. The a n a l y s i s - o f - v a r i a n c e technique was a p p l i e d to the r e s u l t s of each subject i n d i v i d u a l l y . F - s t a t i s t i c s were 5 obtained for each e f f e c t (main and a l l i n t e r a c t i o n s ) . Hays stated that: ... when large numbers of F tests are performed, the experimenter should not pay too much attention to isolated results that happen to be s i g n i f i c a n t . Rather, the p a t t e r n and i n t e r p r e t a b i 1 t i y of r e s u l t s , as well as the strength of association represented by the findings, form a more reason-able s t a t i s t i c for the o v e r a l l evaluation of the experiment. Hays suggested using the omega-squared s t a t i s t i c to measure the strength of association. The interpretation of t h i s s t a t i s t i c i s similar to that of the squared correlation c o e f f i c i e n t . ^ The omega-squared s t a t i s t i c indicates the proportion of variance explained by each e f f e c t . The second issue under consideration in t h i s thesis concerns the amount of insight that the subjects had into t h e i r own judgment processes. To inquire into t h i s , the 7 technique used by Slovic was used. The subjects were asked to allocate 100 points to the f i v e factors to indicate the 31 r e l a t i v e importance of each factor to their judgments after making t h e i r rating of the 32 cases. An "insight index" was c a l c u l a t e d by c o r r e l a t i n g the s u b j e c t i v e weights with normalized omega-squared s t a t i s t i c s for the main e f f e c t s over the five factors. The third issue of concern i s judgment consistency; more s p e c i f i c a l l y , what Ashton calle d consensus or consis-tency across d i f f e r e n t auditors at the same point in time. Consistency was evaluated by correlating each subject's responses with the other subject's responses. The fourth issue dealt with in t h i s experiment i s the t e s t i n g f o r s i g n i f i c a n t d i f f e r e n c e s i n the p a t t e r n of responses due to the subject's cognitive complexity. An analysis-of-variance model, with the. independent variables being categorial variables representing the f i v e cues and the abstract/concrete construct, and the dependent va r i b l e being the subject responses has been applied. It was also considered necessary to determine whether the d i s t r i b u t i o n of responses d i f f e r e d between the two groups for each of the possible combinations of yes/no answers. This was tested by computing for each set of yes-no answers the Kolmogorov-g Smirnov Z - s t a t i s t i c s for the two groups of subjects --abstracts and concretes. In addition, a comparison of the s i g n i f i c a n t e f f e cts for each group was made. Each of the four major issues was analyzed to determine whether s i g n i f i c a n t differences could be found between firms and l e v e l s of subject experience. 32 The f i n a l issue i s the useful n e s s of f r a c t i o n a l f a c t o r i a l design in human judgment research. As discussed l a t e r in this chapter, the use of f r a c t i o n a l f a c t o r i a l designs can confuse the res u l t s of the analysis-of-variance technique. Using the analysis-of-variance r e s u l t s of a subset of the answers to produce answers to a one-half f r a c t i o n a l f a c t o r i a l design, the two sets of r e s u l t s were compared to determine how many s i g n i f i c a n t high-order interactions were lo s t by using a f r a c t i o n a l f a c t o r i a l design, and how many s i g n i f i c a n t main and low-order i n t e r -actions were confused with high-order interactions. Strengths And Weaknesses Of The Experiment This section w i l l discuss the use of the Brunswik lens model, and f a c t o r i a l design in human judamental research. It w i l l also discuss the use of the Paragraph Completion Test as a measuring instrument for cognitive complexity. Brunswik Lens Model 9 The lens model as developed by Brunswik is represented by the diagram in Figure 3.1^ The diagram i s sim i l a r to the shape of a lens thus the name "lens model." The model can be described as follows: the environment in which the judge exists i s represented by the l e f t side of the model and the individual's judgmental processes are represented by the right side of the model. The notation used in Figure 3 i s explained below. 3 3 Y e represents an unknown event (the d i s t a l s tate), which the judge would l i k e to know. r g ^ i s the relat i o n s h i p between the d i s t a l variable and the cue X^. The re l a t i o n s h i p , cal l e d a v a l i d i t y c o e f f i -c i e n t , i s determined by correlating repeated occurrences of the cue and the d i s t a l variable. X^ are cues which may be used to predict the current or future state of the d i s t a l variable. Cues are subject to observation while the d i s t a l variable may or may not be subject to observation. r g ^ i s the rel a t i o n h i p between the cue X^ and the judgment of the d i s t a l variable (Y ). This v a r i a b l e , c a l l e d the u t i l i z a t i o n c o e f f i c e n t , i s calculated by co r r e l a t i n g repeated occurrences of the cue and the individual's judg-ments of the d i s t a l variable. It represents the extent to which an individual u t i l i z e s the cue to predict the d i s t a l v a r i a b l e . Y i s the individual's judgment of the state of the d i s t a l variable. Y and Y w i l l vary because there i s a less than perfect r e l a t i o n s h i p between the unknown state and the predicted value. In the current experiment, the cues (X^s) are repre-sented by the fi v e questions shown in Table V. The Y s variable becomes the subject's evaluations of the degree of weakness in internal control. The Y variable, the "true" e state of the internal control, i s not observable. The 34 values of , the u t i l i z a t i o n coefficent, can be c a l c u l a t -ed but the r e ^ variables, the v a l i d i t y c o e f f i c i e n t s , can not be computed. The Brunswik lens model has been interpreted in three major ways. Brunswik implied that i t should be used as a co r r e l a t i o n a l model. Hammond"*"* interpreted i t as a multiple regression model. A third interpretation is the analysis-of-variance approach. The multiple regression analysis approach w i l l be discussed before the analysis-of-variance method is described. The multiple regression approach is the most frequently used form of the lens model. The l e f t sid° of the lens model may be discribed by the following multiple regression formula: = b ,X, + b „X 0 + ... + b X„ (1) e e l 1 e2 2 en n when Y i s the predicted value of the d i s t a l variable given the cues X1 through X . The multiple c o r r e l a t i o n c o e f f i -1 n cient Rg is indicative of the degree to which the probabil-i s t i c cues serve as sources of information about the state of the d i s t a l variable. The same type of relationships exist on the right hand side of the model, but here the s t a t i s t i c a l r elationships are between the cues and the individual's judgement of YQ which i s Y s. In regression terms, Y = b ,X, + b-X,, + ... + b X s s i 1 s2 2 sn n (2) 35 The multiple c o r r e l a t i o n coefficent R g indicates the degree of r e l a t i o n s h i p between the cues and the i n d i v i d u a l ' s judgment of the d i s t a l variable. Equations (1) and (2) are linear models.. The use of these models assumes a linear relationship between each cue and the d i s t a l variables and between each cue and the individual's judgments. The values of Y . Y , Y , Y can be correlated with e s e s each other to obtain indices with which to evaluate the judgments of an i n d i v i d u a l . R 0 and R are multiple c o r r e l a t i o n c o e f f i c i e n t s that have been discused above. The index r , the c o r r e l a t i o n between YQ and Y s, represents an individual's a b i l i t y to predict the state of the d i s t a l variable. This does not take into consideration that Y g is usually not pe r f e c t l y predictable from a given set of cues. The index r , the optimality index, is the correlation between Y„ and Y . The index r , the matching index, is the c o r r e l a t i o n between Y g and Y . The remaining index, r , indicates the degree to which the multiple regression equation that represents the judgmental strategy of the subject can predict the d i s t a l variable. Two major c h a r a c t e r i s t i c s of the lens model must be stressed. F i r s t , f i v e of the six indices which can be computed require a knowledge of the true value of the d i s t a l variable, Y . This i s not possible in most real-world 36 situations and i t i s not possible in the current study. It is possible to study judgments, however, by focusing on the rig h t side of the model; the individual's judgmental system. Second, were multiple regression models to be used, an assumption of e i t h e r l i n e a r or nonlinear r e l a t i o n s h i p s between the cues and the judgments must be made. The analysis-of-variance model can be used to show both l i n e a r and configural cue u t i l i z a t i o n . C onfigurality i s shown by the interactions generated by the analysis-of-variance computations. In t h i s approach, the cues can be treated as categorial treatment factors instead as continous v a r i a b l e s . The a n a l y s i s - o f - v a r i a n c e approach has been applied in this thesis s o l e l y on the right hand side of the lens model. The e f f o r t has been directed to describing judgmental processes. Limitations of the Lens Model The major l i m i t a t i o n of t h i s study i s that the true state of nature i s not known. Therefore, only the cue u t i l i z a t i o n c o e f f i c i e n t for the subjects can be determined and the other s t a t i s t i c a l measures can not be evaluated (^Q, r , and r ). This i s , of course, at a variance with the nr es o r i g i n a l Brunswik model. It i s , however, a representation of the actual audit process, where the auditor obtains evidence (cues) to determine the true state of nature and inso doing forms judgments. The lens model can therefore be 3 7 viewed as an information processing model representing audit . j , 12 judgment processes. An additional l i m i t a t i o n of the model i s that cues must be divided in discrete lev e l s in order to use the analysis of variance model. This means the cue value must be a discrete value such as "yes" or "no". The analysis of variance model does provide a descrip-tion of individual subjects' judgment processes but i t does not provide an explanation of those processes. Fractional F a c t o r i a l Designs The use of f r a c t i o n a l f a c t o r i a l designs i n audit judgment research had been suggested by Ashton and Brown. They indicated that using a small number of cues makes such studies " u n r e a l i s t i c . " They stated " I t i s an empirical question as to whether the use of a small number of cues contributes to high l e v e l s of consensus, s t a b i l i t y , i n sight, explained variance, and to other findings t y p i c a l of such studies. " 1 3 There are several advantages and disadvantages to the use of f r a c t i o n a l f a c t o r i a l studies. Multifactor studies are more e f f i c i e n t than the experimental approach of manipulating only one factor at a time. The approach of manipulating only one factor provides less information than a multifactor study. Multifactor analysis permits immediate investigation of interaction e f f e c t s and reduces the number of cases required for analysis. However, i f the factors are 38 not independent, the simple e f f e c t s of a f a c t o r vary according to the combination of other factors. This e f f e c t could be l o s t were a f r a c t i o n a l design to be used. Frac-t i o n a l f a c t o r i a l experiments have the advantage of reducing the number of r e p l i c a t i o n s needed. In a f r a c t i o n a l r e p l i c a -tion of a f a c t o r i a l experiment, the high-order interactions cannot be evaluated. An additional disadvantage of frac-t i o n a l designs i s that the e f f e c t s due to main e f f e c t s and low-order interactions are confused with the e f f e c t due to high-order i n t e r a c t i o n s . For example, i n a one-half 5 r e p l i c a t i o n of a 2 f r a c t i o n a l design, main e f f e c t A has an a l i a s of the interaction of B, C, D, and E and the two-way interaction AB has the a l i a s of the interaction of C, D, and E. A s i g n i f i c a n t main e f f e c t or two-factor int e r a c t i o n must therefore be interpreted with caution because the a l i a s e f f e c t causes the e f f e c t and i t s a l i a s to be irrevocably confused. The seriousness of t h i s i s not great i f the 1 4 higher-order interactions are n e g l i g i b l e . Paragraph Completion Test 1 5 Harvey, Hunt and Schroeder formulated a theory of p e r s o n a l i t y s t r u c t u r e which p o s t u l a t e d a continuum of conceptual functioning from very simple (concrete) to very complex (abstract). The instrument with which such func-tioning was measured was a sentence completion t e s t . This test has now been expanded to a paragraph completion t e s t . In this test the subject i s presented with a selected set of 39 stems (eg., Rules When I am in doubt e t c . ) . The subject i s asked to use each stem as a basis for completing the sentence and to write up to two additional sentences. The stems used in the test were selected primarily to assess the complexity of conceptual structures r e l a t i n g to inter-, personal s t i m u l i . Each protocol i s scored according to the l e v e l of c o g n i t i v e s t r u c t u r e i t r e f l e c t s . S c h r o d e r 1 ^ described the s p e c i f i c rules for rating personality struc-ture on the basis of the completed stems. The t e s t - r e t e s t method has been used to test the r e l i a b i l i t y of the instrument over r e l a t i v e l y short periods of time. For a preliminary study at a Midwestern univer-s i t y , two groups of students were tested at i n t e r v a l s of 3 and 9 months. The r e l i a b i l i t y c o e f f i c i e n t for the 3-month group was .67 (n=36) and for the 9-month group was .59 (n=54). This indicates a higher l e v e l of r e l i a b i l i t y for 1 7 t h i s psychological test. The paragraph completion test i s considered useful in t h i s experiment as i t i s a measure of integrative complexity, or the a b i l i t y to integrate and d i f f e r e n t i a t e information. This complexity characterizes the task of auditors who must evaluate large amounts of information in order to express an opinion on the f i n a n c i a l statements of an organization and in doing so also evaluate the organization's internal c o n t r o l . 40 Administration Of The Experiment The administration of the study was conducted between May 1 5, 1 979 and June 28, 1979. The participants were 36 p r a c t i c i n g a u d i t o r s who were employed by f i v e p u b l i c accounting firms i n Vancouver, B r i t i s h Columbia. A l l partici p a n t s had a minimum of one year of p r a c t i c a l audit experience. Firm 1 s u p p l i e d 12 p a r t i c i p a n t s , Firm 2 supplied 6 p a r t i c i p a n t s , Firm 3 supplied 11 p a r t i c i p a n t s , Firm 4 supplied 3 participants and Firm 5 supplied 4 pa r t i c i p a n t s . Firm 3 i s an o f f i c e of the firm from which the f i v e questions on internal control were adapted. Firms 1 through 4 are l o c a l o f f i c e s of international public accounting firms; firm 5 i s the l o c a l o f f i c e of a national Canadian firm. The experiment was conducted through the use of s e l f -administered questionnaires d i s t r i b u t e d by the f i v e firms. The questionnaires consisted of a b r i e f paragraph explaining that the purpose of the materials was an investigation in audit judgments. The subjects were asked to complete the paragraph completion test, which was used to measure t h e i r cognitive complexity. They were then requested to read a short narrative description of a hypothetical company and i t s new computerized system on which the internal control questions were based. The case i s presented in Appendix 1. The auditors were then requested to completed the 32 sets. An example of one set i s shown in Table VI. Following the 32 sets, the "ins i g h t " question asked the subjects to 41 allocate 100 points according to how important they thought each question (cue) was to t h e i r judgments on the degree of weakness in the internal control. Summary This chapter has described the experimental task on which 36 independent a u d i t o r s p a r t i c i p a t e d . The task r e q u i r e d the subject to complete a p s y c h o l o g i c a l t e s t related to cognitive complexity, and to judge the strength or weakness of internal control in a hypothetical company as a f u n c t i o n of f i v e questions which represent p a r t i a l judgments on the weaknesses in internal control. Each subject made 32 such judgments, one each for the 32 possible combinations of yes-no answers to the questions on a scale from 1 to 6. The chapter described the methodology used to examine the data. The analysis-of-variance approach was used to analyze the r e s u l t s of each individual subject in an attempt to determine the r e l a t i v e importance of each cue to the subject. The method for studying each subject's "insight" into t h e i r own judgment processes was described. In addition, the chapter described how the consistency of judgment across the subjects at the same point in time was measured by using the c o r r e l a t i o n c o e f f i c i e n t . The comparison of the r e s u l t s obtained using complete and a one-half f r a c t i o n a l f a c t o r i a l designs to evaluate the 42 usefulness of f r a c t i o n a l f a c t o r i a l design in human judgment research was described. The chapter discussed some advantages and disadvantages of the techniques used in the experiment. F i n a l l y , the actual administration of the experiment was described. 43 Table V - Questions used in current study 1. Based upon your study of the system are the controls over the completeness of processing adequate? 2. Based upon your study of the system are the controls over the maintenance of subsidiary records supporting control accounts adequate? 3. Based upon your study of the system are the controls over the posting of quantities to detailed inventory records adequate? 4. Based upon your study of the system are the controls for supervisory review and approval adequate? 5. Based upon your study of the system are the controls over the checking of extensions and additions adequate? i 44 Table VI - Example of internal control case used in experiment Answers provided to you by your assistant Summary of control area Answer 1 . Based upon your study of the system are the controls over the completeness of processing adequate? NO 2. Based upon your study of the system are the controls over the maintenance of subsidiary records supporting control accounts adequate? YES 3. Based upon your study of the system are the controls over the posting of quantities to detailed inventory records adequate? NO 4. Based upon your study of the system are the controls for supervisory review and approval adequate? YES 5. Based upon your study of the system are the controls over the checking of extensions and additions adequate? YES What i s your evaluation of the internal control? extremely very substantial some not quite adequate weak weak weakness weakness adequate to strong 1 2 3 4 5 6 C i r c l e the number corresponding to your answer. Ashton, 1973, p. 56. 46 FOOTNOTES 1 Canadian I n s t i t u t e of Chartered Accountants, CICA handbook, 1979, pp. 5301-5321, and American Institute of C e r t i f i e d P u b l i c Accountants, Statement on A u d i t i n g  Standards No. 1, 1973, pp. 13-35. 2 For a d e f i n i t i o n of audit f i e l d work see AICPA, State- ment on Auditing Standards No. 1, 1973, pp. 53-70. 3 Cochran, William G. and Cox, Gertrude, Experimental  Design, New York: John Wiley & Sons, Inc., pp. 148-182. 4 Ashton, Robert H., Judgement formation in evaluation of  i n t e r n a l c o n t r o l : an a p p l i c a t i o n of Brunswik's l e n s model, unpublished PhD d i s s e r a t i o n , U n i v e r s i t y of Minnesota, 1973. 5 Hays, William L., S t a t i s t i c s for psychologists, Toronto: Holt, Rinehart and Winston, 1963, p. 410. 6 Lindman, Harold R. , A n a l y s i s of Variance in Complex  Experimental Designs, San Francisco: W.H. Freeman and Company, 1974, pp. 242-245. 7 S l o v i c , Paul, "Analyzing the Expert Judge: A d e s c r i p t i v e study of a stockbroker's decision processes", Journal of  Applied Psychology, August 1969, pp. 255-263. 8 Siegel, S., Nonparametric s t a t i s t i c s for the behavioral  sciences, New York: McGraw-Hill, 1956, p. 130. 9 Brunswik, Egon, The conceptual framework of psychology, International Encyclopedia of Unified Science, Vol. 1, No. 10, Chicago: University of Chicagoo Press, 1952. 10 This discussion of the Brunswik lens model i s based primarily on Beal, Don, G i l l i s , John S., and Stewart, Tom, "The lens model: computational procedures and applications," Perceptual and Motor S k i l l s , 1978, pp. 3-28, and Ashton, 1973. 11 Hammond, Kenneth R.,"Probalistic Functioning and the C l i n i c a l Method," Psychological Review, July 1965, pp. 225-262. 12 W i l l , H.J., "Auditing in Systems P e r s p e c t i v e " , The Accounting Review, October 1974, p. 690. 13 Ashton and Brown, p. 2. 47 14 Cochran and Cox, pp. 244-276. 15 Harvey, O.J., Hunt, D.E., and Schroder, H.M. , Conceptual  systems and personality structure, New York: John Wiley & Sons, 1961. 16 Schoder, Harold M., "Conceptual complexity and person-a l i t y organization," in H.M. Schorder and P. Suedfeld (Eds.), Personality theory and information processing, New York: Ronald Press, 1971, pp. 240-273. 17 Gardiner, Gareth S. and Schroder, Harold M., " R e l i a b i l i -ty and v a l i d i t y of the paragraph completion t e s t : theor-e t i c a l and empirical notes.", Psychological Reports, 1972, pp. 959-962. 48 CHAPTER III RESULTS OF THE EXPERIMENT Introduction The experimental results are presented in t h i s chapter under three p r i n c i p a l headings: judgment d e s c r i p t i o n , judgment i n s i g h t , judgment c o n s i s t e n c y and c o g n i t i v e complexity. The implications of these r e s u l t s are discussed in d e t a i l in the concluding chapter. Appendices 2 and 3 contain the data for the individual subjects. Judgment Description The experimental cases were developed using a f u l l f a c t o r i a l design and analysis-of-variance computations were performed on each auditor's responses for the f u l l frac-t i o n a l and for one-half f r a c t i o n a l f a c t o r i a l designs. This methodology allowed the author to determine the amount of reliance placed on the summary internal control Questions used by each auditor. The computation of the omega-squared s t a t i s t i c i d e n t i f i e d differences in the r e l a t i v e importance of the various questions to each auditor. An o v e r a l l indication of the importance to the auditors of the f i v e questions and a l l the possible interactions of those questions i s shown for the f u l l f r a c t i o n a l f a c t o r i a l design in Table VII. A s i m i l a r set of r e s u l t s for the main e f f e c t s and two-way interactions for the one half f r a c t i o n a l 49 f a c t o r i a l design are shown in Table VIII. The f i r s t column of the tables l i s t s the main e f f e c t s (questions) and the interactions (combinations of questions), the second column shows the number of auditors (out of 36) for whom the F - r a t i o f o r that p a r t i c u l a r e f f e c t was s t a t i s t i c a l l y s i g n i f i c a n t at the .005 l e v e l . There i s no major difference between the r e s u l t s at the .005 level and the .05 l e v e l . Importance Of Main Effects And Interactions One general conclusion that can be drawn from the re s u l t s shown in Tables VII and VIII refers to the impor-tance of the questions (main e f f e c t s ) . When considering a l l the subjects for the f u l l design, the main e f f e c t s accounted for an average 79.99 percent of the t o t a l variance in judgment. On average, the least s i g n i f i c a n t main e f f e c t accounted for more than three times as much variance in judgment as did the most s i g n i f i c a n t i n t e r a c t i o n . Another general conclusion i s that the f i v e internal control questions were not equally important. Question 1 on the completeness of processing accounted for nearly twice the variance in judgment as did question 2 on maintenance of subsidiary records and question 4 on supervisory review, three times as much as question 3, and six times as much as question 5. For the 36 auditors in the f u l l f a c t o r i a l design, the percentage of variance accounted for by the main ef f e c t s ranged from 23% to 93%, the range for interactions was 7% to 77%. 5 0 E f f e c t s Of Firm A f f i l i a t i o n And Level Of Experience An analysis of the importance of main e f f e c t s and interactions by firm a f f i l i a t i o n and by l e v e l of experience provided the r e s u l t s in Tables IX and X. There are no s t a t i s t i c a l l y s i g n i f i c a n t differences between the r e s u l t s for the firms and l e v e l s of experience. Examination of Table IX discloses several differences in the importance of main e f f e c t s between firms. Of p a r t i c u l a r interest i s the difference between firms 4 and 5 and the remaining firms. On average, importance of ques-tions 2 and 4 deviated greatly from firms 1, 2, and 3, the other international accounting firms. A review of t h e i r fundamental audit approaches indicated that firms 1, 2, and 3 emphasized to a higher degree evaluation and understanding of the information system than does firm 4. Firm 5, the Canadian national accounting firm, deviated r a d i c a l l y from the other four firms. Nearly one-half of the judgment variance was accounted for by question 1, only 11% by questions 3 and 4, and less than 1% by question 5. The auditors in firm 5 apparently consider the completeness of processing to be of greater significance to internal control than the other four aspects, e s p e c i a l l y the testing of extensions and additions which accounted for less than 1% of the judgment variance. Table X on the importance of main e f f e c t s and inter-actions by l e v e l of experience reveals major differences 51 between the two lev e l s of experience considered in t h i s experiment. The less experienced group (less than three years of experience) placed less emphasis on the f i r s t two questions on completeness of processing and maintenance of subsidiary records, and they placed s l i g h t l y more emphasis on the fourth and f i f t h questions on supervisory review and testing of extensions and additions than did the more experienced group (over three years of experience). The interactions of the questions showed only minor differences. Judgment Insight Ashton 1 referred to "judgment insight" as the extent of agreement between the subjects subjective description of the manner in which he weights the various questions and the s t a t i s t i c a l description of his cue-weighing process. The s t a t i s t i c a l weights are the omega-sauared s t a t i s t i c s . The subject's subjective cue weighting was assessed by having the subject allocate 100 points between the f i v e questions to r e f l e c t the importance of each auestion. This was done after the subject had evaluated a l l the possible combina-tions of yes/no answers to the f i v e Q u e s t i o n s . Subjective Weights A summary of the subject's subjective weights i s shown in Table XI. When Table XI i s compared with the appropriate average omega-squared s t a t i s t i c s in Tables VII through X, i t can be seen that there i s substantial agreement between the 52 subjective and s t a t i s t i c a l weights. The subjective weights are not d i r e c t l y comparable to the omega-squared s t a t i s t i c s because of the proportion of variance accounted for by the interactions and the error term in the analysis-of-variance. The subjects were asked only to allocate points between the f i v e questions. In reviewing Table XI, i t should be noted that the same general patterns of differences and s i m i l a r i t i e s between firms and l e v e l s of experience in Table X also occured in Table XI. There are no s t a t i s t i c a l l y s i g n i f i c a n t d i f f e r -ences between the r e s u l t s for the firms and l e v e l s of exper ience. Insight Index To assess the degree of judgment insight possessed by each individual subject, an "insight index" was computed. The insight index i s the c o r r e l a t i o n between the subjective weights and the s t a t i s t i c a l weight of the f i v e main e f f e c t s for each subject. The s t a t i s t i c a l weights for the f i v e main eff e c t s were normalized for each subject before co r r e l a t i n g them with the subjective weights. The insight indexes for each subject are shown in Appendix 2. The o v e r a l l mean co r r e l a t i o n i s 0.80. With the exception of one subject whose co r r e l a t i o n was -0.28, a l l correlations were 0.50 or greater, and 83% of a l l subjects with correlations greater than 0.70. Table XII summarizes the insight indexes by firm and by 53 l e v e l of experience. With the exception of firm 3, the firms are not sub s t a n t i a l l y d i f f e r e n t from each other. If the subject with the -0.28 co r r e l a t i o n i s eliminated, the insight index for firm 3 would be 0.79, which would not be v a s t l y d i f f e r e n t from the other firms. There i s a substan-t i a l d ifference between l e v e l s of experience. As one would expect the more experience group has a higher c o r r e l a t i o n than the less experienced group. However, i t should be noted that there are no s t a t i s t i c a l l y s i g n i f i c a n t d i f f e r e n -ces between res u l t s for the firms and l e v e l s of experience. Judgment Consistency Only one type of judgment consistency was investigated by t h i s experiment as discussed in Chapters I and I I . Consensus was defined as the consistency the subjects at the same point in time using the same data. The correlations between the responses of a l l pairs of subjects serve as an estimate of the consensus between t h e i r judgments. This resulted in 666 correlations for the 36 subjects. Average co r r e l a t i o n c o e f f i c i e n t s were computed for each firm and each experience l e v e l ; these are presented in Tables XIII and XIV. In Table XIII, the matrix of correlations represents the mean co r r e l a t i o n between a l l the subjects i s one firm with a l l the subjects in another, or the same firm. In Table XIV, the matrix represents the mean cor r e l a t i o n between a l l the subjects of one experience l e v e l and a l l subjects at another, or the same experience l e v e l s . 54 From Table XIII, i t can be discerned that there i s a high l e v e l of consensus within and between firms, with the possible exception of firm 5. There i s no s t a t i s t i c a l l y s i g n i f i c a n t difference between firms. Of more sign i f i c a n c e are the r e s u l t s shown in Table XIV. The more experienced auditors have a s i g n i f i c a n t l y higher c o r r e l a t i o n with t h e i r peers than does the less experienced group. The l e v e l s of experience are s t a t i s t i c a l l y s i g n i f i c a n t at the .133 l e v e l . Cognitive Complexity The Paragraph Completion Test as described in Chapter II was completed by the subjects p r i o r to evaluating the case as to quality of the internal control. The responses 2 were evaluated according to the established standards by an expert marker. Five of the subjects responses were randomly selected and independently evaluated by another, expert marker; the c o r r e l a t i o n between the two evaluations was 0.95. Of the 36 subjects, only two were considered by the expert markers to have shown evidence of cognitive com-pl e x i t y . Table XV shows the frequency d i s t r i b u t i o n of the conceptual complexity score l e v e l s . Such scores are 3 normally uniformly d i s t r i b u t e d on a scale from 1.0 to 7.0. The scores here are skewed toward the concrete side of the scale. The Kolmogorov-Smirnov two-sample test was applied to determine i f there was a s i g n i f i c a n t difference between the d i s t r i b u t i o n of the responses to the i n t e r n a l c o n t r o l 55 questions for each of the 32 possible combinations of yes/no answers for the two groups, the cognitive complex ones ("abstracts") and the cognitive simple ("concretes"). There were no s i g n i f i c a n t difference between the d i s t r i b u t i o n of responses for the two groups. Of the 32 Z - s t a t i s t i c s , the best possible case for s i g n i f i c a n t differences between the two groups had a p r o b a b i l i t y of 0.218 that a large Kolmogorov-Smirnov Z - s t a t i s t i c would be observed. It i s therefore concluded that the data does not provide convinc-ing evidence in support of differences in the d i s t r i b u t i o n of the responses for each of the 32 possible sets of yes/no answers. When the analysis-of-variance model Y = V b l X l + b2 X2 + b3 X3 + b 4 X 4 + b5 X5 + b6 X6 where Y represents the subjects responses to the f i v e questions represented by the categorial variables to X,. and Xc i s a categorized variable representing the cognitive complex/simple subjects, was evaluated, a l l variables were s t a t i s t i c a l l y s i g n i f i c a n t at the 0.0001 l e v e l . The two cognitive complex subjects each had omega-squared s t a t i s t i c s for one or more interactions greater than one or more of the omega-squared s t a t i s t i c s for the main e f f e c t s . This indicates possible c o n f i g u r a l i t y in these subject's judg-ments . 56 Summary This chapter contains the re s u l t s of the experiment. The four major issues that constitute what were analyzed i n d i v i d u a l l y . The e f f e c t s of firm a f f i l i a t i o n and le v e l of experience were d i s c u s s e d . The major r e s u l t s of the experiment can be summarized as follows: 1. On average, 80% of the varian c e of judgment was accounted for by the main e f f e c t s , the questions to be answered. 2. The subjects subjective descriptions of the extent to which they r e l i e d on the various internal control questions showed s u b s t a n t i a l agreement with the s t a t i s t i c a l descriptions' of t h e i r judgments. 3. The subjects showed a moderately high l e v e l of co r r e l a -tion (.70) between t h e i r judgments. 4. D i f f e r e n c e s between f i r m a f f i l i a t i o n and l e v e l of experience were shown. 5. Differences in the responses between cognitive complex (abstract) and cognitive simple (concrete) subjects were shown to e x i s t . Table VII - Relative importance of main ef f e c t s and interactions to the auditors ( f u l l model) Main e f f e c t No. of auditors Average Percentage or for whom Total Variance interaction s i g n i f i c a n t * Accounted for 1 22 31.92 2 19 14.60 3 1 8 11.35 4 21 1 6. 34 5 14 5.78 1,2 5 1 .45 1,3 6 1.83 1,4 2 1 . 20 1,5 2 1.10 2,3 2 0. 57 2,4 4 0.60 2,5 4 0. 60 3,4 4 0.87 3,5 4 0. 50 4,5 1 0.53 1,2,3 3 0.41 1,2,4 3 0.80 1,2,5 3 0. 66 1,3,4 3 0.49 1,4,5 0 0.43 1,3,5 5 0.98 2,3,4 5 0. 80 2,4,5 1 0.40 2,3,5 2 0. 74 3,4,5 5 0.7.6 1,2,3,4 2 0.55 1,2,3,5 1 0.66 1,2,4,5 1 0.46 1,3,4,5 3 0.51 2,3,4,5 4 1 . 03 1,2,3,4,5 0 0.56 * At the .005 le v e l of si g n i f i c a n c e . 58 Table VIII - Relative importance of main e f f e c t s and two way interactions to the auditors (one half f r a c t i o n a l model) Main ef f e c t No. of auditors Average Percentage of or for whom Total Variance interaction s i g n i f i c a n t * Accounted for 1 24 33.31 2 19 17.19 3 1 7 1 0.83 4 22 16.17 5 16 5.74 1,2 4 2. 62 1,3 6 2.99 1,4 4 1 . 57 1,5 7 1.61 2,3 3 0. 80 2,4 5 1.12 2,5 8 1 . 78 3,4 6 2.1 1 3,5 2 1.19 4,5 0 0.98 at the .005 l e v e l of significance Table IX - Relative importance of main ef f e c t s and interactions to the auditors by firm ( f u l l model) Main e f f e c t or Firm Firm Firm Firm Firm interaction 1 2 3 4 5 1 2 3 4 5 32. 78 1 2.07 13. 24 1 6.60 7.31 31.15 5.23 1 .48 1 8.22 6. 77 28.49 1 8.39 1 1. 73 14.29 5. 78 22. 60 8.88 1 2. 65 30.1 1 4.48 46. 92 15.11 3.44 8.01 0. 72 1,2 1,3 1 ,4 1,5 2,3 2,4 2,5 3,4 3,5 4,5 1,2,3 1,2,4 1,2,5 1,3,4 1,4,5 1,3,5 2,3,4 2,4,5 2,3,5 3,4,5 0.74 1.61 0.35 1.01 0.39 0. 70 0.98 0.77 0.76 0.61 0.1 8 0. 32 0.86 0. 30 0.31 1 . 56 0.89 0. 50 0.34 0. 63 0.93 0. 73 2.39 1 . 73 0.72 0. 76 0.37 0.71 0.19 0. 63 0.59 0. 87 0.44 0. 54 0.32 0.31 0.46 0.37 0.68 0. 58 1 .79 3.37 0.88 0. 66 0.74 1 . 66 0.22 1 . 06 0.1 3 0. 58 0.51 1 .28 0.52 0. 55 0.52 0. 50 0.58 0. 50 0.81 0. 90 1.57 0. 75 1.01 0. 57 0.57 2. 17 0.48 0. 55 0.84 0. 20 0.85 0. 94 0.94 0. 85 0.14 0.32 2.81 0.05 0.29 1.14 3.37 0. 70 2.99 2.07 0.43 0.39 0.95 1.13 0.93 0. 23 0.21 0. 76 0.60 0. 55 0.91 2.06 0.1 6 0. 10 2.1 9 0. 76 1,2,3,4 0.85 0.56 0.35 0.05 0.60 1,2,4,5 1 . 03 0.13 0.31 0. 56 1.41 1,3,4,5 0. 50 0.78 0.42 0.14 0.21 2,3,4,5 0.36 0.33 0. 86 0.42 0.32 1,2,3,4,5 0.78 0.62 1.24 2.1 1 1.02 error 0. 68 0.41 0.38 0. 92 0. 67 60 Table X - Relative importance of main e f f e c t s and two way interactions to the auditors by l e v e l of experience Main Effect Less than Over or 3 years 3 years interaction 1 29. 71 34. 13 2 13.26 1 5.94 3 1 1. 58 11.11 4 16.81 15.86 5 6. 38 5. 18 1,2 1.92 0.98 1,3 1.26 2.40 1,4 0.81 1.59 1,5 1 . 37 0. 84 2,3 0.64 0.50 2,4 1.19 0. 99 2,5 0.84 0.37 3,4 0. 84 0. 90 3,5 0.72 0.28 4,5 0. 57 0.49 1,2,3 0.22 0.60 1,2,4 0. 92 0.69 1,2,5 0.89 0.44 1,3,4 0.47 0. 52 1,4,5 0.61 0.25 1,3,5 1 . 42 0. 54 2,3,4 0.94 0.67 2,4,5 0.61 0. 18 2,3,5 1 .20 0.29 3,4,5 0. 79 0. 74 1,2,3,4 0.65 0.46 1,2,4,5 0.57 0. 75 1,3,4,5 0.57 0.35 2,3,4,5 0. 63 0.38 1,2,3,4,5 • 1.09 0.97 error 0. 53 0. 60 61 Table XI - Values of subjective weights by firm and experience l e v e l Questions Number of Firm 1 2 3 4 5 Subjec 1 30. 25 18. ,54 17. ,88 18. ,75 14. ,58 1 2 2 29. 83 19. ,67 14. , 17 23. , 17 13. , 16 6 3 27. 27 20. ,73 18. ,64 23. ,64 9. ,72 1 1 4 28. 33 16. , 67 13. , 33 28. , 33 13. , 37 3 5 37. 75 22. ,25 17. ,25 18. ,75 4. ,00 4 Level of experience less than 3 years 30. 22 21 . . 75 17. , 14 21 . , 39 9. , 50 1 8 over 3 years 29. 66 1 7. , 55 16. , 94 22. , 17 13. , 68 1 8 overal1 29. 94 19. .65 17. ,04 21 . ,78 1 1 . ,59 36 62 Table XII - Insight index by firm a f f i l i a t i o n and l e v e l of experience Firm Insight index Number of Subjects 1 0.86 12 2 0.84 6 3 0.69 11 4 0.80 3 5 0.83 4 lev e l of experience less than 3 years 0.75 18 more than 3 years 0.84 18 overal1 0.80 36 63 Table XIII - Judgment consensus among auditors in same and d i f f e r e n t firms Firm 1 2 3 4 5 1 .72 .73 .71 .72 .69 2 .74 .72 .73 .68 3 .67 .70 .65 4 .70 .63 5 .62 Average Consensus Index with same with other Number of Firm Firm Firms Subjects 1 .72 .71 12 2 .74 .72 6 3 .67 .70 11 4 .70 .70 3 5 .62 .67 4 64 Table XIV - Judgment consensus among auditors in same experience l e v e l s Experience l e v e l Experience l e v e l less than more than 3 years 3 years Number of Subjects less than 3 years . 6 0 . 7 0 1 8 more than 3 years . 8 0 1 8 TABLE XV - Frequency d i s t r i b u t i o n for conceptual complexity score l e v e l s SCORE 1.0 IXXX 1.1 Ixxxx 1.2 Ixx 1.3 | 1. 4 1 xxx 1.5 I xxxxxxxx 1.6 Ixx 1.7 IX 1.8 Ixx 1.9 1 2.0 Ix 2.1 Ix 2.2 Ix 2.3 Ixxx 2.4 1 2.5 Ixx 2.6 Ix 2.7 1 2.8 1 2.9 1 3.0 Ix 3.1 1 3.2 1 3.3 1 3.4 1 x 66 FOOTNOTES 1 Ashton, 1973, p. 202. 2 Schroder, Driver and Steufert, pp. 185-204. 3 Ibid. 67 CHAPTER IV CONCLUSIONS Summary of the Problem This thesis has been concerned with the consistency of auditor judgments. Additional concerns were differences in judgments due to firm a f f i l i a t i o n and experience, and the a b i l i t y to d i f f e r e n t i a t e and integrate information. In addition, the usefulness of f r a c t i o n a l f a c t o r i a l designs in human judgment research was to be evaluated. A laboratory experiment was conducted to determine the extent of judgment consistency between auditors, firms and le v e l s of experience. The experiment culminated in auditor judgments based on r e l a t i v e , hypothetical findings with respect to fi v e d i f f e r e n t internal control auestions. In addition, the a b i l i t y of the subjects to recognize the extent to which they u t i l i z e d the various questions was assessed. The author constructed a case depicting a hypothetical s i t u a t i o n involving the internal control over sales and inventory in a business organization. In addition to some background information concerning the firm and the account-ing information system, the su b j e c t s were given f i v e i n t e r n a l c o n t r o l questions to be used i n judging the adequacy of the internal control over the sales and i n -ventory system. Each of the fi v e questions could assume two 68 states: adeguate or inadequate control of the s p e c i f i c aspect of i n t e r n a l c o n t r o l . The s u b j e c t s judged the adequacy of the internal control as a function of the 32 possible combinations of the two states. The subjects consisted of 36 auditors from f i v e public accounting firms in the Vancouver, B.C. area. They each held between one and ten years of p r a c t i c a l experience. Summary of the Experimental Results The r e s u l t s of t h i s experiment may be considered applicable only to the task and the individuals involved. Care should be taken in attempting to generalize these r e s u l t s . One of the more s i g n i f i c a n t findings of t h i s research emerges as an assessment of the extent to which the sub-je c t s ' judgments were influenced by any one of the f i v e internal control questions. The f i r s t Q u estion, namely, the control.over the completeness of processing, was considered more important in t h e i r judgments than the other four quest ions. D i f f e r e n c e s in r e l i a n c e upon p a r t i c u l a r i n d i c a t o r s (although in most cases not s t a t i s t i c a l l y s i g n i f i c a n t ) were found to exist among auditors from d i f f e r e n t firms and again among a u d i t o r s with d i f f e r e n t l e v e l s of experience. Auditors with more experience placed more reliance on the f i r s t indicator of completeness of processing than did the less experienced group of subjects. 69 The f i v e main effects accounted for, on average, 80 per cent of the judgment variance. This indicates that the f i v e questions used in the experiment were used almost indepen-dently of each other by the subjects in making t h e i r j udgments. The extent of agreement between s t a t i s t i c a l weights (the omega-squared s t a t i s t i c s ) and subjective weights was assessed and i t was found to be high. The average co r r e l a -t i o n s f o r the more experienced a u d i t o r s and the l e s s experienced auditors were 0.84 and 0.75, respectively. The extent of agreement among the judgments of the subjects, referred to as "consensus", was also found to be high. The correlations between the more experienced group and the less experienced group were 0.80 and 0.60, respec-t i v e l y . The c o r r e l a t i o n between firms averaged between 0.69 and 0.86. The use of the Paragraph Completion Test showed that only two of the 36 subjects could be considered to be c o g n i t i v e l y complex ( a b s t r a c t ) . The Kolmogrov-Smirnov two-sample tests showed no differences in the d i s t r i b u t i o n of responses to the 32 yes/no answers between the two abstract subjects and the 34 concrete subjects. On the other hand, application of a multiple regression model as described in Chapters II and III displayed a s t a t i s t i c a l l y s i g n i f i c a n t difference in the responses between the two groups. 70 The r e s u l t s of the f u l l f r a c t i o n a l f a c t o r i a l model and the one-half f r a c t i o n a l f a c t o r i a l model were reported in Chapter II I . A comparison of the two sets of r e s u l t s indicates that only a few aliases W P T P included in the one-half f a c t o r i a l r e s u l t s . No more than two s i g n i f i c a n t main or interaction e f f e c t s were aliased by one of the one-half f r a c t i o n a l f a c t o r i a l model's e f f e c t s . Implications of the Experiment's Results The most important implication of t h i s research refer to the cognitive complexity portion of the experiment. The subjects have displayed almost l i n e a r models of decision making indicating r e l a t i v e l y simple processes of judqment. Since the r e s u l t s of the Paragraph Completion Test indicate that the subjects are in fact mostly concrete in t h e i r cognitive processes, r e l a t i v e l y simple models of judgment may be expected to apply. The second implication of t h i s research concerns the adequacy of the f r a c t i o n a l f a c t o r i a l model in human judqment research. In this experiment, few a l i a s e s were found to occur when the one-half f r a c t i o n a l f a c t o r i a l model was used. This implies that f r a c t i o n a l f a c t o r i a l models can form useful tools in t h i s area of research. This conclusion, however, i s drawn from a r e l a t i v e l y small sample, using an experiment on a specialized topic, and u t i l i z i n g subjects trained or being trained in the subject matter. 71 A t h i r d implication of t h i s research should be of interest to managing partners of public accounting firms. Although there was a high o v e r a l l c o r r e l a t i o n between judgments by the subjects, a problem of inconsistency in judgments does, in f a c t , e x i s t . Public accounting firms, in applying experiments s i m i l a r to that employed in t h i s research may provide feedback to their s t a f f on the r e l a t i v e c o n s i s t e n c y and i n c o n s i s t e n c y of t h e i r judgments. In addition, these experiments may be used to determine the r e l a t i v e importance of cues, such as the questions used in the experiment, for similar purposes. Implications for Further Research This thesis has addressed a research method for the description of judgments by a group of s p e c i a l i s t s . It i s proposed that the experiment be r e p l i c a t e d , using an even larger sample, to test the re s u l t s of t h i s experiment for sets of both similar and d i s s i m i l a r cues. This research has shown the problem of aliases to be i n s i g n i f i c a n t . Therefore, the use of f r a c t i o n a l f a c t o r i a l designs may be found appropriate in order to test large numbers of cues. The experiments to date have used small numbers of cues (five to eight). To overcome t h i s con-s t r a i n t , an experiment of th i s type may well be conducted in parts to reduce the impact of the time factor on subject response l e v e l s . Each part could consist of a complete set of questions on a relevant audit area such as sales and 72 accounts receivable. Ashton showed that there was l i t t l e variance in the subject decisions over time, thus making the administration of such a multi-phase experiment a viable proposition. This approach may also provide information on the usefulness of an experiment on human information processing in environments characterized by the presence of a large number of cues. The methodology can also be applied to other areas of judgment research. It i s proposed that an experiment evaluating cue u t i l i z a t i o n by production managers could be conducted to determine the ordering or sequence of cue presentation in a management information system. Research of t h i s type has already been completed in the f i e l d s of medical c l i n i c a l judgment and in stockbroker d e c i s i o n -making . ^  However, the approach of a highly structured experiment studying audit judgments is s t i l l cuestionable. The audit process i s a somewhat unstructured a c t i v i t y . What i s suggested here i s the need for a more n a t u r a l i s t i c approach to studying the audit judgment process. A n a t u r a l i s t i c approach might supply subjects with comprehensive l i s t s of information needs, which could be reviewed by the subjects and explanations of why p a r t i c u l a r pieces of information are required be obtained. Yet another approach would be to simulate the audit process supplying the subjects informa-tion as i t was requested. 1 Ashton, op. c i t . 2 S l o v i c , op. c i t . 73 FOOTNOTES 74 BIBLIOGRAPHY Aly, H.F., and Duboff, J.I., " S t a t i s t i c a l vs. Judgmental Sampling: An Empirical Study of Auditing Accounts Receiv-able of a Small Retail Store," The Accounting Review, 1971, pp. 119-128. Report of the 1976-77 committee on human information pro-cessing , American Accounting Association, August 197 7. Ashton, Robert H., Judgment formation in e v a l u a t i o n of i n t e r n a l c o n t r o l : an a p p l i c a t i o n of Brunswik's lens model, unpublished PhD d i s s e r a t i o n , University of Minne-sota, 1 973. Ashton, Robert H., "An Experimental Study of I n t e r n a l Control Judgments," Journa l of Accounting Research , Spring 1 974, pp. 143-1 57^ Ashton, Robert H., "Cue u t i l i z a t i o n and expert judgments: a comparison of independent auditors with other judges", Journal of Applied Psychology, Vol. 59, No. 4, 1974, pp. 437-444. Ashton, Robert H. and Brown, Paul R. , Descriptive Modeling of A u d i t o r ' Judgments, Working Paper 79-6, Graduate School of Businesss, University of Texas at Austin, 1979. Ashton, Robert H. and Kramer, Sandra S., Students as Surro- gates in Behavioral Research: Some Evidence, unpublished man user ipt~J November 19/8. B a r i f f , M.L. and Lusk, E.J., "Cognitive and personality tests for the design of management information systems", Management Science, A p r i l 1977, pp. 820-829. Barrett, M.J., Cognitive s t y l e : an overview of the devel-opmental phase of a decision process research program^ unpublished working paper, University of Minnesota, 1978. Beal, Don; G i l l i s , John S. and Stewart, Tom, "The lens model: computational procedures and a p p l i c a t i o n s , " Perceptual and Motor S k i l l s , 1978, pp. 3-28. Benbasat, Izak and Dexter, A l b e r t S., Value and Events  Approaches to Accounting, Working paper No. 488, Univer-s i t y of B r i t i s h Columbia, Faculty of Commerce, A p r i l 1978. Benbasat, Izak and Schroeder, Roger S., "An Expermental I n v e s t i g a t i o n of Some MIS v a r i a b l e s , " MIS Q u a r t e r l y , March 1977, pp. 37-49. 75 Benbasat, Izak and Taylor, Ronald N., "The impact of cogni-tive styles on information system design", MIS Quarterly, June 1978, pp. 43-54. B i e r i , J. , "Cognitive complexity and judgment of consistent information", in R.P. Abelson, E. Aronson, W.J. McGuire, T.M. Newcomb, M.J. Rosenburg, and P.E. Tannenbaum (Eds.), Theories of cognitive complexity: a sourcebook, Chicago: Rand McNally, 1968. Brunswik, Egon, The conceptual framework of psychology, International Encyclopedia of Unified Science, Vol. 1, No. 10, Chicago: University of Chicago Press, 1952. C a r l i s l e , J . , C o g n i t i v e F a c t o r s in I n t e r a c t i v e D e c i s i o n  Systems, PhD di s s e r a t i b n , Yale University, 1974. Cervancy, N.L. and Dickson, G.W., "An Experimental Evalua-t i o n of Information Overload in a Production Environ-ment," Management Science, June, 1974, pp. 1335-1344. Canadian Institue of Chartered Accountants, CICA handbook, 1 979. Canadian Institue of Chartered Accountants, "Competance and Professsional Development in EDP for the CA: A Report by the CA Computer Course Study Group", CA Magazine, Vol 105, No. 3, pp. 26-70. Cochran, William G. and Cox, Gertrude, Experimental Design, New York: John Wiley & Sons, Inc., 1963. Committee on Auditing Procedure, A Case Study of the Extent of Audit Samples, New York: American I n s t i t u t e of Accountants, 1955. Committee on Human Information Processing of the American Accounting Association, Report of the Committee, August 1977. Corless, J.C., "Assessing P r i o r Distributions for Applying Bayesian S t a t i s t i c s in Auditing," The Accounting Review, 1972, pp. 556-566. Dermer, Jerry D. , "Cognitive Characteristics and the Per-ceived Importance of Information," The Accounting Review, July 1973, pp. 511-519. Dickhaut, John W. and Ronen, Joshua, "Using Selected human performance variables to examine behavior with accounting information systems," Behavioral Experiments in Account- ing , Thomas J. Burns, ed., Columbus, Ohio: The Ohio State University, 1972, pp. 139-219. 76 Dickson, G.W., "Management Information Systems: D e f i n i t i o n s , Problems and Research," SMIS Newsletter, Vol. 1, 1971, pp. 6-12. Doktor, R.H. and Hamilton, W.T., "Cognitive Style and the Acceptance of Management Science Recommendations," Management Science, A p r i l 1973, pp. 884-894. Driver, Michael J . , and Mock, Theodore J.,"Human Information Processing, Decision Style Theory, and Accounting Infor-mation Systems", The Accounting Review, July 1975, pp. 490-508. Gardiner, Gareth S. , and Schroder, Harold M., " R e l i a b i l i t y and v a l i d i t y of the paragraph completion t e s t : theor-e t i c a l and empirical notes," Pscyhological Reports, 1972, pp. 959-962. Gibbins, Michael "A Behavioural Approach to Auditing Research", Symposium on Auditing Research I I , University of I l l i n o i s at Urbana-Champaign, 1977. Gibbins, Michael, "Human inference, h e u r i s t i c s and auditors' judgment processes," CICA Auditing Research Symposium, November 1978, pp. 5A1-5A29. Hamilton, R.E. and Wright, W.F., The Evaluation of Internal  Control over P a y r o l l , Research Paper No 397, Graduate School of Business, Stanford University, December 1977. Hammond, Kenneth R. , " P r o b a l i s t i c Functioning and the C l i n i c a l Method," Psychological Review, July 1965, pp. 225-262. Harvey, O.J., Hunt, D.E. and Schroder, H.M., Conceptual  systems and personality structure, New York: John Wiley & Sons, 1961. Hays, William L. , S t a t i s t i c s for psychologists, New York: Holt, Rinehart and Whinston, 1963. Hoffman, Paul J . , S l o v i c , Paul, and Rorer, Leonard G.," An analysis-of-variance model for the assessment of con-f i g u r a l cue u t i l i z a t i o n in c l i n i c a l judgment", Psycho- l o g i c a l B u l l e t i n , Vol. 69, No. 5, 1 968, pp. 338-349~T Huysmans, J.H.B.M., The Implementation of Operations Re- search , New York: Wiley-Interscience, f970. Joyce, Edward J . , "Expert Judgment in Audit Program Plan-ning," Journal of Accounting Research, Vol. 14, Supple-ment, 1976, pp. 29-60. 77 Keen, Peter G.W., The implications of cognitive s t y l e for individual decision-making, doctoral d i s s e r a t i o n , Harvard University, 1973. Lindman, Harold R. , Analysis of Variance in Complex Experi- mental Designs, San Francisco: W.H. Freeman and Company, 1974. Long, Barabra H. and Z i l l e r Robert C.,"Dogmatism and pre-d e c i s i o n a l information search," J o u r n a l of Ap p l i e d  Psychology, Vol. 49, No. 5, pp. 376-37ls\ Lusk, E.J., "Cognitive Aspects of Annual Reports: F i e l d Indepence/Dependence," Empirical Research in Accounting:  Selected Studies, 1973, pp. 191-20 2. Mason, Richard 0. and Mitr o f f , Ian I., "A program for research on management information systems," Management  Science, January 1973, pp. 475-487. McGhee, Walter; Shields, Michael D.,and Birnberg, Jacob B. , "The Effects of Personality on a Subject's Information P r o c e s s i n g " , The Accounting Review, J u l y 1978, pp. 681-697. McKenney, James L. and Ken, Peter G.W., "How managers' minds work," Harvard Business Review, May-June 1974, pp. 79-90. Newell, A., and Simon, H.A., Human Problem Solving, Engle-wood C l i f f s , NJ: Prentice H a l l , Inc., 1972. Nunnally, Num C , Psychometric theory, Toronto: McGraw-Hill Book Company, 1967. Oppenheim, A. N. , Questionaire design and attitude measure- ment , New York: Basic Books, Inc., 1966. Savich, Richard S., "The Use of Accounting Information in Decision Making", The Accounting Review, July 1977, pp. 642-652. S c o t t , W i l l i a m A., "Cog n i t i v e Complexity and C o g n i t i v e F l e x i b i l i t y , " Sociometry, 1962, Vol. 25, pp. 405-414. Schroder Harold M., "Conceptual complexity and personality organization," in H.M. Schroder and P. Suedfeld (Eds.), Personality theory and information processing, New York: Ronald Press, 1971. Schroder, Harold M., Driver, Michael J . , and Streufert, S i e g f r i e d , Human Information Processing: Individuals and  Groups Functioning in Complex Social Situations, Toronto: Holt, Rinehart and Winston, Inc., 196T~. 78 Schroder, Harold M.,and Suedfeld, Peter, Personality Theory  and Information Processing, New York: The Ronald Company, 1971. Siegel, Sidney, Nonparametric s t a t i s t i c s for the behavioral  sciences, New York: McGraw-Hill, 1956. S l o v i c , Paul, "Analyzing the expert judge: a descriptive study of a stockbroker's decision processes", Journal of  Applied Psychology, August 1969, pp. 225-263. S l o v i c , Paul, "Cue-consistency and c u e - u t i l i z a t i o n i n judgement," American J o u r n a l of Psychology, V o l . 70, 1966, pp. 427-434. Swieringa, Robert J. , "A behavioral approach to internal control evaluation," The Internal Auditor, March/April 1972, pp. 30-45. Swieringa, R.J., Gibbins, M. , Larsson, L. , and Sweeney, J.L., "Experiments in the Heuristics of Human Information P r o c e s s i n g , " J o u r n a l of Accounting Research, V o l . 4, Supplement, 1976, pp. T5"9-195. Taylor, R.N., "Psychological determinants of bounded ration-a l i t y : i m p l i c a t i o n s f o r decision-making s t r a t e g i e s " , Decision Sciences, Vol. 6, No. 3, July 1975, pp. 409-429. Tripdodi, Tony, and B i e r i , James, "Cognitive complexity, preceived c o n f l i c t , and certainty," Journal of Person- a l i t y , 1966, pp. 144-152. Vannoy, Joseph S . , " G e n e r a l i t y of c o g n i t i v e complexity-s i m p l i c i t y as a p e r s o n a l i t y c o n s t r u c t , " J o u r n a l of  Personality and So c i a l Psychology, 1965, pp. 385-396. W i l l , H.J., "Auditing in Systems Perspective", The Account- ing Review, October 1974, pp. 690-706. Witkin, H.A., Dyk, R.G., Patterson, H.F., Goodenough, D.R. and Karp, S.A., Psychological D i f f e r e n t i a t i o n , New York: John Wiley & Sons, 1972. 7 9 APPENDIX 1 CASE STUDY You are the auditor in charge of the year-end audit of the Umpqua Valley Company, Ltd. The purpose of your audit i s to issue a standard short-form audit report. Your firm has performed the annual audit for the past several years, but t h i s i s the f i r s t year that you have been in charge of the f i e l d work. Your firm issued an unqualified opinion l a s t year. The Company i s a wholesale o f f i c e supply and furniture operation. It i s a p u b l i c l y held company with about 1,500 stockholders. During the year under review, sales were about $51 m i l l i o n , an increase of $3 m i l l i o n over the previous year. You have decided that your i n v e s t i g a t i o n of the company's i n t e r n a l c o n t r o l system should be undertaken before you determine the types and extent of audit pro-cedures which should be applied. In reviewing the int e r n a l control, your firm attempts to avoid needless duplication and to obtain better coverage of the system through coord-ination with the a c t i v i t i e s of the internal auditor in those areas that he or she reviews. You have several assistants who w i l l review the existing internal control and the operation of that system in the various areas of the company's a c t i v i t e s ; f o r i n s t a n c e , one a s s i s t a n t w i l l i n v e s t i g a t e the i n t e r n a l c o n t r o l s p e r t a i n i n g to cash receipts; another w i l l review the controls over p a y r o l l ; s t i l l another w i l l review the controls over cash disburse-ments, etc. In conducting these reviews, your assistants w i l l use an internal control questionnaire for each area. You w i l l review the results of each questionnaire; based upon your e v a l u a t i o n of the strength of the e x i s t i n g controls, you w i l l prescribe the audit procedures to be applied in each area. In this experiment you are concerned only with the internal controls over the sales/inventory system. A l l f i n a n c i a l recordkeeping has been computerized. The company uses an IBM System/3 computer l o c a t e d at i t s headquarters. The company i n s t a l l e d a new sales/inventory system this year as part of the continuing plan of upgrading a l l data processing. This system allows sale personnel in the f i e l d to telephone the headquarters sales s t a f f who w i l l enter the orders on-line. This allows the f i e l d personnel to know immediately which items can be shipped and which must be backordered. When the order i s acceptable to the f i e l d personnel, the system updates the accounts receivable and inventory master f i e l d s and generates invoices, backorders, shipping documents and picking l i s t s . For the purposes of t h i s experiment, according to your firm's policy you instructed your assistant to examine f i v e internal control, factors pertaining to the sales/inventory system. Each factor i s in the form of a question which your 8 0 assistant has answered by "yes" or "no." These f i v e ques-t i o n s are summary e v a u l a t i o n s of the i n t e r n a l c o n t r o l q u e s t i o n a i r e s . A f t e r c o n s i d e r i n g the answers to these questions you w i l l r a t e the strength of the i n t e r n a l controls on a scale from 1 (extermely weak) to 6 (adequate to strong). In the experiment you are asked to make 32 such ratings, one for each of 32 possible sets of answers which your assistant might bring to you. You should indicate your rating in each case by c i r c l i n g the appropriate number on the scale from 1 to 6.) 81 APPENDIX 2 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 35. 28 35. 50 22. 92 10.21 2 7.29 1 3.45 22.92 37.76 3 18.66 1 .89 4. 73 13.60 4 7.29 21.01 1 8.94 2.96 5 14.29 1 7. 02 12.12 4. 89 1,2 2.62 0.21 0.76 1.51 1,3 1.17 0. 84 0. 00 2.96 1,4 0.29 0.21 0.19 0.06 1,5 0. 29 0. 84 4. 73 0. 54 2,3 0.00 0.21 0.00 0.06 2,4 2. 62 0.00 0. 1 9 2. 96 2,5 0.29 1.89 1 .70 1.51 3,4 1.17 0.21 0. 19 0. 06 3,5 0.00 0.84 1.70 0.54 4,5 0. 29 0.21 0. 76 0. 54 1,2,3 0.00 0.00 0.1 9 0.54 1,2,4 0.29 0.21 0. 76 0. 54 1,2,5 0.29 0.84 0.76 0.54 1,3,4 0. 00 0.00 0. 76 0. 06 1,4,5 0.29 0.84 0.19 0.06 1,3,5 1.17 0.21 0. 00 2. 96 2,3,4 1.17 0.21 0.00 0.54 2,4,5 0. 29 0.21 0. 19 0. 54 2,3,5 0.00 0.84 0.00 0.54 3,4,5 1.17 0.00 0. 19 1 .51 1,2,3,4 1.17 0.84 1.70 0.06 1,2,3,5 0.00 0.21 0. 19 7.31 1,2,4,5 0.29 0.84 0.76 1.51 1,3,4,5 0. 00 0.21 0. 76 0. 06 2,3,4,5 1.17 0.00 0.00 0.06 1,2,3,4,5 1.17 0.21 1. 70 2.96 t o t a l 100.00 100.00 100.00 100.00 82 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 60. 52 46. 61 33. 43 57. 14 2 5.91 2.58 21.91 8.04 3 3. 78 36. 29 9. 17 14. 29 4 1 1.58 1.45 3.71 0.89 5 8.51 2. 58 0. 08 8. 04 1,2 0.00 0.16 0.08 0.89 1,3 0.24 2. 58 6.14 0. 89 1,4 0.00 0.00 0.08 0.89 1,5 0. 24 0.16 0. 68 0.89 2,3 0.00 0.1 6 1.90 0.89 2,4 0. 24 0.16 1 . 90 0. 00 2,5 0.95 0.00 0.68 0.89 3,4 0. 00 2. 58 0.08 0.89 3,5 0.24 0.1 6 1.90 0.00 4,5 0. 95 0.16 0.08 0. 00 1,2,3 0.24 0.00 0.08 0.00 1,2,4 0. 95 0. 00 0.68 0.00 1,2,5 0.24 1.45 3.71 0.89 1,3,4 0. 24 0. 16 0. 68 0. 00 1,4,5 0.24 0.00 0.68 0.00 1,3,5 0. 00 0. 65 0. 68 0.89 2,3,4 0.95 0.65 0.08 0.00 2,4,5 0.00 0. 16 1 . 90 0. 00 2,3,5 0.24 0.1 6 1.90 0.00 3,4,5 0. 24 0.00 1.90 0.89 1,2,3,4 . 0.24 0.16 1.90 0.89 1,2,3,5 0.00 0.0 1 .90 0.89 1,2,4,5 0.24 0.65 0.08 0.00 1,3,4,5 0. 00 0.16 0. 68 0.00 2,3,4,5 2.13 0.00 0.68 0.8 9 1,2,3,4,5 0. 95 0. 16 0. 68 0. 00 t o t a l 100.00 100.00 100.00 100.00 83 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 34. 43 17.16 12.89 27. 25 2 0.50 9.23 4.21 1 1.04 3 6. 67 22.04 9.47 18. 24 4 40.16 17.16 51.58 22.52 5 0. 50 9. 23 2.37 8. 1 1 1,2 1.38 0.08 0.26 0.90 1,3 2. 70 0. 69 0. 26 0. 90 1,4 0.06 1.91 0.26 0.23 1,5 0. 50 1 .91 1 .05 0. 23 2,3 1.38 0.08 0.00 0.00 2,4 0. 06 0. 08 0. 00 0. 23 2,5 2.70 0.69 0.26 0.23 3,4 0. 06 3. 74 0. 00 0. 23 3,5 1.38 1.91 0.26 0.23 4,5 0. 06 1 . 91 2.37 0. 00 1,2,3 0.50 0.08 0.26 0.23 1,2,4 0. 06 0. 08 0. 26 0.00 1,2,5 0.50 0.08 1.05 0.00 1,3,4 1 .38 0. 08 0. 26 0.00 1,4,5 0.06 0.08 1.05 0.23 1,3,5 1 . 38 6.18 1 .05 3.60 2,3,4 0.50 1.91 1.05 3.60 2,4,5 0. 06 0. 08 2.37 0. 23 2,3,5 0.06 0.08 0.26 0.00 3,4,5 0.50 0.69 0. 26 0.23 1,2,3,4 0.50 0.08 2.37 0.23 1,2,3,5 0. 50 0. 08 1 . 05 0. 23 1,2,4,5 0.50 0.08 1.05 0.0 1,3,4,5 0. 50 1.91 0.0 0.0 2,3,4,5 0.50 0.69 2.37 0.90 1,2,3,4,5 0. 06 0.08 0.00 0. 23 t o t a l 100.00 100.00 100.00 100.00 84 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 51.41 16.97 44. 76 32.34 2 0.38 31.06 1 7.48 1 5.71 3 5.08 25. 89 2. 80 6.68 4 1 5.15 9.92 1 7.48 28.54 5 5.08 0.06 6.29 5. 02 1,2 1.05 0.06 0.70 0.03 1,3 1 . 05 0. 06 0. 00 0. 74 1,4 7.09 2.88 2.80 0.03 1,5 3. 40 0. 06 2. 80 1 .46 2,3 0.38 0.53 0.00 0.03 2,4 0.38 0. 06 0. 70 0. 03 2,5 0.04 0.53 0.00 0.03 3,4 3.40 0. 53 0. 00 0.27 3,5 0.04 0.53 0.00 0.27 4,5 1 . 05 1 . 47 0. 70 0.27 1,2,3 0.04 0.53 0.70 1.46 1,2,4 0. 04 0. 06 0. 70 3. 59 1,2,5 0.04 0.06 0.00 0.03 1,3,4 0. 38 0. 06 0. 00 0.27 1,4,5 0.38 0.53 0.00 0.74 1,3,5 0.04 1 . 47 0. 00 0. 03 2,3,4 0.38 0.53 0.00 0.03 2,4,5 0. 38 0. 06 0. 00 0. 27 2,3,5 0.04 0.53 0.70 0.27 3,4,5 0. 04 2. 88 0. 00 0.27 1,2,3,4 1.05 0.53 0.70 0.27 1,2,3,5 0. 38 0. 06 0. 00 0.27 1,2,4,5 0.04 1.47 0.00 0.03 1,3,4,5 0.38 0. 06 0.00 0. 74 2,3,4,5 0.38 0.06 0.70 0.03 1,2,3,4,5 1 .05 0. 53 0.00 0.27 t o t a l 100.00 100.00 100.00 100.00 85 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 7.80 33.64 29. 78 25.40 2 7.80 1 8.93 9.1 9 2.82 3 28. 43 0.00 9. 19 25. 40 4 23.27 1 4.95 23.53 1.59 5 5. 22 1 8. 93 9. 1 9 0. 18 1,2 1.61 2.10 1.47 1 .59 1,3 1 .61 0. 93 1 . 47 29.81 1,4 0.58 0.93 0.37 2.82 1,5 0. 58 2. 10 1 .47 0.00 2,3 3.1 6 0.23 0.00 1.59 2,4 3.16 0.23 0.37 0.00 2,5 1.61 0.00 0.00 0.00 3,4 0. 06 0. 00 0.37 2. 82 3,5 0.06 0.23 0.00 0.00 4,5 0. 06 0. 23 0.37 0. 18 1,2,3 0.58 0.23 0.37 2.82 1,2,4 0. 58 0. 23 1.47 0.18 1,2,5 1.61 0.93 0.37 0.1 8 1,3,4 1 .61 0. 93 1.47 1 . 59 1,4,5 0.06 0.23 1.47 0.00 1,3,5 0. 06 0. 23 0. 37 0.18 2,3,4 1.61 0.23 0.00 0.1 8 2,4,5 0. 58 0. 93 0. 00 0. 00 2,3,5 1.61 0.93 0.37 0.1 8 3,4,5 0. 06 0. 23 0. 00 0. 00 1,2,3,4 0.58 0.23 0.37 0.00 1,2,3,5 0. 06 0. 00 1.47 0. 00 1,2,4,5 3.1 6 0.00 0.37 0.1 8 1,3,4,5 0. 58 0. 23 0.37 0. 18 2,3,4,5 1.61 0.93 3.31 0.1 8 1,2,3,4,5 0. 58 0. 00 1.47 0. 00 t o t a l 100.00 100.00 100.00 100.00 86 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 50. 00 35. 59 21.05 21. 29 2 1 5.82 1 4.90 8.22 52.58 3 9.57 1 . 97 11. 84 1 .21 4 12.50 20.81 8.22 1 3.95 5 3. 13 1.11 26. 64 1.21 1,2 0.20 1.97 0.33 0.05 1,3 0.20 1.11 1. 32 0.43 1,4 0.78 1.97 0.33 1.21 1,5 0.00 4.43 0. 33 0. 05 2,3 0.00 0.12 0.33 1.21 2,4 0.20 0.49 1 . 32 1 .21 2,5 0.20 0.00 0.00 1.21 3,4 0. 20 1.11 2. 96 0.05 3,5 0.20 0.12 0.33 0.05 4,5 0. 78 0. 00 1 . 32 0. 05 1,2,3 0.00 0.00 0.33 0.43 1,2,4 1 . 76 3.08 0. 00 0.05 1,2,5 0.20 1.11 1.32 0.05 1,3,4 0. 20 0.49 0.33 0.43 1,4,5 0.00 1.11 1.32 0.05 1,3,5 0. 20 0.49 0.33 0. 43 2,3,4 0.00 0.49 1.32 0.05 2,4,5 0. 20 0. 12 2. 96 0. 05 2,3,5 0.00 0.49 1.32 0.43 3,4,5 0. 20 1 .97 1 .32 1 .21 1,2,3,4 0.78 0.12 0.00 0.43 1,2,3,5 0.00 0. 12 0.00 0.05 1,2,4,5 1.76 0.49 0.33 0.05 1,3,4,5 0. 20 3.08 1 . 32 0. 43 2,3,4,5 0.78 1.11 2.96 0.05 1,2,3,4,5 0.00 0.00 0. 33 0. 05 t o t a l 100.00 100.00 100.00 100.00 i» 87 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 16. 92 22.25 28.64 1.62 2 2.50 1 7.58 6.55 14.57 3 28.93 2.47 6. 55 19.84 4 1 6.92 39.56 33.84 14.57 5 2. 50 4.40 6. 55 6.48 1,2 0.90 2.47 1.35 10.12 1,3 0. 90 0. 00 1 .35 1 . 62 1,4 0.10 0.27 2.65 0.40 1,5 0.10 0.27 1 . 35 0.40 2,3 0.10 0.27 1.35 1.62 2,4 4.91 1.10 0.49 3.64 2,5 0.10 0.00 1.35 0.40 3,4 0. 90 0. 27 0.49 1 .62 3,5 0.90 0.27 1.35 0.00 4,5 0.10 0. 00 0.49 0.40 1,2,3 2.50 0.000 0.05 0.40 1,2,4 2. 50 0.27 0.05 6.48 1,2,5 2.50 0.27 0.05 0.00 1,3,4 2. 50 0. 00 0.05 0.40 1,4,5 0.1 0 0.27 0.05 0.00 1,3,5 0. 90 0.00 0.05 0.40 2,3,4 8.1 1 0.2 7 0.05 3.64 2,4,5 0. 10 0. 00 0.05 1 . 62 2,3,5 0.1 0 0.27 0.4 9 3.64 3,4,5 0. 90 2.47 0.05 3.64 1,2,3,4 0.1 0 0.00 0.05 0.00 1,2,3,5 0.10 1.10 0.49 0.00 1,2,4,5 0.1 0 0.27 0.05 0.40 1,3,4,5 0. 10 1.10 0.05 0.00 2,3,4,5 2.50 2.47 1.35 1.62 1,2,3,4,5 0. 10' 0.00 2. 65 0.40 t o t a l 100.00 100.00 100.00 100.00 88 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 50.71 19.47 41 . 02 17.43 2 2.59 40.63 1 9.66 21.29 3 1 1. 87 1 5. 38 1 1 . 89 10. 86 4 19.05 8.65 8.74 25.54 5 2. 59 0. 96 3. 88 8. 16 1,2 0.05 0.00 3.88 0.05 1,3 0.47 0. 24 0.00 0.43 1,4 1.32 0.24 0.24 0.05 1,5 0.05 0. 24 0. 24 0. 05 2,3 0.05 2.1 6 0.97 0.05 2,4 0.47 2. 16 0. 24 8. 16 2,5 0.05 0.24 0.24 0.05 3,4 1 . 32 0. 96 0.24 0.05 3,5 0.4 7 0.00 0.24 0.05 4,5 1 . 32 0. 96 0.97 0.05 1,2,3 0.05 0.96 0.24 0.05 1,2,4 0. 05 0. 00 0. 97 0. 05 1,2,5 0.47 0.96 0.97 0.05 1,3,4 0.47 0. 24 0. 00 0. 43 1,4,5 0.05 0.24 0.24 1.21 1,3,5 0.47 0. 24 0. 00 2.37 2,3,4 0.05 0.24 0.00 0.43 2,4,5 0. 05 0. 24 0. 24 0. 05 2,3,5 0.05 0.24 0.97 1.21 3,4,5 1 .32 0. 00 0. 24 0.05 1,2,3,4 0.47 0.96 0.24 0.43 1,2,3,5 0.47 0.96 0. 24 0. 05 1,2,4,5 0.05 0.00 0.97 0.05 1,3,4,5 2. 59 0. 24 0.97 0.05 2,3,4,5 0.47 2.16 0.97 0.05 1,2,3,4,5 0.47 0. 00 0. 24 1.21 t o t a l 100.00 100.00 100.00 100.00 89 INDIVIDUAL SUBJECT OMEGA-SQUARED RESULTS MAIN EFFECTS AND INTERACTIONS 1 14. 32 45. 73 59. 48 68. 16 2 0.89 39.84 1 9.72 0.00 3 0. 22 1 .83 9. 23 2.47 4 5.59 5.08 2.68 1 8.70 5 2.01 0.81 0.05 0. 00 1,2 1 0.96 1.83 0.05 0.62 1,3 0.89 0.81 0.49 0. 62 1,4 8.05 0.00 0.05 3.86 1,5 8. 05 0. 20 0.05 0. 00 2,3 0.89 0.20 0.49 0.1 5 2,4 0.89 0.20 0.49 0. 00 2,5 3.58 0.00 0.05 0.1 5 3,4 2.01 0. 00 0. 05 2.47 3,5 2.01 0.20 1.37 0.1 5 4,5 0. 22 0. 20 0.49 0. 00 1,2,3 0.22 0.00 0.49 0.1 5 1,2,4 0. 22 0.81 1 . 37 0. 62 1,2,5 2.01 0.20 0.05 0.1 5 1,3,4 0.89 0. 20 0.49 0. 62 1,4,5 3.58 0.00 0.05 0.00 1,3,5 8.05 0.00 0.05 0. 15 2,3,4 0.00 0.00 0.49 0.15 2,4,5 0. 00 0. 20 0.05 0. 15 2,3,5 8.05 0.20 0.49 0.00 3,4,5 2.01 0.81 0. 05 0. 15 1,2,3,4 2.01 0.20 0.05 0.1 5 1,2,3,5 5. 59 0. 00 0.05 0.00 1,2,4,5 0.22 0.00 0.49 0.1 5 1,3,4,5 0. 89 0. 20 0.05 0. 15 2,3,4,5 3.58 0.00 0.49 0.00 1,2,3,4,5 2.01 0.20 0.49 0.00 t o t a l 100.00 100.00 100.00 100.00 APPENDIX 3 INDIVIDUAL SUBJECT CONSENSUS AND INSIGHT SCORES Subject Consensus Insight Number Correlation Score 1 .74 .79 2 .75 .87 3 .71 .75 4 .63 .76 5 .75 .88 6 .70 .96 7 .67 .96 8 .73 .99 9 .69 .91 1 0 .72 .74 11 .65 .72 1 2 .77 .99 13 .71 .84 1 4 .69 .94 15 .78 .97 1 6 .77 .97 17 .65 .50 1 8 .72 .81 19 .76 .89 20 .60 .88 21 .77 .67 22 .72 .58 23 .68 .74 24 .70 .86 25 .62 .64 26 .72 .93 27 . 76 .83 28 .54 -.28 29 .73 .70 30 .72 .87 31 .78 .91 32 .72 .80 33 .43 . 56 34 .74 .97 35 .71 .96 36 .65 .81 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0094795/manifest

Comment

Related Items