Science, Faculty of
Computer Science, Department of
DSpace
UBCV
Ritschel, Nico
Doctor of Philosophy - PhD
University of British Columbia
Today, most programmers are not professional software developers, but end-users with limited training and experience in programming. End-user-friendly programming languages and tools aim to support this type of user, and many use visual programming aids to do so. Block-based programming is a popular visual programming style that has been effectively used in computer science education and is the foundation for many modern end-user programming tools. Because of the popularity of block-based programming, language designers can use a rich set of existing technologies that save them the effort of creating visual programming designs from scratch. However, many language designers ignore that block-based programming was created with learners in mind, who have different needs than end-users. Especially when programs grow in size and complexity, blocks offer little support to help end-users understand and edit programs effectively. In our work, we augment block-based programming with visual features that extend the range of programs that end-users can comprehend and write. In particular, we create languages and environments for the domain of robotics programming that allow end-users to write larger and more expressive programs. We focus on three scenarios that represent challenges that end-users face in this domain: coordinating multiple robots that work in tandem, writing large programs that span several workstations in different locations, and reacting to external signals such as machines or user interactions. For each environment, we first discuss the limitations of existing work in the areas of block-based and end-user programming. We present and discuss the design of our visual extensions with the goal to maintain end-user-friendliness. Finally, we evaluate our work through empirical studies, both formative to inform our designs and summative to demonstrate their benefits. Our designs, and the empirical and analytical process that we applied to create them, both contribute to a stronger understanding of how to build end-user-centric tools. We further believe that although our work focuses on the domain of robotics, these contributions transfer to other areas of end-user programming as well.
Enriching Block-based End-userProgramming with Visual FeaturesbyNico RitschelB.Sc., Technische Universita¨t Darmstadt, 2015M.Sc., Technische Universita¨t Darmstadt, 2017A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)November 2023© Nico Ritschel 2023The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:Enriching Block-based End-user Programming with VisualFeaturessubmitted by Nico Ritschel in partial fulfillment of the requirements forthe degree of Doctor of Philosophy in Computer Science.Examining Committee:Reid Holmes, Professor, Computer Science, UBCSupervisorRonald Garcia, Associate Professor, Computer Science, UBCCo-supervisorIvan Beschastnikh, Associate Professor, Computer Science, UBCSupervisory Committee MemberMichael J. Feeley, Associate Professor, Computer Science, UBCUniversity ExaminerNing Nan, Associate Professor, Sauder School of Business, UBCUniversity ExaminerAmy J. Ko, Professor, The Information School, University of WashingtonExternal ExamineriiAbstractToday, most programmers are not professional software developers, butend-users with limited training and experience in programming. End-user-friendly programming languages and tools aim to support this type of user,and many use visual programming aids to do so. Block-based programmingis a popular visual programming style that has been effectively used in com-puter science education and is the foundation for many modern end-userprogramming tools. Because of the popularity of block-based programming,language designers can use a rich set of existing technologies that save themthe effort of creating visual programming designs from scratch. However,many language designers ignore that block-based programming was createdwith learners in mind, who have different needs than end-users. Especiallywhen programs grow in size and complexity, blocks offer little support tohelp end-users understand and edit programs effectively.In our work, we augment block-based programming with visual fea-tures that extend the range of programs that end-users can comprehendand write. In particular, we create languages and environments for the do-main of robotics programming that allow end-users to write larger and moreexpressive programs. We focus on three scenarios that represent challengesthat end-users face in this domain: coordinating multiple robots that workin tandem, writing large programs that span several workstations in differentlocations, and reacting to external signals such as machines or user inter-actions. For each environment, we first discuss the limitations of existingwork in the areas of block-based and end-user programming. We presentand discuss the design of our visual extensions with the goal to maintainend-user-friendliness. Finally, we evaluate our work through empirical stud-ies, both formative to inform our designs and summative to demonstratetheir benefits. Our designs, and the empirical and analytical process thatwe applied to create them, both contribute to a stronger understanding ofhow to build end-user-centric tools. We further believe that although ourwork focuses on the domain of robotics, these contributions transfer to otherareas of end-user programming as well.iiiLay SummaryLearning how to program is hard, but many people must write programs aspart of their daily work. Block-based programming is a popular tool to helpthese people because it is easy to use and can be adapted to many types ofprograms. However, it has only been used for small and simple programs be-fore, and real programs are often large and complex. In this dissertation, weexplore how useful block-based tools are for programming industrial robots,and what factors are holding them back. We then present ideas on howblock-based tools can support large and complex programs without makingthem harder to learn. We present studies to compare different designs andfind out if they reach the goals we have set out to achieve. We believe thatour ideas and the way we study them can be useful for other areas wherenon-experts have to write programs as well.ivPrefaceChapters 3, 4 and 5 of this dissertation have been adapted from work thatwas previously published or is currently under review for publication. Allremaining chapters are my own original, unpublished work.• Chapter 3 (excluding Section 3.4.2) is based on work that has beenpublished [Nico Ritschel, Vladimir Kovalenko, Reid Holmes, RonaldGarcia, and David C. Shepherd. Comparing Block-based Program-ming Models for Two-armed Robots. Transactions on Software Engi-neering (TSE), 2020]. I was the lead author of this work, have createdthe presented design prototypes, and have designed and conducted thepresented study under supervision of Reid Holmes and Ronald Gar-cia. Vladimir Kovalenko implemented and evaluated the prototypethat is presented in Section 3.4.1. David C. Shepherd was an externalsupervisor and has coordinated the collaboration on this project.The work presented in Section 3.4.2 has been accepted for publication[Felipe Fronchetti, Nico Ritschel, Logan Schorr, Chandler Barfield,Gabriella Chang, Rodrigo Spinola, Reid Holmes, and David C. Shep-herd. Side-by-Side: Block-based Programming for Two-Armed Robots:A Comparative Study. International Conference of Software Engineer-ing (ICSE), 2024]. Felipe Fronchetti, Logan Schorr, Chandler Barfieldand Gabriella Chang developed and evaluated the presented prototypebased on my initial work that is presented throughout the chapter un-der supervision of Rodrigo Spinola and David C. Shepherd. I havesupported them in the design and evaluation of their work, but amnot the lead author.• Chapter 4 is based on work that has been published [Nico Ritschel, Fe-lipe Fronchetti, Reid Holmes, Ronald Garcia, and David C. Shepherd.Can Guided Decomposition Help End-users Write Larger Block-basedPrograms? a mobile robot experiment. Proceedings of the ACM onProgramming Languages 6, OOPSLA2, Article 133. 2022]. I was thelead author of this work, have designed and developed the presentedvprogramming environment, and designed and conducted the presentedstudy under supervision of Reid Holmes and Ronald Garcia. FelipeFronchetti supported the work by providing feedback and ideas onearly design prototypes and the study design. David C. Shepherd wasan external supervisor and has coordinated the collaboration on thisproject.• Chapter 5 is based on unpublished work that is currently being revisedfor future submission [Nico Ritschel, Felipe Fronchetti, Reid Holmes,Ronald Garcia, and David C. Shepherd. Blocks? Graphs? Why NotBoth? Designing and Evaluating a Hybrid Programming Environmentfor End-users]. I was the lead author of this work, designed and de-veloped both presented programming environments, and designed andconducted the presented study under supervision of Reid Holmes andRonald Garcia. Felipe Fronchetti supported the work by investigatingrelated work on graph-based languages and tools, and providing feed-back and ideas on design prototypes and the study design. David C.Shepherd was an external supervisor and has coordinated the collab-oration on this project.The study presented in Chapter 3 has been reviewed by UBC’s Be-havioural Research Ethics Board under BREB Number H18-02963. Thestudies presented in Chapters 4 and 5 have been reviewed by UBC’s Be-havioural Research Ethics Board under BREB Number H21-01434.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . xviDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Connecting Block-based Programming and Visual Program-ming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Contributions of This Work . . . . . . . . . . . . . . . . . . . 92 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1 Block-based Programming . . . . . . . . . . . . . . . . . . . 132.1.1 Early Block-Based Programming Languages . . . . . 142.1.2 Scratch as the Prototype for Modern Block-based Pro-gramming . . . . . . . . . . . . . . . . . . . . . . . . 202.1.3 Modern Block-based Programming Beyond Scratch . 232.1.4 A Language-based Comparison of Block-based Pro-gramming Languages . . . . . . . . . . . . . . . . . . 262.2 Visual Programming . . . . . . . . . . . . . . . . . . . . . . . 282.2.1 Classification and Strategies of Visual ProgrammingLanguages . . . . . . . . . . . . . . . . . . . . . . . . 282.2.2 Block-based Programming and Visual Programming . 32vii2.3 Domain of Inquiry: End-user Robotics Programming . . . . 342.4 Cognitive Dimensions . . . . . . . . . . . . . . . . . . . . . . 373 Block-based Synchronization: Coordination of Two RobotArms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.1 Coordination and Parallelism in End-user Languages . . . . 413.1.1 Coordination in Block-based Languages . . . . . . . . 423.1.2 Coordination in Visual Languages . . . . . . . . . . . 433.1.3 RobotStudio Online YuMi . . . . . . . . . . . . . . . 453.2 Design Approach: Side-by-side Synchronization . . . . . . . 463.2.1 Showing Coordinated Programs Side-By-Side . . . . . 473.2.2 Synchronous Coordinated Movements . . . . . . . . . 483.2.3 Timing and Synchronization of Asynchronous Move-ments . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.2.4 Vertical vs. Horizontal Program Flow . . . . . . . . . 513.3 Comparative Survey . . . . . . . . . . . . . . . . . . . . . . . 533.3.1 Research Questions . . . . . . . . . . . . . . . . . . . 533.3.2 Survey Design . . . . . . . . . . . . . . . . . . . . . . 543.3.3 Survey Results . . . . . . . . . . . . . . . . . . . . . . 573.4 Further Studies on Coordination . . . . . . . . . . . . . . . . 633.4.1 Front-end Prototype Study . . . . . . . . . . . . . . . 643.4.2 Large-scale Physical Robot Study . . . . . . . . . . . 653.4.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . 673.5 Discussion and Summary . . . . . . . . . . . . . . . . . . . . 683.5.1 Using Visual Program Alignment To Represent Com-plex Program Semantics . . . . . . . . . . . . . . . . 693.5.2 Visual Support for Programming Beyond Two-ArmedRobots . . . . . . . . . . . . . . . . . . . . . . . . . . 694 Block-based Decomposition: Mobile Robots Operating Mul-tiple Workstations . . . . . . . . . . . . . . . . . . . . . . . . . 714.1 Decomposition in End-user Languages . . . . . . . . . . . . . 744.2 Design Approach: Guided Decomposition . . . . . . . . . . . 764.2.1 Why Do End-Users Not Use Functions? . . . . . . . 774.2.2 Domain-Specific Program Decomposition . . . . . . . 784.2.3 Aligning Physical and Programmatic Scope . . . . . . 794.2.4 A Visual Programming Environment That SupportsDecomposition . . . . . . . . . . . . . . . . . . . . . . 804.3 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . 814.3.1 Research Questions . . . . . . . . . . . . . . . . . . . 82viii4.3.2 Experimental Design . . . . . . . . . . . . . . . . . . 824.3.3 Experimental Results . . . . . . . . . . . . . . . . . . 854.3.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . 934.4 Discussion and Summary . . . . . . . . . . . . . . . . . . . . 944.4.1 Benefits of Program Decomposition . . . . . . . . . . 954.4.2 Function Usage in Block-Based Systems . . . . . . . . 984.4.3 Domain-Specific Task Support Beyond Mobile Robots 984.4.4 Code Re-use: An Open Problem . . . . . . . . . . . . 995 Block- vs. Graph-based Nested Expressions: Enabling Envi-ronment Interactions through Trigger-Action Programming 1015.1 Limitations of Block-based and Data-flow Programming . . . 1035.1.1 Block-based Programming . . . . . . . . . . . . . . . 1045.1.2 Data-flow Programming . . . . . . . . . . . . . . . . . 1065.2 Design Approach: A Hybrid System . . . . . . . . . . . . . . 1085.3 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . 1105.3.1 Prototype Environments . . . . . . . . . . . . . . . . 1105.3.2 Research Questions . . . . . . . . . . . . . . . . . . . 1105.3.3 Experimental Design . . . . . . . . . . . . . . . . . . 1115.3.4 Experimental Results . . . . . . . . . . . . . . . . . . 1165.3.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . 1215.4 Discussion and Summary . . . . . . . . . . . . . . . . . . . . 1225.4.1 Success and Usability: Blocks Beat Hybrid . . . . . . 1225.4.2 (Why) Do End-users Prefer Graphs? . . . . . . . . . 1235.4.3 Implications for Other Domains . . . . . . . . . . . . 1236 Towards End-user-centric Designs for Domain-specific Pro-gramming Languages . . . . . . . . . . . . . . . . . . . . . . . 1256.1 Empirical Programming Language Design for End-users . . . 1266.2 Limitations of the Visual Design Approach . . . . . . . . . . 1296.3 Pushing our End-user Robotics Designs to New Use Cases . 1316.4 End-user Programming Beyond Robotics . . . . . . . . . . . 1347 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141ixList of Tables2.1 Desirable block-based language properties and comparison ofselected languages based on these properties. Languages aresorted in order of their first release. ✓/X= Language does/-does not have property. (✓)= Language partially has prop-erty; from top to bottom: Alice only supports drag and dropediting for statements, not expressions; Snap! and AIA mixnatural language and traditional programming syntax; Snap!and AIA provide feedback, but error messages often refer tocompiled code rather than blocks. . . . . . . . . . . . . . . . . 272.2 All dimensions introduced by the 13 Cognitive Dimensionsof Notation framework [1]. Descriptions are derived fromGreen et al.’s original definitions [2] and adapted for the pur-pose of evaluating (visual) programming languages. . . . . . . 383.1 Box-plots of the scores of novices for each task’s comprehen-sion questions. Participant numbers (n) are reported in Ta-ble 3.7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583.2 Perceived importance of individual design aspects for noviceparticipants, overall and grouped by their preferred design: 1is least important, 5 is most important. . . . . . . . . . . . . 61xList of Figures1.1 Range of potential programming modalities for end-users (fromleft to right): Programming by example, visual programming,block-based programming and text-based programming. Thisdissertation proposes to study the benefits of merging visualand block-based programming. . . . . . . . . . . . . . . . . . 31.2 A large block-based program for a smart home application [3].The visual highlighting of the program structure does littleto make the program more readable or easier to edit. . . . . . 41.3 Examples of programs with custom blocks in the VPL SimuLinkand the BBPL Scratch. . . . . . . . . . . . . . . . . . . . . . 81.4 Work presented in this dissertation. Aligned with the researchquestions stated in Section 1.2, we target three programmingchallenges and address them with prototypes that combineblocks and visual programming. Then, we analyze the proto-types analytically and empirically. . . . . . . . . . . . . . . . 102.1 Example programs written in the two oldest block-based lan-guages, illustrating the early language features described inthis section. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2 The Alice 2.0 programming environment, which allows usersto create animated 3D scenes using a mix of block-based andmenu-based programming. . . . . . . . . . . . . . . . . . . . . 182.3 Definition of the expression “x > 0” in Alice via four levels ofnested drop-down menus. . . . . . . . . . . . . . . . . . . . . 192.4 Modern version of the Scratch environment that runs in a webbrowser and allows users to create sprite-based 2D animations. 212.5 Synchronization as proposed by the authors of Scratch [4],recreated in a modern version of the environment. Users needto manually define and update variables that can be used in“wait until” blocks. . . . . . . . . . . . . . . . . . . . . . . . . 22xi2.6 Example of type coercion in Snap!: The function supplied to“map” requires two arguments instead of one, but instead ofraising an error, Snap! duplicates the first argument to fillboth holes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.7 Example program in AIA with an error that prevents compi-lation. A colour expression must be connected to the programto fix the error, but this is not obvious to users as other ex-pression blocks look identical. . . . . . . . . . . . . . . . . . . 252.8 Examples of visual languages in different categories. . . . . . 302.9 ABB RobotStudio: A modern development system for pro-fessional robot programmers [5] that features a complex sim-ulator and development environment. . . . . . . . . . . . . . . 342.10 Polyscope: a simplified robot programming environment withvisual features for intermediate users [6] and a less complexsimulator interface. . . . . . . . . . . . . . . . . . . . . . . . . 362.11 CoBlox: a block-based robot programming environment forend-users [7] integrated into ABB RobotStudio. It only sup-ports rudimentary commands, but is easier to use than anyof the other shown systems. . . . . . . . . . . . . . . . . . . . 373.1 Parallel programming in Alice using a “do together” blocknested with “do in order” blocks. . . . . . . . . . . . . . . . . 433.2 Parallelism in Lego Mindstorms EV3. The sole purpose ofthe six blocks highlighted in yellow is to synchronize the topand bottom row. . . . . . . . . . . . . . . . . . . . . . . . . . 443.3 RobotStudio Online YuMi programming environment for two-armed robots. The tool is primarily text-based, but has a setof buttons on the right side of the interface. . . . . . . . . . . 463.4 Example of a block-based program mimicking the physicallayout of the targeted robot, with each arm’s program placedon the same side as the physical arm facing the user. . . . . . 473.5 Example programs using vertical flow. In both programs, onerobot arm hands over an item to the other. . . . . . . . . . . 503.6 Example programs using horizontal flow. In both programs,one robot arm hands over an item to the other, as for thevertical programs in Figure 3.5. . . . . . . . . . . . . . . . . . 523.7 Survey flow (from top to bottom) with participant numbersfor each design treatment. Participants were first categorizedas novices and experts and then randomly assigned one of thedesigns shown in Figures 3.5a, 3.5b, 3.6a and 3.6b. . . . . . . 55xii3.8 SUS scores computed for novice participants’ responses, plot-ted on the SUS percentile curve. Participants rated the ver-tical design candidates significantly better. . . . . . . . . . . . 603.9 SUS scores computed for expert participants’ responses, plot-ted on the SUS percentile curve. Experts gave consistentlyhigher ratings for all designs and had the same preferences asnovices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.10 Fully functional prototype environment Duplo based on thedesign prototype E-V. . . . . . . . . . . . . . . . . . . . . . . 653.11 Participants’ completion times in ascending order. Each barrepresents one participant, and its color indicates the assignedenvironment (blue for Duplo, yellow for ROY). A black circlemarks participants who ran out of time before completing allsub-tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663.12 Design space based on synchronization model and programflow direction. Each of the designs shown in Figures 3.5a,3.5b, 3.6a and 3.6b corresponds to one cell of this table. . . . 684.1 Proposed programming system showing two tasks and side-by-side canvases. The left canvas shows the main programand the right one shows the body of the currently selectedtask(s). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734.2 Structures found in end-user programs. Traditional program-ming languages treat all of them as equal alternatives, but wespeculate that a hierarchical structure benefits end-users themost. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.3 Grammar(s) of the programming language used in the pro-posed system. A program (left) can only contain calls to tasksat the top-level, and that task definitions (right) take placein a separate programming canvas with its own syntax. AsSection 4.2.4 describes, there are some differences betweenthe language’s semantics shown here and its block-based pre-sentation shown in Figure 4.1. Also note that locations aredefined via a visual location picker that is not represented inthis grammar. . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.4 The traditional block-based development environment usedby the TR Cohort of the programming study. It offers thesame command blocks and simulator, but uses a single canvasand provides free-form function blocks that are defined andused in this canvas. . . . . . . . . . . . . . . . . . . . . . . . . 82xiii4.5 Categorization of participants in the TR cohort and theirprograms for the three tasks our study. . . . . . . . . . . . . . 874.6 Categorization of participants in the GD cohort and theirprograms for the three tasks our study. . . . . . . . . . . . . . 884.7 User study results. For box plots: center lines show themedians, box limits indicate the 25th and 75th percentiles.Whiskers extend 1.5 times the interquartile range from the25th and 75th percentiles. Outliers are represented by dots,only outliers < 20 minutes are shown. . . . . . . . . . . . . . 924.8 Examples of programs written by participants for Task 2.(a) shows a subset of an unstructured program (57 blocks intotal); (b) shows the top layer of a hierarchically structuredprogram (61 blocks in total). . . . . . . . . . . . . . . . . . . 964.9 Examples of programs written by participants for Task 3.(a) shows a subset of an unstructured program (58 blocks intotal); (b) shows the top layer of a hierarchically structuredprogram (61 blocks in total). . . . . . . . . . . . . . . . . . . 974.10 Example of a program where movement and stationary in-structions are entangled. It is not possible to entangle move-ment and stationary instructions in this way with our approach. 995.1 A nested expression in the block-based language Scratch: Theentire expression is displayed in a single line and the nestingstructure is difficult to identify. . . . . . . . . . . . . . . . . . 1045.2 Comparison of flow in imperative block-based code and block-based trigger expressions. . . . . . . . . . . . . . . . . . . . . 1055.3 The same program as in Fig. 5.2b represented as a data-flowgraph. The edges make the data-flow explicit and program-mers can manually group and arrange nodes. . . . . . . . . . 1075.4 Hybrid programming environment: the left canvas shows high-level program flow as blocks, the right canvas shows selectedtasks as blocks and triggers as data-flow graphs. . . . . . . . 1095.5 Procedure of the randomized experiment to evaluate the twoprototype environments . . . . . . . . . . . . . . . . . . . . . 1115.6 A more complex trigger with nested operators, used to testparticipants’ program comprehension. . . . . . . . . . . . . . 1155.7 Completion probability over time for the two programmingtasks. Shaded areas are 95% confidence intervals. . . . . . . . 117xiv5.8 Distribution of responses for the Likert items of the survey,split by participant group (Blocks: N=58, Graph: N=52).For the bottom row, bars are aligned around neutral responsesto indicate diverging preferences. . . . . . . . . . . . . . . . . 1196.1 Early design draft for a 3-armed extension of the vertical de-sign with explicit synchronization that we present in Chap-ter 3. It uses 3 columns to represent the individual robotarms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.2 Early design draft for an environment that features both two-armed coordination as presented in Chapter 3 and guideddecomposition as presented in Chapter 4. The left canvasshows the high-level program flow, which is composed of aseries of two-armed programs that are defined in the rightcanvas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133xvAcknowledgmentsFirst and foremost, I would like to thank both of my supervisors, ReidHolmes and Ron Garcia, for their guidance, encouragement and advice overthe past six years. Both my own research and the world around it havetaken turns that nobody could have foreseen when I started my degree. Iam grateful that I had not just one, but two compassionate and dependablementors as sources of support during this time.I would also like to thank Ivan Beschastnikh for his advice and feedbackon my work and the trajectory of my research. It has helped immensely tosteer and improve this dissertation.I have had many other supporters and collaborators throughout my de-gree that I would also like to recognize. The most influential one was with-out doubt David Shepherd, who has been a great mentor throughout all mywork, almost to the degree of being a third informal supervisor. In addition,I would like to thank David’s student Felipe, who was a fantastic collabora-tor on many projects, and Alexander Summers, who has encouraged me totake on teaching, and who has put immense effort into mentoring me duringmy term as instructor.Furthermore, I would like to thank Sebastian Erdweg and Mira Mezini,my previous supervisors during my time at TU Darmstadt. Although myarea of research has shifted substantially since I worked with them, it wastheir encouragement that led me to take on my PhD journey, and I wouldcertainly not be in the same place without them.This research has been generously supported by ABB Inc. and theNatural Sciences and Engineering Research Council of Canada (NSERC).Through my collaborators, it has further been supported by the NaturalScience Foundation (NSF).I have also been supported by my parents and my family, who have beenpatiently waiting for me to finish my degree and have been there for me,both emotionally and financially, the entire time. I apologize for moving sofar away, but enjoy every day we are able to see or talk to each other.Finally, I would like to thank the many members of the Software Prac-tices Lab and the many friends I made within the lab and beyond, of whichxviI only have space to address a few directly. James, talking to you has beenan everlasting source of fun and your (terrible) jokes have lifted me up manytimes. Adam, I am glad for all the runs (both for coffee and in the literalsense) we had over the years. Giovanni, you were a great friend and playingboard games with you was often a highlight of my week. Arthur, thanks forbeing my grad buddy and helping me out whenever you could. Felipe, youradvice has helped me many times as I was stuck at research or in the mid-dle of departmental bureaucracy. Paulette, thank you for always standingup to the department and fighting for our grad student concerns. Braxton,Felix, Joey and Lucas, even though we only shared a short period in the labtogether, I still enjoyed all of it! Neil, Hayley, Siddhesh, Anna and Michael,you were amazing friends, and I will never forget all the amazing times andadventures we had together!xviiDedicationTo AlexMeeting you is the best thing that happened on this journey.xviiiChapter 1IntroductionMillions of people program at work or to solve problems in their daily lives,but only a small fraction are professional software developers. In the UnitedStates, approximately 1.7 million employees were categorized as software de-velopers in 2020 [8]. Most of these employees have received a comprehensiveeducation in computer science and hold at least a Bachelor’s degree in arelated field [8]. However, previous research estimated that between 55 and90 million employees in the U.S. had to program as part of their daily tasksin 2012 [9, 10], a number that has likely grown over the past decade. Theseprogrammers are not trained professionals, but end-user programmers whohave received almost no programming-related education.End-users must program because there are not enough professionalsavailable to write all the code for them. Learning how to program is hard,and even beyond earning a formal degree, it takes novices several years ofeducation and experience before they reach the skill level that is expectedof a programming expert [11]. Further, the majority of everyday tasks re-quires domain knowledge that is at least as hard to acquire as the necessaryprogramming skills. One way to solve this shortage is to make programmingeasier, and make end-users learn only the skills that are absolutely necessaryto solve the task at hand [12]. This type of end-user with limited trainingis sometimes called vernacular developers, in contrast to the professionaldevelopers who traditionally write software [13].Programming environments and languages must support end-user pro-grammers to allow them to succeed. A substantial body of previous researchhas investigated how to teach end-user programmers to use tools that weredesigned for experts [14]. However, training alone can rarely cover the lackof knowledge and experience that end-users are confronted with when theyuse such systems. End-users are more likely to succeed when the tools theyuse are specifically designed with their needs in mind [14, 15].Many beginner-friendly tools and languages use Visual Programming,a form of programming where some or all parts of a program are repre-sented and edited graphically. By using this form of editing, the designersof these systems aim to support users beyond what pure text is capable1of [16]. There also exist a number of purely text-based historical [17] andrecent [18] languages that aim to be beginner-friendly, but even those areoften supported by environments that provide visual aids. The vast major-ity of visual modalities fall into the spectrum that Figure 1.1 illustrates. Onthe extremes, techniques like Programming by Example [19] or Natural Lan-guage Programming [20] try to overcome the need for traditional text-basedprogramming entirely. Programming by example has users record and ex-ecute the intended program behaviour, and natural language programminginterprets instructions in natural language without requiring users to learnany syntax beyond the language they already know. On the other end of thespectrum, work on visual aids for text-based programming like syntax high-lighting or error highlighting aims to make programs and their behavioureasier to comprehend without changing the underlying programming lan-guage at all. Subtle visual aids have been shown to be useful for professionaldevelopers, but the support they provide does not appear to meet the needsof novice programmers [21].In the middle of the visual modality spectrum lie Visual ProgrammingLanguages (VPLs) and Block-based Programming Languages (BBPLs). Bothaim not to replace traditional programming, but to make it easier by combin-ing it with visual elements. VPLs typically make aspects of a program moreexplicit, or changing them more direct [22, 23]. In the example in Figure 1.1,this aspect is the flow of data, but other VPLs might choose higher-level as-pects, such as the scope of data [24], or aspects that are domain-specificlike the layout of a website [25]. BBPLs are often seen as a specific typeof VPLs, but in their pure form they are different. Instead of aspects of aprogram, they visualize the syntax of the underlying programming languageand make programs easier to edit structurally by dragging and droppingsyntactic elements as if they were puzzle pieces. [4]. This makes them quiteversatile: virtually any text-based programming language can be convertedinto a BBPL [26].Block-based programming has reached wide-spread adoption in recentyears, particularly in programming education where its correspondence totext is often seen as a benefit. For example, the TIOBE Index of pop-ular programming languages lists the block-based programming environ-ment Scratch [4] as the 20th-most popular programming language overall in2022 [27]. BBPLs are said to provide first-time programmers with more vi-sual feedback than text-based languages, and to encourage experimentationin an environment that minimizes frustration [4, 28]. However, block-basedsystems like Scratch are not intended for use beyond learning – they pri-marily serve as a stepping stone in turning beginners into expert program-212PrintMy ProgramRecord Runmain() { var value1 = 1; var value2 = 2; print(value1 + value2); }PrintSet value2 to 2Set value1 to When program started 1 + value1 value2Figure 1.1: Range of potential programming modalities for end-users(from left to right): Programming by example, visual program-ming, block-based programming and text-based programming.This dissertation proposes to study the benefits of merging vi-sual and block-based programming.mers [4, 29]. More recent educational work has tried to make this transitioneven easier by creating structural editors that are closer to text-based edit-ing than blocks [30, 31]. This closeness to text does come with limitationsthough: structural editing tools are inherently limited in how much theycan visually support their users.Block-based programming is also used in several recent tools that weredesigned with end-user programmers in mind [3, 7, 32, 33]. Because end-user languages are often limited in scope, tool developers often prefer to usean established, standardized visual design that comes with tried and testedframeworks and tools. Mature block-based frameworks like Blockly [34],which have been evaluated both in terms of usability and the requiredimplementation effort for the overall system, are a good match for theseneeds [33, 35, 36]. However, most end-user BBPLs are just variations of ex-isting educational languages and build on the same principles without con-sidering the differences between traditional learners and end-users. There-fore, they have limitations that make them hard to use in practice, for exam-ple when programs are larger or more complex than those used in teachingsettings [37].In theory, BBPLs can support almost any advanced programming lan-guage feature [26], but their usability benefits diminish if these features areincorporated naively. Instead, the visual representation of low-level programsyntax can distract users in some cases and reduce the readability of pro-grams and the usability of the overall system [38]. Consider the block-basedprogram in Figure 1.2. It looks complex and requires substantial mental ef-fort to read, although the underlying program is less than 20 lines long andmostly comprised of simple variable assignments and arithmetic. This pro-3Figure 1.2: A large block-based program for a smart home applica-tion [3]. The visual highlighting of the program structure doeslittle to make the program more readable or easier to edit.gram illustrates the problem that block-based system designers face whenthey replicate text-based language designs and features, even if they do notaddress the needs of end-users.In this dissertation, we investigate whether block-based systems can bedesigned to be end-user-centric, meaning that they address the needs ofend-users rather than learners. In particular, we explore how BBPLs canbe adapted to enable end-users to write a larger variety of more expressiveprograms without making them harder to learn or use.Thesis statement: By incorporating end-user-centric design features fromvisual programming, block-based programming languages can enable end-users to write programs that solve larger and more complex tasks while re-taining their usability, learnability and ease of implementation.We limit the scope in which we investigate our thesis to one concretearea of industrial end-user programming: robotics. By focusing on one areain which end-users commonly perform programming work, we can tailorlanguages to their specific needs and use abstractions and terminology thatthey are familiar with. However, we consider robotics programming to be a4placeholder for other end-user programming domains, such as home automa-tion [33, 39] or app development [35]. We believe that most observations wemake and design considerations we derive apply to these other domains aswell.We selected robotics as our domain of inquiry for several reasons:1. The target audience of robotics programming languages, and the set-tings in which they are used, are comparatively well-studied and well-defined. Previous work has outlined the settings in which industrialend-users must program robots [7, 40], which forms of training theycan receive in practice [7, 12], and what their expectations are whenprogramming and interacting with robots [41]. This foundation of re-search allows us to reduce the amount of preliminary work necessaryto address the needs of this type of end-user.2. Although modern robotics programming languages provide abstrac-tions for low-level cyber-physical parameters, several aspects of robotprograms challenge end-user programmers: Many non-trivial robotprograms require some form of coordination or interaction betweenmultiple robots or robots and their environment. In addition, evenconceptually simple tasks like picking up and carrying an item can re-quire a large number of programming steps and become complex whena robot must interact with its environment. Programming interfacescan address these challenges by providing abstractions and visual aidsthat offer additional support.3. Robot programming is a domain with substantial related work in bothlanguage modalities that we aim to connect with each other. End-userrobotics programming has been explored previously, by visual pro-gramming work [42, 43] and block-based programming work [36, 44].We can use these design examples from both domains to guide ourwork. At the same time, most previous work has substantial usabilityor scalability limitations [45], leaving room for significant improve-ment.1.1 Connecting Block-based Programming andVisual ProgrammingDue to the growing interest in block-based programming, recent work hasexplored their use outside of education. For example, personal end-users5can now use blocks to automate their homes [33, 39], and professional end-users can use them to develop augmented reality apps [35] and programindustrial robots [7]. In all of these use cases, blocks are intended to bethe only language that an end-user has to learn to solve the task at hand.This means that the ability to transition from blocks to text is not a majorreason that might motivate system designers to use BBPLs. However, thesimilarity between block-based programming and text has other advantagesthat makes BBPLs attractive for end-user systems:1. Block-based programming is an established technique. It allows bothusers and language designers to rely on best practices and transferknowledge from one block-based tool to another. One can even spec-ulate that the continued, widespread use of blocks in child educationcould expose potential end-users to blocks and make it easier for themto use block-based programming tools productively as adults.2. Many visual languages use data structures like graphs to representprograms internally [46]. This requires different compilation or inter-pretation strategies than traditional programs that are represented asAbstract Syntax Trees (ASTs). The structure of block-based programson the other hand matches that of the underlying AST. This meansthat tool creators can not only save the effort of creating a parser fortheir programming language, but can use established language frame-works and tools for AST manipulation and compilation. This makesblock-based tools not only easy to use, but also easy to modify andextend.3. The familiar visual design of block-based systems makes them alsodirectly accessible to people with existing programming experiencein text-based languages. These users can understand the underlyingtext-based code without having to learn an entirely new visual syntax.This means that professionals can easily read and edit block-basedprograms, and support or collaborate with end-users if necessary.Evaluations conducted by the authors of end-user block-based systemsconsistently find them to be easier to learn and use than professional tools [33,35, 36, 47]. However, these findings are always based on small tasks thatmight resemble realistic use cases but are still simplified so that they can besolved by a single user in a single session. There is little concrete evidencethat end-users could solve larger or more complex tasks as well. Further,studies on educational block-based systems that are used outside traditional6learning environments have shown that even when BBPLs support encap-sulation or procedural abstraction to enable the writing of larger programs,users rarely make use of these features [37, 38]. Similarly, some block-basedtools support advanced programming language features like parallelism [4],object-oriented programming [48] or higher-order functions [49], but usersoften ignore or misunderstand them [38]. It therefore appears that exist-ing block-based tools are better suited for writing short, procedural codesnippets than large or complex programs.The designers of VPLs have developed methods to represent encapsula-tion and abstraction that they have demonstrated to be more understand-able to end-users than the traditional approaches used by text-based lan-guages or BBPLs [2, 50]. We call these methods end-user-centric, as theytypically target a specific audience of end-users with existing domain knowl-edge, unlike educational languages that aim to teach general purpose pro-gramming to users with a wide range of backgrounds.For example, Figure 1.3 compares how the VPL SimuLink represents theapplication of a user-defined function in contrast to the BBPL Scratch. Al-though both languages use similar mechanisms for procedural abstraction,their designs differ in important details: SimuLink represents programs asflow graphs, and represents procedures as nodes in this graph. In SimuLink,edges into and out of a procedure represent inputs and outputs; the bodyis fully abstracted and can only be edited in a separate canvas that is in-dependent from the main program. As a result, user-defined procedures inSimuLink are explicitly encapsulated and data flow into and out of eachprocedure is visualized. On the other hand, Scratch presents both the pro-cedure definition and its usage on the same canvas, and even allows usersto freely drag variables between them. Similar to text-based, dynamicallytyped programming languages, scoping restrictions are not enforced until anerror occurs at run-time This increases the likelihood that users introducebugs and misunderstand where data can flow in the program.Procedural abstraction is just one example of a language feature thatvisual languages represent through end-user-centric visualization. Visuallanguages have also demonstrated that they can represent features like par-allelism [51] and higher-order functions [52] in ways that make them moreexplicit and easier for end-users to understand. As a result, many visuallanguages successfully scale to large and complex programs, and they haveproven to be invaluable tools for real industrial end-users [53, 54].In this dissertation, we propose to investigate whether we can designblock-based programming languages that support larger, more complex pro-grams in a similar way to VPLs. We believe that many of the end-user-7(a) Scratch program that uses a custom block to abstract a sequence of if-elseconditionals. The definition is shown next to the main program.(b) SimuLink program that uses a custom block to abstract a mathematicalcomputation. Only the inputs and outputs of the block are shown, unless the usermanually inspects the block.Figure 1.3: Examples of programs with custom blocks in the VPLSimuLink and the BBPL Scratch.8centric methods and strategies that visual languages use to achieve this goalcan be applied to blocks as well. We further hypothesize that by addingvisual, end-user-centric features to blocks, language designers can providebenefits to end-users as they use BBPLs for their programming work.1.2 Contributions of This WorkTo support the thesis statement we presented above, we investigate thefollowing research questions:1. What usability limitations do end-users encounter when using existingblock-based programming languages?2. Can we extend BBPLs with end-user-centric features to overcomethese limitations in the area of robotics without losing their end-user-friendliness?3. What implications does the addition of end-user-centric features toBBPLs for robot programming have for their existing usability andimplementation benefits?In the rest of this section, we provide additional context and details regardingthese questions and the work we propose.Figure 1.4 outlines the structure of this dissertation. It refers to theresearch questions above. We have selected two types of robots that serveas use cases for our end-user programming tools: two-armed robots that arestationary, and mobile robots that can move within a defined perimeter andhave one attached robot arm. Both of these robot types are commonly usedin practice, but there exists little work that allows end-users to use themto their full potential. In particular, we have identified three programmingchallenges that end-users face and that existing tools do not address:1. synchronizing two robot arms to solve coordinated tasks. Thisscenario is difficult for end-users because it directly forces them towrite programs that contain parallelism and coordination. When theyprogram two robot arms, users must actively determine when armswait for each other or perform synchronized actions. Existing block-based tools have attempted to make this and similar types of parallelprogramming accessible. However, all existing, pure block-based sys-tem designs suffer severe limitations regarding visibility or abstraction.To tackle this scenario, we present a design that uses a visual approach9➟To: Above Place➟➟➟➟Pick-upTo: Hand-over ATo: Hand-over BTo: PlaceTo: ➟AbovePick-upTo: ➟AbovePick-upTo: ➟To: Above PlaceLR➟To: Above Place➟➟➟➟Pick-upTo: Hand-over ATo: Hand-over BTo: PlaceTo: ➟AbovePick-upTo: ➟AbovePick-upTo: ➟To: Above PlaceLRMove to Above Place .Open Hand.Wait.Move to Place .Move to Above Place .Wait.Wait.Wait.Open Hand.Wait.Wait.Close Hand.Move to Pick up .Move to Above Pickup .Move to Handover A . Move to Handover B .Move to Above Pickup .Open Hand.Wait.Wait.Wait.Wait.Open Hand.When is pressed: Close Hand.analytical discussions of features and trade-offs, using the “13 Cognitive Dimensions of Notations” frameworkempirical methods, such as controlled studies and experiments with end-user participantsTo address RQ2: We design prototypes of block-based programming tools that incorporate visual programming features to target each programming challenge in the context of robotics:To address RQ1: We identify three challenges that end-users face when using block-based tools:coordinating parallel programs decomposing large programs embedding nested expressions into imperative programs To address RQ3: We evaluate the prototypes during and after the design process, using:Figure 1.4: Work presented in this dissertation. Aligned with theresearch questions stated in Section 1.2, we target three pro-gramming challenges and address them with prototypes thatcombine blocks and visual programming. Then, we analyze theprototypes analytically and empirically.10to improve visibility by displaying two programs side-by-side, and thenhighlights interaction points between the two robot arms to make themeasier to understand for end-users.2. programming mobile robots to solve tasks across multipleworkstations. This scenario poses two separate challenges to end-users: First, when a robot’s task spans multiple workstations, therobot must operate in multiple different spatial contexts. For exam-ple, objects or tools that are necessary to solve a task may or may notbe present at different workstations at given points in time, and end-users must understand this dependency between the program theycreate and the robot’s current surroundings. Second, multi-stationrobot programs often grow in size as the number of workstations andsteps to complete increases. Without structuring programs and us-ing appropriate abstraction, end-users can experience difficulties withreading or editing such large programs. In traditional programminglanguages, users solve both of these challenges using procedural ab-straction. However, as we illustrated in Section 1.1, BBPLs do notsufficiently convey the purpose and correct usage of this form of ab-straction to end-users. Instead of teaching users how to use abstractionthrough external means such as manuals or tutorials, we present a de-sign for this scenario that guides users in decomposing their programsas they work on a task. Our design splits programs into low-level tasksthat take place in a single location and are connected by a separate,high-level structure. These tasks are placed in separate editing can-vases, which are arranged in a way that makes it easy to navigate andkeep track of the different program components.3. programming mobile robots to interact with their environ-ment. This usage scenario extends the previous one by assumingthat robots do not just perform tasks as a single predetermined se-quence, but react to inputs from their environment. By enabling thisinteraction, we both extend the range of tasks that a robot can ac-complish and potentially allow the same robot to work on multipleparallel tasks. This use case aligns closely with how workers concep-tualize robots in industrial settings: they consider robots to be theirassistants and would like to assign them tasks that help in their dailyroutine [41]. To allow a robot to work on multiple tasks, potentiallyeven those that serve multiple different end-user programmers, requiresa clear program structure and model of interaction with the environ-11ment. For this reason, we propose a design for an editing system thatfocuses on triggers to control and schedule the robot’s interactionswith its surroundings. In particular, we propose a hybrid environmentthat connects block-based programming and visual dataflow program-ming for defining dependencies and conditions that trigger the robotto perform specific work.We first answer research question RQ1 by analyzing each of the proposedusage scenarios and the limitations that end-users encounter when using ex-isting tools. We then answer RQ2 and RQ3 by designing and evaluatinglanguage and environment prototypes to overcome these limitations. Forour evaluation of each design, we combine our findings from RQ1 with anempirical assessment through user studies and experiments. Chapter 3 de-scribes our work on two-armed synchronization and coordination, Chapter 4the work on programming mobile robots across multiple workstations, andChapter 5 the work on making mobile robots interact with their environ-ment. In Chapter 6, we provide a discussion of our work and a reflection onwhich elements can be applied to other domains within and beyond robotics.12Chapter 2BackgroundIn this Chapter, we give an overview of the history of block-based program-ming and the accomplishments and limitations of several seminal block-based tools. We also present and categorize visual programming languagesbased on their properties and features, some of which we set out to transferto block-based programming. Finally, we introduce the framework of Cogni-tive Dimensions of Notation (CDN) that provides us with terminology andcategorizations that we use in later Chapters.2.1 Block-based ProgrammingBlock-based programming has gained substantial popularity over the pastdecade. It has been successfully used in Computer Science education in high-schools [28] and universities [55], and in personal [33] and industrial [7] end-user programming. This is remarkable, considering that early block-basedlanguages did not receive much mainstream attention. The Scratch [4] en-vironment, released in 2010, was the first block-based programming systemthat found widespread popularity, and it has inspired many other languagesthat apply block-based programming principles to new application domains.In this Section, we present a selection of block-based programming lan-guages that we consider to be influential or historically relevant. Centeringaround Scratch as the environment that influenced modern block-based pro-gramming the most, we describe languages before Scratch in Section 2.1.1,the principles introduced by Scratch itself in Section 2.1.2, and more recentwork building on top of Scratch in Section 2.1.3. In addition to providing ahistorical background, we outline the contribution of each new block-basedtool at a language or environmental level. In addition to individual contri-butions of previous tools, we highlight one exemplary challenge that almostall block-based tool designers face and that illustrates the range of choicesthey can make: how to visually represent types using blocks. Finally, wesummarize the features and properties of each discussed tool in Section 2.1.4,although we revisit some of them in later chapters because they relate toour own contributions.13(a) BridgeTalk (b) LogoBlocksFigure 2.1: Example programs written in the two oldest block-basedlanguages, illustrating the early language features described inthis section.2.1.1 Early Block-Based Programming LanguagesThe oldest language that we consider to be related to modern block-basedprogramming environments is BridgeTalk, which was released in 1987 [56].BridgeTalk was designed to allow users to convert informal problem de-scriptions into functional program code. Remarkably, BridgeTalk’s authorsalready distinguish between learners and end-users as two potential usergroups that they aim to support: in their vision, aspiring programmerscould learn BridgeTalk as an introductory language to prepare them forprofessional languages like Pascal, and end-user programmers could writeall their programs in BridgeTalk without the need to “graduate” from it.However, though BridgeTalk underwent multiple design iterations and userstudies, it was never fully implemented.At a first glance, the language, shown in Figure 2.1a, looks remarkablysimilar to modern block-based languages: BridgeTalk uses puzzle-like shapesfor blocks that visualize the structure of a program explicitly. As its authors14explain, they chose this design to address common issues they perceivedwhen beginners write their first programs. For example, they found thatbeginners often struggle with understanding the flow between lines of nestedcode like the loop shown in Figure 2.1a. In addition, beginners are oftenunaware of the full syntax of a language, and don’t know whether they cantranslate a single step in their informal plan into a single instruction ormust decompose it into multiple steps. BridgeTalk supports its users byproviding a visual template for which syntactic elements are necessary andhow programs can be composed from them.Another noticeable design aspect of BridgeTalk is that it is much moreverbose than text-based programming languages. Each syntactical elementis labeled and explained in plain language that can be understood withoutexternal documentation. This is a stark contrast to other popular program-ming languages from the time BridgeTalk was developed, which relied onsyntax that was efficient to write for experts, but hard to learn and readwithout memorizing abbreviated commands and syntax. BridgeTalk’s au-thors chose this design with two goals in mind. First, they wanted to givenovice users confidence in their code, and avoid syntax that users could mis-interpret if they did not have the necessary background knowledge. Second,they aimed to implicitly teach users the common programming terminologyof the time as they were repeatedly using the same blocks labeled with it.Finally, another design goal of BridgeTalk is to provide support for high-level abstractions to guide the thinking of programmers. From a modernperspective, where high-level language features are commonly found even inprofessional languages, this goal might not be immediately clear from exam-ining programs like the one in Figure 2.1a. This program not only illustratesthe detailed step-by-step computation of a while-loop, but also illustrateshow the absence of non-integer variables requires programmers to resort toworkarounds like using “magic numbers” to represent special states. How-ever, this program demonstrates BridgeTalk’s high-level language goals inthe contexts of its time, for example by showing how the language distin-guishes between a “Counter Plan” and the “Running Total Plan”. Bothof these blocks represent simple arithmetic addition, but BridgeTalk usesdistinct blocks for them to illustrates that one is used as a loop iterationcounter while the other is used for mathematical computation.All of BridgeTalk’s design elements we describe above made their wayinto modern block-based languages. However, BridgeTalk has two substan-tial limitations compared to later languages that only become apparent whenwriting code instead of just reading it. First, BridgeTalk programs are notassembled by dragging and attaching blocks, but rather by iterating through15each block’s unfilled syntactic connectors and selecting a possible new blockto fill each gap. Second, all blocks have a completely fixed shape, and blockscan only be nested in ways supported by their shape. This means that in Fig-ure 2.1a, one cannot place a “Compute Plan” block in the “Get New Value”slot of the loop block because it does not have the correct height for the slot.It also means that loops cannot be nested inside other loops, and the sameapplies for conditionals. BridgeTalk’s authors discuss potential workaroundsfor this issue but also argue that larger programs or deeper nesting of blocksmight confuse novices. They claim that discouraging novices from writingcode that is too complex for them to understand could be a feature in-stead of a limitation. However, as newer block-based languages have shown,BridgeTalk’s limitations were severe enough to outweigh this benefit.After several years without follow-up work related to BridgeTalk beingreleased, Sten Mino¨r re-introduced the idea of composing programs throughvisual blocks in 1992 [57]. This work is primarily an analysis and criticaldiscussion of structural editing tools of its time and does not contribute anynew language designs or evaluations. However, it presents brief programsnippets that resemble BridgeTalk’s design ideas, and introduces the idea ofusing a mouse pointer to drag and drop blocks onto the programming canvas.Remarkably, Mino¨r does not explicitly reference BridgeTalk, and none ofthe later work on blocks we discuss in this Chapter cites his work either.It appears that although Mino¨r’s work fills a gap in the development ofblock-based tools, it has had little influence on other tools, and that paralleldevelopments like the introduction of drag-and-drop were coincidental.Following Mino¨r’s conceptual work, LogoBlocks [58] was the next fully-fledged language that re-introduced the idea of programming with blocks.Notably, like Mino¨r, the authors of LogoBlocks do not cite BridgeTalk as asource of inspiration, and although they share the same design ideas, thereappears to be no direct connection between the two languages. Instead, theauthors of LogoBlocks were inspired by the text-based programming lan-guage Logo [17] and the learning theory of constructionism. The theory ofconstructionism focuses primarily on educating children and aims to supportlearning through experimentation rather than following a fixed curriculumof instructions and learning materials [59]. Logo and LogoBlocks supportthis idea by providing only a small number of syntactic elements that areeasy to memorize and supported by the visual shapes of blocks. Throughthis syntax, children can quickly progress from guided learning to self-drivenexperimentation and create sharable artifacts as part of their learning pro-cess [60].BridgeTalk and LogoBlocks are designed for different audiences and pur-16poses: BridgeTalk’s targets adults to introduce them to established pro-gramming concepts and patterns, while LogoBlocks targets children and isonly loosely connected to traditional programming languages. These dif-ferences might also explain the different domains that the two languagestarget. BridgeTalk aims to support arbitrary computation, and LogoBlocksfocuses on the more tangible use case of programming Lego robots. Despitetheir different origins, the two languages share a remarkable number of de-sign elements, including the ideas of using puzzle pieces to represent syntaxand providing high-level abstractions to their users. However, LogoBlocksovercomes BridgeTalk’s two main limitations: it allows users to assembleprograms via drag-and-drop, and it allows blocks to scale in size as neededas long as they are connected to an existing block through the right con-nector. LogoBlocks is also the first language that uses different shapes tovisually distinguish between integer and boolean variables, as shown in Fig-ure 2.1b. The round outer connector of the boolean expression “sensora <10” is easily distinguishable from the rectangular connector of the integers“sensora” and “10” it contains. LogoBlocks also uses colours to highlightsemantic differences between blocks, for example the constant “10” and themutable sensor variable “sensora”.Another notable distinction between BridgeTalk and LogoBlocks is theuse of textual labels. As the two programs in Figure 2.1 illustrate, LogoB-locks can represent a larger program in substantially less screen space thanBridgeTalk. To achieve this goal, LogoBlocks uses abbreviations for some ofLogo’s commands, as shown in Figure 2.1b where the blocks labeled “a” or“b” are short forms of Logo’s “motor a, on” statement to turn on a motor.This is a stark contrast to BridgeTalk’s verbose block descriptions, and canmake LogoBlocks much harder to read without background knowledge.Alice 2.0 is a block-based educational storytelling environment that wasreleased in 2006 and received wide-spread attention in the computer scienceeducation community [61]. It extends the text-based scripting language Al-ice, which was released in 1998, and provides a block-based programmingenvironment for it [62]. Alice primarily targets high-school students andallows them to program the behaviour of characters and objects in ani-mated 3D scenes. This re-framing of programming as a way to tell storieswas considered to be one of Alice’s primary contributions at the time [63]and has inspired a number of later, not necessarily block-based, educationalenvironments and computer science curricula [64, 65].Focusing on the block-based aspects of Alice 2.0, one of the most notabledesign choices is the closeness of blocks to text and the goal of allowing usersto transition to traditional programming. In fact, Alice is the first instance17Figure 2.2: The Alice 2.0 programming environment, which allowsusers to create animated 3D scenes using a mix of block-basedand menu-based programming.of a system that was not initially designed for block-based programming,but converted from text at a later point in time. Alice 2.0 was specificallydesigned to support the transition between blocks and text, and both use thesame underlying language. This programming language is object-orientedand similar to Java, but replaces some keywords with natural language andremoves others entirely, like parentheses and semicolons. Before execution,programs are converted into actual Java code, which modern versions ofAlice allow users to view as well [66]. Users can therefore follow the entirepipeline from blocks to simplified text-based code to traditional Java withina single programming environment.Despite its merits, Alice also serves as a first example of some of theinherent limitations of block-based programming. Alice is statically typedand aims to support object-oriented programming similar to Java. However,to be friendlier to learners, Alice aims to reduce the number of different typesthat users must consider. As Figure 2.2 illustrates, most commands in Aliceare method calls on objects in the current scene, using parameters with aset of primitive types like strings, numbers, colours and directions. Each ofthese blocks uses a different shape to indicate what its type is and whereit can be inserted into the surrounding program. However, Alice’s type18Figure 2.3: Definition of the expression “x > 0” in Alice via four levelsof nested drop-down menus.19system also supports user-defined classes and subtyping using inheritance.Therefore, an unbounded number of types can appear in an Alice program,so simple shapes cannot indicate which types are compatible.Alice overcomes the challenge of supporting object-oriented block typesby abandoning the block-based puzzle metaphor when composing values.Expressions that appear within these statements cannot be dragged anddropped into place like the statement blocks themselves, but must be se-lected using drop-down menus. These menus are automatically filtered toshow only values with legal types. Although this aspect of Alice’s designensures that users cannot compose ill-typed programs, it also leads to a lessconsistent and often tedious user experience than in most other block-basedlanguages. For example, as Figure 2.3 shows, users must navigate throughfour levels of nested menus to define a simple conditional expression like“x > 0”. Solving the visual challenges of supporting object-oriented pro-gramming remains an open problem with which even modern block-basedlanguages struggle, as we describe in Section 2.1.3.2.1.2 Scratch as the Prototype for Modern Block-basedProgrammingTo this day, the Scratch programming environment remains the most widelyused block-based programming system. Figure 2.4 shows a screenshot of amodern iteration of Scratch. The left side of the environment shows thetoolbox from which users can drag blocks into the programming canvas inthe center. When blocks are attached to a trigger block like the “whenspace key pressed” block in Figure 2.4, they become an executable programthat runs whenever the corresponding event occurs. Most blocks containcommands that manipulate a 2D sprite on a canvas on the right side ofthe environment, but they can also execute arbitrary code such as playingsounds or showing prompts to the user.Scratch is not directly based on an existing text-based language, but itfeatures a wide variety of commonly used programming constructs like vari-ables, conditionals, loops and procedure-like custom blocks. Like LogoB-locks, Scratch’s authors clearly name the text-based programming languageLogo [17] and the underlying philosophy of constructionism [60] as theirprimary inspiration. Following the ideas of constructionism, the creators ofScratch name tinkerability as one of their fundamental design principles [4].By calling Scratch tinkerable, its authors mean that the environment encour-ages experimentation: since Scratch is designed to prevent any syntacticalerrors, it allows users to run their code at any point in time. This way,20Figure 2.4: Modern version of the Scratch environment that runs ina web browser and allows users to create sprite-based 2D ani-mations.they can test programs even if they are incomplete. In addition, Scratch ig-nores unused code blocks that are disconnected from a program. Therefore,users can quickly swap out commands without having to explicitly delete ordisable them like in text-based languages.Another design principle of Scratch is simplicity, which is similar toBridgeTalk’s goal of providing high-level abstractions. For example, Scratchoffers users a separate block to change the coordinates of a sprite to an abso-lute value or to a relative one based on its current position, and also providesseparate blocks for changing the x-coordinate, the y-coordinate or both coor-dinates. All of these commands could be handled by a single block and somearithmetic computation, but Scratch aims to provide easier to understandalternatives. Compared to previous languages, Scratch also adds features tosimplify its language for beginners: It is the first language to consistentlylabel all of its blocks in natural language instead of typical programminglanguage syntax. It is also the first language to provide automated hygienefor names. When users in Scratch rename variables and function blocks, the21Figure 2.5: Synchronization as proposed by the authors of Scratch [4],recreated in a modern version of the environment. Users needto manually define and update variables that can be used in“wait until” blocks.environment automatically finds and renames references to these blocks aswell. Most professional code editors provide this feature of hygienic renam-ing as a refactoring [67], but Scratch makes this way of managing names thedefault.Despite their efforts to make Scratch simple to use, Scratch’s authorspoint out that their language is sub-optimal in some regards. Scratch pro-vides almost 100 primitive blocks, and despite efforts to organize them intocategories, the list of options can easily overwhelm users who are new tothe environment. Restricting the number of primitive blocks is essential forblock-based languages since, unlike text-based languages, they typically ex-pose users to a list of all the syntactic options available to them. Scratch’sauthors recognize this issue in their original presentation of the environ-ment and suggest that the number of primitive blocks should be minimizedif possible. They also pose block selection as an open problem that couldbe targeted by future work. However, the number of available blocks inScratch, as well as the design of its toolbox, have not changed significantlyin the years following its initial release.To demonstrate how hard it can be to find a small, expressive set ofprimitive blocks, consider a Scratch feature that was prominently presentedby its authors from the start and is still present to this day: concurrency.Scratch allows users to write multi-threaded programs, for example by cre-ating multiple triggers that react to the same event. It also provides a block22to broadcast a message to other threads, and a block to pause a thread untila specific message is received or a condition is satisfied. Scratch’s authorsclaim that these blocks are expressive enough to program a wide range ofconcurrent programs. For this reason they have intentionally refrained fromadding further, higher-level primitives for concurrency. However, in prac-tice it is often non-trivial to synchronize threads in Scratch by using just theavailable primitives. For example, Figure 2.5 shows how Scratch’s authorspropose to synchronize two concurrent threads using a lock. This solutionnot only adds substantial clutter to the code but also requires users to befamiliar with variables. Experienced programmers might also believe thatthe shown solution could still suffer from a race condition if a thread switchoccurs between the “wait” and the “set busy to 1” block. Only with ad-ditional knowledge about Scratch’s concurrency semantics, which restrictsthread switches to specific points of a program, does this example becomefully comprehensible. This feature of Scratch is not only difficult to learn,but can actively mislead learners if they believe that the lock shown in Fig-ure 2.5 is thread-safe in all programming languages.2.1.3 Modern Block-based Programming Beyond ScratchScratch was initially released as a stand-alone desktop application and quicklyattracted interest from a wider audience. Google’s Blockly framework, re-leased in 2012, creates a block-based environment similar to Scratch di-rectly in a web browser [34]. Blockly was originally designed to compileblock-based programs to JavaScript code, which it can run directly fromwithin the programming environment. However, it also supports other tar-get programming languages and can therefore be used to program almostany system as long as there exists an infrastructure to compile or run theproduced code. Blockly is still under active development today and mostblock-based environments, including modern Scratch, use it as their techno-logical foundation.A popular alternative to Scratch is Snap!, released in 2013 by UC Berke-ley [49]. Snap! replicates most of the features of Scratch, but also adds ad-vanced programming language features to the system. Most notably, Snap!supports collections, higher-order functions, basic object-oriented program-ming, and continuations. Snap! is primarily used in supervised educationalsettings such as college classes [68]. Therefore, although it demonstratesthat block-based languages can support advanced programming languagefeatures, the environment itself does little to explain their purpose to begin-ners. Learners probably cannot understand features like continuations just23Figure 2.6: Example of type coercion in Snap!: The function suppliedto “map” requires two arguments instead of one, but instead ofraising an error, Snap! duplicates the first argument to fill bothholes.by tinkering with the blocks, and Snap!’s built-in documentation even statesthat “continuations are too complicated to explain in a help screen.”Snap!’s direct translation of advanced programming concepts into blocksexhibits further drawbacks regarding how types are represented. Snap!’ssupport for lists and higher-order functions exposes users to an unboundednumber of types that can occur in a program. Some later work has pro-posed designs for representing a wider range of types in blocks [69], butSnap! does not use any comparable system. Instead, it uses a single shapefor all types and, unlike Alice or Scratch, does not prevent users from as-sembling ill-typed programs. Snap! attempts to convert types implicitly asprograms execute, which avoids frustrating errors in some cases, but canalso lead to unexpected semantics. This design decision does not necessarilyhelp beginner programmers, as studies have found that similar behaviourin traditional languages like JavaScript can confuse even professional de-velopers [70]. Take for example the program in Figure 2.6, which uses thehigher-order function “map” that expects a function with one parameteras its argument. When the built-in addition operator, which requires twoarguments, is inserted instead, Snap! attempts to infer the user’s inten-tion. The output of the program suggests that Snap! fills all argumentsof the addition operator with the list argument. Although this allows theprogram to run and the results might even match the user’s expectations, itis not obvious that this way to use the “map” function is not the intendedone. Therefore, beginners might inadvertently learn a wrong lesson whentinkering with Snap!’s features.Although Snap!’s implicit type conversions are intended to avoid user-facing error messages and the resulting frustration, users can still writeSnap! programs that produce error messages. These error messages arenot created by Snap!’s user interface, but by the JavaScript engine that24Figure 2.7: Example program in AIA with an error that prevents com-pilation. A colour expression must be connected to the programto fix the error, but this is not obvious to users as other expres-sion blocks look identical.executes the code that Snap! generated. Therefore, they refer to codethat is not actually visible to users and can be hard to interpret, especiallyfor beginners that are only familiar with Snap!’s block-based language andnot JavaScript. The existence of syntax errors further violates the designprinciple introduced by Scratch that every program that can be created isexecutable. Snap! therefore foregoes many of the benefits of Scratch byfocusing on features over design consistency.The final block-based programming environment we discuss in this sec-tion is App Inventor for Android (AIA). This popular block-based program-ming system primarily targets adult learners and allows them to write mobilephone apps [48]. AIA provides both a graphical editor for the app’s user in-terface and a block-based editor for the app’s functionality. AIA particularlyfocuses on making programs tinkerable by supporting live programming [71].By using a special companion app, users can test their apps immediately ontheir phone and even inject new code while the app is executing.Like Snap!, AIA supports several programming language features be-yond Scratch, such as collections and objects. AIA takes a more measuredapproach to adding features, such as omitting higher-order functions andproviding no way for users to define custom classes. It also does not per-form any implicit type conversions, but instead enforces type correctnesswhen blocks are attached to one another. However, AIA does not visuallyrepresent block types like Scratch or Alice. This approach can lead to con-fusing situations as demonstrated by Figure 2.7: The left block has an emptysocket that must be filled with an appropriate value. AIA indicates this us-ing a red circle and a tooltip. However, all three blocks on the right look likethey could fit in the empty socket, despite only one having the correct type.Users might try to attach the others, which seems visually possible but isnot supported by the editor. This underlying type check that explains this25mismatch is completely hidden from the user. Furthermore, a user cannottest a program before all errors in the program are fixed. Even when AIAprograms are free of compile-time errors, they can still raise run-time errors,which often refer to the compiled Java code instead of the blocks. Thisbehaviour can cause frustration and limits how easily users can experimentwith their programs in practice, despite support for tinkering being amongAIA’s primary goals [71].2.1.4 A Language-based Comparison of Block-basedProgramming LanguagesGuided by the languages we presented and discussed in the previous sections,we have identified a list of shared properties of most modern block-basedlanguages. In Table 2.1 we show a summary of these properties and whetherthe languages succeed or fail at achieving them. We do not list Blocklyin Table 2.1, since it is intended as a framework to develop block-basedlanguages rather than a language itself. Its most significant contributionis to enable block-based programming in a web browser without installingadditional software. AIA, Snap! and more recent versions of Scratch areweb-based, but we omit this property in Table 2.1 since it is less relatedto language design than the environment’s implementation. We also omitthe simplicity of the language’s syntax. This property is highly relevantwhen designing block-based languages, but we believe that it is possible toperform a meaningful comparison between the ones we discussed becausethey serve vastly different purposes. For example, LogoBlocks uses fewerdifferent blocks than any of the more recent languages but is also limited toa very specific domain.As the direct comparison shows, early block-based languages have con-sistently improved upon each other. Scratch combined the strengths of itspredecessors and was the first environment that combined their desirableproperties with the exception of supporting complex data types. Based onthe languages we have presented, it appears that supporting complex datatypes and representing those types as shapes are mutually exclusive proper-ties. Alice is the only language that achieved both, but at the price of usingdrop-down menus instead of drag and drop to assemble blocks. Remark-ably, the more recent environments also forego some desirable properties,like programs being always executable when they can be assembled in theeditor. This might suggest that modern languages consider functionality tobe as or even more important than beginner-friendliness or consistency withtheir own design principles.26Feature BridgeTalk LogoBlocks Alice Scratch Snap! AIAIntegration of language editor and compiler ✓ ✓ ✓ ✓ ✓ ✓Deep nesting of blocks is possible X ✓ ✓ ✓ ✓ ✓Syntax-directed editing using drag and drop X ✓ (✓) ✓ ✓ ✓Support for complex data types X X ✓ X ✓ ✓Shapes accurately represent compatible data types ✓ ✓ ✓ ✓ X XAutomated hygiene for all names X X X ✓ ✓ ✓Programs are readable like natural language X X X ✓ (✓) (✓)Programs are always valid and executable X X X ✓ X XImmediate feedback between editor and run-time X X ✓ ✓ (✓) (✓)Table 2.1: Desirable block-based language properties and comparison of selected languages based on theseproperties. Languages are sorted in order of their first release. ✓/X= Language does/does not haveproperty. (✓)= Language partially has property; from top to bottom: Alice only supports drag anddrop editing for statements, not expressions; Snap! and AIA mix natural language and traditionalprogramming syntax; Snap! and AIA provide feedback, but error messages often refer to compiledcode rather than blocks.272.2 Visual ProgrammingText has always been the dominating modality for creating computer pro-grams. Although programming environments have evolved to provide visualaids and tools, a text editor usually remains the centerpiece of professionalsoftware development. However, visual programming environments are usedby millions of non-professionals. spreadsheet software likeMicrosoft Excel isone of the most popular programming tools for end-users, and its cell-basedprogramming paradigm can hardly be represented as plain text [72]. Creat-ing and representing programs visually is not a recent idea either: the visualgraphics editor Sketchpad from 1964 is often considered to be the first exam-ple of visual programming [73]. Since then, designers have created thousandsof other visual languages, many of which are highly domain-specific or targetaudiences that are not professional programmers [46].In the remainder of this section we give an overview of the domain ofvisual languages. As there exists a much larger body of work for visual lan-guages than for blocks, but only a small fraction is directly relevant for ourwork, we do not provide a detailed historical overview here. Such overviewscan be found in the multitude of surveys that have been conducted on visualprogramming over the years [46, 74, 75]. Instead, we focus primarily on howthe wide range of previous visual languages can be categorized, as well asthe strategies found in those languages that can be useful for our work. Sec-tion 2.2.1 presents this overview of previous work, and Section 2.2.2 discussesthe relationship between block-based programming and visual programming.2.2.1 Classification and Strategies of Visual ProgrammingLanguagesThe range of visual languages that are used for programming or have beenpreviously proposed is substantial. Visual programming in practice rangesfrom spreadsheet-based approaches like Microsoft Excel to flow graph lan-guages like LabView [76], and dozens of other language designs. In 2017,Erwig et al. performed the largest and most recent survey of visual languagesto date [46]. They surveyed 484 languages that were published between 1995and 2014, 340 of which they classified as visual programming languages. No-tably, Erwig et al. did not include any block-based languages in their study,as they do not consider them to be visual languages. We discuss this de-cision further in Section 2.2.2. The authors assigned a combination of 17tags to each language, which allowed them to also categorize languages thatcombine multiple design paradigms. By filtering languages based on cer-28tain tag combinations, they were able to identify higher-level patterns like“flowchart” or “spreadsheet”. Their results can be seen as an overview ofthe domain of visual programming languages.Erwig et al. initially distinguish VPLs according to syntax:• Graph languages that define objects and how they relate to each other,allowing programs to be interpreted as graphs. They can be distin-guished further based on typical graph properties like whether edgesare directed or which graph components are labeled.• Partition languages that divide space into regions, suggesting a ge-ometric interpretation. The survey sub-categorizes these languagesbased on if they are open or closed, the former meaning that they allowspace to remain undivided as ”background” (as in a Euler diagram)and the latter, meaning that they require a division of all availablespace (as in a spreadsheet).The general categorization resembles that of other authors [74, 77]. Thesurvey found that 91% of the languages can be classified as graph or partitionlanguages, with 36% being only graph-based and 23% being only partition-based. It also found that the number of unclassified languages decreasedover time, showing a trend towards standardization. This standardizationalso means that the binary categorization suffices to describe the majorityof the recent visual programming languages.In addition to syntax, Erwig et al. classify languages semantically as:• Graphical or external based on whether their semantic domain matchesthe graphical representation used in the language or not. A languagethat displays image manipulations visually is considered graphical, asare spreadsheets since numbers and formulas are their semantic do-main.• Static or dynamic, depending on whether the language models a datastructure or describes computation.Erwig et al. remark that although the division into static and dynamiclanguages is ultimately the deciding factor of whether a language is a pro-gramming language, categorizing a language can be rather subjective. Forexample, there is often a duality between data structures and implicit com-putation embedded in them (e.g., state machines). For their survey, theauthors decided to classify all languages that could be seen as describingcomputation as dynamic. The survey found that 72% of visual languages29(a) SimuLink: flowchart language(b) Spider Diagram: eulerdiagram language(c) Excel: spreadsheet languageFigure 2.8: Examples of visual languages in different categories.are purely external and 21% purely graphical, and that 70% of languagesare dynamic and 30% are static. These findings indicate that the majorityof visual languages are indeed programming languages, and that most areused in an external (i.e., not domain-inherent) visual context.By combining their tags into higher-level categories, Erwig et al. foundthat the majority of their studied corpus of data is made up by a small setof visual language types. Figure 2.8 shows an example language from eachof these categories:• 41% were classified as flowchart languages that describe graph-basedcomputation,• 25% were identified as tabular languages that use closed partitioning(of which 15% are spreadsheet languages that also use formulas),• and 11% were categorized as Euler diagram languages that use openpartitioning and labels (of which 6% also use graph-based elements).30Erwig et al.’s classification can provide an overview of what types ofvisual paradigms exist in the wild, but not why language authors have chosenthese specific designs. Assuming that most visual languages were designeddeliberately and carefully, it seems reasonable to assume that their overallgoal was to support their users in a way that goes beyond the capabilitiesof pure text. However, visual languages are not the only means to achievethis goal. Statecharts [78] and the more modern Unified Modelling Language(UML) [79] are just two examples of visualizations for program structuresand behaviour that were designed to aid developers during the programdesign process. More subtle visual aids like syntax or error highlighting areeven more common, and found in almost every modern editor. This raisesthe question of whether there is a fundamental difference between programvisualization and visual programming.Burnett [23] described four strategies that visual languages use to sup-port the programmer. Whether, and to what extent, these strategies arepresent in visualizations can be used to distinguish visual programming fromother uses of visualization:1. Concreteness means presenting data to the user in a way that they arefamiliar with or that matches their mental image without the need toabstract from it. For example, a visual language could represent a 2-dimensional object concretely by drawing it instead of listing geometricproperties.2. Directness aims to enable manipulation of data in a way that resem-bles the user’s intention as directly as possible. For example, a visuallanguage could allow users to change the position or size of a geo-metric object via drag and drop instead of editing a numeric “height”property.3. Explicitness tries to explicitly present semantic information to the userwithout the necessity to infer data or perform mental calculations.For example, a visual language could visualize a relation between twoobjects by drawing an explicit connection between them instead ofmaking the user infer that connection themselves based on textualdescriptions.4. Liveness attempts to give the user immediate and automated feedbackabout any modification to the program. For example, a visual languagecould reflect any updates to an object’s properties immediately whenthe change occurs instead of waiting for the user to manually refreshthe program.31Not all visual programming languages use all of the aforementionedstrategies to the same extent, but missing a strategy entirely can be anindication that a system does not fall into this category. For example, syn-tax highlighting does not increase the concreteness or directness of text, andis therefore better categorized as a visual aid and not as a visual language.Program visualizations like UML on the other hand do not use directnessor liveness and should therefore not be considered languages themselves.2.2.2 Block-based Programming and Visual ProgrammingSurveying work on both block-based programming languages and visual pro-gramming languages reveals an interesting discrepancy between the two do-mains: block-based programming language creators and those who researchblock-based languages frequently classify them as visual programming lan-guages and therefore see themselves as part of this domain [4, 28, 80]. How-ever, Erwig et al.’s survey of visual languages does not consider any block-based languages to be part of this domain [46]. It is not difficult to imagineblock-based languages as a graph-based language that uses blocks as nodesand their connections as edges. In fact, previous work has used such a repre-sentation to statically analyze block-based code [81]. However, at the sametime blocks are intentionally similar to traditional text, and can be seen asa pure visual aid to make editing or reading programs easier for beginners.We investigate this disagreement further in the remainder of this section.To understand why researchers might not consider block-based languagesto be visual programming languages, it is useful to apply Burnett’s classi-fication of visual languages [23]. We discuss this classification in detail inSection 2.2.1, but as a reminder: the four strategies that visual languagesshould apply are concreteness, directness, explicitness and liveness.At first glance it can be challenging to see where block-based languagesapply any of the above strategies. Some languages create blocks that rep-resent graphical sprites (e.g., Scratch, Snap!) or other objects (e.g., Alice,AIA), but all of the properties of the sprites and objects are representedas plain text instead of a more concrete form.1 Furthermore, none of thelanguages use techniques, like bi-directional editing [82], that would allowusers to manipulate sprites or objects directly and reflect these changes inthe block-based program. However, block-based languages do employ someVPL strategies in the context of the syntax of the underlying text language.For example, the puzzle shapes of blocks explicitly visualize the syntax of1One notable exception are colours in Scratch, Snap! and AIA that are represented asa coloured square instead of a name or RGB values32the underlying language and give users a more concrete way to see how theycan combine primitives. In languages where data types are accurately rep-resented by shapes, one could also argue that blocks make the language’stype system more manifest. Scratch and AIA apply another VPL strategywhen they emphasize the idea of tinkerability: tinkerability is similar toliveness in that it allows users to quickly switch between programming andtesting and get immediate feedback on their code. Nonetheless, the block-based languages we investigated only partially match the definition of VPLsabove. It is therefore understandable why researchers disagree on whetherthey consider block-based programming to be part of the VPL domain.When comparing block-based and visual languages, one may ask thequestion: why do block-based languages not apply more of the VPL strate-gies? We believe that the answer to this question reveals the most funda-mental difference between visual and block-based languages: modern block-based programming, which was heavily influenced by languages like Scratch,was designed with the goal to closely resemble text-based programming. Asmost language designers intend blocks to be used in educational settings,they want users to have an exit strategy [29] and be able transfer theirknowledge when they transition to text-based languages. However, it is im-portant to note that this overall goal does not always align with the waysthat block-based programming is used today. For example, languages forpersonal and industrial use cases may not even have a text-based equivalentthat users could learn after blocks [7, 33].In addition to pedagogical reasons, block-based language developers gainbenefits from keeping their languages similar to text: it simplifies their trans-lation to text-based code. In all block-based languages we have investigatedin our review, it is straight-forward to strip away the visual layer and treatthe text contained in the blocks as code. In many cases, blocks can be com-piled to textual code line-by-line. However, even if the translation is moreelaborate, block-based languages form a tree that approximates an AbstractSyntax Tree (AST). There are substantially more and mature frameworksand techniques to transform and generate code from ASTs than for any vi-sual language formalisms. Language designers can benefit from these toolsand create new languages faster and with a focus on design aspects ratherthan implementation constraints. As a consequence, there are even efforts tomake block-based language development itself accessible to non-professionalsby providing block-based design tools [83, 84].33Figure 2.9: ABB RobotStudio: A modern development system forprofessional robot programmers [5] that features a complex sim-ulator and development environment.2.3 Domain of Inquiry: End-user RoboticsProgrammingTraditional programming environments for industrial robots are complexand difficult to use. Figure 2.9 shows RobotStudio [5], one such system,that targets professional developers. RobotStudio provides code editing andproject management tools, similar to modern Integrated Development Envi-ronments (IDEs). It further allows users to test their programs through acomplex 3D simulator, and via debugging tools that are integrated with realrobot hardware. Other robotics IDEs, such as the ROS Development Stu-dio [85] provide similar features and a have comparable level of complexity.Robotics IDEs like RobotStudio provide a substantial amount of featuresto support their users. However, these features were designed with users inmind that have a thorough understanding of both programming in generaland in the context of robotics. RobotStudio uses the propriety languageRAPID [86], while other tools provide frameworks that build on text-basedgeneral purpose programming languages like C++ or Python. They allhave in common that they were designed to give programmers low-level34control over the robot they target, sacrificing usability and learnability forthis purpose.Researchers and industrial developers have both tried to make robotprogramming accessible to end-users as well. They typically propose newprogramming systems along the spectrum we presented in Figure 1.1, us-ing text-based, block-based, visual or fully automated programming ap-proaches [16, 87]. Automated approaches, which aim to eliminate the needfor traditional programming altogether, are particularly popular in robotics.For this reason, Biggs and MacDonald’s survey of programming systemsfrom 2003 sorted all robot programming systems into just two categoriesmanual or automatic [88]. The category of manual systems includes all toolsthat use any form of visual, block-based or text-based programming. As thesurvey did not explicitly focus on end-user systems, professional roboticsprogramming languages like RAPID [86] are also covered by this category.Automatic tools strive to eliminate the need for traditional program-ming by using alternative modalities, such as lead-through programming [89]or demonstration-based learning [90]. In lead-through programming, usersphysically guide a robot arm by moving it along a path that is then recordedand replayed when a program is executed. This form of programming is idealfor end-users as it is very direct and requires little training. However, it islimited to simple programs and collaborative robot hardware, which is safeto be used around humans. Demonstration-based learning is conceptuallysimilar, but does not require users to directly interact with the robot armthat executes their program. Instead, users record the intended motion usinga teach pendant, a different robot model, or gesture recognition. Althoughthis form of programming is still easy to learn, it is less direct and oftenless precise [91]. A number of recent systems have been proposed to solvethis issue, for example by using mixed and virtual reality [92–94], but theirpractical benefits remain untested.Compared to automated programming, manual systems typically sup-port a wider range of programs and do not require specific robot hardware.However, like other end-user and educational systems, they must make us-ability trade-offs based on their intended audience. For example, the block-based programming tools Open Roberta [44] and BEESM [39] target learnerswith no experience in robotics. They provide high-level commands and basicsimulators to test programs, which makes them similarly easy to learn anduse as other block-based tools such as Scratch. However, the functionalityand range of robots that they support is limited, and they face similar issuesas other block-based tools that we discussed in Section 2.1.3. Other man-ual systems, such as the RC+ Express [95] and Polyscope [6], propose more35Figure 2.10: Polyscope: a simplified robot programming environmentwith visual features for intermediate users [6] and a less com-plex simulator interface.complex programming environments and target intermediate users with pro-gramming experience. Figure 2.10 shows Polyscope, which features a simplesimulator and a tree-based visual programming language.A particular system that has influenced our work is CoBlox, a block-based programming environment for collaborative robot arms [7]. CoBlox,shown in Figure 2.11, combines manual and automated robot programmingin a single system. Users can use a block-based programming environmentto define the overall structure of their programs and then use lead-throughprogramming to define the target position for each arm movement. CoBloxis embedded into RobotStudio and can target both real robots and the samesimulator as programs written in RAPID. An evaluation by the authors ofCoBlox found that end-users can learn how to use CoBlox faster and solvesimple programming tasks more effectively than with other commerciallyavailable tools [36]. Although this initial evaluation was performed usinga robot simulator, a later evaluation has confirmed that end-users can alsoprogram real robots using CoBlox after completing a short training [47].Despite its promising initial evaluations, CoBlox is inherently limited36Figure 2.11: CoBlox: a block-based robot programming environmentfor end-users [7] integrated into ABB RobotStudio. It onlysupports rudimentary commands, but is easier to use thanany of the other shown systems.in its feature-set. It only supports linear programs that contain a numberof movements and target a single robot arm. Features like variables andloops are supported in theory, but CoBlox does not provide any support orinstructions for end-users on how to use them. They were also never used inany of the evaluations of CoBlox [36, 47], and all used tasks were only a fewblocks long. However, we see the positive results of its usability evaluationsas initial evidence that block-based programming is suitable to support end-users as they program industrial robots. This work has motivated us toinvestigate how block-based robotics can support a richer set of tasks.2.4 Cognitive DimensionsSome areas of VPL research are relevant to block-based languages, regard-less of how one considers the relation between the two domains. One streamof research that is particularly useful is the framework of 13 Cognitive Di-mensions of Notation (CDN) [1, 2], which can be applied to both BBPLsand VPLs. Table 2.2 lists the 13 dimensions introduced by the CDN frame-work, as well as a brief description of the language aspect that each of themaddresses. Since its inception, the CDN has been used to rationalize and37Dimension DescriptionAbstraction Gradient What levels and forms of abstraction does thelanguage support or require?Closeness of Mapping How directly does the language represent thenotation of the underlying domain?Consistency How difficult is it to infer parts of the languagefrom knowledge of other parts?Diffuseness How many different elements (e.g., keywords,syntactic rules, visual elements) make up thelanguage or its visual representation?Error-proneness To what degree does the language’s designcause or prevent user errors?Hard Mental Operations How complex is the mental model that a userhas to maintain as they read or write programsin the languages?Hidden Dependencies How difficult is it to identify which parts of aprogram relate to or depend on each other?Premature Commitment Does the language require users to make deci-sions (e.g., on program structure) before theyare aware of all relevant information?Progressive Evaluation Is it possible to evaluate or receive feedbackon partially complete code?Role-expressiveness How clear is the structure of programs and thepurpose of individual code fragments?Secondary Notation Are there ways to annotate or structure codeinformally, beyond the language’s syntax?Viscosity How much effort is required to make changesto existing programs?Visibility How difficult is it to read a program, and nav-igate it if does not fit onto a single screen?Table 2.2: All dimensions introduced by the 13 Cognitive Dimen-sions of Notation framework [1]. Descriptions are derived fromGreen et al.’s original definitions [2] and adapted for the purposeof evaluating (visual) programming languages.38and compare the designs of over 500 visual notations and languages [96]. Re-cent work has also applied the CDN to block-based languages [97, 98] andfound it particularly useful when evaluating differences between block-basedlanguages and trade-offs in their designs.Some dimensions of the CDN can be directly mapped to block-basedlanguage terminology. For example, the dimension of whether a languagesupports Progressive Evaluation is highly related to the idea of a language’sliveness or tinkerability. Other dimensions describe problems that are par-ticularly simple to solve in block-based languages due to their close relationto text-based program code. For example, all block-based languages sinceAlice that we investigated address the dimension of enabling Secondary No-tation by allowing users to add comments to their code. However, somecognitive dimensions are particularly useful for discussing the benefits andlimitations that arise from block-based language design choices.Take Error-proneness as one example of a cognitive dimension that canbe applied to multiple aspects of block-based languages. Error-pronenessmeasures the degree to which a notation leads users to make mistakes. Thatblock-based languages have a less error-prone syntax is one of the most ob-vious benefits compared to textual languages. Modern environments useadditional measures to avoid errors on a visual level. For example, Scratchand most other recent environments automatically align blocks so that dis-connected blocks cannot overlap or be closely adjacent to one another, sousers do not misinterpret them as being connected [29]. However, differentblock-based languages prevent or discourage semantic mistakes to signifi-cantly degrees. For example, all block-based languages since Scratch thatwe investigated support the capture-avoiding renaming of variables, but onlyAIA warns users when they try to use variables outside of scope [97]. As an-other example, Snap! automatically attempts to convert mismatched blocktypes, similar to runtime type coercions in languages like JavaScript. Justlike type coercions, this feature that can simplify some programs but is alsoknown to cause hard-to-predict behaviour that often leads to bugs [70].We use the CDN primarily because of the terminology it provides forcomparing visual programming language features and related notation. Inlater chapters, we refer to the dimensions listed in Table 2.2 without furtherintroduction, but we highlight them to clarify that we refer to terms of theCDN.39Chapter 3Block-based Synchronization:Coordination of Two RobotArmsOne of the challenges of industrial robot programming is controlling how arobot interacts with its environment. In traditional systems programming,the preferred strategy to control such interactions is to hide the details ofthe environment behind abstractions as much as possible. This is possibleto some degree for robot programming as well, for example when it comes tocontrolling physical parameters that control the robot’s motors or sensors.However, a robot’s high-level interactions with its environment, like how itfinds target locations, picks up objects, or interacts with machines, are oftenat the core of robot programming and cannot be hidden from the user.A specific case in robot-environment interaction is that of two robotsworking cooperatively. When multiple robots interact with one another,they can solve tasks more effectively than if each was performing work in-dependently [99]. However, programming this interaction is difficult, evenfor professionals: similar to when processes and threads interact in tradi-tional system design, the individual programs must be carefully ordered toprevent race conditions and unexpected robot behaviour. At the same time,traditional strategies to present and teach parallel programming [100] canbe difficult to transfer to robots. The state of a program, in particular, canbe challenging to reason about in robot programming, as interactions do notjust affect isolated variables, but can change the entire physical state of therobot’s environment.Block-based programming languages like Scratch or Alice support con-currency similar to professional programming languages, but do little tomake it easier to understand for beginners. In Section 2.1.2, we have alreadyillustrated how it is possible to synchronize two Scratch threads through alock. In this chapter, we revisit this style of synchronization and providean overview of other approaches, including some from the broader areas40of visual programming and program visualization. We then present ourown contribution: a number of novel designs to visually represent programcoordination in block-based languages, and their analytical and empiricalevaluation. The work is based on the papers “Comparing Block-based Pro-gramming Models for Two-armed Robots” (published in Transactions ofSoftware Engineering in 2020) [101] and “Side-by-Side: A Case Study En-abling Parallel Programming of Robotic Arms by End-Users” (submitted tothe International Conference of Software Engineering 2024) 2. In the firstpaper, we explored four candidate design options that all display two block-based programs side-by-side and visually connect them at points where theyinteract. The four designs differ with respect to their synchronization modeland program flow presentation. We conducted a controlled survey of 273professional users (110 without previous programming experience) to deter-mine which one is the easiest to understand and which one our participantsprefer. We then implemented a front-end prototype of the best design andconducted a smaller interactive experiment with 11 industrial participantsto confirm that it is viable in practice. The second paper went beyondfront-end experiments and evaluated a fully functional environment proto-type that controls a physical two-armed robot. We found that participantswere substantially more successful at solving realistic assembly tasks usingour block-based system than when they used a text-based, commerciallyavailable tool with similar functionality.3.1 Coordination and Parallelism in End-userLanguagesParallel programming is typically considered an advanced topic in computerscience education. It is only taught after students have mastered sequentialprograms. This path to understanding parallelism is not suitable for end-users who have a specific task in mind that they want to solve. They mustbe able to learn sequential and coordinated programming at the same time.Several block-based languages try to support parallel programming andsimplify it to some degree. However, to our knowledge, no previous block-based work has attempted to visualize parallelism or represent it in a waythat goes beyond the capabilities of text. Therefore, they do little to support2The author of this dissertation is not the first author of the second paper, but hascontributed to the study design, evaluation and presentation of the results. We focusprimarily on the first paper in this chapter, but summarize the findings of the secondpaper in Section 3.4.2.41end-users as they reason about which parallel program (inter)actions occursimultaneously. We present related block-based work and discuss its limita-tions in Section 3.1.1. Some prior work attempts to support parallelism inthe visual programming language space, but none of those we surveyed area good fit for enabling end-users to coordinate multiple robots. We presentand discuss those attempts in Section 3.1.2. Finally, we briefly introduceRobotStudio Online YuMi, a commercially available, primarily text-basedtool that has inspired our work and that we used as a baseline for our eval-uation in Section 3.1.3.3.1.1 Coordination in Block-based LanguagesPrevious block-based systems use a variety of techniques to make coordi-nated parallel programming beginner-friendly. In Chapter 2, we introducedthe block-based languages Scratch and Alice, both of which support paral-lelism primarily in the context of animation. As presented there, Scratchsupports the parallel execution of multiple triggers. Figure 2.5 illustratesthe synchronization of two such triggers: similar to traditional thread syn-chronization, users can set up a simple lock and make the threads wait forthe lock to be available. An expert in systems programming might expectsuch a lock set-up and find it easy to read and understand. However, for anend-user who might still be struggling with more fundamental programmingconcepts like mutable variables, this program can be confusing and distractfrom the actual commands that are executed once the synchronization hasoccurred. Scratch’s approach is also error-prone, as end-users must eitherreplicate the lock (e.g. after looking up such a set-up in a tutorial or onlinereference) or devise their own, potentially flawed, implementation.Alice proposed a different technique for synchronization that requires lesslow-level programming. Therefore, it might be a better fit for beginners. Itfeatures “do together” blocks that enable parallel animations [61]. Thistype of block, shown in Figure 3.1, allows users to animate multiple objectsat once by placing commands inside a “do together” block. To implementmore complex sequences of ordered and parallel animations, users can nest“do together” and “do in order” blocks. At the end of each block, Aliceautomatically ensures that all nested blocks have finished executing.Both Scratch and Alice provide general-purpose abstractions for paral-lelism that can be used for different scenarios across multiple domains. How-ever, neither Scratch nor Alice provide users with visual feedback or scaffold-ing to clarify which commands are executed synchronously or in which order.In contrast to Scratch and Alice, the Parallel Snap! [102] environment for42Figure 3.1: Parallel programming in Alice using a “do together” blocknested with “do in order” blocks.programming distributed systems focuses on domain-specific high-level ab-stractions such as MapReduce [103] or the producer-consumer pattern. Thisfocus allows the environment to make these specific abstractions for paral-lelism very end-user friendly and easy to understand. Parallel Snap! alsouses animated visualizations to demonstrate parallelism to users. However,Parallel Snap! does not integrate visual elements directly into the block-based programming language. Users must execute their programs to see theeffects of the parallel computation. Parallel Snap! therefore extends block-based programming with novel program visualizations rather than visualprogramming features.3.1.2 Coordination in Visual LanguagesVisual programming languages aim to simplify parallel programming as well.In particular, visual data-flow languages, which are often purely functional,have explored the parallel computation of data [74]. As data-flow languageslike VIVA [104] or CODE [105] demonstrate, a compiler or interpreter forthese languages can compute data dependencies automatically and coordi-nate parallel computation without any additional user input. However, thistechnique does not generalize to other visual languages, even those that arerepresented by a flow graph.One example of a graph-based visual robotics language with more com-43Figure 3.2: Parallelism in Lego Mindstorms EV3. The sole purposeof the six blocks highlighted in yellow is to synchronize the topand bottom row.plex coordination is Lego Mindstorms EV3. This language, which representsprograms as a control-flow graph, allows each node to have multiple outgoingedges. All nodes connected in this way execute in parallel [106]. Figure 3.2shows an example program that uses parallelism in EV3. Both the top andthe bottom row of the program are executed concurrently.As Figure 3.2 illustrates, EV3’s approach allows users to arrange theirprograms to look like a timeline from left to right or top to bottom. How-ever, this requires that the users manually arrange each node on the canvasto match the intended program flow. To ensure that two commands executesimultaneously, users must manually synchronize them. They need to man-ually add a complex barrier construct (highlighted in yellow in Figure 3.2).Besides adding visual clutter, this also requires users to correctly use othernon-trivial language features like variables and loops.EV3 demonstrates that visualizing parallel computation alone is not suf-ficient to make coordination accessible to end-users. EV3 users might be ableto write concurrent programs, but they must manually combine low-level in-structions and ensure that they work reliably and as intended. This issue iscomparable to the one we previously discussed for the block-based languageScratch, where synchronizing multiple threads is also tedious and requiresa lot of low-level effort. However, since EV3 programs target robots, thisshortcoming becomes even more critical. For example, although handlersfor different user interactions in Scratch might not necessarily interact, mul-44tiple commands in EV3 that control the same robot must almost certainlybe coordinated explicitly.3.1.3 RobotStudio Online YuMiRobotStudio Online YuMi (ROY) is a programming interface created byrobot manufacturer ABB to control their two-armed robot YuMi. It is de-signed to demonstrate the range of tasks that two robot arms can solve whenthey cooperate. Therefore, ROY is specifically designed to be beginner-friendly and accessible to end-users who would be overwhelmed by tradi-tional tools like ABB’s RobotStudio [5].ROY’s programming approach is primarily text-based, but it uses a num-ber of graphical interface elements to make programming easier for begin-ners. As Figure 3.3 illustrates, ROY’s main interface contains three panels:two text-based program editors and a panel on the right with graphical but-tons. The two editors contain the program code for each robotic arm thatis targeted by the environment. Each program is written directly in theRAPID programming language, although boilerplate code, such as functionheaders and variable definitions, is hidden from the user. To assist users asthey write RAPID code, ROY provides a series of buttons on the right todefine commonly used robot commands. In addition to opening and closingthe grippers attached to each arm, some buttons capture the current posi-tion of each robot arm and generate a movement command that uses thecaptured position as a target. This way of capturing locations by directlymanipulating a physical robot is called lead-though programming and is com-monly used in end-user robotics systems [7, 90]. As users compile and runtheir code, the two programs are combined with their respective boilerplatecode and automatically deployed onto their respective robot arms.Although ROY’s graphical programming support appears to enable thetext-free creation of programs similar to block-based or visual programming,it has a major limitation. All of the described buttons can only be used toinsert new code at the currently selected line number. To edit existing com-mands, users must manually edit the text, and to change captured locations,they must first switch into a separate “Edit this arm” interface that showsonly one arm’s program and displays the normally hidden boilerplate codeincluding variable definitions. Existing variables can be overridden usingthis interface without having to manually re-write the program, but thisfeature is easy to miss and switching between different modes complicatesthe programming significantly.45Figure 3.3: RobotStudio Online YuMi programming environment fortwo-armed robots. The tool is primarily text-based, but has aset of buttons on the right side of the interface.3.2 Design Approach: Side-by-sideSynchronizationRobot programs can be coordinated in many ways, depending on the taskthat the user is trying to complete. In our work we focus on coordinatingtwo robot arms in two distinct ways:• Synchronous coordination - Some tasks require both robot armsto move at the same time and speed, although potentially in differentdirections. Examples of such tasks are carrying, turning, folding ortearing an item.• Asynchronous coordination - Other tasks require both robot armsto move independently, potentially waiting for each other at some pointbefore continuing. Examples of such tasks include handing an itemfrom one arm to another, and holding an item in place with one armwhile the other one stirs, screws, or welds.In this section we present our process of designing a programming lan-guage that is end-user friendly and supports both synchronous and asyn-chronous coordination. We focus on our analytical design considerationsand the decisions they motivated. For our design discussions, we use the46(a) Two-armed robot carrying anitem with two armsMove to Above Place Right .Move to Above Place Left .Open Hand.Open Hand.Move to Place Left . Follow movement.Move to Above Place Left . Follow movement.Move to Above Pick Left . Follow movement.Close Hand.Move to Above Pickup Left .Close Hand.Move to Pick up Left . Move to Pick up Right .Open Hand.Move to Above Pickup Right .Open Hand.When is pressed: (b) Block-based program for carryingan item with both armsFigure 3.4: Example of a block-based program mimicking the physicallayout of the targeted robot, with each arm’s program placedon the same side as the physical arm facing the user.terminology of the Cognitive Dimensions of Notations that we introducein more detail in Section 2.4, and we italicize and underline the respectiveCDN terms. We conducted several small-scale informal studies with noviceprogrammers to guide our design work, and we refer to them throughout thissection. Our larger empirical evaluation of the resulting designs candidates,is presented separately in Section 3.3.3.2.1 Showing Coordinated Programs Side-By-SideRobot programmers often must write code based on imprecise, high-leveltask descriptions. They need to simultaneously determine the necessarysteps for each robot arm and the necessary coordination between them. Forbeginners, this process can be challenging even for tasks where no coordi-nation is necessary. The first goal of our design process was therefore tofind ways for novices to effectively read and write code for two independentrobot arms.In early design drafts, we used a single, linear program where each com-mand had a parameter that specified which arm should move. We also triedusing separate Move Right Arm and Move Left Arm blocks. Both designsintertwine the movements of the two arms and force users to distinguishthem in their heads. This is a Hard Mental Operation. Furthermore, thesedesigns also reduce the Visibility of each arm’s individual actions.To improve our prototype, we took inspiration from the overall design47of ROY (shown in Figure 3.3). Displaying the programs for the two robotarms side-by-side not only uses the available screen space efficiently, butalso matches the perspective of the user when programming and testing.Figure 3.4a shows the users’ typical view of a two-armed robot. They arefacing the robot from the front with one arm on the left and one on the right.To connect both programs, we use a single trigger block that spans bothcolumns and has two tabs for successor nodes. This integrates well with theinterlocking jigsaw aesthetic of block-based languages. The resulting layoutcan be seen in the first 5 rows of Figure 3.4b.One important detail of the design shown in Figure 3.4b is that it requiresslightly different blocks for each arm. For example, the Open Hand blocksin the two columns must have their jigsaw tabs aligned either on the leftor the right of each block. Using different blocks for each arm means thatprogram fragments cannot be easily moved between arms. This leads to anincreased Viscosity of our language. One way to overcome this limitation isto automatically adapt the blocks depending on which arm it is assigned to,mirroring their layout as they get dragged across the center of the canvas.However, in our later prototype implementations we found this behaviourdifficult to achieve in practice. We therefore opted for a visually less cleanbut practically feasible design that does not center blocks around the middleand aligns them on the left for both arms.3.2.2 Synchronous Coordinated MovementsTwo columns are an effective way to present two independent programsside-by-side. However, our goal was to support coordinated tasks where therobot arms don’t act independently.Consider a task where the robot in Figure 3.4a is supposed to placethe held Lego piece on the table. This task requires both arms to movedownward simultaneously. This could be implemented by two side-by-sideMove blocks, one for each arm. However, a movement like this requiresprecision to make sure that the timing, speed and distance of both armsfit exactly. This increases the system’s Error-proneness. It also leaves thedependency of the two movements implicit, creating a Hidden Dependency .To simplify defining synchronous movements across arms, we introducenew block types that span both columns. The first type, which we refer toas a Follow block, is shown in Figure 3.4b. This block lets the user definethe target location for one arm and computes the other arm’s movementsautomatically. It makes both arms move in parallel by ensuring that theirdistance and speed are matched. The second new block type is a Mirror48block. It also matches the speeds of the arms but makes them move inopposite directions. This is necessary for tasks like folding of objects.An advantage of both new block types is that they fit well how beginnersthink about synchronous movements. Their designs and labels match theverbal descriptions that early participants intuitively gave for synchronousmovements. This suggests that the Closeness of Mapping between theseblocks and the way beginners think about tasks is beneficial for linking thecode to the physical domain.3.2.3 Timing and Synchronization of AsynchronousMovementsA typical robot program like the one in Figure 3.4b combines both syn-chronized and individual commands. When the arms move independently,the actions that are presented side-by-side may not always take the sameamount of time. Later commands may however require a specific order ortiming of arm movements. At some point, one of the two arms may need towait for the other one to achieve this timing.We showed our design draft as presented in Figure 3.4b to a small groupof novice participants. We explained to them that the robot might takesignificantly less time to open its hand than it does to move its arm. Wethen asked them about their interpretation of the timing of the followingcommands. The answers given by most participants suggested that theyread the program as a timeline. They assumed that each row of blocks,starting from the top and moving down, is executed simultaneously for boththe left and the right robot arm. This means that they expected the armsto wait if one is faster than the other.Based on this feedback, we drafted a design that implicitly synchronizesthe arms after each row of blocks. We added Wait blocks in cases whereonly one arm is required to move. These blocks ensure that all blocks areconnected in the jigsaw-style of block-based languages. An example programusing this design is shown in Figure 3.5a. It tells one robot arm to pick upan item and hand it to the other arm, and then tells the other arm to grabit and put it down.The design shown in Figure 3.5a has a significant limitation: since ev-ery block has exactly one counterpart in the other arm’s program, only onecommand can ever be executed simultaneously with another command. Ifa short command is combined with a long one, this can introduce unneces-sary wait times. An example for this is the first four rows of the programin Figure 3.4b. Although the first four commands for each arm require49Move to Above Place .Open Hand.Wait.Move to Place .Move to Above Place .Wait.Wait.Wait.Open Hand.Wait.Wait.Close Hand.Move to Pick up .Move to Above Pickup .Move to Handover A . Move to Handover B .Move to Above Pickup .Open Hand.Wait.Wait.Wait.Wait.Open Hand.When is pressed: Close Hand.(a) Design (I-V): after each com-mand, the arms implicitly wait for eachother.Move to Above Place .Open Hand.Move to Place .Move to Above Place .Wait for each other.Open Hand.Wait for each other.Close Hand.Wait for each other.Move to Handover A .Move to Above Pickup .Close Hand.Move to Pickup .Move to Above Pickup . Move to Handover B .Open Hand. Open Hand.When is pressed: (b) Design (E-V): explicit barrierblocks mark points of synchronizationFigure 3.5: Example programs using vertical flow. In both programs,one robot arm hands over an item to the other.no coordination, the arms would still wait for each other after every step.Therefore, the example program as it is written here wastes time wheneverarm movements take longer to execute than opening a robot arm’s hand.Efficiency could be improved by swapping the order of commands, but thismay not be obvious to beginners. There are also other cases where the orderof commands is relevant and the design is not expressive enough to writemore efficient programs.A second drawback of the presented design becomes visible in Figure 3.4b.If only one arm should be active for an extended period of time, multipleWait blocks must be inserted for the other arm. This is necessary to retainConsistency with the block-based jigsaw design, which does not allow gapsbetween blocks. However, it adds visual clutter to the design and increasesits Diffuseness. It can further reduce the Visibility of those command blocksthat describe active robot behavior.We considered ways to solve the problems of the presented design whilekeeping it readable like a timeline. One way is to modify the height of50blocks based on how long they take to execute. Another is to add indicatorsthat warn users about wait times. However, in practice it is not possibleto predict the exact length of each robot command. The same commandmight even take a different amount of time across multiple runs. We cannottherefore rely on this information.An alternative approach to synchronization can be found in traditionalparallel programming: barriers can force threads to wait until all of themhave reached the same point in program execution. They synchronize con-current programs explicitly. We have created an alternative design based onthis approach. It does not require blocks to wait for each other after eachrow, but uses Barrier blocks instead. Figure 3.5b shows the hand-overprogram from Figure 3.5a re-written in this design.A design with explicit barriers allows the execution of any number ofcommands in parallel. This makes it strictly more expressive than the designshown before in Figure 3.5a. It comes however with the drawback thatprograms cannot always be read as a timeline. Take the program fromFigure 3.4b as an example: when interpreted based on this design, it is notclear which of the first commands for each arm are executed simultaneously.As Figures 3.5a and 3.5b illustrate, programs may only need a small num-ber of Barrier blocks. This is especially the case since two-column blockslike Follow blocks also act as a barrier. Users must think about timingwhen it matters for the program’s correctness. This makes the Diffusenessof this design lower than for the previous design. However, this does notnecessarily make programs shorter, as Figures 3.5a and 3.5b show.The two presented designs that use implicit and explicit synchronizationboth come with individual trade-offs. An analytical approach cannot reliablydetermine which of the two design alternatives is easier to comprehend.Answering this question empirically motivated us to conduct the survey wepresent in Section 3.3.3.2.4 Vertical vs. Horizontal Program FlowOur preliminary studies indicated that beginners intuitively read side-by-side programs, such as shown in Figures 3.4b, like a timeline. This intuitionmight be influenced by previous experiences with widely available commer-cial tools for audio, video or animation editing. These tools use similartime-line visualizations and show events occurring in parallel on multipletime-aligned tracks.Unlike our previously presented design drafts, most other tools presenttime as flowing horizontally. We are not aware of research that investigated51➟To: Above Place➟➟➟➟Pick-upTo: Hand-over ATo: Hand-over BTo: PlaceTo: ➟AbovePick-upTo: ➟AbovePick-upTo: ➟To: Above PlaceLR(a) Design (I-H): after each command, the arms wait for each other➟To: Above Place➟➟➟➟Pick-upTo: Hand-over ATo: Hand-over BTo: PlaceTo: ➟AbovePick-upTo: ➟AbovePick-upTo: ➟To: Above PlaceLR(b) Design (E-H): explicit barrier blocks mark points of synchronizationFigure 3.6: Example programs using horizontal flow. In both pro-grams, one robot arm hands over an item to the other, as forthe vertical programs in Figure 3.5.if mapping time to a vertical or horizontal axis is more effective. We assumethat most applications choose a horizontal design to maximize screen spaceor for other practical, layout-related reasons. Nonetheless, the likely pre-exposure of many users to horizontal timeline visualizations might influencetheir intuitions and comprehension.Figures 3.6a and 3.6b show horizontal versions of the programs we previ-ously presented in Figures 3.5a and 3.5b. As the figures show, a horizontallayout requires more modifications than simply rotating the code. SinceEnglish text flows from the left to right, labels that are placed next toeach other horizontally consume significantly more space than when theyare placed vertically. This leads to screen space being used less effectivelyand drastically reduces the Visibility of the overall program context. Wetherefore decided to use icons instead of text-based labels for all commands,with the exception of user-named locations.Previous research has shown that icons are less effective for comprehen-sion than textual labels [107]. However, our design drafts use fewer differentsymbols than this previous work. We therefore speculated that there mightbe an acceptable trade-off between using icons and and being able to presenttime flow horizontally.In addition to icons, the horizontal design comes with another trade-52off. Unlike in the vertical design, the two rows of the horizontal layout donot directly correspond to the positions of the left and right robot arm.Mapping the top row to the left arm and the bottom to the right one mightbe a Hard Mental Operation for users.The trade-offs between the vertical and horizontal design alternatives aresimilar to those of the different synchronization alternatives. It is hard todraw conclusions by only using an analytical approach. We therefore decidedto evaluate the impact of both the synchronization model and the programflow orientation in the comparative survey we present in Section 3.3.3.3 Comparative SurveyWe wanted to empirically validate how well potential users can comprehendand use each design to decide which one is worth further development. Toconduct a study on a larger scale, we designed an automated survey that wecould distribute to a large number of participants.3.3.1 Research QuestionsOur study was designed to answer the following research questions:RQ1: How well do novices comprehend our candidate designs?As part of our design process we created mock-ups of sample programswritten in each design candidate, like those shown in Figures 3.5a, 3.5b, 3.6aand 3.6b. We used these mock-ups to validate our designs. We designed aseries of three tasks, each based on one example program, and a series ofcomprehension questions. We divided participants into randomized groupsthat were each shown a different design candidate throughout the survey.By grading each participant’s results for these tasks, we measured how welleach design allowed them to comprehend realistic example programs writtenin it.The first task did not use parallelism and was intended to introduceparticipants to block-based robot programming. The second task used syn-chronous parallelism and was based on the program shown in Figure 3.4b.The third task used asynchronous parallelism and was based on the programshown in Figure 3.5a.RQ2: Which design(s) do novices prefer?We see the differences in comprehensibility as a major factor in deter-mining the quality of our designs. However, users may prefer a design forreasons unrelated to their ability to understand it. We therefore asked par-53ticipants to fill out a short questionnaire asking how usable the prototypewas.We also wanted to give participants the opportunity to directly comparethe designs. We showed each participant one alternative to the design theywere originally assigned and let them rate its usability. We also asked themwhich one they would prefer overall. Table 3.7 shows the order in whicheach participant group was shown the tasks and design alternatives.RQ3: Which factors influence the preference and overall opinionof novices?We also wanted to find out which factors contributed the most to eachparticipant’s preferences. We pre-selected a number of differences, suchas program flow orientation, or using icons or text as labels, and askedparticipants how important each of these differences was to their decision.We also gave them the opportunity to name other factors and to provideus with an open-ended explanation of their decision. These questions werepart of the usability questionnaire shown in Table 3.7.RQ4: Do experts have different preferences than novices?Although our language designs are intended to be novice-friendly, theyshould also be usable by expert programmers. Since experts may have dif-ferent needs and preferences than novice end-users, we included both in ourstudy. We define novices as users with no significant experience with anyprogramming language, no previous robotics programming experience, andno professional role related to software development. We end our analysisby comparing the preferences of both groups.3.3.2 Survey DesignTable 3.7 shows an overview of our survey structure and the participantcount for each part of the survey. As we recruited participants, we providedthem with a brief demographic questionnaire to classify them as novicesand experts. We then asked them to complete three tasks to measure howwell they could comprehend our design candidates. Finally, we asked theparticipants to rate the usability of the designs they saw and indicate whichfactors influenced their ratings.Recruitment: To recruit participants for our survey we reached out to bothemployees of a large, multi-national robotics company, and the broader pub-lic by advertising in online communities focused on engineering. We had 313responses, with participants from a wide range of professional backgrounds(e.g. engineers, software developers, administrators). Of those participants,we include 273 who completed at least one task of the survey in our results.54yIntroduction 273 Participants Started SurveyRecruitment 110 Novices 163 ExpertsTask 1 50 Vertical 60 Horizontal 77 Vertical 86 HorizontalTask 2 45 Vertical 53 Horizontal 55 Vertical 68 HorizontalTask 2 (alt.) 41 Vertical 50 Horizontal 54 Vertical 65 HorizontalTask 3 22 I-V 14 E-V 25 I-H 23 E-H 20 I-V 31 E-V 28 I-H 32 E-HTask 3 (alt.) 22 I-H 14 E-H 25 I-V 23 E-V 18 I-H 31 E-H 27 I-V 31 E-VUsabilityQuestionnaire84 Completed Questionnaire 107 Completed QuestionnaireCompleted 191 Participants Completed SurveyFigure 3.7: Survey flow (from top to bottom) with participant num-bers for each design treatment. Participants were first catego-rized as novices and experts and then randomly assigned one ofthe designs shown in Figures 3.5a, 3.5b, 3.6a and 3.6b.Introduction and Randomization: Immediately after our participantshad completed our demographic questionnaire, we showed them a one minutelong training video. This video introduced them to the overall idea of robotprogramming and the kind of robot that our designs target. It also showedthem how a robot performs a simple pick-and-place procedure with onearm. It did not show any programming interface to ensure that we didn’tbias participants.After participants had finished watching the video, we divided theminto randomized groups. Each was assigned to a different design candidateamong the four we have presented previously. All groups followed the sameremaining survey procedure except for the mock-up programs they wereshown and minor changes in survey texts to match the individual layout ofeach design.Tasks: We wanted to evaluate the participants’ ability to comprehend pro-grams written in our candidate designs on both a syntactic and a semanticlevel. For this purpose we designed three tasks with increasingly complexsample programs that were inspired by realistic usage scenarios. For eachquestion, we also gave participants a use case description. We then askedthem to answer a series of questions to evaluate their comprehension. Weintentionally focused on questions related to timing and parallelism, as theseare the novel aspects of our design.To avoid overwhelming participants, we ordered the three tasks follow-55ing this video by increasing complexity. Each task consisted of a use casedescription, a mock-up program and a number of comprehension questions.Example questions were ”select all rows/columns during which the rightrobot arm carries the cube” or ”which blocks need to be modified to changethe cube’s target destination”. Participants were allowed to leave questionsunanswered and continue with the rest of the survey.Task 1 was designed to be a warm-up task that only uses one arm andgives participants a chance to get familiar with the block-based program-ming environment. We showed participants a one-armed pick-and-placeroutine. A similar routine was used by Weintrop et. al. in their CoBloxexperiments [36]. Since this task did not contain any coordination, we onlydistinguish between two different designs for this task, based on the usedprogram flow direction. We asked two questions to determine whether par-ticipants had a basic understanding of the environment, one question aboutthe (completely linear) timing of commands, and one question about howthey would change the item’s target destination.Task 2 used parallelism, but was intentionally similar to Task 1 so thatparticipants could identify the same overall structure. The vertical mock-upfor this task was identical to the one presented in Figure 3.4b. We askedparticipants two questions to evaluate if they had a basic understanding ofthe function of rows and columns in the given program. Then we askedthem to identify the purpose of the given program. Finally we asked themtwo questions about the timing of parallel commands. The first of thesequestions allowed two possibly correct answers, based on the parallelismmodel the participant intuitively assumed. The second question then verifiedif participants were applying their chosen model consistently to the wholeprogram.Task 3 was the most complex task. It showed participants an uncom-mented 30 second video of a robot arm handing an item to another arm,and an incorrect program attempting to reproduce this behaviour. Fig-ures 3.5a through 3.6b show the correct versions of this program in each ofthe evaluated design alternatives. The version used in the survey howeverhad the Open and Close block in the middle of each program swapped. Thiserror would cause one arm to drop the held item before the other one couldgrab it. We asked participants one general question about the timing ofthe program that was independent of the contained error. We also askedtwo questions about identifying the erroneous blocks. The final questionsuggested ways to fix the error, and asked participants to identify at leastone that could work.After the second and third tasks, we showed participants an alterna-56tive mock-up using a different design candidate. This prepared them tocompare the designs during our follow-up questionnaire. To avoid confusionbetween the shown designs, we decided not to show participants the alterna-tive synchronization model but only the alternative program flow direction.Table 3.7 shows the resulting order in which each participant group wasshown each design.Usability and Preference Survey: To capture participant preferences,we solicited their opinions of the two design alternatives they were shown.After working on all three tasks of our survey, we asked them to rate theusability of each design via standardized questions from the System UsabilityScale (SUS) [108]. Since some of the SUS questions only apply in a contextwhere users can use a system actively, we asked only 6 of the 10 questionsdefined by the SUS and then computed an overall score. According toprevious research [109], this gives results that are comparable to the fullquestionnaire after scaling the resulting score.Besides rating usability, we also asked participants to indicate their over-all preference between the two used design alternatives on a 5-point scale.We also asked how important some of the design differences were to theirpreference. We further allowed participants to list other factors for theirdecision. We asked about three major differences between the vertical andhorizontal designs: the direction of program flow, the type of labeling usedfor blocks (icons vs. text) and the uniformity (or lack of uniformity) of blocksizes. Since each participant used either implicit or explicit synchronization,we rely on the usability survey to compare these design types.3.3.3 Survey ResultsAll of the following results are based on the 273 participants who bothgave consent to their data being collected and completed at least one of thetasks assigned to them. We classified 110 participants as novices and 163 asexperts. The numbers of participants that finished each individual task arelisted in Table 3.7. A total of 191 participants (84 novices and 107 experts)finished the entire survey.RQ1: How well do novices comprehend our candidate designs?Based on the 110 novice responses, Table 3.1 summarizes the distri-butions of comprehension scores for each task and design. The table alsoshows the average scores for each task. The overall average scores of partic-ipants on Task 2 and 3 were high, even though these tasks were non-trivial.However, novice participants performed substantially worse on Task 1 thanTasks 2 or 3.57DesignCandidateTask 1: 1-armedPick and PlaceTask 2: 2-armedPick and PlaceTask 3a: 2-armedHandoverImpl., Vert.Expl., Vert.0 25 50 75 100 0 25 50 75 1000 25 50 75 100Impl., Hor.Expl., Hor.Mean score: 55% Mean score: 75% Mean score: 67%Table 3.1: Box-plots of the scores of novices for each task’s compre-hension questions. Participant numbers (n) are reported in Ta-ble 3.7.We did not expect participants to perform worse on Task 1, since thistask was intended to be the easiest and did not involve any form of coordi-nation between arms. A closer inspection of the results shows that the lowoverall score can be attributed to a single comprehension question that only32% of all novices answered correctly. This question asked participants howthey would change the target destination of an object carried by the robotarm. It was the only question that involved the manipulation of locations.We assume that a lack of training on the meaning of locations is the mostlikely cause for this finding.Participants who used the horizontal design performed worse on averageon Task 1. The difference is substantial and statistically significant (29%worse; independent samples t-test assuming equal variances finds 𝑡 (108) =90.54, 𝑝 < .0001). For Tasks 2 and 3, there is also a difference in theaverage values, but it is smaller and not statistically significant (< 10%worse; independent samples t-test assuming equal variances for Task 2 finds𝑡 (96) = 1.06, 𝑝 = .15; one-way ANOVA for Task 3 finds 𝐹 (3, 80) = 0.22,𝑝 = .88).RQ1 Summary: The data shows that participants’ performance washigh for all tasks, except for the warm-up task, independently of the useddesign.RQ2: Which design(s) do novices prefer?We measured participants’ preferences using the SUS scale, which rangesfrom 0 to 100 points. This scale should be interpreted linearly, but wecan instead convert SUS scores into a percentile rank which compares our58designs to other systems that were evaluated using the same scale. We usedonly 6 of the 10 questions from the standard questionnaire, but previousresearch indicates that the scaled scores are still equivalent to the samepercentiles [109].Figure 3.8 shows the raw scores of our designs on the SUS percentilecurve. Meta-studies find that the average SUS score across studies is 66 [109].The scores of all our designs are above this average, ranging from the 60th tothe 80th SUS percentile. There is almost no difference (< 1 point) betweenthe average SUS scores of the designs using implicit and explicit synchroniza-tion. We do see however a substantial and statistically significant 8-pointdifference in favor of the vertical designs over the horizontal ones. A one-wayANOVA finds 𝐹 (3, 164) = 5.41, 𝑝 = .0014. A post-hoc Tukey HSD finds thatvertical designs are always better than horizontal ones (all 𝑝 < .05).In addition to calculating SUS scores, we also asked participants di-rectly which design they prefer. Of all participants, 57% answered that theyslightly or strongly preferred the vertical design they were shown. Another39% slightly or strongly preferred a horizontal design. Remarkably, only 4%of all participants had no preference at all.RQ2 Summary: The data shows that the majority of novice partici-pants preferred the vertical program flow, but have no preference betweensynchronization models.RQ3: Which factors influence the preference and overall opinionof novices?Pre-Selected Factors: We showed all participants two designs: theone we assigned to them and an alternative one that uses the same synchro-nization model but a different program flow orientation. We asked themwhich factors had the biggest influence on their preference ratings. Thefirst row of Table 3.2 summarizes how participants rated three pre-selecteddesign differences as Likert items on a 5-point scale. Overall, participantsfound the differences in labeling (text vs. icons) to be the most importantfactor, closely followed by the orientation of program flow. It seemed lessimportant for participants whether blocks had a uniform size.We conducted two additional analyses: first, we investigated whetherparticipants that preferred different designs had different factors that in-fluenced them. We grouped the factors by preferred design as shown inTable 3.2. An independent-sample t-test assuming equal variances findsno statistically significant difference between the two groups for any factor(from left to right: 𝑡 (78) = 0.35, 𝑝 = .36; 𝑡 (78) = 0.86, 𝑝 = .20; 𝑡 (78) = 0.52,𝑝 = .30). Second, we verified that participants did not always prefer their as-5901020304050607080901000 10 20 30 40 50 60 70 80 90 100SUS PercentileRaw SUS Score SUS Percentile Curve Design I-V (Avg. Score: 79.4) Design E-V (Avg. Score: 78.7) Design I-H (Avg. Score: 70.8) Design E-H (Avg. Score: 70.2)Figure 3.8: SUS scores computed for novice participants’ responses,plotted on the SUS percentile curve. Participants rated thevertical design candidates significantly better.signed design over the alternative they were shown later. A one-way ANOVAfinds no statistically significant difference between participants based onwhich design they were assigned (𝐹 (3, 77) = 0.27, 𝑝 = .85).Qualitative Feedback: In addition to rating the importance of pre-selecteddesign differences, we allowed participants to provide written feedback onwhat they liked about their preferred design. Almost all participants usedthis opportunity, resulting in comments from 78 of our 84 novice partic-ipants. We used open coding to categorize participants’ responses basedon named attributes and keywords. This resulted in 48 unique codes, eachqualified as being positive/negative and assigned to one or more of the de-signs.Positive comments about program comprehension were common for alldesign alternatives, but were found more frequently for the vertical designs.The most frequently used and uniformly distributed codes were understand-able (45 overall mentions) and simple (44 overall mentions). The code intu-itive was more frequently used for the vertical designs (30 mentions) thanthe horizontal ones (12 mentions). Some participants also mentioned thata vertical design was compact (6 mentions), but none said this about any60Orientation Icons/Text UniformityOverall1 2 3 4 5 1 2 3 4 5 1 2 3 4 5Pref. VTPref. HRTable 3.2: Perceived importance of individual design aspects fornovice participants, overall and grouped by their preferred de-sign: 1 is least important, 5 is most important.horizontal design.Participants’ preference for the vertical design was further explained bytheir consistent comments on key issues. One participant commented, “Be-cause I work vertically in Excel a lot, the vertical format was more in-tuitive.”, which was echoed in many other comments, such as “I like thevertical due to the normal flow of scrolling down in the computer screen”and “information is normally presented top to bottom on a screen”. Theusers’ prior experience with vertical flow in other settings may have led toa preference for it in this setting. Another clear reason that users preferredthe vertical flow was that “it is easier to associate left and right arms withleft and right columns, rather than rows”, mentioned by many users, whocommented that “the natural left and right orientation of the vertical pro-gramming was a big positive” and “the left-right distinction is more clearrather than having to remember that top is left and bottom is right.”There were however codes that highlighted strengths of the horizontaldesign as well. The code shows time flow was used 8 times for horizon-tal designs compared to only 3 times for the vertical designs. Further, 21participants complimented the icon labels used in the horizontal designs. Incontrast, only 8 participants mentioned the text labels of the vertical designspositively, and 2 participants criticized them. These results are consistentwith the previous observation that although the majority of participants pre-ferred the vertical design, there is a minority with a substantial preferencefor the horizontal design.Those users that preferred the horizontal design also explained theirpreference consistently. One participant commented “The icons in the hori-zontal example made it easy to understand in one glance which actions areperformed”; this was echoed by others, such as “the horizontal flow’s iconswere very useful and intuitive” and “[I] like the horizontal’s use of picturesover text”. Although the icons were the most positively mentioned property,6101020304050607080901000 10 20 30 40 50 60 70 80 90 100SUS PercentileRaw SUS Score SUS Percentile Curve Design I-V (Avg. Score: 81.6) Design E-V (Avg. Score: 81.1) Design I-H (Avg. Score: 74.8) Design E-H (Avg. Score: 74.1)Figure 3.9: SUS scores computed for expert participants’ responses,plotted on the SUS percentile curve. Experts gave consistentlyhigher ratings for all designs and had the same preferences asnovices.a few users also commented that “it was easier to mentally determine thetiming of the program’s actions”, “the horizontal design clearly gave me atimeline”, and “I liked that it follows the design of most timelines, whichfeels most natural.” However, perhaps the most relevant comment that sev-eral users made was the suggestion that we “bring the visual aids of theactions [(the icons]) from the horizontal version to the vertical version.”The synchronization model did not seem to influence the prevalence ofany specific code. In addition, only a small number of comments mentionedthe used synchronization model. Some participants said that they likedthe used way of synchronization (5 mentions for the explicit and 4 for theimplicit variant). This suggests that the synchronization model had only aminor influence on the opinion of participants.RQ3 Summary: The data shows that the way blocks are labeled (textor icon-based) influenced participants’ opinion the most, followed by theprogram flow orientation.RQ4: Do experts have different preferences than novices?Figure 3.9 shows the SUS scores we calculated based on the responses62of expert participants. The experts not only performed better in all com-prehension tasks, but also gave higher usability ratings for all design alter-natives. They did however show the same preferences between the designalternatives as novices. The difference between the designs using implicitand explicit synchronization models is negligible (< 1 point), but there is asubstantial 7-point difference between the ones using vertical and horizontalprogram flow. A one-way ANOVA finds that there is a significant differencein our results (𝐹 (3, 206) = 4.80, 𝑝 = .0030). Unlike for novices, a post-hocTukey HSD however only finds a statistically significant difference betweenDesign E-V and the Designs I-H and E-H (both 𝑝 < .05).To identify factors that influenced expert users we applied the sameapproach as for RQ3 to the results for expert users. In addition to collectingimportance ratings for the pre-selected differences, we also coded 91 writtencomments. However, we did not see any noticeable difference in either ofthe results compared to novices.RQ4 Summary: Overall, expert participants gave higher average us-ability ratings and showed the same preference for designs with verticalprogram flow as novices.3.4 Further Studies on CoordinationIn our previously presented survey, we found that users could comprehendall designs equally well. We further found that the users preferred verticalprogram flow but had no preference between synchronization models. Asdiscussed in Section 3.2, our designs with explicit synchronization are moreexpressive than those with implicit synchronization. We therefore decidedto continue the development of our design candidate that presents programflow vertically and uses explicit synchronization.Our survey measured how well participants could comprehend givenmock-up programs, but we were not able to verify if participants could writethese programs themselves. To validate the usability of the design candidatethat we implemented, we conducted two follow-up studies. The first studyis a small-scale user study on a front-end prototype implementation of thedesign we deemed best, and included in our paper “Comparing Block-basedProgramming Models for Two-armed Robots”. The second experiment isa larger controlled experiment conducted on a physical robot and is theprimary contribution of the paper “Side-by-Side: A Case Study EnablingParallel Programming of Robotic Arms by End-Users” that we contributedto. In the following, we describe the first study in detail and provide a brief63summary of the findings of the second study.3.4.1 Front-end Prototype StudyTo complement the findings of our previously presented survey, we conductedan interactive study using a front-end prototype of the design we deemedbest. This study was smaller than our survey but provided important insightinto how novices could use our design to solve realistic robotics tasks. Itassessed the following research question:RQ5: Can users solve realistic coordinated robot programmingtasks using our prototype?We recruited 11 participants at a large office site of a multinational en-gineering company. Participants came from a diverse set of professionalbackgrounds, similar to those of our comparative survey. We classified par-ticipants as novices or experts based on the same classification questions asfor our survey.We showed all participants a short introduction video to demonstrate theprototype to them. The video explained the available blocks and showed howto write a simple program that picks up and then drops a cube. The partic-ipants had the opportunity to pause the video and ask the study supervisorclarification questions at any point.After they finished watching the video, we asked participants to solvetwo tasks. For each task we gave users a text that described the intendedbehaviour of the robot. We also gave them a picture that showed a numberof pre-defined locations that they could use in programs. The first taskwas to create a two-armed pick-and-place program, similar to Task 2 of oursurvey. The second task was to create a program to hand an item over fromone arm to the other, similar to Task 3 of our survey. All participants solvedthe tasks in the same order.We wanted to provide participants with a way to test their programs.Since we could not support testing on a real robot, we instead offered partic-ipants to manually ”simulate” their current program for them. When par-ticipants asked to test their programs, the study supervisor would imitatethe robot’s movements with their arms. However, only 2 of the participantsused this opportunity before submitting their final solutions.We measured the time that participants needed to complete each task.We also graded their final results for correctness. After they completed bothtasks, we gave participants an SUS usability questionnaire, similar to theone we used in our survey. Since this experiment involved interaction withour system, we included all 10 official SUS questions [108].64Figure 3.10: Fully functional prototype environment Duplo based onthe design prototype E-V.All of our 11 participants finished the interactive study. We classified 7of them as experts and 4 as novices. Only two of the experts had previouslyused any robot programming tools, but the others had some other previousprogramming experience.A total of 10 participants (3 novices) correctly solved Task 1 and 7participants (2 novices) correctly solved Task 2. Participants took 4 to 5minutes on average for each task, with the longest taking 11 minutes for onetask. We therefore conclude that most participants could solve two realisticcoordinated robot programming tasks using our prototype.3.4.2 Large-scale Physical Robot StudyBased on the promising findings of our survey and prototype study, weimplemented a fully functional prototype with the name Duplo. As illus-trated in Figure 3.10, the system uses the program flow and synchronizationmechanism that we presented in our design candidate E-V in Figure 3.5b.However, due to technical limitations, we were unable to implement blocksthat span across the programs for both arms or the visual mirroring of blocksdepending on the side of the canvas they are placed on.To evaluate Duplo, we compared it to RobotStudio Online YuMi [110]that we described in Section 3.1.3. ROY is the only robotics tool of which65●●●●●●●●●●●●●●●●●●●0255075100MinutesFigure 3.11: Participants’ completion times in ascending order. Eachbar represents one participant, and its color indicates the as-signed environment (blue for Duplo, yellow for ROY). A blackcircle marks participants who ran out of time before complet-ing all sub-tasks.we are aware that targets end-users and supports coordinated programs thattarget two robot arms at once. In addition, ROY provides a similar set offeatures as Duplo, with many commands being available in both languages.This makes ROY an ideal candidate for evaluating the impact that theblock-based editing paradigm and the synchronized blocks have on end-usersprogramming real-world robots.To compare the two programming environments, we performed a ran-domized controlled experiment with 52 participants. Our participants, pri-marily university students with little to no programming experience, wererandomly assigned to one of the two programming systems and given a brief15-minute long introduction on how to use it. They were then presentedwith a 105-minute long programming task that consisted of multiple stagesin which participants had to coordinate two robot arms to jointly carry andassemble a series of items. We recorded whether participants completed eachstage of the programming task, how long they took to do so, the numberand kind of programming mistakes they made, and how many attempts theyneeded to test their programs.Figure 3.11 summarizes our findings regarding completion times and suc-cess of both participant groups. We found that on average, participants whoused the Duplo block-based environment could solve the given tasks fasterand with greater success. Compared to only 46% of ROY users who suc-cessfully completed all stages of the programming task, 80% of Duplo userssolved all stages. Participants who used Duplo were also faster, requiringonly 53 minutes on average compared to 75 minutes on average for ROYusers. A closer analysis of the two participant groups shows that Duplousers made fewer programming mistakes and produced cleaner code with66fewer unused commands or redundant definitions of robot locations. Thesedifferences might explain their better overall performance and demonstratethe benefits of our design for coordinated block-based programming.3.4.3 LimitationsOur studies have a number of limitations, which we discuss at this point.Participant population: Our comparative survey was open to the gen-eral public. However, due to the channels through which it was advertised,we expect the majority of our participants to have a North American orEuropean background. It is further likely that a disproportionate numberof participants are employees of the same multi-national engineering com-pany. The participants we recruited for our first prototype study were alsoemployees of this company in a single location, and those recruited for oursecond, larger study were all undergraduate students at a North Americanuniversity. Despite being broad overall, these recruiting methods might haveintroduced biases, of which we consider the cultural ones to be most relevantfor our results. In particular, North American or European users of our toolsmight prefer a different reading direction than those from other countries.It remains an open question whether our findings generalize to users fromdifferent cultural backgrounds.User Training: All of our studies used training methods for our partici-pants that were practical but not necessarily realistic. For our survey wehad to limit the amount of time spent on introducing users to our designs,and for our other studies we had to limit how much time users can spendfamiliarizing themselves with the given environment and tasks. This mighthave negatively affected their comprehension. It might for example explainwhy users had trouble solving Task 1 of our survey, even though it was de-signed as a warm-up task. Despite the intended beginner-friendliness of ourdesigns, we would expect real industrial end-users to be trained more thor-oughly. In particular, we would expect any real-world training to involveinteractive testing and experimentation due to fewer time constraints.Task Choice: Time limitations have not just affected the training butalso the tasks that we could provide to our users. We have aimed to selectrealistic tasks of different styles and sizes across our studies, and motivatedour choices for each of them We are however aware that they only coversome of the potential usage scenarios for our designs. We have verifiedthat programs for other common tasks can be implemented in our designs.Further studies in the field are required to validate whether our findings canbe generalized beyond those used by us.67Implicit Synchronization,Vertical Flow (I-V)Implicit Synchronization,Horizontal Flow (I-H)Explicit Synchronization,Vertical Flow (E-V)Explicit Synchronization,Horizontal Flow (E-H)Expressivenesshigherlowerlower higherPerceived UsabilityFigure 3.12: Design space based on synchronization model and pro-gram flow direction. Each of the designs shown in Figures 3.5a,3.5b, 3.6a and 3.6b corresponds to one cell of this table.Attrition Bias: The survey results we presented include the partial re-sults of 81 participants who completed some but not all tasks. Participantsmay have left the survey because they were frustrated and found the surveytasks too challenging. This might have led to the artificially inflated com-prehension scores or usability ratings for later tasks. To test for this bias,we examined the partial results of participants who did not complete thesurvey. We did not find a notable difference in their scores. We found nocorrelation between the assigned designs and drop-out rates. We thereforeassume any bias caused by participants who dropped out is negligible.3.5 Discussion and SummaryIn this Chapter, we have presented and analyzed four block-based designprototypes that use visual programming elements, and evaluated them an-alytically and empirically. Figure 3.12 shows an overview of the findings ofour survey: explicit synchronization allows users to express a wider range ofprograms, and vertical program flow was perceived as more usable by ourparticipants. This outcome motivated us to conduct two follow-up studies.Both of these studies confirmed that the design with explicit synchronizationand vertical flow allows end-users to successfully program robot arms. Inthe remainder of this section, we discuss these findings and how they mightapply to other programming areas as well.683.5.1 Using Visual Program Alignment To RepresentComplex Program SemanticsOur analytical approach and preliminary studies alone do not explain whyour explicit synchronization was accessible to end-users. The design usesexplicit barriers for synchronization, which are an established concept intraditional parallel programming to synchronize threads. However, usingbarriers correctly in parallel programs is usually considered to be a diffi-cult task that is too advanced to teach to novices. Our experiments haveshown that beginners could understand barriers quickly and with minimalexternal guidance. They could comprehend programs with barriers just asfast as those that used implicit line-by-line synchronization. This is surpris-ing since implicit synchronization is less expressive and therefore seems likeit should be easier to understand. Barriers also did not seem to match theway that participants of our preliminary studies intuitively read coordinatedprograms.We believe that the addition of visual elements to blocks has played acrucial role in making our design end-user friendly. We showed the com-mands for each arm side-by-side and visualized the barrier as a single blockthat connects both arms. The design of barriers in our follow-up studies didnot have the same visual representation of barriers as a single block, but stillaligned them vertically and therefore created a visual connection. Unlike intraditional text-based programs, novices do not need to manually find andalign concurrent actions. We believe this to be a powerful visual featureof our language that was not present in any previous work. The feature isalso unique to block-based programming as it cannot be easily replicatedin text-based tools where programmers manually determine the layout ofprogram code.3.5.2 Visual Support for Programming Beyond Two-ArmedRobotsOur findings advance the area of robotics by making the programming oftwo-armed robots accessible to end-users. Building on previous research thathas shown that end-users can learn to successfully program single-armedrobots [7, 36], our own observations confirm and extend those previous re-sults. Remarkably, although we asked novices to solve more complex pro-gramming tasks than the previous work, we did not give them substantiallymore guidance or learning time. This did not seem to negatively impacttheir learning or performance.69Based on our findings, we suspect that neither the previous work wesurveyed in Section 3.1 nor our own designs and tasks have pushed novicesto the limit of their learning abilities. Our findings also suggest that with theright level of support, end-users can understand and even write programsthat feature extensive parallelism. It appears that end-users do not need tohave mastered simpler use cases, such as single-armed programming, beforethey can learn how to write complex programs that feature coordination. Itis up to future work to explore whether it is possible to introduce even morecomplex models of parallelism to end-user languages and how to representthem visually.All our design candidates have two inherent limitations that we considerchallenging to overcome but that would substantially improve their expres-sive power. First, all are restricted to coordinating two robot arms at a time,and second, they do not support of any flow of data between the parallelprograms. We discuss the challenges that arise with the addition of morearms in Section 6.3. However, incorporating data-flow is a particularly com-plex extension, as the presence of variables and mutation alone is a commonsource of confusion for beginner programmers [111]. Our designs avoid theselanguage features as they are not necessary for the use case we targeted. In-stead, we focus only on physical state, for example in the form of whethera robot hand is open or closed. This state is inherently arm-specific andeasy to inspect and reason about for end-users. We believe that any formof data-flow that is more complex and introduces new sources of errors willpose a substantial new challenge that goes beyond the scope of our currentprototypes and likely requires substantial follow-up work.70Chapter 4Block-based Decomposition:Mobile Robots OperatingMultiple WorkstationsStationary robot arms can support factory workers with a wide range oftasks, but their lack of mobility can hinder their use in practice. Theirlimited range can make it tedious to move items between different process-ing or assembly steps. More importantly though, when robots are fixed to apre-determined location, they cannot easily be re-purposed or autonomouslyswitch between multiple tasks. This means that stationary robots are pri-marily useful in contexts where their task is known ahead of time and thechance of any changes to their environment is low [112].Mobile robots that can move autonomously between physical locationshave increased in popularity over the previous years. [40, 112] Compared tostationary robots, this makes them substantially more flexible and allowsthem to solve tasks that are spread across multiple work sites. In addition,mobile robots can autonomously switch between different tasks and worksites. For example, when a mobile robot is idle, it can temporarily work ona different task. Therefore, a single mobile robot can do the same work asmultiple stationary robots.Assigning multiple responsibilities and more complex tasks to a singlerobot means that the complexity of the corresponding program grows aswell. When tasks are distributed over multiple sites, programmers mustkeep track of which operations are executed at each site and at which pointsthe robot has to switch its location. Particularly for end-users, managinglarge programs can be hard. They must simultaneously focus on both thebigger picture and the individual steps of the task that they want to solve.Traditionally, programming languages offer support for structuring codeand decomposing it into smaller, mostly self-contained fragments such asfunctions or methods. Most block-based languages support abstraction andprogram decomposition as well, often in the form of user-defined procedures71or first-order functions. However, just because abstraction mechanisms areoffered in principle does not mean that users can apply them to the concreteprogram they try to solve. All available evidence suggests that users of block-based systems tend not to decompose their programs [113], even thoughdecomposition is crucial to successfully write larger programs. We thereforespeculate that existing block-based systems do not sufficiently support theirusers through the set of features they offer to them.In this chapter, we present a block-based programming environmentthat guides users as they decompose their programs. This work is basedon the paper “Can Guided Decomposition Help End-Users Write LargerBlock-Based Programs? A Mobile Robot Experiment” (published and pre-sented in OOPSLA 2022) [114]. The system we present specifically targetsthe creation of programs for mobile robot workers. These consist of high-level commands that move the mobile robot between different workstationsand low-level commands that direct the robot to execute local tasks suchas picking up an item. The presented programming environment makesuse of this common task structure and explicitly divides programs into ahigh-level movement component and a low-level local component. Insteadof traditional functions, the environment features “tasks”, which resembleparameter-less procedures, but impose domain-specific restrictions that limithow they can be composed and which instructions can be used within andoutside of them. To support this language design visually and make pro-grams easier to navigate, the environment is divided into two side-by-sidecanvases, one for high-level and one for low-level programming. Figure 4.1shows our prototype and an example program with a single task selectedand displayed on the right canvas.We hypothesize that compared to a traditional block-based environment,our design candidate encourages users to give their programs a sensible,domain-specific structure. We further hypothesize that this structure en-courages users to re-use code rather than create code clones. These specula-tive benefits come with a potential cost: in comparison to free-form proce-dural abstraction as it is supported by current block-based languages, usersmust follow the program structure that our environment imposes on them,which might not always be best. However, we hypothesize that for mobilerobots, this cost is minimal in practice, as few end-users would be able to finda better solution in a traditional environment than the one our environmentimposes on programs.To validate our hypotheses and evaluate whether our environment canmake larger programs easier to understand and write for end-users, we con-ducted an experimental evaluation with 92 novice participants recruited via72Figure 4.1: Proposed programming system showing two tasks andside-by-side canvases. The left canvas shows the main pro-gram and the right one shows the body of the currently selectedtask(s).AMT. We randomly assigned each participant to either use our guided pro-gramming environment or a traditional block-based programming environ-ment with support for custom blocks. We asked all participants to solve aseries of three tasks that cover cases of interest: a small task that would nottypically require decomposition, a large task with an obvious and easy todecompose structure, and another large task with a more challenging struc-ture that is not optimally suited for our guided approach. Consistent withprevious studies [28, 35, 36, 115], we found that participants from both co-horts performed well on the small task. However, for the tasks that requiredlarger programs (and even the one that was easy to decompose), many par-ticipants in the cohort that used the traditional environment did not useabstractions or structure their programs at all. These participants were lesssuccessful at completing the given tasks. In comparison, the participantswho used the environment with guided program decomposition were over20% more likely to complete their tasks successfully, even for the task wherethis decomposition was sub-optimal.734.1 Decomposition in End-user LanguagesBlock-based programming was created with learners in mind, who usuallywrite brief and self-contained program snippets that serve a single purpose.Nonetheless, several popular block-based programming languages try to sup-port larger programs by allowing users to define custom blocks. These blockswork similarly to custom functions and are highly expressive in theory, butas we outlined in Section 1.1, their integration into block-based environ-ments is often shallow. For example, Figure 1.3a shows the definition andusage of a custom block in Scratch [4], which resemble the definition andcall of a procedure in a traditional, text-based language. Also as in text-based languages, there is no explicit separation between the function bodyand the surrounding program built into the environment. This means thatusers can freely move code between the different scopes of the program andcreate programs that any expert programmer would immediately identify asfaulty. For example, Scratch does not enforce scoping: it alllows a functionparameter like x in Figure 1.3a to be dragged into the main program bodyand misused outside of its definition range. Scratch does not provide anyfeedback in these cases, and its design feature of not presenting run-timeerrors to the programmer can cause even more confusion as beginners tryto understand the intended semantics of procedures.Some block-based programming languages go even further than Scratchin terms of replicating established abstraction and decomposition mecha-nisms from traditional programming languages. OpenRoberta [44] supportsfirst-order functions that return a value, Snap! supports higher-order func-tions [49], and App Inventor for Android supports object-oriented program-ming [48]. However, none of these languages attempts to guide programmersin how to use these features or to visualize their semantics. Therefore, un-less these tools are accompanied by external support, such as a tutor or anextensive tutorial, they are hardly a good fit for end-users or other noviceusers.Unfortunately, there is limited empirical data available on how end-usersdecompose their code when they are left to their own devices. Surveys ofpublic repositories for block-based code suggest that few programs use anydecomposition at all, but it is unclear what percentage of those code basesare written by adults or meant to be used productively [37, 38]. The samesurveys also find that many block-based programs contain long sequencesof instructions and frequent code duplication. Work that focuses on pro-fessional developers suggests that this style of code can be detrimental toprogram comprehension, even for those professionals [116]. One expects in-74(a) Nostructure(b) Free-form structure (c) Hierarchical structureFigure 4.2: Structures found in end-user programs. Traditional pro-gramming languages treat all of them as equal alternatives, butwe speculate that a hierarchical structure benefits end-users themost.experienced programmers, such as end-users, to be even more obstructedby poorly structured code, and the observations we present in Section 4.3support this assumption.Drawing from previous work, and our own observations of end-userswriting block-based programs, we identify two common patterns in end-userprograms: one subset of users always creates programs as a single, linear se-quence of commands without any attempt made to decompose the code. Wecall this style unstructured programming and illustrate it in Figure 4.2a. Theother subset of users appears to be aware of the need for decomposition andattempts to use the tools available to them (e.g. functions) to structure theircode. They create structures as shown in Figure 4.2b, which we call free-form programming. These structures can take any shape that is supportedby the programming language, potentially mixing different styles and usingfeatures ad-hoc instead of planning ahead. As a result, the code is ofteninconsistently structured, more complicated than necessary, and thereforeharder to understand and more error-prone.A wide and unrestricted program design space can benefit experiencedusers, but beginners are more likely to be overwhelmed by this flexibility.In fact, learning how to use functions has been identified as a ThresholdConcept in computer science education: understanding how to use functional75abstraction dramatically benefits learners, but is challenging to learn [117].If parameters are used to call functions, learners struggle to understandthe semantics of pass-by-value versus pass-by-reference [118]. Users alsostruggle to understand the related concepts of variable scoping, such aswhether they can access global variables within a function, and are confusedby corner cases such as variable shadowing [118]. In addition, users mightaccidentally discover and try to use techniques like recursion, which canlead to confusion and misconceptions [119]. Therefore, we speculate that asystem that reduces the abstraction decision space and provides guidance tousers, illustrated by Figure 4.2c, can benefit these users and improve theirability to reasonably structure their work and consequently construct largerprograms.4.2 Design Approach: Guided DecompositionWe propose a programming system that provides more guidance to noviceusers, such as end-users, than existing environments. The goal of this systemis to reduce the design space to those decisions that these users can makeeffectively, and to explicitly give them the information needed to make them.We believe that programming domains typically come with a set of embed-ded structures and hierarchies that can facilitate decomposition. End-usersthat are familiar with a domain have an informal understanding of these de-composition rules and how to apply them. A system that targets end-usersshould build on top of this understanding, provide the necessary formaliza-tion for domain-specific conventions, and enable programmers to decomposetheir code in a way that matches these conventions.Our system, shown in Figure 4.1, supports a fixed hierarchy of param-eterless functions, which we call tasks. Figure 4.3 shows the grammar ofthe underlying programming language and illustrates how tasks effectivelydivide the language into two distinct sub-languages: one where tasks arecomposed and one where they are defined. Our system further supports thishierarchical design with an environment that splits the program editor intotwo side-by-side canvases, where one provides the user with an overview oftheir program and the other shows the current task. Our hypothesis is thatthis approach for program decomposition and code navigation offers usersa simplified, yet powerful way to structure their programs in accordancewith their understanding of the problem they are trying to solve. Note thatwe use the Cognitive Dimensions of Notation throughout this section. Weintroduced this framework in Section 2.4 and underline and italicize terms76⟨program⟩→ ‘when started do’⟨task⟩*⟨task⟩→ ‘do’ ⟨taskname⟩ ‘at’⟨stationname⟩⟨taskdef ⟩→ ‘to’ ⟨taskname⟩ ‘:’⟨statement⟩*⟨statement⟩→ ‘pick from’ ⟨location⟩| ‘place at’ ⟨location⟩| ‘turn item’ (‘clockwise’ |‘counter-clockwise’)Figure 4.3: Grammar(s) of the programming language used in theproposed system. A program (left) can only contain calls totasks at the top-level, and that task definitions (right) takeplace in a separate programming canvas with its own syntax.As Section 4.2.4 describes, there are some differences betweenthe language’s semantics shown here and its block-based pre-sentation shown in Figure 4.1. Also note that locations aredefined via a visual location picker that is not represented inthis grammar.to indicate that they are based on an established definition.4.2.1 Why Do End-Users Not Use Functions?Kallia and Sentance [117] noted two factors that contribute to beginners’lack of function usage, and that we believe must be overcome if we want tomake them accessible to end-users. First, functions are abstract program-ming mechanisms that do not directly match the tasks users are trying toaccomplish. Translating a high-level task into appropriate sub-tasks, that isperforming functional decomposition, is a Hard Mental Operation, even forprofessional programmers [120] and likely moreso for novices [121]. As theydecompose their main task into sub-tasks, users may become overzealous,for example creating many functions with only a single block (top right ofFigure 4.2b). The unconstrained nature of task decomposition can quicklylead to confusing callchains (e.g., the calling relationships in Figure 4.2b), oreven unintended recursive definitions (that likely lead to non-termination).These potential misuses of functions contribute to users creating programsstructured like in Figure 4.2b, where functions can confuse inexperiencedusers more than they help them.Second, beginner programmers with no prior experience using functionsmay not see their benefit until it is too late [122, 123]. Novice programmersmay perceive functions as requiring Premature Commitment, as they havean up-front cost in terms of adding blocks and the understanding program77flow, and they must decide whether to use them before they know how largetheir program will ultimately become. Thus, users begin writing programsthat have no structure, and as these programs grow they become unwieldy.Block-based systems have a low Viscosity in theory as they allow usersto quickly re-arrange programs via drag-and-drop and have built-in hygienefor names. However, compared to refactorings that are found in professionaldevelopment tools, these features provide little guidance to programmers.This is especially unhelpful for novices, who might not have a clear vision ofhow to improve a program’s structure, even if they are aware of its currentissues. As we will see in Section 4.3, inexperienced users tend to simply splitprograms into arbitrary chunks, which provides minor Visibility benefits, butlittle benefit for their understanding of the code. For this reason, we alsodo not believe that simply highlighting overly long functions, or restrictingthe maximum length of continuous code blocks, can be a solution to thisproblem.4.2.2 Domain-Specific Program DecompositionTo overcome the factors that prevent inexperienced users from using func-tional abstraction, we have developed an alternative design for a block-basedenvironment that encourages the systematic decomposition of programs.Our language design can be summarized with two main points: first, togive decomposition more meaning to end-users, and to better align withtheir programming goals, we introduce “tasks”. Tasks are parameterlessprocedures that are assigned to a single workstation and can only containinstructions that relate to this workstation. Second, we make tasks manda-tory by splitting the environment and the underlying language into twodistinct components: the main program editor, which can only be used tocompose tasks, and a task editor, which defines the instructions of each task.Tasks are designed to explicitly support end-users as they create pro-grams for the mobile robotics domain. By explicitly catering to the kindsof programs end-users create for this domain, our design aims to naturallyguide end-users to create tasks that promote the single responsibility prin-ciple. Each task is assigned to a specific workstation, and the instructionsavailable to define a task are limited to those that can be executed locally atthis workstation. Therefore, the task body primarily contains instructionsthat move the robot’s arm and manipulate the workstation (e.g. by pickingand placing items). Movement between workstations happens implicitly asthe robot switches between tasks.To reduce the decision space for inexperienced users, our system design78imposes a domain-specific program structure We believe the hierarchical pro-gram structure with a fixed two-layer call graph makes our system easy tounderstand, yet expressive enough to decompose larger mobile robot workertasks. We therefore do not offer any additional, more complex mechanismsfor decomposing programs beyond tasks, such as globally or locally definedfunctions. The system imposes certain restrictions on users. First, usersmust decompose all programs into tasks, even small programs where an en-tirely flat program structure would be straight-forward to read. This limita-tion can reduce the Visibility of small programs, but also prevents program-mers from making a Premature Commitment to a flat program structurewhen programs must eventually grow larger. We further believe that mak-ing large programs easier to comprehend supports end-users where they needit the most. Second, the hierarchical design with two fixed levels rules outprograms with multiple decomposition layers (e.g. through nested functioncalls) or recursion. This can be a limitation for experienced programmersand lead to redundant code, potentially increasing the resulting language’sViscosity compared to one without these restrictions. However, we considerthe limitations to be beneficial in the context of end-users, who are morelikely to make mistakes when the control flow of a program becomes com-plex. In fact, many other block-based systems also prevent or warn usersabout using recursion [49, 62], but do so in more intrusive ways such asdisabling blocks or showing error messages.4.2.3 Aligning Physical and Programmatic ScopeComplex scoping rules can be confusing and unintuitive for novice program-mers [118]. Therefore, we have a system that simplifies the concept of pro-grammatic scope and matches it with the physical scope in which the mobilerobot conducts work. Since each task is localized at a single workstation, atask should only contain commands that can be executed within the phys-ical scope of its assigned workstation. For example, movement commandscannot take place within a task, and if there are different types of worksta-tions there might be other restrictions based on the physically available toolsand the workstation layout. This is analogous to how most programminglanguages have built-in mechanisms to limit the lexical scope that is acces-sible from within a given block of code. However, when those rules becomecomplex, they can increase the Error-proneness of code and without suffi-cient feedback, understanding them can become a Hard Mental Operation.Block-based languages can go beyond the traditional checks that text-basedlanguages perform to enforce scoping rules. For example, these systems can79actively filter the syntax they present to users and provide only the validoptions. In a structured programming system like the one we propose, wecan leverage this feature.To provide end-users with further guidance, our system design presentsonly those commands that are within the physical scope of a robot task.Similar to previous work on block-based robot programming [7, 101], oursystem supports only one variable type: locations. These locations, for ex-ample the target coordinates where an item is supposed to be placed, canbe defined using the programming environment. Previous work has allowedusers to select target locations physically by moving the robot arm into theintended position [7]. That work does not discuss their design decision inthe context of CDN terminology, but we consider it highly beneficial for theCloseness of Mapping that users experience between the programming en-vironment and the physical system. For our system, which currently relieson a virtual robot simulator, we had to replace this approach with a rea-sonable alternative. We chose a visual picker for target locations, which isa less direct representation than physical human-robot interaction, but stillsubstantially more direct than a purely text-based specification. We havefurther provided useful feedback in the form of meaningful domain-specificerror messages and warnings such as “Robot is already carrying an item!”when programs try to carry multiple things at once. We cannot compare thevisual location picker used by our system to the physical approach used inprevious work. However, we believe that both are reasonable, user-friendlychoices.In previous work, all locations were always defined globally [7, 101].However, since our system allows programs to span multiple workstations,this approach is no longer appropriate. Instead, locations must be specific tothe workstation at which they are accessible. Since our system assigns eachtask to exactly one workstation, we decided to make user-defined locationstask-specific, similar to local variables defined within traditional functions.This straight-forward mechanism, made possible by matching physical andprogrammatic scoping, saves users the Hard Mental Operation of manuallyfiltering the available locations.4.2.4 A Visual Programming Environment That SupportsDecompositionOur design is implemented as a multi-canvas environment, as shown in Fig-ure 4.1. The main program appears in the left panel of the figure. Themiddle panel of the figure shows the task that has been selected in the main80program. A robot simulator is shown in the right panel of Figure 4.1 as ourprototype implementation uses a simulation environment. This environmentoffers high Visibility, concurrently displaying the main body of the programon the left canvas and the body of the currently selected task in the middlecanvas. As users edit tasks, we concretize their editing by including thecurrently selected workstation in the task definition’s header. This has noimpact on the language’s semantics or the ability to re-use a task for otherworkstations. However, the concretization might mislead users into thinkingthat they are only changing one instance of a task at a time. To highlightthat they are in fact changing all instances of a task at once, other instancesare also highlighted in the main program when any of them is selected.Previous work has either made all function definitions visible at onceon the same canvas (e.g. Scratch), or attempted to spread code over anarbitrary number of small, isolated canvases [124]. Although the latterapproach shares some of the benefits to our design, it suffers either fromVisibility issues when too much code needs to be displayed, or introducesadditional Hard Mental Operations as users must customize their environ-ment manually. Our design on the other hand uses a fixed number of can-vases that are populated automatically and that follow the overview-detailparadigm that is commonly used in interface design and information vi-sualization [125]. This approach tries to balance Visibility with avoidingHard Mental Operations.4.3 Experimental EvaluationPrevious studies of block-based environments and how end-users create pro-grams in them have used small example tasks [36, 115]. Due to their smallsize (usually 20 blocks or less), these programs are not complex enoughto require the use of decomposition. Our study focuses on how end-usersperform when tackling larger tasks that do not fit on a single screen.In this study of 92 self-identified end-users recruited via AMT, we trainedparticipants to use a block-based programming environment. We then askedthem to solve 3 tasks of increasing size and difficulty. One participant groupused a traditional block-based programming environment with support forparameterless functions, and the other used the environment with guideddecomposition presented in Section 4.2. We measured how participants de-composed their programs in each environment, and how the use of decom-position affected their success in solving the given tasks. In the remainderof this section, we describe our study in detail.81Figure 4.4: The traditional block-based development environmentused by the TR Cohort of the programming study. It offersthe same command blocks and simulator, but uses a single can-vas and provides free-form function blocks that are defined andused in this canvas.4.3.1 Research QuestionsOur study investigated the following research questions:RQ1 How do end-users decompose their programs in a traditional block-based environment?RQ2 Does the use of program decomposition impact the task success rateand task completion time of end-user programmers when writing largerprograms?RQ3 How does an environment with guided program decomposition changethe way end-users write larger block-based programs?RQ4 Does guided program decomposition impact the task success rate andtask completion time of end-user programmers when writing largerprograms?4.3.2 Experimental DesignWe recruited our participants through the platform Amazon MechanicalTurk (AMT), which pays users to conduct online tasks. Using a short pre-questionnaire, we selected only those users that indicated that they hadless than one year of programming experience and did not have experience82programming industrial robots. Participants that passed this screen wererandomly divided into two cohorts. The first cohort (which we call TR fromnow on) used a traditional block-based environment to complete the study,and the second cohort (which we call GD) used the novel system with guideddecomposition that we presented in Section 4.2.Figure 4.4 shows the traditional block-based programming environmentthat participants in TR used; Figure 4.1 shows the novel environment thatthose in GD used. Both environments include an editor on the left sideand 2D simulator on the right side. The editor for the GD cohort requiredtwo canvases, as described in Section 4.2. The editor for cohort TR wasa single canvas with the option to add simple, parameterless functions asthey are supported by many existing block-based environments. As existingblock-based environments use a wide range of terms to describe functionsto their users, we decided to call them recipes in our own interface to matchthe terminology used in a previous industrial robot programming environ-ment [7]. This environment also featured a separate “Move to Station X”command to replace the implicit robot movements in the environment withguided decomposition. All remaining commands, such as those for pickingand placing items, were identical to the other environment, matching thethe statements listed in Figure 4.3.We used a series of 3 tutorials to train all participants to use a block-based programming system. The first two tutorials focused on the system’score functionality: commands to move from station to station and to pickup, carry, turn, and place items. The third tutorial taught users to decom-pose programs (using functions or tasks, respectively), and to re-use code bycalling the same function or task multiple times. The tutorials’ content andflow were as identical as possible for each cohort. All tutorials are availableon the previously linked website. Although we intended the tutorials to takeapproximately 15 minutes in total, we did not limit the time that partici-pants were allowed to spend on each tutorial. A small number participantsin both cohorts exceeded the intended tutorial time (see Figure 4.7), butthe majority completed them without our initial time estimate.After participants completed the tutorials, we asked them to solve aseries of 3 tasks. Each task consisted of a brief description and an imageshowing the intended outcome of the task. A simulator (see Figure 4.4)allowed participants to test their solutions, and they were allowed an unlim-ited number of attempts. However, we did limit the time that participantswere allowed to take for each task, giving them one final chance to submittheir solution after they exceeded the maximum allowed time. Independentof whether a participant’s solution was correct, we saved their final attempt83for later evaluation.The three tasks, in the order we gave them to participants, were:Task 1: A short, toy-sized task with a time limit of 10 minutes that weintended as a warm-up for participants. Users were asked to move twoboxes between stations, which can be accomplished using just 13 blockswithout any decomposition, or using 17 blocks in the system that requirestask decomposition.Task 2: A larger task with a time limit of 15 minutes that we designedto be repetitive and therefore benefit substantially from code re-use. Userswere asked to move a stack of 3 boxes, one at a time, from one side of aworkstation to the other, for four different workstations. The task providedparticipants with a functioning program for a single workstation, consistingof 15 blocks, which they had to apply identically or with slight variations toother workstations. We specifically chose this task because it consists of anumber of spatially isolated and therefore easy to separate sub-tasks. Thismeant that users who chose to decompose their programs could avoid re-dundant work by re-using or at least efficiently duplicating code. We believethat this task is representative of the type of work for which our proposeddomain-specific decomposition is the most effective. We therefore expectedusers of the traditional environment who decomposed their programs to doso in a similar way as we imposed on users of the other system. The task alsofeatured some clear redundancy and therefore potential for re-using code inboth systems, reducing the minimum number of necessary blocks from 56without any decomposition (or 62 in the system that required task-baseddecomposition) to just 46 in an optimal solution.Task 3: Another large task with a time limit of 15 minutes, although forthis task the ideal structure was less obvious since the components of thetask were not spatially isolated. Users were asked to move boxes betweenstations, each time swapping places with another box. For this task, partici-pants started without being given code, but we asked them to solve an easierpart of the problem with a single box on each side first, and then approach amore complex version with two boxes that could be solved by re-using partsof their code. We believe that this task is representative of a type of workwhere our proposed decomposition strategy is sub-optimal, since it requiressplitting tasks into more (and smaller) sub-programs than users might findintuitive. This also becomes clear when comparing block numbers: A solu-tion without any decomposition can solve this task using 48 blocks, and anoptimal solution with code re-use requires 40 blocks; the strategy imposedby our presented system requires 54 blocks with or 64 without re-use. We84therefore see it as useful for both evaluating if such overly strict decompo-sition requirements harm our participants’ performance, but also whetherthose users without guidance are able to find a (potentially different) styleof decomposition that is useful for them.4.3.3 Experimental ResultsIn the following section we discuss the results of our study, aligned with theresearch questions we presented in Section 4.3.1. As a reminder, we refer tothe cohort of participants who used a traditional block-based programmingsystem as TR, and to those we used our proposed novel system as GD.RQ1: Program Decomposition in a Traditional SystemFigure 4.5 shows how participants in the TR cohort used functionalabstraction, separated into Figures 4.5a, 4.5b, and 4.5c, which correspondto Tasks 1, 2, and 3, respectively. The right of each figure also shows howmany participants re-used code as a result of applying functions.For Task 1, the shortest task, participants used an average of 13.8 blocks(median: 13), of which 8.5 blocks (median: 8) were statements3, and definedan average of 0.3 (median: 0) functions. Participants defined between 0 and2 functions, with 8 using at least one. The following bar chart shows thefull distribution of function usage4:Only 8 (16%) out of 49 participants used functions for this task at all, and 41(84%) wrote their program as one continuous block of code. Therefore, theaverage number of sequential statement blocks contained in a single chunkof code (either the main program or a function) is 8.1 blocks (median: 8),which is close to the total average program size. As shown in Figure 4.5a,none of the 8 participants that defined 1 or 2 functions re-used code (bycalling the same function more than once).For Task 2, a larger task, participants used an average of 54.1 (median:55) blocks, of which 29.9 (median: 31) were statements, and defined anaverage of 2.1 (median: 3) functions. Each participant defined between 0and 4 functions, with 29 using at least one. The distribution is shown indetail in the following bar chart:3When we refer to statements in this sub-section, we include function calls and instruc-tions for the robot, but not locations or function headers4Light green bars (to the left) show successful tasks, dark red bars (to the right) showfailed tasks. The height of the bar corresponds to the number of functions, dots representsolutions where no functions were used.85A higher percentage of participants used functions for this task. However,19 (39%) did not use any functions, specifying their entire program as asingle, continuous block. As shown in the middle of Figure 4.5b, of theprograms that contained functions, the majority (69%) were structured in away where each function describes the robot’s actions at a workstation. Thisstyle of program decomposition is the one that we impose on users in theguided environment. The remaining 9 programs (31%) were decomposed indifferent ways, for example by extracting specific pick-and-place sequences.We tried to categorize these programs but were unable to find any patternsof note that appeared in 3 or more programs. Overall, participants used anaverage of 21.1 sequential statement blocks (median: 10) per code chunk(function or main program), with those participants who did not use anyfunctions heavily skewing the distribution.Of the 29 function-using participants, 25 (85%) gave their functions cus-tom names. The remaining 4 (15%) did not change the default names as-signed by the editor (i.e. “do something” with numeric suffixes to ensureuniqueness). However, only 10 of the 29 participants that used functions(34%) called any of them more than once, as shown on the right of Fig-ure 4.5b, which meant that there were many missed opportunities for codereuse.For Task 3, another larger task, the average solution contained 49.0blocks (median: 49), of which 31.7 (median: 31) were statement blocks.Each participant defined between 0 and 4 functions, with 13 participantsusing at least one, as detailed in the following bar chart:Only 13 (28%) of our participants used functions at all, a lower percentagethan for Task 2, resulting in only 0.7 functions being used on average overall(median: 0). This led to less structure overall and even less code re-usethan for Task 2, as shown in the middle-right of Figure 4.5c. Notably, 33participants (72%) constructed their entire program as a single, continuousblock of code, as seen in the middle-bottom of Figure 4.5c. Of the remainingparticipants who did decompose their programs, 5 (38%) used a very coarsestructure that split their code into exactly two sub-programs, following thetwo parts outlined in the task’s description text. Only 3 participants (23%)used a workstation-based style that resembled the one used by the envi-ronment with guided decomposition. The remaining 5 participants (38%)used different styles that split the task into 3 or more parts and showed nocommon decomposition pattern. Overall, the average number of sequentialstatements used per code chunk (main program or function) is 25.4 blocks(median: 27), which mostly representative of those participants who did not86(a) Task 1(b) Task 2(c) Task 3Figure 4.5: Categorization of participants in the TR cohort and theirprograms for the three tasks our study.use functions. When just considering function users, the sequential state-ment number is 11.9 (median: 11), which still suggests of a fairly coarsedecomposition of code.RQ1 Summary: When using a traditional block-based system, mostend-users did not use functions to decompose larger tasks, resulting ina single, unstructured code block for 65% of the submitted solutions forTasks 2 and 3.RQ2: Impact of Traditional Decomposition on Success and TimeTasks 2 and 3 were complex, and thus we hypothesized that participantsthat used functions would have better success. As can be seen in the middleof Figures 4.5b and 4.5c, most participants that failed did not use functionsto structure their programs. For Task 2, of the 29 participants who used87(a) Task 1(b) Task 2(c) Task 3Figure 4.6: Categorization of participants in the GD cohort and theirprograms for the three tasks our study.functions only 5 (17%) failed, and of the 19 participants that did not utilizefunctions, 9 (47%) failed. For Task 3, of the 13 participants who usedfunctions only 3 (23%) failed, and of the 33 participants that did not utilizefunctions, 18 (55%) failed. Taking both tasks together, when a participantused at least one function to solve a complex task, they succeeded 81% ofthe time, whereas when they did not, they succeeded 52% of the time.Although we saw a correlation between function usage and success ratesfor both Tasks 2 and 3, we only saw a substantial difference in how fastparticipants solved a task for Task 2. For Task 2, the participants that usedfunctions finished in 9.1 minutes on average, and the participants that didnot use functions finished in 13.4 minutes. For Task 3, the participants thatused functions finished in 14.5 minutes, and the participants that did not88use functions finished in 14.0 minutes.RQ2 Summary: End-users that added functions to their programs weremore likely to be successful when working on larger tasks.RQ3: Program Decomposition in an Environment with Domain-Specific GuidanceFigure 4.6 shows how participants in the GD cohort decomposed theircode to solve each task. By design, the development environment withguided program decomposition required all participants in this cohort todecompose their programs, including the toy-sized Task 1. To avoid ambi-guity between our three study tasks and the “tasks” that the environmentoffered participants as a way to decompose their programs (see Section 4.2),we refer to the latter as “sub-tasks”. Since the environment with guideddecomposition does not allow differences in decomposition style, we divideparticipants based on the number of sub-tasks they created in Figure 4.6.For Task 1, participants used an average of 16.3 blocks (median: 17),of which 7.8 (median: 8) were statements5, and defined an average of 3.8sub-tasks (median: 4). The total number of blocks is higher than for the TRcohort. After a manual inspection of the code written by participants, wefind that this difference can be primarily attributed to the overhead causedby additional sub-task definitions. The average number of sequential state-ments per code chunk (main program or sub-task) is 1.6 blocks (median: 1).Each sub-task only contained an average of 2.0 blocks (median: 2) and 1.0statement blocks (median: 1). This level of decomposition is quite extreme,and demonstrates that the system we propose is not optimized for such smallprograms where decomposition is arguably unnecessary. Each participantdefined between 1 and 4 sub-tasks, with all successful participants using 3 or4. The distribution of sub-task usage is detailed in the following bar chart:As visualized on the right side of Figure 4.6a, the vast majority of partic-ipants did not re-use code. However, although this task was short, it didhave potential for re-using a sub-task, which two participants successfullyutilized.For Task 2, participants used 60.3 blocks on average (median: 63), ofwhich 28.1 (median: 31) were statements, and defined an average of 3.7 sub-tasks (median: 4). Similar to Task 1, the average program length is thereforeslightly higher than for the other cohort, although the average number ofstatements is slightly lower. The average number of sequential statements5When we use the term “statements” here, we include sub-task calls and robot instruc-tions, but not locations or headers.89per code chunk (main program or sub-task) is 6.1 blocks (median: 6). Eachparticipant defined between 0 and 4 sub-tasks, with all successful ones using3 or more. The full distribution is detailed in the following bar chart:Overall, we saw a very high consistency in how participants solved thistask. Almost all participants split their program into four about equally-sized sub-tasks that contained the code for a single workstation. However,although two of the workstations had identical instructions, only 6 partici-pants (14%) re-used code between them. Furthermore, all but 5 participants(12%) edited the default names for their functions, making use of this op-portunity to document their code.For Task 3, participants averaged 59.9 blocks (median: 65), of which anaverage of 31.2 blocks (median: 31) were statements, and defined an averageof 12.3 functions (median: 14). As for the previous tasks, the total programsize is larger than for the TR cohort. Each participant defined between 0and 18 sub-tasks, with none of the successful participants using fewer than10. The distribution is detailed in the following bar chart:As visualized in the middle of Figure 4.6c, participants needed manyfunctions to solve this task. Nine participants (22%) solved the task suc-cessfully with 12 sub-tasks or fewer, 22 (55%) with 13-15 sub-tasks, and2 (5%) used more than 15 sub-tasks. As a result, the average number ofsequential statements per code chunk (main program or sub-task) is only3.6 on average (median: 3), which is much lower than for the TR cohort.A total of 28 participants (70%) used names other than the default namesfor their sub-tasks, again indicating an interest in documenting code in thisway.Despite the restrictions on how programs can be decomposed in theguided environment, participants’ programs showed some variation in solv-ing this task. All of them were structured around workstations, but someparticipants followed a more stringent order of operations while others at-tempted to optimize their solutions. For example, approximately two thirdsof the programs written by successful participants contain redundant move-ments between workstations that are likely explained by them trying to keepthe order of operations executed by the robot as uniform as possible. How-ever, only 7 (18%) of the participants re-used sub-tasks at all, despite thepotential for re-use enabled by this uniformity.RQ3 Summary: Users with guidance employ substantially more programdecomposition but still only rarely re-use code.90RQ4: Impact of Decomposition Guidance on Success and TimeThe effect of the decomposition guidance provided to cohort GD onsuccess can perhaps best be seen by re-examing the bar charts we producedfor RQ2 and comparing them with the charts for RQ3. Participants in theGD cohort had much more success, as indicated by the almost entirely greencharts in Section 4.3.3. Figure 4.7 shows a more direct comparison of thetwo cohorts; the left side of this figure details success rates and the rightside details task times.For the first task, which was relatively simple, both cohorts had highsuccess rates (92% and 95%, shown on left of Figure 4.7). However, forTask 2, the TR cohort’s performance dropped significantly (71%) and theGD cohort’s performance remained high (91%). Though Task 3 provedto be the most difficult task for both cohorts, 83% of the participants inthe GD cohort still managed to solve it successfully, versus 54% for theTR cohort. We performed a chi-squared test of independence to analyzethe statistical significance of the relation between participants’ cohort andtheir success on each task. For Task 1, we did not find the relation to besignificant: 𝜒2Task1(1, 𝑁 = 92) = 0.46, 𝑝 = .496. For Task 2 and Task 3, wedid find the relation to be significant: 𝜒2Task2(1, 𝑁 = 91) = 5.64, 𝑝 = .018;𝜒2Task3(1, 𝑁 = 86) = 7.72, 𝑝 = .005.To ensure that the slight difference in tutorial presentation between co-horts did not lead to some participant group having more time or experiencewith the system, we also analyzed how long participants spent working onboth the tutorials and tasks. As shown on the right of Figure 4.7, there wereno practical differences between tutorial times except on Task 3. To investi-gate this difference, we performed an independent two-tailed t-test, but wedid not find a statistically significant difference: 𝑡Tut1(90) = −0.65, 𝑝 = .517;𝑡Tut2(90) = 1.40, 𝑝 = .165; 𝑡Tut3(90) = −1.20, 𝑝 = .233.For times spent on tasks, the differences we observed showed a similartrend as the success rates. For the first task, the average time of 4.7 minutes(median: 4 min.) was similar to that of the TR cohort (average: 4.8 min.,median: 4 min.). However, for Task 2, participants spent 9.2 minutes onaverage (median: 8 min.), which is less time than the average of the TRcohort (average: 10.8 min., median: 10 min.). For Task 3, the differencebetween the cohorts was even more pronounced, as participants only spent12.4 minutes on this task on average (median: 12.5 min.) compared to theTR cohort’s 14.2 minutes (median: 15.5 min.).91Figure 4.7: User study results. For box plots: center lines show the medians, box limits indicate the 25thand 75th percentiles. Whiskers extend 1.5 times the interquartile range from the 25th and 75thpercentiles. Outliers are represented by dots, only outliers < 20 minutes are shown.92An independent two-tailed t-test further found a statistically significantrelation between the cohorts and task times for Task 3, but not for Task 1and Task 2: 𝑡Task1(90) = −0.151, 𝑝 = .880; 𝑡Task2(89) = −1.89, 𝑝 = .062;𝑡Task3(84) = −2.57, 𝑝 = .006.RQ4 Summary: Users of the guided environment were significantly moresuccessful when solving larger tasks.4.3.4 LimitationsHere we discuss some of the limitations of our study.Participant Population: We recruited participants via the AMT plat-form. We did not collect detailed demographic information from participantsbeyond screening them on programming experience. Previous work suggeststhat although the pool of AMT workers is not entirely representative of thegeneral population of the US, potential biases are comparable to those ofother recruitment methods (e.g. recruiting students) [126]. Although wedo not have clear evidence that AMT participants are an acceptable proxyfor the overall end-user demographic we aimed to investigate, we believethat their results provide initial evidence we hope to be investigated furtherthrough future, more in-depth studies.Recruiting Process: We recruited the 92 participants of our study intwo waves, 55 and 37 participants respectively. We planned to have 50-100participants in total, but we began evaluating the results of our first 55participants when making the decision to add a second wave of participants,introducing a risk of bias. We found little difference in all aspects of the twowaves, but out of an abundance of caution we separated the two waves inthe supplemental data to ensure transparency.Tasks and Training: We aimed to train participants in a way that istime-efficient and comparable to the quality of training that end-users mightreceive in real-life. In addition, we designed the training methods to be assimilar as possible across both of our participant cohorts. We assume thatparticipants would perform better across all our tasks with more extensivetraining, but the high success rate on Task 1 suggests that our trainingmethod was sufficient for teaching participants the foundations of program-ming mobile robot workers. Another potential limitation is that the smallnumber of tasks we used in our study cannot fully represent the wide rangeof possible programming tasks that end-users might encounter in industrialpractice, even within the domain of programming a mobile robot worker.These tasks and their wording might not match real industrial practice,and real applications might provide more detailed or precise instructions93that make it easier to determine a program’s optimal structure. Our studyfurther does not investigate how large or complex end-user programs canbecome, and whether our decomposition approach can scale further or if itis only applicable to a limited range of program sizes. Beyond the scope wehave targeted, there are also further scenarios in which mobile robots can beused (e.g., where the robot conducts work while moving instead of stoppingat dedicated workstations). These scenarios are not compatible with thelanguage and decomposition strategy that we have presented here, and it isnot certain that our work or findings are transferable to them.Experimental Measures: For our experiment, we evaluated our proposedsystem as a whole, which consists of several components that we described inSection 4.2. We discussed the potential benefits of each component, but ourexperiment cannot validate to which degree each component is responsiblefor the differences that we observed between the two cohorts. In addition, wecompared our proposed system to one that supports parameterless functionsin a style that is conceptually similar to ours, but not necessarily representa-tive of all abstraction and decomposition mechanisms offered by block-basedsystems. Other block-based systems do offer features like function parame-ters, and the lack of those features might have held back participants of theTR cohort as they wrote their programs.Furthermore, other factors are other potentially when evaluating thequality of a real program than those chosen by us for this experiment. Forexample, run-time performance might be relevant in practice, and it mightbe a goal for real programmers to minimize the time a robot spends movingbetween stations. Since such factors are highly dependent on the givensituation and robot model, we decided not to instruct our participants tocare for them, and therefore also do not include them in our evaluation.For our qualitative evaluation, we focused on the number of functions used,the overall program structure and code re-use. We also reported on othermetrics, like the number of overall blocks or statements in a program. Webelieve that the performance differences we observed, despite the almostidentical program size between cohorts, shows that block numbers alone arenot a sufficient indicator for how easily an end-user can understand a givenprogram.4.4 Discussion and SummaryIn this section, we discuss the implications of our approach and the findingsfrom Section 4.3. As in Section 4.2, we use terminology from the frame-94work of Cognitive Dimensions of Notation [2] and highlight these terms byunderlining and italicizing them.4.4.1 Benefits of Program DecompositionOur findings, both for RQ2 and RQ4, confirm that program decomposi-tion can substantially improve the performance of end-users as they solvemoderately large programming tasks. For the TR cohort, we found thatusers that decomposed their programs were more successful in solving tasks.This could imply that decomposition helped them program more effectively,but also leaves room for the interpretation that users who performed bet-ter at programming also found it easier to decompose their programs. Ourfindings for the GD cohort provide a clearer insight into this connectionbetween decomposition and programming performance. By giving our par-ticipants an environment that required them to decompose their programsand guided them towards a reasonable domain-specific decomposition, wesaw their success rates improve substantially.It remains an open question how exactly program decomposition benefitsbeginners, and whether advice given to professionals on how to ideally struc-ture a code base [116] also applies to end-users. However, a closer look atsome example programs suggests that any structure at all, even if it was notnecessarily the most intuitive or most concise, already has positive effects ona programs comprehensibility. For example, Figure 4.8 shows two programscreated by participants during our user study (Task 2). Both of these pro-grams correctly solve the task, yet they highlight the differences in programcomprehensibility. Figure 4.8a shows the solution of a participant in theTR cohort: an unstructured program with 57 individual blocks. Programssimilar to this one were written by many of the TR cohort’s participantsacross all three tasks. In stark contrast, Figure 4.8b shows a solution from aparticipant in the GD cohort. The main program body is only 4 lines long,and each task’s name summarizes (at least superficially) what is happeninginside the task’s body. Therefore, even though the total length of the pro-gram is slightly longer at 61 blocks, and even though the bodies of “MoveStack” and “Move Stack2” are identical and could have been replaced by asingle, re-used task, it is still easier to understand its structure and relate itto the given task.Notably, block-based programs already provide certain Visibility bene-fits to users compared to text-based languages. For example, even in anunstructured program like in Figure 4.8a, the different block colours canhelp users distinguish the individual commands and identify the points at95(a) TR Cohort: No structure(excerpt)(b) GD Cohort: HierarchicalstructureFigure 4.8: Examples of programs written by participants for Task 2.(a) shows a subset of an unstructured program (57 blocks intotal); (b) shows the top layer of a hierarchically structuredprogram (61 blocks in total).which the robot moves between workstations. However, as our experimentdemonstrates, this visual aid alone is not a substantial enough measure tohighlight a program’s structure. We believe that beyond guiding users touse an at least rudimentary level of abstraction, another important bene-fit of decomposition is the ability to name program components. For ex-ample, readers of the program in Figure 4.8a would need to perform theHard Mental Operation of chunking and summarizing the program’s partsmanually. In Figure 4.8b on the other hand, the user was able to describethat each task moves a stack of boxes. Previous work has argued for theimportance of Secondary Notation like names and comments, especially fornovices [98]. However, especially block-based languages often lack placeswhere users can use Secondary Notation or introduce barriers that makethem difficult to access for beginners.Figure 4.9 shows another example of the same pattern, but for Task 3.As described in Section 4.3.2, this task was intentionally chosen to containa lot of steps that move the robot between workstations, resulting in a96(a) TR Cohort: No structure(excerpt)(b) GD Cohort: HierarchicalstructureFigure 4.9: Examples of programs written by participants for Task 3.(a) shows a subset of an unstructured program (58 blocks intotal); (b) shows the top layer of a hierarchically structuredprogram (61 blocks in total).very fine-grained task separation when following our guided decompositionapproach. Therefore, the program shown in Figure 4.9b is rather long andnot structured in a way that an expert would likely consider optimal. Yet,compared to the unstructured programming style shown in Figure 4.9a thatthe majority of participants used for this task, the program is still morereadable, and as for the previous example, the participant has used tasknames to summarize their code. We believe that this example illustrateshow even a (from an expert perspective) sub-optimal program structure canhelp end-users understand their programs better.974.4.2 Function Usage in Block-Based SystemsOur results validate previous findings and demonstrate that end-users canquickly learn how to solve small, toy-sized tasks, such as Task 1 in our study,in a block-based environment [35, 36, 115]. However, we extended these find-ings showing that, when not supported by the programming environment,end-users became less successful as tasks grew larger, such as for Task 2 andTask 3. We believe this is because they either do not use functions, or whenthey do use functions, they struggle to use them systematically.When considering why end-users in the TR cohort tended not to usefunctions, even though our study explicitly trained them to do so, a lackof exposure to these features is not a reasonable explanation. Upon fur-ther comparison of the results for Task 2 (which offered a straightforwardway to structure programs) and Task 3 (where the ideal structure was lessobvious), we found that users were much more likely to structure their pro-grams in Task 2. This supports our assumption that end-users understandthe need to structure their programs, and attempt to do so when they areable. However, in all but the most straightforward applications, the opti-mal program structure might only become apparent after a large amountof code has already been written. At this point, a novice user might havealready Prematurely Committed to a specific structure (or to no particularstructure) and it might be difficult for them to manually re-structure theircode.What causes these end-users to struggle with functions? There are manymissteps that end-users can make when using unrestricted functions. Con-sider Figure 4.10, where movement and stationary instructions are entan-gled within a single function call. When viewing the main program, theend-user cannot predict which functions might contain a hidden robot basemove, giving each function call a potential side-effect. This Visibility issueis solved naturally in our prototype, by systematically requiring all robotbase movements to be defined at the top level, as shown in Figure 4.9b bythe “at Station X” directive. This fixed Abstraction Gradient constrains theprograms end-users could potentially write, forcing them to avoid confusingside-effects.4.4.3 Domain-Specific Task Support Beyond Mobile RobotsThis work demonstrates that a system that guides users in decomposing theirprograms can support end-users as their programs grow in size. However,other end-user domains, such as home automation, app development or data98Figure 4.10: Example of a program where movement and station-ary instructions are entangled. It is not possible to entanglemovement and stationary instructions in this way with our ap-proach.collection from websites, might require a different approach to structuringprograms. In particular, different domains likely need different forms ofenvironmental support to map the kinds of goals end-user programmershave to abstractions such as tasks or functions. We believe that findingtasks that are compatible with the domain-specific expectations of end-usersis a challenging, but not impossible exercise. Therefore, we hope that futurework in identifying domain-specific tasks, for other domains, will allow thosedomains to benefit from techniques and customized tool support similar towhat we present in this work.4.4.4 Code Re-use: An Open ProblemAlthough participants in the GD cohort of our study performed better thanthose of the TR cohort on many metrics, there is one result that is similarfor both cohorts: a lack of code re-use. Regardless of task, only a smallfraction of either cohort called any function or task more than once (9%of TR, 12% of GD). As a result, the average sizes of programs with andwithout decomposition were about the same, which eliminates one of themajor benefits of modularizing programs.We did not expect users to employ substantial amounts of code re-usesince they were not explicitly instructed to write code that is concise. Fur-thermore, both environments did not allow users to parameterize their func-tions or tasks, limiting the potential for re-usability. However, Task 2 in par-99ticular had obvious opportunities for code re-use, as illustrated by Figure 4.8:participants had to write an identical 12-block sequence of instructions twicefor two different workstations, but very few used this potential to substan-tially reduce the amount of code they had to write. This complete absenceof re-use in most users’ programs is surprising, considering that we explicitlytaught participants how to re-use code as part of the tutorial sequence theyhad to complete.Because we did not anticipate this lack of re-use, we did not ask partici-pants about their reasons for not re-using code. One of the primary benefitsof supporting re-use is the reduction of the language’s Viscosity, as eachrepetition requires additional work if code changes become necessary. Wespeculate that participants might not have seen a benefit in putting effortinto re-using the same function or task, and for this limited study they mighthave been correct in this assessment. For instance, we did not tell partici-pants that they had to write programs that were easy to maintain or modifylater. This lack of instruction might have caused them to prefer code clones,as they might have been more familiar with the idea of copy-and-paste thanthey were with re-use.We did try to make code re-use easier for participants. As Figure 4.1shows, our guided environment highlighted all calls to the same task andshowed the relevant panes on the right as a visual stack. However, this in-terface may have been insufficient, as the visual cue is subtle and thereforeeasy to miss. As we did not provide further explicit explanation, it mighthave even confused our participants. Finding a more effective way to en-courage code re-use, especially in the presence of parameters, remains anopen problem for future work.100Chapter 5Block- vs. Graph-basedNested Expressions:Enabling EnvironmentInteractions throughTrigger-Action ProgrammingHuman workers on factory floors or in warehouses often envision using mo-bile robots as co-workers to which they can assign tasks that would betedious for a human to solve [41]. The environment introduced in Chapter 4contributes to realizing this vision. Allowing end-users to program mobilerobots and assign them tasks that span multiple physical locations givesthem the opportunity to automate routines and workflows that previouslyrequired a human worker.Program size is only one of the challenges when programming mobilerobots. Another one is that interaction with the robot’s environment is oftenmore complex when it takes place across a large physical space. For exam-ple, a mobile robot may be programmed to operate one or more machinesand also remotely react to commands given by human workers. To offset thecost of a robot, it might even be used for multiple purposes, switching be-tween tasks on demand [127]. These use cases cannot be addressed througha single, linear program, even if end-users can make changes over time. In-stead, the programming system must represent environmental signals andinteractions and allow end-users to connect them to robot programs.The programming system we introduced in Chapter 4 focuses solely onprogramming what actions a mobile robot should execute. In this chapter,we extend this system with a focus on interaction with the robot’s environ-ment, and add support for programming when actions should execute. Weintroduce support for triggers, a simplified form of event-based programmingfound in popular end-user tools like If This Then That (IFTTT) [128]. Trig-101gers can represent a range of useful environmental stimuli, for example byallowing a robot to reacting to machine signals, user inputs or sensor data.This enables new types of programs, such as those that operate machineryin multiple time-delayed steps, or those that contain multiple tasks that areexecuted on demand.Building an end-user system that supports triggers raises a number ofdesign challenges and decisions. In theory, the design of existing digital au-tomation systems like IFTTT can serve as a template for triggers in robotprogramming. Once a program is deployed, a scheduling system can checktrigger conditions and execute programs without further input from theend-user. However, two factors differentiate robot programming from dig-ital automation. First, most digital tasks that tools like IFTTT are usedfor (e.g., sending an email or displaying a reminder message) are concep-tually atomic, and either instantaneous or easy to resolve in parallel. Onthe other hand, even simple robotic tasks might involve dozens of individualsteps and take minutes to complete. Therefore, it is likely that situationsarise where a new trigger requires handling before the previous one has beenfully resolved. Furthermore, it is hard or impossible to interleave multiplerobotics tasks that are assigned to the same robot (or the same machinesor workstations, if multiple robots are in use). Handling such situations, forexample by queuing up triggers, in a way that is understandable for end-users can be difficult. Second, using block-based programming to representtriggers introduces a new challenge that takes place before program execu-tion. Defining a trigger involves constructing a complex logical expression,a form of programming that is different than defining sequences of impera-tive commands, and that we speculate to be represented poorly in existingblock-based languages.In our work, we primarily focus on the challenge of representing expres-sions in blocks because compared to scheduling, this challenge is more closelyrelated to our overall topic of making block-based programming accessibleto end-users. We conducted small-scale preliminary evaluations on how toschedule triggers in a way that is understandable to end-users. Eventually,we chose a design that was reasonably easy to implement in a prototypeand appeared simple enough for most of our preliminary participants to beused in our follow-up evaluations. However, in these evaluations, which weconducted at a far larger and more rigorous scale, we focus entirely on inves-tigating the current means of representing nested expressions in block-basedprogramming languages, and propose alternative designs.In this chapter, we present two prototype environments: one that uses ablock-based design that nests expressions vertically rather than horizontally,102and one that is a hybrid between blocks and data-flow programming. Wechoose data-flow programming as a potential alternative to blocks becauseit is a mature and popular visual paradigm and assumed to be a particularlygood fit for computing expressions [129]. On the other hand, data-flow pro-gramming is less effective when modeling imperative, stateful computationslike those commonly found in the context of robotics. A hybrid of the tworepresentations has the potential to combine the best properties of both.We conducted a controlled experiment that compares two environmentsfor mobile robot programming based on the two design options outlinedabove: one exclusively uses blocks while the other is a hybrid of block-basedand data-flow programming. We recruited 113 end-users via the Prolific on-line platform to compare the two prototypes. We trained each participant touse their respective environment and asked them to complete two complextasks that required writing both nested expressions for triggers and imper-ative robot code. We then evaluated their ability to comprehend isolated,more complex examples of triggers that were represented as blocks or asa graph. Finally, we had them rate their environment individually and incomparison to the alternative.We find that participants performed better on average when using apurely block-based environment, both when writing programs and when try-ing to understand them. These results are supported by the participants’ratings, which were higher for those who used blocks exclusively. Remark-ably, when asked to compare the environment they used with the alternative,participants who used blocks were more interested in using a graph-basedenvironment than those who used data-flow graphs were in switching toblocks. We believe that our observations illustrate the attraction that data-flow programming has for end-users, but also provide evidence for theirpractical limitations. For the domain of robotics, we find that introducinggraphs as a situational alternative does not ultimately benefit end-users.For other domains, our findings raise the questions about whether blocksare underused and whether they could outperform data-flow programming.5.1 Limitations of Block-based and Data-flowProgrammingIn this section we analyze and compare the strengths and limitations ofblock-based programming and data-flow programming.103Figure 5.1: A nested expression in the block-based language Scratch:The entire expression is displayed in a single line and the nestingstructure is difficult to identify.5.1.1 Block-based ProgrammingThere is limited evidence about which design elements of block-based lan-guages most benefit novice users. However, one of the design goals ofScratch and Alice was to make programs match the flow of natural languagetext [4, 62]. Therefore, to reach this goal, programs should be readable (andexecuted) in an order that follows the flow of the block shapes. For exam-ple, in Figure 5.2a, a robot is instructed to execute a sequence of commands.This flow is represented by the arrangement of the blocks, allowing the pro-gram to be read top-to-bottom and left-to-right: “To move stack of blocksat Station A: pick up item from top left; place item at bottom right; ...”.This flow matches the flow of English language text and can be adapted toother natural languages with different flow by mirroring the block shapes.Block-based languages like Scratch strive to retain some similarity be-tween blocks and text for expressions. Scratch does not use traditionalsyntax like parentheses, but instead relies entirely on block shapes and doesnot provide any customization options for how users can format their pro-grams. An example of this approach’s limitations can be seen in Figure 5.1,which is representative for branching in Scratch and other popular block-based languages. In this example, the visual clutter and hard-to-identifyblock boundaries obscure the nesting structure of the expression, and edit-ing it can require disassembling and rebuilding the entire nested block viadrag-and-drop. The main benefit that blocks offer in this case is guidance onhow the different types of sub-expressions can be composed (round shapesare integers, pointy shapes are booleans). However, unlike for text-basedrepresentations, programmers cannot add line breaks or spacing to improvethe readability of the overall expression. Furthermore, the closeness to nat-ural language does little to improve the program’s readability. In particular,since English does not provide a way to indicate how grammatical clauses104(a) Program: flows top-to-bottom. (b) Expression: flows inside-out.Figure 5.2: Comparison of flow in imperative block-based code andblock-based trigger expressions.are nested, the text alone leaves ambiguity that is only resolved by closelyexamining the block shapes.Figure 5.2b shows an alternative way to represent nested expressions incode, which is the one we use for the purely block-based prototype envi-ronment we evaluate in this paper. In this representation, expressions havethe shape that block-based languages traditionally use for statements, plac-ing one on each line and concatenating them top-to-bottom. Compared toScratch, where an entire expression is placed in a single line like a run-on sen-tence, this style resembles a nested bullet point list in natural language text.By default, all blocks are combined using the “and” operator, but explicitoperators such as “any of” or “all of” can be inserted to nest expressions.Since each sub-expression is presented on its own line, with operators addingindentation to inner expressions, the program is indented similarly to state-ments inside control-flow blocks. To avoid deeper nesting than necessary,the representation provides an explicit drop-down selection of negated values(e.g., “running” vs. “not running”) instead of using negation blocks.The representation we chose in Figure 5.2b aims to improve the clarityof the contained expression, but does not resolve all potential limitationsof representing expressions using blocks. It still forces programmers to readand write expressions inside-out instead of top-to-bottom. Another potentialissue of this representation is that it re-uses the visual shape of statementsfor expressions, which might cause confusion when using both in the sameenvironment. For example, a programmer might expect to be able to attachthe blocks in Figure 5.2b to the program in Figure 5.2a.1055.1.2 Data-flow ProgrammingData-flow graphs do not visualize the text-based syntactic structure of a pro-gram. Instead, they introduce a notation that captures the same semantics,but with a focus on the flow of information rather than order of execution.For a nested expression, a data-flow graph represents the flow of informa-tion from sub-expressions to their surrounding expressions, and thereforealso their dependencies and the order in which they must be evaluated tomatch direction of the flow. As a result, data-flow graphs can leave parts ofthe actual execution order up to the system that evaluates them. Figure 5.3shows the data-flow graph for the expression that the block-based programin Figure 5.2b computes, with data flowing from the individual environmen-tal signals to the trigger at the bottom of the graph. The evaluation of thenodes does not need to follow the order that Figure 5.2b implies, but canbe changed and optimized as needed.When computing a single side-effect-free expression, such as the triggershown in Figure 5.2b, the resulting data-flow graph is always tree-shaped.This tree closely matches the program’s syntax tree for most programminglanguages. For example, if one were to draw the syntax tree of the block-based program in Figure 5.2b, it would be structurally almost identical toFigure 5.3. In fact, the only difference for this example is the presence ofexplicit and-operator nodes in Figure 5.3, which we added for clarity, butcould be omitted.The closeness of data-flow graphs and syntax trees makes it trivial toconvert between a block-based expression as shown in Figure 5.2b and itsgraph representation as shown in Figure 5.3. From the programmer’s per-spective, graph-based representations might be beneficial for understandinga program. Specifically, data-flow graphs make the order in which expres-sions get evaluated and how they depend on each other explicit and can bearranged so that the resulting flow can be read from top to bottom.Data-flow graphs can be generated for all kinds of programs and notjust those that consist of a single side-effect-free expression like that in Fig-ure 5.2b. In the general case, the graph may not be tree-shaped, since datacan flow to multiple target nodes, such as environmental state or multi-ple output values. However, the more complex structure makes data-flowgraphs less useful as a visual tool. This is especially the case if computationis stateful and the scope of the state is potentially global, such as a robot’sphysical environment. For example, none of the commands in the robotprograms shown in Figure 5.2a share explicit data flow, but they all affect(and are affected by) the state of the robot’s physical environment. The106Figure 5.3: The same program as in Fig. 5.2b represented as a data-flow graph. The edges make the data-flow explicit and program-mers can manually group and arrange nodes.block-based program represents this dependency by defining a fixed orderin which the commands must be executed. Without introducing additionalabstraction, such as nodes that represent the environment and edges thatmodel interactions with it, a data-flow graph would look almost identical.In terms of user experience, data-flow graphs have further advantagesand disadvantages. One major advantage is that they resemble flow charts,a widely used notation outside of programming that many potential end-users are familiar with. Furthermore, data-flow graphs can give users thefreedom to arrange and format their code as they prefer, without affectingits semantics. For example, a user can group related sub-graphs together,similar to how nodes that contribute to the same combinators are groupedin Figure 5.3. A fundamental disadvantage of graph editing compared toarranging blocks is that editing a graph requires twice the work: in additionto adding or editing nodes, users must also add or update the correspondingedges. Graphs give users the opportunity to manually arrange their code,but maintaining this arrangement also induces effort as users likely mustmake manual adjustments as they edit their code. As Figure 5.3 shows, agraph can also demand substantially more space than blocks, and compress-ing it into a small canvas can make it harder to read.1075.2 Design Approach: A Hybrid SystemBlock-based and data-flow programming both have distinct benefits for end-users and seem to be best suited for different types of programs. Althoughit is often possible to easily convert programs between the two paradigms,their user interfaces differ too much to seamlessly switch between them whileprogramming. End-users further might not be able to tell which approachis preferable for writing a given program or program component. Therefore,we propose an environment that selects the program representation thatprovides the most benefit for the user for each part of a program.To achieve our goal of merging the two program representations, we drawinspiration from our previous work [114] (described in Chapter 4) that hasintroduced a user interface that splits programs over two distinct canvases:one that displays the high-level flow of the program and one that definesthe localized tasks that this flow connects. Users can navigate the programhierarchically via these two canvases, selecting one task at a time on the leftthat then gets displayed on the right. The previous work has focused onguiding users as they decompose programs by restricting the scope of theavailable syntax and variables in each canvas, and found that this approachcan benefit end-users significantly when they write large programs.We extend this work by not only supporting sequentially composed tasks,but also triggers that define when task sequences get executed. Users cannow use the left canvas to select both tasks and triggers, whose definitionis displayed on the right. Figure 5.4 shows the environment, with a triggerselected in the left canvas and its definition displayed as a data-flow graphon the right. When a task is selected instead of a trigger, the right canvasdisplays its definition as a block-based program like in Figure 5.2a.This hybrid approach has several key benefits. First, it can provide thesame guidance on program decomposition that we found to be beneficial inChapter 4. This guidance is likely even more important for programs thatcontain triggers, as this new feature increases the amount of code end-userswill need to write. Second, the environment can be realised by connectingtwo existing frameworks: The block-based programming system Blockly [34]and the graph editor MXGraph [130]. As cost plays an important role forprofessional tool developers, being able to rely on existing, feature-rich codebases increases the chances of a similar system being adopted commercially.Third, by keeping the two paradigms distinct, we can directly compare thehybrid to the purely block-based design we introduce in Section 5.1.1 withoutintroducing other design differences that might affect our observations.108Figure 5.4: Hybrid programming environment: the left canvas shows high-level program flow as blocks, theright canvas shows selected tasks as blocks and triggers as data-flow graphs.1095.3 Experimental EvaluationTo evaluate the effect of block-based and hybrid editing on end-user pro-grammers, we conducted a controlled experiment. We discuss our experi-mental design and procedure in this section.5.3.1 Prototype EnvironmentsWe evaluated two prototype environments in the experiment: one purelyblock-based and one that combines blocks and data-flow graphs. In Sec-tion 5.1, we argue the blocks can represent the computation of nested ex-pressions, but that this representation comes with caveats. We introduced ahybrid environment that aims to overcome the limitations of blocks by inte-grating them with data-flow graphs to represent the computation of triggers.From the experiment, we hoped to gain further empirical insights into thelearnability and usability trade-offs between the block-based and the hybridenvironment.To keep the two programming environments under evaluation as simi-lar as possible, they provide identical syntax and editing functionality forprogramming imperative robot tasks, and an almost identical syntax fordefining triggers. We further provided both participant groups with anidentical simulator that shows a simplified top-down view of a mobile robotand its environment. This simulator allowed them to quickly execute theirprograms and test their correctness within the same browser window thatthey used for writing their programs.5.3.2 Research QuestionsWe set out to answer the following research questions:RQ1: Can end-users create robot programs using a mix of trigger-based andtask-based programming?Through this research question, we aim to validate the overall designof the programming environment described in Section 5.2. Previous workfound that a programming environment with two separate programmingcanvases can help end-users structure complex programs and allow them toprogram mobile robots with high success [101]. Trigger-based programming,the element we add to the design explored by the previous work, is a widelyused paradigm in practice [131, 132]. Nonetheless, these approaches maynot work in tandem and triggers may not be suitable for the domain wechose for the experiment.110Pre-screening+ first tutorial(n=179)Study grouprandomization(n=163)Second and third tutorial(n=114)Two interactive programming tasksParticipantRecruitmentUsabilitySurvey (n=110)Two programcomprehension tasks (n=110)ComparativeSurvey (n=110)(n=113)Figure 5.5: Procedure of the randomized experiment to evaluate thetwo prototype environmentsRQ2: Does the representation of triggers affect participants’ performanceas they read and write programs?We also aim to compare participant performance across the two proto-type environments. By answering this research question, we hope to gain ad-ditional insight into the trade-offs between the two approaches and whethera hybrid environment is effective in mitigating the trade-offs between blocksand data-flow graphs.RQ3: How do participants perceive both representations of triggers afterusing one of them?In addition to collecting empirical measurements, we also asked partici-pants how they perceived their assigned environment. We further asked par-ticipants whether they prefer one environment over the other, and whetherthey have any comments about their programming experience or their pref-erences.5.3.3 Experimental DesignWe used the Prolific [133] online platform for the study, which pays par-ticipants a fixed compensation to participate in studies and other researchactivities. We had no direct interaction with the participants other thanaddressing rare cases of technical difficulties. Instead, participants followeda fully automated study procedure that is outlined in Figure 5.5 and thatwe discuss in detail below.RecruitmentWe performed a power analysis [134] to estimate the numberof participants necessary to reach statistically meaningful results. Setting𝛼 = 0.05 and expecting an effect size of 𝑤 = 0.3, the power analysis deter-mined that a minimum of 88 participants are necessary to reach a powerof 1 − 𝛽 = 0.8 in a simple 𝜒2-test. Rounding conservatively, we thereforeplanned to recruit 100 participants.Participants were recruited through an advertisement on Prolific that111contained the study description, an estimate of the study duration, anddetails about the offered compensation, which was 12£6. We recruited par-ticipants worldwide and in multiple waves to avoid over-representing specificcountries or population groups. We discuss potential biases that might havebeen introduced by the Prolific platform and the recruitment process inSection 5.3.5.Pre-screening To ensure that the participants are end-users, we performedtwo filtering steps. First, we used a filter provided by Prolific to limit ad-vertising to participants who did not report having computer programmingskills when they created a profile on the platform. Second, we presentedpotential participants with a brief pre-screening survey that asked themwhether they had previous experience with software development, robotics,block-based, or visual programming languages. Only participants who didnot report any previous experience were allowed to participate in the study,while the remainder received 0.30£ for their efforts.Automated bots and inattentive participants are a known issue when us-ing online panels and crowdworking platforms [135]. We chose the Prolificplatform instead of the more commonly used AMT as the platform claimsto carefully curate their participant pool and prevent automated or dupli-cate submissions [133]. To further ensure that participants engaged withthe study and also have the technical capabilities to use the programmingenvironment, we also included a brief introductory tutorial as part of thepre-screening process. This tutorial asked potential participants to programa robot to pick and place a block, with detailed instructions how to do so.The tutorial was designed to take less than 5 minutes to complete, did notinclude any triggers, and was identical for all users. The task was intendedto be easy to complete, but could not be finished by randomly clicking on thescreen. Those who successfully completed the first tutorial received a smallcompensation of 1£ and were added to the pool of pre-screened participants.Randomization and Training After pre-screened participants providedconsent to participate in the study and have their data collected, we assignedeach of them randomly to one of the two environmental prototypes. We thenpresented them with a series of two tutorials that built on top of the initialtutorial that they completed during the pre-screening process. These tutori-als introduced them to more aspects of the programming environment. Thesecond tutorial introduced each participant to writing programs that spanmultiple workstations and are split across multiple tasks. A third tutorial6Prolific is based in the United Kingdom and calculates payments in British pounds.Participants from other countries received equivalent amounts in their local currency.112introduced them to triggers and how to program the robot to interact withmachines. The third tutorial was different for both participant groups dueto the different methods used to program triggers, but we designed bothtutorial versions to be as similar as possible. Overall, we expected a partic-ipant to take less than 20 minutes for both tutorials, but did not enforce acut-off time.Programming Tasks To answer RQ1 and RQ2, we aimed to present par-ticipants with programming tasks that feature both triggers and robot pro-grams that are complex enough to test the programming abilities they hadacquired during their training. We also wanted to test both the practicalusability of blocks and data-flow graphs, and how they support the partic-ipants when they need to reason about the activation of triggers. For thisreason, we used tasks that were larger and more complex than those usedin previous studies [7, 101, 114]. As larger tasks took participants longerto work on each of them, we had to limit their number to ensure that par-ticipants would not get exhausted or drop out of the study. We eventuallysettled on two tasks in a fixed order that covered the concepts we set out toinvestigate.Task 1 is large but conceptually simple. Two buttons are used to control arobot and make it pick up items from two different workstations. In bothcases, the items should be carried to a machine that processes them, andafter the machine finishes, the robot should pick up the items and transportthem to a target destination. For this task, we decided to give straight-forward natural language instructions for the required logic of the triggers,placing the focus on execution rather than creativity or abstract logicalthinking.Task 2 is more complex than Task 1, as it requires one robot to simulta-neously manage two item processing pipelines. We provided participantswith a program for managing one of the two pipelines. Although it seems atfirst that duplicating this given code is all that would be necessary to solvethe task, the resulting programs for each pipeline interfere and cause therobot to inadvertently place items from one pipeline onto the other. Thisinterference can only be avoided by adding more logic to the triggers of thesecond pipeline. This task requires fewer programming steps than Task 1due to the given partial solution, but resolving the interference (which is notexplained in the task description) demands a more thorough understandingof the trigger semantics.After we conducted several pilot experiments, we expected participantsto be able to solve the two tasks in approximately 20 minutes per task. Someparticipants took longer than 20 minutes, but few of the successful partici-113pants exceeded 30 minutes. We therefore decided on 30 minutes as a cut-offtime, at which point we would save the participant’s work and redirect themto the next part of the study. Within this cut-off time, participants had anunlimited number of attempts to test their solution using the simulator weprovided them. By inspecting the state of the simulated environment, wewere able to automatically determine whether a solution was correct, andforward the participant to the next task.Usability Survey After participants finished working on both tasks, eithersuccessfully or by exceeding the time limit, we redirected them to a usabilitysurvey. To answer RQ3, we asked participants separately about their opinionon the learnability of the environment they used, and how easy it was toread and write programs. We provided participants with a set of 5-pointLikert items for their rating. RQ3 focuses on how participants perceivedtriggers, but we asked participants to rate both the definition of triggersand construction of robot programs separately. We hoped that doing sowould encourage participants to focus on only one of these two aspects of theenvironment for each question. We also gave participants the opportunityto explain their opinion or provide additional feedback using a free textquestion.Program Comprehension Questions The triggers that we asked partic-ipants to write during the two programming tasks were reasonably complexgiven the scope of the study, but real triggers might grow even larger. Dueto the limited time we could give the participants to solve each task, wehad to limit the number of items, workstations, and corresponding signalsthat we could ask them to consider. The code that participants had to writeconsisted of two to four signals and at most two operators in a single trigger,which is less than shown in Figure 5.2b and 5.3. Furthermore, only one ofthe two tasks required participants to read existing triggers.To evaluate whether our findings, especially those for RQ2, hold for morecomplex triggers, we added two program comprehension tasks to the exper-iment. One task tested whether users could comprehend a trigger similar tothe examples in Figure 5.2b and 5.3. Figure 5.6 shows the block-based ver-sion of the other task, which focused on whether users could understand atrigger with more nesting levels. We asked participants to select the correctnatural language description of each program’s behaviour out of four optionsin randomized order. Both of these triggers use signals and operators withwhich the participants are familiar, but do not combine them in a way thatrelates to any of the previously seen tasks or to each other. We intentionallychose this design so that participants would need to read the given codecarefully without having to learn new concepts beyond those tested by the114Figure 5.6: A more complex trigger with nested operators, used totest participants’ program comprehension.programming tasks.Comparative Survey As the last step in the experiment, we investigatedhow participants perceived the design of the environment to which they werenot assigned. To answer this question, which is related to RQ3, we showedparticipants the same programs that they had just seen during the programcomprehension component of the experiment, as well as the correct naturallanguage description of the program. We then asked participants to rate thisenvironment on three 5-point Likert items in comparison to the one they hadseen up to this point. Like in the usability survey, we distinguished betweenthe perceived learnability of the design and the ease of reading and writingprograms. We also gave participants a final opportunity to provide writtencomments about their ratings.The comparative survey is an addition to the experiment that requiredlittle extra participant time as we re-used the same programs as for the pre-vious component of the experiment. This comes with two important caveatsthat should be considered when interpreting the results. First, participantshave not had a chance to actually use the alternative design and judge itsusability beyond a first impression. Second, we expect participants to bebiased as they have seen the other design first and are more familiar with it.However, both of these caveats apply equally to both experimental groups.Therefore, we compared responses between groups rather than relying onabsolute rating numbers.1155.3.4 Experimental ResultsFollowing the study procedure outlined in Figure 5.5, a total of 179 partic-ipants successfully completed the pre-screening and 163 of them agreed toparticipate in the randomized experiment. Of those participants, 49 decidedto withdraw from the study before completing the additional programmingtutorials. Unfortunately, we have no insight into why they decided not toparticipate further. It might have been because they struggled to follow thetutorials, or because the study required too much effort for the compen-sation we provided. Participants who did complete the tutorials took 28.8minutes on average (SD=10.2) to do so, with is slightly longer than antici-pated. Upon closer inspection, the longer completion time can be attributedto the third tutorial, which introduced triggers. It took participants 16.2minutes to complete on average (SD: 6.0) while the first two tutorials stayedwithin their expected duration. The first tutorial took 5.1 minutes on aver-age (SD=2.1) and the second tutorial took 7.6 minutes on average (SD=4.1).One participant was excluded for neither attempting any programming tasksnor providing any survey responses.In the remainder of this Section, we present the data for the 113 par-ticipants who completed at least the programming component of the ex-periment. On average, these participants were 28.6 years old (SD=9.6); 44(39%) identified as female and 68 (60%) identified as male. One participantopted not to disclose their gender. Participants listed 23 distinct countries ofresidence, and 78 (69%) identified themselves as at least part-time employed.When we refer to the evaluated groups, we will call them Graph andBlocks for brevity, as the only difference between them was the program-ming paradigm used to define triggers. For all statistical tests, we use𝑝 < 0.05 as significance threshold.RQ1: Overall Effectiveness of Trigger-based and Task-based Pro-grammingFor each participant in the experiment, we recorded whether they fin-ished each programming task and how long it took them if so. For Task 1, 68out of 113 participants (60%) were successful, and for Task 2 50 participants(44%) were successful.We further collected the participants’ final programs as well as inter-mediate programs whenever they decided to run tests. The number of testruns ranges widely between participants for both tasks. On average, partic-ipants ran their programs 7.04 times (SD=5.83) for Task 1 and 8.43 times(SD=6.86) for Task 2, which includes the final, successful run if there was1160 5 10 15 20 25 300.00.20.40.60.81.0Task 1 Compl. Prob. BlocksGraph0 5 10 15 20 25 30Time (min)0.00.20.40.60.81.0Task 2 Compl. Prob. BlocksGraphFigure 5.7: Completion probability over time for the two program-ming tasks. Shaded areas are 95% confidence intervals.one. The median of runs is substantially lower at 4 runs for Task 1 and6 runs for Task 2, illustrating that most participants only ran their codea few times while a few used up to 34 attempts. We manually examinedsuccessful and unsuccessful participants’ testing behaviour individually butdid not find any notable differences.Finally, we also measured participants’ ability to comprehend two morecomplex examples of triggers. We found that 84 (76%) of the 110 partici-pants who completed the program comprehension tasks correctly answeredthe first comprehension question and 78 (69%) correctly answered the sec-ond question. Most participants who correctly answered one of the twoquestions also answered the other one correctly; 68 participants (62%) cor-rectly answered both questions.RQ1 Summary: Overall, 60% of participants successfully solved the firstand 44% solved the second task.RQ2: Performance Differences Between the EnvironmentsIn addition to analyzing the overall participant performance, we alsoanalyzed the individual groups and compared their results. Of the 59 Blocks117participants, 42 (71%) solved Task 1 and 30 (51%) solved Task 2. Of the 54Graph participants, 29 (54%) solved Task 1 and 20 (37%) did the same forTask 2.To analyze both the completion times and success rates of the partici-pants under the given time limit, we performed survival analysis for bothgroups and tasks [136]. The survival function 𝑆 (𝑡) models the probabilityof a participants’ survival over a given period of time. In the context ofthe study, a participant “survives” as long as they have not yet success-fully finished a task. Since we care for the probability of participants havingcompleted the task after a given time frame, we plot 1 − 𝑆 (𝑡) in Figure 5.7.We fit Kaplan–Meier estimators [136] on the success rates and times ofthe participants. Figure 5.7 shows the resulting 95% confidence intervalsfor the completion probabilities. We also performed a log-rank test [137]for each task to determine if the difference between the two curves is sta-tistically significant. We found 𝜒2Task1(df=1; N=113) = 4.92, 𝑝 = 0.03 and𝜒2Task2(df=1; N=113) = 3.08, 𝑝 = 0.08. Therefore, only the difference forTask 1 is statistically significant.For both tasks, we found only minor differences in the testing behaviourof the two groups: Participants in the Blocks group ran their programs 6.93times (median=4, SD=6.34) for Task 1 and 8.30 times (median=5, SD=7.46)for Task 2, compared Graph participants executing 7.15 runs (median=5,SD=5.45) for Task 1 and 8.61 runs (median=6, SD=6.44) for Task 2.For the comprehension questions, we also found that participants in theBlocks group performed better than those in the Graph group. Participantsin Blocks understood the trigger shown to them correctly in 50 (85%) and46 (78%) out of 58 cases for the two respective tasks. The Graph usersunderstood the trigger in 34 (65%) and 32 (62%) out of 52 cases for thetwo respective tasks. We find 𝜒2Comp1(df=1; N=110) = 6.59, 𝑝 = 0.01 and𝜒2Comp2(df=1; N=110) = 4.20, 𝑝 = 0.04, and therefore a statistically signifi-cant difference for both tasks.RQ2 Summary: Participants in Blocks were more successful at solvingTask 1 as well as the program comprehension questions than those inGraph. We did not see a statistically significant difference between thegroups for Task 2.RQ3: Participants’ Perception of the EnvironmentsIn addition to measuring participant performance, we also asked them torate the environment they used regarding its learnability and the readabilityand writability of programs. We did so with two sets of questions to separatethe trigger editor from the surrounding programming environment for robot118Blocks Graph0%25%50%75%100%LearnabilityBlocks Graph0%25%50%75%100%ReadabilityBlocks Graph0%25%50%75%100%Writability54321Blocks Graph0%25%50%75%100%Blocks Graph0%25%50%75%100%Blocks Graph0%25%50%75%100%5432150% 0% 50%BlocksGraph50% 0% 50%BlocksGraph50% 0% 50%BlocksGraph210-1-2TasksTriggersComparisonFigure 5.8: Distribution of responses for the Likert items of the sur-vey, split by participant group (Blocks: N=58, Graph: N=52).For the bottom row, bars are aligned around neutral responsesto indicate diverging preferences.tasks that was identical for both groups. We provided participants withthree 5-point Likert items, which we code as 1 for a strongly negative ratingand 5 for a strongly positive rating. The resulting matrix of responses isshown in Figure 5.8.Participants from both groups used the same editing interface for robottasks and gave it highly similar ratings as expected. The average ratings forthe task editor are 3.75 for learnability (Blocks: 3.72, Graph: 3.77), 3.76for readability (Blocks: 3.74, Graph: 3.77) and 3.54 for writability (Blocks:3.55, Graph: 3.54). For the trigger component of the used environment, wefind larger differences. The average ratings here are 3.17 for learnability(Blocks: 3.41, Graph: 2.92), 3.22 for readability (Blocks: 3.45, Graph:2.96) and 3.04 for writability (Blocks: 3.34, Graph: 2.73). These ratingssuggest that the participants found block-based triggers more usable overall,119with a particularly strong preference when rating how easy it was to writeprograms.The collected usability ratings should not be interpreted on an abso-lute scale as users rate usability with a substantial bias towards higher rat-ings [138]. We did not use a standardized questionnaire, and therefore onlyuse the usability ratings to compare the two participant groups to each other.This type of comparison can also be affected by statistical biases, but theeffects are typically small [139].We performed a one-way MANOVA across all six Likert items. Thisanalysis finds that 𝐹 (df=6, 103) = 2.34, 𝑝 = 0.04, indicating a statisticallysignificant difference between the groups. Individual significance tests forthe task items find: FTaskLearn(df=1, 108) = 0.06, 𝑝 = 0.81, FTaskRead(df=1,108) = 0.02, 𝑝 = 0.88, and FTaskWrite(df=1, 108) = 0.01, 𝑝 = 0.95. For thetrigger items, they find FTrigLearn(df=1, 108) = 5.58, 𝑝 = 0.02, FTrigRead(df=1,108) = 5.00, 𝑝 = 0.03, and FTrigWrite(df=1, 108) = 8.13, 𝑝 < 0.01. There-fore, the differences between all three ratings of the trigger component arestatistically significant, and those for the task items are not.In addition to asking participants to rate their assigned environment, wealso showed them triggers as they would be represented in the alternativeenvironment. We asked them to indicate which of the representations theyprefer, using the same three items (learnability, readability, and writability)as for the previous survey. We code responses on a scale from -2 (strongpreference for the other environment) to 2 (strong preference for assignedenvironment).The participants in the Graph group gave average ratings of 0.19 forlearnability, 0.09 for readability, and 0.19 for writability, indicating a weakpreference for their own design. For the Blocks group on the other hand,participants gave average ratings of -0.41 for learnability, -0.46 for readabil-ity, and -0.45 for writability, indicating a preference for the representationused by the Graph group. A one-way MANOVA across the three items finds𝐹 (3,106) = 3.49, 𝑝 = 0.02, and individual tests find FCompLearn(1, 108) =9.05, 𝑝 < 0.01, FCompLearn(1, 108) = 5.95, 𝑝 = 0.02, and FCompLearn(1, 108)= 9.61, 𝑝 < 0.01 for the comparative ratings. Therefore, all differences arestatistically significant.RQ3 Summary: Participants in Blocks gave significantly higher usabil-ity ratings for their assigned trigger editor than those in Graph. However,both groups expressed a preference for the graph-based editor in directcomparison.1205.3.5 LimitationsHere we discuss limitations of the experiment.Participant Recruitment: We recruited participants using the Prolificonline platform as it made it feasible to pick a large sample of end-userswithout being limited to a single geographical location or employer. Asdescribed in Section 5.3, we applied a variety of measures to reduce the risksof using a crowdworking platform. We also provide a range of demographicdetails about the participants in Section 5.3.4 that lead us to believe thatwe captured a wide range of participant backgrounds. Nonetheless, we areaware that the sample might not fully capture the large variety of potentialend-users who might be asked to write robot programs in practice.Attrition: As stated in Section 5.3.4, a substantial number of users de-cided to withdraw from the experiment before completing the programmingtutorials. These participants have been fairly evenly distributed betweenboth participant groups (24 in Blocks, 27 in Graph). However, we haveno information about why these participants withdrew, and our inability toanalyze their data might introduce bias into the experiment.Task Selection: To ensure that we could ask the participants to completereasonably complex programming tasks, we had to limit the number of taskswe asked them to solve. We could have selected a larger pool of tasksand randomly assigned them to participants, but this would have requiredsplitting the participants into more groups and limited the statistical powerof our results. As a result, we had to select only two programming and twocomprehension tasks to represent a wider range of practical scenarios. Wejustified our selection in Section 5.3, but the external validity of our resultsis limited without further validation on different tasks.Training and Time Limits: Due to the limited time available duringthe study, we had to restrict both the training we could provide to theparticipants and the time that they were allowed to work on each task.Participants might have performed better overall if they had received moreextensive training, and more participants might have completed the tasksif given more time. Similarly, participants might have been able to provideus with more detailed feedback and been able to provide a more distinctfeedback on aspects of the system. For example, we see in Figure 5.8 thatparticipants showed little distinction between learnability, readability andwritability in their responses. With more time available to allow them tobecome familiar with the systems, and with room for more tasks that fo-cus on one of the three aspects, we might have received more meaningfulresponses. However, these limitations affected both participant groups to a121similar extent, leading us to believe that they would not have significantlyaltered the outcome of the study.5.4 Discussion and SummaryIn this section, we discuss and interpret our findings. We also include se-lected participant quotes from the 46 total written comments collected usingthe survey at the end of the experiment.5.4.1 Success and Usability: Blocks Beat HybridThe results of our experiment confirm the findings of previous studies [36,114] that an end-user friendly programming environment allows end-users toprogram mobile robots with little training. Compared to those studies, weinvestigated substantially more complex tasks that involved writing robotprograms as well as logic that determines when the program gets executed.Therefore, it is unsurprising that even the better-performing of the twoparticipant groups had a lower success rate than participants in previousstudies. Examining Figure 5.7, it is likely that a more generous time limitmight have allowed more participants to finish the tasks. Nonetheless, weconsider it remarkable that more than half of the participants could solvethe tasks despite the necessary constraints.Our findings also support our assumption that the trade-offs we pre-sented in Section 5.1 are relevant for end-users and affect how they writecode. We found that participants were faster and more successful writingprograms in an environment that uses blocks as its only representation forcode. This might be unsurprising, considering the additional effort requiredto learn two distinct editing styles. Some participants echoed this senti-ment, writing about block-based trigger editing that “[i]t’s more similar toprogramming tasks,” and about graph-based editing that “[t]he lines andstructure are relatively complex.” However, few comments mention issuesregarding the different editing experiences between the two styles, with theexception of a few rare comments like “I couldn’t find how I could changethe lines, so I ended up scrapping it and starting over.” Instead, severalparticipants commented on data-flow graphs being difficult to read in gen-eral, criticizing “random arrows [that] had two separate ends and were veryconfusing,” and that “it appears like there is a hierarchy even when there isnot.”Participants not only found it harder to write programs using data-flowgraphs, but also struggled to read and understand them. This is not just122supported by comments like the previously mentioned ones, but also by theresults of the program comprehension tasks. For both tasks, users that werepresented with a data-flow graph performed significantly worse at identifyingwhat the given trigger was supposed to do. Notably, for the comprehensiontasks we showed users the trigger code in isolation. Therefore, we speculatethat data-flow graphs might be harder to read for end-users even outside ofthe context of a hybrid programming environment.5.4.2 (Why) Do End-users Prefer Graphs?One finding of the survey does not fit with the other results: in a directcomparison, end-users prefer data-flow graphs over blocks. One reason forthis observation could be that data-flow graphs look more interesting orvisually appealing. All users had exposure to blocks, but those users whoused them exclusively did never see data-flow graphs before the point wherethey had to rate them. They might have perceived them as a more elaboratevisualization and the overall system as more complex or mature. Therefore,novelty alone (from the perspective of the user) might be a reason why ourparticipants showed a preference for graphs.Another potential reason for participants’ preference could be that weonly presented users with data-flow graphs such as the one shown in Fig-ure 5.3, which we manually arranged to be easy to read for the given task.Compared to the block-based equivalent program, some users wrote that“[t]he other one gets more clean and organized.” and that it “looks muchbetter and more intuitive than what I used.” We speculate that users com-pared our arrangement to the block-based program in Figure 5.2b, whichleaves no room for visual customization or aids. Some users also explicitlystated that the block-based code “looks very messy” and raised concernsthat it “can get very spaghetti-y, very fast.” While some participants statedabout graphs that “in larger scale this looks better to program”, we havediscussed the higher demand for space as a potential downside of graphswhen it comes to larger programs.5.4.3 Implications for Other DomainsBoth block and data-flow programming have been adapted to a variety ofend-user programming domains. However, to our knowledge there exist nohybrid systems like the one we presented here. Our findings can be inter-preted as justifying this lack of hybrid systems. It appears that even minoradjustments to the visual presentation of block-based languages such as123those we performed in our entirely block-based system can achieve superiorresults despite theoretical limitations and caveats.Remarkably, our results also provide initial evidence that end-users arebetter at understanding blocks than data-flow graphs even when each isused in isolation. We observed this in our program comprehension tasks,and the participant comments support this finding. This seems to contra-dict the more widespread commercial adoption of data-flow graphs comparedto blocks. Although blocks are used in select commercial robotics applica-tions [32], data-flow graphs have found wide-spread use across several otherdomains, particularly in game development [140, 141] and Internet-of-Thingsprogramming [142]. Our findings raise the question of whether exploringblock-based approaches might bring benefits for end-users in these domainsas well.124Chapter 6Towards End-user-centricDesigns for Domain-specificProgramming LanguagesIn Chapters 3, 4 and 5, we present tools that expand the range of roboticsapplications that end-users can develop. All of the presented prototypes arefounded in block-based programming and the underlying design principlesand features we present in Chapter 2. However, each prototype extendsbeyond block-based programming alone and integrates concepts from vi-sual programming into its design. In summary, the concepts used by eachprototype are:Chapter 3: The side-by-side presentation of two programs, their alignmentto represent parallelism, and the use of synchronized commands thatconnect the two programs. In contrast to traditional block-based codethat resembles text with a linear flow from top to bottom, this proto-type adds a second dimension that represents the assignment of com-mands to a specific robot arm.Chapter 4: The decomposition of a large robot program into multiple spa-tially localized tasks that can be navigated using an overview-detaileditor. In contrast to traditional block-based environments that pro-vide users with free-form functional decomposition and a full set ofsyntactic options at all times, this prototype guides users to ensurethat their programs follow a specific, domain-appropriate structure.Chapter 5: The representation of nested trigger expressions as data-flowgraphs in combination with blocks for sequential commands. In con-trast to traditional block-based systems that follow a single visualmodality and represent all code using it, this prototype aims to maxi-mize the benefits of multiple visual presentations for different elementsof a program.125In this chapter, we reflect on the design approach we have taken todevelop the prototypes we present in this work. We discuss the lessons welearned about empirically designing languages and tools that target end-users. We further discuss some of the limitations and caveats of this work,and how we can generalize the design elements and techniques we use toother settings within the domain of robotics, and to other domains whereend-users program.6.1 Empirical Programming Language Design forEnd-usersDesigners of programming tools and languages often prioritize novel featuresin their work that make their systems more expressive and provide users withuseful information about their programs. Factors such as how easy it is tolearn and use a new system are often secondary, and are evaluated fromthe perspective of the authors, who are inherently familiar with their owndesigns. There exist a number of historical studies of how end-users [143]and professional developers [144] use programming tools. These studieshave established general guidelines for designers. For example, that it isimportant to provide users with abstraction and encapsulation layers, andthat it is easier to read instructions (in user interfaces or as programminglanguage keywords) in natural language than in abstract notation. However,as programming languages, their features, and the ways in which they areused evolve over time, it becomes less clear if and how these guidelines applyto modern systems. Unfortunately, it is rare for studies to be replicated tosee if they stand the test of time. Iinstead of informing design, modernempirical work is often only used to demonstrate the benefits of languagesand tools after the design process has been completed, and its scope andrigor may be too limited to inform future work [145].Recently, programming language researchers have intensified their ef-forts to incorporate methods from the field of Human-Computer Interaction(HCI) into their work. The most comprehensive and generally applicablelanguage design process that has been presented is called the ProgrammingLanguage Iterative Evaluation and Refinement System (PLIERS) [146]. Thissystem proposes a methodology to develop programming languages itera-tively and with feedback from potential users, evaluating languages not justsummatively but also formatively.PLIERS aims to be broadly useful for programming language research,but focuses on languages and tools for professional developers. In the context126of end-user-centric work, it can only be applied with caveats. For example,PLIERS assumes that potential users have previous programming experiencethat can provide them a frame of reference. If users have used establishedprogramming tools, they can actively compare new designs to the ones theyhave used previously and evaluate perceived trade-offs on their own. End-users, on the other hand, often have no previous exposure to programmingand therefore need more guidance on which novel aspects of a tool theyshould evaluate and which ones are already established. In addition, theymight also feel overwhelmed by terminology used in questions and mightneed more context or clearer examples to be able to provide valuable insightsabout specific language features.PLIERS does mention one of the biggest challenges we observed in ourwork: the trade-off between creating a system that meets the needs of pro-grammers by offering them a wide range of expressiveness, but also keepingthe system simple enough to be usable. However, this trade-off is even morerelevant for end-users than it is for professionals. It can be harder to find acompromise that is acceptable for end-users, since they need a lower barrierof entry than more experienced programmers, but still have expressivenessrequirements that a system needs to meet. Professionals who plan to usea system over a long period of time are also more likely than end-users toactively look out for new features and be able to learn the ones they discoverin a self-directed manner [147]. Addressing this challenge does not requirean entirely new design approach, but additional attention to usability andshort-term learnability, and might limit whether findings from studies onprofessional developers apply to end-user tools.PLIERS also has limitations when applied to languages that are nottext-based. In particular, PLIERS proposes the rapid development of mock-ups and prototypes to gather user feedback and compare design candidates.However, this form of iteration is far easier when tools are entirely text-based and little effort is needed to make superficial changes to designs. Forexample, it might be a matter of minutes to replace a keyword in a text-basedprogramming language with one that might be easier to understand, butchanging the design of a visual language can require substantial developmenteffort. Similarly, it can be easier to provide text-based programs to usersand have them understand how they could make changes to them as themodality is familiar to them. In contrast, the necessary interactions withvisual tools are often harder to predict just from looking at a program or animage of a tool’s user interface.Although we conducted most of our design work before PLIERS waspublished, we use similar techniques as those proposed by its authors. Our127language design process follows an iterative structure where early drafts arerefined by direct end-user feedback or by analysing them through frameworkssuch as the 13 Cognitive Dimensions of Notations (CDN) [1]. Our work onside-by-side coordination in Chapter 3 is particularly influenced by both theCDN and by formative feedback gathered through a large end-user survey.PLIERS mentions analytical work and the CDN, but puts more emphasison empirical studies to inform design decisions. However, based on ourown work and experience, we believe that a thorough theoretical usabilityanalysis is essential to inform the available design options and to guideempirical work. As the authors of PLIERS note themselves, user feedbackis not always sound and reasonable, and we therefore believe in presentingusers only with design alternatives that are consistent with the overall goalsfor the system.In addition to formative studies, PLIERS outlines how user studies andexperiments can evaluate new languages summatively. We have also con-ducted such summative evaluations in our work. In particular, we performeda controlled experiment to evaluate our environment for mobile robot pro-gramming in Chapter 4, and a similar one in our follow-up work on coor-dination in Section 3.4.2). PLIERS outlines some of the challenges of suchevaluations, notably the difficulty of separating different language aspectsto judge their impact separately. For example, in Chapter 4 we evaluatedboth language and environmental design choices at once, leaving it unclearwhich of them contributed how much to our overall results. The authorsof PLIERS suggest conducting several isolated studies on features, whichcomes with the price of either having to recruit more participants or findingresults that are statistically less meaningful. In our work, we decided tofocus on a bigger-picture comparison to demonstrate the possible impact ofguided decomposition for end-users, leaving a more detailed analysis of theindividual design choices for future work.Another insight that we gained through our work lies in the recruitmentof study participants. PLIERS suggests recruiting senior undergraduate orgraduate students as participants for studies as they have programming ex-perience and are easily available to researchers at universities. Whether stu-dents can actually substitute for professional developers is questionable, butthey are even less likely to be good replacements for end-users. This is notjust because students might have more programming experience than end-users, but also because real end-users are likely more mature and have dif-ferent needs [148]. In our work, we found that recruiting end-users throughonline platforms such as MTurk or Prolific is an effective way to find a largenumber of participants as long as measures are taken to counter the draw-128backs of these platforms. Bots and participants with low motivation to workon study tasks can bias the results of studies and limit the amount of usefulfeedback that can be gathered. However, from our experience, the use ofpre-screenings and interesting, non-trivial tasks can lead to insightful results.Notably, we did recruit students (from different areas) for our follow-up workon two-armed coordination (see Chapter 3.4.2). For experiments that musttake place in-person, students can be a reasonable compromise as long asthey are screened for factors such as programming experience.A challenge that the authors of PLIERS identified, but that we believeto be less significant for end-users, is the lack of time to train participantsor ask them to complete complex tasks, which limits a study’s externalvalidity. Time restrictions are a consistent limiting factor that we identifiedin all of our studies, but a limited amount of training and time for taskcompletion is to be expected for end-user systems. When evaluating systemsfor professionals, it is often hard to predict how differently developers wouldinteract with a system after weeks or months of active use. For end-userson the other hand, we do not typically expect them to develop the samedegree of competence using a system, as programming is only a small partof their daily work. Conducting a shorter study with limited training andsmall tasks can therefore be a realistic simulation of their user experience.6.2 Limitations of the Visual Design ApproachOur work introduces domain-specific visual elements into block-based pro-gramming while retaining most of the benefits of block-based systems. How-ever, the extensions we present still come at a cost, which affects both theimplementation and end-user experience. In this section, we discuss someof these limitations, while the larger question of whether our designs cangeneralize beyond a single domain is addressed in Sections 6.3 and 6.4.Regarding implementation, our systems do introduce overhead, both indesign and implementation, compared to creating an off-the-shelf block-based editor. There exist a number of tools to create block-based program-ming systems with minimal or even no programming knowledge [34, 84]. Al-though our systems are using the same established frameworks targeted bythese tools, they still require substantial implementation effort and customsolutions for design challenges like the ones we presented in each chapter.Nonetheless, our systems require a smaller amount of design and imple-mentation effort to build than brand new visual languages. For example,the two-canvas system presented in Chapter 4 needs to coordinate multiple129block-based canvases, but does not re-implement the underlying user inter-faces and workflows, such as dragging and attaching blocks. Similarly, thesystem we introduce in Chapter 5 combines block-based and graph-basedediting, both of which rely on established editing tools that only need to beconnected to create a single program across both editors.Our designs also introduce new user experience challenges that arise in-dependently of their implementation. Users who might have already used atraditional block-based environment might find the differences in how theyinteract with the system unexpected or frustrating. As we have illustratedin Chapter 2, traditional block-based systems also show a substantial rangein their design, so we consider this a rather small risk. However, modernblock-based tools are moving towards consolidation around a more rigid setof frameworks and design practices, which our work does not fully com-ply with. We therefore expect this trade-off between standardization andcustom-tailored tools to become more relevant for future work.Another aspect of user experience that our work does not yet addressis accessibility. Although block-based programming may appear inherentlyinaccessible at first due to its focus on mouse-based and highly visual inter-actions, previous work has demonstrated that it can be adapted for visuallyimpaired audiences [149, 150]. Some of these modifications, like using highercontrast colours or offering visual customization options, can be directly ap-plied to our work as well. None of our presented designs extend the meaningof colours or block shapes beyond the way they are already present in tradi-tional block-based systems. By reducing the amount and number of differenttypes of blocks on screen, they might even help alleviate some of the difficul-ties that visually impaired users might have reading block-based programs.However, some features that were designed with blind users in mind requireconverting programs into text. These features are harder to replicate byour designs due to their greater emphasis on visual features and thereforereduced closeness to text-based code.In the context of our chosen domain of robot programming, we believethat accessibility challenges are not limited to the programming interface,but also other human-robot interactions such as positioning or testing anddebugging programs. Substantial additional work is needed to investigatethe accessibility challenges of this domain and develop accessible designguidelines. In the scope of this thesis, we cannot address these issues, butnonetheless, we believe that our designs could be adapted for blind or visu-ally impaired users. For example, our side-by-side design presented in Chap-ter 3 could be translated into a text-based grid that can be navigated witha four-way cursor and read out to the user. Previously proposed ideas such130as uniquely identifying blocks [149] could be used to indicate correspond-ing left- and right-armed command blocks to users. Notably, our designcandidate with implicit line-by-line synchronization might be more suitablefor such an interface as it is more regular in its grid-like structure. Ourother designs in Chapters 4 and 5 could see similar navigation adaptions. Inparticular, our guided decomposition approach might be suitable for usersof non-mouse-based interfaces because it reduces the number of availableblocks in each canvas as well as the number of total blocks on screen.6.3 Pushing our End-user Robotics Designs toNew Use CasesThe designs that we present in our work address specific use cases for robotsand are custom-tailored for their purposes. On a language level, we havetypically chosen a subset of commands that is reasonable as a minimal vi-able prototype. Our designs might be easily adapted or extended by addingnew types of blocks to support additional robot commands. However, moreelaborate extensions that interact with the visual elements that we incorpo-rated into the designs might be more challenging to realise and could leadto unexpected and severe drawbacks.As an example of an extension we found surprisingly difficult to achieve,consider the design we introduce in Chapter 3 for coordinating two robotarms. Field studies have shown that few practical tasks require coordinatingmore than two robots directly [99]. Nonetheless, one might wonder whethermore tracks could be added to extend our prototype beyond two-armedrobots. We have created early drafts of how such a design could look, asillustrated in Figure 6.1. This example shows a program where three robotarms hand over an item from one robot arm to another and then to a thirdone. To support the third robot arm, we introduce a third column thatblocks can be placed in, with programs still being executed top-to-bottomwith explicit synchronization blocks to make the arms wait for each other.Although the example in Figure 6.1 is conceptually similar to the two-armed program shown in Figure 3.5b, it suffers from significant usabilitydrawbacks. First of all, the more robot arms a program gets distributedacross, the more space remains unused, reducing the visibility of the code.The example is also harder to read sequentially, as it flows in two dimensionsinstead of just one. In this example, the program flow is still linear, from thetop-left to the bottom-right, but other programs might flow back to the leftor even jump arbitrarily between the columns. Although this example only131Open Hand.Move Arm to Place .Move Arm to Above Place .—————— Wait for each other —————— Open Hand.—————— Wait for each other —————— Close Hand.—————— Wait for each other —————— Move Arm to Second Handover Middle . Move Arm to Second Handover Right .—————— Wait for each other —————— Open Hand.—————— Wait for each other —————— Close Hand.—————— Wait for each other —————— Move Arm to First Handover Left .Close Hand.Move Arm to Pickup .Move Arm to Above Pickup .Open Hand. Left Arm, do: Middle Arm, do: Right Arm, do:Move Arm to First Handover Middle .Figure 6.1: Early design draft for a 3-armed extension of the verticaldesign with explicit synchronization that we present in Chap-ter 3. It uses 3 columns to represent the individual robot arms.shows three arms, one can imagine that any larger number of arms wouldfurther harm the program’s readability.Another challenge is finding a reasonable model of synchronization. Oneway to represent synchronized commands would be to introduce a globalbarrier that halts all programs for all arms. This form of synchronizationis easy to understand, but might be unreasonably inefficient in practice.Therefore, in the example we show here, we chose a different form of syn-chronization where only two arms are waiting for each other at a time. Thisbehaviour leaves room for concurrent commands that the third arm can ex-ecute. For example, if the first command “Move Arm to Second HandoverRight” was a particularly complex movement that required time to perform,the arm could execute it while the other two arms perform the first han-dover. However, the way we represent this synchronization in blocks onlyworks when two adjacent arms are synchronized as shown in Figure 6.1.If, for example, the left and right arm were to interact, a new block thatskips the middle column would be required, which is difficult to represent132Move Arm to Handover Left . Follow Movement.Close Hand. Close Hand.Move Arm to Pick-up Left . Move Arm to Pick-up Right .Carry Item with Arm A and Arm B .Carry Item with Arm B and Arm C .When program is startedCarry Item with Arm A and Arm B .Figure 6.2: Early design draft for an environment that features bothtwo-armed coordination as presented in Chapter 3 and guideddecomposition as presented in Chapter 4. The left canvas showsthe high-level program flow, which is composed of a series oftwo-armed programs that are defined in the right canvas.visually and might be misunderstood by users. Adding more arms wouldundoubtedly complicate such a design further.Although our first draft as shown in Figure 6.1 is flawed, one should notconclude that the design we presented in Chapter 3 is inherently incompat-ible with more robot arms. To address the issues above, we have createdfurther drafts with the goal to represent a pipeline of robot arms. Thesearms might be physically located in a way so that we only need to considerpairwise interactions, as shown in Figure 6.1. Such a pipeline could executeprograms in parallel, with the left arm picking up a new item once it hashanded the previous one over to the middle arm. Unlike the shown example,the robot arms would effectively execute a loop with the goal to optimize thethroughput of executed commands (and therefore processed items). Such adesign might be useful for certain real use cases of robot arms, and it mightbe worthwhile to investigate the representation of loops in block-based lan-guages further. We have decided not to continue this design path in thisdissertation, but consider it a promising area of future work.Another interesting example of how our work could be generalized isto combine our work from Chapters 3 and 4. In Chapter 4, we present anenvironment that supports guided decomposition for mobile robot programs.133We discuss the importance of finding an appropriate form of decompositionfor other programs in Section 4.4. However, coordination is one concreteexample of a potential use case for guided decomposition. Figure 6.2 showsan early design draft for an environment for programming multiple robotarms using the same concepts as we presented in Chapter 4. In this design,tasks are parameterized by specifying the robot arms that are supposed tointeract instead of workstations where tasks take place. Selecting a task inthe left canvas shows its definition, which uses the side-by-side layout thatwe present in Chapter 4. By selecting different robot arms, tasks can bere-used for multiple purposes.Compared to our previous draft shown in Figure 6.1, the design in Fig-ure 6.2 limits all robot interactions to two arms at a time. However, italso structures programs around this limitation. This design principle issimilar to the one we used for mobile robots in Chapter 4, where programswere decomposed based on space rather than coordination. In principle,this design could make programs more readable, but it might also structurethem in a fragmented way if many different combinations of robot armsneed to interact. Because of the implementation limitations that we dis-cussed in Section 3.4.2, we could not pursue this design draft further as itwould have required too many separate canvases and too much screen space.Nonetheless, we believe that this draft demonstrates the potential for guideddecomposition in a different style than mobile robots.Overall, the design drafts shown here demonstrate some of the possibledesign space for using our prototypes in the context of robot programming.In theory, one might also combine our trigger-based design prototypes shownin Chapter 5 to create a single environment that shows off all our work inone combined system. However, we consider it unlikely that a system thatrequires end-users to learn such a large number of features and concepts iseffective in practice without adding further support. We therefore advocatefor selecting the necessary features for a concrete use case carefully andopting for their careful integration over sheer quantity. For this reason, wehave separated our prototypes rather than creating a single environmentthat aims to fit all purposes at once.6.4 End-user Programming Beyond RoboticsSeveral of the visual features that we incorporated into block-based program-ming throughout our work are highly specific to the domain of robotics.The presentation of two programs side-by-side to match the alignment of134physical robot arms and decomposition of programs based on the physicalworkstations that a mobile robot operates are just two examples of thesedomain-specific adaptations. Although we have shown in previous chaptershow each of these features benefits end-users in robotics, they also limit howour block-based designs could be used in other domains. For example, mostprograms outside of robotics do not consider physical space or the arrange-ment of hardware components. Even in cases where the layout of a systemis relevant, such as in distributed computing, tool designers might intention-ally hide these aspects of the system rather than incorporating them intotheir designs, for example by using abstractions to separate them from therest of the program code.We believe that some of our experimental findings have direct impli-cations beyond specific domains. We discuss these implications in Sec-tions 3.5.2, 4.4.3 and 5.4.3. However, we believe that there are more funda-mental design features that go beyond our concrete prototypes and transferto other domains as well. As in previous chapters, we use the terminol-ogy of the Cognitive Dimensions of Notations to discuss them here anditalicize and underline CDN terms.One of the most central concerns that of all our designs aim to address isthe Visibility of programs and their semantics. The need for programs andtheir semantics to be visible is well-understood in the context of visual pro-gramming languages [2], but usually receives less focus in the developmentof block-based programming tools. Block-based programming provides someinherent visibility features, such as the colour coding of different blocks andthe use of shapes to highlight connections between commands. However, alesson we learned from our own language designs is that this support is notsufficient to make complex programs easy to read and understand. Visualprogramming features can make programs more visible on multiple levels, asour design for coordination illustrates. Using horizontal screen space moreeffectively keeps larger amounts of code visible, following the data visual-ization principle of “eye beats memory” [151]. Aligning and connecting thetwo coordinated programs also makes their interactions more visible andhighlights the order in which they are executed. Our other designs followa similar approach: they aim to maximize the amount of code that can bedisplayed and the ease of navigating it, but also address semantic concerns,for example by highlighting data flow in nested expressions.As a consequence of increasing the visibility of complex parts of a pro-gram, all of our designs also reduce the Error-proneness of programs. Recallour initial example of a block-based program that is hard to read from Fig-ure 1.2. In this program, blocks were ordered and scaled based on the135syntax of the underlying text-based program, resulting in often arbitraryconnections between aligned blocks. In contrast, our designs direct the pro-grammer’s attention to the parts of the program that are the hardest tocomprehend and edit. In the example of our program decomposition design,we even decided to structure the entire programming interface and navi-gation around tasks and the physical space in which they take place. Bymaking complex elements of a program explicit first-class features of theprogramming environment, we ensure that end-users do not neglect them ormake errors based on a lack of information.In addition to focusing the attention of end-user programmers on thehardest to understand parts of the programs they edit, all of our designsalso aim to simplify these parts and make them easier to comprehend. Bymaking properties of the programs explicit, an inherent goal of visual pro-gramming, they reduce the frequency of Hard Mental Operations. In mostcases, we use the layout of program code to make mental operations easier,for example by arranging code in a way that matches the order of executionin both our designs for coordination and trigger evaluation. Compared totext, blocks provide the ability to do so as it is not necessary to arrange pro-grams strictly top-to-bottom and left-to-right. However, we also go beyondlayout as we support users in making decisions about their programs. Al-though our design for program decomposition is the most obvious examplefor simplifying the decision space for end-user programmers, we also use thistechnique more subtly in our other designs. For example, in our coordina-tion prototype, we provide only a single mechanism to synchronize the tworobot arms, compared to the low-level approaches offered by languages likeScratch and Mindstorms EV3. Similarly, in our trigger-based environment,we explicitly divide programs into tasks and triggers with distinct syntacticelements that cannot be combined arbitrarily. We believe that although thisreduction of possible program semantics can limit the expressiveness of theoverall system, it is often essential to support end-user friendliness.One final aspect that is shared by our systems, but less explicit in ourdesigns is a focus on limiting the language’s Diffuseness. A programminglanguage is more diffuse the more different elements or concepts a user has tolearn to understand it. Admittedly, as all our systems are prototypes, theyare likely less diffuse on a syntactic level than a production system mightbe, as they do not need to support all potential commands that the robothardware supports. However, our systems also follow this design principleon a different level, as they aim to re-use existing block-based concepts andfeatures when there is no clear benefit to replacing them. For example, weconsidered the addition of arrows and other markers to highlight synchro-136nization points in our two-armed environment in Chapter 3, but found thataligning the blocks vertically is sufficient for most users and fits with theexisting jigsaw puzzle design of blocks. Re-using design concepts not onlyreduces the learning effort for end-users, but also ensures that the imple-mentation of the resulting systems is simpler and closer to that of purelyblock-based ones. Remarkably, the findings of our experiment in Chapter 5support this design decision as users appear to prefer a more uniform block-based system over one that merges two different design approaches with thegoal to combine their strengths.137Chapter 7ConclusionEnd-users already make up a large fraction of the software developmentecosystem, and their number is likely to grow further. At the same time,only a fraction of programming tools target end-users, and there exist fewestablished standards, or well-understood design principles, to support toolbuilders who create environments for end-user programmers. As a conse-quence, end-users often must rely on alternatives such as educational orprofessional tools that are not created with them in mind. This situationputs a burden both on end-users who lack support, and on tool designerswho are expected to target a diverse range of potential users.Adapting professional or educational tools for end-users is often challeng-ing. Tool creators may underestimate this challenge due to the misconcep-tion that tools that are easy to use are also easy to design. State-of-the-arttools for professional developers are often complex because they are feature-rich, so it might seem that removing some of those features is all that isneeded to make them easier to learn and use. Furthermore, tools and lan-guages that are meant for students can appear like a blueprint for how tomake programming more accessible for other audiences as well. However,both approaches commonly lead to systems that are inconsistent in theirdesigns and do not allow end-user programmers to reach their full potential.Without rigorous evaluation and feedback from end-users, it is not pos-sible to make informed decisions regarding the trade-offs that arise whendesigning systems, but this is especially true in end-user-centric systems.Almost all designers of programming environments are experts in softwaredevelopment themselves, which makes it hard for them to intuitively judgethe usability of their own tools. The end-user tools that see wide-spreadadoption are usually those that have been tested thoroughly to truly fit theneeds of their target audience. Unfortunately, successful tools are often pro-prietary and backed by strong commercial interests. Naturally, commercialvendors have little interest in sharing the insights gathered from their stud-ies with potential competitors, or informing them of decisions made duringdevelopment.We believe that independent research into end-user-centric systems plays138an important role in making new designs and techniques available to a largernumber of tool developers and users. In this work, we have explored howblock-based programming, which is already the basis for a large numberof systems, can be enriched with visual features. As stated in our thesisstatement, we set out to expand the range of programs that end-users canwrite using blocks, supporting both larger and more complex tasks whilemaintaining ease of use. We focused on three areas that pose challenges forend-users: coordinating parallel programs, decomposing large programs, andembedding nested expressions into imperative programs. For each area, wedeveloped design candidates, analyzed them, and evaluated them empiricallythrough studies with end-user participants. Our findings show that thisapproach can benefit end-users and make them more likely to overcome theprogramming challenges they encounter, pushing the boundaries of whatend-users are capable of.In Chapter 3, we present an environment targeting two robot-arms thatconduct work in tandem. Coordinating programs to perform tasks in par-allel is challenging, even for professional software developers. For this rea-son, teaching environments often establish other fundamental concepts first,such as mutable state or the control-flow in multi-threaded programs. Inour comparative study on different language designs, we have found that itis possible to introduce end-users to coordination in a way that does notrequire these fundamental prerequisites. We observed that end-users canunderstand the flow of two parallel programs as long as the code visuallymatches the physical layout of the robot arm set-up. We further foundthat as long as we direct the programmers’ attention to points where thearms interact, they can also understand the behaviour of asynchronous se-quences of commands in-between. These empirical insights, combined witha language design based on analytical insights, allowed us to build a coordi-nated programming system that end-users were more effective at using thancomparable commercial tools.In Chapter 4, we introduce an environment that guides end-users in writ-ing larger programs. Work that focuses on beginner programmers frequentlyignores the challenges of growing code bases altogether. Even most end-usertools are evaluated on small, toy-sized programs only. At the same time,tools for professionals assume that developers are already familiar with ab-straction and decomposition techniques. This leaves a gap that is difficultto bridge, especially for end-users who primarily care about a problem athand rather than general-purpose software design patterns. In our work, wecarefully analyzed the domain that our environment addresses, and designeda system that provides end-users with a way to structure and navigate their139code. In the controlled experiment we performed, we gained insights intohow and why end-users fail to decompose their code, and our system allowsthem to overcome the challenges they face. As a result, many of them wereable to write substantially larger programs than in a traditional system.In Chapter 5, we extend the environment from Chapter 4 with supportfor environmental triggers. These triggers introduce a new style of program-ming to the system, as they are defined as logical expressions that may haveseveral nesting levels. Existing block-based tools are ill-equipped for thisstyle of programming. Data-flow programming on the other hand is specifi-cally designed for this type of program, but a poor fit for imperative robotcommand sequences. In our work, we created an experimental new envi-ronment that combines block-based and data-flow programming in a singlesystem. On paper, this hybrid system allows users to use style of editingthe fits best to the task at hand, but it also has drawbacks due to its highercomplexity and inconsistent program editing experience. In a controlledstudy, we compared the hybrid system with a purely block-based one andfound that it was outperformed by the block-based tool. We believe thatthis result validates previous findings on the benefits of block-based pro-gramming. Furthermore, it illustrates the importance of evaluating systemdesign prototypes empirically to determine the impact of design trade-offs.We hope that our contributions support further research on the use ofblock-based designs in end-user programming. We have outlined whetherand how it is possible to adapt the design approaches we chose for our workto other domains and programming challenges that end-users face. Many ofour designs are only prototypes, and our experimental findings need furtherwork to investigate how they compare to experiences of using such systemsoutside of controlled study environments. In particular, we expect that fieldstudies could lead to new insights when they study end-users that work in aself-driven fashion on real tasks that are directly relevant to them. However,we hope that this work contributes towards supporting end-users in creatingincreasingly complex programs and accomplishing their real-world goals.140Bibliography[1] Thomas RG Green. Cognitive dimensions of notations. People andcomputers V, pages 443–460, 1989. → pages x, 37, 38, 128[2] Thomas R. G. Green and Marian Petre. Usability analysis of visualprogramming environments: a ‘cognitive dimensions’ framework.Journal of Visual Languages & Computing, 7(2):131–174, 1996. →pages x, 7, 37, 38, 95, 135[3] Iva´n Ruiz-Rube, Jose´ Miguel Mota, Tatiana Person, Jose´Mar´ıa Rodr´ıguez Corral, and Juan Manuel Dodero. Block-baseddevelopment of mobile learning experiences for the internet of things.Sensors, 19(24):5467, 2019. → pages xi, 3, 4[4] John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, andEvelyn Eastmond. The scratch programming language andenvironment. Transactions on Computing Education (TOCE), 10(4):1–15, 2010. → pages xi, 2, 3, 7, 13, 20, 22, 32, 74, 104[5] ABB Ltd. Robotstudio suite. URL: https://new. abb.com/products/robotics/robotstudio, 2023. → pages xii, 34, 45[6] PolyScope manual. https://s3-eu-west-1.amazonaws.com/ur-support-site/44018/Software Manual en Global.pdf, 2013. Accessed:2023-03-01. → pages xii, 35, 36[7] David Weintrop, David C Shepherd, Patrick Francis, and DianaFranklin. Blockly goes to work: Block-based programming forindustrial robots. In Blocks and Beyond Workshop (B&B), pages29–36, 2017. → pages xii, 3, 5, 6, 13, 33, 36, 37, 45, 69, 80, 83, 113[8] US Department of Labor. Occupational outlook handbook. URL:https://www.bls.gov/ooh/, 2021. → page 1[9] Christopher Scaffidi, Mary Shaw, and Brad Myers. The ‘55mend-user programmers’ estimate revisited. Institute for SoftwareResearch, International, Carnegie Mellon University, 2005. → page 1141[10] Amy J Ko, Robin Abraham, Laura Beckwith, Alan Blackwell,Margaret Burnett, Martin Erwig, Chris Scaffidi, Joseph Lawrance,Henry Lieberman, Brad Myers, et al. The state of the art in end-usersoftware engineering. ACM Computing Surveys (CSUR), 43(3):1–44,2011. → page 1[11] Leon E Winslow. Programming pedagogy—a psychological overview.ACM Sigcse Bulletin, 28(3):17–22, 1996. → page 1[12] Brian Dorn and Mark Guzdial. Learning on the job: characterizingthe programming knowledge and learning strategies of web designers.In Proceedings of the Conference on Human Factors in ComputingSystems (CHI), pages 703–712. ACM, 2010. → pages 1, 5[13] Mary Shaw. Myths and mythconceptions: What does it mean to bea programming language, anyhow? Proceedings of the ACM onProgramming Languages, 4(HOPL):1–44, 2022. → page 1[14] Susan Wiedenbeck, Patti L Zila, and Daniel S McConnell. End-usertraining: an empirical study comparing on-line practice methods. InProceedings of the Conference on Human factors in computingsystems (CHI), pages 74–81, 1995. → page 1[15] Brian James Dorn. A case-based approach for supporting theinformal computing education of end-user programmers. PhD thesis,Georgia Institute of Technology, 2010. → page 1[16] Barbara Rita Barricelli, Fabio Cassano, Daniela Fogli, and AntonioPiccinno. End-user development, end-user programming andend-user software engineering: A systematic mapping study. Journalof Systems and Software, 149:101–137, 2019. → pages 2, 35[17] Wallace Feurzeig and George Lukas. Logo—a programming languagefor teaching mathematics. Educational Technology, 12(3):39–46,1972. → pages 2, 16, 20[18] Abhinandan Pal and Anubhab Ghosh. Qiwi: A beginner friendlyquantum language. In Companion Proceedings of the 2022 ACMSIGPLAN International Conference on Systems, Programming,Languages, and Applications: Software for Humanity, pages 78–80,2022. → page 2[19] Daniel Conrad Halbert. Programming by example. PhD thesis,University of California, Berkeley, 1984. → page 2142[20] Rada Mihalcea, Hugo Liu, and Henry Lieberman. Nlp (naturallanguage processing) for nlp (natural language programming). InInternational Conference on intelligent text processing andcomputational linguistics, pages 319–330. Springer, 2006. → page 2[21] Christoph Hannebauer, Marc Hesenius, and Volker Gruhn. Doessyntax highlighting help programming novices? Empirical SoftwareEngineering, 23(5):2795–2828, 2018. → page 2[22] Brad A Myers. Taxonomies of visual programming and programvisualization. Journal of Visual Languages & Computing, 1(1):97–123, 1990. → page 2[23] Margaret M Burnett. Visual programming. Wiley Encyclopedia ofElectrical and Electronics Engineering, 2001. → pages 2, 31, 32[24] Wayne Citrin, Richard Hall, and Benjamin Zorn. Programming withvisual expressions. In Proceedings of Symposium on VisualLanguages, pages 294–301. IEEE, 1995. → page 2[25] Kerry Shih-Ping Chang and Brad A Myers. A spreadsheet model forusing web service data. In 2014 IEEE Symposium on VisualLanguages and Human-Centric Computing (VL/HCC), pages169–176. IEEE, 2014. → page 2[26] Azusa Kurihara, Akira Sasaki, Ken Wakita, and Hiroshi Hosobe. Aprogramming environment for visual block-based domain-specificlanguages. Procedia Computer Science, 62:287–296, 2015. → pages2, 3[27] TIOBE The Software Quality Company. TIOBE index.https://www. tiobe. com/tiobe-index, 2023. → page 2[28] David Weintrop. Block-based programming in computer scienceeducation. Communications of the ACM, 62(8):22–25, 2019. →pages 2, 13, 32, 73[29] Neil Fraser. Ten things we’ve learned from blockly. In 2015 IEEEBlocks and Beyond Workshop (Blocks and Beyond), pages 49–50.IEEE, 2015. → pages 3, 33, 39[30] Neil CC Brown, Amjad Altadmri, and Michael Ko¨lling. Frame-basedediting: Combining the best of blocks and text programming. In1432016 International Conference on Learning and Teaching inComputing and Engineering (LaTICE), pages 47–53. IEEE, 2016. →page 3[31] Cyrus Omar, Ian Voysey, Michael Hilton, Jonathan Aldrich, andMatthew A Hammer. Hazelnut: a bidirectionally typed structureeditor calculus. ACM SIGPLAN Notices, 52(1):86–99, 2017. → page3[32] ABB Ltd. Wizard easy programming. URL: https://new.abb.com/products/robotics/application-software/wizard, 2020. → pages3, 124[33] Nayeon Bak, Byeong-Mo Chang, and Kwanghoon Choi. Smart block:A visual programming environment for smartthings. In 2018 IEEE42nd Annual Computer Software and Applications Conference(COMPSAC), volume 2, pages 32–37. IEEE, 2018. → pages3, 5, 6, 13, 33[34] Neil Fraser et al. Blockly: A visual programming editor. URL:https://code. google. com/p/blockly, 2013. → pages 3, 23, 108, 129[35] Jose´ Miguel Mota, Iva´n Ruiz-Rube, Juan Manuel Dodero, andInmaculada Arnedillo-Sa´nchez. Augmented reality mobile appdevelopment for all. Computers & Electrical Engineering, 65:250–260, 2018. → pages 3, 5, 6, 73, 98[36] David Weintrop, Afsoon Afzal, Jean Salac, Patrick Francis, BoyangLi, David C Shepherd, and Diana Franklin. Evaluating coblox: Acomparative study of robotics programming environments for adultnovices. In Proceedings of the Conference on Human Factors inComputing Systems (CHI), pages 1–12, 2018. → pages3, 5, 6, 36, 37, 56, 69, 73, 81, 98, 122[37] Gregorio Robles, Jesu´s Moreno-Leo´n, Efthimia Aivaloglou, andFelienne Hermans. Software clones in scratch projects: On thepresence of copy-and-paste in computational thinking learning. In2017 IEEE 11th International Workshop on Software Clones(IWSC), pages 1–7. IEEE, 2017. → pages 3, 7, 74[38] Felienne Hermans, Kathryn T Stolee, and David Hoepelman. Smellsin block-based programming languages. In 2016 IEEE Symposium on144Visual Languages and Human-Centric Computing (VL/HCC), pages68–72. IEEE, 2016. → pages 3, 7, 74[39] Mazyar Seraj, Serge Autexier, and Jan Janssen. Beesm, ablock-based educational programming tool for end users. InProceedings of the 10th Nordic Conference on Human-ComputerInteraction, pages 886–891, 2018. → pages 5, 6, 35[40] Roland Siegwart, Illah Reza Nourbakhsh, and Davide Scaramuzza.Introduction to autonomous mobile robots. MIT press, 2011. →pages 5, 71[41] Bohkyung Chun and Heather Knight. The robot makers: Anethnography of anthropomorphism at a robotics company. ACMTransactions on Human-Robot Interaction (THRI), 9(3):1–36, 2020.→ pages 5, 11, 101[42] James P Diprose, Bruce A MacDonald, and John G Hosking. Ruru:A spatial and interactive visual programming language for novicerobot programming. In 2011 IEEE Symposium on Visual Languagesand Human-Centric Computing (VL/HCC), pages 25–32. IEEE,2011. → page 5[43] Shih-Chung Kang, Wei-Tze Chang, Kai-Yuan Gu, and Hung-LinChi. Robot Development Using Microsoft Robotics Developer Studio.Chapman and Hall/CRC, 2019. → page 5[44] Beate Jost, Markus Ketterl, Reinhard Budde, and ThorstenLeimbach. Graphical programming environments for educationalrobots: Open roberta-yet another one? In 2014 IEEE InternationalSymposium on Multimedia, pages 381–386. IEEE, 2014. → pages5, 35, 74[45] Ignas Plauska, Romas Lukas, and R Damasevicius. Reflections onusing robots and visual programming environments for project-basedteaching. Elektronika ir Elektrotechnika, 20(1):71–74, 2014. → page 5[46] Martin Erwig, Karl Smeltzer, and Xiangyu Wang. What is a visuallanguage? Journal of Visual Languages & Computing, 38:9–17, 2017.→ pages 6, 28, 32[47] Nico Ritschel, Anand Ashok Sawant, David Weintrop, Reid Holmes,Alberto Bacchelli, Ronald Garcia, Chandrika KR, Avijit Mandal,145Patrick Francis, and David C Shepherd. Training industrial end-userprogrammers with interactive tutorials. Software: Practice andExperience, 2022. → pages 6, 36, 37[48] David Wolber, Hal Abelson, Ellen Spertus, and Liz Looney. AppInventor. ” O’Reilly Media, Inc.”, 2011. → pages 7, 25, 74[49] Brian Harvey, Daniel D Garcia, Tiffany Barnes, Nathaniel Titterton,Daniel Armendariz, Luke Segars, Eugene Lemon, Sean Morris, andJosh Paley. Snap!(build your own blocks). In Proceeding of theTechnical Symposium on Computer Science Education (SIGCSE),pages 759–759, 2013. → pages 7, 23, 74, 79[50] Margaret M Burnett, Marla J Baker, Carisa Bohus, Paul Carlson,Sherry Yang, and Pieter Van Zee. Scaling up visual programminglanguages. Computer, 28(3):45–54, 1995. → page 7[51] Florent Devin, Pierre Boulet, J-L Dekeyser, and Philippe Marquet.Gaspard: a visual parallel programming environment. InProceedings. International Conference on Parallel Computing inElectrical Engineering, pages 145–150. IEEE, 2002. → page 7[52] Jo¨rg Poswig, Guido Vrankar, and Claudio Morara. Visavis: ahigher-order functional visual programming language. Journal ofVisual Languages & Computing, 5(1):83–111, 1994. → page 7[53] SD Peacock. The use of simulink for process modelling in the sugarindustry. In Proc. S. Afr. Sug. Technol. Ass, volume 76, pages444–455, 2002. → page 7[54] Aleksander Malinowski and Hao Yu. Comparison of embeddedsystem design for industrial applications. Transactions on industrialinformatics, 7(2):244–254, 2011. → page 7[55] David Weintrop and Uri Wilensky. Comparing block-based andtext-based programming in high school computer science classrooms.ACM Transactions on Computing Education (TOCE), 18(1):1–25,2017. → page 13[56] Jeffrey G Bonar and Blaise W Liffick. A visual programminglanguage for novices. Technical report, PITTSBURGH UNIV PALEARNING RESEARCH AND DEVELOPMENT CENTER, 1987.→ page 14146[57] Sten Mino¨r. Interacting with structure-oriented editors. InternationalJournal of Man-Machine Studies, 37(4):399–418, 1992. → page 16[58] Andrew Begel. Logoblocks, 1996. → page 16[59] Seymour Papert. Constructionism vs. instructionism. In Proceedingsfrom Japanese Educators Conference, 1980. → page 16[60] Idit Ed Harel and Seymour Ed Papert. Constructionism. AblexPublishing, 1991. → pages 16, 20[61] Caitlin Kelleher, Randy Pausch, and Sara Kiesler. Storytelling alicemotivates middle school girls to learn computer programming. InProceedings of the conference on Human factors in computingsystems (CHI), pages 1455–1464, 2007. → pages 17, 42[62] Matthew John Conway. Alice: Easy-to-learn three-dimensionalscripting for novices. PhD thesis, University of Virginia, 1998. →pages 17, 79, 104[63] Caitlin Kelleher and Randy Pausch. Using storytelling to motivateprogramming. Communications of the ACM, 50(7):58–64, 2007. →page 17[64] Quinn Burke and Yasmin B Kafai. The writers’ workshop for youthprogrammers: digital storytelling with scratch in middle schoolclassrooms. In Proceedings of the 43rd ACM technical symposium onComputer Science Education, pages 433–438, 2012. → page 17[65] Hatice Yildiz Durak. Digital story design activities used for teachingprogramming effect on learning of programming concepts,programming self-efficacy, and participation and analysis of studentexperiences. Journal of Computer Assisted Learning, 34(6):740–752,2018. → page 17[66] Wanda Dann, Dennis Cosgrove, Don Slater, Dave Culyba, and SteveCooper. Mediated transfer: Alice 3 to java. In Proceedings of the43rd ACM technical symposium on Computer Science Education,pages 141–146, 2012. → page 18[67] William F Opdyke. Refactoring: An aid in designing applicationframeworks and evolving object-oriented systems. In Proc.SOOPPA’90: Symposium on Object-Oriented ProgrammingEmphasizing Practical Applications, 1990. → page 22147[68] Michael Ball, Lauren Mock, Jonathan McKinsey, Zachary Machardy,Daniel Garcia, Nathaniel Titterton, and Brian Harvey. Oh, snap!enabling and encouraging success in cs1. In Proceedings of the 46thACM Technical Symposium on Computer Science Education, pages691–691, 2015. → page 23[69] Sorin Lerner, Stephen R Foster, and William G Griswold.Polymorphic blocks: Formalism-inspired ui for structured connectors.In Proceedings of the 33rd Annual ACM Conference on HumanFactors in Computing Systems, pages 3063–3072, 2015. → page 24[70] Michael Pradel and Koushik Sen. The good, the bad, and the ugly:An empirical study of implicit type conversions in javascript. In 29thEuropean Conference on Object-Oriented Programming (ECOOP2015). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2015. →pages 24, 39[71] Jeffrey Schiller, Franklyn Turbak, Hal Abelson, Jose´ Dominguez,Andrew McKinney, Johanna Okerlund, and Mark Friedman. Liveprogramming of mobile apps in app inventor. In Proceedings of the2nd Workshop on Programming for Mobile & Touch, pages 1–8,2014. → pages 25, 26[72] Robin Abraham, Margaret Burnett, and Martin Erwig. Spreadsheetprogramming. Wiley Encyclopedia of Computer Science andEngineering, pages 2804–2810, 2007. → page 28[73] Ivan E Sutherland. Sketchpad a man-machine graphicalcommunication system. Simulation, 2(5):R–3, 1964. → page 28[74] Daniel D Hils. Visual languages and computing survey: Data flowvisual programming languages. Journal of Visual Languages &Computing, 3(1):69–101, 1992. → pages 28, 29, 43[75] Marat Boshernitsan and Michael Sean Downes. Visual programminglanguages: A survey. Computer Science Division, University ofCalifornia, 2004. → page 28[76] G Michael Vose. Labview: Laboratory virtual instrument engineeringworkbench. Byte, 11(9):84–92, 1986. → page 28[77] Kim Marriott, Bernd Meyer, and Kent B Wittenburg. A survey ofvisual language specification and recognition. In Visual languagetheory, pages 5–85. Springer, 1998. → page 29148[78] David Harel. Statecharts: A visual formalism for complex systems.Science of computer programming, 8(3):231–274, 1987. → page 31[79] Grady Booch. The unified modeling language user guide. PearsonEducation India, 2005. → page 31[80] Chun-Yen Tsai. Improving students’ understanding of basicprogramming concepts through visual programming language: Therole of self-efficacy. Computers in Human Behavior, 95:224–232,2019. → page 32[81] Andreas Stahlbauer, Christoph Fra¨drich, and Gordon Fraser.Verified from scratch: Program analysis for learners’ programs. In2020 35th IEEE/ACM International Conference on AutomatedSoftware Engineering (ASE), pages 150–162. IEEE, 2020. → page 32[82] Brian Hempel, Justin Lubin, and Ravi Chugh. Sketch-n-sketch:Output-directed programming for svg. In Proceedings of the 32ndAnnual ACM Symposium on User Interface Software andTechnology, pages 281–292, 2019. → page 32[83] Neil CC Brown, Jens Mo¨nig, Anthony Bau, and David Weintrop.Panel: Future directions of block-based programming. In Proceedingsof the 47th ACM technical symposium on computing scienceeducation, pages 315–316, 2016. → page 33[84] Mauricio Verano Merino and Koen van Wijk. Workbench forcreating block-based environments. In Proceedings of the 15th ACMSIGPLAN International Conference on Software LanguageEngineering, pages 61–73, 2022. → pages 33, 129[85] Ricardo Tellez. The ros development studio by the construct. URL:https://www. theconstructsim.com/the-ros-development-studio-by-the-construct/, 2017. → page 34[86] ABB Robotics. Technical reference manual: Rapid instructions,functions and data types. ABB Robotics, 2014. → pages 34, 35[87] Gopika Ajaykumar, Maureen Steele, and Chien-Ming Huang. Asurvey on end-user robot programming. ACM Computing Surveys(CSUR), 54(8):1–36, 2021. → page 35149[88] Geoffrey Biggs and Bruce MacDonald. A survey of robotprogramming systems. In Proceedings of the Australasian conferenceon robotics and automation, pages 1–3, 2003. → page 35[89] Zengxi Pan, Joseph Polden, Nathan Larkin, Stephen Van Duin, andJohn Norrish. Recent progress on programming methods forindustrial robots. In ISR 2010 (41st International Symposium onRobotics) and ROBOTIK 2010 (6th German Conference onRobotics), pages 1–8. VDE, 2010. → page 35[90] Brenna D Argall, Sonia Chernova, Manuela Veloso, and BrettBrowning. A survey of robot learning from demonstration. Roboticsand autonomous systems, 57(5):469–483, 2009. → pages 35, 45[91] Guilherme Boulhosa Rodamilans, Emı´lia Villani, Lu´ıs GonzagaTrabasso, Wesley Rodrigues de Oliveira, and Ricardo Suterio. Acomparison of industrial robots interface: force guidance system andteach pendant operation. Industrial Robot: An InternationalJournal, 43(5):552–562, 2016. → page 35[92] Joa˜o Neves, Diogo Serrario, and J Norberto Pires. Application ofmixed reality in robot manipulator programming. Industrial Robot:An International Journal, 2018. → page 35[93] Enes Yigitbas, Ivan Jovanovikj, and Gregor Engels. Simplifying robotprogramming using augmented reality and end-user development. InHuman-Computer Interaction–INTERACT 2021: 18th IFIP TC 13International Conference, Bari, Italy, August 30–September 3, 2021,Proceedings, Part I 18, pages 631–651. Springer, 2021.[94] Andrzej Burghardt, Dariusz Szybicki, Piotr Gierlak, Krzysztof Kurc,Paulina Pietrus´, and Rafa l Cygan. Programming of industrial robotsusing virtual reality and digital twins. Applied Sciences, 10(2):486,2020. → page 35[95] Epson rc+ express manual. https://files.support.epson.com/far/docs/epson rc express 6-axis rev.5 en.pdf,2022. Accessed: 2023-03-01. → page 35[96] Mohammad Hadhrawi, Alan F Blackwell, and Luke Church. Asystematic literature review of cognitive dimensions. In PPIG,page 3, 2017. → page 39150[97] Franklyn Turbak, David Wolber, and Paul Medlock-Walton. Thedesign of naming features in app inventor 2. In 2014 IEEESymposium on Visual Languages and Human-Centric Computing(VL/HCC), pages 129–132. IEEE, 2014. → page 39[98] Robert Holwerda and Felienne Hermans. A usability analysis ofblocks-based programming editors using cognitive dimensions. InProceedings of the Symposium on visual languages and human-centriccomputing (VL/HCC), pages 217–225. IEEE, 2018. → pages 39, 96[99] Soenke Kock, Timothy Vittor, Bjo¨rn Matthias, Henrik Jerregard,Mats Ka¨llman, Ivan Lundberg, Roger Mellander, and MikaelHedelind. Robot concept for scalable, flexible assembly automation:A technology study on a harmless dual-armed robot. In Proceedingsof the 2011 International Symposium on Assembly andManufacturing (ISAM), pages 1–5. IEEE, 2011. → pages 40, 131[100] Sean Massung and Cinda Heeren. Visualizing parallelism in cs 2. In2013 IEEE International Symposium on Parallel & DistributedProcessing, Workshops and Phd Forum, pages 1252–1256. IEEE,2013. → page 40[101] Nico Ritschel, Vladimir Kovalenko, Reid Holmes, Ronald Garcia, andDavid C Shepherd. Comparing block-based programming models fortwo-armed robots. Transactions on Software Engineering (TSE),2020. → pages 41, 80, 110, 113[102] Annette Feng, Eli Tilevich, and Wu-chun Feng. Block-basedprogramming abstractions for explicit parallel computing. InProceedings of the 2015 Blocks and Beyond Workshop, pages 71–75.IEEE, 2015. → page 42[103] Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified dataprocessing on large clusters. Communications of the ACM, 51(1):107–113, 2008. → page 43[104] Steven L Tanimoto. Viva: A visual language for image processing.Journal of Visual Languages & Computing, 1(2):127–139, 1990. →page 43[105] James C Browne, Syed I Hyder, Jack Dongarra, Keith Moore, andPeter Newton. Visual programming and debugging for parallel151computing. Parallel & Distributed Technology: Systems &Applications, 3(1):75–83, 1995. → page 43[106] Daniele Benedetelli. Lego Mindstorms EV3 Laboratory: Build,Program, and Experiment with Five Wicked Cool Robots. No StarchPress, 2013. → page 44[107] Susan Wiedenbeck. The use of icons and labels in an end userapplication program: an empirical study of learning and retention.Behaviour & Information Technology, 18(2):68–82, 1999. → page 52[108] John Brooke et al. SUS - a quick and dirty usability scale. Usabilityevaluation in industry, 189(194):4–7, 1996. → pages 57, 64[109] James R Lewis and Jeff Sauro. The factor structure of the systemusability scale. In International conference on human centereddesign, pages 94–103. Springer, 2009. → pages 57, 59[110] ABB Ltd. RobotStudio Online YuMi. URL: https://apps.microsoft.com/store/ detail/9NBLGGH2SQFM, 2015. → page 65[111] Shuchi Grover and Satabdi Basu. Measuring student learning inintroductory block-based programming: Examining misconceptionsof loops, variables, and boolean logic. In Proceedings of the 2017ACM SIGCSE technical symposium on computer science education,pages 267–272, 2017. → page 70[112] George Michalos, Panagiotis Karagiannis, Nikos Dimitropoulos,Dionisis Andronas, and Sotiris Makris. Human robot collaboration inindustrial environments. The 21st century industrial robot: whentools become collaborators, pages 17–39, 2022. → page 71[113] Kashif Amanullah and Tim Bell. Evaluating the use of remixing inscratch projects based on repertoire, lines of code (loc), andelementary patterns. In Proceedings of the Frontiers in EducationConference (FIE), pages 1–8, 2019. → page 72[114] Nico Ritschel, Felipe Fronchetti, Reid Holmes, Ronald Garcia, andDavid C Shepherd. Can guided decomposition help end-users writelarger block-based programs? a mobile robot experiment.Proceedings of the ACM on Programming Languages, 6(OOPSLA2):233–258, 2022. → pages 72, 108, 113, 122152[115] Mateus Carvalho Gonc¸alves, Ota´vio Neves Lara, Raphael Wincklerde Bettio, and Andre´ Pimenta Freire. End-user development of smarthome rules using block-based programming: a comparative usabilityevaluation with programmers and non-programmers. Behaviour &Information Technology, pages 1–23, 2021. → pages 73, 81, 98[116] Martin Fowler. Refactoring: Improving the Design of Existing Code.Addison-Wesley, 1999. → pages 74, 95[117] Maria Kallia and Sue Sentance. Computing teachers’ perspectives onthreshold concepts: Functions and procedural abstraction. InProceedings of the 12th workshop on primary and secondarycomputing education, pages 15–24, 2017. → pages 76, 77[118] Cazembe Kennedy and Eileen T Kraemer. What are they thinking?eliciting student reasoning about troublesome concepts inintroductory computer science. In Proceedings of the Koli CallingInternational Conference on Computing Education Research, pages1–10, 2018. → pages 76, 79[119] Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Ja¨rvinen. Astudy of the difficulties of novice programmers. ACM SIGCSEBulletin, 37(3):14–18, 2005. → page 76[120] Chiu-Liang Chen, Shun-Yin Cheng, and Janet Mei-Chuen Lin. Astudy of misconceptions and missing conceptions of novice Javaprogrammers. In Proceedings of the International Conference onFrontiers in Education: Computer Science and ComputerEngineering (FECS), page 1, 2012. → page 77[121] Simon Rose, MP Jacob Habgood, and Tim Jay. An exploration ofthe role of visual programming tools in the development of youngchildren’s computational thinking. Electronic journal of e-learning,15(4):pp297–309, 2017. → page 77[122] Jeff Kramer. Is abstraction the key to computing? Communicationsof the ACM, 50(4):36–42, 2007. → page 77[123] Orit Hazzan. Reflections on teaching abstraction and other softideas. ACM SIGCSE Bulletin, 40(2):40–43, 2008. → page 77[124] Andrew Bragdon, Robert Zeleznik, Steven P Reiss, SumanKarumuri, William Cheung, Joshua Kaplan, Christopher Coleman,153Ferdi Adeputra, and Joseph J LaViola Jr. Code Bubbles: A workingset-based interface for code understanding and maintenance. InProceedings of the Conference on Human Factors in ComputingSystems (CHI), pages 2503–2512, 2010. → page 81[125] Niklas Elmqvist and Jean-Daniel Fekete. Hierarchical aggregation forinformation visualization: Overview, techniques, and designguidelines. Transactions on Visualization and Computer Graphics(TVCG), 16(3):439–454, 2009. → page 81[126] Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis.Running experiments on amazon mechanical turk. Judgment andDecision making, 5(5):411–419, 2010. → page 93[127] Fei Chen, Kosuke Sekiyama, Ferdinando Cannella, and ToshioFukuda. Optimal subtask allocation for human and robotcollaboration within hybrid assembly system. Transactions onAutomation Science and Engineering (T-ASE), 11(4):1065–1075,2013. → page 101[128] Steven Ovadia. Automate the internet with “if this then that”(IFTTT). Behavioral & social sciences librarian, 33(4):208–211,2014. → page 101[129] Wesley M Johnston, JR Paul Hanna, and Richard J Millar.Advances in dataflow programming languages. ACM computingsurveys (CSUR), 36(1):1–34, 2004. → page 103[130] JGraph Ltd. mxgraph. URL: https://jgraph.github.io/mxgraph/,2017. → page 108[131] Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and Michael LLittman. Practical trigger-action programming in the smart home.In Proceedings of the Conference on Human Factors in ComputingSystems (CHI), pages 803–812, 2014. → page 110[132] Blase Ur, Melwyn Pak Yong Ho, Stephen Brawner, Jiyun Lee, SarahMennicken, Noah Picard, Diane Schulze, and Michael L Littman.Trigger-action programming in the wild: An analysis of 200,000 iftttrecipes. In Proceedings of the Conference on Human Factors inComputing Systems (CHI), pages 3227–3231, 2016. → page 110154[133] Stefan Palan and Christian Schitter. Prolific. ac—a subject pool foronline experiments. Journal of Behavioral and ExperimentalFinance, 17:22–27, 2018. → pages 111, 112[134] Jacob Cohen. Statistical power analysis. Current directions inpsychological science, 1(3):98–101, 1992. → page 111[135] Michael Chmielewski and Sarah C Kucker. An mturk crisis? shifts indata quality and the impact on study results. Social Psychologicaland Personality Science, 11(4):464–473, 2020. → page 112[136] Manish Kumar Goel, Pardeep Khanna, and Jugal Kishore.Understanding survival analysis: Kaplan-meier estimate.International journal of Ayurveda research, 1(4):274, 2010. → page118[137] Stefan Wellek. A log-rank test for equivalence of two survivorfunctions. Biometrics, pages 877–881, 1993. → page 118[138] Aaron Bangor, Philip T Kortum, and James T Miller. An empiricalevaluation of the system usability scale. International Journal ofHuman–Computer Interaction, 24(6):574–594, 2008. → page 120[139] Joost CF De Winter and Dimitra Dodou. Five-point likert items: ttest versus mann-whitney-wilcoxon. Practical Assessment, Research& Evaluation, 15(11):1–12, 2010. → page 120[140] Brenden Sewell. Blueprints visual scripting for unreal engine. PacktPublishing Ltd, 2015. → page 124[141] Lucas Bertolini. Hands-On Game Development without Coding:Create 2D and 3D games with Visual Scripting in Unity. PacktPublishing Ltd, 2018. → page 124[142] Milica Lekic´ and Gordana Gardasˇevic´. Iot sensor integration tonode-red platform. In Proceedings of the International SymposiumInfoteh-Jahorina (Infoteh), pages 1–5, 2018. → page 124[143] Lance A Miller. Programming by non-programmers. InternationalJournal of Man-Machine Studies, 1974. → page 126[144] Elliot Soloway and Kate Ehrlich. Empirical studies of programmingknowledge. IEEE Transactions on software engineering, 10(5):595–609, 1984. → page 126155[145] Andreas Stefik and Stefan Hanenberg. Methodological irregularitiesin programming-language research. Computer, 50(8):60–63, 2017. →page 126[146] Michael Coblenz, Gauri Kambhatla, Paulette Koronkevich, Jenna LWise, Celeste Barnaby, Joshua Sunshine, Jonathan Aldrich, andBrad A Myers. Pliers: a process that integrates user-centeredmethods into programming language design. ACM Transactions onComputer-Human Interaction (TOCHI), 28(4):1–53, 2021. → page126[147] Shareen Mahmud, Jessalyn Alvina, Parmit K Chilana, Andrea Bunt,and Joanna McGrenere. Learning through exploration: how children,adults, and older adults interact with a new feature-rich application.In Proceedings of the 2020 CHI Conference on Human Factors inComputing Systems, pages 1–14, 2020. → page 127[148] Philip J Guo. Older adults learning computer programming:motivations, frustrations, and design opportunities. In Proceedings ofthe Conference on Human Factors in Computing Systems (CHI),pages 7070–7083. ACM, 2017. → page 128[149] Stephanie Ludi and Mary Spencer. Design considerations to increaseblock-based language accessibility for blind programmers via blockly.J. Vis. Lang. Sentient Syst., 3(1):119–124, 2017. → pages 130, 131[150] Varsha Koushik, Darren Guinness, and Shaun K Kane. Storyblocks:A tangible programming game to create accessible audio stories. InProceedings of the 2019 CHI Conference on Human Factors inComputing Systems, pages 1–12, 2019. → page 130[151] Tamara Munzner. Visualization analysis and design. CRC press,2014. → page 135156
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-ShareAlike 4.0 International
Graduate
Holmes, Reid
Garcia, Ronald (Professor of computer science)
Enriching block-based end-user programming with visual features
Text