Software Engineering with an Agile Development Framework/Whole process
Principles
editPrototyping: Build something in order to answer specific question, learn from it and move on.
Client satisfaction
Client involvement
Documentation (and evidence portfolio)
How to read book
editSectors {{Adfmetabox |title |Bite |2 hours |Inputs |Evidence |Stakeholders }}
| title |
|
|
5 work streams
editThe rainbow sector diagrams in the book are intended as templates for steering your own project. In the first two iterations we have linked the boxes, in the third iteration the sectors are blank.
The trick is to get from the inputs on the left to the outputs on the right.
1. Draw the activities you need to get from the inputs to the outputs. If something is not driving towards the output, then ask yourself, why are we doing this?
2. Pay attention to all five workstreams.
- Evaluate
- Steer
Examples
editCapstone Project
editThe capstone project is an important component of most Information Technology degrees, the Bachelor of Information Technology (B.InfoTech) at Otago Polytechnic is no exception. A complete list of projects is online at www.otagopolytechnic.ac.nz
Fincher et al. (2001) describe “keys for success” for projects. While they intend these measures to be used in establishing programmes, rather than specific projects, they do present a useful yardstick for successful projects. The Otago B.InfoTech is perhaps unique in its strengths in complete systems, including both hardware and software.
A significant challenge in the design of capstone courses is relationship between process and product. As academics we argue that a strong process will result in a good product but instructors face little direction in the identification of a suitable process. A major issue is that of systematically incorporating prototypes into the software design and development lifecycle. In this paper we have identified a development framework which successfully does that integration. This framework was developed from exemplar projects and provides a best system framework. The framework is also used to explore the development process of weaker projects.
Role of development methodology
editThis chapter aims to explore the role of development methodology in successful capstone projects, in particular the extent, nature and integration of prototyping.
In computing development projects the choice of development methodology is as perennial a subject for debate as is the choice of programming language. A methodology has to be selected for any given project and the selection may be based on a myriad of factors. Many educational institutions contain a capstone project whereby students, often in groups, work to complete a significant project. The choice of development process becomes pivotal in a capstone project (Beasley 2003; Chamillard and Goold 2003;. Bridgeman 2003, Clear 2003). In computing education, the choice is complicated by the introduction of the teaching imperative. The issue we face is finding not only the best approach for development but the best approach for teaching it.
Clear et al. (2001) described what they call the “process versus product tension” (p95) in the design of capstone courses. They refer to the decision as to whether the focus of the capstone course should be on the development process, or on the product of that development: a working system. They argue that as the capstone course emulates work as a professional “it is advisable that an acceptable methodology or process be adopted and followed. What specific methodology or adaptation thereof is chosen is less important than the use of some formal process”. Chamillard and Braun (2002) describe the problem thus:
| “ | “Given the importance of process in real software development activities, we want to ensure that our students get appropriate exposure to process issues. On the other hand, we also do not want our students to believe that if they follow an appropriate process, it does not matter if they generate a working product! We must therefore also provide the appropriate emphasis on the product the students are developing to ensure it meets the project requirements”. (p229) | ” |
What then, is an appropriate process for capstone projects? Traditionally, most capstone courses have used a derivation of the Software Development Life Cycle (SDLC, e.g. that described by Hoffer et al. 1999). In recent years, instructional designers have mixed this with components of agile modelling and especially prototyping (Fincher et al. 2001).
Hakim and Spitzer (2000) argue that the “challenge is to systematically incorporate prototypes into the software design and development lifecycle”.
In this paper we aim to identify a framework for successful projects.
Exemplar projects
editIn the following sections we examine the qualitative evidence of the development process of exemplar projects. Groups were selected to give a range of project category. Quotes are given from the students’ reflective reviews to portray the complexity of the decisions and process.
 Case study ACKI
|
|
Group: Kevin Barclay
The ACKI development aimed to create a box that sits between computer and keyboard, for users with limited mobility (Barclay et al. 2001). This was a generic solution for the needs of unique users. The box emulated the keyboard to feed analog or digital input form any device. Several aspects of the development process stand out. A stable testbed was established meaning that a detailed testing protocol could be established with an integrated test routine. The group kept a detailed log book of process and defined test outcomes. Different levels of prototype met different needs, the testbed was used to investigate technical issues, while whiteboard pens emulated joysticks to explore level of control, what does input device need to do. This project aligns with an Agile Development Framework even though it was a hardware based project. For each component “miniature lifecycles” were used with an – analyse-design-test cycle.
|
| Case study School network
|
|
Group: Boudewyn Erkens, Lorna Lou, Rebecca Sidaway
This project aimed to meet the ICT needs of a local primary school. In addition to network design and implementation, the group worked closely with the teachers in meeting professional development needs. While the project was hardware and training, the group followed the topdown structure of the SDLC. The training needs in particular were met through training needs analysis and then lessons were designed and implemented. The network solution was prototyped using a rapid installation and borrowed equipment which the school used for a month to see if the solution met their needs.
|
5.7 Alternative CAD interface
The SDLC was used as the basis for a project that aimed to develop a drawing board interface to architectural CAD systems (Lawton and Meikle 2003). Much of the project was development and configuration of hardware and then user testing of each iteration. The team developed many different prototype systems, and although using many of the same components, they built largely from scratch each time on a rig they developed.
“Aspects of prototyping were revisited again and again ... and again. Some aspects proved to be overly time consuming, and in one case a decision had to be discarded for this very reason.”
“Paper” interfaces will be developed with the assistance of architectural draughting students. The students will be asked to evaluate each version of the interface and contribute suggestions or criticisms. This will influence the following draught of the interface, as well as the requirements of the menu file (the interface code for the CAD application).
The hardware will be assembled and installed for testing with potential users. As with the criticisms, observations and suggestions will be evaluated and incorporated into the system. This stage is repeated until the functional requirements have been met”
The iterative stages of analysis, logical design and prototyping were run concurrently. Aspects of the project involving prototyping included a drawing surface, the construction of the frame, a pointing device, the projection system and the customisation of the AutoCAD interface. Problems were encountered, as well as unforeseen benefits of the system.
5.8 SMS The development of a system for the control of several remote devices via a cellphone was the goal of this group. This system involved the development and integration of several hardware and software components. The group followed an SDLC process, developing a hardware and software prototype in analysis: “we threw together an application…after getting this test application working we came up with a few problems with our design…we now had our functional requirements nailed down, and signed off, and could continue in-depth planning…”
The group developed a working system “prototype” and in implementation replaced major components sequentially.
5.9 Edubuilder The construction industry is an area where there are multiple layers of complexity, interacting roles, differing timescales and consequences of decisions. This is a prime target for a good interactive learning experience supported by technology. This project undertook to provide a content management system in an area where there is a vast store of expert knowledge available but few good information sources that take advantage of interactive media. The group followed the SDLC and began by working through a huge pile of building standards and struggled to find an underlying theme to hold it together. They used several metaphors before coming back to the building itself and the project plan chart. They developed a diagrammatic prototype and in discussion with several “clients” it became apparent that the different timescales, contexts and user views of a Gannt chart, that this was the elusive concept. To test this premise the group developed several prototypes to prove the concept, these ranged from a single sketch to a semi-functional interactive mockup of the system. These prototypes were used to explicitly test various aspects of the premise. In order to implement this concept the group realised some complex technical procedures were needed. The group used a carefully managed testing regime that identified a need for a test, the development of code, a test procedure and recording. As requirements were well defined, a stable system was developed and a rigorous testing regime utilised, the implementation at the end of the project was remarkably rapid yet the deliverable was robust and useful. 5.10 Information system for suit hire Despite being an ideal candidate for a traditional information system, the client for this project was not entirely convinced that he needed a computerised system. To alleviate this concern the group developed an early prototype on the computer that mirrored the paper based system. The client was convinced that he would gain efficiency while not losing the basis of the paper based system. This prototype also allowed the development of strong functional requirements. The group was also able to test their understanding of the current system by “operating” the prototype in parallel with the existing system, at the clients’ premises. Once analysis was completed, the group discarded the prototype. They then went through normal logical and physical design involving several prototypes. The client was unavailable for five months during this stage but the group was able to test the prototypes according to the strong functional requirements they had developed. The resulting system was substantially different from the original prototype in operation of the business but it retained the look of the paper based system. 5.11 Familtrak A familiarisation tour (“famil”) is a trip taken by people in the travel agency to familiarise themselves with the products they are selling. They are supposed to write up an account on their return but hardly ever do. The client for this project wanted a system to facilitate the reporting and dissemination of such. This was much more than a simple database, the solution involved a webpage, XML, GIS and a high degree of interactivity (Hamilton 2003). .The initial functional requirements were worked out between the group and the client. The group then undertook several weeks of validating and improving the functional requirements. This included much observation of work patterns, a questionnaire and a paper based prototype. A prototype was first used to represent the set of functions the system would be required to carry out. In logical design, a second paper based prototype was used to develop the structure of the system. The group first tested the prototype entirely on paper with the client, and then in a very clever move scanned in the paper and linked it together, resulting in a low fidelity but high interactivity prototype. This meant that “testing was completed before any code was written”. The was released to the client for actual production use three months before due date and the group was then able to work on perfecting the system and developing some of the more difficult aspects (linking spatial and multimedia aspects). 6 Exemplar Processes From the exemplar projects some themes emerge. These we distil into an exemplar development framework.
1. “SDLC” identified methodology 2. Prototyping used 3. Strong functional requirements used 4. Functional requirements tested with low fidelity but high interactivity prototypes 5. Stable platform for development developed early 6. Prototypes part of integrated testing plan 7. Early functional deliverable to client 8. Robust final deliverable 9. Still need ‘normal’ design – prototyping doesn't replace 10. Prototypes used in communication with client 11. Maturation by revolutionary (cf evolutionary). Staged replacement for hardware.
Some of these factors are perhaps expected measures of quality; it is not surprising that they appear here: we did not look at spelling or good intra-group communication but they would be expected on such a framework also. What we have though is evidence that the exemplar groups do these things. Some factors are surprising; we did not expect to find the maturation process so similar for the exemplar projects.
7 Validation In order to further explore the importance of the development framework, we undertook an analysis of the full set of documented projects. The first author worked through the documentation with a third person who did not know the history of the projects to assign a five-point scale to each of eleven factors. These scores are compared to the grades for the projects. It should be noted that this analysis uses grade used as measure for quality and we recognise that there are other factors at play here. We also recognise the potential bias of the authors in making this assessment, although this should have been partially mitigated by the impartial third person. A simple summation of the eleven factors provides a very high correlation (r=0.756) with the grade received for each project (Figure 3: for this and subsequent statistics the exemplar projects are excluded). Further, the relationship is not affected by the type of project.
Figure 3: Process score summation is closely related to quality, this is unaffected by the project type.
Figure 4 shows all the projects assessed according to the development framework. All factors are positively correlated with the grade. Some anomalous projects can be identified: Project “78 HW A” was a hardware procurement project that did not follow the framework, whereas “174 IS A-” was a retrospective project where processes had been followed but evidence was lacking resulting in a slightly lower grade. Most of the constituent factors that make up this score could be considered general measures of quality. Other measures such as efficacy of group work, or spelling, would probably also have positive relationships. This is still useful, we would also identify these factors as critical success factors. Some of the factors are not quite so obvious; the type of implementation – revolutionary or staged replacement as opposed to evolutionary – is striking in the pattern of being carried out by better groups.
Figure 4: Projects assessed by development framework, ordered from top by grade. The darker shading indicates a 4 or 5 where this factor is used. Exemplar projects above the line.
Perhaps more important than the overall correlations of the process factors, is the relationships between them at different ends of the spectrum – what are the poorer groups doing differently? By examining Figure 4 carefully (or looking at the correlations – not shown), some interesting patterns emerge. Poorer groups did less prototyping, which is understandable; they did less of other things too. What is critical is that when they did prototyping they did not also do “normal design” nor have stable systems, early delivery etc. Poorer groups that used prototypes in early stages also had very weak functional requirements, did not have strong testing plans etc., in other words the prototype became the development in total. Groups in the middle ranges that did prototypes tended to have weaker logical and physical design, instead they saw the development as one of making the prototype robust. These patterns can also be seen in the comments of the marking panel for groups in the lower half of the grades (Table 2).
Wrote code as proof of concept prototype then tried to design it. 'one half of the team was designing and the other half coding and both at different rates'
Technical decisions made in getting prototype out haunted later development good process limited by technical skills
Functional prototype developed early but without design, that bad design stuck, Digitised a bad process as some stages of SDLC missed.
Initial concepts done with prototype, anything deemed to be too hard never got considered as a real requirement SDLC but skipped from initial idea to implementation
Couldn't achieve much technical complexity so scoped down to make prototype deliverable
After successful prototype almost no progress for the rest of the year. At last minute, they independently implement the rest of it and retrospectively did SDLC analysis
Prototype developed for proof of concept stuck SDLC very weak.. Client happy but insufficient technical complexity
Palm development. Never had stable platform. Couldn't make it work.
Table 2: Comments from marking panel for C and fail groups
8 Conclusion Hakim (2000) argued that the “challenge is to systematically incorporate prototypes into the software design and development lifecycle”. In this paper we have identified a development framework which successfully does that integration. This framework was developed from exemplar projects and provides a best system framework.
Stein (2002) also identifies properties common to successful projects. But cautions “it is true however, for each property I can find … less successful project(s) containing the same property”. By examining how the development framework compares with what poorer groups are doing, we have been able to identify how this framework might apply to weaker groups. In particular, we have demonstrated the risk of weaker groups doing prototypes instead of normal design, or conversely, sticking to a naïve view of the SDLC that results in poorly tested designs.
This paper has not considered the level of technical complexity of a project. Goold (2003) discusses the effect of both technical skills of group members and the scope of the proposed project. It would be worth examining the complexity of projects, and how this relates to both the grade and the development framework adopted. We found that the framework applies equally well across different types of project (see also Avrahani 2002).
The tension between product and process can be lessened by adopting a process that can be seen to produce good products while being flexible and robust. We believe that the development framework developed in this paper will provide a foundation for capstone courses.
9 Acknowledgements We would like to thank the years of students for providing such a wealth of information. Also the work of the other lecturers on this course. Patricia Haden, Phoebe Eden-Mann and Zuzette van Vuuren helped with analysis. This research was carried out under Otago Polytechnic Category B Ethical Approval. References
Barclay, K., Mann, S., Brook, P. and Doonan, A. (2001) Development and Testing of an Adaptive Computer Keyboard Interface Paper in the Proceedings of 14th Annual Conference of the National Advisory Committee on Computing Qualifications Napier, p13-22 Beasley, R. E. (2003). “Conducting a successful senior capstone course in computing.” The Journal of Computing in Small Colleges 19(1): 122–131. Bridgeman, N. (2003). Project success: defining, designing, constructing and presenting a capstone project. 16th Annual NACCQ, Palmerston North, NACCQ.211-216 Clear, T. (2003). “The waterfall is dead, long live the waterfall!” SIGSCE Bulletin 35(4): 13–14. Clear, T., F. H. Young, M. Goldweber, P. M. Leidig and K. Scott (2001). “Resources for instructors of capstone courses in computing.” ACM SIGCSE Bulletin 33(4): 93-113. Chamillard, A. T. and K. A. Braun (2002). The Software Engineering Capstone: Structure and Tradeoffs. Proceedings of the 33rd SIGCSE technical symposium on computer science education, Cincinnati, Kentucky.227-231 Fincher, S., M. Petre and M. Clark, Eds. (2001). Computer Science Project Work: Principles and Pragmatics. London, Springer. Garrett, P., D. Youngman, J. McCormack, C. Rosescu and S. Mann (2003). Characteristics of success - virtually there. Proceedings of the 16th Annual NACCQ.269-272 Goold, A. (2003). Providing process for projects in capstone courses. Proceedings of the 8th annual conference on Innovation and Technology in Computer Science Education, Thessalonki, Greece, SIGCSE ACM.26-29 Hakim, J. and T. Spitzer (2000). Effective prototyping for usability. ACM Special Interest Group for Design of Communications, Proceedings of IEEE professional communication society international professional communication conference and Proceedings of the 18th annual ACM international conference on Computer documentation: technology & teamwork.47-54 Hamilton, M. (2003). Familtrack. Proceedings of the 16th Annual NACCQ, NACCQ.484 Hoffer, J. A., J. F. George and J. S. Valacich (1998). Modern Systems Analysis and Design. Reading USA, Benjamin Cummings. Lawton, B. and A. Meikle (2003). Alternative CAD user interface. Proceedings of the 16th Annual NACCQ, NACCQ.487 Ponting, D., L. Quarrie and G. Robertson (2003). Serve i.t. right: hospitality training CD-rom. Proceedings of the 16th Annual NACCQ.495 Rudd, J., K. Stern and S. Isensee (1996). “Low vs High-fidelity debate.” Interactions 3(1): 76–85. Singh-Cosgrave, B., Sinclair-Fox,C., McLellan, G., Mann, S. and McGregor, G. (2001) Interactive CD for Teaching Development Based Subjects Concise Paper in the Proceedings of 14th Annual Conference of the National Advisory Committee on Computing Qualifications Napier, p460 Stein, M. V. (2002). “Using large vs. small group projects in capstone and software engineering courses.” The Journal of Computing in Small Colleges 17(4): 1–6.
4 Results
4.1 Initial findings It is worth investigating external features of the projects to indicate development characteristics of successful projects. Between 2001 and 2003, 65 projects have been completed. 35% gained As, 32% Bs, 16% C and 17% failed. The projects can classified into eight categories as shown in Table 1 along with marks distributions for each. With some small exceptions (“total systems” are all successful, ‘hardware’ has a higher than expected failure rate), there are no strong relationships between the type of project and the grade. Good grades are possible whatever the type of project.
Table 1: Classification of projects and grade distribution.
Total System 3 Hardware 15 Applied software 7 Information system 13 Multimedia 12 Content management 6 Network 3 Other 6 Total 65
Figure 1: Effect of group size on project grade.
There is an effect of group size, as shown in Figure 1. While groups of two or three show similar patterns, the students working alone have a much higher chance of failure. We believe that there are two underlying factors here: the quality of students (i.e. why are they working alone), and that they miss the benefits of working in a group.
Most students follow a document centred development approach. The students were wrong however, size does not count, the rumour about assigning grades by weight is not true. Figure 1 demonstrates a very weak relationship between the final grade and the number of pages in the documentation (n=49, complete documentation available in the library). Once above a minimum threshold of around 100 pages, any grade is possible. There are a cluster of top grades at around 260 pages. The projects with greatest amount of documentation did not get top grades and clearly poor projects cannot be rescued by a mass of documentation.
Figure 2: Weak relationship between pages and grade (n=49)
Good projects can not be identified by type of project nor by the quantity of work. So we come back to the question, what are the good groups doing?