«Abstract We conducted an industrial case study of a distributed team in the USA and the Czech Republic that used Extreme Programming. Our goal was to ...»
The XP-EF has been used to structure several XP case studies. In studies conducted with two small, co-located teams (one at IBM and one at Sabre Airline Solutions™), improvements in post-release quality, pre-release quality, and productivity were observed after the teams changed from a more traditional process to XP [35, 52]. Another study with a larger team at Sabre Airlines Solutions™ showed that the team yielded above-average post-release quality and average to above-average productivity relative to industry averages when the team utilized the XP methodology . These case study results suggest that the usage of XP can help improve the business-related results (such as quality and productivity) of teams operating within certain contexts. However, there has been little research on the use of XP in a GSD environment where the informal communication required in XP may become strained. Xiaohu, et al. discuss how XP practices can reduce communications issues in a GSD project and provide a limited discussion of an GSD enhancement project that used XP .
3. Research method Our research is an industrial case study of an entire project from initial requirements definition through the end of product development. Case studies are used to collect data through observation of a project in an unmodified, contextual setting . Case studies are powerful techniques for pursuing “how” or “why” research questions, when the investigator has little control over the events, and when the focus is on a contemporary phenomenon with some real-life context .
The research questions we sought to investigate were to determine what factors allowed GSD XP teams to create successful products. This includes understanding both the everyday mechanics of team process, as well as an evaluation of the final product in terms of quality, productivity, and customer satisfaction. This data was gathered using the XP-EF. Furthermore, we use a qualitative research approach called grounded theory  to preserve the complexity of our case study data. The intent is to generate or discover a theory that is “grounded” in data from the field. This approach is suitable for our study since we were not investigating any preconceived theories of what might make a GSD XP team successful or unsuccessful. In this research, we do not provide formal hypothesis testing or draw any general conclusions. However, we conjecture about some communication practices for GSD XP teams based upon evidence gathered from this case study.
The qualitative information was supplied primarily from two sources. The first source was an informal email prompt sent to all members of the project management team in the U.S. as well as the lead developer in the Czech Republic. The remaining developers were not contacted due to their limited availability during the case study analysis. Five out of seven recipients responded; responses were received from the project manager, tracker, development manager, development lead, and one of two proxy customers. The responses ranged from one page of
prose to three pages of detailed lists. The prompt is:
One thing that I have not heard much about is the problems with the project. So, what do you feel were the biggest hurdles, obstacles, shortcomings, or problems throughout the … project? Be as specific or as
as you like.
The second main qualitative resource was interviews. A face-to-face interview was conducted with the customer and the proxy customer after the system had been in production for three months. The interview questions may be found in the Appendix. A similar face-to-face interview was conducted with the development manager. Both interviews took approximately 45 minutes. A final, 30 minute follow-up interview was conducted over the phone with the customer approximately six months after the system had been put into production.
We also collected a variety of quantitative data. A simple script was used to count the number of source lines of code (thousand lines of executable code or KLOEC), classes, and methods via naming conventions in the code.
Measurements of effort were gathered from the XPlanner1 project tracking tool. The data on the team, process factors, and the project outcome described in Section 4 was structured using XP-EF V1.4 . We used both qualitative and quantitative data collection methods in trying to capture a more holistic and contextual portrayal of the factors under study . Convergence and agreement between the results of multiple methods enhances the belief that the results are valid and not a methodological artifact .
4. Team and project description
This section describes the Tekelec-Radiante project. From this point forward, we refer to the project as the F-15 project. The XP-EF details of the F-15 project that are not described in this section may be found in . We also use the term “team” in this paper to refer to all of the project personnel in both the USA and the Czech Republic.
We do, at times, refer to the teams individually, but a non-qualified use of the term implies the entire group. This is in accordance with the way in which the team viewed itself: not as two separate entities working at a great distance apart, but as one group striving to achieve a common goal. The customer noted that cooperation and teamwork were the biggest success factors in this project, and that the level of commitment was extraordinary.
4.1. Team factors
We begin by describing the development team, located in the Czech Republic, and the project management team, located in the USA. The number of developers varied during the course of the project, from four at the beginning, to seven near the delivery deadline, to two during the maintenance phase. The developers had experience in http://www.xplanner.org telecommunications development but were required to learn many new and proprietary interfaces for the F-15 system, which created a non-trivial learning curve. The customer for the F-15 project was a representative from the customer service department whose task it was to train new customers on the use of Tekelec systems. The team members and their roles are summarized in Table 1 to provide a reference point for the rest of the paper.
4.2. Process factors This section describes the technologies used during the software development process. A more complete description of the team’s use of the XP process can be found in . The technological factors described in Table 2 give an overview of the software development approach taken during this project.
The team implemented many tenets of the XP process, while customizing others to fit their needs. The team practiced test-driven development (TDD) and unit tested all code. Over 75% of the work was spent in pairs (as recorded by the XPlanner tool) and followed rigorous coding standards. The developers integrated their code into a build machine at least once per day on which a regression test suite was run every eight hours. Also, the developers practiced collective code ownership, though the majority of work on certain pieces of code was often done by one pair. Only the lead developers took part in planning meetings, which departs from standard XP practice. This was primarily due to scheduling reasons, as the planning meetings took place at the end of the Czech workday and often extended well-after closing hours.
Throughout the project, the project management personnel and the development lead took part in release and iteration planning meetings. During the first month of the project, these meetings served as the primary means of communication between the two groups. The planning meetings were held in a conference room with the project management team with the development lead in the Czech Republic on a speakerphone. During these meetings, user stories were defined and added to the coming iteration to be worked on by the developers. One shortcoming of this approach is that it involved only one perspective from the development team, that of the development lead. The absence of the development team during planning meetings runs contrary to XP process . Other developers were involved with the call only if they had a specific question for a member of the project management team; developers were typically not involved with the decision-making process. Another obstacle noted by the tracker was that project management personnel explained system components on a whiteboard, which the development lead could not see. XP advocates these informal, whiteboard meetings, but the information could not be shared with the development team.
The team’s primary means of project management was through the XPlanner tool. XPlanner contained electronic versions of user stories and a mechanism for tracking the amount of effort developers spent on each user story. This information was used to analyze the team’s velocity, which is the amount of effort (in hours) the team completed in the previous iteration. The velocity was used to determine the number of features that should be scheduled for the next iteration. The XPlanner tool was accessible on-line by the entire team, including customers. Developers used the tool every 1-3 days to update the time spent working on user stories. During iteration planning meetings, the discussion centered on the information contained in XPlanner, which was updated by the project tracker during the meeting. He would then prompt the development lead (via speakerphone) to reload the current page whenever a change had been made. Change tracking was not available in XPlanner. As discussed in greater detail in Section 5.3, the team used also used a mailing list for project management activities. The mailing list allowed individual developers to communicate directly with customers to answer questions regarding feature specifications. All members of the development and project management teams were included on the mailing list.
The team also used daily status meetings between the development manager and the development team to discuss progress and technical issues. Maznevski and Chudoba  advocate the “regular rhythm” of such meetings for effective distributed teams. During these short status meetings (in the form of “stand up meetings” or Scrum meetings ), the team would have voice communication either via speakerphone or Voice Over IP (VOIP) teleconferencing software. Later in the project, these meetings also used whiteboard software so that the development manager and the development team could synchronously view and edit whiteboard contents. The meetings lasted from 15-30 minutes and involved each developer briefly recounting what he did on the previous day, any problems he encountered, and what he expected to do the current day. This meeting also served as a forum for the developers to discuss any issues that needed to be resolved with the project management team. The development manager could also use this time to explain design details or project planning details that the developers needed to know.
4.3. Project factors This section describes the characteristics of the project to help generate a better understanding of the size and scope of the system. This information is summarized in Table 3.
4.4. Project outcome This section discusses the F-15 project outcome measures that concern the business-oriented results of the project. We also provide information from a customer interview conducted after the project had been in production.
The project outcome measures are summarized in Table 4.
Table 4: Project outcome Outcome measure F-15 project Pre-release Quality (test defects/KLOEC) N/A Post-release Quality (post-release defects/KLOEC) 1.62 defects/KLOEC Customer Satisfaction Capability – Neutral (interview) Reliability – Satisfied Communication – Very Satisfied KLOEC/person month 1.22 KLOEC/PM
2.32 KLOEC/PM (including test code) User stories/person month 8.53 stories/PM Putnam Productivity Parameter  15782.72 The timeframe is presented roughly corresponding to releases. From iteration 1-1 through 1-3, the team had no definite customer. Iterations 2-1 through 2-5 represent the time period from when a customer joined the project through final system delivery. Iterations 3-1 forward represent the maintenance phase of the project. The reference line at 2-5 represents the project’s delivery deadline.
We use two measures of quality related to defects in the system. A pre-release quality measure (a surrogate measure of quality ) was not available since results of pre-release testing were not recorded. Just prior to release, the system was tested by the customer at a functional/system level, and bugs were recorded as user stories.
Post-release quality reflects the number of defects found by the customer once the system is in production. The customer entered post-release bugs he encountered as user stories in XPlanner. The post-release defect numbers for the F-15 project are lower than published industry averages for new software projects . In an interview, the customer also stated that he was satisfied with the reliability of the product, noting only one fatal crash that was resolved immediately after the product was shipped.