Skip to main content Link Menu Expand (external link) Document Search Copy Copied
CS4973/7580, Spring 2023

Reflection Paper Due Tuesday Mar 14, 11:00am EDT

The structure of this class is designed to provide a broad survey of many topics in software engineering practice and research:

  • Software Process
  • Modularity and Design
  • Mining Software Repositories
  • Open Source
  • Testing
  • Continuous Integration
  • Devops
  • Expertise and knowledge sharing
  • Security
  • Software engineering in specific domains

The goal of the reflection paper is for you to select a topic of particular personal interest, identify five research articles (that are not required readings for this course), read them, and organize your reaction to the topic in a reflection paper. The reflection paper provides you to an opportunity to apply critical reasoning skills in the context of the research papers that we discuss in class, and to dig deeper into a topic of your own particular interest.

This is not a literature review: the goal with this paper is for you to provide some editorial critiques of the work and express your own opinions, rather than to provide a fair and direct summary of the contents of each paper. The expected length of the reflection paper is roughly 2,500 words.

There are three main deliverables for the reflection paper:

  1. Paper proposal
  2. The reflection paper
  3. A short presentation in class of your findings

The objectives for the reflection paper are to:

  • Compare and contrast different software testing and analysis approaches
  • Critically analyze research projects
  • Cultivate new ideas for research directions

Once you have selected the papers that you will respond to, consider one (or more) of the following prompts:

  • What are common themes or goals between these papers? Do they share similar assumptions (in terms of how software is designed and tested)?
  • How do these works complement each other?
  • Based on your own experiences developing software, do you believe that the evaluations in these papers provide sufficient evidence to draw the conclusions that are claimed?
  • If you have experience with other related works in software engineering, how do you think those works compare?
  • Are there next-steps that you think could be applied to extend the ideas presented in these papers?
  • Were there parts of some of these works that you thought were poorly explained, and could have been written in another way to be more accessible?

You should consider these prompts as jumping-off points: feel free to respond to one or more of these questions in your paper, or alternatively, something completely different. The most important part is that your paper should convey your own reactions to the six papers that you have selected — demonstrating that you read the papers and thought about them.

This is an individual assignment. The goal of this assignment is to provide your personal reflection to the topic of your choice. Papers that are substantially similar to each other (or to third-party content) will receive a score of 0, with no opportunity for a revision. If you have any questions or concerns of what constitutes plagiarism, please reach out to the instructor immediately.

Formatting

Feel free to use whatever word processing environment you prefer. We strongly encourage that you use some editor that will help you manage a bibliography — like LaTeX + BibTex, or Word + Endnote. Please clearly identify the one paper from the required reading list for the course plus the five additional papers that you intend to discuss — one way to do that might be to include them in your bibliography, and cite them in the first paragraph of your paper.

Presentation

To share your experience with the rest of the class, prepare a lightning talk (8 minutes) that summarizes what you learned about this topic, and your reaction to it. The goal of the lightning talk is to share one or two things that you learned that were interesting to you: an open (or closed) problem in that research area, some methodology or tool that addresses a known problem, etc. The presentation will be graded entirely on a “you did it” or “you didn’t” basis, as long as you show up at the assigned time and talk about the topic.

Grading

Your reflection paper will be graded on the scale of (Unacceptable, Check-, Check, Check+). The criteria for each grade are described below.

To receive the grade of Check, the paper/presentation must satisfy all of these criteria:

  • The paper is approximately 2,500 words (plus or minus a few hundred is OK; if you are far short of 2,500 you might consider deepening your exploration; if you are far over 2,500 words, you might consider condensing your efforts)
  • The paper responds to one of the articles on the course’s required reading list, plus the five articles that were approved by the course staff via the paper proposal
  • The paper demonstrates a thoughtful understanding of the papers selected
  • The paper uses relevant examples from the papers discussed in class to support and describe your reactions to the works
  • The paper uses language that is understandable, and generally does not distract from the content of the work
  • The presentation describes a concept related to the paper.

To receive the grade of Check-, the paper must satisfy all of these criteria:

  • The paper is between 1,500-2,500 words
  • The paper responds to one of the articles on the course’s required reading list, plus the five articles that were approved by the course staff via the paper proposal
  • The paper demonstrates a limited understanding of the papers selected, focusing primarily on summarizing the content of the papers without demonstration of higher-level understanding of them

Submissions that do not meet the criteria for “Check-“ will receive the grade of “Unacceptable.” Submissions that exceed the qualities outlined for “Check” (e.g. include additional, insightful connections between the papers discussed in class and other works, uses particularly sophisticated and engaging language, and generally demonstrates an exceptional and thorough understanding of the selected papers) may receive the grade of “Check+”. As per the course policies, note that a grade of “Check” is sufficient to receive an A in the class.

Submission

Submit your paper on Canvas. The paper is due on March 14 at 11:00am. No late submissions will be accepted. Presentations will occur in-class on March 14 and 16.

Revision

Students who make a good faith effort on the paper (e.g. submit a document on time that is conformant to the general guidelines explained above), and receive a grade below “Check” will receive detailed feedback on their paper, and be allowed to resubmit the paper (within 2 weeks), under the understanding that the highest grade of the submissions will be used. Students who receive a grade of “Check” may not resubmit their work to attempt to achieve a “Check+” (again, note that in this situation, this would have no impact on your course grade).

List of all articles selected by students

We have aggregated all of the articlds selected by students for their reflection papers into lists organized by topic (topics with more articles listed reflect topics that more students selected for their reflection papers)

Modularity and design

Open source

  • Neamtiu, I., Xie, G., Chen, J. (2013). Towards a better understanding of software evolution: an empirical study on open-source software. Journal of Software: Evolution and Process, 25(3), 193-218.
  • Comino, S., Manenti, F. M. (2011). Dual licensing in open source software markets. Information Economics and Policy, 23(3-4), 234-242.
  • Capra, E., Francalanci, C., Merlo, F. (2008). An empirical study on the relationship between software design quality, development effort and governance in open source projects. IEEE Transactions on Software Engineering, 34(6), 765-782.
  • Tan, X., Zhou, M., Fitzgerald, B. (2020, June). Scaling open source communities: An empirical study of the Linux kernel. ACM/IEEE 42nd International Conference on Software Engineering (pp. 1222-1234).
  • Qiu, H. S., Li, Y. L., Padala, S., Sarma, A., Vasilescu, B. (2019). The signals that potential contributors look for when choosing open-source projects. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-29..
  • J. Coelho and M. T. Valente, “Why modern open source projects fail,” in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, (New York, NY, USA), p. 186–196, Association for Computing Machinery, 2017.
  • C. Mendez, H. S. Padala, Z. Steine-Hanson, C. Hilderbrand, A. Horvath, C. Hill, L. Simpson, N. Patil, A. Sarma, and M. Burnett, “Open source barriers to entry, revisited: A sociotechnical perspective,” in Proceedings of the 40th International Conference on Software Engineering, ICSE ’18, (New York, NY, USA), p. 1004–1015, Association for Computing Machinery, 2018.
  • X. Tan, M. Zhou, and B. Fitzgerald, “Scaling open source communities: An empirical study of the linux kernel,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, ICSE ’20, (New York, NY, USA), p. 1222–1234, Association for Computing Machinery, 2020.
  • E. Dias, P. Meirelles, F. Castor, I. Steinmacher, I. Wiese, and G. Pinto, “What makes a great maintainer of open source projects?,” in Proceedings of the 43rd International Conference on Software Engineering, ICSE ’21, p. 982–994, IEEE Press, 2021.
  • T. Dey, A. Karnauch, and A. Mockus, “Representation of developer expertise in open source software,” in Proceedings of the 43rd International Conference on Software Engineering, ICSE ’21, p. 995–1007, IEEE Press, 2021.
  • D. Riehle, P. Riemer, C. Kolassa, and M. Schmidt, “Paid vs. volunteer work in open source,” in Proceedings of the Annual Hawaii International Conference on System Sciences, 01 2014.

Testing

  • Dirk Beyer, Adam J Chlipala, Thomas A Henzinger, Ranjit Jhala, and Rupak Majumdar. 2004. Generating tests from counterexamples. In Proceedings. 26th International Conference on Software Engineering. IEEE, 326–335.
  • Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs.. In OSDI, Vol. 8. 209–224.
  • Koen Claessen and John Hughes. 2000. QuickCheck: a lightweight tool for random testing of Haskell programs. In Proceedings of the fifth ACM SIGPLAN international conference on Functional programming. 268-279.
  • George Fink and Matt Bishop. 1997. Property-based testing: a new approach to testing for assurance. ACM SIGSOFT Software Engineering Notes 22, 4 (1997), 74-80.
  • Thomas A Henzinger, Ranjit Jhala, Rupak Majumdar, and Gregoire Sutre. 2002. Lazy abstraction. In Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 58-70.
  • Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation. 213–223.
  • “Web accessibility evaluation methods: A systematic review. Design, User Experience, and Usability”, Nuñez et al.
  • “A brief survey of current software engineering practices in continuous integration and automated accessibility testing”, Sane
  • “Aid: An automated detector for gender-inclusivity bugs in OSS Project Pages”, Chatterjee et al.
  • “Mobile accessibility guidelines adoption under the perspective of developers and designers”, de Almeida and Gama
  • “Evaluating responsive web design’s impact on Blind Users”, Nogueira et al.
  • “Accessibility of UI frameworks and libraries for programmers with visual impairments”, Pandey et al.
  • Lin, Jun-Wei, Reyhaneh Jabbarvand, and Sam Malek. “Test transfer across mobile apps through semantic mapping.” 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2019.
  • Behrang, Farnaz, and Alessandro Orso. “Test migration between mobile apps with similar functionality.” 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2019.
  • Mariani, Leonardo, et al. “Semantic matching of gui events for test reuse: are we there yet?.” Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 2021.
  • Bajammal, Mohammad, and Ali Mesbah. “Semantic Web Accessibility Testing via Hierarchical Visual Analysis.” 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021.
  • Walsh, Thomas A., Phil McMinn, and Gregory M. Kapfhammer. “Automatic detection of potential layout faults following changes to responsive web pages (N).” 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2015.
  • “Automated Model-Based Android GUI Testing Using Multi-Level GUI Comparison Criteria”, Baek and Bae
  • “Automated testing of virtual reality application interfaces.”, Bierbaum et al.
  • “On the efficiency of automated testing”, Böhme and Paul
  • Automated Accessibility Testing of Mobile Apps”, Eler et al.
  • Robert Nilsson and Jeff Offutt. 2007. Automated Testing of Timeliness : A Case Study. (2007).
  • T.Y Chen, S.C. Cheung, and S.M. Yiu. Metamorphic testing: a new approach for generating next test cases.
  • Kyle Dewey, Jared Roesh, and Ben Hardekopf. Language fuzzing using constraint logic programming.
  • Kyle Dewey, Jared Roesch, and Ben Hardekopf. Fuzzing the Rust Typechecker UsingCLP.
  • Sungjae Hwang, Sungho Lee, Jihoon Kim, and Sukyoung Ryu. JUSTGen: Effective Test Generation for Unspecified JNI Behaviors on JVMs.
  • Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. Finding and Understanding Bugs in C Compilers.
  • Alex Groce, Chaoqiang Zhang, Eric Eide, Yang Chen, and John Regehr. Swarm testing. In Proc. ISSTA, pages 78–88, July 2012
  • Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Transactions on Software Engineering, vol. 37, no. 5. pp. 649–678, 2011.
  • G. Petrović, M. Ivanković, B. Kurtz, P. Ammann, and R. Just, “An industrial application of mutation testing: Lessons, challenges, and research directions,” in Proceedings - 2018 IEEE 11th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2018, Jul. 2018, pp. 47–53.
  • G. Petrovic, M. Ivankovic, G. Fraser, and R. Just, “Practical Mutation Testing at Scale: A view from Google,” IEEE Transactions on Software Engineering, vol. 48, no. 10. Institute of Electrical and Electronics Engineers Inc., pp. 3900–3912, Oct. 01, 2022.
  • R. J. Lipton and F. G. Sayward, “Hints on test data selection: Help for the practicing programmer,” Computer (Long Beach Calif), vol. 11, no. 4, pp. 34–41, 1978.
  • M. Ivankovi, G. Petrovi, R. Just, and G. Fraser, “Code coverage at Google,” in ESEC/FSE 2019 - Proceedings of the 2019 27th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Aug. 2019, pp. 955–963.

DevOps

Expertise and knowledge sharing

  • R. Kavitha and M. Irfan Ahmed, “A knowledge management framework for agile software development teams,” in 2011 International Conference on Process Automation, Control and Computing, pp. 1–5, 2011.
  • S. Mirhosseini and C. Parnin, “Docable: Evaluating the executability of software tutorials,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conferencve and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, (New York, NY, USA), p. 375–385, Association for Computing Machinery, 2020.
  • A. Horvath, M. X. Liu, R. Hendriksen, C. Shannon, E. Paterson, K. Jawad, A. Macvean, and B. A. Myers, “Understanding how programmers can use annotations on documentation,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, (New York, NY, USA), Association for Computing Machinery, 2022.
  • T. D. LaToza, M. Arab, D. Loksa, and A. J. Ko, “Explicit programming strategies,” Empirical Software Engineering, vol. 25, no. 4, p. 2416–2449, 2020.
  • “Social media and tacit knowledge sharing: Developing a conceptual model”, Panahi et al.
  • “An exploratory study of live-streamed programming”, Alaboudi and LaToza
  • “How social and communication channels shape and challenge a participatory culture in software development”, Storey et al.
  • “Analyzing user comments on YouTube coding tutorial videos”, Poché et al.
  • “Influence of social and technical factors for evaluating contribution in GitHub”, Tsay et al.
  • S. Baltes and S. Diehl, “Towards a theory of software development expertise,” ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 187-200, 2018.
  • F. O. Bjørnson and T. Dingsøyr, “Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used,” Information and Software Technology, vol. 50, no. 11, pp. 1055-1068, 2008.
  • M. Zhou and A. Mockus, “Developer fluency: achieving true mastery in software projects,”FSE ‘10: Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering, pp. 137-146, 2010.
  • M. Caulo, B. Lin, G. Bavota, G. Scanniello and M. Lanza, “Knowledge Transfer in Modern Code Review,” ICPC ‘20: Proceedings of the 28th International Conference on Program
  • C. Treude and M.-A. Storey, “Effective communication of software development knowledge through community portals,” ESEC/FSE ‘11: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, pp. 91-101, 2011.

Security

Software engineering in specific domains

Data science

Interactive theorem provers

  • A. W. Appel, “Coq’s vibrant ecosystem for verification engineering (invited talk),” in Proceedings of the 11th ACM SIGPLAN international conference on certified programs and proofs, 2022, pp. 2–11.
  • F. van Doorn, G. Ebner, and R. Y. Lewis, “Maintaining a library of formal mathematics,” in Intelligent computer mathematics: 13th international conference, CICM 2020, bertinoro, italy, july 26–31, 2020, proceedings, 2020, pp. 251–267.
  • K. Roe and S. Smith, “CoqPIE: An IDE aimed at improving proof development productivity: (Rough diamond),” in Interactive theorem proving: 7th international conference, ITP 2016, nancy, france, august 22-25, 2016, proceedings 7, 2016, pp. 491–499.
  • A. Celik, K. Palmskog, M. Parovic, E. J. G. Arias, and M. Gligoric, “Mutation analysis for coq,” in 2019 34th IEEE/ACM international conference on automated software engineering (ASE), 2019, pp. 539–551.
  • T. Ringer, R. Porter, N. Yazdani, J. Leo, and D. Grossman, “Proof repair across type equivalences,” in Proceedings of the 42nd ACM SIGPLAN international conference on programming language design and implementation, 2021, pp. 112–127.

Healthcare

  • Alshayban, A., Ahmed, I., & Malek, S. (2020, June). Accessibility issues in android apps: State of affairs, sentiments, and ways forward. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (pp. 1323-1334).
  • Perera, H., Hussain, W., Whittle, J., Nurwidyantoro, A., Mougouei, D., Shams, R. A., & Oliver, G. (2020, June). A study on the prevalence of human values in software engineering publications, 2015-2018. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (pp. 409-420).
  • Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020, April). A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-12).
  • Garde, S., & Knaup, P. (2006). Requirements engineering in health care: the example of chemotherapy planning in paediatric oncology. Requirements Engineering, 11, 265-278.
  • Sutcliffe, A., Sawyer, P., Liu, W., & Bencomo, N. (2021, May). Investigating the potential impact of values on requirements and software engineering. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) (pp. 39-47). IEEE.

Program understanding

© 2022 Jonathan Bell. Released under the CC BY-SA license