Homework Assignment #1 — Paper Presentation Sign-up

HW1 is due Sunday, September 3, at 11:59PM Central time. No grace period is allowed. If your submission is late, you get zero points.

Students are required to select a research paper from the list provided by the instructor (see Research Paper List below) and present the paper in an assigned lecture (listed on Lectures).

How to Submit HW1

Each team is required to email the instructor and the GSI with the subject CS8395-HW1 before the deadline. In the email, you must list at least 3 papers in order (indicating your preference order) so that we can assign all the papers based on preference and conflicts. Note that the paper assignment is FCFS. If we haven't received an email from your team before the deadline, we will randomly assign a paper to your team.

Presentation Format

There are 19 teams in this course in Fall 2023. Each team will only present one research paper (selected from the Research Paper List below). In one lecture, two papers will be presented (i.e., two teams present in one lecture).

Each paper presentation is expected to follow the rules:

Research Paper List

We provide 19 papers for the teams to select from:.

  1. *[Ayewah2008] Using Static Analysis to Find Bugs. FindBugs was an important milestone to real-world software. Google's ErrorProne and Synopsys's Coverity are industrial-strenth tools that are popular today.
    Yueke Zhang & Xiaohan Wang
  2. *[Weimer2010] Automatic Program Repair with Evolutionary Computation GenProg was one of the first approaches on (heuristic) automatic program repair (published in ICSE one year before which receives the Most Influential Paper in ICSE 2019).
    Yuxin Hu & Weiheng Qiu
  3. *[Memon2017] Taming Google-Scale Contonuous Testing This paper was published in ICSE-SEIP 2017. The SEIP (SE in Practice) companion track focuses on software engineering practices in industry. This paper talks about experiences of program testing on real-world large scale programs (i.e., Google).
    David Gao
  4. *[Parnin2011] Are Automated Debugging Techniques Actually Helping Programmers? While researchers in software engineering provide so many tools every year, do they actually help develpers? If not, why? How can we learn from developers' behaviors to imprivde tool designs? This paper is a great example of human factor studies in SE that looked at tool usage problems.
    Zihan Fang
  5. *[Feng2023] Efficiency Matters: Speeding Up Automated Testing with GUI Rendering Inference GUI testing is a unique testing task in SE and usually time-consuming. This paper presents an approach to dynamically shorten the waiting time for GUI rendering thus to reduce testing time. This paper was published in ICSE 2023.
    Jeffrey Pan
  6. *[Zhao2023] Demystifying Privacy Policy of Third-Party Libraries in Mobile Apps This paper was published in ICSE23 and explored potential issues in mobile app bahaviors that may violate user privcay policies.
    Azar Hasan
  7. *[Shin2010] Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities While static anlaysis and dynamic analysis are commonly used to detect software vulnerabilities, this paper presents an empirical way instead. This paper was published in TSE in 2010.
    Chi Huynh
  8. *[Tian2022] Learning to Construct Better Mutation Faults This paper proposed a DL-based mutation techniqe to construct mutation faults that better present real faults and boost downstream applications. This paper received the best paper award of ASE2022.
    Sandhya Vankataramaiah & Arpit Ojha
  9. *[Huang2019] Distilling Neural Representations of Data Structure Manipulation using fMRI and fNIRS This was the first paper that investigated the data structure manipulation process and revealed its relationship with sptial ability on the neurological level. It also for the first time used and compared two medical imaging modalities in one study and provided guidelines for future studies. This paper received the SIGSOFT Distinguished Paper Award in ICSE 2019.
    Manda Li
  10. *[Busjahn2015] Eye Movements in Code Reading: Relaxing the Linear Order Code reading is an important activity in SE. How do developer read code? This paper used eye tracking to explore such behaviors. This paper was published in ICPC 2015, an SE conference that focuses on program comprehension.
    Surya Chand Rayala & Sunny Ashutoshbhai Bhatt
  11. *[Dell2012] "Yours is Better!" Participant Response Bias in HCI Researchers usually run human studies to validate the improvement of a design for users. However, the design of human studies can easily introduce biases from the participants. This sounds terrifying. How should we design a human study?
    Runxuan Yu
  12. *[Behroozi2022] Asynchronous Technical Interviews: Reducing the Effect of Supervised Think-Aloud on Communication Ability Coding interviews are a critical test for programmers to land a job. But many aspects can lead to unfair judgement of programmers' performance in the interviews. This paper suggests a new format of conducting job interview. This paper was published in FSE22.
    Dung Nguyen
  13. *[Murphy-Hill2019] Do Developers Discover New Tools On The Toilet? While many tools are developed to ease software developers' work, most are not even recognized. This paper at Google explored a simple but turned-out-to-be useful solution to this problem. Also the title of this paper is quite cute:) This paper received the SIGSOFT Distinguished Paper Award in ICSE 2019.
    Jackson Rusch
  14. *[vanBreukelen2023] "STILL AROUND": Experiences and Survival Strategies of Veteran Women Software Developers. This paper targeted at a minority group in SE: older women (primarily) and presented a qualitative study on ageism and sexism in modern software companies. Ageism and sexism are a pervasive issue in SE and this paper demonstated quite intersting and inspiring findings and discussions. This paper received the ACM SIGSOFT Distinguished Paper Award in ICSE2023.
    Mary Stirling Brown
  15. *[Hindle2012] On the Naturalness of Software This was the milestone paper that analyzed/treated programming languages as a comparison with natural languages (i.e., English). It opened the wide interdisciplinary research between software engineering and NLP, especially on the predication for software tasks. Published in 2012, this paper later received the award of Most Influential Paper in ICSE 2022.
    Rohan Kola
  16. *[Guo2021] GraphCodeBERT: Pre-Training Code Representations with Data Flow This paper presents a great example on using domain knowledge in software engineering (i.e., AST, dataflow) in AI models to build AI tools for software tasks.
    Zoe Luther & Alexy Skoutnev
  17. *[Fang2023] A Four-Year Study of Student Contribution to OSS vs. OSS4SG with a Lightweight Intervention Open source software are not only for technical purposes. More and more OSS projects aim to solve societal issues. Meanwhile, people have suggested societal problems might better motivate the participation of programming. This paper studied how students contribute to general OSS and social good OSS projects over 4 years and investigated if the participation in social good projects has an impact on students' contribution to OSS in general. It also presented an intervention in college class settings that might improve students' contribution to social good projects. This paper was published in FSE2023.
    Qiantong Wang
  18. *[Terrel2017] Gender Difference and Bias in Open Source: Pull Request Acceptance of Women versus Men This was the first paper that investigated the acceptance rate of pull requests regarding contributors' genders to reveal a systematic bias against female developers in open source.
    Tito Ebiwonjumi

Presentation Content and Grading

This section should not surprise you about what a "good presentation" looks like. However, it is always useful to list the criterion explicitly.

Here I want to gratefully acknowledge Prof. Kevin Moran at George Mason Univeristy for sharing the presentation criteria in his course with me.

Presentation Content

A research paper presentation should include the following content:

  1. Overview of Motivation : Here you should tell the story of why the problem that the paper is tackling is important. This part should also include necessary background introduction. As we discussed in our lectures, it is very important to know your audience first. Do they have enough background to understand the motivation?
  2. Key Idea of Research : Here, crystallize the key idea or novelty behind the paper for the audience in an engaging way.
  3. Approach Description or Study Design: Here, describe the details of the approach or designed study in a way that is relatable to the audience. Include any necessary background required to understand the approach or study methods.
  4. Evaluation & Results: Provide an overview of the most important takeaway results with supporting evidence. Provide any necessary background about the evaluation methods used.
  5. Discussion Questions: At the end of the presentation, you are encouraged to include a set of discussion questions to drive the course of class discussion. As the presenter, you are also responsible to lead the Q&A session after the presentation. Thus, some prepared dicussion questions can be very useful.

Grading

This paper presentation will be graded out of 100 points (it takes 20% in the final grade) using the criteria below.

Category

Professional (100%)

Adequate (75%)

Needs Work (50%)

Serious Problems (25%)

Grade
Content Full grasp (more than needed) of material in initial presentation and in answering questions later, includes interesting discussion questions Solid presentation of material and answers all questions adequately but without elaboration, adequate discussion questions Less than a full grasp of the information revealed rudimentary presentation and answers to questions, discussion questions unclear No grasp of information, some misinformation, and unable to answer questions accurately, no discussion questions 40%
Visual Aids Visuals explain and reinforce the rest of the presentation, presentation has text on slides only where needed Visuals relate to rest of presentation, but fall short in explaining key topics, too much text on slides Visuals are too few or not sufficiently related to the rest of the presentation Visuals not used or are superfluous 20%
Organization Information presented in a logical interesting sequence that is easy for the audience to follow and tells the story of the paper Information is presented in a logical sequence that is easy for the audience to follow but is not engaging or exciting Presentation jumps between topics making it difficult to follow the story of the paper. Audience cannot follow presentations because it follows no logical sequence 10%
English No misspelled words or grammatical errors No more than two misspelled words or grammatical errors Three-five more misspelled words or grammatical errors More than 5 misspelled words or grammatical errors 10%
Elocution Speaks clearly, correctly and precisely, loud enough for audience to hear and slowly enough for easy understanding Speaks clearly, pronounces most words correctly, loud enough to be easily heard, and slow enough to be understood Speaks unclearly, mispronounces many major terms, and speaks too softly or rapidly to be easily understood Mumbles, mispronounces most important terms, and speaks too softly or rapidly to be understood 10%
Eye Contact Eye contact constant; minimal or no reading of notes Eye contact maintained except when consulting notes, which is too often Some eye contact but mostly reading from notes No eye contact, reads from notes exclusively 10%

Peer Reviews: Every student except for the presenters will be required to review the presentation ("Peer Reviews" takes 15% in your final grade). You will receive an online form at the beginning of the presentation via Piazza. For each grading criterion, you are asked to grade for the presenters. There are also a few extra questions for you to answer in the form. Your feedback will be shared with the presenters anonymously after the presentation. For the audience, every peer review is 5 points for your participation. But the instructor has the right to cut off at most 2 points if your feedback is meaningless. Students must fill evaluation forms independantly (even if you have a team of two).

Finally, the grading for each presentation will be 50% (getting rid off one highest and one lowest score, then take the average) from the audience and 50% from the instructors.