Homework Assignment #1 — Paper Presentation Sign-up

HW1 is due Friday, September 2, at 11:59PM Central time. No grace period is allowed. If your submission is late, you get zero points.

Students are required to select a research paper from the list provided by the instructor (see Research Paper List below) and present the paper in an assigned lecture (listed on Lectures).

How to Submit HW1

Each team is required to email the instructor with the subject CS8395-HW1 before the deadline. In the email, you must list at least 3 papers in order (indicating your preference order) so that we can assign all the papers based on preference and conflicts. Note that the paper assignment is FCFS. If we haven't received an email from your team before the deadline, we will randomly assign a paper to your team.

Presentation Format

There are 13 teams in this course in Fall 2022. Each team will only present one reserch paper (selected from the Research Paper List below). In one lecture, two papers will be presented (i.e., two teams present in one lecture).

Each paper presentation is expected to follow the rules:

Research Paper List

We provide 16 papers for the teams to select from:.

  1. *[Ayewah2008] Using Static Analysis to Find Bugs. FindBugs was an important milestone to real-world software. Google's ErrorProne and Synopsys's Coverity are industrial-strenth tools that are popular today.
    Adit Negi
  2. *[Weimer2010] Automatic Program Repair with Evolutionary Computation GenProg was one of the first approaches on (heuristic) automatic program repair (published in ICSE one year before which receives the Most Influential Paper in ICSE 2019).
    Presented by: Rajay Preddie
  3. *[Fowler2018] Fuzz Testing for Automotive Cyber-security This paper was selected as an example of interdisciplinary research between software engineering and cyber physical system. This is a relatively new area with a lot of potentials.
    Presented by: Hunter Baxster
  4. *[Hindle2012] On the Naturalness of Software This was the milestone paper that analyzed/treated programming languages as a comparison with natural languages (i.e., English). It opened the wide interdisciplinary research between software engineering and NLP, especially on the predication for software tasks. Published in 2012, this paper later received the award of Most Influential Paper in ICSE 2022.
    Presented by: Minh Vu
  5. *[Huang2019] Distilling Neural Representations of Data Structure Manipulation using fMRI and fNIRS This was the first paper that investigated the data structure manipulation process and revealed its relationship with sptial ability on the neurological level. It also for the first time used and compared two medical imaging modalities in one study and provided guidelines for future studies. This paper received the SIGSOFT Distinguished Paper Award in ICSE 2019.
    Presented by: Jay Barot
  6. *[Parnin2011] Are Automated Debugging Techniques Actually Helping Programmers? While researchers in software engineering provide so many tools every year, do they actually help develpers? If not, why? How can we learn from developers' behaviors to imprivde tool designs? This paper is a great example of human factor studies in SE that looked at tool usage problems.
    Presented by: Skyler Grandel
  7. *[Dell2012] "Yours is Better!" Participant Response Bias in HCI Researchers usually run human studies to validate the improvement of a design for users. However, the design of human studies can easily introduce biases from the participants. This sounds terrifying. How should we design a human study?
    Presented by: Zifeng Liang & Rooney Gao
  8. *[Lee2015] Comparing the Effectiveness of Online Learning Approaches on CS1 Learning Outcomes Software engineering education (also CS education) is an essential topic in 21st centry. It is a common approach for novice programmers to learn programming on online platforms. Different platforms use different tasks to train novices. Which works and which doesn't?
    Presented by: Joyce Horn Fonteles & Eduardo (Edward) Davalos Anaya
  9. *[Kelleher2007] Storytelling Alice Motivates Middle School Girls to Learn Computer Programming This paper presents a programming environment, Storytelling Alice, to teach middle school girls to program. It shows that such an environment can better motivate and engage users.
    Presented by: Di Liu & Yuzhe Yao
  10. * [Cherubini2007] Let's Go To the Whiteboard: How and Why Software Developers Use Drawings We see diagrams often in software development, but we still don't really know how and why developers create such visual images.
    Presented by: David Seo
  11. *[Guo2021] GraphCodeBERT: Pre-Training Code Representations with Data Flow This paper presents a great example on using domain knowledge in software engineering (i.e., AST, dataflow) in AI models to build AI tools for software tasks.
    Jack Long & Vamsi Immanneni
  12. *[Chaparro2019] Assessing the Quality of the Steps to Reproduce in Bug Reports This paper leverages learning models in AI/NLP to automatically identify and assess the quality of reported steps to reproduce bugs in bug reports. This paper received the SIGSOFT Distinguished Paper Award in FSE 2019.
    Rithik Reddy & Jamison Homata
  13. *[Tufano2022] Using Reinforcement Learning for Load Testing of Video Games This is a very new paper leveraging reinforcement learning to train smart agents as players to automatically test the performance of video games.
    Presented by: Nicholas Potteiger & Sanidhya Mangal
  14. *[Humbatova2021] DeepCrime: Mutation Testing of Deep Learning Systems Based on Real Faults Testing is a classic topic in SE research. However, how to test deep learning systems remains an open research problem. This paper proposes an approach to leverage mutation testing to test DL systems. This research topic belongs to the area of SE4AI.
    Presented by: Nitish Nimma
  15. *[Terrel2017] Gender Difference and Bias in Open Source: Pull Request Acceptance of Women versus Men This was the first paper that investigated the acceptance rate of pull requests regarding contributors' genders to reveal a systematic bias against female developers in open source.
    Presented by: Tabitha See Ya Lee
  16. *[Miller2022] "Did You Miss My Comment or What?" Understanding Toxicity in Open Source Discussions This was the first paper that tried to understand the characteristics of open source toxicity. What can we do to make open source a more inclusive, welcome and friendly place to sustain the particiaption? This paper received the SIGSOFT Distinguished Paper Award in ICSE 2022.
    Presented by: Zach Karas & Yifan Zhang

Presentation Content and Grading

This section should not surprise you about what a "good presentation" looks like. However, it is always useful to list the criterion explicitly.

Here I want to gratefully acknowledge Prof. Kevin Moran at George Mason Univeristy for sharing the presentation criteria in his course with me.

Presentation Content

A research paper presentation should include the following content:

  1. Overview of Motivation : Here you should tell the story of why the problem that the paper is tackling is important. This part should also include necessary background introduction. As we discussed in our lectures, it is very important to know your audience first. Do they have enough background to understand the motivation?
  2. Key Idea of Research : Here, crystallize the key idea or novelty behind the paper for the audience in an engaging way.
  3. Approach Description or Study Design: Here, describe the details of the approach or designed study in a way that is relatable to the audience. Include any necessary background required to understand the approach or study methods.
  4. Evaluation & Results: Provide an overview of the most important takeaway results with supporting evidence. Provide any necessary background about the evaluation methods used.
  5. Discussion Questions: At the end of the presentation, you are encouraged to include a set of discussion questions to drive the course of class discussion. As the presenter, you are also responsible to lead the Q&A session after the presentation. Thus, some prepared dicussion questions can be very useful.

Grading

This paper presentation will be graded out of 100 points (it takes 20% in the final grade) using the criteria below.

Category

Professional (100%)

Adequate (75%)

Needs Work (50%)

Serious Problems (25%)

Grade
Content Full grasp (more than needed) of material in initial presentation and in answering questions later, includes interesting discussion questions Solid presentation of material and answers all questions adequately but without elaboration, adequate discussion questions Less than a full grasp of the information revealed rudimentary presentation and answers to questions, discussion questions unclear No grasp of information, some misinformation, and unable to answer questions accurately, no discussion questions 40%
Visual Aids Visuals explain and reinforce the rest of the presentation, presentation has text on slides only where needed Visuals relate to rest of presentation, but fall short in explaining key topics, too much text on slides Visuals are too few or not sufficiently related to the rest of the presentation Visuals not used or are superfluous 20%
Organization Information presented in a logical interesting sequence that is easy for the audience to follow and tells the story of the paper Information is presented in a logical sequence that is easy for the audience to follow but is not engaging or exciting Presentation jumps between topics making it difficult to follow the story of the paper. Audience cannot follow presentations because it follows no logical sequence 10%
English No misspelled words or grammatical errors No more than two misspelled words or grammatical errors Three-five more misspelled words or grammatical errors More than 5 misspelled words or grammatical errors 10%
Elocution Speaks clearly, correctly and precisely, loud enough for audience to hear and slowly enough for easy understanding Speaks clearly, pronounces most words correctly, loud enough to be easily heard, and slow enough to be understood Speaks unclearly, mispronounces many major terms, and speaks too softly or rapidly to be easily understood Mumbles, mispronounces most important terms, and speaks too softly or rapidly to be understood 10%
Eye Contact Eye contact constant; minimal or no reading of notes Eye contact maintained except when consulting notes, which is too often Some eye contact but mostly reading from notes No eye contact, reads from notes exclusively 10%

Peer Reviews: Every student except for the presenters will be required to review the presentation ("Peer Reviews" takes 15% in your final grade). You will receive an online form at the beginning of the presentation via Piazza. For each grading criterion, you are asked to grade for the presenters. There are also a few extra questions for you to answer in the form. Your feedback will be shared with the presenters anonymously after the presentation. For the audience, every peer review is 5 points for your participation. But the instructor has the right to cut off at most 2 points if your feedback is meaningless. Students must fill evaluation forms independantly (even if you have a team of two).

Finally, the grading for each presentation will be 50% (getting rid off one highest and one lowest score, then take the average) from the audience and 50% from the instructor.