Paper Presentation and Implementation

The purpose of this segment is to deepen your understanding and engagement with cutting-edge RL research through the detailed study, presentation, and where applicable, implementation of a significant paper in the field. This exercise aims to emulate the process of a conference presentation combined with a critical review session.

Project Stages

  • Paper Presentation (rolling deadlines, earliest Oct 13): Team-based paper overview delivered as a 12–15 minute presentation with supporting slides.
  • Midterm Project (due Oct 24): Reproduce the primary results from the selected paper, delivering code, analysis, and discussion of deviations.
  • Final Project (due Dec 5): Extend the reproduced work with a substantive modification such as a new environment or algorithmic improvement, accompanied by the final report, code, and video.

Team Sign-Up

Before getting started, please team up and sign up for your chosen paper on the following Google Sheet:
Team Sign-Up Sheet.

Presentation Structure

Each team will be responsible for the following components:

Presentation

  • Duration:
    • Standard Papers: 12-15 minutes.
    • Survey Papers: 30-45 minutes, presented in a tutorial style.
  • Format:
    • Slides: Prepare a slide deck to guide your talk.
    • Analysis: Cover the paper’s background, problem statement, methodology, experiments, key findings, and implications. Highlight the significance of the work and its contributions to the field.
      • Background: Discuss prior works leading up to the publication of the paper.
      • Problem Statement: Clearly outline the primary problem the paper addresses.
      • Motivation: Discuss why the problem is important and what drives the research.
      • Method: Summarize the approach or methodology the paper proposes, highlighting any novel techniques or theories.
      • Experiments: Provide a short analysis and comment on the experiments conducted in the paper, focusing on their design, execution, and the validity of the conclusions drawn from them.
      • Future Work: Discuss works that have resulted from or built upon the paper.
      • Critique: Offer a critical analysis of the paper’s strengths, weaknesses, impact, and contributions. Anticipate potential criticisms and questions that might arise during the presentation, simulating the author’s rebuttal to peer review comments.
  • Content Distribution:
    • 5 minutes dedicated to providing necessary background and context.
    • 7-10 minutes focused on the main material of the paper.

Quiz Questions:

  • Prepare 5 quiz questions with answers to ensure that the audience has grasped the key concepts of the paper. These questions should be crafted in such a way that they can be answered by attendees based solely on the information from your presentation.
  • Ensure the questions are Canvas-friendly (e.g., multiple-choice, true/false) and auto-gradable.
  • Questions should take no more than 5 minutes to answer and focus on the key takeaways from your presentation.
  • All students are required to watch the recorded presentations. There will be an in-class quiz to answer the associated quiz questions during the following week.
  • All students are required to peer review the recorded presentations before the in-class quiz. A Rubric and Canvas assignment will be posted each week to complete the peer review.

Midterm Structure

Implementation

  • Goal: Implement a minimal version of the main algorithm or method presented in the paper, using a simple environment to demonstrate the paper’s features.
  • Code Quality:
    • The code should be completely self-contained with detailed installation instructions.
    • It is recommended to include a Python notebook to walk through the main points, where math and code can be demonstrated side by side.
  • Submission Guidelines:
    • Submit the codebase in one of the following formats:
      • A GitHub repository link (included in the report).
      • A zip file uploaded directly with your submission.
    • The code should include:
      • A README file with clear instructions on how to set up and run the implementation.
      • Necessary dependencies and reproducibility details.

Report

  • Length: Approximately 2 pages.
  • Content:
    • Introduction: Briefly summarize the core idea of the paper and its significance.
    • Modification Description: Clearly explain your proposed modification.
    • Motivation: Justify why your modification is interesting or valuable.
    • Feasibility: Explain why your modification is achievable given your time and resources.
  • Submission Guidelines:
    • Please submit in PDF format.

Exceptions: - If implementing the paper is impractical (e.g., if it’s a survey paper or requires resources beyond the scope of the course), teams must inform the instructors one week ahead of the deadline, providing a detailed justification for the lack of implementation. - Explain the barriers to implementation and discuss the theoretical implications or applications of the paper instead.

Final Structure

Purpose

The final deliverable builds on your midterm submission. You will now implement the proposed modification (a meaningful modification or extension to a research paper), conduct experiments, and report on your findings. This is your opportunity to apply what you’ve learned, showcase creative ideas, and demonstrate a strong understanding of your chosen method.

Part 1: Modification Implementation and Evaluation

  • Description: Implement the modification proposed in your midterm submission. Conduct experiments to evaluate the effect of your modification. Compare your modification results to the original baseline method and provide an analysis of the performance, strengths, or limitations. The grade will NOT depend on the success of your original idea. However, the instructors will expect a proper analysis of a failure. Why did your algorithm succeed or fail? Please try to provide additional evidence- just like a (short) conference paper.

  • Deliverable: A functional codebase that implements your modification, and accompanying results and analysis.

Part 2: Presentation Video

  • Description:
    • Create a 7-minute presentation video summarizing your project. The video should include:
      • A brief overview of the original paper and its core idea.
      • A summary of your reproduction and modification.
      • A description of your experimental setup.
      • Key findings and observations.
      • Challenges, takeaways, and future work (if applicable).
  • Deliverable:
    • Upload your video to a platform (YouTube, Google Drive, Dropbox, etc.) and include the link in your final report.

Part 3: Final Report (2–4 pages, not a strict limit)

  • Submit a PDF report that includes the following:
    • Introduction and Background: Briefly summarize the core idea of the paper and your reproduction work.
    • Modification Description: Clearly describe the modification you implemented and the motivation behind it.
    • Experimental Setup: Describe your evaluation strategy. Include any datasets used, training setup, and evaluation metrics.
    • Results and Discussion: Include graphs, tables, or visuals comparing your modification to the original method. Discuss your observations.
  • Presentation Video Link
    • Provide a link to your presentation video.

Part 4: Team Contribution Survey

Each team member will fill out a survey describing each member’s contribution to the project (e.g., implementation, experimentation, writing, etc.).

Part 5: Code

  • Submit the codebase in one of the following formats:
    • A GitHub repository link (included in the report), or
    • A zip file uploaded directly with your submission.
  • The code should include:
    • A README file with setup and run instructions.
    • Any necessary scripts and dependencies for reproducing your experiments.

General Guidelines

  • Engagement: Encourage interactive discussion and feedback during your presentation to simulate a real conference talk environment.
  • Preparation: In addition to your slides, consider preparing additional materials or handouts if they help clarify complex concepts discussed in the paper.
  • Feedback: After your presentation, you will receive feedback from both peers and instructors. Use this feedback to refine your understanding and presentation skills.