Course Overview

This seminar focuses on Deep Representation Learning, exploring how deep neural networks acquire hierarchical and versatile representations during training. The course combines theoretical insights with practical applications, guiding students through the process of analyzing, evaluating, and improving representations in deep learning models. Participants will reproduce and adapt research papers, write scientific reports, and present their findings.

Key Topics

• Analysis and evaluation of deep neural network representations
• Methods for improving interpretability and reusability of representations
• Comparison of representations across different networks
• Learning task-specific vs. universal representations

Course Structure

The seminar is project-oriented, with students working in groups to reproduce and adapt research papers. The course is divided into 13 sessions, including lectures, student presentations, and hands-on support sessions. Key milestones include:

• Exposé/Proposal Presentation: Students present their project ideas and plans.
• Project Idea Presentations: Students share their progress and challenges.
• Final Presentations: Students present their completed projects and findings.

Prerequisites

• Background in deep learning (e.g., ANNs with TensorFlow or PyTorch)
• Familiarity with linear algebra, statistics, and information theory
• Experience with a deep learning framework (TensorFlow or PyTorch)

Resources

• Course Materials on Stud.IP

Session Timeline

Session 1: 17.10.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Lecture
• Description: Introduction and Organization

Session 2: 24.10.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Lecture
• Description: Scientific Work

Session 3: 07.11.2023, 10:00-12:00

• Presenter: Students
• Style: Presentations
• Description: Exposé/Proposal Presentations

Session 4: 14.11.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Lecture
• Description: Literature Research and LaTeX Introduction

Session 5: 21.11.2023, 10:00-12:00

• Presenter: Students
• Style: Presentations
• Description: Project Idea Presentations 1

Session 6: 28.11.2023, 10:00-12:00

• Presenter: Students
• Style: Presentations
• Description: Project Idea Presentations 2

Session 7: 05.12.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Lecture/Support
• Description: Experiments 1: How to Calculate

Session 8: 12.12.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Support
• Description: Experiments 2: How to Calculate

Session 9: 19.12.2023, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Lecture
• Description: Scientific Writing 1: How to Write

Session 10: 09.01.2024, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Support
• Description: Scientific Writing 2: How to Write

Session 11: 16.01.2024, 10:00-12:00

• Presenter: Lukas, Ulf, Yusuf
• Style: Work
• Description: The Review Process: How to QA

Session 12: 23.01.2024, 10:00-12:00

• Presenter: Students
• Style: Presentations
• Description: Final Presentations 1

Session 13: 30.01.2024, 10:00-12:00

• Presenter: Students
• Style: Presentations
• Description: Final Presentations 2

Extended Descriptions

Session 1: Introduction and Organization

• Explain the course structure and expectations.
• Tips for finding research topics and searching for papers.
• Guidance on accessing resources (e.g., university VPN, Google Scholar, arXiv).

Session 2: Scientific Work

• Introduction to the scientific method.
• Steps for reproducing and adapting research papers.
• Teamwork and version control with Git.

Session 3: Exposé/Proposal Presentations

• Students present their project proposals.
• Feedback and group formation.

Session 4: Literature Research and LaTeX Introduction

• How to conduct a literature review.
• Introduction to LaTeX and BibTeX for scientific writing.
• Tools like Zotero for organizing references.

Session 5 & 6: Project Idea Presentations

• Students present their project ideas, progress, and challenges.
• Feedback and guidance for next steps.

Session 7 & 8: Experiments 1 & 2

• Hands-on support for running experiments.
• Guidance on using tools like Git, VS Code, and EduVPN.

Session 9 & 10: Scientific Writing 1 & 2

• Structuring a scientific paper (abstract, introduction, related work, methods, experiments, discussion).
• Avoiding plagiarism and proper citation practices.

Session 11: The Review Process

• Introduction to the peer-review process.
• Students review each other’s work using a review form.

Session 12 & 13: Final Presentations

• Students present their completed projects and findings.
• Feedback and discussion.

Files and Resources

• 20230913_RL-Meeting-Board.pdf: Digital board notes from the discussion session on 13.09.2023.
• 20230920_Session-Plan.txt: Updated session plan from 20.09.2023.
• 20230920_Representation-Learning-Introduction.txt: Topics from the ICLR Conference for the introduction.
• 20230927_Meeting-Board.jpg: Image of the meeting board with a timeline for sessions.
• 20230927_Session-Plan.txt: Updated session plan from 27.09.2023.
• 20231003-Session-Plan-Summary.md: Summary of the session plans.