Overview

Data management systems are the corner-stone of modern applications, businesses, and science (including data). If you were excited by the topics in 4111, this graduate level course in database systems research will be a deep dive into classic and modern database systems research. Topics will range from classic database system design, modern optimizations in single-machine and multi-machine settings, data cleaning and quality, and application-oriented databases.

The class places a heavy emphasis on paper reading and discussion. The point is to practice reading papers critically, discussing facets of the papers, implementing ideas in research papers, and conducting research. As such, students will be expected to read papers in depth and conduct a semester-long research project.

Ideally, you will be comfortable with reading code that is not yours, open to trying different software systems, and willing to actively participate in and lead discussions.

Course Expectations

• Students are expected to actively participate in class discussions; participation is mandatory.
• Students should be comfortable reading research papers; you will read, answer questions, and comment on the readings before class.
• Students should be comfortable coding data processing systems; familiarity with database query execution, some machine learning, and algorithms is expected.
• Students should be comfortable conducting a research project and writing up the results in a research paper format.

Participation

Participation is mandatory. See the first class’ slides

Slack

• Heavy use of Slack, Google Drive features.
• Discussion about the next class will be in the #class-DATE slack channel.
• You are expected to ask and answer questions on slack.

Students are given a role to focus on in the next class’s discussion.

• Before each class
  • By Monday 10PM: start slack private message with teammates
  • By Tuesday 10PM: post to next class’ slack channel at least one question about the papers.
  • By Wednesday 10PM:
    • add team slides to the class’ Google Slides.
    • Respond to 1+ question in Slack
• During each class
  • One person from each role takes ~5 minutes to discuss the paper from your role’s perspective
  • Discuss and ask questions

Project (semester long)

You will pursue a semester long research project related to this course. The project is a significant part of the course grade.

FAQ

What’s the relationship between the different database courses? (solid lines are prereqs, blue dashed lines are recommended).

What’s the difference from 6111 Advanced Database Systems?

• Both 6111 and this class (6113) are continuations of 4111, in that they dive deeper into the research literature and more advanced topics. 6111 focuses a lot on information retrieval and the web (essentially, viewing the web as a big “database”). This class focuses on the research and design of database internals and systems design.
• Take a look at the readings for both classes to get a better sense of the difference!

What’s the difference from 4112 Database System Implementation?

• 4112 is a continuation of 4111. It dives deeper into the topics typically mentioned in the second half of 4111, such as indexing, query optimization, transactions and concurrency control, and durability. This class primarily covers more “established” content found in database management textbooks, and may sometimes refer to papers when appropriate.
• In contrast, 6113 is heavily oriented around reading research papers and conducting research. 6113 will read research literature that ultimately led to some of the ideas covered in 4112, but will also cover more advanced database systems research as well.

Can I take both 6111 and 6113 for credit?

• Yes.

Collaboration/Copying Policy

Refer to Columbia’s academic honesty policy if you are at all unsure.

You must write all the code you hand in for the programming assignments, except for code that we give you as part of the assignment. You are not allowed to look at anyone else’s solution, you are not allowed to look at solutions from previous years, and you are not allowed to look at solutions from other universities. You may discuss the assignments with other students, but you may not look at or use each other’s code. The same rule holds for the question assignments: you must write all answers yourself, not look at others’ answers, but you can discuss the questions with others at a high level. You are also not allowed to look for or at solutions to the assignments on the Internet. You can search for small pieces of code that solve small parts of your assignments, and you may use tutorials to learn, however if you copy any code from anywhere, we request that you identify the origin in a comment in the code.

Your reviews must be written originally, and be based on your own understanding and thoughts about the reading. Copying or paraphrasing content written by others is not allowed.

Be advised that we will be running all assignments through the MOSS code similarity tool, which is very accurate even after significant amount of obfuscation, so we will identify and report anyone who attempts to breach this rule. We will include in our tests solutions from previous years both from Columbia and elsewhere. Both copy-ers and copy-ees will be punished. You are responsible for protecting your code and homeworks from others and not leaving them lying around in publicly open directories.

Finally, you may discuss the questions for each assignment with other students, but you may not look at other students’ answers. You must write your answers yourself.