--- title: "Data Byte: Submission Limits On Autograders" description: Research conducted by Mimir on submission limits for automated grading systems for computer science courses. image: https://www.mimirhq.com/hubfs/Rotating.002.png --- Go back to [![](https://www.mimirhq.com/hubfs/Fall%202018%20Website/mimir-classroom-logotype-color-no-padding.svg) ![](https://www.mimirhq.com/hubfs/Fall%202018%20Website/mimir-classroom-logo-color-no-padding.svg)](https://www.mimirhq.com/) Features [Programming Projects](https://www.mimirhq.com/classroom/programming-projects) [Programming Assignments](https://www.mimirhq.com/classroom/programming-assignments) [Mimir IDE](https://www.mimirhq.com/classroom/ide) [Plagiarism Detection](https://www.mimirhq.com/classroom/plagiarism-detection) [Jupyter Notebooks](https://www.mimirhq.com/classroom/jupyter) [Student Analytics](https://www.mimirhq.com/classroom/analytics) [Course Management](https://www.mimirhq.com/classroom/course-management) [Content Library](https://www.mimirhq.com/classroom/content-library) Use Cases Built to save time, help students, and encourage improvement. [Streamline Coursework](https://www.mimirhq.com/classroom/streamline-coursework) [Improve Learning Experiences](https://www.mimirhq.com/classroom/improve-learning-experiences) [Reward Hard Work](https://www.mimirhq.com/classroom/reward-hard-work) [Pricing](https://www.mimirhq.com/classroom/pricing) [Get Started](https://class.mimir.io/register) [Log In](https://class.mimir.io/login) ### Mimir Classroom [Schedule a live demo](https://www.mimirhq.com/classroom/demo) [Pricing](https://www.mimirhq.com/classroom/pricing) ### Features [Programming Projects](https://www.mimirhq.com/classroom/programming-projects) [Programming Assignments](https://www.mimirhq.com/classroom/programming-assignments) [Online IDE](https://www.mimirhq.com/classroom/ide) [Plagiarism Detection](https://www.mimirhq.com/classroom/plagiarism-detection) [Course Management](https://www.mimirhq.com/classroom/course-management) [Content Library](https://www.mimirhq.com/classroom/content-library) [Jupyter Notebooks](https://www.mimirhq.com/classroom/jupyter) [Student Analytics](https://www.mimirhq.com/classroom/analytics) ### Use Cases [Streamline Coursework](https://www.mimirhq.com/classroom/streamline-coursework) [Improve Learning Experiences](https://www.mimirhq.com/classroom/improve-learning-experiences) [Reward Hard Work](https://www.mimirhq.com/classroom/reward-hard-work) # Data Byte: Submission Limits On Autograders ![Picture of Prahasith Veluvolu](https://www.mimirhq.com/hubfs/VeluvoluMimirHeadshot-1.jpg) Prahasith Veluvolu April 9, 2019 Every single day we receive tens of thousands of students submissions across hundreds of programming projects on Mimir Classroom. Since we can automatically evaluate all of these submissions, instructors can choose to allow unlimited submissions on projects without adding more work to their plate; however, limiting submissions is also an option and can alter how students tackle projects. We decided to take a look at projects that limit submissions to better understand the impact.  Instructors can set the submission limit to a number greater than 0 or select unlimited submissions. Pulling data from 2014 to early 2019, we found that we had a pretty close split between projects that allowed for unlimited submissions (44%) and those that had a maximum limit set (56%).  *![Rotating.002](https://www.mimirhq.com/hs-fs/hubfs/Rotating.002.png?width=1920&name=Rotating.002.png)*     *The graph below shows the distribution of projects by submission limit. About 30% of projects allowed for less than 10 submissions and we found no correlation between the level of the course and the submission limit.* *![Rotating.004](https://www.mimirhq.com/hs-fs/hubfs/Rotating.004.jpeg?width=1920&name=Rotating.004.jpeg)* *Looking at student submission patterns based on the limits imposed on the project resulted that limiting submissions to 20 or more was equivalent to allowing unlimited submissions. More interestingly, when students have fewer than 10 submissions, they are more likely to get a passing score in their initial submission.* We spoke to instructors who offer fewer than ten submissions and those that offer unlimited and learned the following: - Instructors that offer unlimited submissions often hide the results of a few of their automated test case until after the due date (using one of Mimir Classrooms features). This way students still have to debug on their own to make sure their code covers all the requirements. - Instructors that offer fewer than ten submissions usually do not hide any of their automated tests and ask students to debug on their own before submitting and treat every submission like a final one.   If you use automated grading in your computer science courses and would like to share you experience with us, reach out at [hello@mimirhq.com](mailto:hello@mimirhq.com).  ![](https://www.mimirhq.com/hubfs/Fall%202018%20Website/mimir-logotype-reverse-no-padding.svg) [Privacy Policy](https://www.iubenda.com/privacy-policy/546498/full-legal) [Terms of Service](https://class.mimir.io/lms/tos.html) © Copyright 2020 Mimir – All Rights Reserved Title and school names/logos on this page are provided for identification purposes only. The views and opinions expressed are those of the individual only and do not necessarily reflect the position of their respective organizations.