October 19, 2017
At Mimir (YC S15) we handle automated grading (among other features) for instructors teaching computer science courses. This goal has left our servers with a rather unusual workload. Unlike most services which need to scale for millions of small, simple requests, we have to scale for thousands of extremely CPU and IO intensive tasks to compile and run and test student code.
This means that there are really two parts to our backend infrastructure. The first, which I’ll call “classroom” from here on out, handles simple requests for our app, such as loading a list of active projects and displaying due dates. The second, which I’ll call “Odin” from here on out, handles intensive tasks such as compiling a Java program and running it against a test case. When we wrote our first iteration of Mimir Classroom in 2014, we were focused on delivering an MVP rather than a scalable solution, so we wrote both of these aspects into a CakePHP web app. Due to our own lack of development and dev-ops skill (we were freshmen in college at the time), this app was messy, not scalable, and not as reliable as it needed to be. When we got accepted into Y Combinator, it was clear we needed to rethink our stack in a full rewrite of the platform.
Our next iteration led us to choose two different stacks for classroom and for Odin (this will be a story for another time told by another one of Mimir’s co-founders, Jacobi). I led development on classroom and chose Ruby on Rails. For now we’ll focus on the choices we made for our front-end and back-end.
This choice was the easiest. PostgreSQL is by far the most well supported, full featured SQL database in existence. It’s open source and community driven. Storage has never been our largest scaling challenge, so we did not have the need for faster but less reliable and less comfortable NoSQL databases.
When we were making the decision of what framework we wanted to use for the back-end we had a few options. The first was to continue using CakePHP, but this was quickly ruled out since CakePHP was slow and years of PHP development takes it’s toll on a developer. We found that instead of benefiting from the features available in CakePHP, we found ourselves consistently fighting with the PHP language and Cake’s ORM. The other two options we considered were Django and Rails. Both offered great support, modern languages, powerful abstractions, and huge communities. We ultimately decided on Ruby on Rails since I had previous experience using it and was confident in my ability to teach my cofounders so they could contribute.
One thing that many React developers may find interesting is that I have not mentioned Redux. Redux is a solution for state management that can make writing functions to update the state more intuitive. While Redux is a great library and will work great on some teams, we found it simply added too much boilerplate to make it efficient to push new features. Even Dan Abramaov, creator of Redux, admits that you might not need redux. In place of Redux, we utilize React’s default state management solution alongside one I built myself, which works much like React’s built in state but extracted into it’s own component. I would encourage anyone considering state management solutions to “write their own” Redux. Redux’s code is so simple you can write it yourself in an afternoon, and from there you can think of your own philosophy on how state updates should be communicated.
In the end, we ended up rewriting every page to use React, rather than just using it on certain pages that needed interactivity. This made navigation between pages much snappier and allowed us to reuse convenient elements like sorting and searching on nearly every page.