WhyParseFailedHeader

Why Parse Failed

With the impending shutdown of the Parse hosted service today, I want to reflect on why I think Parse failed and what the service meant to me. As a solopreneur and 8+ year iOS developer who has used Parse in all of his major projects, I think I have a worthy perspective on the value and shortcomings of the Parse service.

Parse got me to where I am

I’ve used Parse in all three of my most popular projects: Slide (5m+ installs), Falcross (350k+) and Jake and Amir (25k+). I don’t know if I’d have finished any of them without it.

As a budding developer who knew nothing about databases and had limited server software/deployment skills, Parse was a godsend. Parse provided an accessible abstraction to the “put this data online somewhere” problem, and allowed me to get familiar with the fundamental concepts of data storage, push notifications, server-side code, error logging and more without having to spend my time and patience on making sure these things worked in the first place.

Parse allowed me to focus on the products I was creating and not worry about the backend, which made me feel powerful and inspired me to put as much effort into making my apps great as I possibly could.

The Facebook acquisition

In April 2013, after I’d been using Parse for about 9 months, it was acquired by Facebook in an apparent bid to start providing mobile app infrastructure services to developers. Whether this was the first nail in Parse’s coffin is up for debate, but at the time I was mostly excited that Parse had access to significantly greater human and capital resources.

What killed Parse?

Many people have analyzed the business reasons that Facebook may have shuttered Parse, but most analyses haven’t focused on the nitty gritty technical details, and I believe that technical shortcomings played a large role in the decision to nix the service.

In this post I will focus on how Parse failed as a technology product, a topic with which I am intimately familiar.

Parse Technology: The Good

Mobile SDKs: The Parse iOS SDK was generally iterated upon quickly and without breaking changes. It eventually gained the ability to use custom subclasses to represent Parse objects which was useful in an OO sense. It also gained the ability to store objects locally for offline use, which actually ended up as a disaster (see “The Ugly”).

Query System: The Parse.Query mechanism provided an abstraction of MongoDB for reading, writing an updating database records. The API was simple, consistent across platforms, and features like scoresQuery.include('user') allowed for easy joins across collections.

Parse Dashboard & Data Browser: The dashboard was the ultimate one-stop shop for managing your app. Browsing database collections in realtime, including sorting, filtering, and row-level editing, were all a breeze with the Data Browser. Other sections provided many useful metrics like API requests per second, DB usage, number of active installations, recent push notifications, server logs, and more. One of my favorites was Jobs, a lovely UI for scheduling repeating or one-off background tasks which still beats anything I’ve found on npm.

Cloud Code & CLI: With beforeSave, afterDelete, and other model hooks, it was easy to validate or alter your data, or perform additional actions when something changed. The CLI made it easy to get started with CloudCode and lightning fast to deploy it, and even had an interactive dev mode that would deploy your changes every time a file was saved. Magical.

Cloud Modules: There were many integrations for sending e-mail, content moderation, and more. Nowhere near the scope you could get from the npm registry, but just enough to be intriguing and approachable to a young developer who’s new to APIs.

Push Notifications: Parse mobile SDKs saved device tokens automatically with relation to the current user, so you could easily target some or all of a user’s devices. Push channels made it easy for users to opt-in to wide-reaching notifications about specific topics of interest.

Design & Documentation: The Parse website and dashboard felt “Premium” and conveyed a certain sense of order and seriousness. The docs, while not perfect, did a very good job of getting your integration to a working state without being confusing.

The Bad

Multi-User Development

One project I did involved multiple developers working on one Parse codebase. I won’t get into too many details, but let it suffice to say that it was really annoying to do. Each user basically had to create their own Parse project, manually keep it in sync with the latest git HEAD, commit and push changes, and then one person had to log back into the “production” app, make sure they pulled the right changes from git, and deploy via the CLI. This invariably lead to multiple people manually pushing to production.

I’d love to know if anyone made this work well, but from what I can tell, Parse was designed with a single developer in mind.

Service Limits

Aside from documented service limits, like 600 req/s and 1000 objects per query, Parse had some really crippling undocumented limits. There are multiple sources where the community has compiled these limits. Some of the most heinous ones include:

  • 100 line log limit: Want to catch a rare error or log something in production? Better be quick! There’s no way to send these logs to a logging service like Papertrail.
  • Unknown server capacity: Want to do something simple like resize an image on the server? The SDK supports it, but it fails randomly due to limited resources.
  • Push notifications are not guaranteed to be instant: They can arrive after 30 minutes or an hour. Goodbye chat apps or anything time-sensitive.

Support Options

I was hoping the increase in resources from the Facebook acquisition would allow the team to tackle some rough edges that needed to be smoothed out, especially with support. That didn’t happen.

They eventually shut down the in-house support website in favor of telling people to submit to StackOverflow or the Facebook Bug Tracker, and they shut down their Enterprise tier which offered phone support and was the only guaranteed way to get in contact with a member of the Parse team.

Having no options to even pay for guaranteed support made Parse an immediate “no” for any serious decision maker in a startup.

I have to give a special shout-out to Héctor Ramos who seemingly single-handedly answered the flood of support questions and gave thoughtful answers for literally years.

The Ugly

Database Automation & ACLs

Parse abstracted away database management completely, giving you no control of indexes and instead relying on a completely automated system to create appropriate indexes based on query patterns. This made the product simple, but started to break down when large collections were queried with multiple patterns, causing tons of indexes to be automatically added.

The need for additional indexes was exacerbated by Parse’s multi-layered access control system, which added some combination of _wperm, _rperm and _acl keys to any custom object saved, and added _rperm to any queries for these objects. Because of MongoDB 2.x’s inability to use more than one index for sorted queries, every key you wanted to sort by needed a compound index with _rperm, not to mention the additional compound indexes required for sorting by multiple keys.

You can see the result of this below on my mLab cluster which contains the migrated Parse data for Falcross. The collection in question has 30 million objects with <10 small keys per object. These are the indexes required to support just a few simple query patterns without grinding the cluster to a halt:

Screen Shot 2017-01-29 at 10.11.45 PM

This 13.4GB collection has a whopping 22.2GB of indexes and the degraded write performance that comes with them. I need to point out that the 7 indexes above were created by me, by hand, after migrating my Parse data. I canceled my initial migration which was slated to take days with the unbelievable 13 indexes that the automated Parse system had created on this same simple collection. What a waste!

This unfortunate scenario is just as much a shortcoming of MongoDB as it is of Parse. I’ve since migrated the above collection to PostgreSQL and it occupies less than 10GB of space including the 4 indexes that it needs. Lesson learned: don’t use a document store for relational data.

I’ve gained the sense that database ops are complicated for a reason and that creating a system to automatically optimize a database is really, really hard. It’s a challenge that fits the scope of a whole startup, not a feature in a BaaS startup.

I do wonder if some company will come along and create some middle ground of abstraction between, say, Parse and Heroku Postgres, where a relational database is mostly abstracted away, but you can at least add/remove columns and manage the indexes yourself.

Local Datastore

This highly anticipated feature was Parse’s solution to store app data for offline use. It launched in December 2014 on iOS and looked amazing on the surface.

Unfortunately, upon implementing the LDS, I noticed almost immediately that pinning more than 1000 objects made any offline queries unbearably slow and pegged my device at 100% CPU for multiple seconds. I submitted a bug report and was told that this was “by design”. I was pissed, as you can see in my fiery response to the Parse team.

This taught me a painful lesson about the danger of using a proprietary platform and was a major turning point for me. My future plans for Falcross pretty much all revolved around being able to store things offline, and Parse’s solution wasn’t even close to being good enough for use in production. I saw no way to move forward in the timeframe that was necessary to keep supporting myself with my app income, so I went and joined my friend’s company.

As I later learned when Parse open-sourced their SDKs, the reason it was so bad was because it fetched the entire SQLite table for whichever entity you were querying and then filtered the objects in-memory in a proprietary way. What the actual what?

Even today, 2 years later, the feature looks to be still broken and I doubt it will ever get fixed.

Conclusion

Parse tried to do a lot of things, and it succeeded at many of them. It was the gold standard of the “mobile backend as a service” product category and shaped the direction for similar services, but ultimately failed to create a product that was solid enough to rely on for any serious project.

It’s clear that the team’s inexperience with databases led them to make some irreversibly bad engineering decisions, like basing the whole product on MongoDB, or designing the Local Datastore the way they did. Many of the thousands of questions on StackOverflow are about the limitations of the query system.

After the Facebook acquisition, I think the Parse team knew that their platform couldn’t be used at scale without serious overhauls, and so they dropped the Enterprise option and focused on the independent/hobbyist dev market. Unfortunately, there’s no money in that–who would pay for a platform that is only technically capable of supporting side projects that will either never grow to scale or will necessitate a full rewrite to grow to scale?

Thank you, Parse team

Thank you to Ilya and whole Parse team for being brave enough to break new ground. Your creation allowed me to achieve dreams I never thought possible before I knew about Parse. Your design decisions have influenced my engineering practice and I know that many of the brilliant ideas from the Parse platform will live on in other products.

And a special thanks to Facebook for being an unrelenting supporter of open source software and doing a graceful shutdown of the service with a year’s notice. I’m sure the people who continue to use the Parse platform on their own servers are beyond grateful for the opportunity to continue their projects that they’ve invested so many hours in.

What’s next

There are hosted alternatives to Parse out there, and even some companies who will host your Parse Server to provide a “Parse-like” experience. But since I don’t know a lot about either of those things, I’ll leave it to the reader to figure out what’s the best option for her.

As for me, my year serving as CTO of Onerent taught me a ton about wrangling my own stack, and that’s what I’ll be doing from now on. I love the freedom of building my backend however I like, and I feel so much more capable with RDBMS and node than I ever did with Parse.

That being said, I hope that something comes along that will give young developers the same feeling of empowerment that I felt when I found Parse. There is a special educational value in working with a highly abstracted backend. Whether there’s any money to be made there is a toss-up.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>