7 practical design principles in my apps

Who is this post for?

Another set of design principles?

I will give you a run-down of what I will be writing about (so that you can see if it’s not wasting your time):

• how to properly handle data deletion in your app without breaking connections
• how to properly save data in your app to avoid messy intermediate states
• where to actually store data (memory, configuration properties, database, cloud)
• how many ‘player’ data models do you need and why?
• handling interruptions, players exiting game in the middle
• passing data between screens / levels
• I’ll keep adding more topics over time.

Before we start, one more disclaimer:

These design patterns are partially taken from various different official programming patterns but are mostly based on experience and what worked for me. These are heuristics, not laws; adapt them to your domain and constraints. I’m sure there is room for improvement which I am always open to!

Data deletion

If you work in IT, especially in software development when building a new system from scratch, maybe you had a case when your manager asked you something like “Hey Jarek, we can now enter and edit data, it gets displayed perfectly, all dashboards are running… can you implement a delete functionality now? Should be easy, no?“. And then the manager gets surprised with an estimate of many sprints / weeks. Why is deleting data such a cursed subject?

In most real systems, clean deletion of interconnected data is hard and error-prone….

This is my statement taken from my experience. No matter how big or small the system, at some point when you delete data that is inter-connected to other data will cause some issues. The problem is, it is also almost impossible to avoid not having data deletion. So now, what to do to make it as safe as possible? I compiled a list of rules that I follow, feel free to get inspired but don’t feel forced to follow them to the letter.

• Avoid immediately deleting data, no matter how small – for anything beyond trivial caches or logs. I always employ the “is_archived” approach in my database tables. Whenever the user wants to delete something, I first mark it as “is_archived = true” and then exclude it from all UI views. Why? First, the user sometimes makes mistakes and might want to revert data, second – I want to be extra careful to perform the real deletion somewhere in the backend at some point.
• Soft delete has a cost though – it can be tempting to just delete the data immediately, because doing it in the “soft” way adds a lot of effort, complicates the queries, has some performance implications because there’s more data to go through, indexing and purging can become a hassle… But in my opinion, it is almost always worth it, especially for critical data.
• Set up cleanup mechanisms of archived data (and log them well!) – having your data marked as archived is one thing, but at some point it would be good to actually get rid of it. For that, prepare special classes or entire modules that handle only data cleanup. First of all, you will have it centralized in one place so it’s easier to manage, second you will be fully in control of the deletion process. Make sure to log every start and finish as well as all errors or deviations from happy paths.
  • A side hint for enterprise-grade systems – log every single deletion of any kind of data like crazy. Especially who did it, when and if possible why. I cannot count how many times I had to investigate at work “who deleted my data, surely it was not me!” (usually it was that person…)
• Always delete in transactions – in case this is not obvious, imagine you delete a Player from your app. The player has tied game history to him, statistics, friends, notifications and maybe 20 other things. Now imagine this process fails somewhere halfway… In distributed systems it might be a bit more challenging to achieve transactional deletion, but there are ways to achieve these boundaries / have compensating actions. The rule is simple, either everything goes or nothing goes.
• Be transparent with the user what it means to ‘delete’ – I always put a disclaimer / popup that when deleting data it will still be available and recoverable within some period of time. Especially if the user wants to delete some essential data, maybe for deleting small trash it’s not that critical. Still, I want the user to know that “delete” is actually a process and not just something you do willy-nilly.
• Some data just shouldn’t be deleted – in some cases it’s just better to keep data persisted… if your app is not enterprise-grade / customer facing, maybe it’s better to keep the data marked as archived forever than face data corruption issues.

Saving data

Let’s look at the list of suggestions and patterns that helped me handle the data persistence layer of my apps and games.

• Avoid persisting every intermediate step of wizard-like flows – build the full aggregate in memory and persist once at the boundary. In your code, try to have your temporary objects that you manipulate in memory. It is usually a bad practice to have a process (for example when creating a game room) and save in the database every step along the way. So for example, in my Game Room application when creating a game room, there are multiple steps the user has to go through – select game, configure it, add players. Every game room, session and everything is saved in the database but only as the very last step. Until then, I have a ‘temporaryGameRoom’ object containing everything the user has configured. This centralizes everything in a single easy-to-manage object and allows to do all operations in transactional way.
• Alternatively, use draft mode – if your creation process takes time and the user is not able to finish it in less than a minute, then a middle solution is to have a place to save the draft before actually committing it to the main database tables. This can be in memory, configuration files or even separate database tables. As long as you are able to purge this information later easily, have it restored for the user and publish it directly to main data storage.
• Don’t forget to clean your drafts though – so that you don’t end up with so-called “eternal zombie drafts”. I achieve this by using some background jobs that get triggered every day (for example) and whatever draft data I can immediately purge, I try to do so… but there’s always something that will be left behind (because for example, the user threw his phone in the toilet in the middle of a wizard-flow).
• Data validation and sanitization – don’t forget to trim data you save to the database to avoid unnecessary spaces in the beginning or the end. Also, try to define unacceptable symbols that might mess up anything in your application. Test your app with data entered in different alphabets (Japanese, Chinese, Arabic, Cyrillic etc.). Finally, if you expect a number – set the text field to expect a number, if you expect an email – set it to an email type.
• Try to avoid surprises to the user – most likely you will have a lot of data that you see as essential for basic functionalities of your app, in my case a UserAccount entity goes together with a PlayerUser in my GameRoom app. But I always try to be transparent saying that if you create an account in my app, you will get this and that, and I will be creating this and that in addition. Creating stuff under the hood and then having the user be surprised can sometimes lead to bad user experience.
• Centralize your DB/file operations – whether you call it Repository or DAO, the goal is to have one abstraction that owns persistence logic. Even if you use some dedicated libraries you will most likely have to write some code to handle all the CRUD operations. It’s better to have that centralized in one place rather than allow any class to just directly access your DB objects or run SQL queries wherever you please.
• Define early your loading screens – this sounds minor but to me it was actually very critical and helpful. At some point you will want to easily show a full screen loading screen for whatever reason with whatever label fits the current context. Prepare that as soon as possible and make it reusable. There’s nothing worse than clicking a button and then either having the screen freeze or nothing happens for few seconds until some lengthy network operation is done.
• Define early your logging mechanisms and crashlytics – a good logging system will save you a lot of hassle in the future. There’s probably dozens of good logging frameworks that you can reuse, I don’t recommend writing anything from scratch. Just make sure the framework saves the logs on device and is smart enough to self-clean the logs. Then, once your app becomes live, having any cloud system to monitor crashes and exceptions (like Crashlytics from Firebase) is even more important. You can test your app like crazy but users will find bugs you would never dream of. I usually put logs in:
  • All exception handlings (catch blocks)
  • All exceptional situations (like, something that was supposed to be not null is null, I write it as a warning)
  • All DB operations in general, especially in the centralized class handling my CRUDs
  • Meaningful stage transitions (e.g. my game room transitions from lobby to in progress state)

Where to store data

• Memory vs local DB/file vs online DB – when developing apps most likely these will be your three main persistence layers. For simple apps maybe even two or even one.
• Memory for short-lived data that doesn’t need to survive phone being thrown in the toilet – my general rule is that in memory goes the temporary states of objects, anything that is short-lived and most importantly – if the user decides to throw his phone in the toilet or force closes the app – when he opens it again, the missing memory objects won’t corrupt the entire app.
• Local DB/file to avoid data loss – for that specific case, local DB or file is more important. If you have some data that has to survive the toilet test or more than a few minutes, it’s better to have it saved in the local DB or file. So that when the user restarts the app, he will be able to reload the data into the memory.
• Online DB for collaboration and cross-device backup – you know how a lot of apps offer premium with a “sync to cloud” benefit? Yeah, that’s probably a fancy way of saying that your data will be saved in some cloud like Firestore or something else. It’s usually a premium benefit since cloud databases always have some cost involved, but the main use case is really to ensure the user doesn’t lose his data if he decides to buy a new phone. And the other obvious use case is being able to share stuff between different users, you won’t be able to avoid some central point where you have to store that kind of data.
• Should the local DB be replicated in online DB? – in my opinion that’s a bad practice mostly because of economical reasons. For indie apps with cost constraints, blindly mirroring the entire local DB to the cloud is often unnecessary and expensive; selectively sync only what you need for collaboration and backup. As mentioned above, cloud storage / reads / writes always come with a cost. So maybe, it doesn’t make sense to backup your entire local DB in the cloud DB if you are not planning to really use it? Furthermore, if your Player local SQL table contains 50 columns, but you only want to share displayName and email with the world, then store only displayName and email in the cloud DB. The less the, better in this case.
• Keys and unique identifiers – first of all, my rule is that each data source owns its own unique IDs to identify the entries. So my local SQL has an id column that is completely different than id column of my firestore or firebase realtimedb ids. Why? Because I treat these IDs as local, private identifiers of data. If you need something that binds all the data together (e.g. you have Player SQL table and Player collection in Firestore), then decide on your own custom public unique ID – in this case most likely email, but it can be anything. In my case, for players and users, a public ID is the Auth0 ID that is generated by Auth0 system I use for authentication. Since every player who wants to play my online games needs to have Auth0 account, I use the special public ID that Auth0 generates (and again, this is not the actual DB id, it’s a special public ID meant for sharing across apps).
• UUIDs – in my databases, for my private IDs I usually go with UUIDs. This is probably highly debatable; some prefer simple integers, others composite keys… I just go with UUID because it’s easy to generate and it’s (almost) always unique.

How many player data models you need?

This topic is quite important one and touches a lot of official design patterns when it comes to data architecture of your apps. As usual, there are two ways to approach things: quick and dirty or abstract and complex. And in my experience, for this particular case, the abstract and complex is the way to go, unfortunately. It can be really tempting to have only one Player data model and then use it in all contexts and all different storage places (local SQL, cloud)… but in the long run, this will create so many problems and most likely some blockers. So, let’s take a look at the bullet points:

• One domain data model and one per repository type or bigger context – this would be best visualized by an example. I have a simple Player data model. When I started I thought I wouldn’t need much more, but now I have the following:
  • PlayerUser – top-level player to which statistics are tied
  • PlayerGameRoom – a player instance created from PlayerUser (or from scratch) that is relevant only to the game room lobby
  • PlayerGameSession – a player instance created from PlayerGameRoom when a specific game session starts (actual game). Because 1 game room can have multiple game sessions and each game session has its own statistics tied to the players.
  • And now each of the above has duplication for my SQL repository (because the columns in the DB or not 1-1 the same as my domain fields) and for my Firestore (another set of fields that need to be mapped to a JSON model).
  • There are different auto mappers that can be used to map between the models (like MapStruct), but in my case, since it’s quite simple, I just do it manually (debatable decision most likely, but I find these auto mappers overcomplicated and cumbersome). In larger teams, I used these mappers a lot and of course they are not all that bad, they just add a bit of extra ceremony to the whole process.
  • This creates a total of 9 Player-related classes! For smaller apps, maybe it’s an overkill, but in my experience, I prefer to invest in this overkill from the start rather than have a headache in the future.
• What happens if I wouldn’t do it like this and had only 1 data class?
  • I would have to handle all the mappings in one place which already sounds dirty because one class would handle domain logic and 2 persistence layers.
  • I wouldn’t be able to say that this specific Player with this specific configuration at the given time was part of that game room and that game session – that was already a blocker for me, because this means if I edited / deleted the PlayerUser, then it could mess up the entire game history.
  • In case of game rooms that have multiple game sessions, if one player left the game room, it would break the entire game room. I would have to use some flags or other tricks to indicate the player is absent.
• In general, I feel that in these cases it does make sense to overcomplicate the data model:
  • You need to keep historical data that is up to date in that given snapshot of time.
  • You want to track statistics and tie other data to your data model and you don’t want a deletion or modification to mess up the statistics.
  • You start to create a large amount of flags or enums to indicate a state of object – for me, that’s a clear indication that it’s better to have a dedicated data model class instead of having for example “PlayerState = [GameRoom, GameSession, Global]” etc. You will surely get lost in the business logic tied to all these types.
• Do I really need to duplicate my Player data model 9 times with so many duplicated data? – And that’s the best part of object-oriented programming where abstractions and inheritance comes to the rescue. You can easily define a PlayerAbstract or PlayerBase class that holds some common fields across all data models (like id, displayName, createdDate, avatarUrl, etc.). And then every next class just extends this abstraction with their own fields.

I tried the quick-and-dirty approach with simplified data models – it has always bitten me in the ass…

Interruptions and edge cases

Here I can give some tips on how to approach these topics. They’re all very specific, and there aren’t many golden rules, but let’s give it a try with some examples:

• Clean code, separation of concerns, proper data storage places – this is most likely the only golden rule to make your life as easy as possible when it comes to edge cases. In one of the previous sections I described how the Player data model looks like in my app and one of the triggers that made me go for this complex and clean approach was when I was testing the app and decided to see what happens when I delete a player… I caught that quite early on of course, it was pretty obvious.
• Interruptible data should rely on memory as little as possible – in the earlier section I also mentioned about locations where data is saved – memory, local DB/file and cloud. If you have a case that a user can leave in the middle of your session and it turns out most of the session data was saved in memory… well then you are screwed. The general rule is that memory objects should always be re-creatable from the local DB or cloud. So when the player rejoins the game, he should just synchronize the current state of game and then be able to continue immediately. It sounds easier said than done, and before you get it right you might need to rewrite your data models or service classes a few times…
• If you can, be strict and block edge cases – sometimes the easiest way to go is to simply block an edge case from happening. In one of my apps, if the user wanted to suddenly leave an ongoing session, I just plainly say “You are about to leave the session, you won’t be able to come here and all data will be lost”. I just realized that handling all the recreation of data and everything was such a complex feature and prone to so many errors that I just decided not to do it. Let’s hope it doesn’t become a source of negative feedback about the app, but so far I think my users are very understanding of such decisions.

Passing data around

When it comes to passing data around, there are probably dozens of different patterns. I would focus on the context of mobile apps since that’s my main area of development. Since I work mostly with Flutter, you’ll see the words Screen and Widget a lot. The latter not being a classical widget you would put in your Android home screen, but more like a reusable UI component.

• Central Singleton point – having a central singleton class that is alive throughout the lifecycle of the app session makes things quite easy. This is just one option and while it sounds like an easy and tempting solution, there are some downsides. First of all, the singleton approach only make sense if there can always be one session of such an object a a time. Additionally, you put a lot of dependency on that single class, meaning that all screens and widgets will rely on this single class object having the proper data inside. Furthermore, most likely your screens and widgets won’t live independently without that central class, which kills reusability. Lastly, it will make your unit tests much harder to be executed because you need to achieve the right state of that singleton class for each of your test case. Therefore, there are cases where having the central singleton makes sense, but it should not be overused as the primary solution for everything.
• Moving data from screen to screen / widget – this approach creates a clear chain. For example, a Player screen might show a list of players and if you select one of them, you would proceed to PlayerDetails screen. You could again use the central singleton object to store the “selectedPlayer”, but then you overly rely on singletons again. Another viable approach is simply to pass the Player object from one screen to the other. And in this particular context, it actually makes a bit more sense. Imagine you can edit your player from the Player screen but also directly from the Game Lobby screen which might be using completely different helper classes. Therefore, if you just have the PlayerDetails screen accept the Player object, this means this screen can be called from any place in your app and doesn’t rely on a single singleton class that might not be available in some other context.
• General rule – if there is possibility for reusability – go for passing data directly to the widget/screen. If multiple screens are in a kind of a flow or single session – go for the singleton class.
• Flutter side-note on state management – in Flutter you will find already some existing libraries to manage state management of objects, such as Riverpod, BLoC, Provider. Personally, I use Riverpod especially when listening to changes from firestore / Firebase RealtimeDB – it works instantly whenever there is a change. And another side note – if you use Firebase with your mobile app, use Firebase RealtimeDB for anything that has to be synchronized immediately after it changed. If you can survive a bit of delay, use Firestore.

Final thoughts

Thank you for going through this blog post, I hope my experiences brought you some food for thought that might be useful in your domain and context. As usual, drop a comment in case you have an opinion or want to point out any mistake I might have in my thinking – as everyone, I learn as I go and perhaps in 5 years what I wrote here won’t make any more sense… but that’s life in the IT business. Thank you!