- We are currently updating our API using a completely new backend. As part of this work, and seeking for creating a faster API, a commit was introduced with the goal of improving some technical aspects of data fetching and concurrency handling. Although the change accomplished the desired effect, the current in-memory cache data was not updated to reflect the changes performed.
- This led to unavailability of parts of the application retrieving information using the method introduced by the change.
What we’ve done
- After identifying the problem, the cache was purged to allow filling it with the new data structure
What’s the impact
- The change was deployed to production at 10:28 am of April 23. 12 minutes was the time between the change was deployed to production and the cache was purged.
- During that time, 13 users were affected by the unavailability of retrieving parts of the projects information.
What we are doing to help
- Those kind of changes are rare. However, we identified the pattern of code changes that may affect the in-memory cache structure and defined means to purge the cache when this happens.