A while back we started getting alerts in the middle of the night on low disk space. A quick investigation led us to one of our ETL tasks. Every night the task was fired to eliminate duplicate dumps, and free up some space. This is a short story about how we found our silver bullet and solved the issue without adding a single byte of storage.
I often talk about making Django scale but what does it actually mean? It means getting consistent performance regardless of the amount of data. In this article we tackle The last nail in Django admin's scalability coffin - the paginator.
Common table expressions, also known as the WITH clause, are a very useful feature. They help break down big queries into smaller pieces which makes it easier to read and understand. But, when used incorrectly they can cause a significant performance hit.
Every team has a unique development style. Some teams implement localization and require translations. Some teams are more sensitive to database issues and require more careful handling of indexes and constraints. In this article we describe how we enforce our own development style using the Django check framework, the inspect and the ast modules from the Python standard library.
ORMs offer great utility for developers but abstracting access to the database has its costs. Developers who are willing to poke around the database and change some defaults often find that great improvements can be made.
Django Admin search fields are great, throw a bunch of fields in search_fields and Django will handle the rest. The problem with search field begins when there are too many of them. This is how we replaced Django search with text filters for specific fields, and made Django admin much faster.
A few weeks ago we encountered a major performance regression in one of our main admin pages. The page took more than 10 seconds to load (at best) and hit the query execution timeout at worst. When we investigated the issue, we found that the date hierarchy was the cause for most of the time spent loading the admin page. In the article we describe how we significantly improved the performance of Django Admin date hierarchy
The date hierarchy is a great feature but it comes at a price. On very large tables, the way date hierarchy is implemented can make an admin page nearly unusable. In this article we describe the limitations of the date hierarchy, and suggest a way to overcome them.
We had a large codebase and a lot of tests. Unfortunately, a lot of our tests were a relic from when we were using fixtures extensively. In this article, we describe a different approach that reduced the number of fixtures we maintain.
The days of desktop systems serving single users are long gone. Web applications nowadays are serving millions of users at the same time. With many users comes a wide range of new problems: concurrency problems. In this article we describe two approaches for managing concurrency in Django models.