Please note that this post primarily focuses on understanding the concept behind Generic Foreign Keys and does not delve into the debate of whether they are the best choice from a database design perspective or whether they should be used.

Foreign Keys

A foreign key is a critical element in a database table, often comprising one or more columns, whose values must correspond to the values in another table's column(s). FOREIGN KEY constraints play a crucial role in maintaining referential integrity, ensuring that if a value in one column (A) references a value in another column (B), then column B must exist.

However, what if we want to establish a reference to any table using a foreign key? Consider a scenario where we want to keep track of "likes" for various types of content, such as posts and courses.

class Post(models.Model):
    ...

class Like(models.Model):
    post = models.ForeignKey(
        Post, 
        on_delete=models.SET_NULL, 
        null=True, 
        blank=True
    )
    user = models.ForeignKey(
        User,
        on_delete=models.SET_NULL, 
        null=True, 
        blank=True
    )

Suppose we introduce a new model, "Course," and we want to allow users to like courses as well.

We face a choice:

• Create separate models to track likes for each type of content or

• Employ a single "Like" model with a Generic Foreign Key.

Generic Foreign Key

To understand how Generic Foreign Keys work, let's revisit SQL's standard foreign key concept. In a typical foreign key, one column references the primary key of a predefined table. We must devise a schema that accommodates this flexibility to make it generic and reference any table.

In a typical foreign key setup, a single column references the primary key of a specific, predefined table. For instance, consider the following SQL code:

CREATE TABLE posts (
    post_id SERIAL PRIMARY KEY,
    user_id INT NOT NULL,
    content TEXT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    -- Add any other columns related to a post here
);

FOREIGN KEY (user_id) REFERENCES users(user_id);

Here, the “user_id” column in the "posts" table references the “user_id” column in the "users" table. This enforces referential integrity and ensures that the values in the "user_id" column of "posts" correspond to existing values in the "users" table.

Now, let's consider a scenario where we want to create a reference to various tables, not just a specific one. SQL doesn't provide a built-in mechanism for this. To achieve this flexibility, we need to adjust our schema definition.

To make a foreign key generic and capable of referring to any table, we can create two columns:

1. A column that stores the primary key value without the constraints of a foreign key.

2. Another column indicates which table the reference pertains to.

In this way, we create a more versatile structure:

CREATE TABLE likes (
    like_id SERIAL PRIMARY KEY,
    user_id INT NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    content_id INT NOT NULL,
    content_type VARCHAR(255) NOT NULL,
    -- Add any other columns related to likes here
);

-- You can define foreign keys to link to the User table.
-- FOREIGN KEY (user_id) REFERENCES User(user_id);

In this new setup, the “content_id” column stores the primary key value, and the “content_type” column specifies the table to which it refers. This approach allows us to reference multiple tables dynamically based on the value in “content_type”.

By employing this schema modification, we can achieve the flexibility needed to create generic foreign keys, enabling us to reference various tables within our database.

INSERT INTO Like (user_id, content_id, content_type)
VALUES (1, 123, 'posts'); 
-- Assuming user with ID 1 liked post with ID 123

INSERT INTO Like (user_id, content_id, content_type)
VALUES (2, 456, 'courses'); 
-- Assuming user with ID 2 liked course with ID 456

INSERT INTO Like (user_id, content_id, content_type)
VALUES (3, 789, 'digital_goods'); 
-- Assuming user with ID 3 liked digital goods with ID 789

We can effectively implement generic foreign keys by querying the "content_type" key from the "likes" table and joining it with the corresponding table in separate queries.

ContentType Model

In Django, table names differ from model names, consisting of the Django app and model name combined. Django provides an inbuilt model for tracking defined models in your project, known as "ContentType."

Instead of manually storing this information, we reference the "ContentType" table when defining a Generic Foreign Key in a Django Model.

Django includes a contenttypes application that can track all of the models installed in your Django-powered project, providing a high-level, generic interface for working with your models.1

Here's how it looks in Django:

from django.contrib.contenttypes.fields import GenericForeignKey
from django.contrib.contenttypes.models import ContentType
from django.db import models


class Like(models.Model):
    user = models.ForeignKey(
        User,
        on_delete=models.SET_NULL, 
        null=True, 
        blank=True
    )
    content_type = models.ForeignKey(
        ContentType, 
        on_delete=models.CASCADE
    )
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey("content_type", "object_id")

    class Meta:
        indexes = [
            models.Index(fields=["content_type", "object_id"]),
        ]

Note the index on fields "content_type" and "object_id"; it optimizes fetching records for specific content types, such as Posts, Courses, and Digital Goods.

GenericForeignKey

There are three parts to setting up a GenericForeignKey:

1. Give your model a ForeignKey to ContentType. The usual name for this field is “content_type”.

2. Give your model a field that can store primary key values from the models you’ll be relating to. For most models, this means a PositiveIntegerField. The usual name for this field is “object_id”.

3. Give your model a GenericForeignKey, and pass it the names of the two fields described above. If these fields are named “content_type” and “object_id”, you can omit this – those are the default field names GenericForeignKey will look for.

Unlike for the ForeignKey, a database index is not automatically created on the GenericForeignKey, so it’s recommended that you use Meta.indexes to add your own multiple column index. This behavior may change in the future.2

Unlike ForeignKey, a database index is not automatically created on the GenericForeignKey, so adding your custom multiple-column index using Meta.indexes is advisable.

Working with the ORM

There are two approaches to working with GenericForeignKeys:

1. Passing "content_type" and "object_id" separately or

2. Using "content_object" directly.

Passing content_type and object_id Separately

Explicit Control

Using content_type and object_id separately provides more explicit control over the foreign key relationship. You can set these fields independently, allowing you to manipulate the like object's references as needed.

Performance

In some cases, especially when working with a large number of likes, directly setting content_type and object_id can be more efficient because it avoids the overhead of creating and managing a content object instance.

Complex Relationships

Passing them separately can be beneficial when dealing with complex relationships or situations where you must perform additional logic based on the content type and object ID.

Example

# Creating a like for a Post
post_content_type = ContentType.objects.get(
    app_label="cms",
    model="post"
)
user = User.objects.get(id=1)
like = Like(
    user=user,
    object_id=123,    # ID of the post
    content_type=post_content_type,
)
like.save()

# Creating a like for a Course
course_content_type = ContentType.objects.get(
    app_label="cms",
    model="course"
)
user = User.objects.get(id=1)
like = Like(
    user=user,
    object_id=456,    # ID of the course
    content_type=course_content_type,
)
like.save()

# Creating a like for a DigitalGood
dg_content_type = ContentType.objects.get(
    app_label="cms",
    model="digital_good"
)
user = User.objects.get(id=1)
like = Like(
    user=user,
    object_id=789,    # ID of the Digital Good
    content_type=dg_content_type,
)
like.save()

Using content_object Directly

Convenience

The content_object approach is more convenient and concise in many cases. It allows you to work with content objects directly without explicitly setting content_type and object_id.

Readability

Code using content_object tends to be more readable and self-explanatory, especially for developers less familiar with the underlying database schema.

Django's Design Philosophy

Django's ORM is designed to make everyday tasks straightforward, and using content_object aligns with this philosophy.

In most cases, using content_object directly is recommended because it simplifies your code and improves readability. However, suppose you have specific use cases that require fine-grained control over the foreign key relationship or are working with many records and need to optimize performance. In that case, you might choose to pass content_type and object_id separately.

Example

# Creating a like for a Post
post = Post.objects.get(id=123)
user = User.objects.get(id=1)
like = Like(user=user, content_object=post)
like.save()

# Creating a like for a Course
course = Course.objects.get(id=456)
user = User.objects.get(id=2)
like = Like(user=user, content_object=course)
like.save()

# Creating a like for a DigitalGood
digital_good = DigitalGood.objects.get(id=789)
user = User.objects.get(id=3)
like = Like(user=user, content_object=digital_good)
like.save()

Conclusion

In conclusion, Generic Foreign Keys in Django provide a powerful way to handle references to multiple tables within your database schema dynamically. While traditional foreign keys bind to specific tables, Generic Foreign Keys can reference various tables based on a content type indicator. This flexibility empowers developers to build more adaptable and extensible data models, enhancing the versatility of their Django projects. Whether you choose to pass “content_type” and “object_id” separately or use the convenient “content_object” Understanding and utilizing Generic Foreign Keys can significantly improve your data modelling capabilities in Django applications.

The Default Structure

project/
|-- app/
|   |-- migrations/
|   |   |-- __init__.py
|   |-- __init__.py
|   |-- admin.py
|   |-- apps.py
|   |-- models.py
|   |-- tests.py
|   |-- views.py
|-- project/
|   |-- __init__.py
|   |-- settings.py
|   |-- urls.py
|   |-- asgi.py
|   |-- wsgi.py
|-- manage.py
|-- requirements.txt

When we create an app in Django, a folder is made with the name of the app, and typically, it consists of a migrations folder and several files to configure the admin, models, tests, and views.

While it is an excellent structure already, the problem arises when the project grows. One of my projects had almost 80+ models. So, we can’t have a single Python file with thousands of lines. I mean, we can, but then it becomes a lot less readable.

So then, how do we do it? Making sure we have scope to grow it to a certain point where it is still easier to expand without affecting the readability of it.

The Solution

Strategy 1: Organizing Models and Views

Over time, the original problem revolved around the increasing size of our Django project's models and view files. Django doesn't strictly enforce placing all models inside the models.py file.

Instead, it offers the flexibility to structure our project how we see fit. We've devised a simple yet effective solution to address this issue: organizing models and views by placing them in separate files within dedicated directories. The updated project structure would resemble the following:

project/
|-- app/
|   |-- migrations/
|   |   |-- __init__.py
|   |-- __init__.py
|   |-- admin.py
|   |-- apps.py
|   |-- models/
|   |   |-- __init__.py
|   |   |-- model1.py
|   |   |-- model2.py
|   |   |-- ...
|   |-- views/
|   |   |-- __init__.py
|   |   |-- view1.py
|   |   |-- view2.py
|   |   |-- ...
|   |-- tests.py
|-- project/
|   |-- __init__.py
|   |-- settings.py
|   |-- urls.py
|   |-- asgi.py
|   |-- wsgi.py
|-- manage.py
|-- requirements.txt

By adopting this approach, we maintain a clean and modular project structure, ensuring that our models and views remain organized and readable as our project grows.

models Directory

The presence of an __init__.py file in the models’ directory serves several important purposes:

Namespace Organization

By declaring the models’ directory as a package with __init__.py, we establish an explicit namespace for our models. This allows us to organize and structure our models into multiple files while maintaining them within the same logical package. This becomes increasingly beneficial as our project grows and we introduce numerous models.

Import Convenience

The __init__.py file allows us to easily import our models from other parts of our app or project. For instance, we can import a model like this:

from app.models import MyModel

views Directory

Similarly, in the views directory, the __init__.py file plays a vital role:

Namespace Organization

It aids in organizing our views into separate files within the views package. Each view file can focus on specific views or functionality, enhancing the overall code maintainability.

Import Convenience

With the __init__.py file in place; we can effortlessly import our view functions or classes from the views directory into other parts of our code. This includes usage in our URL routing (urls.py) or different views.

By including these __init__.py files, we improve our Django project's organization, structure, and maintainability, making it easier to manage as it continues to evolve and expand.

Strategy 2: Taking It a Step Further

While Strategy 1 addresses our problem effectively, we can refine it further to enhance our project's structure and maintainability.

But before we delve into these improvements, let's take a moment to discuss the concept of apps in Django.

Creating Apps in Django

Before we proceed, it's essential to understand the technical definition of an app within a Django project, as per the official documentation1:

Applications include some combination of models, views, templates, template tags, static files, URLs, middleware, etc. They’re generally wired into projects with the INSTALLED_APPS setting and optionally with other mechanisms such as URLconfs, the MIDDLEWARE setting, or template inheritance.

Dividing Apps Based on Business Functions

When structuring my Django projects, dividing them based on the core business functions they serve is crucial. This approach enhances project organization and makes managing and maintaining over time easier.

For instance, let's take the example of a Content Management System (CMS) I built in Django. Here's how I typically divide it into multiple apps:

1. CMS App: I create models and views for the fundamental shared resources that define the CMS in this app. This includes entities like categories, tags, posts, and more.

2. IAM (Identity and Access Management) App: To handle user authentication, extend user models, and manage roles and permissions, I create a dedicated IAM app. This keeps authentication-related code separate and well-organized.

3. Notification App: For managing various types of notifications, whether they are in-app notifications, emails, browser notifications, or others, I established a distinct notification app. This centralized approach ensures efficient notification handling throughout the project.

By structuring the project this way, instead of randomly creating apps for every model, we align each app with a specific business function. This promotes clarity and maintainability and streamlines the development process, making it easier to scale and expand the project as business requirements evolve.

This strategy enhances the organization of the Django project and aligns development efforts with the underlying business objectives, ensuring a more cohesive and efficient project structure.

Creating Resources in the Django App

Let's delve deeper into what constitutes a resource in my context of a Django app:

What is a Resource?

I define a resource as a cohesive collection of components that work together to fulfil a specific functionality. These components typically include Models, Views or ViewSets, related Tests, and Serializers. I further split the app into resources to maintain an organised codebase, each representing a distinct functional unit.

Resource Structure

The project structure I typically follow looks like the following:

project/
|-- app/
|   |-- resource1/
|   |   |-- models.py
|   |   |-- views.py
|   |   |-- admins.py
|   |   |-- serializers.py
|   |   |-- filters.py
|   |   |-- tests/
|   |   |   |-- test_1.py
|   |   |   |-- test_2.py
|   |   |   |-- ...
|   |-- resource2/
|   |   |-- models.py
|   |   |-- views.py
|   |   |-- admins.py
|   |   |-- serializers.py
|   |   |-- filters.py
|   |   |-- tests/
|   |   |   |-- test_1.py
|   |   |   |-- test_2.py
|   |   |   |-- ...
|   |-- migrations/
|   |   |-- __init__.py
|   |   |-- ...
|   |-- urls.py
|-- project/
|   |-- __init__.py
|   |-- settings.py
|   |-- urls.py
|   |-- asgi.py
|   |-- wsgi.py
|-- manage.py
|-- requirements.txt

Model as a Resource

Regarding whether the name of a model can be considered a resource, it depends. A resource typically encompasses similar models that are closely related. If the models within the models.py file of a resource directory are closely related and serve a common purpose, then naming the resource after the primary model makes sense. However, if the models are not closely related, keeping them in separate resource directories is advisable.

This project structure has proven effective for me over time, but it's essential to remember that project organization can vary based on individual preferences and project requirements. Feel free to adapt and modify it to suit your needs and preferences.

Conclusion

In conclusion, the structure and organization of a Django project play a pivotal role in its maintainability and scalability. Throughout this discussion, we've explored strategies and best practices for creating a well-structured Django project and how to divide it into smaller, manageable units called resources.

By dividing our project into resources based on business functions, we achieve clarity and maintainability, making it easier to navigate and extend as our project evolves. Each resource encapsulates related models, views, tests, serializers, and filters, creating a modular and organized codebase.

It's important to note that while the project structure outlined here has proven effective, it's not a one-size-fits-all solution. Adaptations can be made to align with specific project requirements and personal preferences.

Ultimately, a well-organized Django project, with clear resource boundaries and thoughtful structuring, contributes significantly to project success. It promotes maintainability, teamwork, and adaptability, ensuring your Django application remains robust and adaptable as it grows and evolves.

I did not get any confirmations; hence I did not do the session today. But, we will do a session tomorrow from 11 AM.

Please let me know if you have any other questions. Below are the details to join the session.

Kuldeep Pisda is inviting you to a scheduled Zoom meeting.
Topic: Getting Started | Code with KD
Time: May 30, 2021, 11:00 AM Mumbai, Kolkata, New Delhi
Join Zoom Meeting
https://zoom.us/j/98279956004?pwd=NGJiUk02Y0wrVkxJdi91YW0yTDdTQT09
Meeting ID: 982 7995 6000
Passcode: P8Xr0q

Thanks and Regards,
Kuldeep Pisda

Topics I will be covering.

1. What is web development?

2. Different aspects of web development

3. Web Development for non-CS students

4. Getting started with web development

5. Roadmap for Web Development as a Career Option

Feel free to share any other topics/questions that we should cover. I am thinking of a 10-12 AM tomorrow, i.e. 29th May slot for the session. Feel free to suggest yours if you have any other suggestions.

I am a full stack developer and currently working at Goldcast Inc as a backend developer. I am starting a series on Web Development which will consist of weekly live sessions. I was hoping you could find me on LinkedIn.

Subscribe now

In the meantime, tell your friends!