IT4each

Drop Shipping Online Store on Django (Part 6)

6. Adding a Scraping (Parsing) Module and Autofilling the Database Directly From Another Website!

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

How to get information from another site?

Of course, the most convenient (and, by the way, more reliable) way will be the interaction of our site with the provider site via API. Indeed, this requires

on the supplier site such an opportunity was organized (access to it via API),
and that the administration of the supplier site give us a login and password for this access.

The Django framework allows us to organize such interaction with your own means, without resorting to installing third-party packages. However, the Django REST ftamework (DRF) package will do the job best.

Nevertheless, in our case, we will use another method - reading and extracting the necessary information directly from the HTML page. This action is called scraping (parsing) of the site.

Two popular Python libraries will be used for this purpose: beautifulsoup4 and requests. You can install them using two terminal commands:

pip install beautifulsoup4
pip install requests

Web page structure

Typically, data on a product page is grouped into blocks. Inside the blocks, the same type of data is under the same selectors (see figure):

If we download and parse an HTML page with a list of products, we can get a structured list of data. Specifically for our case, for each data block, we need to get the following dictionary:

{
    'name': 'Труба профильная 40х20 2 мм 3м', 
    'image_url': 'https://my-website.com/30C39890-D527-427E-B573-504969456BF5.jpg', 
    'price': Decimal('493.00'), 
    'unit': 'за шт', 
    'code': '38140012'
 }

Action plan

Create scraping.py module in shop app
In this module, create a scraping() function that can:

Get page HTML code (by package request)
Process the resulting HTML code (by package beautifulsoup4)
Save result in database

Test the function scraping() “manually”
Add a button to start scraping on the site page

The plan is ready - let's start its implementation!

Create a scraping (parsing) module and get the HTML code using the requests package

Obviously, the script responsible for reading information from another site should be placed in a separate shop application module: shop/scraping.py. The scraping() function will be responsible for sending a request to URL_SCRAPING, reading data and writing this data to the Product table of the project database.

First of all, we need to get the HTML code of the product data page for further processing. This task will be assigned to the requests module:

import requests

def scraping():
    URL_SCRAPING = 'https://www.some-site.com'
    resp = requests.get(URL_SCRAPING, timeout=10.0)
    if resp.status_code != 200:
        raise Exception('HTTP error access!')

    data_list = []
    html = resp.text

It makes sense to immediately see what you got. To do this, let's add code that will count the number of characters in the html object and at the same time print this object itself:

html = resp.text
    print(f'HTML text consists of {len(html)} symbols')
    print(html)


if __name__ == '__main__':
    scraping()

The shop/scraaping.py module does not require any Django settings (at least not yet), so you can run it like a regular Python script:

HTML text consists of 435395 symbols
<!DOCTYPE html>
<html lang="ru">
  <head>
    <link rel="shortcut icon" type="image/x-icon" href="/bitrix/templates/elektro_light/favicon_new.ico"/>
    <meta name="robots" content="index, follow">
<meta name="keywords" content="Профильные трубы, уголки">
<meta name="description" content="Цены на профильные трубы, уголки от  руб. Описание. Характеристики. Отзывы. Скидки на  профильные трубы, уголки.">
    <meta name="viewport" content="width=device-width, initial-scale=1.0 user-scalable=no"/>
    <meta name="msapplication-TileColor" content="#ffffff">

As you can see, the result really looks like an HTML page.

The first part of the task is solved - access to the site data is obtained, and those 435,395 characters that are displayed on the screen contain all the information we need. All we now need is to simply extract this information and store the result in the database.

Processing the resulting HTML code with the BeautifulSoup package

Further processing will be most conveniently carried out using the beautifulsoup4 module. To do this, we first need to create a soup object, which is a nested data structure of an HTML document:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'html.parser')

More information on how to get started with this package can be found on the man page: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#quick-start

You can also read more about the beautifulsoup4 CSS selectors here: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors Further on the supplier's page, we will be most interested in the product block - layout of repeating product cards with a similar data structure. You can get a list of elements of the same type from the soup object using the select() method, where the CSS selector of this block is specified as an argument. In our case it will be class=”catalog-item-card”:

blocks = soup.select('.catalog-item-card ')

In the loop, we can access each block and at the same time see what is inside the block object. This is how the modified code will look like:

html = resp.text

    soup = BeautifulSoup(html, 'html.parser')
    blocks = soup.select('.catalog-item-card ')

    for block in blocks:
        print(f'HTML text consists of {len(block.text)} symbols')
        print(50 * '=')
        print(block.text)
        break

And this is how the printed block.text object will look like:

HTML text consists of 382 symbols
==================================================
<div class="catalog-item-card" itemprop="itemListElement" itemscope="" itemtype="http://schema.org/Product">
<div class="catalog-item-info">
<div class="item-all-title">
<a class="item-title" href="/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/" itemprop="url" title="Труба профильная 40х20 2 мм 3м">
<span itemprop="name">Труба профильная 40х20 2 мм 3м</span>
</a>

As you can see, the number of characters in the block has been reduced to 382. Which greatly simplifies our task.

We can parse these blocks into elements of interest to us using the soup.select_one() method, which, unlike the select() method, does not select all elements of the page, that satisfies the condition (method argument), but only the first matched element. It is also important to remember that the text obtained with the soup.select_one() object can be extracted using the text method. Thus, applying this method with certain arguments, we fill almost the entire data dictionary, with the exception of the code field:

soup = BeautifulSoup(html, 'html.parser')
    blocks = soup.select('.catalog-item-card ')

    for block in blocks:
        """{
        'name': 'Труба профильная 40х20 2 мм 3м', 
        'image_url': 'https://my-website.com/30C39890-D527-427E-B573-504969456BF5.jpg', 
        'price': Decimal('493.00'), 
        'unit': 'за шт', 
        'code': '38140012'
        }
        """
        data = {}
        name = block.select_one('.item-title[title]').get_text().strip()
        data['name'] = name

        image_url = URL_SCRAPING_DOMAIN + block.select_one('img')['src']
        data['image_url'] = image_url

        price_raw = block.select_one('.item-price ').text
        # '\r\n \t\t\t\t\t\t\t\t\t\t\t\t\t\t493.00\t\t\t\t\t\t\t\t\t\t\t\t  руб. '
        price = re.findall(r'\S\d+\.\d+\S', price_raw)[0]
        price = Decimal(price)
        data['price'] = price   # 493.00

        unit = block.select_one('.unit ').text.strip()
        # '\r\n \t\t\t\t\t\t\t\t\t\t\t\t\t\tза шт\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'
        data['unit'] = unit  # 'за шт'

Generating an additional link to go to the detail page and getting the product code

Upon closer examination of the product block, it turned out that some of the information on the product is located on another page of the supplier's website - on the product detail page. The link itself to go to this page is in the block block.

Therefore, we will have to repeat here the same algorithm that we used a few steps ago to get data for the block block:

Generate link to detail page
Follow this link and read with requests.get() the new HTML code of this page already - the detail page
Save the received data in a new object Beautiful Soup
Extract the code number using the same method soup.select_one()

# find and open detail page
        url_detail = block.select_one('.item-title')
        # <a class="item-title" href="/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/" itemprop="url" title="Труба профильная 40х20 2 мм 3м">

        url_detail = url_detail['href']
        # '/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/'

        url_detail = URL_SCRAPING_DOMAIN + url_detail

        html_detail = requests.get(url_detail).text
        soup = BeautifulSoup(html_detail, 'html.parser')
        code_block = soup.select_one('.catalog-detail-property')
        code = code_block.select_one('b').text
        data['code'] = code

        data_list.append(data)

        print(data)

If we do everything right, we will end up with a list of dictionaries with data for each block.

Adding error handling

The success of site scraping depends on some parameters and circumstances. And most of them do not depend on our Django code, namely:

Availability (or inaccessibility of the provider site)
Changing page layout
Internet connection problems
and so on…

The success of site scraping depends on some parameters and circumstances. And most of them do not depend on our Django code, namely:

Availability (or inaccessibility of the provider site)
Changing page layout
Internet connection problems
and so on…

class ScrapingError(Exception):
    pass


class ScrapingTimeoutError(ScrapingError):
    pass


class ScrapingHTTPError(ScrapingError):
    pass


class ScrapingOtherError(ScrapingError):
    pass

And then we make changes to the code:

try:
        resp = requests.get(URL_SCRAPING, timeout=10.0)
    except requests.exceptions.Timeout:
        raise ScrapingTimeoutError("request timed out")
    except Exception as e:
        raise ScrapingOtherError(f'{e}')

    if resp.status_code != 200:
        raise ScrapingHTTPError(f"HTTP {resp.status_code}: {resp.text}")

Saving the received data in the database

As you can see, the product is added only if it is not already in the database. The search is performed by the product code number (field value code ).

Despite the fact that in the scraping.py function itself, the data is already written to the database, we still return the data_list list. Just in case).

However, if we now try to reproduce this script, we will get an error:

"/home/su/Projects/django-apps/Projects/drop-ship-store/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 67, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Process finished with exit code 1

The thing is that now the script accesses the database, which means that you need to get the Django settings. You can run this code to check in management/commands (more on this can be found here: https://docs.djangoproject.com/en/4.0/howto/custom-management-commands/) But we will do otherwise: we will immediately add the launch page and check the operation of the scraping() function already there.

Transferring scraping control to the site page

The algorithm for adding a new page remains the same:

Comes up with a url that will call it (shop/fill-database/)
Add urls.py configurator to shop
Set urls.py to link url and view (path('fill-database/', views.fill_database, name='fill_database'),
Move (copy) the file from the template to the project
Create a view in the module shop/views.py
Checking the result!

If, after the successful completion of all these points, we go to the admin panel, we will see that, after running the script, the Product table is filled with new values:

Now everything is ready for the last step: adding the pages of the online store directly and the code that will manage them. But we will deal with this in the next seventh and last lesson.

You can learn more about all the details of this stage from this video (RU voice):

To the next stage of the project

Drop Shipping Online Store on Django (Part 5)

5. Creation of Database Functionality Using TDD (Test Driven Development)

In this course, we will go through all the stages of creating a new project together:

Why do programmers write tests?

Most programmers don't like to write tests.

It is understandable - few people like to do double work: first write and debug code, and then spend the same amount of time (if not more!) on writing tests.

Well, if the project will be supported further. Then the time spent on writing tests is guaranteed to pay off: it will be possible to make sure that creating a new one did not break the old one in seconds.

However, if the project is closed, then the cost of tests will be added to the loss of time for writing the main code.

So writing tests is a bit like investing - in principle, a good thing, but risky. Therefore, to write tests or not to write - everyone chooses for himself, depending on each specific situation.

Nevertheless, there is an approach in programming that allows you, if not to win, then at least almost not to lose when writing tests. And this approach is called "Test Driven Development" (TDD) ("Development through testing"). The idea is very simple: we FIRST write tests, and only then we write the code itself, which must pass these tests.

And what is the advantage here? And the plus is that in this case, the time spent on checking the code is completely eliminated. After all, even the very first run of a freshly written code will be done "not by hand", but by tests!

And an additional bonus of this approach is that the check itself becomes more systematic, and therefore more reliable. Because what and how to test is thought out in advance, and not done impromptu. Which drastically reduces the chances of missing something important.

Actually, this is why Django has a well-developed code testing toolkit. And that is why already in the first training example on the website of this framework, a whole lesson out of eight is devoted to the consideration of an example of the implementation of this principle: "Development through testing" or "Test Driven Development" (TDD).

In our training mini-course, we will consider the test-driven development approach when creating a database, since the main computational load or the main business logic will be located in the shop/models.py module, which is responsible for the interaction between database tables data.

So, our tasks at this stage:

Formulate the basic requirements for the structure of the database
Create a database design (draw tables and relationships between them)
Formulate requirements for the business logic of the database
Implement these requirements into test code
Write code that passes these tests

Requirements for the structure of the database

To simplify the task (after all, this is a mini-course!) we will stipulate three assumptions:

Our products are flat (no categorization)
We have dropshipping, and therefore there is no warehouse and there can be no stock leftovers (an infinite stock of goods for each item is assumed)
Our cart is an order with the status cart. Just after clicking the “Proceed to payment” button, this status changes to waiting_for_payment

Creating a database structure

This block diagram satisfies all the above requirements:

Therefore, in the shop/models.py module, we create the following structure:

class Product(models.Model):
    name = models.CharField(max_length=255, verbose_name='product_name')
    code = models.CharField(max_length=255, verbose_name='product_code')
    price = models.DecimalField(max_digits=20, decimal_places=2)
    unit = models.CharField(max_length=255, blank=True, null=True)
    image_url = models.URLField(blank=True, null=True)
    note = models.TextField(blank=True, null=True)

class Payment(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    amount = models.DecimalField(max_digits=20, decimal_places=2, blank=True, null=True)
    time = models.DateTimeField(auto_now_add=True)
    comment = models.TextField(blank=True, null=True)

class Order(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    # items = models.ManyToManyField(OrderItem, related_name='orders')
    status = models.CharField(max_length=32, choices=STATUS_CHOICES, default=STATUS_CART)
    amount = models.DecimalField(max_digits=20, decimal_places=2, blank=True, null=True)
    creation_time = models.DateTimeField(auto_now_add=True)
    payment = models.ForeignKey(Payment, on_delete=models.PROTECT, blank=True, null=True)
    comment = models.TextField(blank=True, null=True)

class OrderItem(models.Model):
    order = models.ForeignKey(Order, on_delete=models.CASCADE)
    product = models.ForeignKey(Product, on_delete=models.PROTECT)
    quantity = models.PositiveIntegerField(default=1)
    price = models.DecimalField(max_digits=20, decimal_places=2)
    discount = models.DecimalField(max_digits=20, decimal_places=2, default=0)

Чтобы объекты этих классов понятно отображались при отладке и в админке, полезно будет добавить в каждый класс методы __str__, а также упорядочить все записи в таблицах по порядку, например:

class Meta:
        ordering = ['pk']

    def __str__(self):
        return f'{self.user} --- {self.time.ctime()} --- {self.amount}'

The final step in creating a database is to create migrations and apply them:

python manage.py makemigrations
python manage.py migrate

And the final touch is to add control and display of the database to the admin panel:

from django.contrib import admin

from shop.models import Product, Payment, OrderItem, Order

admin.site.register(Product)
admin.site.register(Payment)
admin.site.register(OrderItem)
admin.site.register(Order)

Database business logic requirements

When a user selects a product, the product should be added in the selected quantity to the Cart, which should be created automatically (unless, of course, it has not already been created by this moment)

A cart that has not changed to an Order status within 7 days should be automatically deleted on the first call to the get_cart(user) method

At each new addition (deletion, change) of the quantity (or price) of goods, the total amount of the order should be automatically recalculated

After completing the set/change of the Cart and proceeding to payment, the Cart should change its status (if it is not empty!) and become an Order awaiting payment (waiting_for_payment)

The get_unpaid_orders(user) method is required, which will allow you to get the total amount of unpaid orders (status=waiting_for_payment) for the specified user

The get_balance(user) method is required, which will allow you to get the balance on the account of the specified user

Changing the status to waiting_for_payment automatically starts checking the balance of the current user. If the balance amount >= the amount of the order, then the Order changes its status to paid. This simultaneously creates a payment equal to (minus) the amount of the order (which immediately after payment reduces the balance of the client's account by the amount of the order)

Making a payment automatically triggers a mechanism to check all outstanding orders, starting with the oldest one. If the payment amount entered is sufficient to pay for several orders awaiting payment, then all these orders change their status to paid. At the same time, payments equal to (minus) the amount of each order are formed (which immediately after payment reduces the balance of the client's account by the amount of orders)

As you can see, the list of checks is quite impressive. Even a single manual execution of all the listed tests is "very hard work", which can take 15 minutes, no less.

But we know well from our own experience that a rare code starts working without errors the first time. Consequently, manual testing would take hours of our time. And each new refactoring of the code is again hours for verification. Not to mention the fact that during manual testing, you can forget to check something, or forget to delete test values from the database.

In general, as we will see for ourselves further, writing tests will take less time from the first time, which means that it will pay off its costs already on the first test run!

Write tests (transfer the requirements for the database to the test code)

When writing tests, keep the following in mind:

Application-level tests already have a ready-made module tests.py
Django tests are based on unittests
By default, before running EACH test, its own test database is created, which is immediately deleted after the test is completed.
It follows from this that a “just created” test database simply cannot physically have test values. This means that these database test values must be created each time with each new test.

It is most convenient to use fixtures for these purposes - special files (usually in json format) containing values for the test database. The “just created” test database automatically loads these values into itself before each test.
If you are not yet familiar with the concept of fixtures, then the easiest way to get acquainted with them is to manually populate the database and then dump these values with the command: python manage.py dumpdata -o mydata.json.gz (more on this here: https://docs.djangoproject.com /en/4.0/ref/django-admin/#dumpdata)
Fixtures that contain data to populate the test database must be added to the application in the shop/fixtures/data.json
You must specify

fixtures = [
        "shop/fixtures/data.json"
    ]

You need to remember that a superuser must be created in the test database. Therefore, if the superuser is not in the fixtures, then it must be created in the setUp(self)

An example of the contents of the test file shop/tests.py is shown below:

from django.test import TestCase,
from shop.models import *

class TestDataBase(TestCase):
    fixtures = [
        "shop/fixtures/data.json"
    ]

    def setUp(self):
        self.user = User.objects.get(username='root')

    def test_user_exists(self):
        users = User.objects.all()
        users_number = users.count()
        user = users.first()
        self.assertEqual(users_number, 1)
        self.assertEqual(user.username, 'root')
        self.assertTrue(user.is_superuser)

    def test_user_check_password(self):
        self.assertTrue(self.user.check_password('123'))

Writing code that passes these tests

An example of a code snippet used in the lecture is shown below:

def change_status_after_payment(payment: Payment):
    """
    Calling the method after creating a payment and before saving it.
    First need to find total amount from all payments (previous and current) for the user.
    If total amount >= the order amount, we change status and create negative payment.
    """
    user = payment.user
    while True:
        order = Order.objects.filter(status=Order.STATUS_WAITING_FOR_PAYMENT, user=user) \
            .order_by('creation_time') \
            .first()
        if not order:
            break

        total_payments_amount = get_balance(payment.user)
        if order.amount > total_payments_amount:
            break

        order.payment = payment
        order.status = Order.STATUS_PAID
        order.save()
        Payment.objects.create(user=user, amount=-order.amount)


@receiver(post_save, sender=Payment)
def on_payment_save(sender, instance, **kwargs):
    if instance.amount > 0:
        # pay_for_waiting_orders(instance)
        change_status_after_payment(instance)


@receiver(post_save, sender=Order)
def on_order_save(sender, instance, **kwargs):
    if Order.objects.filter(status='2_waiting_for_payment'):
        pay_for_waiting_orders(instance)

You can learn more about all the details of this stage from this video (RU voice):

To the next stage of the project

Drop Shipping Online Store on Django (Part 4)

4. Registering and Authorizing Users on a Django Site

In this course, we will go through all the stages of creating a new project together:

Registration and Authorization in Django

A Little bit of Theory

As has been repeatedly mentioned, the format of this course is purely practical. But without defining the concepts of session, cookies, GET- and POST requests it will be very difficult to understand the essence what we will do next. Therefore, a few words about this, nevertheless, are worth saying.

The Concept of Session and Cookies

How does the site "distinguish" "its" users from "strangers"?

Let's open the Storage → Cookies tab in the browser and try to register on a site. And we will see that at the time of registration, a new key session_id appears, which exists exactly as long as we save our authorization on this site. And it is this very session code that we can see in the django_session table if we open the database of this site.

However, if we log out, this session_id will disappear from both the database and browser cookies. The fact is that after the user authorizes on the site, the server creates a special digital label in its database, a copy of which it sends to the user's browser, and the browser stores this digital label in its memory. This is the session key, which is stored in the browser's memory. These data stored in the browser's memory are called cookies.

(Just in case, the SQLite browser mentioned in the video can be installed using these commands:
$ sudo add-apt-repository -y ppa:linuxgndu/sqlitebrowser
$ sudo apt-get update
$ sudo apt-get install sqlitebrowser )

As long as the user is logged in to the site, the session_id in the cookies of his browser matches the session_id stored in the database. That allows the user to visit any pages of the site to which he has access.

To do this, the browser sends session_id there with each new request to the server, and as long as this code matches the code in the server database, the user remains authorized.

If the user logs out, the server will delete his session_id from its database. And on those pages of the site where an authorized login is required, this user will not be able to log in.

If the user logs in again, he will again be able to visit any page of the site. But it will already be a completely different session with a different session_id.

Model User

All tables in a Django database (DB) are described using models. These are special Python classes that inherit from the Model class.

One model describes one database table. And each line of this model describes one field of the table.

In more detail we will analyze examples of models in the next 5th lesson. For now, let's just take note that the User user model, which has a certain set of basic fields and methods, already exists by default. It is always created in Django during the first migration. In particular, there are already fields username, email, password, is_staff (whether the user is an employee) and < b>is_superuser.

POST and GET Requests

New unfamiliar abbreviations usually inspire fear. To get rid of at least half of this fear, we immediately note that all the requests that we have talked about so far are GET requests. Really, isn't it scary at all? Then let's move on!

So, when accessing the server, different types of requests are possible. In general, the HTTP protocol has a lot of them, but in this course we will use only two:

GET request - when we just need to display some page
and POST request - when we need to send data to the server to store it in the database

To send a GET request, it is enough to specify a regular url. But the transmission of a POST request MANDATORY implies the presence of a form and a csrf token.

Forms, csrf-token

A form in HTML is a special construct that is enclosed in a <form></form> tag. This can also contain input fields with a single <input> tag, which is usually paired with a <label></label> tag, which explains that it is necessary to enter in the field input.

The form also contains a button (input field) with type="submit", clicking on which is exactly the command to send data to the server.

The current form looks like this in HTML code:

<form method="post" action="/your-url/">
    <label for="username">User name: </label>
    <input id="username" type="text" name="username" value="{{ username }}">
    <input type="submit" value="OK">
</form>

Passing URL to Other Applications, register.html and login.html Pages

In the previous lesson, we have already learned how to pass control from the configurator main/urls.py to the view. However, another option is also good practice: passing the url from the main configurator to another application's configurator.

To do this, in main/urls.py we will add a line, thanks to which all authorization requests starting with 'auth/' will be transferred for processing to another configurator - authentication /urls.py:

from django.contrib import admin
from django.urls import path, include

from company import views as views_company

urlpatterns = [
    path('', views_company.home_page, name='index'),
    path('about/', admin.site.urls),
    path('admin/', admin.site.urls),
    path('auth/', include('authentication.urls')),
]

Therefore, in authentication/urls.py there will be requests corresponding to authorization, registration and logout:

from django.urls import path

from authentication import views

urlpatterns = [
    path('login/', views.login_user, name='login'),
    path('register/', views.register, name='register'),
    path('logout/', views.logout_user, name='logout'),
]

Now, to begin with, we will make simple views that will be launched on these three requests in the authentication application:

def login_user(request):
    return render(request, ‘auth/login.html’)

def register(request):
    return render(request, ‘auth/register.html’)

def logout_user(request):
    pass

Nothing is clear yet about logging out - there is no page for it, so instead of the code we will simply write pass.

Well, actually, the template is not needed here. And you need a special logout function logout (request).

However, if we limit the view to just this function, we get an error: The view authentication.views.logout_user didn't return an HttpResponse object. It returned None instead.

Therefore, at the end of the view, we will have to return the transition to some page. For example, home. This can be done using the redirect('index') function, where 'index' is the address name in authentication/urls.

Creating a Form in the form.py Module

Obviously, now the construction for all views in the authorization application will not be as simple as it was for the index view. Because now we have a new task: to read the data from the HTML page, transfer it to the server and then to the User table in our database. And, as mentioned above, everything that is transferred to the database should only be transmitted using a form and a POST request.

To create forms in Django, a special forms.py module is provided, in which you can first create an "input form" object and describe its properties and characteristics in detail. The form is usually associated with a specific table in the database. Therefore, when creating a form, you should:

Indicate which table this form belongs to
Select a list of fields (you can select all fields and specify a specific list)
Define a list of required fields
Describe other field properties
and so on

Since we are using a ready-made Django's User table with a rather voluminous list of standard fields, it is imperative to specify in our form which fields will be used:

from django import forms
from django.contrib.auth.models import User


class LoginForm(forms.Form):
    username = forms.CharField()
    password = forms.CharField(widget=forms.PasswordInput())

The last line says that when entering a password, a special widget will be used that allows you to hide the symbol when filling out the field.

In the next lesson, we'll see that creating a form looks very similar to creating a database table.

Getting Data From HTML Page

Thus, the following options will need to be added to our login_user view:

Pass LoginForm to login.html
Ensure secure data entry in this form
Add a control button to the page, with which the user can send data to the server
Ensure data acceptance on the server, their validation
And, if necessary, add the received data to the appropriate database table

You can pass any data to the page using an additionally created context dictionary. In our version, by the login_form key, we will add an instance of the class of the newly created form LoginForm():

def login_user(request):
    context = {'login_form': LoginForm()}
    return render(request, 'auth/login.html', context)

This is quite enough to transfer data to an html page, but to receive data from the page, you will need to add data processing on a POST request.

def login_user(request):
    context = {'login_form': LoginForm()}

    if request.method == 'POST':
        login_form = LoginForm(request.POST)
        if login_form.is_valid():
            username = login_form.cleaned_data['username']
            password = login_form.cleaned_data['password']
            user = authenticate(username=username, password=password)
            if user:
                login(request, user)
                return redirect('index')

    return render(request, 'auth/login.html', context)

At the same time, on the html page, you will need to add:

Form tags <form></form>
In the form tag, be sure to specify the POST method and the url by which this POST request should be processed
Add <input> tag with name parameter equal to form field name
And finally, add the <input> tag (or <button>) with the type='”submit” parameter by click on which the user will be able to transfer data to the server.

That is, the form fragment in the HTML code will look something like this:

<form method="post" action="{% url 'login' %}">
    {% csrf_token %}
    <label class="form-label" for="id_username">Login</label>
    <input type="text" class="form-input" id="id_username"
                               name="username" placeholder="Login" required>
    <label class="form-label" for="id_password">Password</label>
    <input type="password" class="form-input" id="id_password"
                               name="password" placeholder="Password" required>
    <button type="submit" class="request-quote-btn">Log In</button>
</form>

The transfer of data from a user is a very sensitive operation with which a potential attacker can try to break into the server. Therefore, for additional security, in addition to the already existing session key, another secret key csrf_token is passed to the page. It is valid only for one data transfer from a particular form, and only for the data transfer of this particular form. If something goes wrong and a new form input is required, the server will generate a new csrf_token on a new request.

Handling Data Entry Errors

Filling out the form may be with errors - this is completely normal. It is not normal if the server "silently" does not accept this data and does not "explain" to the user why.

There are at least two ways.

1.) The simplest solution is to simply tell the user that the input is not correct. To do this, a new attention field is simply added to the context dictionary. In addition, it will be correct to return the form with exactly the data that did not pass.

if user:
    login(request, user)
    return redirect('index')
context = {
    'login_form': LoginForm(request.POST),
    'attention': f'The user with name {username} is not registered in the system!'}

2.) Another, more elegant and neat way: tell the user exactly what errors occurred during the validation process. And here you will need changes directly in the form itself:

def clean(self):
    cleaned_data = super().clean()
    username = cleaned_data.get('username')
    password = cleaned_data.get('password')

    try:
        self.user = User.objects.get(username=username)
    except User.DoesNotExist:
        raise forms.ValidationError(f'User with username: [{username}] does not exist!')

    if not self.user.check_password(password):
        raise forms.ValidationError('Could not log in using these email and password')

    return cleaned_data

Now these errors will be added to the LoginForm() form class instance. True, in order to read them on the authorization page, minor changes to the HTML code will also be required:

{% for error in login_form.non_field_errors %}
    <div class="alert alert-danger">{{ error }}</div>
{% endfor %}

At this point, the mission of creating an authorization page can be considered completed!

Creating a Registration View Using Generic Views

In addition to the view functions already well known to us, there are also view classes that inherit from generic views. Moreover, the latter is preferred when solving complex problems, since generic views look more compact and are easier to read.

We will start our acquaintance with the topic of generic views with only one option for now - the TemplateView class. (Later in this course, we will look at several more types of generic views).

As an example, let's rewrite our register function-based view into a RegisterView using the TemplateView class:

from django.views.generic import TemplateView

class RegisterView(TemplateView):
    template_name = 'auth/register.html'

This class has get and post methods to handle GET and POST requests:

def get(self, request):
        user_form = RegisterForm()
        context = {'user_form': user_form}
        return render(request, 'auth/register.html', context)

    def post(self, request):
        user_form = RegisterForm(request.POST)
        if user_form.is_valid():
            user = user_form.save()
            user.set_password(user.password)
            user.save()
            login(request, user)
            return redirect('index')

        context = {'user_form': user_form}
        return render(request, 'auth/register.html', context)

It is worth paying special attention to the method of creating a new user record in the User table, since we will continue to use this method further:

First, a user object is created - an instance of the model (class) User (but sometimes this can be done using the save() method class ModelForm).
Further (if necessary), this object can be changed / supplemented (in the example above, we store the string value of the password in the hash sum format).
But the new object is added directly to the database using another save() method, the Django model method.

The form RegisterForm, which is used here in the post method, should be mentioned separately.

Forms of the ModelForm Class

The previous LoginForm form was created based on the Form class, which is a fairly simple variant that looks very similar to the Model class of models.

Forms created with the ModelForm class are more advanced. Here we no longer need to prescribe each field of the form - just specify the name of the model that will be used in this form, and the list of required fields. After that, the format of the fields from the model attached to the form will automatically be transferred to the form itself.

New form example:

class RegisterForm(forms.ModelForm):
    class Meta:
        model = User
        fields = ('username', 'email', 'password')

Embedding a Form in HTML-code

In the previous LoginForm example, we attempted to embed a completed form into an HTML page using the {{ login_form }} tag. However, we quickly abandoned this idea - the styles of the standard Django form and the layout styles of our login.html page were too different.

But there is a solution. It is only necessary to display the fields not “at once”, but to do it gradually, in a cycle. Moreover, in this case, Django allows you to “break” the individual output for each form field into three components:

Output directly to the input field itself.
The field labels (title) associated with this field.
Input errors related to this field.

An example of HTML code that implements the above:

{% for field in user_form %}
<div class="row">
    <div class="col-12 col-lg-4 col-md-4 offset-md-4">
        <div class="quote-form-wrapper">
            <span style="color: red">{{ field.errors }}</span>
            {{ field.label_tag}}
            {{ field }}
        </div>
    </div>
</div>
{% endfor %}

You can learn more about all the details of this stage from this video (RU voice):

To the next stage of the project

Drop Shipping Online Store on Django (Part 3)

3. Adding an HTML-Template and Static Files to a Django Project. Creating a Base and Main Pages

In this course, we will go through all the stages of creating a new project together:

Adding an HTML-Template and Static Files to a Django Project

So, our non-standard checklist (in the form of a modified HTML template) has been created and now it's time to start "crossing out" its items. Let's start with the main page.

First of all, you need to make sure that the tamplates directories for html files and static for static files (css, js, and image files) have been added to the project root. And that the paths of these directories relative to the base directory were added to the project settings file settings.py.

Actually, it is not difficult to make sure of this: the actual tree diagram of our project is presented on the slide below:

├── authentication
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── db.sqlite3
├── main
│   ├── asgi.py
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── requirements.txt
├── shop
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── static
├── templates
└── venv

Now we copy the static files: we copy the entire contents of the asserts folder of the HTML template to the static project folder. Next, we copy the main page, as long as it is (that is, without highlighting the common part into the base page base-page.html for now). Let's just copy the index.html template file into the templates folder of our project.

Next, we need to remember that in order to display a page in a Django project, at least 2 steps are required:

Specify the url of this page, by which it will be called in the project.

And create a special function (or class) called view that will run when the specified link is clicked and display our page.

Now in our project, in addition to the main folder, there are two more application folders: the shop folder and the authentication folder. All three of these folders already have, or can be added a couple of files urls.py - views.py. But only one of them: a pair of main from the main folder can provide a url that does not have the additional insertion of url fragments in the form of /auth/ or /shop/ . In other words, links (urls) from the main folder will lead to the "clean" address of the domain name.

Of course, most often the home page was displayed at the address of a domain name. So we have decided on the location of the urls.py - views.py pair, it remains only to add the views file, which is still missing there, to the main folder. py. And then make the appropriate entries in these files:

main/urls.py

from django.contrib import admin
from django.urls import path
from shop import views

urlpatterns = [
    path('', views.index, name='index'),
    path('admin/', admin.site.urls),
]

Next, let's create a simple view, whose tasks will only include rendering (creating) the specified template. To create it, you can use a ready-made view file from one of our applications, or create a new views.py file in the main folder.

main/views.py

from django.shortcuts import render


def index(request):
    return render(request, 'index.html')

We try to run it, and we see that the main page is displayed, but the styles on it are completely absent. It's understandable: and links to styles should now be written in completely different paths (before, static files were in the asserts folder, and now it is in the static folder), and the way these links are written should also be completely different.

Firstly, every HTML page must now begin with a static load command {% load static %}.
And secondly, in all links, the path to all static files should now look different

<link rel="stylesheet" href="../construct/assets/css/style.css">

but like this:

<link rel="stylesheet" href="{% static '/css/style.css' %}">

We refresh the page, and we see that the picture has changed, but apart from the inscription Loading... nothing appears. So you need to update all links to the main page static files. And especially the js-files at the end of the page...

Well, now you can see something. Certain styles have appeared, although not all. In order for all of them to appear, you need to sequentially "walk" through all the links and carefully and carefully change them. As you can see, you can't do without a minimum knowledge of HTML on the back-end.

Extract Common Part of All Project Pages to Base Bage

Well, in order not to get up twice, we immediately optimize our work on bringing statics in accordance with the new requirements for all pages.

First of all, you need to pay attention to the fact that all pages of our HTML template have one common part, which includes the header (including the main menu) and footer. These elements are present on absolutely all pages. Therefore, it makes no sense to first duplicate the insertion of the same code, and then also suffer with changing links to static files. It will be much more reasonable if we separate this common part into a separate base page base-page.html, and leave insertion points for unique content from the project pages we need on this base page.

Now every HTML page, except for the base one, will have exactly the same structure as the index.html page:

{% extends 'base-page.html' %}
{% load static %}

{% block title %}
    < Page Name >
{% endblock title %}

{% block container %}
    < Current Page Content >
{% endblock container %}

That is, now when the page index.html is loaded, the base page base-page.html is loaded first (extends 'base-page.html') , then the statics for this page are loaded (load static), and then the current values of the < Page Name > and < Current Page Content > blocks are added to the specified insertion points.

All other pages will be similarly designed. For example, for the following page about-us.html, you need to do the same operations:

Add a new line with a new url to main/urls.py;
Add a new view to main/views.py;
And add the about-us.html page to the templates folder.

main/urls.py

from django.contrib import admin
from django.urls import path
from shop import views

urlpatterns = [
    path('', views.index, name='index'),
    path('about/', views.about, name='about'),
    path('admin/', admin.site.urls),
]

Next, let's create the simplest view, the tasks of which will only include rendering the specified template. To create it, you can use a ready-made view file from one of our applications, or create a new views.py file in the main folder.

main/views.py

from django.shortcuts import render


def about(request):
    return render(request, 'about-us.html')

And the last third step: add the about-us.html page to the templates folder. The modified HTML code of this page, in fact, will now consist only of the code that was previously located between the header and footer. And the structure of the page about-us.html will be no different from the page index.html.

index.html

{% extends 'base-page.html' %}
{% load static %}

{% block title %}
    < Page Name >
{% endblock title %}

{% block container %}
    < Current Page Content >
{% endblock container %}

Thus, after creating the base page, we can place in the templates of all other pages of the site only the original HTML code that is not in the base template. Of course, without forgetting to give a link to the base template. And that's it! This incredibly reduces code duplication and makes it easy to make changes to all pages of the site at once, just changing the code on the base page.

You can learn more about all the details of this stage from this video (RU voice):

To the next stage of the project

Drop Shipping Online Store on Django (Part 2)

2. Project Planning. Select and Update HTML-template

In this course, we will go through all the stages of creating a new project together:

Planning

In the last lesson, we prepared the development environment, created a new project in it, added the necessary settings to this project, and even created two new applications that will be responsible for registration/authorization and for the operation of the online store. In other words, we have completed the initial typical operations, which correspond to the preparatory stage of 99.9 (9)% of such projects. And now is the time to draw up a detailed plan of our further actions.

Project planning is, in fact, not an easy task. For these purposes, even a special scientific discipline "Project management" has been created. Of course, the format of our mini-course doesn't allow us to delve too deeply into this. And, nevertheless, a few words about project management still need to be said.

I think I won’t open America to anyone if I say that a project can be completed without any plan. It will just take longer and cost more. (If it works at all!)

Therefore, the hallmark of a good plan is that in each of its paragraphs it answers at least 4 basic questions:

What exactly do we do?
How do we do it?
What resources (What resources are used to run the current step?)
When (By what date?)

If there are answers to these questions for all points of the plan, we can be congratulated: we really have a plan! Well, if not, then we do not have a plan, but just a dream!

But what if we are going to do something that has never been done before? And/or if not one of the four questions, not one answer?

The answer is very simple: you need to highlight this unknown direction, "Terra incognito" in a separate plan. Plan of research work.

But what about the destination, with the timing? After all, in research work it is very difficult to determine the value?

And this problem is also solvable: where it is impossible to determine, you can always set a deadline that will not allow this research work to go to infinity.

Well, then everything is simple: if we have found answers to all unknown questions BEFORE the deadline, then we move on to the second stage: we draw up the results of the research in the usual way and start work. Well, if we didn’t meet it, then it’s likely that we didn’t take up our business. Therefore, we inevitably disappear with this idea and begin to work on the outcome of another task.

Fortunately, the problem we face in this mini-course is the most common and trivial one. Therefore, here you just need to carefully rewrite all the stages of our plan in the checklist, and, as you complete them, gradually cross them out, move them from line to line.

However, in our mini-course we will go even further, and as such a "checklist" we will take a ready-made HTML template, where each HTML page will correspond to a stage in our plan. The order of work will be as follows: we transferred the page from the template to the project - consider crossing out the line in the checklist! And so on until we "delete" (crossing out) all HTML pages from the template!

This idea seems completely absurd. But do not rush to draw conclusions!

Where can I get an HTML template for a project?

There are at least three answers to this question:

Do it yourself,
Order from professionals,
Use the services of a store of ready-made templates.

All of these options have their pros and cons:

First option will require time, design skills and the skills of a HTML coder. That is, it will not suit everyone.
The second option assumes that you have some financial reserve, and, oddly enough, a VERY clear understanding of what exactly you want to get in the end. It is important to remember here that the cost of a good front-end can be a sum with many zeros, and that it will be very difficult for a beginner to figure out where the production need ends and the waste of money begins. Moreover, good professionals are able not only to do well, but also to convince well)
Therefore, the best option for everyone, and especially for startup owners, is to visit the online store of ready-made templates. Here you can not only see different options, but even touch (in the sense of "click"). The prices here are quite democratic, and there are even completely free options. For example, the free template that is used in our project was taken from this online store: Template Monster . True, at present this particular option is not available there, but still, there are a sufficient number of similar and free options. By the way, please note that only HTML5 templates will work for us.

Planning the pages of the future template

So, the selected template is downloaded from the Template Monster website and unpacked into a separate folder. If we open the index.html file in the browser, it may seem that we have a ready-made project in front of us: all (or almost all) buttons and links work and lead to the necessary pages. With one, however, significant caveat: all these beautiful pages are filled with not quite the information that we need. Therefore, right now we have to determine which pages we will leave, which ones we will delete, and which, on the contrary, we will add.

Let's now decide on the set of pages of our future template. Our project will require the following pages:

Home pag.
Registration page.
Authorization page (login)
Product catalog page. Here, control will be added to select a product and place it in the basket in the required quantity
Product detail page - page with more detailed information on the product
Cart page - a list of products selected for purchase and payment.
Scraping control page. Must be available only to company staff and NOT available to customers

Obviously, all of these pages (except for the detail page) should have links in the main menu. And it is also desirable that the registration / authorization menu be separate from the main one.

Website functionality planning

The database will be as simple as possible (after all, our course is mini, not maxi).

The product table will be “flat”, i.e. no categories/subcategories. The measure of the item will be “pieces”. The selected product with the set quantity is automatically added to the shopping cart. If there is no basket for this user yet, it is created, if it already exists, it is added.

If the goods from the Cart are sent for payment, the Cart is cleared and the Order appears with the status "waiting for payment".

To account for payments, a table of payments must be created. If the amount of the credited payment is greater than or equal to the amount of the order, then the Order changes its status to “paid”, and a “negative” payment with a “minus amount” is formed in the payment table itself. Thus, to determine the balance of the client's account, it will not be necessary to create a new balance table. In addition, this scheme will also solve the issue of online payment: when connecting an online payment aggregator, the system will only need to create a payment object equal to the amount of the payment received. This will be enough to automatically transfer the Order to the paid category.

And we provide for work with bank payments:

After each payment, we automatically check for unpaid orders. If there are several of them, then we pay starting from the “oldest” one.
When creating an Order awaiting payment, we also check if there is a balance on the client's account. And if the amount of the balance is greater than or equal to the amount of the order, we automatically transfer the Order to the category of paid ones and subtract the amount of the order from the balance.

We're done with the planning and moving on to turning the HTML template we just downloaded into the "checklist" of our project.

Template update

So, the selected template is downloaded from the Template Monster website and unpacked into a separate folder. If we open the index.html file in the browser, it may seem that we have a ready-made project in front of us: all (or almost all) buttons and links work and lead to the necessary pages. With one, however, significant caveat: all these beautiful pages are filled with not quite the information that we need. Therefore, right now we have to determine which pages we will leave, which ones we will delete, and which, on the contrary, we will add.

Of course, save the home page index.html (HOME).
You can also save the about-us.html (ABOUT) page. Since, according to legend, we are a construction company, then someday we will have to give information about ourselves. In this project, we will not post anything on this page, but we will leave it as a reserve for the future.

On this, the choice of finished pages ended - the rest of the pages went through to collect with the help of "glue" and "scissors" from what was on the deleted pages.

Registration Page register.html (Register);
And login page login.html (Log In).

Kstati, dlya ssylok na stranitsy registratsii/avtorizatsii bylo ispol'zovano gotovoye menyu vybora yazykov na glavnoy stranitsy. Ssylki na sleduyushchiye stranitsy byli razmeshcheny v glavnom menyu:

Stranitsa otobrazheniya vsego spiska tovarov internet-magazina shop.html (SHOP). Tam zhe budet dobavlena vozmozhnoct' vybora tovara v nuzhnom kolichestve.
Stranitsa detalizatsii vybrannogo tovara shop-details.html (punkta menyu na glavnoy stranitsy u neyo net - otkryvayetsya po knopke "Detail", raspolozhennoy ryadom s kazhdym tovarom.
Stranitsa korziny cart.html (CART)
I stranitsa zapuska skripta skreypinga (parsinga) tovarov so storonnego sayta fill-products.html (FILL-DATABASE)

Nu, vot, kazhetsya i vso: vse osnovnyye momenty plana uchteny, i vse nuzhnyye stranitsy dobavleny.

Show more 859 / 5,000 Translation results

By the way, for links to the registration/authorization pages, a ready-made language selection menu on the main page was used. Links to the following pages have been placed in the main menu:

The page for displaying the entire list of goods of the online store shop.html (SHOP). There will also be added the ability to select goods in the right quantity.
The detail page for the selected product shop-details.html (it does not have a menu item on the main page - it opens by clicking the "Detail" button next to each product.
Cart page cart.html (CART)
And the page for launching the script for scraping (parsing) products from a third-party site fill-products.html (FILL-DATABASE)

Well, that seems to be all: all the main points of the plan have been taken into account, and all the necessary pages have been added.

And in conclusion of the topic of planning, it is necessary to say about a very important point: when planning, you should always remember that the IT project plan is not a sacred cow. When implementing the plan, details may appear that we did not foresee at the beginning. Like, for example, when planning this mini-course, I did not think about the fact that the second lesson turns out to be completely theoretical. Therefore, I moved the item “Selecting and finalizing a template” from the next lesson to this one.

Summary: if, as the project progresses, it becomes necessary to adjust the plan, adjust it without hesitation!

You can learn more about all the details of this stage from this video (RU voice):

To the next stage of the project

Drop Shipping Online Store on Django

1. Introduction. Who is this course for? Preparing the Development Environment

Introduction

This article is a presentation of the new Drop Shipping Store on Django course, where in 7 lessons you will create a full-fledged online store from free materials that are in the public domain, which can trade anything on a dropshipping basis. If anyone is not in the know, then dropshipping is a way of organizing a business in which the seller places on his virtual storefronts goods that he does not have, but which he can receive at any time from his regular supplier. Therefore, the task of the owner of such an online store is to receive an order and payment from the client, and then transfer the information about the order and the received payment to the supplier, minus his commission.

Everything you need to create this online store: programs, necessary training materials and even video tutorials are in the public domain. You just need to carefully follow everything that will be written and shown in the materials for this course.

In this course, we will go through all the stages of creating a new project together:

As mentioned above, all videos that describe in detail (and most importantly, show!) The development process are in the public domain. Therefore, carefully following the recommendations on the screen, you will eventually get the real working code of the entire project. However, for those who are immediately interested in the final result, there is the option of accessing an archived copy of the project after each lesson. This access can be obtained through a small donation here at this link (registration required).

Well, the site itself, which will turn out in the end, can be viewed in the video at the end of this article.

Who is this course for?

For everyone who is interested in web development, regardless of the level of knowledge and experience!

Those who are far from programming and who have been dreaming of their own online store for a long time will be able to get acquainted with the main stages of creating a web project, get an idea of the capabilities of the Django framework and understand the amount of work that is necessary for implementation these possibilities. This will allow, firstly, to formulate the tasks for your own project more accurately and in more detail, and, secondly, to specify the requirements for the contractor to implement these same tasks. This will significantly reduce the time for discussing these tasks with the contractor, and hence the overall costs of the project.

For those who consider themselves to be Familiar with the basics of the Python language this will be a great opportunity to try their hand at creating their own project. Actually, if you don’t know Python yet, then only two courses separate you from this category: Python for beginners and Python Basics. After passing them, you will be able to independently adjust this project for many of your tasks and save the result in your own repository. Well, and what you can’t implement on your own yet, you can order a third-party contractor (including us!). In any case, finalizing a part of the project is not at all the same as doing the project from scratch!

Well, if you are familiar with Python OOP, have already tried your hand at the Django website and created your first Polls application there, then this course will suit you perfectly. Thanks to it, you can independently implement most of the tasks in your application. Well, the execution of that insignificant part of the project, which you cannot do yourself yet, can always be ordered by a specialist. Including us :-).

Preparing the Development Environment

We will be working on a Linux operating system, and as an IDE we will use the Community (free) version of PyCharm. The choice of Linux is not accidental: it is on this OS that all servers that use the Django framework work. Therefore, if you work on Windows, then the best solution would be to additionally install a virtual machine with Linux OS. How to do this is described in detail and shown in these videos:

In addition, we will need somewhere to save checked and debugged versions of our code, and a more convenient place than a repository has not yet been invented for this purpose. Therefore, if you have not yet created an account with SSH access in any repository, then it makes sense to read See this article on our website, which details and shows how to create an account on GutHub and add SSH keys to it.

After carrying out all the above additional preparations, we can finally proceed directly to the creation of the project itself in the following order:

create a virtual environment for our project;
install the Django package on it;
create the project itself;
change the settings.py file responsible for the settings of our project;
add two new applications to the project:
- application responsible for registering and authorizing users (authentication);
- and the application of the online store itself (shop);
make the first migration;
create a superuser;
add to .gitignore .idea for PyCharm;
and "push" everything done to the repository.

After completing all the points of this checklist, we will try to run the project. Hooray - our project is already running! And all that remains for us is to "slightly" modify its functionality!

We go into the admin panel under the login and password as the newly created superuser. As you can see, there are already user and user group models.

We haven't added this structure yet, but it comes by default in Django. Therefore, if desired, we can add the first users for testing even before we authorize users on the site. That is, directly from the admin panel!

The results of all this preparatory work of the current stage, you can find the link at the bottom of the video, which will lead you to the course Drop Shipping Store on Django (RU voice). And, if you have paid for the course, then you can download an archive copy of this stage (that is, what has been done so far) (registration required).

You can learn more about all the details of this stage from this video (RU voice):

Infinite Dependent Drop Down Lists in в Google Sheets (Part 2)

This article is a continuation of Infinite Dependent Dropdowns in Google Sheets.

Changes and additions to the code are described here, namely:

Automatic creation of sheets Nome Data_2;
Automatic addition of all formulas on sheet Nome;
When editing the row of dependent (linked lists) created earlier, the values and formulas to the right of the edited cell are automatically deleted.

function onEdit(e) {

  var row = e.range.getRow();
  var col = e.range.getColumn();
  var list_name = e.source.getActiveSheet().getName();
  var name = e.value;
  var oldName = e.oldValue;
  var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Home');
  var mask = JSON.stringify(sh.getRange(row, 1, 1, col).getValues()[0]);
  var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Home');
  var colMax = sh.getLastColumn();
  
  if(list_name === "Home" && name !== oldName && col < colMax) {
    fillColumn(row, col, mask);
  }
}

function onOpen() {
  var ui = SpreadsheetApp.getUi();
  // Or DocumentApp or FormApp.
  ui.createMenu('Custom Menu')
      .addItem('Create sheets', 'createSheets')
      .addToUi();
}


function fillColumn(row, col, mask) {

// clear dataVal and Value
  var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Home');
  var colMax = sh.getLastColumn();
  sh.getRange(row, col + 1, 1, colMax).clearDataValidations().clearContent();

// find date
  var sd = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Data");
  var arrData = sd.getDataRange().getValues();
  var arrData_2 = [];
  arrData_2.push(arrData[0]);
  
  var iMax = arrData.length - 1;
  for(var i=1; i<=iMax; i++) {
    if(JSON.stringify(arrData[i].slice(0, col)) == mask) {
      arrData_2.push(arrData[i]);
    }
  }
   
// clear Data_2
  var sd_2 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Data_2");
  sd_2.getDataRange().clearContent();

// insert data
  sd_2.getRange(1, 1, arrData_2.length, arrData_2[0].length).setValues(arrData_2);
  
// add dataVal
  col++;
  var list = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K'];
  var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Home');
  sh.getRange(row, col).setDataValidation(SpreadsheetApp.newDataValidation()
  .setAllowInvalid(false)
  .requireValueInRange(sh.getRange('Data_2!$' + list[col - 1] + '$2:$' + list[col - 1] + '$1000'), true)
  .build());
}

function createSheets() {

// is exist Home?
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sd = ss.getSheetByName('Data')
  
// create if not exist
  if(!ss.getSheetByName('Home')) {
    ss.insertSheet('Home', 0);
// create Data Val
    var sh = ss.getSheetByName('Home');
    sh.getRange('Home!A2:A20').setDataValidation(SpreadsheetApp.newDataValidation()
      .setAllowInvalid(false)
      .requireValueInRange(sh.getRange('Data!$A$2:$A'), true)
      .build());
    sh.getRange(1, 1, 1, 10).setValues(sd.getRange(1, 1, 1, 10).getValues()).setFontWeight('bold');

  };

// is exist Data_2?
  if(!ss.getSheetByName('Data_2')) {
   
// create if not exist
    var k = ss.getNumSheets();
    ss.insertSheet('Data_2', k + 1);
    var sd_2 = ss.getSheetByName('Data_2');
    sd_2.getRange(1, 1, 1, 10).setValues(sd.getRange(1, 1, 1, 10).getValues()).setFontWeight('bold');
  };
}

Now, in this version of the program, it is possible to automatically create all the sheets of the file necessary for the script to work (including data formatting and validation) by pressing just one(!) button from the user menu.

All you need for this:

create a sheet named "Data"
copy the data needed for linked (dependent) lists into it
copy and paste this script into a file
refresh page
in the appeared custom menu select: Create sheets

You can get more information from this video (RU voice):

ATTENTION!
This article has a continuation: Infinite Dependent Dropdowns in Google Sheets (part 3)

Infinite Dependent Drop Down Lists in в Google Sheets

The article considers the option of creating dependent drop-down lists on the Google Spreadsheet sheet, practically unlimited neither in the number of linked elements, nor in the number of rows on the sheet.

The program script is shown below:

function onEdit(e) {

  let col = e.range.getColumn();
  let list_name = e.source.getActiveSheet().getName();
  let name = e.value;
  
  if(list_name=="Home") {
    fillColumn(col, name);
  }
}

function fillColumn(col, name) {

// find date
  var sd = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Data");
  var iMax = sd.getLastRow();
  var col_data = [];
  
  for(var i=2; i<=iMax; i++) {
    var x = sd.getRange(i, col).getValue();
    if(x == name) {
      col_data.push(sd.getRange(i, col+1).getValue());
    }
  }
   
// clear Data_2
  var sd_2 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Data_2");
  sd_2.getRange(2, col+1, sd_2.getLastRow(), sd_2.getLastColumn()).clearContent();


// insert data
  iMax = col_data.length;

  for(var i=2; i<=iMax+1; i++) {
    sd_2.getRange(i, col+1).setValue(col_data.shift());
  }
}

Please note: sheet names must EXACTLY match the names specified in the script!

Main sheet: "Home"

datasheet: "Data",

intermediate data sheet: "Data_2"

These names in the script are highlighted in red, so, if desired, it will not be difficult to find them to replace with the names of your sheets.

This article has a continuation: Infinite Dependent Dropdowns Lists in Google Sheets (Part 2)

Now, in the new version of the program, it is possible to automatically create all the sheets of the file necessary for the script to work (including data formatting and validation) by pressing just one (!) button from the user menu.

More information you can find in this video (RU voice):

How to create and add SSH-key to your GitHub-account

In order to access the repository on GitHub via ssh, you first need to create an account itself.

Further, after authorization, you need to go to your account settings (click on your avatar icon in the upper right corner of the screen and select settings) --> SSH and GPG keys), and then click the New SSH key button (also on the top right).

Now in these two opened fields (Title and Key) you need to insert the name of the key (any) and the code of the public part of the key itself (copy the contents of the key).

Where to get them? First of all, it makes sense to check if you have generated your key before and simply forgot about it. To do this, go to the Home folder and look for keys there in the hidden .ssh folder (you must first add the ability to view hidden files in the Navigator). Another way to find this folder is through the terminal window with the command:

$ ls -al ~/.ssh

If there are keys, then we will see at least 4 lines of text

drwx------ 2 su su 4096 сеп 24 23:24 .
drwxr-xr-x 23 su su 4096 сеп 24 23:24 ..
-rw------- 1 su su 419 сеп 24 23:24 id_ed25519
-rw-r--r-- 1 su su 104 сеп 24 23:24 id_ed25519.pub

Well, if not, then only the top two. In this case, we will have to generate the keys ourselves:

$ ssh-keygen -t ed25519 -C "my_email@example.com"

The system will inform about the beginning of the generation process and specify under what name the file with the key should be saved:

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/su/.ssh/id_ed25519):

Just agree (Enter).

Next, the system will inform you about the creation of the .ssh directory and prompt you to enter a password phrase (it is not necessary to enter):

Created directory '/home/su/.ssh'.
Enter passphrase (empty for no passphrase):

If you have entered a password, then the next step will require you to enter confirmation, if not, then just press Enter.

And as a result, the password will be generated and something like this will be displayed on the screen:

Enter same passphrase again:
Your identification has been saved in /home/su/.ssh/id_ed25519
Your public key has been saved in /home/su/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:FOkr66ZIZN5eTZ1omlY77s1GkgZtWa+A3NNDwEKCzcE my_email@example.com
The key's randomart image is:
+--[ED25519 256]--+
| =oo..oo |
| . E.. o.o |
| . *.= . |
| +.Ooo.. |
| o oS=oo |
| + . .B=.o |
| o . =++o |
| . o oo. +. |
| . o+..o.o |
+----[SHA256]-----+

Now you need to go to the .ssh folder and copy the public part of the key, which is in the file with the extension: .pud (id_ed25519.pub).

We need to copy the contents of this particular file and paste it into the Key field of the key settings screen on GitHub. Next, you need to come up with a name for this key (in case there are several keys) and enter this name in the Title field. After filling in both fields and clicking the Add SSH key button, adding the key can be considered completed. And now it would be nice to make sure that everything works.

To do this, we will create a test project in our repository, copy it to the local machine, try to make changes to our project and "push" the result to the repository.

Clicks the "plus" to the left of the avatar in the upper right corner of GitHub, between the "bell" and the avatar --> New repository

Next, in the page that opens, enter the project name under Repository name, write a few words about the project in the Description section, mark "add" for Add a README file > and Add .gitignore, and in the ".gitignore" options, choose Python. And at the end of everything, click Create repository.

The project has been created, and now we copy it to a folder on the local computer, for which we select the latter in the HTTPS / SSH switcher and copy the link to our newly created project in the GitHub repository.

But before copying, we need to make sure that the git package is installed on our computer. This can be done with the command:

$ git --version

If it turns out that git is not installed, then we use the following commands;

$ sudo apt-get update
$ sudo apt-get install git

Now, directly in the folder where we want to place this project, open the terminal window (right mouse button --> "Open in Terminal"), and then, in the terminal itself, enter:

$ git clone git@github.com:it4each/my-project.git

The first time we clone, we will most likely end up with something like this:

Cloning into 'my-project'...
The authenticity of host 'github.com (140.82.121.4)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

We answer yes and the project is successfully added to the specified folder on our computer. In addition, the GitHub address has been added to the list of known hosts (see below).

Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com,140.82.121.4' (RSA) to the list of known hosts.
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (4/4), done.

And if we now once again examine the contents of the keys folder, we will see a new file (known_hosts) in it with a list of known hosts, where the GitHub host is also added:

$ ls -al ~/.ssh drwx------ 2 su su 4096 сеп 25 10:49 .
drwxr-xr-x 23 su su 4096 сеп 24 23:39 ..
-rw------- 1 su su 411 сеп 24 23:43 id_ed25519
-rw-r--r-- 1 su su 102 сеп 24 23:43 id_ed25519.pub
-rw-r--r-- 1 su su 884 сеп 25 10:49 known_hosts

Now, for the project to work, we need to create a virtual environment. This requires the packages pip3 and virtualenv. If these packages are not installed yet, use the commands:

# install pip3
$ sudo apt-get install python3-pip  

# install virtual environment
$ sudo pip3 install virtualenv

To create a virtual environment, go to the project folder and then run the installation command:

$ cd my-project

$ virtualenv venv -p python3

The virtual environment has been created. Now it needs to be run:

user@user:~/Projects/django-app/my-project$ source venv/bin/activate

(venv) user@user:~/Projects/django-app/my-project$

The proof that the virtual environment is running is the appearance of the venv folder in parentheses - (venv) in the system response.

Install the first package - the Django package. And immediately after installing it, we create a new project (to create it inside the project folder, add a dot to the end of the command):

$ pip3 install Django

$ django-admin startproject main .

We remember all installed packages in the requirements.txt file:

$ pip3 freeze > requirements.txt

Now there are enough files in our folder to make the first commit and save the work done in the GitHub repository. To do this, first add all the created files with the add --all command, then create a commit commit and send this commit to the repository with the push command:

$ git add --all

$ git commit -am "first commit"

$ git push

If everything went without errors, then the following files should be added to the repository:

[main 0a792bb] first commit
7 files changed, 200 insertions(+)
create mode 100644 main/__init__.py
create mode 100644 main/asgi.py
create mode 100644 main/settings.py
create mode 100644 main/urls.py
create mode 100644 main/wsgi.py
create mode 100755 manage.py
create mode 100644 requirements.txt

Please note that the largest folder of our project - the virtual environment folder venv was not included in the repository, because by default it is located in the .gitignore file - in the list of exclusion files for copying to remote storage. If you remember, GitHub offered us to add this file when creating the my-project project.

It should be noted that there is no .idea folder for PyCharm service files in .gitignore. Therefore, if you use this IDE to create your project, don't forget to add an exception for the .idea folder yourself.

For more information on adding an SSH key and creating a project, watch this video (RU voice):

Dropdown Lists in Django Admin

What's the idea?

For those who are not yet familiar with dependent dropdown lists and have not yet had time to appreciate the incredible set of conveniences that they provide when working with spreadsheets, I recommend get to know this article. (There is also a link to the video)

Those who do not need to advertise the obvious advantages of dependent dropdown lists, sooner or later "rest" against the natural, and alas, inevitable limitations of spreadsheets:

requiring sufficiently high qualifications for staff,
lack of "foolproof",
complexity in the organization of protection and distribution of access rights,
difficulty in organizing automatic data exchange with other information sources (websites, databases)
etc. etc.

This is especially true for small business owners who are well acquainted with the symptoms of "growing pains": what was incredibly helpful yesterday has become a hindrance and brake today.

To this end, an attempt was made to combine the advantages of spreadsheets and databases. And the administrative panel of the popular Django framework, written in the Python programming language, was chosen as the place for such a "polygon".

Why Django?

The main argument for this choice was the administrative panel of this framework. The Django admin has a tabular structure, so it looks like a spreadsheet. Therefore, improvements to the level of compliance with the specified parameters here require the most minimal.

Well, such pleasant "little things" as the incredible popularity of this framework, its reliability and performance just strengthened this choice.

Preparing the project and setting up the development environment

Python is the child of Unix OS. Therefore, choosing a unix-like operating system (Linux or macOS) would be a completely logical decision. Moreover, it is not at all necessary to switch to a computer with another OS: you can simply install a virtual machine on your computer with any operating system and install Linux on it already. And also install PyCharm - a convenient and free development environment.

How to do this is described in detail and shown in these videos (RU voice!):

After successfully installing all of the above, you can proceed to create a project. To do this, you need to create a directory with the name of the project django-dropdown-admin, enter this directory and start the process of creating a virtual environment and then to start it:

$ virtualenv venv - p python3

$ source venv/bin/activate

The virtual environment is running, as indicated by the parenthesized folder name that appears at the beginning of the command line: (venv).

Next, install Django:

(venv) $ pip install django

Create a project (note the dot at the end of the command line!):

(venv) $ django-admin startproject main .

And start the dropdown application:

(venv) $ django-admin startapp dropdown

You will see something like this as a result:

.
├── dropdown
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── main
│   ├── asgi.py
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
└── venv

From now on, all further actions will be more convenient to perform in the PyCharm IDE. (see video at the end of this article)

1. Change settings.py

First of all, let's make changes to the settings - the main/settings.py file:

1.) Add the newly created application to INSTALLED_APPS

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # === my apps =======
    'dropdown',
]

2.) Cancel complex validation of user passwords for debug mode

# Password validation
# https://docs.djangoproject.com/en/4.0/ref/settings/#auth-password-validators

if not DEBUG:
    AUTH_PASSWORD_VALIDATORS = [
        {
            'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
        },
        {
            'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
        },
        {
            'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
        },
        {
            'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
        },
    ]

3.) And specify the path for the static files (it will be required to change the admin js script)

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.0/howto/static-files/

STATIC_URL = 'static/'

STATICFILES_DIRS = [
    BASE_DIR / "static",
]

To logically complete the last action, immediately add a new directory for static files to the project directory: static

2. Create a database (DB)

As you know, Django uses ORM (Object-relational mapping), so to create a database, you just need to describe its structure in the models.py file and then carry out migrations, which, according to the description of this structure, will create the necessary database tables and establish all necessary links between the fields of these tables.

The contents of the file dropdown/models.py:

from django.db import models


class Category(models.Model):
    name_cat = models.CharField(max_length=250)

    def __str__(self):
        return self.name_cat


class Subcategory(models.Model):
    cat = models.ForeignKey(Category, on_delete=models.CASCADE,
                            verbose_name='category')
    name_subcat = models.CharField(max_length=250)

    def __str__(self):
        return self.name_subcat


class Good(models.Model):
    subcat = models.ForeignKey(Subcategory, on_delete=models.CASCADE,
                               verbose_name='subcategory')
    name_good = models.CharField(max_length=250)
    price = models.DecimalField(max_digits=6, decimal_places=2)

    def __str__(self):
        return self.name_good


class OrderItem(models.Model):
    order = models.ForeignKey("dropdown.Order", on_delete=models.CASCADE, verbose_name="order")
    cat = models.ForeignKey(Category, on_delete=models.CASCADE, verbose_name='cat')
    subcat = models.ForeignKey(Subcategory, on_delete=models.CASCADE, verbose_name='subcat')
    name_good = models.ForeignKey(Good, on_delete=models.CASCADE, verbose_name='good')
    quantity = models.PositiveIntegerField(default=0)
    amount = models.DecimalField(max_digits=9, decimal_places=2)

    def __str__(self):
        return f'{self.name_good} + {self.quantity}'


class Order(models.Model):
    order_id = models.PositiveIntegerField(unique=True)
    order_date = models.DateTimeField(auto_now=True)
    total_quantity = models.PositiveIntegerField(default=0)
    total_amount = models.DecimalField(max_digits=9, decimal_places=2)

    def __str__(self):
        return str(self.order_id)


class AllowedCombination(models.Model):
    cat = models.ForeignKey(Category, on_delete=models.CASCADE)
    subcat = models.ForeignKey(Subcategory, on_delete=models.CASCADE)
    good = models.ForeignKey(Good, on_delete=models.CASCADE)

    def __str__(self):
        return f'{self.cat} {self.subcat} {self.good}'

    class Meta:
        ordering = ['pk']

Now that the database structure is described, it is necessary to implement it in the database itself, for which we first create migrations in the terminal window with the first command (python manage.py makemigrations):

(venv) $ python manage.py makemigrations 

Migrations for 'dropdown':
  dropdown/migrations/0001_initial.py
    - Create model Category
    - Create model Good
    - Create model Order
    - Create model Subcategory
    - Create model OrderItem
    - Add field subcat to good
    - Create model AllowedCombination

And then, with the following command (python manage.py migrate), we apply all these migrations to our database:

(venv) $ python manage.py migrate

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, dropdown, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying dropdown.0001_initial... OK
  Applying sessions.0001_initial... OK

3. Preliminary configuration of the Admin panel (administrative panel of the Django framework)

In the simplest version, we can work with our database directly from the Admin. By the way, this is the rare case when "simplest" does not mean "worst" at all. Quite the contrary - the standard Django admin panel already contains incredibly convenient and powerful tools for adding, changing and deleting data from the database.

And all that is needed for this is just to make a minimal description of the Admin panel structure in the file dropdown/admin.py using the following code:

from django.contrib import admin

from .models import (AllowedCombination, Category, Good, Order, OrderItem,
                     Subcategory)

admin.site.register(Category)
admin.site.register(Subcategory)
admin.site.register(Good)


class OrderItemInline(admin.TabularInline):
    model = OrderItem
    extra = 0


class OrderAdmin(admin.ModelAdmin):
    inlines = [OrderItemInline]


admin.site.register(Order, OrderAdmin)


class OrderItemAdmin(admin.ModelAdmin):
    pass


admin.site.register(OrderItem, OrderItemAdmin)


class AllowedCombinationAdmin(admin.ModelAdmin):
    list_display = ['cat', 'subcat', 'good', ]


admin.site.register(AllowedCombination, AllowedCombinationAdmin)

To enter the Admin panel, it remains to take the last step - create a Superuser.

Therefore, we enter the terminal window again and first enter the command to create the Superuser (python manage.py createsuperuser), and then sequentially answer the system's questions:

(venv) $ python manage.py createsuperuser
Username (leave blank to use 'su'): root
Email address: root@root.com
Password: 
Password (again): 
Superuser created successfully.

4. Filling the database

Now is the time to check what we got. Run the site using the command python manage.py runserver

(venv) $ python manage.py runserver

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
April 06, 2022 - 17:33:16
Django version 4.0.3, using settings 'main.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

and go to the admin panel by url: http://127.0.0.1:8000/admin/

And we log in as a Superuser, using the login and password that we entered when creating the superuser user.

In our version, there will be three dependent drop-down lists:

Categories
Subcategories
and directly Goods

You must first fill them in, and then fill in the table of Allowed Combinations (AllowedCombination) on the basis of which our script will offer values in the drop-down lists. Well, something like this:

Of course, in each case, all tables, the names of categories and subcategories, the depth of nesting of all these categories-subcategories-sub-subcategories, and, of course, the values themselves, will be completely different. The proposed option is just an illustration of the idea itself.

5. Make Lists Dependent

Everything that was before - the most that is, the usual and standard Django. Which knows how to work with lists, but does not yet know how to make them dependent, that is, to suggest the values of the subsequent list, taking into account the choice of the previous one.

To teach Django new functionality, you need to add a little "magic":

Add a new form (file dropdown/forms.py
Add a new js script (file static/js/restricted-model-choice-field.js
Apply changes to an existing file dropdown/admin.py

Add file dropdown/forms.py:

import json
from typing import Any

from django import forms
from django.forms import Media, widgets

from .models import Good, OrderItem, Subcategory


class RestrictedSelect(widgets.Select):
    @property
    def media(self):
        media = super().media
        media += Media(js=["js/restricted-model-choice-field.js"])
        return media


class BoundRestrictedModelChoiceField(forms.BoundField):
    def get_restrictions(self):
        restrictions = {}

        restrict_on_form_field = self.form.fields[self.field.restrict_on_form_field]
        # Можно оптимизировать
        for restricting_object in restrict_on_form_field.queryset:
            allowed_objects = self.field.queryset.filter(**{self.field.restrict_on_relation: restricting_object})
            for obj in allowed_objects:
                restrictions.setdefault(obj.id, set()).add(restricting_object.id)

        return restrictions

    def build_widget_attrs(self, attrs, widget=None):
        attrs = super().build_widget_attrs(attrs, widget)

        restrictions = self.get_restrictions()
        restrictions = {k: [str(v) for v in vs] for k, vs in restrictions.items()}
        attrs["data-restrictions"] = json.dumps(restrictions)

        bound_restrict_on_form_field = self.form[self.field.restrict_on_form_field]
        attrs["data-restricted-on"] = bound_restrict_on_form_field.html_name

        return attrs


class RestrictedModelChoiceField(forms.ModelChoiceField):
    widget = RestrictedSelect

    def __init__(self, *args, restrict_on_form_field: str = None, restrict_on_relation: str = None, **kwargs):
        super().__init__(*args, **kwargs)

        if not restrict_on_form_field:
            raise ValueError("restrict_on_form_field is required")
        self.restrict_on_form_field = restrict_on_form_field

        if not restrict_on_relation:
            raise ValueError("restrict_on_relation is required")
        self.restrict_on_relation = restrict_on_relation

    def get_bound_field(self, form, field_name):
        return BoundRestrictedModelChoiceField(form, self, field_name)


class OrderItemForm(forms.ModelForm):
    class Meta:
        model = OrderItem
        fields = "__all__"

    subcat = RestrictedModelChoiceField(
        Subcategory.objects.all(),
        restrict_on_form_field="cat",
        restrict_on_relation="allowedcombination__cat",
    )
    name_good = RestrictedModelChoiceField(
        Good.objects.all(),
        restrict_on_form_field="subcat",
        restrict_on_relation="allowedcombination__subcat",
    )

Add a static file static/js/restricted-model-choice-field.js:

(function(){
  function throwError(baseElement, message) {
    console.error(message, baseElement);
    throw new Error(message);
  }

  function reset(baseElement) {
    baseElement.value = "";
  }

  function getRestrictedFields() {
    return Array.prototype.slice.apply(document.querySelectorAll("[data-restrictions]"));
  }

  function getFieldsRestrictedOn(baseElement) {
    let elements = getRestrictedFields();
    return elements.filter(e => getRestrictedOnField(e) === baseElement);
  }

  function isFormsetTemplate(baseElement) {
    return baseElement.name.indexOf("__prefix__") >= 0;
  }

  function getRestrictedOnField(baseElement) {
    if (isFormsetTemplate(baseElement)) {
      return null;
    }

    let fieldName = baseElement.getAttribute("data-restricted-on");
    if (!fieldName) {
      throwError(baseElement, "data-restricted-on is undefined");
    }

    if (fieldName.indexOf("__prefix__") >= 0) {
      fieldName = cleanDynamicFormSetName(baseElement, fieldName);
    }

    let form = baseElement.closest("form");
    if (!form) {
      throwError(baseElement, "The field is not inside a form");
    }

    let fields = form.querySelectorAll(`[name=${fieldName}]`);
    if (fields.length == 0) {
      throwError(baseElement, `Could not find field ${fieldName}`);
    }

    if (fields.length > 1) {
      console.warn(`Found multiple fields ${fieldName}`);
    }
    return fields[0];
  }

  function cleanDynamicFormSetName(baseElement, fieldName) {
    let prefixIx = fieldName.indexOf("__prefix__");
    let selfPrefix = baseElement.name.slice(prefixIx);
    let prefixMatch = selfPrefix.match(/\d+/);
    if (!prefixMatch) {
      throwError(baseElement, `Cannot detect dynamic formset prefix: ${baseElement.name}`);
    }

    return fieldName.replace("__prefix__", prefixMatch[0]);
  }

  function getRestrictions(baseElement) {
    let restrictionsJson = baseElement.getAttribute("data-restrictions");
    if (!restrictionsJson) {
      throwError(baseElement, "data-restrictions is undefined");
    }

    return JSON.parse(restrictionsJson);
  }

  function updateOptionList(baseElement) {
    if (isFormsetTemplate(baseElement)) {
      return;
    }

    let refField = getRestrictedOnField(baseElement);
    if (!refField) {
      throwError(baseElement, "Could not find refField");
    }

    let restrictions = getRestrictions(baseElement);

    let options = Array.prototype.slice.apply(baseElement.querySelectorAll("option"));
    options.forEach(option => {
      if (!option.value) {
        option.hidden = false;
        return;
      }

      let allowedOnValues = restrictions[option.value] || [];
      option.hidden = allowedOnValues.indexOf(refField.value) < 0;
    });
  }

  function clearOptionList(baseElement) {
    let options = Array.prototype.slice.apply(baseElement.querySelectorAll("option"));
    options.forEach(option => {
      option.hidden = true;
    });
  }

  document.addEventListener("change", event => {
    let element = event.target;
    getFieldsRestrictedOn(element).forEach(baseElement => {
      reset(baseElement);
      updateOptionList(baseElement);
      baseElement.dispatchEvent(new Event("change", {bubbles: true}));
    });
  });

  document.addEventListener("DOMContentLoaded", () => {
    getRestrictedFields().forEach(baseElement => {
      if (isFormsetTemplate(baseElement)) {
        clearOptionList(baseElement);
      } else {
        updateOptionList(baseElement);
      }
    });
  })
})();

And we make changes to the already existing dropdown/admin.py file:

from django.contrib import admin

from .forms import OrderItemForm
from .models import (AllowedCombination, Category, Good, Order, OrderItem,
                     Subcategory)

admin.site.register(Category)
admin.site.register(Subcategory)
admin.site.register(Good)


class OrderItemInline(admin.TabularInline):
    model = OrderItem
    form = OrderItemForm
    extra = 0


class OrderAdmin(admin.ModelAdmin):
    inlines = [OrderItemInline]


admin.site.register(Order, OrderAdmin)


class OrderItemAdmin(admin.ModelAdmin):
    form = OrderItemForm


admin.site.register(OrderItem, OrderItemAdmin)


class AllowedCombinationAdmin(admin.ModelAdmin):
    list_display = ['cat', 'subcat', 'good', ]


admin.site.register(AllowedCombination, AllowedCombinationAdmin)

Ready! Now in our Admin panel, when filling in the Order table, we can use the Dependent Dropdown Lists:

And the general list of files of our project now looks like this:

.
├── db.sqlite3
├── dropdown
│   ├── admin.py
│   ├── apps.py
│   ├── forms.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   ├── models.py
│   ├── __pycache__
│   │   ├── admin.cpython-38.pyc
│   │   ├── apps.cpython-38.pyc
│   │   ├── __init__.cpython-38.pyc
│   │   └── models.cpython-38.pyc
│   ├── tests.py
│   └── views.py
├── main
│   ├── asgi.py
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-38.pyc
│   │   ├── settings.cpython-38.pyc
│   │   ├── urls.cpython-38.pyc
│   │   └── wsgi.cpython-38.pyc
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── static
│   └── js
│       └── restricted-model-choice-field.js
└── venv

More details about the creation of this project are described (and, most importantly, shown!) In this video (RU voice!):

Tags list

Apps Script Arrays Java Script asynchronous code asyncio coroutine Django Dropdown List Drop Shipping Exceptions GitHub Google API Google Apps Script Google Docs Google Drive Google Sheets multiprocessing Parsing Python regex Scraping ssh Test Driven Development (TDD) threading website monitoring zip

Blog

6. Adding a Scraping (Parsing) Module and Autofilling the Database Directly From Another Website!

How to get information from another site?

Web page structure

Action plan

Create a scraping (parsing) module and get the HTML code using the requests package

Processing the resulting HTML code with the BeautifulSoup package

Generating an additional link to go to the detail page and getting the product code

Adding error handling

Saving the received data in the database

Transferring scraping control to the site page

5. Creation of Database Functionality Using TDD (Test Driven Development)

Why do programmers write tests?

Requirements for the structure of the database

Creating a database structure

Database business logic requirements

Write tests (transfer the requirements for the database to the test code)

Writing code that passes these tests

4. Registering and Authorizing Users on a Django Site

Registration and Authorization in Django

A Little bit of Theory

The Concept of Session and Cookies

Model User

POST and GET Requests

Forms, csrf-token

Passing URL to Other Applications, register.html and login.html Pages

Creating a Form in the form.py Module

Getting Data From HTML Page

Handling Data Entry Errors

Creating a Registration View Using Generic Views

Forms of the ModelForm Class

Embedding a Form in HTML-code

3. Adding an HTML-Template and Static Files to a Django Project. Creating a Base and Main Pages

Adding an HTML-Template and Static Files to a Django Project

2. Project Planning. Select and Update HTML-template

Planning

Where can I get an HTML template for a project?

Planning the pages of the future template

Website functionality planning

Template update

1. Introduction. Who is this course for? Preparing the Development Environment

Introduction

Who is this course for?

Preparing the Development Environment

What's the idea?

Why Django?

Preparing the project and setting up the development environment

1. Change settings.py

2. Create a database (DB)

3. Preliminary configuration of the Admin panel (administrative panel of the Django framework)

4. Filling the database

5. Make Lists Dependent

Tags list