Blog

Drop Shipping Online Store on Django (Part 7)

7. Data Visualization Using Forms and Views

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

Add pages: shop.html, shop-details.html and cart.html

Our project is nearing completion, from which we are literally a few last steps away.

The most important functional part, which accounts for all the basic operations with the database, was completed in the 5th lesson. Those, in fact, we already have all the business logic and we just have to add a user interface for displaying database data and changing them.

To warm up, let's first do what we already know well - copy the last 3 pages from the template to the templates/shop folder that are still there: shop.html, shop-details.html and cart.html. And let's try to open them in the project.

To do this, first of all, we will remove from them what is already in the base template. Next, add links to these pages to the shop/urls.py configurator.

urlpatterns = [
    path('fill-database/', views.fill_database, name='fill_database'),
    path('', TemplateView.as_view(template_name='shop/shop.html'), name='shop'),
    path('cart_view/',  
         TemplateView.as_view(template_name='shop/cart.html'), name='cart_view'),
    path('detail/<int:pk>/',  
         TemplateView.as_view(template_name='shop/shop-details.html'), 
         name='shop_detail')
]

By the way, pay attention: not only links are shown here, but also the views themselves! Before us is an example of that rare case when the view can not be imported from the views.py module. The point is that template_name (the only required parameter for TemplateView) can be specified in the as_view() method as a kwargs argument. And then the view will “run” directly from the shop/urls.py configurator!

And the final touch - let's add links to the SHOP and CART pages to the main menu in order to immediately check if we made any mistakes in the layout when embedding the base template. There is nowhere to add a link to the detail page yet. Therefore, to check this page for errors, it will have to be called “manually”.

New Generic View Types: ListView and DetailView

We started our acquaintance with the generic view in the 4th lesson, and already then we were convinced how powerful and concise Django's tool is. And from the example above, we also learned that TemplateView can generally be written in one line directly in the shop/urls.py configurator.

Two other representatives of this category, namely: ListView and DetailView, have the same amazing properties. In the simplest version of the view, you can specify only the names of the model and template. And this will be enough for the html template to get all the necessary information and be able to display all the rows of the Products table (ListView), or all the fields of a single product whose pk will be listed at the end of the url (DetailView).

(To be fair, it's not even necessary to specify a template name - by default, this name is already included in the generic view settings. More information can be found here: Class-based views)

Creating ProductsListView

When going to the product page via the link /shop/, the user expects to see the entire list of products. Therefore, we change the already created test view TemplateView, which was created solely to test the template, with a new one - ProductsListView:

from django.views.generic import ListView

from shop.models import Product


class ProductsListView(ListView):
    model = Product
    template_name = 'shop/shop.html'

Do not forget to also make changes to the shop/urls.py configurator:

urlpatterns = [
    path('', views.ProductsListView.as_view(), name='shop'),
    path('cart_view/',
         TemplateView.as_view(template_name='shop/cart.html'), name='cart_view'),
    path('detail/<int:pk>/',
         TemplateView.as_view(template_name='shop/shop-details.html'),
         name='shop_detail'),
    path('fill-database/', views.fill_database, name='fill_database'),
]

By default, all data from the Product model will automatically be passed to the template as an object_list object. Therefore, by creating a cycle through object_list in the html template, we will get access to all product products, which means we can get the values of all fields of interest to us:

# one block
                {% for product in object_list %}
					<div class="col-12 col-lg-4 col-md-6 item">
                        <div class="card" style="width: 18rem;">
                            <form method="post" action="">
                                <img src="{{product.image_url}}" class="card-img-top" alt="...">
                                <div class="card-body">
                                    <h5 class="card-title"><b>{{ product.name}}</b></h5>
                                    <p class="card-text">
                                        {{ product.description }}
                                    </p>
                                </div>
                                <ul class="list-group list-group-flush">
                                    <li class="list-group-item">Price: {{ product.price }}</li>
                                    <li class="list-group-item">
										{% csrf_token %}
										<label class="form-label" for="id_quantity">Quantity:</label>
										<input type="number" name="quantity" value="1" min="1"
											   required id="id_quantity"/>
                                    </li>
                                </ul>
                                <div class="card-body">
                                    <button class="learn-more-btn" type="submit">buy now</button>
                                    <a class="contactus-bar-btn f_right" href="">detail</a>
                                    <br><br>
                                </div>
                            </form>
                        </div>
                    </div>
                {% endfor %}

The shop/shop.html template now contains 4 identical test blocks. Replacing one of them with the proposed option, we will get a complete list of all blocks with all the values of the product table.

The same concise and elegant solution exists in Django for displaying a single selected object. Only now it inherits not ListView, but DetailView:

class ProductsDetailView(DetailView):
    model = Product
    template_name = 'shop/shop-details.html'

And don't forget to change the name of the view in shop/urls.py:

urlpatterns = [
    path('', views.ProductsListView.as_view(), name='shop'),
    path('cart_view/',
         TemplateView.as_view(template_name='shop/cart.html'), name='cart_view'),
    path('detail/<int:pk>/', views.ProductsDetailView.as_view(),
         name='shop_detail'),
    path('fill-database/', views.fill_database, name='fill_database'),
]

Now that the actual product list is displayed, we can add a link to the detail page to the product loop in the shop/shop.html template:

<div class="card-body">
	<button class="learn-more-btn" type="submit">buy now</button>
	<a class="contactus-bar-btn f_right" href="{% url 'shop_detail' product.pk %}">detail</a>
	<br><br>
</div>

Pay attention to how a new compound link {% url 'shop_detail' product.pk %} is created: after a space, from the name url comes the product number in the database - product. pk. Of course, this result also needs to be verified.

Adding the selected item to the cart

So we have come to one of the most crucial moments - to filling the cart with the selected product. Of course, this logic can also be implemented using generic view. Moreover, this method is considered preferable, because, as you know, the more complex the task, the more understandable and concise the generic view code looks, compared to the code of a regular function.

But, this option may not be very clear for beginners. Therefore, to solve this problem, let's return to functions and forms again.

Let's start with the form. We have to fill in the OrderItem table, which is connected by ForeignKey to the Product and Order tables (models). Therefore, in fact, the only unknown field that we have to enter is the Quantity field, and we can take all other data from other tables. Therefore, our AddQuantityForm form will consist of only one field:

from django import forms

from shop.models import OrderItem


class AddQuantityForm(forms.ModelForm):
    class Meta:
        model = OrderItem
        fields = ['quantity']

Now back to the view, which we will call add_item_to_cart. And here our task is extremely simplified - we do not need to create a GET request. This already does the view ProductListView. Therefore, all that is required from the add_item_to_cart view is to receive and process the POST request:

@login_required(login_url=reverse_lazy('login'))
def add_item_to_cart(request, pk):
    if request.method == 'POST':
        quantity_form = AddQuantityForm(request.POST)
        if quantity_form.is_valid():
            quantity = quantity_form.cleaned_data['quantity']
            if quantity:
                cart = Order.get_cart(request.user)
                # product = Product.objects.get(pk=pk)
                product = get_object_or_404(Product, pk=pk)
                cart.orderitem_set.create(product=product,
                                          quantity=quantity,
                                          price=product.price)
                cart.save()
                return redirect('cart_view')
        else:
            pass
    return redirect('shop')

As you can see, if the form has passed validation, then the quantity object is created. We also remember that all OrderItem objects do not exist on their own, but are necessarily tied to some kind of cart or order. The get_cart method that we have already created in the previous lessons is able to provide us with the desired cart - the cart object. The product object, whose quantity we just confirmed, is easily obtained by request for pk=pk. By the way, product can be done using the get() method, but the get_object_or_404() variant is considered more reliable, which can handle a 404 error if the object with the desired pk will not be in the database.

Thus, we have already received all the fields necessary to create a new object. Therefore, now using cart.orderitem_set.create() we create a new model object OrderItem, and using the cart.save() method, we fix the connection of this object with an order basket.

The last thing left for us now is to add a new url to the configurator:

path('add-item-to-cart/<int:pk>', views.add_item_to_cart, name='add_item_to_cart'),

and then add this new url to the action attribute of the form tag on the shop/shop.html page:

<form method="post" action="{% url 'add_item_to_cart' product.pk %}">

Now you're ready to add orders to your shopping cart. True, we can only see the result in the admin panel.

And a very important touch, which almost remained behind the scenes. All users should be able to look at the product catalog. But only registered users can choose a product and add it to the cart. To solve the problem of protecting the add_item_to_cart view from unauthorized access by unauthorized users, the decorator @login_required will help. As you can see, this decorator will automatically redirect an unlogged user to the 'login' login page.

Cart management: display a list of items

Конечно же, добавленные в корзину позиции хотелось бы видеть не только в админке. Тем более, что у нас уже всё для этого готово. Кроме вью. Им и займёмся.

По сути, для отображения элементов корзины необходимо получить данные двух моделей:

  • Модели заказа Order (из которой с помощью метода get_cart(user) получаем объекта cart)
  • И модели OrderItem (для order=cart)

И затем передать эти данные в шаблон с помощью словаря context:

From the cart object in the template, only data related to the cart itself will be retrieved (in our case, only the order amount). We will get the data for each position of the item order as a result of the loop over the items object:

{% for item in items %}
    <div class="row">
        <div class="col-12 col-md-1 item">
            &nbsp;&nbsp;&nbsp;{{ forloop.counter }}
        </div>
        <div class="col-12 col-md-4 item">
            {{ item.product }}
        </div>
        <div class="col-12 col-md-2 item">
            {{ item.quantity }}
        </div>
        <div class="col-12 col-md-2 item">
            {{ item.price }}
        </div>
        <div class="col-12 col-md-2 item">
            {{ item.amount }}
        </div>
        <div class="col-12 col-md-1 item">
        </div>
    </div>
{% endfor %}

Cart Management: Deleting Items

Ошибаться может каждый. Поэтому пользователь должен иметь возможность удалять лишние позиции из корзины.

Как мы уже хорошо усвоили - все изменения базы данных должны проходить только через форму и метод POST. И удаление позиции в том числе.

Здесь стоит отметить, что для удаления элементов модели в Django имеется очень удобное generic view - DeleteView, для которого не нужно ни создавать отдельную форму в модуле shop/forms.py, ни специально описывать метод POST. Всё это уже создано в DeleteView по умолчанию:

@method_decorator(login_required, name='dispatch')
class CartDeleteItem(DeleteView):
    model = OrderItem
    template_name = 'shop/cart.html'
    success_url = reverse_lazy('cart_view')

    # Проверка доступа
    def get_queryset(self):
        qs = super().get_queryset()
        qs.filter(order__user=self.request.user)
        return qs

The only thing we have added here (changed, to be more precise) is the get_queryset method, which filters the OrderItem model data request by user.

We also do not need any additional output using the GET method: as in the case of adding a position to the cart, we will use the ready-made data that cart_view kindly provides us.

All that remains for us is to add a form to EVERY (!!!) position of the cart (fortunately, they are all displayed in a loop anyway) and a new url to call a new view CartDeleteItem.

Changes in the shop/cart.html template:

<div class="col-12 col-md-1 item">
    <form method="post" action="{% url 'cart_delete_item' item.pk %}">
        {% csrf_token %}
        <button type="submit" style="color: blue"><u>delete</u></button>
    </form>
</div>
Добавление в shop/urls.pyl:
path('delete_item/<int:pk>', views.CartDeleteItem.as_view(), name='cart_delete_item'),

Cart management: proceed to create an order

After everything you need has been successfully added to the cart, and everything superfluous has been safely removed from it, all that remains for us is to complete the recruitment process, change the cart status from STATUS_CART to STATUS_WAITING_FOR_PAYMENT and thus proceed to pay for the order.

The make_order method itself has long been created by us. It remains only to add a button to the basket sheet, upon pressing which this method will be launched.

The task for the view will be very simple - find the desired basket and apply the make_order method to it:

@login_required(login_url=reverse_lazy('login'))
def make_order(request):
    cart = Order.get_cart(request.user)
    cart.make_order()
    return redirect('shop')

After changing the status, there will be a redirect to the shop/shop.html page. After connecting online payment, this redirect can be replaced by a transition to the payment aggregator page. And you will also need to remember to add a new link to shop/urls.py:

path('make-order/', views.make_order, name='make_order'),

And add this link to the button on the cart page:

<a class="contactus-bar-btn f_right" href="{% url 'make_order' %}">
    Process to Payment
</a>

Conclusion

Our project has been completed. Of course, a lot was left behind the scenes: confirmation of registration by email, logging, connecting online payment, deployment, etc. etc.

However, the main functionality of the online store has been created. And improvement, as you know, never ends or ends.

In any case, if you have any questions, you know who to ask: it4each.com@gmail.com.

Good luck in creating your own online store and see you on new courses!

You can learn more about all the details of this stage from this video (RU voice):





Read more >>

Drop Shipping Online Store on Django (Part 6)

6. Adding a Scraping (Parsing) Module and Autofilling the Database Directly From Another Website!

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

How to get information from another site?

Of course, the most convenient (and, by the way, more reliable) way will be the interaction of our site with the provider site via API. Indeed, this requires

  • on the supplier site such an opportunity was organized (access to it via API),
  • and that the administration of the supplier site give us a login and password for this access.

The Django framework allows us to organize such interaction with your own means, without resorting to installing third-party packages. However, the Django REST ftamework (DRF) package will do the job best.

Nevertheless, in our case, we will use another method - reading and extracting the necessary information directly from the HTML page. This action is called scraping (parsing) of the site.

Two popular Python libraries will be used for this purpose: beautifulsoup4 and requests. You can install them using two terminal commands:

pip install beautifulsoup4
pip install requests

Web page structure

Typically, data on a product page is grouped into blocks. Inside the blocks, the same type of data is under the same selectors (see figure):

If we download and parse an HTML page with a list of products, we can get a structured list of data. Specifically for our case, for each data block, we need to get the following dictionary:

{
    'name': 'Труба профильная 40х20 2 мм 3м', 
    'image_url': 'https://my-website.com/30C39890-D527-427E-B573-504969456BF5.jpg', 
    'price': Decimal('493.00'), 
    'unit': 'за шт', 
    'code': '38140012'
 }

Action plan

  • Create scraping.py module in shop app
  • In this module, create a scraping() function that can:
    1. Get page HTML code (by package request)
    2. Process the resulting HTML code (by package beautifulsoup4)
    3. Save result in database
  • Test the function scraping() “manually”
  • Add a button to start scraping on the site page

The plan is ready - let's start its implementation!

Create a scraping (parsing) module and get the HTML code using the requests package

Obviously, the script responsible for reading information from another site should be placed in a separate shop application module: shop/scraping.py. The scraping() function will be responsible for sending a request to URL_SCRAPING, reading data and writing this data to the Product table of the project database.

First of all, we need to get the HTML code of the product data page for further processing. This task will be assigned to the requests module:

import requests

def scraping():
    URL_SCRAPING = 'https://www.some-site.com'
    resp = requests.get(URL_SCRAPING, timeout=10.0)
    if resp.status_code != 200:
        raise Exception('HTTP error access!')

    data_list = []
    html = resp.text

It makes sense to immediately see what you got. To do this, let's add code that will count the number of characters in the html object and at the same time print this object itself:

html = resp.text
    print(f'HTML text consists of {len(html)} symbols')
    print(html)


if __name__ == '__main__':
    scraping()

The shop/scraaping.py module does not require any Django settings (at least not yet), so you can run it like a regular Python script:

HTML text consists of 435395 symbols
<!DOCTYPE html>
<html lang="ru">
  <head>
    <link rel="shortcut icon" type="image/x-icon" href="/bitrix/templates/elektro_light/favicon_new.ico"/>
    <meta name="robots" content="index, follow">
<meta name="keywords" content="Профильные трубы, уголки">
<meta name="description" content="Цены на профильные трубы, уголки от  руб. Описание. Характеристики. Отзывы. Скидки на  профильные трубы, уголки.">
    <meta name="viewport" content="width=device-width, initial-scale=1.0 user-scalable=no"/>
    <meta name="msapplication-TileColor" content="#ffffff">

As you can see, the result really looks like an HTML page.

The first part of the task is solved - access to the site data is obtained, and those 435,395 characters that are displayed on the screen contain all the information we need. All we now need is to simply extract this information and store the result in the database.

Processing the resulting HTML code with the BeautifulSoup package

Further processing will be most conveniently carried out using the beautifulsoup4 module. To do this, we first need to create a soup object, which is a nested data structure of an HTML document:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'html.parser')

More information on how to get started with this package can be found on the man page: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#quick-start

You can also read more about the beautifulsoup4 CSS selectors here: https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors Further on the supplier's page, we will be most interested in the product block - layout of repeating product cards with a similar data structure. You can get a list of elements of the same type from the soup object using the select() method, where the CSS selector of this block is specified as an argument. In our case it will be class=”catalog-item-card”:

blocks = soup.select('.catalog-item-card ')

In the loop, we can access each block and at the same time see what is inside the block object. This is how the modified code will look like:

html = resp.text

    soup = BeautifulSoup(html, 'html.parser')
    blocks = soup.select('.catalog-item-card ')

    for block in blocks:
        print(f'HTML text consists of {len(block.text)} symbols')
        print(50 * '=')
        print(block.text)
        break

And this is how the printed block.text object will look like:

HTML text consists of 382 symbols
==================================================
<div class="catalog-item-card" itemprop="itemListElement" itemscope="" itemtype="http://schema.org/Product">
<div class="catalog-item-info">
<div class="item-all-title">
<a class="item-title" href="/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/" itemprop="url" title="Труба профильная 40х20 2 мм 3м">
<span itemprop="name">Труба профильная 40х20 2 мм 3м</span>
</a>

As you can see, the number of characters in the block has been reduced to 382. Which greatly simplifies our task.

We can parse these blocks into elements of interest to us using the soup.select_one() method, which, unlike the select() method, does not select all elements of the page, that satisfies the condition (method argument), but only the first matched element. It is also important to remember that the text obtained with the soup.select_one() object can be extracted using the text method. Thus, applying this method with certain arguments, we fill almost the entire data dictionary, with the exception of the code field:

soup = BeautifulSoup(html, 'html.parser')
    blocks = soup.select('.catalog-item-card ')

    for block in blocks:
        """{
        'name': 'Труба профильная 40х20 2 мм 3м', 
        'image_url': 'https://my-website.com/30C39890-D527-427E-B573-504969456BF5.jpg', 
        'price': Decimal('493.00'), 
        'unit': 'за шт', 
        'code': '38140012'
        }
        """
        data = {}
        name = block.select_one('.item-title[title]').get_text().strip()
        data['name'] = name

        image_url = URL_SCRAPING_DOMAIN + block.select_one('img')['src']
        data['image_url'] = image_url

        price_raw = block.select_one('.item-price ').text
        # '\r\n \t\t\t\t\t\t\t\t\t\t\t\t\t\t493.00\t\t\t\t\t\t\t\t\t\t\t\t  руб. '
        price = re.findall(r'\S\d+\.\d+\S', price_raw)[0]
        price = Decimal(price)
        data['price'] = price   # 493.00

        unit = block.select_one('.unit ').text.strip()
        # '\r\n \t\t\t\t\t\t\t\t\t\t\t\t\t\tза шт\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'
        data['unit'] = unit  # 'за шт'

Generating an additional link to go to the detail page and getting the product code

Upon closer examination of the product block, it turned out that some of the information on the product is located on another page of the supplier's website - on the product detail page. The link itself to go to this page is in the block block.

Therefore, we will have to repeat here the same algorithm that we used a few steps ago to get data for the block block:

  • Generate link to detail page
  • Follow this link and read with requests.get() the new HTML code of this page already - the detail page
  • Save the received data in a new object Beautiful Soup
  • Extract the code number using the same method soup.select_one()

# find and open detail page
        url_detail = block.select_one('.item-title')
        # <a class="item-title" href="/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/" itemprop="url" title="Труба профильная 40х20 2 мм 3м">

        url_detail = url_detail['href']
        # '/catalog/profilnye_truby_ugolki/truba_profilnaya_40kh20_2_mm_3m/'

        url_detail = URL_SCRAPING_DOMAIN + url_detail

        html_detail = requests.get(url_detail).text
        soup = BeautifulSoup(html_detail, 'html.parser')
        code_block = soup.select_one('.catalog-detail-property')
        code = code_block.select_one('b').text
        data['code'] = code

        data_list.append(data)

        print(data)

If we do everything right, we will end up with a list of dictionaries with data for each block.

Adding error handling

The success of site scraping depends on some parameters and circumstances. And most of them do not depend on our Django code, namely:

  • Availability (or inaccessibility of the provider site)
  • Changing page layout
  • Internet connection problems
  • and so on…

The success of site scraping depends on some parameters and circumstances. And most of them do not depend on our Django code, namely:

  • Availability (or inaccessibility of the provider site)
  • Changing page layout
  • Internet connection problems
  • and so on…

class ScrapingError(Exception):
    pass


class ScrapingTimeoutError(ScrapingError):
    pass


class ScrapingHTTPError(ScrapingError):
    pass


class ScrapingOtherError(ScrapingError):
    pass

And then we make changes to the code:

try:
        resp = requests.get(URL_SCRAPING, timeout=10.0)
    except requests.exceptions.Timeout:
        raise ScrapingTimeoutError("request timed out")
    except Exception as e:
        raise ScrapingOtherError(f'{e}')

    if resp.status_code != 200:
        raise ScrapingHTTPError(f"HTTP {resp.status_code}: {resp.text}")

Saving the received data in the database

As you can see, the product is added only if it is not already in the database. The search is performed by the product code number (field value code ).

Despite the fact that in the scraping.py function itself, the data is already written to the database, we still return the data_list list. Just in case).

However, if we now try to reproduce this script, we will get an error:

"/home/su/Projects/django-apps/Projects/drop-ship-store/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 67, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Process finished with exit code 1

The thing is that now the script accesses the database, which means that you need to get the Django settings. You can run this code to check in management/commands (more on this can be found here: https://docs.djangoproject.com/en/4.0/howto/custom-management-commands/) But we will do otherwise: we will immediately add the launch page and check the operation of the scraping() function already there.

Transferring scraping control to the site page

The algorithm for adding a new page remains the same:

  • Comes up with a url that will call it (shop/fill-database/)
  • Add urls.py configurator to shop
  • application
  • Set urls.py to link url and view (path('fill-database/', views.fill_database, name='fill_database'),
  • Move (copy) the file from the template to the project
  • Create a view in the module shop/views.py
  • Checking the result!

If, after the successful completion of all these points, we go to the admin panel, we will see that, after running the script, the Product table is filled with new values:

Now everything is ready for the last step: adding the pages of the online store directly and the code that will manage them. But we will deal with this in the next seventh and last lesson.

You can learn more about all the details of this stage from this video (RU voice):




To the next stage of the project



Read more >>

Drop Shipping Online Store on Django (Part 5)

5. Creation of Database Functionality Using TDD (Test Driven Development)

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

Why do programmers write tests?

Most programmers don't like to write tests.

It is understandable - few people like to do double work: first write and debug code, and then spend the same amount of time (if not more!) on writing tests.

Well, if the project will be supported further. Then the time spent on writing tests is guaranteed to pay off: it will be possible to make sure that creating a new one did not break the old one in seconds.

However, if the project is closed, then the cost of tests will be added to the loss of time for writing the main code.

So writing tests is a bit like investing - in principle, a good thing, but risky. Therefore, to write tests or not to write - everyone chooses for himself, depending on each specific situation.

Nevertheless, there is an approach in programming that allows you, if not to win, then at least almost not to lose when writing tests. And this approach is called "Test Driven Development" (TDD) ("Development through testing"). The idea is very simple: we FIRST write tests, and only then we write the code itself, which must pass these tests.

And what is the advantage here? And the plus is that in this case, the time spent on checking the code is completely eliminated. After all, even the very first run of a freshly written code will be done "not by hand", but by tests!

And an additional bonus of this approach is that the check itself becomes more systematic, and therefore more reliable. Because what and how to test is thought out in advance, and not done impromptu. Which drastically reduces the chances of missing something important.

Actually, this is why Django has a well-developed code testing toolkit. And that is why already in the first training example on the website of this framework, a whole lesson out of eight is devoted to the consideration of an example of the implementation of this principle: "Development through testing" or "Test Driven Development" (TDD).

In our training mini-course, we will consider the test-driven development approach when creating a database, since the main computational load or the main business logic will be located in the shop/models.py module, which is responsible for the interaction between database tables data.

So, our tasks at this stage:

  • Formulate the basic requirements for the structure of the database
  • Create a database design (draw tables and relationships between them)
  • Formulate requirements for the business logic of the database
  • Implement these requirements into test code
  • Write code that passes these tests

Requirements for the structure of the database

To simplify the task (after all, this is a mini-course!) we will stipulate three assumptions:

  • Our products are flat (no categorization)
  • We have dropshipping, and therefore there is no warehouse and there can be no stock leftovers (an infinite stock of goods for each item is assumed)
  • Our cart is an order with the status cart. Just after clicking the “Proceed to payment” button, this status changes to waiting_for_payment

Creating a database structure

This block diagram satisfies all the above requirements:

Therefore, in the shop/models.py module, we create the following structure:

class Product(models.Model):
    name = models.CharField(max_length=255, verbose_name='product_name')
    code = models.CharField(max_length=255, verbose_name='product_code')
    price = models.DecimalField(max_digits=20, decimal_places=2)
    unit = models.CharField(max_length=255, blank=True, null=True)
    image_url = models.URLField(blank=True, null=True)
    note = models.TextField(blank=True, null=True)

class Payment(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    amount = models.DecimalField(max_digits=20, decimal_places=2, blank=True, null=True)
    time = models.DateTimeField(auto_now_add=True)
    comment = models.TextField(blank=True, null=True)

class Order(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    # items = models.ManyToManyField(OrderItem, related_name='orders')
    status = models.CharField(max_length=32, choices=STATUS_CHOICES, default=STATUS_CART)
    amount = models.DecimalField(max_digits=20, decimal_places=2, blank=True, null=True)
    creation_time = models.DateTimeField(auto_now_add=True)
    payment = models.ForeignKey(Payment, on_delete=models.PROTECT, blank=True, null=True)
    comment = models.TextField(blank=True, null=True)

class OrderItem(models.Model):
    order = models.ForeignKey(Order, on_delete=models.CASCADE)
    product = models.ForeignKey(Product, on_delete=models.PROTECT)
    quantity = models.PositiveIntegerField(default=1)
    price = models.DecimalField(max_digits=20, decimal_places=2)
    discount = models.DecimalField(max_digits=20, decimal_places=2, default=0)

Чтобы объекты этих классов понятно отображались при отладке и в админке, полезно будет добавить в каждый класс методы __str__, а также упорядочить все записи в таблицах по порядку, например:

class Meta:
        ordering = ['pk']

    def __str__(self):
        return f'{self.user} --- {self.time.ctime()} --- {self.amount}'

The final step in creating a database is to create migrations and apply them:

python manage.py makemigrations
python manage.py migrate

And the final touch is to add control and display of the database to the admin panel:

from django.contrib import admin

from shop.models import Product, Payment, OrderItem, Order

admin.site.register(Product)
admin.site.register(Payment)
admin.site.register(OrderItem)
admin.site.register(Order)

Database business logic requirements

  1. When a user selects a product, the product should be added in the selected quantity to the Cart, which should be created automatically (unless, of course, it has not already been created by this moment)

  2. A cart that has not changed to an Order status within 7 days should be automatically deleted on the first call to the get_cart(user) method

  3. At each new addition (deletion, change) of the quantity (or price) of goods, the total amount of the order should be automatically recalculated

  1. After completing the set/change of the Cart and proceeding to payment, the Cart should change its status (if it is not empty!) and become an Order awaiting payment (waiting_for_payment)

  2. The get_unpaid_orders(user) method is required, which will allow you to get the total amount of unpaid orders (status=waiting_for_payment) for the specified user

  3. The get_balance(user) method is required, which will allow you to get the balance on the account of the specified user

  1. Changing the status to waiting_for_payment automatically starts checking the balance of the current user. If the balance amount >= the amount of the order, then the Order changes its status to paid. This simultaneously creates a payment equal to (minus) the amount of the order (which immediately after payment reduces the balance of the client's account by the amount of the order)

  2. Making a payment automatically triggers a mechanism to check all outstanding orders, starting with the oldest one. If the payment amount entered is sufficient to pay for several orders awaiting payment, then all these orders change their status to paid. At the same time, payments equal to (minus) the amount of each order are formed (which immediately after payment reduces the balance of the client's account by the amount of orders)

As you can see, the list of checks is quite impressive. Even a single manual execution of all the listed tests is "very hard work", which can take 15 minutes, no less.

But we know well from our own experience that a rare code starts working without errors the first time. Consequently, manual testing would take hours of our time. And each new refactoring of the code is again hours for verification. Not to mention the fact that during manual testing, you can forget to check something, or forget to delete test values ​​from the database.

In general, as we will see for ourselves further, writing tests will take less time from the first time, which means that it will pay off its costs already on the first test run!

Write tests (transfer the requirements for the database to the test code)

When writing tests, keep the following in mind:

  • Application-level tests already have a ready-made module tests.py
  • Django tests are based on unittests
  • By default, before running EACH test, its own test database is created, which is immediately deleted after the test is completed.
  • It follows from this that a “just created” test database simply cannot physically have test values. This means that these database test values must be created each time with each new test.

  • It is most convenient to use fixtures for these purposes - special files (usually in json format) containing values ​​for the test database. The “just created” test database automatically loads these values ​​into itself before each test.
  • If you are not yet familiar with the concept of fixtures, then the easiest way to get acquainted with them is to manually populate the database and then dump these values ​​with the command: python manage.py dumpdata -o mydata.json.gz (more on this here: https://docs.djangoproject.com /en/4.0/ref/django-admin/#dumpdata)
  • Fixtures that contain data to populate the test database must be added to the application in the shop/fixtures/data.json
  • directory
  • You must specify
  • as an attribute of the test case class

fixtures = [
        "shop/fixtures/data.json"
    ]

  • You need to remember that a superuser must be created in the test database. Therefore, if the superuser is not in the fixtures, then it must be created in the setUp(self)
  • method

An example of the contents of the test file shop/tests.py is shown below:

from django.test import TestCase,
from shop.models import *

class TestDataBase(TestCase):
    fixtures = [
        "shop/fixtures/data.json"
    ]

    def setUp(self):
        self.user = User.objects.get(username='root')

    def test_user_exists(self):
        users = User.objects.all()
        users_number = users.count()
        user = users.first()
        self.assertEqual(users_number, 1)
        self.assertEqual(user.username, 'root')
        self.assertTrue(user.is_superuser)

    def test_user_check_password(self):
        self.assertTrue(self.user.check_password('123'))

Writing code that passes these tests

An example of a code snippet used in the lecture is shown below:

def change_status_after_payment(payment: Payment):
    """
    Calling the method after creating a payment and before saving it.
    First need to find total amount from all payments (previous and current) for the user.
    If total amount >= the order amount, we change status and create negative payment.
    """
    user = payment.user
    while True:
        order = Order.objects.filter(status=Order.STATUS_WAITING_FOR_PAYMENT, user=user) \
            .order_by('creation_time') \
            .first()
        if not order:
            break

        total_payments_amount = get_balance(payment.user)
        if order.amount > total_payments_amount:
            break

        order.payment = payment
        order.status = Order.STATUS_PAID
        order.save()
        Payment.objects.create(user=user, amount=-order.amount)


@receiver(post_save, sender=Payment)
def on_payment_save(sender, instance, **kwargs):
    if instance.amount > 0:
        # pay_for_waiting_orders(instance)
        change_status_after_payment(instance)


@receiver(post_save, sender=Order)
def on_order_save(sender, instance, **kwargs):
    if Order.objects.filter(status='2_waiting_for_payment'):
        pay_for_waiting_orders(instance)

You can learn more about all the details of this stage from this video (RU voice):




To the next stage of the project



Read more >>

Drop Shipping Online Store on Django (Part 4)

4. Registering and Authorizing Users on a Django Site

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

Registration and Authorization in Django

A Little bit of Theory

As has been repeatedly mentioned, the format of this course is purely practical. But without defining the concepts of session, cookies, GET- and POST requests it will be very difficult to understand the essence what we will do next. Therefore, a few words about this, nevertheless, are worth saying.

The Concept of Session and Cookies

How does the site "distinguish" "its" users from "strangers"?

Let's open the Storage → Cookies tab in the browser and try to register on a site. And we will see that at the time of registration, a new key session_id appears, which exists exactly as long as we save our authorization on this site. And it is this very session code that we can see in the django_session table if we open the database of this site.

However, if we log out, this session_id will disappear from both the database and browser cookies. The fact is that after the user authorizes on the site, the server creates a special digital label in its database, a copy of which it sends to the user's browser, and the browser stores this digital label in its memory. This is the session key, which is stored in the browser's memory. These data stored in the browser's memory are called cookies.

(Just in case, the SQLite browser mentioned in the video can be installed using these commands:
$ sudo add-apt-repository -y ppa:linuxgndu/sqlitebrowser
$ sudo apt-get update
$ sudo apt-get install sqlitebrowser
)

As long as the user is logged in to the site, the session_id in the cookies of his browser matches the session_id stored in the database. That allows the user to visit any pages of the site to which he has access.

To do this, the browser sends session_id there with each new request to the server, and as long as this code matches the code in the server database, the user remains authorized.

If the user logs out, the server will delete his session_id from its database. And on those pages of the site where an authorized login is required, this user will not be able to log in.

If the user logs in again, he will again be able to visit any page of the site. But it will already be a completely different session with a different session_id.

Model User

All tables in a Django database (DB) are described using models. These are special Python classes that inherit from the Model class.

One model describes one database table. And each line of this model describes one field of the table.

In more detail we will analyze examples of models in the next 5th lesson. For now, let's just take note that the User user model, which has a certain set of basic fields and methods, already exists by default. It is always created in Django during the first migration. In particular, there are already fields username, email, password, is_staff (whether the user is an employee) and < b>is_superuser.

POST and GET Requests

New unfamiliar abbreviations usually inspire fear. To get rid of at least half of this fear, we immediately note that all the requests that we have talked about so far are GET requests. Really, isn't it scary at all? Then let's move on!

So, when accessing the server, different types of requests are possible. In general, the HTTP protocol has a lot of them, but in this course we will use only two:

  • GET request - when we just need to display some page
  • and POST request - when we need to send data to the server to store it in the database

To send a GET request, it is enough to specify a regular url. But the transmission of a POST request MANDATORY implies the presence of a form and a csrf token.

Forms, csrf-token

A form in HTML is a special construct that is enclosed in a <form></form> tag. This can also contain input fields with a single <input> tag, which is usually paired with a <label></label> tag, which explains that it is necessary to enter in the field input.

The form also contains a button (input field) with type="submit", clicking on which is exactly the command to send data to the server.

The current form looks like this in HTML code:

<form method="post" action="/your-url/">
    <label for="username">User name: </label>
    <input id="username" type="text" name="username" value="{{ username }}">
    <input type="submit" value="OK">
</form>

Passing URL to Other Applications, register.html and login.html Pages

In the previous lesson, we have already learned how to pass control from the configurator main/urls.py to the view. However, another option is also good practice: passing the url from the main configurator to another application's configurator.

To do this, in main/urls.py we will add a line, thanks to which all authorization requests starting with 'auth/' will be transferred for processing to another configurator - authentication /urls.py:

from django.contrib import admin
from django.urls import path, include

from company import views as views_company

urlpatterns = [
    path('', views_company.home_page, name='index'),
    path('about/', admin.site.urls),
    path('admin/', admin.site.urls),
    path('auth/', include('authentication.urls')),
]

Therefore, in authentication/urls.py there will be requests corresponding to authorization, registration and logout:

from django.urls import path

from authentication import views

urlpatterns = [
    path('login/', views.login_user, name='login'),
    path('register/', views.register, name='register'),
    path('logout/', views.logout_user, name='logout'),
]

Now, to begin with, we will make simple views that will be launched on these three requests in the authentication application:

def login_user(request):
    return render(request, ‘auth/login.html’)

def register(request):
    return render(request, ‘auth/register.html’)

def logout_user(request):
    pass

Nothing is clear yet about logging out - there is no page for it, so instead of the code we will simply write pass.

Well, actually, the template is not needed here. And you need a special logout function logout (request).

However, if we limit the view to just this function, we get an error: The view authentication.views.logout_user didn't return an HttpResponse object. It returned None instead.

Therefore, at the end of the view, we will have to return the transition to some page. For example, home. This can be done using the redirect('index') function, where 'index' is the address name in authentication/urls.

Creating a Form in the form.py Module

Obviously, now the construction for all views in the authorization application will not be as simple as it was for the index view. Because now we have a new task: to read the data from the HTML page, transfer it to the server and then to the User table in our database. And, as mentioned above, everything that is transferred to the database should only be transmitted using a form and a POST request.

To create forms in Django, a special forms.py module is provided, in which you can first create an "input form" object and describe its properties and characteristics in detail. The form is usually associated with a specific table in the database. Therefore, when creating a form, you should:

  • Indicate which table this form belongs to
  • Select a list of fields (you can select all fields and specify a specific list)
  • Define a list of required fields
  • Describe other field properties
  • and so on

Since we are using a ready-made Django's User table with a rather voluminous list of standard fields, it is imperative to specify in our form which fields will be used:

from django import forms
from django.contrib.auth.models import User


class LoginForm(forms.Form):
    username = forms.CharField()
    password = forms.CharField(widget=forms.PasswordInput())

The last line says that when entering a password, a special widget will be used that allows you to hide the symbol when filling out the field.

In the next lesson, we'll see that creating a form looks very similar to creating a database table.

Getting Data From HTML Page

Thus, the following options will need to be added to our login_user view:

  • Pass LoginForm to login.html
  • Ensure secure data entry in this form
  • Add a control button to the page, with which the user can send data to the server
  • Ensure data acceptance on the server, their validation
  • And, if necessary, add the received data to the appropriate database table

You can pass any data to the page using an additionally created context dictionary. In our version, by the login_form key, we will add an instance of the class of the newly created form LoginForm():

def login_user(request):
    context = {'login_form': LoginForm()}
    return render(request, 'auth/login.html', context)

This is quite enough to transfer data to an html page, but to receive data from the page, you will need to add data processing on a POST request.

def login_user(request):
    context = {'login_form': LoginForm()}

    if request.method == 'POST':
        login_form = LoginForm(request.POST)
        if login_form.is_valid():
            username = login_form.cleaned_data['username']
            password = login_form.cleaned_data['password']
            user = authenticate(username=username, password=password)
            if user:
                login(request, user)
                return redirect('index')

    return render(request, 'auth/login.html', context)

At the same time, on the html page, you will need to add:

  • Form tags <form></form>
  • In the form tag, be sure to specify the POST method and the url by which this POST request should be processed
  • Add <input> tag with name parameter equal to form field name
  • And finally, add the <input> tag (or <button>) with the type='”submit” parameter by click on which the user will be able to transfer data to the server.

That is, the form fragment in the HTML code will look something like this:

<form method="post" action="{% url 'login' %}">
    {% csrf_token %}
    <label class="form-label" for="id_username">Login</label>
    <input type="text" class="form-input" id="id_username"
                               name="username" placeholder="Login" required>
    <label class="form-label" for="id_password">Password</label>
    <input type="password" class="form-input" id="id_password"
                               name="password" placeholder="Password" required>
    <button type="submit" class="request-quote-btn">Log In</button>
</form>

The transfer of data from a user is a very sensitive operation with which a potential attacker can try to break into the server. Therefore, for additional security, in addition to the already existing session key, another secret key csrf_token is passed to the page. It is valid only for one data transfer from a particular form, and only for the data transfer of this particular form. If something goes wrong and a new form input is required, the server will generate a new csrf_token on a new request.

Handling Data Entry Errors

Filling out the form may be with errors - this is completely normal. It is not normal if the server "silently" does not accept this data and does not "explain" to the user why.

There are at least two ways.

1.) The simplest solution is to simply tell the user that the input is not correct. To do this, a new attention field is simply added to the context dictionary. In addition, it will be correct to return the form with exactly the data that did not pass.

if user:
    login(request, user)
    return redirect('index')
context = {
    'login_form': LoginForm(request.POST),
    'attention': f'The user with name {username} is not registered in the system!'}

2.) Another, more elegant and neat way: tell the user exactly what errors occurred during the validation process. And here you will need changes directly in the form itself:

def clean(self):
    cleaned_data = super().clean()
    username = cleaned_data.get('username')
    password = cleaned_data.get('password')

    try:
        self.user = User.objects.get(username=username)
    except User.DoesNotExist:
        raise forms.ValidationError(f'User with username: [{username}] does not exist!')

    if not self.user.check_password(password):
        raise forms.ValidationError('Could not log in using these email and password')

    return cleaned_data

Now these errors will be added to the LoginForm() form class instance. True, in order to read them on the authorization page, minor changes to the HTML code will also be required:

{% for error in login_form.non_field_errors %}
    <div class="alert alert-danger">{{ error }}</div>
{% endfor %}

At this point, the mission of creating an authorization page can be considered completed!

Creating a Registration View Using Generic Views

In addition to the view functions already well known to us, there are also view classes that inherit from generic views. Moreover, the latter is preferred when solving complex problems, since generic views look more compact and are easier to read.

We will start our acquaintance with the topic of generic views with only one option for now - the TemplateView class. (Later in this course, we will look at several more types of generic views).

As an example, let's rewrite our register function-based view into a RegisterView using the TemplateView class:

from django.views.generic import TemplateView

class RegisterView(TemplateView):
    template_name = 'auth/register.html'

This class has get and post methods to handle GET and POST requests:

def get(self, request):
        user_form = RegisterForm()
        context = {'user_form': user_form}
        return render(request, 'auth/register.html', context)

    def post(self, request):
        user_form = RegisterForm(request.POST)
        if user_form.is_valid():
            user = user_form.save()
            user.set_password(user.password)
            user.save()
            login(request, user)
            return redirect('index')

        context = {'user_form': user_form}
        return render(request, 'auth/register.html', context)

It is worth paying special attention to the method of creating a new user record in the User table, since we will continue to use this method further:

  • First, a user object is created - an instance of the model (class) User (but sometimes this can be done using the save() method class ModelForm).
  • Further (if necessary), this object can be changed / supplemented (in the example above, we store the string value of the password in the hash sum format).
  • But the new object is added directly to the database using another save() method, the Django model method.

The form RegisterForm, which is used here in the post method, should be mentioned separately.

Forms of the ModelForm Class

The previous LoginForm form was created based on the Form class, which is a fairly simple variant that looks very similar to the Model class of models.

Forms created with the ModelForm class are more advanced. Here we no longer need to prescribe each field of the form - just specify the name of the model that will be used in this form, and the list of required fields. After that, the format of the fields from the model attached to the form will automatically be transferred to the form itself.

New form example:

class RegisterForm(forms.ModelForm):
    class Meta:
        model = User
        fields = ('username', 'email', 'password')

Embedding a Form in HTML-code

In the previous LoginForm example, we attempted to embed a completed form into an HTML page using the {{ login_form }} tag. However, we quickly abandoned this idea - the styles of the standard Django form and the layout styles of our login.html page were too different.

But there is a solution. It is only necessary to display the fields not “at once”, but to do it gradually, in a cycle. Moreover, in this case, Django allows you to “break” the individual output for each form field into three components:

  • Output directly to the input field itself.
  • The field labels (title) associated with this field.
  • Input errors related to this field.

An example of HTML code that implements the above:

{% for field in user_form %}
<div class="row">
    <div class="col-12 col-lg-4 col-md-4 offset-md-4">
        <div class="quote-form-wrapper">
            <span style="color: red">{{ field.errors }}</span>
            {{ field.label_tag}}
            {{ field }}
        </div>
    </div>
</div>
{% endfor %}

You can learn more about all the details of this stage from this video (RU voice):




To the next stage of the project



Read more >>

Drop Shipping Online Store on Django (Part 3)

3. Adding an HTML-Template and Static Files to a Django Project. Creating a Base and Main Pages

Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

In this course, we will go through all the stages of creating a new project together:

Adding an HTML-Template and Static Files to a Django Project

So, our non-standard checklist (in the form of a modified HTML template) has been created and now it's time to start "crossing out" its items. Let's start with the main page.

First of all, you need to make sure that the tamplates directories for html files and static for static files (css, js, and image files) have been added to the project root. And that the paths of these directories relative to the base directory were added to the project settings file settings.py.

Actually, it is not difficult to make sure of this: the actual tree diagram of our project is presented on the slide below:

├── authentication
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── db.sqlite3
├── main
│   ├── asgi.py
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── requirements.txt
├── shop
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── static
├── templates
└── venv

Now we copy the static files: we copy the entire contents of the asserts folder of the HTML template to the static project folder. Next, we copy the main page, as long as it is (that is, without highlighting the common part into the base page base-page.html for now). Let's just copy the index.html template file into the templates folder of our project.

Next, we need to remember that in order to display a page in a Django project, at least 2 steps are required:

  • Specify the url of this page, by which it will be called in the project.
  • And create a special function (or class) called view that will run when the specified link is clicked and display our page.

  • Now in our project, in addition to the main folder, there are two more application folders: the shop folder and the authentication folder. All three of these folders already have, or can be added a couple of files urls.py - views.py. But only one of them: a pair of main from the main folder can provide a url that does not have the additional insertion of url fragments in the form of /auth/ or /shop/ . In other words, links (urls) from the main folder will lead to the "clean" address of the domain name.

    Of course, most often the home page was displayed at the address of a domain name. So we have decided on the location of the urls.py - views.py pair, it remains only to add the views file, which is still missing there, to the main folder. py. And then make the appropriate entries in these files:

    main/urls.py

    from django.contrib import admin
    from django.urls import path
    from shop import views
    
    urlpatterns = [
        path('', views.index, name='index'),
        path('admin/', admin.site.urls),
    ]

    Next, let's create a simple view, whose tasks will only include rendering (creating) the specified template. To create it, you can use a ready-made view file from one of our applications, or create a new views.py file in the main folder.

    main/views.py

    from django.shortcuts import render
    
    
    def index(request):
        return render(request, 'index.html')

    We try to run it, and we see that the main page is displayed, but the styles on it are completely absent. It's understandable: and links to styles should now be written in completely different paths (before, static files were in the asserts folder, and now it is in the static folder), and the way these links are written should also be completely different.

    • Firstly, every HTML page must now begin with a static load command {% load static %}.
    • And secondly, in all links, the path to all static files should now look different

    <link rel="stylesheet" href="../construct/assets/css/style.css">

    but like this:

    <link rel="stylesheet" href="{% static '/css/style.css' %}">

    We refresh the page, and we see that the picture has changed, but apart from the inscription Loading... nothing appears. So you need to update all links to the main page static files. And especially the js-files at the end of the page...

    Well, now you can see something. Certain styles have appeared, although not all. In order for all of them to appear, you need to sequentially "walk" through all the links and carefully and carefully change them. As you can see, you can't do without a minimum knowledge of HTML on the back-end.

    Extract Common Part of All Project Pages to Base Bage

    Well, in order not to get up twice, we immediately optimize our work on bringing statics in accordance with the new requirements for all pages.

    First of all, you need to pay attention to the fact that all pages of our HTML template have one common part, which includes the header (including the main menu) and footer. These elements are present on absolutely all pages. Therefore, it makes no sense to first duplicate the insertion of the same code, and then also suffer with changing links to static files. It will be much more reasonable if we separate this common part into a separate base page base-page.html, and leave insertion points for unique content from the project pages we need on this base page.

    Now every HTML page, except for the base one, will have exactly the same structure as the index.html page:

    {% extends 'base-page.html' %}
    {% load static %}
    
    {% block title %}
        < Page Name >
    {% endblock title %}
    
    {% block container %}
        < Current Page Content >
    {% endblock container %}

    That is, now when the page index.html is loaded, the base page base-page.html is loaded first (extends 'base-page.html') , then the statics for this page are loaded (load static), and then the current values ​​of the < Page Name > and < Current Page Content > blocks are added to the specified insertion points.

    All other pages will be similarly designed. For example, for the following page about-us.html, you need to do the same operations:

    • Add a new line with a new url to main/urls.py;
    • Add a new view to main/views.py;
    • And add the about-us.html page to the templates folder.

    main/urls.py

    from django.contrib import admin
    from django.urls import path
    from shop import views
    
    urlpatterns = [
        path('', views.index, name='index'),
        path('about/', views.about, name='about'),
        path('admin/', admin.site.urls),
    ]

    Next, let's create the simplest view, the tasks of which will only include rendering the specified template. To create it, you can use a ready-made view file from one of our applications, or create a new views.py file in the main folder.

    main/views.py

    from django.shortcuts import render
    
    
    def about(request):
        return render(request, 'about-us.html')

    And the last third step: add the about-us.html page to the templates folder. The modified HTML code of this page, in fact, will now consist only of the code that was previously located between the header and footer. And the structure of the page about-us.html will be no different from the page index.html.

    index.html

    {% extends 'base-page.html' %}
    {% load static %}
    
    {% block title %}
        < Page Name >
    {% endblock title %}
    
    {% block container %}
        < Current Page Content >
    {% endblock container %}

    Thus, after creating the base page, we can place in the templates of all other pages of the site only the original HTML code that is not in the base template. Of course, without forgetting to give a link to the base template. And that's it! This incredibly reduces code duplication and makes it easy to make changes to all pages of the site at once, just changing the code on the base page.

    You can learn more about all the details of this stage from this video (RU voice):




    To the next stage of the project



    Read more >>

    Drop Shipping Online Store on Django (Part 2)

    2. Project Planning. Select and Update HTML-template

    Attention! If you're having trouble following the previous step, you can visit relevant lesson , download the archive of the previous step, install it, and start this lesson exactly where the previous one ended!

    In this course, we will go through all the stages of creating a new project together:

    Planning

    In the last lesson, we prepared the development environment, created a new project in it, added the necessary settings to this project, and even created two new applications that will be responsible for registration/authorization and for the operation of the online store. In other words, we have completed the initial typical operations, which correspond to the preparatory stage of 99.9 (9)% of such projects. And now is the time to draw up a detailed plan of our further actions.

    Project planning is, in fact, not an easy task. For these purposes, even a special scientific discipline "Project management" has been created. Of course, the format of our mini-course doesn't allow us to delve too deeply into this. And, nevertheless, a few words about project management still need to be said.

    I think I won’t open America to anyone if I say that a project can be completed without any plan. It will just take longer and cost more. (If it works at all!)

    Therefore, the hallmark of a good plan is that in each of its paragraphs it answers at least 4 basic questions:

    • What exactly do we do?
    • How do we do it?
    • What resources (What resources are used to run the current step?)
    • When (By what date?)

    If there are answers to these questions for all points of the plan, we can be congratulated: we really have a plan! Well, if not, then we do not have a plan, but just a dream!

    But what if we are going to do something that has never been done before? And/or if not one of the four questions, not one answer?

    The answer is very simple: you need to highlight this unknown direction, "Terra incognito" in a separate plan. Plan of research work.

    But what about the destination, with the timing? After all, in research work it is very difficult to determine the value?

    And this problem is also solvable: where it is impossible to determine, you can always set a deadline that will not allow this research work to go to infinity.

    Well, then everything is simple: if we have found answers to all unknown questions BEFORE the deadline, then we move on to the second stage: we draw up the results of the research in the usual way and start work. Well, if we didn’t meet it, then it’s likely that we didn’t take up our business. Therefore, we inevitably disappear with this idea and begin to work on the outcome of another task.

    Fortunately, the problem we face in this mini-course is the most common and trivial one. Therefore, here you just need to carefully rewrite all the stages of our plan in the checklist, and, as you complete them, gradually cross them out, move them from line to line.

    However, in our mini-course we will go even further, and as such a "checklist" we will take a ready-made HTML template, where each HTML page will correspond to a stage in our plan. The order of work will be as follows: we transferred the page from the template to the project - consider crossing out the line in the checklist! And so on until we "delete" (crossing out) all HTML pages from the template!

    This idea seems completely absurd. But do not rush to draw conclusions!

    Where can I get an HTML template for a project?

    There are at least three answers to this question:

    1. Do it yourself,
    2. Order from professionals,
    3. Use the services of a store of ready-made templates.

    All of these options have their pros and cons:

    1. First option will require time, design skills and the skills of a HTML coder. That is, it will not suit everyone.
    2. The second option assumes that you have some financial reserve, and, oddly enough, a VERY clear understanding of what exactly you want to get in the end. It is important to remember here that the cost of a good front-end can be a sum with many zeros, and that it will be very difficult for a beginner to figure out where the production need ends and the waste of money begins. Moreover, good professionals are able not only to do well, but also to convince well)
    3. Therefore, the best option for everyone, and especially for startup owners, is to visit the online store of ready-made templates. Here you can not only see different options, but even touch (in the sense of "click"). The prices here are quite democratic, and there are even completely free options. For example, the free template that is used in our project was taken from this online store: Template Monster . True, at present this particular option is not available there, but still, there are a sufficient number of similar and free options. By the way, please note that only HTML5 templates will work for us.

    Planning the pages of the future template

    So, the selected template is downloaded from the Template Monster website and unpacked into a separate folder. If we open the index.html file in the browser, it may seem that we have a ready-made project in front of us: all (or almost all) buttons and links work and lead to the necessary pages. With one, however, significant caveat: all these beautiful pages are filled with not quite the information that we need. Therefore, right now we have to determine which pages we will leave, which ones we will delete, and which, on the contrary, we will add.

    Let's now decide on the set of pages of our future template. Our project will require the following pages:

    1. Home pag.
    2. Registration page.
    3. Authorization page (login)
    4. Product catalog page. Here, control will be added to select a product and place it in the basket in the required quantity
    5. Product detail page - page with more detailed information on the product
    6. Cart page - a list of products selected for purchase and payment.
    7. Scraping control page. Must be available only to company staff and NOT available to customers

    Obviously, all of these pages (except for the detail page) should have links in the main menu. And it is also desirable that the registration / authorization menu be separate from the main one.

    Website functionality planning

    The database will be as simple as possible (after all, our course is mini, not maxi).

    The product table will be “flat”, i.e. no categories/subcategories. The measure of the item will be “pieces”. The selected product with the set quantity is automatically added to the shopping cart. If there is no basket for this user yet, it is created, if it already exists, it is added.

    If the goods from the Cart are sent for payment, the Cart is cleared and the Order appears with the status "waiting for payment".

    To account for payments, a table of payments must be created. If the amount of the credited payment is greater than or equal to the amount of the order, then the Order changes its status to “paid”, and a “negative” payment with a “minus amount” is formed in the payment table itself. Thus, to determine the balance of the client's account, it will not be necessary to create a new balance table. In addition, this scheme will also solve the issue of online payment: when connecting an online payment aggregator, the system will only need to create a payment object equal to the amount of the payment received. This will be enough to automatically transfer the Order to the paid category.

    And we provide for work with bank payments:

    • After each payment, we automatically check for unpaid orders. If there are several of them, then we pay starting from the “oldest” one.
    • When creating an Order awaiting payment, we also check if there is a balance on the client's account. And if the amount of the balance is greater than or equal to the amount of the order, we automatically transfer the Order to the category of paid ones and subtract the amount of the order from the balance.

    We're done with the planning and moving on to turning the HTML template we just downloaded into the "checklist" of our project.

    Template update

    So, the selected template is downloaded from the Template Monster website and unpacked into a separate folder. If we open the index.html file in the browser, it may seem that we have a ready-made project in front of us: all (or almost all) buttons and links work and lead to the necessary pages. With one, however, significant caveat: all these beautiful pages are filled with not quite the information that we need. Therefore, right now we have to determine which pages we will leave, which ones we will delete, and which, on the contrary, we will add.

    1. Of course, save the home page index.html (HOME).
    2. You can also save the about-us.html (ABOUT) page. Since, according to legend, we are a construction company, then someday we will have to give information about ourselves. In this project, we will not post anything on this page, but we will leave it as a reserve for the future.

    On this, the choice of finished pages ended - the rest of the pages went through to collect with the help of "glue" and "scissors" from what was on the deleted pages.

    1. Registration Page register.html (Register);
    2. And login page login.html (Log In).

    Kstati, dlya ssylok na stranitsy registratsii/avtorizatsii bylo ispol'zovano gotovoye menyu vybora yazykov na glavnoy stranitsy. Ssylki na sleduyushchiye stranitsy byli razmeshcheny v glavnom menyu:

    1. Stranitsa otobrazheniya vsego spiska tovarov internet-magazina shop.html (SHOP). Tam zhe budet dobavlena vozmozhnoct' vybora tovara v nuzhnom kolichestve.
    2. Stranitsa detalizatsii vybrannogo tovara shop-details.html (punkta menyu na glavnoy stranitsy u neyo net - otkryvayetsya po knopke "Detail", raspolozhennoy ryadom s kazhdym tovarom.
    3. Stranitsa korziny cart.html (CART)
    4. I stranitsa zapuska skripta skreypinga (parsinga) tovarov so storonnego sayta fill-products.html (FILL-DATABASE)

    Nu, vot, kazhetsya i vso: vse osnovnyye momenty plana uchteny, i vse nuzhnyye stranitsy dobavleny.

    Show more 859 / 5,000 Translation results

    By the way, for links to the registration/authorization pages, a ready-made language selection menu on the main page was used. Links to the following pages have been placed in the main menu:

    1. The page for displaying the entire list of goods of the online store shop.html (SHOP). There will also be added the ability to select goods in the right quantity.
    2. The detail page for the selected product shop-details.html (it does not have a menu item on the main page - it opens by clicking the "Detail" button next to each product.
    3. Cart page cart.html (CART)
    4. And the page for launching the script for scraping (parsing) products from a third-party site fill-products.html (FILL-DATABASE)

    Well, that seems to be all: all the main points of the plan have been taken into account, and all the necessary pages have been added.

    And in conclusion of the topic of planning, it is necessary to say about a very important point: when planning, you should always remember that the IT project plan is not a sacred cow. When implementing the plan, details may appear that we did not foresee at the beginning. Like, for example, when planning this mini-course, I did not think about the fact that the second lesson turns out to be completely theoretical. Therefore, I moved the item “Selecting and finalizing a template” from the next lesson to this one.

    Summary: if, as the project progresses, it becomes necessary to adjust the plan, adjust it without hesitation!

    You can learn more about all the details of this stage from this video (RU voice):






    To the next stage of the project

    Read more >>

    Drop Shipping Online Store on Django

    1. Introduction. Who is this course for? Preparing the Development Environment

    Introduction

    This article is a presentation of the new Drop Shipping Store on Django course, where in 7 lessons you will create a full-fledged online store from free materials that are in the public domain, which can trade anything on a dropshipping basis. If anyone is not in the know, then dropshipping is a way of organizing a business in which the seller places on his virtual storefronts goods that he does not have, but which he can receive at any time from his regular supplier. Therefore, the task of the owner of such an online store is to receive an order and payment from the client, and then transfer the information about the order and the received payment to the supplier, minus his commission.

    Everything you need to create this online store: programs, necessary training materials and even video tutorials are in the public domain. You just need to carefully follow everything that will be written and shown in the materials for this course.

    In this course, we will go through all the stages of creating a new project together:

    As mentioned above, all videos that describe in detail (and most importantly, show!) The development process are in the public domain. Therefore, carefully following the recommendations on the screen, you will eventually get the real working code of the entire project. However, for those who are immediately interested in the final result, there is the option of accessing an archived copy of the project after each lesson. This access can be obtained through a small donation here at this link (registration required).

    Well, the site itself, which will turn out in the end, can be viewed in the video at the end of this article.

    Who is this course for?

    For everyone who is interested in web development, regardless of the level of knowledge and experience!

    1. Those who are far from programming and who have been dreaming of their own online store for a long time will be able to get acquainted with the main stages of creating a web project, get an idea of ​​​​the capabilities of the Django framework and understand the amount of work that is necessary for implementation these possibilities. This will allow, firstly, to formulate the tasks for your own project more accurately and in more detail, and, secondly, to specify the requirements for the contractor to implement these same tasks. This will significantly reduce the time for discussing these tasks with the contractor, and hence the overall costs of the project.

    2. For those who consider themselves to be Familiar with the basics of the Python language this will be a great opportunity to try their hand at creating their own project. Actually, if you don’t know Python yet, then only two courses separate you from this category: Python for beginners and Python Basics. After passing them, you will be able to independently adjust this project for many of your tasks and save the result in your own repository. Well, and what you can’t implement on your own yet, you can order a third-party contractor (including us!). In any case, finalizing a part of the project is not at all the same as doing the project from scratch!

    3. Well, if you are familiar with Python OOP, have already tried your hand at the Django website and created your first Polls application there, then this course will suit you perfectly. Thanks to it, you can independently implement most of the tasks in your application. Well, the execution of that insignificant part of the project, which you cannot do yourself yet, can always be ordered by a specialist. Including us :-).

    Preparing the Development Environment

    We will be working on a Linux operating system, and as an IDE we will use the Community (free) version of PyCharm. The choice of Linux is not accidental: it is on this OS that all servers that use the Django framework work. Therefore, if you work on Windows, then the best solution would be to additionally install a virtual machine with Linux OS. How to do this is described in detail and shown in these videos:

    In addition, we will need somewhere to save checked and debugged versions of our code, and a more convenient place than a repository has not yet been invented for this purpose. Therefore, if you have not yet created an account with SSH access in any repository, then it makes sense to read See this article on our website, which details and shows how to create an account on GutHub and add SSH keys to it.

    After carrying out all the above additional preparations, we can finally proceed directly to the creation of the project itself in the following order:

    • create a virtual environment for our project;
    • install the Django package on it;
    • create the project itself;
    • change the settings.py file responsible for the settings of our project;
    • add two new applications to the project:
      • application responsible for registering and authorizing users (authentication);
      • and the application of the online store itself (shop);
    • make the first migration;
    • create a superuser;
    • add to .gitignore .idea for PyCharm;
    • and "push" everything done to the repository.

    After completing all the points of this checklist, we will try to run the project. Hooray - our project is already running! And all that remains for us is to "slightly" modify its functionality!

    We go into the admin panel under the login and password as the newly created superuser. As you can see, there are already user and user group models.

    We haven't added this structure yet, but it comes by default in Django. Therefore, if desired, we can add the first users for testing even before we authorize users on the site. That is, directly from the admin panel!

    The results of all this preparatory work of the current stage, you can find the link at the bottom of the video, which will lead you to the course Drop Shipping Store on Django (RU voice). And, if you have paid for the course, then you can download an archive copy of this stage (that is, what has been done so far) (registration required).

    You can learn more about all the details of this stage from this video (RU voice):






    Read more >>

    Tags list

        Apps Script      Arrays Java Script      asynchronous code      asyncio      coroutine      Django      Dropdown List      Drop Shipping      Exceptions      GitHub      Google API      Google Apps Script      Google Docs      Google Drive      Google Sheets      multiprocessing      Parsing      Python      regex      Scraping      ssh      Test Driven Development (TDD)      threading      website monitoring      zip