How to create a fullstack application using Django and Python Part 16

Sunday, September 15, 2024 at 8:48 PM | 7 min read

Last modified on Thursday, July 16, 2026 at 12:06 PM

#fullstack development, #macOS, #django, #beautifulsoup4, #soupsieve, #html5lib, #html parser, #python3, #series, #tests, #unittest

Beautiful bowl of soup garnished with herbs and surrounded by a flower and a broken half-bowl of almonds

Photo by Ella Olsson on unsplash.com

Important Note: Before committing anything to Git or pushing anything to remote, please visit How to create a fullstack application using Django and Python Part 4 where I discuss how to add the python-dotenv package to the Django site and why it is crucial to do it. This article assumes you have a working knowledge of Git.

Table of Contents

Update (July 2026): This walkthrough was written against Django 5.1, which reached full end-of-life in December 2025 and is no longer receiving security patches. Run python -m django --version to check which version you have.

I wrote this series as a live process, so sometimes it might seem confusing, but it all works out in the end.

Adding Test Client using Soup Sieve

As I mentioned in section 15, there is a way of more generically verifying HTML inputs in templates/signup.html. I can do that with a package called Soup Sieve.

Soup Sieve is an effective selector add-on for powerful Beautiful Soup 4. It comes pre-installed with BeautifulSoup4 4.7.0 and above, and uses the CSS selector API to query a document. In my case, it can be used to find the HTML inputs in my templates/signup.html and verify they exist there. This time I won't need to pass in complete inputs to the .assertContains() method. That was a bit of an inconvenient and long-winded approach.

Installing Beautiful Soup 4, Soup Sieve, and html5lib HTML parser

I have to install Beautiful Soup 4 before installing Soup Sieve. In fact, it's meant to be used alongside Beautiful Soup 4:

# first:
python3 -m pip install beautifulsoup4
# then:
python3 -m pip install soupsieve
# then (very important):
python3 -m pip install html5lib # needed for parsing and has to be installed separately

I also need to install html5lib. It parses the HTML that Beautiful Soup/Soup Sieve scrapes (that's what they do!).

Make sure to install all three packages at the root directory of the project (in my case django-boards directory) where the venv directory resides. Also make sure that your virtual environment has been activated before installing the packages. Then add bs4, soupsieve, and html5lib to INSTALLED_APPS in settings.py:

INSTALLED_APPS = [
    ...
    'soupsieve',
    'bs4',
    'html5lib',
]

Update July 6, 2026: I still have soupsieve and bs4 listed in INSTALLED_APPS today, but I don't actually import or use them anywhere in the project — I later realized this step wasn't necessary, since these are plain Python packages rather than Django apps with models or config to register.

Creating my Test Client using Beautiful Soup 4 and Soup Sieve

First, I commented out my existing test_form_inputs in accounts/tests/test_signup_tests.py. Instead of relying on the generic test I originally had (although you could go that way if you want), I could inspect my signup.html template in the browser when I go to the url http://127.0.0.1:8000/signup/. All input attributes are in there, and much easier to read!

What I did was get all the input ids and pass them in like the following:

# add following imports first in accounts/tests/test_signup_tests.py:
import bs4
import soupsieve as sv

# then add the refactored test_form_inputs test:
def test_form_inputs(self):
    """
    The view must contain five inputs: csrf, username, email, password1, password2
    """
    self.assertContains(self.response, "<input", 7)

    # add the refactored test_form_inputs test:
    self.response = self.client.get(reverse("signup"))
    text = """
    <form method="post" novalidate="" class="signup-form">
        <input type="hidden" name="csrfmiddlewaretoken" value="5bzfyc9iidGoyInd3IYNlTrBGVLNVo09hNqsSjydsbrvupjtRELqgD8siJf94pup">

        <div class="form-group">
            <label for="id_username">Username:</label>
            <input type="text" name="username" maxlength="150" autofocus="" class="form-control " required="" aria-describedby="id_username_helptext" id="id_username" data-np-intersection-state="visible">

            <small class="form-text text-muted">
                Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.
            </small>
        </div>

        <div class="form-group">
            <label for="id_email">Email:</label>
            <input type="email" name="email" maxlength="254" class="form-control " required="" id="id_email" data-np-intersection-state="visible">
        </div>

        <div class="form-group">
            <label for="id_password1">Password:</label>
            <input type="password" name="password1" autocomplete="new-password" class="form-control " aria-describedby="id_password1_helptext" id="id_password1" data-np-intersection-state="visible">


            <small class="form-text text-muted">
                <ul><li>Your password can’t be too similar to your other personal information.</li><li>Your password must contain at least 8 characters.</li><li>Your password can’t be a commonly used password.</li><li>Your password can’t be entirely numeric.</li></ul>
            </small>

        </div>

        <div class="form-group">
            <label for="id_password2">Password confirmation:</label>
            <input type="password" name="password2" autocomplete="new-password" class="form-control " aria-describedby="id_password2_helptext" id="id_password2" data-np-intersection-state="visible">


            <small class="form-text text-muted">
                Enter the same password as before, for verification.
            </small>

        </div>

        <div class="form-group">
            <label>Password-based authentication:</label>
            <div id="id_usable_password" class="form-control "><div>
            <label for="id_usable_password_0"><input type="radio" name="usable_password" value="true" class="form-control " id="id_usable_password_0" checked="">
        Enabled</label>

        </div>
        <div>
            <label for="id_usable_password_1"><input type="radio" name="usable_password" value="false" class="form-control " id="id_usable_password_1">
        Disabled</label>

        </div>
        <div>
            <small class="form-text text-muted">
                Whether the user will be able to authenticate using a password or not. If disabled, they may still be able to authenticate using other backends, such as Single Sign-On or LDAP.
            </small>
        </div>

        <button type="submit" class="btn btn-primary btn-block">Create an account</button>
    </form>
    """
    soup = bs4.BeautifulSoup(text, "html5lib")
    sv.select(
            "form:is(.signup-form)",
            soup,
    )
    print(
        sv.select(
            "form:is(.signup-form)",
            soup,
        )
    )

You might also notice that I did not have to import the html5lib library. I only had to add it to INSTALLED_APPS in settings.py.

Breaking down the test_form_inputs test
  1. First I do as I have done to date. I get the "signup" view with self.response = self.client.get(reverse('signup')).
  2. Then I create a text object which contains the value of the element I want to target and the class it contains. I added the class beforehand. Soup Sieve/BeautifulSoup needs to be able to find a class, or an element selector, etc., in order to be able to parse the correct HTML. I wanted to be and needed to be more specific than the form element selector, so I chose signup-form as the class selector. It clearly describes its target element. All I needed to add was the form element, because it contains all the inputs I want to grab and verify in the test.
  3. Then I create a soup object (required) and set it to bs4.BeautifulSoup(text, 'html5lib') (also required in order to work), followed by sv.select() where I select the .signup-form class of the signup form so that I can view the entire form when I run my tests. The other thing I include is the line self.assertContains(self.response, "<input", 7). This tells me that I am looking for seven inputs. And since there actually are seven inputs in the signup form in my case, the entire test_form_inputs function passes. I print out the results of sv.select( "form:is(.signup-form)", soup, ) to see what actually returns from sv.select("form:is(.signup-form)", soup,). And that is it!

Finding all the input tags in the signup.html template using Beautiful Soup 4

I could also find just the input tags inside the signup.html template form using Beautiful Soup 4:

# place at the bottom of the test_form_inputs function
for tag in soup.find_all('input'):
    print(tag)

Which, for me, returned the following in Terminal:

<input name="csrfmiddlewaretoken" type="hidden" value="5bzfyc9iidGoyInd3IYNlTrBGVLNVo09hNqsSjydsbrvupjtRELqgD8siJf94pup"/> input tag
<input aria-describedby="id_username_helptext" autofocus="" class="form-control" data-np-intersection-state="visible" id="id_username" maxlength="150" name="username" required="" type="text"/> input tag
<input class="form-control" data-np-intersection-state="visible" id="id_email" maxlength="254" name="email" required="" type="email"/> input tag
<input aria-describedby="id_password1_helptext" autocomplete="new-password" class="form-control" data-np-intersection-state="visible" id="id_password1" name="password1" type="password"/> input tag
<input aria-describedby="id_password2_helptext" autocomplete="new-password" class="form-control" data-np-intersection-state="visible" id="id_password2" name="password2" type="password"/> input tag
<input checked="" class="form-control" id="id_usable_password_0" name="usable_password" type="radio" value="true"/> input tag
<input class="form-control" id="id_usable_password_1" name="usable_password" type="radio" value="false"/>

This also proves (in a different way) that all seven inputs located in the signup form actually do exist, and it also confirms that there ARE seven of them.

Update July 6, 2026: I later discovered there are actually seven inputs instead of five. See the change below:

# in accounts/tests/test_signup_tests.py
def test_form_inputs(self):
    """
    The view must contain five inputs: csrf, username, email, password1, password2
    """
    # Note: I later discovered there are actually 7 inputs — see the breakdown below
    self.assertContains(self.response, "<input", 7)

Running the accounts tests

After I complete creating the test, I can run the accounts tests:

python3 manage.py test accounts

The following is returned:

======================================================================
FAIL: test_csrf (accounts.tests.test_view_password_reset.PasswordResetConfirmTests.test_csrf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mariacam/Python-Development/django-boards/django_boards/accounts/tests/test_view_password_reset.py", line 110, in test_csrf
    self.assertContains(self.response, 'csrfmiddlewaretoken')
  File "/Users/mariacam/.pyenv/versions/3.12.5/lib/python3.12/site-packages/django/test/testcases.py", line 623, in assertContains
    self.assertTrue(
AssertionError: False is not true : Couldn't find 'csrfmiddlewaretoken' in the following response
b'\n<!DOCTYPE html>\n<html lang="en">\n  <head>\n    <meta charset="utf-8">\n    <meta name="description" content="A forum dedicated to all things Django" />\n    <meta name="keywords" content="django, python3" />\n    <title>\n      \n  \n    Reset your password\n  \n\n    </title>\n    <link rel="stylesheet" href="/static/css/bootstrap.min.css">\n    <link rel="stylesheet" href="/static/css/app.css">\n    \n  <link rel="stylesheet" href="/static/css/accounts.css">\n\n  </head>\n  <body>\n    \n  <div class="container">\n    <h1 class="text-center logo my-4">\n      <a href="/">Django Boards</a>\n    </h1>\n    \n  <div class="row justify-content-center">\n    <div class="col-lg-6 col-md-8 col-sm-10">\n      <div class="card">\n        <div class="card-body">\n          \n            <h3 class="card-title">Reset your password</h3>\n            <div class="alert alert-danger" role="alert">\n              It looks like you clicked on an invalid password reset link. Please try again.\n            </div>\n            <a href="/password-reset/" class="btn btn-secondary btn-block">Request a new password reset link</a>\n          \n        </div>\n      </div>\n    </div>\n  </div>\n\n  </div>\n\n    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/2.9.2/umd/popper.min.js"\n            integrity="sha512-2rNj2KJ+D8s1ceNasTIex6z4HWyOnEYLVC3FigGOmyQCZc2eBXKgOxQmo3oKLHyfcj53uz4QMsRCWNbLd32Q1g=="\n            crossorigin="anonymous"\n            referrerpolicy="no-referrer"></script>\n    <script src="https://code.jquery.com/jquery-3.7.1.min.js"\n            integrity="sha256-/JqT3SQfawRcv/BIHPThkBvs0OEvtFFmqPF/lYI/Cxo="\n            crossorigin="anonymous"></script>\n    <script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.3.3/js/bootstrap.min.js"\n            integrity="512-ykZ1QQr0Jy/4ZkvKuqWn4iF3lqPZyij9iRv6sGqLRdTPkY69YX6+7wvVGmsdBbiIfN/8OdsI7HABjvEok6ZopQ=="\n            crossorigin="anonymous"\n            referrerpolicy="no-referrer"></script>\n  </body>\n</html>\n'

======================================================================
FAIL: test_form_inputs (accounts.tests.test_view_password_reset.PasswordResetConfirmTests.test_form_inputs)
The view must contain two inputs: csrf and two password fields
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mariacam/Python-Development/django-boards/django_boards/accounts/tests/test_view_password_reset.py", line 122, in test_form_inputs
    self.assertContains(self.response, '<input', 3)
  File "/Users/mariacam/.pyenv/versions/3.12.5/lib/python3.12/site-packages/django/test/testcases.py", line 614, in assertContains
    self.assertEqual(
AssertionError: 0 != 3 : Found 0 instances of '<input' (expected 3) in the following response
b'\n<!DOCTYPE html>\n<html lang="en">\n  <head>\n    <meta charset="utf-8">\n    <meta name="description" content="A forum dedicated to all things Django" />\n    <meta name="keywords" content="django, python3" />\n    <title>\n      \n  \n    Reset your password\n  \n\n    </title>\n    <link rel="stylesheet" href="/static/css/bootstrap.min.css">\n    <link rel="stylesheet" href="/static/css/app.css">\n    \n  <link rel="stylesheet" href="/static/css/accounts.css">\n\n  </head>\n  <body>\n    \n  <div class="container">\n    <h1 class="text-center logo my-4">\n      <a href="/">Django Boards</a>\n    </h1>\n    \n  <div class="row justify-content-center">\n    <div class="col-lg-6 col-md-8 col-sm-10">\n      <div class="card">\n        <div class="card-body">\n          \n            <h3 class="card-title">Reset your password</h3>\n            <div class="alert alert-danger" role="alert">\n              It looks like you clicked on an invalid password reset link. Please try again.\n            </div>\n            <a href="/password-reset/" class="btn btn-secondary btn-block">Request a new password reset link</a>\n          \n        </div>\n      </div>\n    </div>\n  </div>\n\n  </div>\n\n    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/2.9.2/umd/popper.min.js"\n            integrity="sha512-2rNj2KJ+D8s1ceNasTIex6z4HWyOnEYLVC3FigGOmyQCZc2eBXKgOxQmo3oKLHyfcj53uz4QMsRCWNbLd32Q1g=="\n            crossorigin="anonymous"\n            referrerpolicy="no-referrer"></script>\n    <script src="https://code.jquery.com/jquery-3.7.1.min.js"\n            integrity="sha256-/JqT3SQfawRcv/BIHPThkBvs0OEvtFFmqPF/lYI/Cxo="\n            crossorigin="anonymous"></script>\n    <script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.3.3/js/bootstrap.min.js"\n            integrity="512-ykZ1QQr0Jy/4ZkvKuqWn4iF3lqPZyij9iRv6sGqLRdTPkY69YX6+7wvVGmsdBbiIfN/8OdsI7HABjvEok6ZopQ=="\n            crossorigin="anonymous"\n            referrerpolicy="no-referrer"></script>\n  </body>\n</html>\n'

----------------------------------------------------------------------
Ran 46 tests in 10.107s

FAILED (failures=2)
Destroying test database for alias 'default'...

I am addressing these failures and will post the fixes in a future post when I have figured them out.

Code associated with this post

To view the code associated with this post, please visit 5cee77d.

Conclusion

In this section, I demonstrate an addition to the original test_form_inputs test in which I use the .assertContains() method to verify the existence of specific inputs residing in the signup form. I install Beautiful Soup 4 and Soup Sieve along with the html5lib HTML parser, and then re-create the test_form_inputs test using those packages. The test is much shorter, and my last approach to discovering the ids of the inputs I want to grab by iterating over the signup form input elements is quicker and much easier to read!