Intro to Programming with Python – #5

In my last post, we went over how functions work. We realized how we can significantly reduce our code length and maintain it in an orderly manner using functions. There are other ways of achieving this. One such way is working with modules and packages in python

In this post we will discuss about what is a python module, why is it necessary to have multiple modules rather than one and how packages come handy in dealing with a combination of modules. The post also describes how this approach helps us increase our efficiency in finding errors in code.

Before going ahead, find assignment solutions for previous post here.

PYTHON MODULE

A module in python is any .py file. This means take any python file and it is what we call a module in python. So all the files we made from the time of first post till now or all the files we will create in future if they have a .py extension then they are Python Modules.

Well that was easy. But what is the benefit of thinking like this? Let’s talk about the usage of python module!

HOW TO WORK WITH PYTHON MODULES?

Python gives us a very simple but powerful tool to literally import functions or other entities present in one module into another. Consider each module as a part of computer – you have a CPU, a GPU, a RAM, a motherboard and what not. They all have their special purpose and they all have to work in conjunction with each other. Modules give us similar modularity. We can take one function from one module and use it in other module. We can take every entity in one module and use it in another or multiple others!

So how does it work? How can we achieve this modularity? Python gives us three important keywords – import and from along with an operator called the dot operator ( . ) for this purpose. Let’s build two different python files (read modules) to understand them!

File: simple_utils.py

def add(li1, li2):
    sum_list = []
    for i in range(len(li1)):
        sum_list.append(li1[i] + li2[i])
    return sum_list

def sub(li1, li2):
    diff_list = []
    for i in range(len(li1)):
        diff_list.append(li1[i] - li2[i])
    return diff_list

def mul(li, val):
    new_li = []
    for i in range(len(li)):
        new_li.append(li[i] * val)
    return li1

def pow_li(li, pow):
    new_li = []
    for i in range(len(li)):
        new_li.append(li[i] ** pow)
    return new_li

def pow_li_rev(li, val):
    new_li = []
    for i in range(len(li)):
        new_li.append(val ** li[i])
    return new_li

pi = 3.141592653589793
e = 2.718281828459045

Here,

  • add(): adds two lists element wise
  • sub(): subtracts elements second list from first list
  • mul(): Multiply value in val to every element in list li
  • pow_li(): returns a list with each element being element of list li raised to power pow
  • pow_li_rev(): returns a list with each element being val raised to power element of list li
  • pi: is mathematical constant
  • e: is mathematical constant

File: playgorund.py

def area_of_circles(radius_li):
    pi = 3.141592653589793
    new_li = []
    for radius in radius_li:
        new_li.append(pi * radius ** 2)
    return new_li

def exp_li(li):
    e = 2.718281828459045
    new_li = []
    for i in range(len(li)):
        new_li.append(e ** li[i])
    return new_li

Here,

  • area_of_circles(): Takes a list of radii of various circles and returns a list of their areas!
  • exp_li(): Takes a list and returns a list with each element being e to the power of elements of the input list.

Now, let’s think about it. What if we had access to functions from simple_utils.py in playground.py? How much less lines we might have needed? Let’s write area_of_circles() and exp_li() function assuming that we have access to all the entities (including pi and e) in simple_utils.py.

def area_of_circles(radius_li):
    radius_sq = pow_li(radius_li, 2)
    areas = mul(radius_sq, pi)
    return areas

def exp_li(li):
    return pow_li_rev(li, e)

Here, we could have returned area_of_circles() in more concise manner as well:

def area_of_circles(radius_li):
    return mul(pow_li(radius_li, 2), pi)

Now that we are loving it, let’s do it! First note – both simple_utils.py and playgorund.py should be in same directory!

We can import functions and entities in four possible ways!

import simple_utils

Here, import tells python to import the simple_utils. But there is a catch! Now you can’t directly call pow_li() or any other entity in simple_utils.py. To call these functions, we will use dot operator. For example the pow_li() function will be called as simple_utils.pow_li(…). Dot operator tells python that pow_li() is a function that belongs to module simple_utils.

from simple_utils import *

Here * is not the multiplication operator. It represents everything that is present in the simple_utils.py module. Now we can call every function from simple_utils without using the dot operator.

from simple_utils import pow_li, pow_li_rev, mul, pi, e
Here, we are importing everything and using ‘,‘ to separate all the imports from one another.
import simple_utils.pow_li
import simple_utils.pow_li_rev
import simple_utils.mul
import simple_utils.pi
import simple_utils.e
Here, we aren’t importing the module but just what we need. In this scenario, we will call functions as we called functions in first way using the dot operator, example simple_utils.pow_li(…).

So, our final playground.py will look something like this:

from simple_utils import pow_li, pow_li_rev, mul, pi, e

def area_of_circles(radius_li):
    return mul(pow_li(radius_li, 2), pi)

def exp_li(li):
    return pow_li_rev(li, e)

We just reduced 12 lines of code to just 5!

PYTHON PACKAGES

Now that you have got a gist of how we can use python modules, let’s talk about how we can mantain large chunks of files (read modules) associated to a single project but in a different way.

Consider having 100 files in your project. Some working to provide utility functions, some working to mantain data and the rest working to handle User Interface (of course, a project can have many other things as well). Every category have one thing in common, they require help from modules of other categories. Example: User Interface modules might require utility functions to work on data that they get from data modules.

Going by the rule of modules – “each file should be present in the same directory”. This can cause a massive chaos in case of failure of any one function in any one module!

To sort this issue, we can use what we call a Python Package. So how to use Python Packages?

HOW TO USE PYTHON PACKAGES?

Say, you have 5 files that handle User Interface (UI):
  • forms.py
  • products.py
  • about.py
  • home.py
  • blog.py
5 files that handle data:
  • form_data.py
  • products_data.py
  • about_data.py
  • home_data.py
  • blog_data.py
3 files that have utilities:
  • blog_utils.py
  • product_utils.py
  • other_utils.py
And at last a file to run the project – main.py which starts everything! If our project resides in directory – super_cool_project, wouldn’t it be cool that we have a project structure such that all UI modules are present in a sub-directory named ui, all data modules are present in a sub-directory named data and all utility modules are present in a sub-directory named utils? Leaving only main.py in the root directory of the project:
super_cool_project
|----data
|    |----about_data.py
|    |----blog_data.py
|    |----forms_data.py
|    |----home_data.py
|    |----products_data.py
|
|----ui
|    |----about.py
|    |----blog.py
|    |----forms.py
|    |----home.py
|    |----products.py
|
|----utils
|    |----blog_utils.py
|    |----product_utils.py
|    |----other_utils.py
|
|----main.py
Python Packages allow us to achieve this exact thing. To be precise, here we convert sub-directories data, ui and utils into packages. This can be done by adding __init__.py modules (either empty or with some code) inside these sub-directories.
super_cool_project
|----data
|    |----__init__.py
|    |----about_data.py
|    |----blog_data.py
|    |----forms_data.py
|    |----home_data.py
|    |----products_data.py
|
|----ui
|    |----__init__.py
|    |----about.py
|    |----blog.py
|    |----forms.py
|    |----home.py
|    |----products.py
|
|----utils
|    |----__init__.py
|    |----blog_utils.py
|    |----product_utils.py
|    |----other_utils.py
|
|----main.py

HOW TO IMPORT MODULES FROM PACKAGES

Now that we have structured our project in packages, we need a way to access entities from them! Luckily, python’s import and from keywords (which we used for modules) work here as well.
  • Say, you want to import a function display_about() from about.py module present in ui package in the main.py module. We can achieve this by following line of code added to main.py:
from ui.about import display_about
  • Say you want to import a function get_blog_data() from blog_data.py module present in data package in the blog.py module present in ui package. Since we are running this program using main.py module, i.e., from root directory, python knows about all packages in the directory. So, we can achieve this by following line of code added to blog.py:
from data.blog_data import get_blog_data
  • Now that we have dealt with root and sub-package dependency as well as sub-package to sub-package dependencies, let’s deal with one package dependency! What if blog_data.py module requires a function connect_to_database() from home_data.py. How can we achieve this? In this case, dot operator ( . ) plays a significant role as it will represent our current package. So, we can add following line of code to blog_data.py.
from .home_data import connect_to_database

MODULES AND PACKAGES FOR ERROR CHECKING & RECTIFICATION

Let’s try something out! Add following code at the end of your playground.py module from the module section above:

from simple_utils import add

li1 = [1, 2, 3, 4]
li2 = [2, 3, 4]
print(li1, '+', li2, '=', add(li1, li2))

Now let’s run our playground.py file:

Let’s look at the error carefully! We got some error stating – list index out of range. And we got it in simple_utils.py (as stated by the File path line) in line 4 of the file in function add() at the exact line – sum_list.append(li1[i] + li2[i]).

So python being a nice friend it is, states that we reached an index that wasn’t in one of the lists we sent in add() function of simple_utils.py module. Now we know the exact place of the error!

Let’s rectify it! What does add() function do? It adds two files by iterating over the range of length of first list! Hmmm. What is the length of the first list? li1 has a length of 4. What is the length of second list? li2 has the length of 3. That’s where the issue is. If we apply range() over len(li1), we get maximum value of 3 but we know that the last index in li2 is 2.

How can we rectify this? Let’s take the simplest approach for now and change li2 to length 4.

li1 = [1, 2, 3, 4]
li2 = [2, 3, 4, 0]
print(li1, '+', li2, '=', add(li1, li2))

Output:

[1, 2, 3, 4] + [2, 3, 4, 0] = [3, 5, 7, 4]

Isn’t it cool how we ordered our code and achieved an easy way out of difficult problems? This might not look much right now. Wait till you work on a big project. 😉

This brings us to the end of this post. No assignments this time! Just try to assimilate what you have learned till now and try to come up with ideas that will make our assignments and codes from previous posts better! 🙂

LINKS TO SOME USEFUL PYTHON BOOKS

  1. Learn Python 3 The Hard Way
  2. Automate the Boring Stuff with Python
  3. Python Crash Course

This Post Has One Comment

Leave a Reply

Close Menu

SUBSCRIBE FOR WEEKLY POST UPDATES <3