Intro to Programming with Python – #5
November 2, 2018
In my last post, we went over how functions work. We realized how we can significantly reduce our code length and maintain it in an orderly manner using functions. There are other ways of achieving this. One such way is working with modules and packages in python
In this post we will discuss about what is a python module, why is it necessary to have multiple modules rather than one and how packages come handy in dealing with a combination of modules. The post also describes how this approach helps us increase our efficiency in finding errors in code.
Before going ahead, find assignment solutions for previous post here.
A module in python is any .py file. This means take any python file and it is what we call a module in python. So all the files we made from the time of first post till now or all the files we will create in future if they have a .py extension then they are Python Modules.
Well that was easy. But what is the benefit of thinking like this? Let’s talk about the usage of python module!
HOW TO WORK WITH PYTHON MODULES?
Python gives us a very simple but powerful tool to literally import functions or other entities present in one module into another. Consider each module as a part of computer – you have a CPU, a GPU, a RAM, a motherboard and what not. They all have their special purpose and they all have to work in conjunction with each other. Modules give us similar modularity. We can take one function from one module and use it in other module. We can take every entity in one module and use it in another or multiple others!
So how does it work? How can we achieve this modularity? Python gives us three important keywords – import and from along with an operator called the dot operator ( . ) for this purpose. Let’s build two different python files (read modules) to understand them!
def add(li1, li2): sum_list =  for i in range(len(li1)): sum_list.append(li1[i] + li2[i]) return sum_list def sub(li1, li2): diff_list =  for i in range(len(li1)): diff_list.append(li1[i] - li2[i]) return diff_list def mul(li, val): new_li =  for i in range(len(li)): new_li.append(li[i] * val) return li1 def pow_li(li, pow): new_li =  for i in range(len(li)): new_li.append(li[i] ** pow) return new_li def pow_li_rev(li, val): new_li =  for i in range(len(li)): new_li.append(val ** li[i]) return new_li pi = 3.141592653589793 e = 2.718281828459045
- add(): adds two lists element wise
- sub(): subtracts elements second list from first list
- mul(): Multiply value in val to every element in list li
- pow_li(): returns a list with each element being element of list li raised to power pow
- pow_li_rev(): returns a list with each element being val raised to power element of list li
- pi: is mathematical constant
- e: is mathematical constant
def area_of_circles(radius_li): pi = 3.141592653589793 new_li =  for radius in radius_li: new_li.append(pi * radius ** 2) return new_li def exp_li(li): e = 2.718281828459045 new_li =  for i in range(len(li)): new_li.append(e ** li[i]) return new_li
- area_of_circles(): Takes a list of radii of various circles and returns a list of their areas!
- exp_li(): Takes a list and returns a list with each element being e to the power of elements of the input list.
Now, let’s think about it. What if we had access to functions from simple_utils.py in playground.py? How much less lines we might have needed? Let’s write area_of_circles() and exp_li() function assuming that we have access to all the entities (including pi and e) in simple_utils.py.
def area_of_circles(radius_li): radius_sq = pow_li(radius_li, 2) areas = mul(radius_sq, pi) return areas def exp_li(li): return pow_li_rev(li, e)
Here, we could have returned area_of_circles() in more concise manner as well:
def area_of_circles(radius_li): return mul(pow_li(radius_li, 2), pi)
Now that we are loving it, let’s do it! First note – both simple_utils.py and playgorund.py should be in same directory!
We can import functions and entities in four possible ways!
Here, import tells python to import the simple_utils. But there is a catch! Now you can’t directly call pow_li() or any other entity in simple_utils.py. To call these functions, we will use dot operator. For example the pow_li() function will be called as simple_utils.pow_li(…). Dot operator tells python that pow_li() is a function that belongs to module simple_utils.
from simple_utils import *
Here * is not the multiplication operator. It represents everything that is present in the simple_utils.py module. Now we can call every function from simple_utils without using the dot operator.
from simple_utils import pow_li, pow_li_rev, mul, pi, e
import simple_utils.pow_li import simple_utils.pow_li_rev import simple_utils.mul import simple_utils.pi import simple_utils.e
So, our final playground.py will look something like this:
from simple_utils import pow_li, pow_li_rev, mul, pi, e def area_of_circles(radius_li): return mul(pow_li(radius_li, 2), pi) def exp_li(li): return pow_li_rev(li, e)
We just reduced 12 lines of code to just 5!
Now that you have got a gist of how we can use python modules, let’s talk about how we can mantain large chunks of files (read modules) associated to a single project but in a different way.
Consider having 100 files in your project. Some working to provide utility functions, some working to mantain data and the rest working to handle User Interface (of course, a project can have many other things as well). Every category have one thing in common, they require help from modules of other categories. Example: User Interface modules might require utility functions to work on data that they get from data modules.
Going by the rule of modules – “each file should be present in the same directory”. This can cause a massive chaos in case of failure of any one function in any one module!
HOW TO USE PYTHON PACKAGES?Say, you have 5 files that handle User Interface (UI):
super_cool_project |----data | |----about_data.py | |----blog_data.py | |----forms_data.py | |----home_data.py | |----products_data.py | |----ui | |----about.py | |----blog.py | |----forms.py | |----home.py | |----products.py | |----utils | |----blog_utils.py | |----product_utils.py | |----other_utils.py | |----main.py
super_cool_project |----data | |----__init__.py | |----about_data.py | |----blog_data.py | |----forms_data.py | |----home_data.py | |----products_data.py | |----ui | |----__init__.py | |----about.py | |----blog.py | |----forms.py | |----home.py | |----products.py | |----utils | |----__init__.py | |----blog_utils.py | |----product_utils.py | |----other_utils.py | |----main.py
HOW TO IMPORT MODULES FROM PACKAGESNow that we have structured our project in packages, we need a way to access entities from them! Luckily, python’s import and from keywords (which we used for modules) work here as well.
- Say, you want to import a function display_about() from about.py module present in ui package in the main.py module. We can achieve this by following line of code added to main.py:
from ui.about import display_about
- Say you want to import a function get_blog_data() from blog_data.py module present in data package in the blog.py module present in ui package. Since we are running this program using main.py module, i.e., from root directory, python knows about all packages in the directory. So, we can achieve this by following line of code added to blog.py:
from data.blog_data import get_blog_data
- Now that we have dealt with root and sub-package dependency as well as sub-package to sub-package dependencies, let’s deal with one package dependency! What if blog_data.py module requires a function connect_to_database() from home_data.py. How can we achieve this? In this case, dot operator ( . ) plays a significant role as it will represent our current package. So, we can add following line of code to blog_data.py.
from .home_data import connect_to_database
MODULES AND PACKAGES FOR ERROR CHECKING & RECTIFICATION
Let’s try something out! Add following code at the end of your playground.py module from the module section above:
from simple_utils import add li1 = [1, 2, 3, 4] li2 = [2, 3, 4] print(li1, '+', li2, '=', add(li1, li2))
Now let’s run our playground.py file:
Let’s look at the error carefully! We got some error stating – list index out of range. And we got it in simple_utils.py (as stated by the File path line) in line 4 of the file in function add() at the exact line – sum_list.append(li1[i] + li2[i]).
So python being a nice friend it is, states that we reached an index that wasn’t in one of the lists we sent in add() function of simple_utils.py module. Now we know the exact place of the error!
Let’s rectify it! What does add() function do? It adds two files by iterating over the range of length of first list! Hmmm. What is the length of the first list? li1 has a length of 4. What is the length of second list? li2 has the length of 3. That’s where the issue is. If we apply range() over len(li1), we get maximum value of 3 but we know that the last index in li2 is 2.
How can we rectify this? Let’s take the simplest approach for now and change li2 to length 4.
li1 = [1, 2, 3, 4] li2 = [2, 3, 4, 0] print(li1, '+', li2, '=', add(li1, li2))
[1, 2, 3, 4] + [2, 3, 4, 0] = [3, 5, 7, 4]
Isn’t it cool how we ordered our code and achieved an easy way out of difficult problems? This might not look much right now. Wait till you work on a big project. 😉
This brings us to the end of this post. No assignments this time! Just try to assimilate what you have learned till now and try to come up with ideas that will make our assignments and codes from previous posts better! 🙂