Photo by the author Canva
# Entry
When you are novel in Python, you usually utilize the “DO” loop when you need to process data collection. Do you want to level out a list of numbers? It loops through them. Do you want to filter or add them? The loop again. It is more intuitive to us as people, because our brain thoughts and works sequentially (one thing at once).
But that doesn’t mean computers must. They can utilize what is called Vector thinking. Basically, instead of looping every element to perform the operation, you give the entire Python list how, “Hey, here is the list. Perform all operations at the same time.”
In this tutorial, I will give you a gentle introduction to how it works, why it matters, and we will discuss some examples to see how beneficial it can be. Let’s start.
# What is vector thinking and why does it matter?
As discussed earlier, vector thinking means that instead of sequential operation of operations, we want to perform them collectively. This idea is actually inspired by Matrix surgery and vectors in mathematics, and your code is much faster and more readable. Libraries such as Numpy allow you to implement vectorized thinking in Python.
For example, if you need to multiply the list of numbers by 2, instead of accessing each element and performing operations one by one, you will multiply the entire list at the same time. This has the main benefits, such as reducing the majority of Python costs. Every time you heat a Python loop, the translator must do a lot of work, such as checking types, managing objects and supporting loop mechanics. Thanks to the vectorized approach, you reduce this by processing with loose. It is also much faster. We will see it later with an example of impact on performance. I visualized what I just said in the form of a picture so that you could understand the idea of what I am talking about.
Now that you have an idea what it is, let’s see how you can implement it and how it can be useful.
# A elementary example: temperature conversion
There are various temperature conventions used in different countries. For example, if you know the Fahrenheit scale and the data is provided in Celsius, here you can convert it using both approaches.
// Loop approach
celsius_temps = [0, 10, 20, 30, 40, 50]
fahrenheit_temps = []
for temp in celsius_temps:
fahrenheit = (temp * 9/5) + 32
fahrenheit_temps.append(fahrenheit)
print(fahrenheit_temps)Exit:
[32.0, 50.0, 68.0, 86.0, 104.0, 122.0]// Vectorized approach
import numpy as np
celsius_temps = np.array([0, 10, 20, 30, 40, 50])
fahrenheit_temps = (celsius_temps * 9/5) + 32
print(fahrenheit_temps) # [32. 50. 68. 86. 104. 122.]
Exit:
[ 32. 50. 68. 86. 104. 122.]Instead of dealing with each element individually, we turn the list into a NumPy board and utilize a formula for all elements at the same time. Both process data and provide the same result. In addition to the fact that the NumPy code is more concise, you may not notice the time difference now. But we’ll discuss it soon.
# Advanced example: mathematical operations on many boards
Let’s take another example in which we have many boards and we must calculate the profit. Here’s how you can do it with both approaches.
// Loop approach
revenues = [1000, 1500, 800, 2000, 1200]
costs = [600, 900, 500, 1100, 700]
tax_rates = [0.15, 0.18, 0.12, 0.20, 0.16]
profits = []
for i in range(len(revenues)):
gross_profit = revenues[i] - costs[i]
net_profit = gross_profit * (1 - tax_rates[i])
profits.append(net_profit)
print(profits)
Exit:
[340.0, 492.00000000000006, 264.0, 720.0, 420.0]Here we manually calculate the profit for each entry:
- Subtraction of costs from revenues (gross profit)
- Apply tax
- Appendix the result to the novel list
It works well, but it is a lot of manual indexing.
// Vectorized approach
import numpy as np
revenues = np.array([1000, 1500, 800, 2000, 1200])
costs = np.array([600, 900, 500, 1100, 700])
tax_rates = np.array([0.15, 0.18, 0.12, 0.20, 0.16])
gross_profits = revenues - costs
net_profits = gross_profits * (1 - tax_rates)
print(net_profits)Exit:
[340. 492. 264. 720. 420.]The vectorized version is also more legible and simultaneously performs elementary operations in all three boards. Now I don’t just want to repeat “it’s faster” without solid proof. And you can think: “What is the canal about?” But now, when you saw how to implement it, let’s look at the performance difference between them.
# Performance: The numbers are not lying
The difference I am talking about is not just a noise or a taoretical thing. It is measurable and proven. Let’s look at a practical reference point to understand how many improvements you can expect. We will create a very huge set of data 1,000,000 and perform the operation (x^2 + 3x + 1 ) on each element using both approaches and compare time.
import numpy as np
import time
# Create a huge dataset
size = 1000000
data = list(range(size))
np_data = np.array(data)
# Test loop-based approach
start_time = time.time()
result_loop = []
for x in data:
result_loop.append(x ** 2 + 3 * x + 1)
loop_time = time.time() - start_time
# Test vectorized approach
start_time = time.time()
result_vector = np_data ** 2 + 3 * np_data + 1
vector_time = time.time() - start_time
print(f"Loop time: {loop_time:.4f} seconds")
print(f"Vector time: {vector_time:.4f} seconds")
print(f"Speedup: {loop_time / vector_time:.1f}x faster")Exit:
Loop time: 0.4615 seconds
Vector time: 0.0086 seconds
Speedup: 53.9x fasterIt’s more than 50 times faster !!!
This is not a miniature optimization, it will make data processing tasks (I’m talking about huge data sets) much more feasible. I utilize a nummy for this tutorial, but Panda is another library built on NUMPY. You can also utilize it.
# When not vectorize
Just because something works in most cases does not mean that this is an approach. In programming your “best” approach always depends on the problem. Vectorization is great when you perform the same operation on all elements of the data set. But if your logic covers sophisticated conditions, earlier ending or operations depending on earlier results, and then stick to the loop -based approach.
Similarly, working with very miniature data sets, the costs of configuring vector operations can prevail over the benefits. So just utilize it where it makes sense and do not force him where not.
# Wrapping
Continuing cooperation with Python, challenge yourself to see the possibilities of vectorization. When you reach for the “For” loop, stop and ask if there is a way to express the same operation using the numba or panda. Most often it exists, and the result will be a code that is not only faster, but also more elegant and easier to understand.
Remember that the goal is not to eliminate all loops from the code. Is to utilize Proper tool for work.
Canwal Mehreen Kanwal is a machine learning engineer and a technical writer with a deep passion for data learning and AI intersection with medicine. He is the co -author of the ebook “maximizing performance from chatgpt”. As a Google 2022 generation scholar for APAC, it tells diversity and academic perfection. It is also recognized as a variety of terradate at Tech Scholar, Mitacs Globalink Research Scholar and Harvard Wecode Scholar. Kanwalwal is a scorching supporter of changes, after establishing FemCodes to strengthen women in the STEM fields.