# Entry
Why do you exploit Python? For many people it comes down to “just because,” but it really shouldn’t. Python is a powerful general-purpose programming language with a plain syntax highlighted by a Pythonic approach to managing logic and data that happens to have become a popular language for data science, machine learning, and artificial intelligence precisely for these reasons. Python is basic to master, but you can spend many years working to refine your skills and master the core mechanics of the language, working to go from a beginner to a professional who is able to write proficient, maintainable systems.
With this in mind, today we’ll look at five core concepts that every Python developer should have in their toolkit.
# 1. List expressions and generator expressions
Python is eminent for its readability. Expression list they allow you to replace awkward loops with a single line of code. However, the real pro here is knowing when to exploit a generator expression instead of saving memory.
// The Awkward Way (For Loop)
Let’s start with the unskilled, “clunky” non-Python way of doing things:
numbers = range(1000000)
squared_list = []
for n in numbers:
if n % 2 == 0:
squared_list.append(n ** 2)// Pythonic way (list comprehension)
Now let’s look at a Pythonic way to solve the same task:
# Concise and faster execution
squared_list = [n ** 2 for n in numbers if n % 2 == 0]
# The "Must-Know" Twist: Generator Expressions
# If you only need to iterate once and don't need the whole list in memory:
squared_gen = (n ** 2 for n in numbers if n % 2 == 0)Exit:
List size: 4,167,352 bytes
Generator size: 200 bytesHere’s why this is vital, aside from people saying “this is how you do it in Python”: List comprehension is faster than .append(). Generator expressions (using parentheses) are “lazy” – they create elements one at a time, allowing you to process huge sets of data without consuming system memory.
Let’s see how to exploit the generator, one call at a time, using the generator expression:
numbers = range(1000000)
squared_gen = (n ** 2 for n in numbers if n % 2 == 0)
# Values are computed only when requested, not all at once
print(next(squared_gen))
print(next(squared_gen))
print(next(squared_gen))Exit:
# 2. Decorators
Decorators is a way to modify the behavior of a function or class without permanently changing its source code. Think of them as wrappers for other features.
// Clumsy way
If you want to record the running times of several different functions, you can manually add a timecode to each individual function.
import time
def process_data():
start = time.time()
# ... function logic ...
end = time.time()
print(f"process_data took {end - start:.4f}s")
def train_model():
start = time.time()
# ... function logic ...
end = time.time()
print(f"train_model took {end - start:.4f}s")
def generate_report():
start = time.time()
# ... function logic ...
end = time.time()
print(f"generate_report took {end - start:.4f}s")Note that the repetitions make the problem obvious: the same four lines duplicated in every function. Let’s see how the decorator function can fix this.
// The Pythonic way
Here’s a more Pythonic take on this task.
import time
from functools import wraps
def timer_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f"{func.__name__} took {end - start:.4f}s")
return result
return wrapper
@timer_decorator
def heavy_computation():
return sum(range(10**7))
heavy_computation()Exit:
heavy_computation took 0.0941sSee how timer_decorator() “wraps” heavy_computation() function, and when the latter is invoked, it becomes subordinated to the former and experiences its benefits.
Decorators promote “don’t repeat yourself base (DRY). They are necessary for logging, authentication, and caching in production environments.
# 3. Context managers (with Statements)
Managing resources such as files, database connections, or network sockets is a common source of errors. If you forget to close a file, it will either leak memory or lock the file from other processes.
// Clumsy way
Here we open a file, exploit it and force close it when it is no longer needed.
f = open("data.txt", "w")
try:
f.write("Hello World")
finally:
# Simple to forget!
f.close()// The Pythonic way
The with statement would lend a hand us with the above.
# File is automatically closed here, even if an error occurs
with open("data.txt", "w") as f:
f.write("Hello World")Not only is it more concise, but the logic is simpler and easier to understand, plus you get the easy-to-forget close() for free, because “assembly” and “dismantling” take place reliably. For data-related tasks, this is useful when connecting to SQL databases or handling huge input/output (IO) tasks.
# 4. Self-control *args AND **kwargs
Sometimes you don’t know how many arguments will be passed to a function. Python handles this elegantly by using “packing” operators. Even as a beginner who may not have hired them, you have undoubtedly seen these “packaging” operators at some point.
// Python example
Here’s the Python way of handling it:
*args(non-keyword arguments): A “packer” operator that collects additional data positional arguments in low order. This is used when you don’t know how many elements will be passed to the function.**kwargs(keyword arguments): A “packer” operator that collects additional data named arguments to the dictionary. Used for optional settings or named parameters.
def make_profile(name, *tags, **metadata):
# name is the named argument
print(f"User: {name}")
# tags is a tuple
print(f"Tags: {tags}")
# metadata is a dictionary
print(f"Details: {metadata}")
make_profile("Alice", "DataScientist", "Pythonist", location="NY", seniority="Senior")Exit:
User: Alice
Tags: ('DataScientist', 'Pythonist')
Details: {'location': 'NY', 'seniority': 'Senior'}This is the secret to versatile libraries like Scikit-Learn Or Matplotlib. It allows you to pass any number of configuration settings to functions, making your code incredibly adaptable to changing requirements.
# 5. Dunder’s methods (magic methods)
“Dunder” means double underline (e.g __init__). Officially special methods (but more commonly called magic methods), these methods allow custom objects to emulate Python’s built-in behavior.
// The Pythonic way
Let’s see how to exploit magic methods to add automatic behavior to our classes.
class Dataset:
def __init__(self, data):
self.data = data
def __len__(self):
return len(self.data)
def __str__(self):
return f"Dataset with {len(self.data)} items"
# Create a dataset instance
my_data = Dataset([1, 2, 3])
# Calls __len__
print(len(my_data))
# Calss __str__
print(my_data)Exit:
Using the built-in __len__ AND __str__ dunders, our custom class gets useful features for free.
Dunder methods form the basis of Python’s object-oriented protocol. By implementing methods such as __getitem__ Or __call__you can make your classes behave like lists, dictionaries, or even functions, which will lead to much more intuitive APIs.
# Summary
Mastering these five concepts means moving from scripting to software development. Using list expressions for speed, decorators for pure logic, context managers for security, *args/**kwargs when it comes to flexibility and dunder methods for object-oriented power, you are laying a foundation on which you can build further Python knowledge.
Matthew Mayo (@mattmayo13) has a master’s degree in computer science and a university diploma in data mining. As editor-in-chief of KDnuggets & Statologyand contributing editor at Machine learning masteryMatthew’s goal is to make sophisticated data science concepts accessible. His professional interests include natural language processing, language models, machine learning algorithms, and exploring emerging artificial intelligence. It is driven by a mission to democratize knowledge in the data science community. Matthew has been coding since he was 6 years senior.