I Made My Python Code 10x Faster with These Little-Known Tricks

📘 Premium Read: Access my best content on Medium member-only articles — deep dives into Java, Spring Boot, Microservices, backend architecture, interview preparation, career advice, and industry-standard best practices.

🎓 Top 15 Udemy Courses (80-90% Discount): My Udemy Courses - Ramesh Fadatare — All my Udemy courses are real-time and project oriented courses.

▶️ Subscribe to My YouTube Channel (176K+ subscribers): Java Guides on YouTube

▶️ For AI, ChatGPT, Web, Tech, and Generative AI, subscribe to another channel: Ramesh Fadatare on YouTube

If you’ve been coding in Python long enough, you’ve probably run into performance issues. Maybe your script is running too slow. Maybe it crashes when processing large data. Or maybe your service times out during peak traffic.

That was me a year ago — until I started focusing on small, practical Python performance improvements that most developers overlook.

These aren’t massive rewrites or advanced C extensions. They’re simple, low-effort tips that can boost your speed by 10x or more, especially when used together.

In this article, I’ll walk you through the exact tricks I used — backed by real examples — to optimize slow Python code without sacrificing readability.

---

The Problem

I was working on a data-processing pipeline in Python. The task was to parse millions of records, clean the data, and generate reports.

• Initial runtime: 8 minutes
• After optimization: under 50 seconds

No Cython. No Just-In-Time compilers. Just pure Python with some smart tweaks.

---

Trick 1: Use Built-in Functions Instead of Manual Loops

Python’s built-in functions (like sum(), any(), all(), max(), min(), sorted()) are implemented in C and are much faster than writing your own loops.

Example:

# Slow
total = 0
for num in numbers:
    total += num

# Fast
total = sum(numbers)

In one of my scripts, switching from a manual loop to sum() shaved off 3 seconds when processing a 10M row dataset.

Takeaway: Whenever possible, prefer Python’s built-in functions. They’re optimized at the interpreter level.

---

Trick 2: Avoid Repeated Lookups in Loops

Every time you access an object’s method or attribute inside a loop, Python performs a lookup. Caching it into a local variable makes execution much faster.

Example:

# Slower
for user in users:
    user.do_work()

# Faster
do_work = users[0].do_work
for _ in users:
    do_work()

Even with built-in methods like arr.__getitem__, caching can save thousands of lookups in tight loops.

Takeaway: Cache methods or attributes when they are repeatedly used inside loops.

---

Trick 3: Use List Comprehensions Instead of Append Loops

List comprehensions are faster and cleaner than building lists manually.

Example:

# Slow
result = []
for item in data:
    result.append(process(item))

# Fast
result = [process(item) for item in data]

In benchmarks, list comprehensions can be 30–40% faster than append-based loops for large datasets.

---

Trick 4: Use Generators for Large Datasets

If you don’t need all results at once, use generators. They save memory and improve performance.

Example:

# Slower - loads all items
result = [process(x) for x in data]

# Faster - yields one at a time
result = (process(x) for x in data)

Or define generator functions using yield:

def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line.strip()

Use case: Processing large log files line by line without crashing your system.

---

Trick 5: Use Sets for Membership Checks

Checking if an item exists in a list is O(n). Checking in a set is O(1).

Example:

# Slow
if item in my_list:
    ...

# Fast
if item in my_set:
    ...

I replaced if id in list_of_ids (8 million items) with a set and reduced runtime from 27 seconds to 0.5 seconds.

---

Trick 6: Use enumerate() Instead of Manual Indexing

enumerate() is faster and cleaner than using range(len()).

# Slower
for i in range(len(data)):
    print(i, data[i])

# Faster
for i, value in enumerate(data):
    print(i, value)

It makes the code more Pythonic and less error-prone.

---

Trick 7: Use Local Variables Inside Loops

Python accesses local variables faster than globals or object attributes.

# Slower
def compute():
    for i in range(1000000):
        self.process(i)

# Faster
def compute():
    local_process = self.process
    for i in range(1000000):
        local_process(i)

This small tweak can yield huge performance gains when calling functions in tight loops.

---

Trick 8: Avoid Unnecessary Function Calls

Function calls in Python are relatively expensive. If a value doesn’t change inside a loop, compute it once outside.

# Slower
for user in users:
    threshold = get_threshold()
    if user.score > threshold:
        ...

# Faster
threshold = get_threshold()
for user in users:
    if user.score > threshold:
        ...

This avoids millions of redundant calls.

---

Trick 9: Profile Before You Optimize

Don’t guess where the slowdown is — measure it.

Use time, cProfile, or profiling tools:

import time

start = time.time()
process_data()
print(f"Time taken: {time.time() - start:.2f}s")

Or:

python -m cProfile my_script.py

For memory, try memory_profiler.

Rule: Optimize based on data, not assumptions.

---

Trick 10: Use the Right Data Structures

The wrong data structure can ruin performance. Use the right one for the job:

Task	Best choice
Lookup by key	dict
Fast membership checks	set
Fast FIFO queue	collections.deque
Count items	collections.Counter
Grouping	defaultdict(list)

Example:

from collections import defaultdict

grouped = defaultdict(list)
for item in data:
    grouped[item.category].append(item)

This is both faster and cleaner than manually handling dictionaries.

---

Bonus: Use NumPy or Pandas for Heavy Computation

For numeric computation or data analysis, NumPy and Pandas crush raw Python loops.

# Slow
squared = [x * x for x in range(1000000)]

# Fast NumPy
import numpy as np
arr = np.arange(1000000)
squared = arr ** 2

NumPy can be up to 100x faster than plain Python.

If you’re working with data, Pandas also provides vectorized operations that outperform loops significantly.

---

Additional Performance Tips

• Use multiprocessing/threading for parallelism when tasks are independent.
• Lazy evaluation: Don’t compute things you don’t need.
• Batch database/API calls instead of looping with single queries.
• Avoid string concatenation in loops; use join() instead.
• Upgrade Python: Each new release comes with performance improvements.

---

Summary of All Tricks

Trick	Key Idea
1. Use built-in functions	Faster than manual loops
2. Cache lookups	Avoid repeated method/attribute calls
3. List comprehensions	Faster than append loops
4. Generators	Save memory with large datasets
5. Use sets	Membership checks in O(1) time
6. Use enumerate()	Cleaner and faster indexing
7. Local variables	Faster than globals in loops
8. Precompute constants	Avoid repeated function calls
9. Profile your code	Measure before optimizing
10. Data structures	Use dict, set, deque, defaultdict
Bonus	NumPy/Pandas for numeric/data-heavy workloads

---

Final Thoughts

Most Python code can be improved without rewriting it in C or resorting to advanced hacks. The secret is to write clean, readable code — and then optimize the bottlenecks that matter most.

In my experience, applying just three or four of these tricks together resulted in a 10x speed boost in a real-world data pipeline.

Good code is readable. Great code is both readable and fast.