Python
Completed
-
Section 1: Getting Started
-
Section 2: Core Syntax and Data Types
-
Section 3: Collections
-
39: Set Operations: Union, Intersection, Difference
-
Section 4: Control Flow
-
Section 5: Functions
-
Section 6: Turtle Graphics and Early Practice Projects
-
Section 7: Working with Files and I/O
-
Section 8: Regular Expressions
-
Section 9: Object-Oriented Python
-
Section 10: Error Handling
-
Section 11: Modules and Packages
-
Section 12: Iterators, Generators, and Functional Tools
-
Section 13: Decorators and Metaprogramming
-
Section 14: Concurrency and Parallelism
-
Section 15: Working with Dates, Times, and Numbers
-
Section 16: Standard Library Deep Dive I: Data Structures
-
Section 17: Standard Library Deep Dive II: System and Introspection
-
Section 18: Standard Library Deep Dive III: Security and Encoding
-
Section 19: Standard Library Deep Dive IV: Text and Data Utilities
-
Section 20: Networking and Web Basics
-
Section 21: Working with Databases
-
Section 22: Testing and Quality
-
Section 23: Advanced Typing
-
Section 24: Context Managers and Resource Handling
-
Section 25: Text, Unicode, and Binary Data
-
Section 26: More Functional and Iteration Tools
-
Section 27: Data Validation and Configuration
-
Section 28: Working with Images and Media
-
Section 29: Property-Based and Documentation Testing
-
Section 30: Packaging and Deployment
-
Section 31: Performance and Internals
-
Section 32: Design Patterns in Python
-
Section 33: GUI Programming
-
Section 34: Security Basics
-
Section 35: Data Structures and Algorithms
-
Section 36: Practical Projects
-
Section 37: Capstone Projects
-
Section 38: Interview and Algorithm Practice
-
Section 39: Writing Idiomatic Python
248: The gc Module and Garbage Collection Internals
You've probably internalised the idea that Python handles memory for you. For the most part, it does. You create an object, you stop using it, and it vanishes. But there's a specific scenario where Python's primary mechanism—reference counting—completely fails. I want to show you exactly how that happens and how the gc module steps in to save us.
The Reference Counting Trap
Let's start with a simple scenario. I'll create two objects that point to each other. This happens all the time in real-world code, like in a doubly-linked list or a parent-child relationship in a GUI tree.
import gc
import sys
class Node:
def __init__(self, name):
self.name = name
self.link = None
def __del__(self):
print(f"Cleaning up {self.name}")
# Create a circular reference
a = Node("Node A")
b = Node("Node B")
a.link = b
b.link = a
# Now, let's "delete" them from our local scope
del a
del b
print("Local variables deleted.")
If you run this, you'll notice something frustrating: the __del__ methods never fire. I've deleted the variables a and b, so why are the objects still alive? It's because Node A still has a reference to Node B, and Node B still has one to Node A. Their reference counts never hit zero. They are effectively ghosts—unreachable by your code, but still occupying RAM.
Forcing a Cleanup
This is where the gc (Garbage Collector) module comes in. While reference counting is the first line of defense, the cyclic garbage collector is the backup. It doesn't run every second; it triggers based on thresholds. I can, however, force it to run right now to see if it can spot these orphans.
# Continuing from the previous snippet...
print("Manually triggering GC...")
unreachable_count = gc.collect()
print(f"GC collected {unreachable_count} objects.")
Now, the __del__ messages finally print. By calling gc.collect(), I told Python to stop relying on reference counts and instead perform a "cycle detection" scan. It looks for groups of objects that are reachable from each other but unreachable from the "root" of the program. I'll be honest: in a small script, you'll never notice this. But in a long-running server process, these "ghost" objects can lead to a slow, agonizing memory leak.
Peeking at the Garbage Collector's To-Do List
I'm curious about what the GC is actually tracking. We can use gc.get_objects() to see every object Python is currently monitoring. Warning: this list is huge because it includes everything from internal modules to every single string you've defined. To make it useful, I'll filter it.
# Let's create another cycle
x = Node("X")
y = Node("Y")
x.link = y
y.link = x
# Find all Node objects currently in memory
all_nodes = [obj for obj in gc.get_objects() if isinstance(obj, Node)]
print(f"Found {len(all_nodes)} Node objects in memory.")
This is a powerful debugging tool. If you suspect a memory leak, you can snapshot gc.get_objects(), perform an action in your app, snapshot again, and see which objects are growing in number without being cleaned up.
The Hierarchy of Garbage
Python doesn't scan every single object every time the GC runs—that would be a performance nightmare. Instead, it uses "generations." New objects start in Generation 0. If they survive a GC collection, they get promoted to Generation 1, and eventually Generation 2.
The logic is simple: most objects die young. Why spend CPU cycles scanning a long-lived configuration object over and over again? Let's look at the current thresholds:
print(gc.get_threshold())
# Output is usually (700, 10, 10)
This means when the number of allocations minus deallocations exceeds 700, Gen 0 is collected. If Gen 0 has been collected 10 times, Gen 1 is collected, and so on. If you're writing a high-performance application and you notice "stutters" in your execution, it might be the GC hitting a Gen 2 collection. You can actually tune these values using gc.set_threshold(), though I'd suggest leaving them alone unless you have a very specific reason to touch them.
📋 Practical Task
Exercise: Detecting and Breaking Circular Reference Leaks in a Graph Structure
You are building a simple Graph data structure where nodes have a list of neighbors. Because the edges are bidirectional, circular references are inevitable.
- Create a
GraphNodeclass with anameand aneighborslist. Implement a__del__method that prints when a node is destroyed. - Create three nodes (A, B, and C) and connect them in a triangle (A→B, B→C, C→A).
- Delete the local references to A, B, and C using
del. - Verify that the nodes are not deleted immediately.
- Use the
gcmodule to manually trigger a collection and prove that the circular references were identified and cleaned up. - Print the total number of unreachable objects found during that specific collection.
There are no comments for now.