Python queue Module

The queue module in Python provides a way to create and manage different types of queue data structures. It includes implementations for FIFO (First-In-First-Out) queues, LIFO (Last-In-First-Out) queues, and priority queues. These data structures are particularly useful for thread-safe task management.

Table of Contents

1. Introduction
2. Key Classes and Methods
   • Queue
   • LifoQueue
   • PriorityQueue
   • put
   • get
   • task_done
   • join
   • empty
   • full
3. Examples
   • Using FIFO Queue
   • Using LIFO Queue
   • Using Priority Queue
   • Thread-Safe Task Management
4. Real-World Use Case
5. Conclusion
6. References

Introduction

The queue module provides a way to create and manage different types of queue data structures that are thread-safe. This module is particularly useful in multi-threaded applications where multiple threads need to communicate or share data in a synchronized manner.

Key Classes and Methods

Queue

A FIFO (First-In-First-Out) queue.

import queue

q = queue.Queue()

LifoQueue

A LIFO (Last-In-First-Out) queue, similar to a stack.

import queue

lq = queue.LifoQueue()

PriorityQueue

A priority queue where entries are kept sorted and retrieved in order.

import queue

pq = queue.PriorityQueue()

put

Adds an item to the queue.

q.put(item)

get

Removes and returns an item from the queue.

item = q.get()

task_done

Indicates that a formerly enqueued task is complete.

q.task_done()

join

Blocks until all items in the queue have been processed.

q.join()

empty

Returns True if the queue is empty, False otherwise.

is_empty = q.empty()

full

Returns True if the queue is full, False otherwise.

is_full = q.full()

Examples

Using FIFO Queue

import queue

q = queue.Queue()
q.put('first')
q.put('second')
q.put('third')

while not q.empty():
    print(q.get())

Output:

first
second
third

Using LIFO Queue

import queue

lq = queue.LifoQueue()
lq.put('first')
lq.put('second')
lq.put('third')

while not lq.empty():
    print(lq.get())

Output:

third
second
first

Using Priority Queue

import queue

pq = queue.PriorityQueue()
pq.put((2, 'second'))
pq.put((1, 'first'))
pq.put((3, 'third'))

while not pq.empty():
    print(pq.get())

Output:

(1, 'first')
(2, 'second')
(3, 'third')

Thread-Safe Task Management

import queue
import threading
import time

def worker(q):
    while True:
        item = q.get()
        if item is None:
            break
        print(f'Working on {item}')
        time.sleep(1)
        print(f'Finished {item}')
        q.task_done()

q = queue.Queue()
threads = []

for i in range(4):
    t = threading.Thread(target=worker, args=(q,))
    t.start()
    threads.append(t)

for item in range(10):
    q.put(item)

q.join()

for i in range(4):
    q.put(None)
for t in threads:
    t.join()

Output:

Working on 0
Working on 1
Working on 2
Working on 3
Finished 0
Working on 4
Finished 1
Working on 5
Finished 2
Working on 6
Finished 3
Working on 7
Finished 4
Working on 8
Finished 5
Working on 9
Finished 6
Finished 7
Finished 8
Finished 9

Real-World Use Case

Producer-Consumer Problem

The queue module is often used to implement the producer-consumer problem in multi-threaded applications.

import queue
import threading
import time

def producer(q, n):
    for i in range(n):
        print(f'Producing {i}')
        q.put(i)
        time.sleep(0.5)
    q.put(None)  # Sentinel to signal the consumer to stop

def consumer(q):
    while True:
        item = q.get()
        if item is None:
            break
        print(f'Consuming {item}')
        q.task_done()

q = queue.Queue()
n = 10

producer_thread = threading.Thread(target=producer, args=(q, n))
consumer_thread = threading.Thread(target=consumer, args=(q,))

producer_thread.start()
consumer_thread.start()

producer_thread.join()
q.join()
consumer_thread.join()

Output:

Producing 0
Producing 1
Consuming 0
Producing 2
Consuming 1
Producing 3
Consuming 2
Producing 4
Consuming 3
Producing 5
Consuming 4
Producing 6
Consuming 5
Producing 7
Consuming 6
Producing 8
Consuming 7
Producing 9
Consuming 8
Consuming 9

Conclusion

The queue module in Python provides a robust and thread-safe way to implement various types of queues. It is particularly useful for multi-threaded applications where tasks need to be managed and synchronized between multiple threads. With FIFO, LIFO, and priority queue implementations, the queue module covers a wide range of use cases for task management and inter-thread communication.

References

• Python queue module documentation