"python thread safety - are lists thread-safe?" Code Answer

By thread-safety people usually mean whether a piece of code is re-entrant or not. That means if a function can be safely called again before the earlier function call has returned.

Consider the append method of a Python List. If I have a List l = [1, 2, 3], can I safely call l.append(4) and l.append(5) from two different threads and get [1, 2, 3, 4, 5] (or [1, 2, 3, 5, 4]) in return? If yes, then the code is thread-safe.

When people talk about Python's thread-safety in general, they mostly mean whether Python's basic data-structure operations are reentrant or not.

Python's List data structure in the CPython implementation is, indeed, thread-safe. The same guarantee doesn't hold for other implementations.

By Dani G on October 23 2022

A list can be made thread-safe using a mutual exclusion (mutex) lock.

Specifically, each add, delete, update, and read of the list must be protected by a lock.

This can be achieved by wrapping a list with a new class. The class can then expose the specific operations that we require on the list and protect those operations on the internal list using a lock.

An instance of the threading.Lock can be used to protect the list.

You can learn more about the mutex lock here:

How to Use a Mutex Lock in Python
This lock must be acquired prior to an operation on the list and released once the operation on the list has completed.

For example:

...
# acquire the lock
lock.acquire()
# operate on the list
...
# release the lock
lock.release()
This ensures that only one thread at a time is able to operate on the list.

The context manager interface on the lock may be used to simplify this operation, allowing the lock to be released automatically.

For example:

...
# acquire the lock
with lock:
	# operate on the list
	...
This adds overhead, making each list operation much slower than it otherwise would be without the lock. The benefit is confidence that race conditions with list have been removed entirely.

If your design is to have some wrapped functions call other wrapped functions, then a reentrant lock is preferred, in order to avoid a deadlock.

This can be achieved via the threading.RLock class. It will allow the lock to be acquired by the same thread again.

By Robert Karczmarczyk on November 14 2022

lists themselves are thread-safe. in cpython the gil protects against concurrent accesses to them, and other implementations take care to use a fine-grained lock or a synchronized datatype for their list implementations. however, while lists themselves can't go corrupt by attempts to concurrently access, the lists's data is not protected. for example:

l[0] += 1

is not guaranteed to actually increase l[0] by one if another thread does the same thing, because += is not an atomic operation. (very, very few operations in python are actually atomic, because most of them can cause arbitrary python code to be called.) you should use queues because if you just use an unprotected list, you may get or delete the wrong item because of race conditions.

By Sean Morgan on September 10 2022