How can I implement shared and exclusive access for reading and writing?

[canyone2015]

How to allow multiple threads to read a shared resource concurrently while ensuring exclusive access for threads that need to modify it?

It's a very popular question, so I decided that it should be discussed.

In C++ I can do this:

MyClass my_data;
std::shared_mutex my_mutex_for_my_data;
...
// Thread 1
{
std::unique_lock<std::shared_mutex> lock(my_mutex_for_my_data); // [access guarded by a single unique_lock]
// do smth with my_data
}
...
// Threads 2, 3, 4
{
std::shared_lock<std::shared_mutex> lock(my_mutex_for_my_data); // [access guarded by multiple shared_locks]
// do smth with my_data
}
...

How can I do this with aiologic in the most appropriate way?

Also, it makes sense to consider not only threads but also async tasks.

[x42005e1f]

I plan to add phase-fair readers-writer locks in the future, but not in the near future. Here is the expected content of 0.15.0:

1. Documentation for all existing high-level primitives: concept explanation, usage examples, API reference.
2. Universal decorators: aiologic.synchronized() (async-aware alternative for wrapt.synchronized()) and aiologic.lowlevel.*_checkpoints() (for more convenient checkpoint tuning).
3. New methods for locks: lock.green_owned() and lock.async_owned() (inspired by Joshuaalbert/FairAsyncRLock).
4. Truly fair Condition.wait() and Condition.wait_for().
5. Maybe something else.

Since I now see that readers-writer locks are really needed by more than just me, I will raise the priority and add them in one of the next versions after 0.15.0: in 0.16.0 or 0.17.0. Until then, you can take any existing implementation of readers-writer locks for threading, such as elarivie/pyReaderWriterLock, and do two things:

1. Replace the underlying locks with alternatives from aiologic.
2. Add related asynchronous methods.

After that, you should be able to use readers-writer locks in both threads and tasks. However, that would not be enough to support checkpoints, and there may (or may not) be deadlocks when using two different APIs in the same thread.

[x42005e1f]

I think it makes sense to create a non-aiologic primitive so you do not have to wait too long.

Theoretically (and, as I will show below, practically too), readers-writer locks can be derived from reentrant locks. Let the acquire methods support identifier passing. Then we have possible situations where different tasks pass the same identifier. How to handle such situations?

1. If there is no owner, both tasks become one as a group.
2. If there is an owner and it does not match the identifier, both tasks wait for a release.
3. If there is an owner and it matches the identifier, both tasks increment the counter.
4. Both tasks from the first group in the queue are notified on release.
5. Since reentrancy must be preserved, each task must have its own counter.

What is curious is that we have actually invented a higher-level primitive. Instead of a waiting queue consisting of events (items of type aiologic.lowlevel.Event), we now have a waiting queue consisting of event vectors (items of type aiologic.Event). Reentrancy is now handled separately for each task, and multiple tasks can own the primitive (either in a single thread or in multiple threads). This is a group-level primitive, so let's call it GLock (yes, I know this naming is confusing, and I will think about it later).

In my opinion, the concept of groups shows what we are dealing with quite well. It is similar to groups in UNIX: a file can be owned by a personal group (the group identifier is equal to the user identifier), or it can be owned by a group of several users, in which case all of them can work with the file.

I said that readers-writer locks can be derived from reentrant locks. And indeed, using a shared identifier for readers would allow them to work simultaneously, while writers could be given no shared identifier and then work separately. But I actually derived a more general primitive: if there are only two phases (reader and writer) in readers-writer locks, there can be as many phases as you like in GLock.

I have already implemented GLock for you. It uses an internal lock to synchronize its own state, so it does not follow the aiologic style. But it works, and can be successfully used as a phase-fair readers-writer lock. Available here: x42005e1f/a50d0744013b7bbbd7ded608d6a3845b.

Here is how you can use GLock for readers and writers:

base = GLock()  # group-level!

reading = GLock(base, default_group="reading")
writing = GLock(base)

with reading:  # for sync reader
    ...

async with writing:  # for async writer
    ...

It is also useful when your code is coroutine-safe but not thread-safe, and you want to synchronize thread-level access without blocking the event loop. To do this, just pass default_group_factory, for example:

lock = GLock(default_group_factory=aiologic.lowlevel.current_thread_ident)  # thread-level!

Note that in GLock, the group is ignored when the lock has already been acquired by the current task. This means that instead of the expected upgrade (from reading to writing) or downgrade (from writing to reading), you will get an increment of the current task's counter when the lock is reacquired. Correctly supporting upgrading & downgrading without introducing additional methods requires significant changes, in particular lock acquiring in release methods with shielding from external cancellation (since the lock is only released when all tasks in the group have decremented their counter to zero).