cinder/doc/source/contributor/threading.rst
Jay S. Bryant 1423480fb6 Make doc/source directory compliant with design in spec
The following spec defines what each project's doc/source
directory is supposed to look like:

https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html

I had not yet moved existing content to follow this design.
This patch does that, moving the devref to the
'contributor' directory.  It also moves the CLI
related documentation into the 'cli' directory.  I have
updated the autodoc generation to now create the api
documentation in 'doc/source/contributor/api'.

This patch also creates a template for future documentation
contribution.  I have created all of the directories
recommended by the spec and have included documentation
as to what should go in each directory.

The index file is updated to point at the new locations for
existing content.

'doc/.gitignore' is updated so that it won't complain about the
automatically generated 'doc/contributor/api' directory.

Change-Id: I55c50fa0b7c1d06c91e40dbcfd11b1c8e8378aa6
2017-07-19 15:59:02 -05:00

2.7 KiB
Raw Blame History

Threading model

All OpenStack services use green thread model of threading, implemented through using the Python eventlet and greenlet libraries.

Green threads use a cooperative model of threading: thread context switches can only occur when specific eventlet or greenlet library calls are made (e.g., sleep, certain I/O calls). From the operating system's point of view, each OpenStack service runs in a single thread.

The use of green threads reduces the likelihood of race conditions, but does not completely eliminate them. In some cases, you may need to use the @utils.synchronized(...) decorator to avoid races.

In addition, since there is only one operating system thread, a call that blocks that main thread will block the entire process.

Yielding the thread in long-running tasks

If a code path takes a long time to execute and does not contain any methods that trigger an eventlet context switch, the long-running thread will block any pending threads.

This scenario can be avoided by adding calls to the eventlet sleep method in the long-running code path. The sleep call will trigger a context switch if there are pending threads, and using an argument of 0 will avoid introducing delays in the case that there is only a single green thread:

from eventlet import greenthread
...
greenthread.sleep(0)

In current code, time.sleep(0does the same thing as greenthread.sleep(0) if time module is patched through eventlet.monkey_patch(). To be explicit, we recommend contributors use greenthread.sleep() instead of time.sleep().

MySQL access and eventlet

There are some MySQL DB API drivers for oslo.db, like PyMySQL, MySQL-python etc. PyMySQL is the default MySQL DB API driver for oslo.db, and it works well with eventlet. MySQL-python uses an external C library for accessing the MySQL database. Since eventlet cannot use monkey-patching to intercept blocking calls in a C library, queries to the MySQL database using libraries like MySQL-python will block the main thread of a service.

The Diablo release contained a thread-pooling implementation that did not block, but this implementation resulted in a bug and was removed.

See this mailing list thread for a discussion of this issue, including a discussion of the impact on performance.