openstack/cinder
Code Issues Proposed changes

Fix reporting old stats

Browse Source 
For the stats reporting we have a decoupled implementation where the
gathering of stats from the backend and the submission of these stats to
the schedulers are run in independent periodic tasks.

The problem with this implementation is that we'll be getting old stats
all the time because of the threading model we are using.

Since we have greenthreads we will not switch threads until we do I/O,
so if the _publish_service_capabilities periodic task is invoked first
we won't switch to the _report_driver_status until we send the
capabilities update.  And even if we start the _report_driver_status
task first as soon as it does some I/O to query the backend we'll be
switching to sending the cached data to the schedulers.

It may not sound like a big problem, but when you are quickly creating a
great number of big volumes having an additional minute old data can be
problematic.

This patch changes the way we do the status reporting and ensures that
we are always sending the newest data.

There could only be 1 drawback to this change, and that is if a driver
takes longer than 1 minute to gather the stats, because in this case a
newly started scheduler will take a little longer to have stats for this
backend.  But it doesn't really happen, because the schedulers actually
request the volumes for their capabilities on start up.

Change-Id: Id203a3a8b8c09e6415dcd26cc5ae684f702b8457
Closes-Bug: #1750878
This commit is contained in:
Gorka Eguileor 2018-02-21 18:38:36 +01:00
parent b4681905dc
commit b8076c55d5
2 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
cinder/manager.py
View File

@ -182,7 +182,6 @@ class SchedulerDependentManager(ThreadPoolManager):
"""Remember these capabilities to send on next periodic update."""
self.last_capabilities = capabilities
@periodic_task.periodic_task
def _publish_service_capabilities(self, context):
"""Pass data back to the scheduler at a periodic interval."""
if self.last_capabilities:

2
cinder/volume/manager.py
View File

@ -2374,7 +2374,6 @@ class VolumeManager(manager.CleanableManager,
LOG.info("Migrate volume completed successfully.",
resource=volume)
@periodic_task.periodic_task
def _report_driver_status(self, context):
# It's possible during live db migration that the self.service_uuid
# value isn't set (we didn't restart services), so we'll go ahead
@ -2507,6 +2506,7 @@ class VolumeManager(manager.CleanableManager,
return volume_stats
@periodic_task.periodic_task
def publish_service_capabilities(self, context):
"""Collect driver status and then publish."""
self._report_driver_status(context)