Reduce the flush frequency of dbcounter plugin

This relaxes the limits for dbcounter to make it flush stats to the
database less often. Currently every thirty seconds or 100 hits, we
write a stats line to the database. In some services (like keystone)
this can trigger more than one write per second because of the
massive number of SELECT calls that service makes.

This removes the hit limit and decreases the mandatory flush interval
to once a minute. Hopefully this will manifest as lower load on the
database triggered by what would be readonly operations.

Change-Id: I43a58532c0541075a2d36408abc50a41f7994bda
This commit is contained in:
Dan Smith 2023-07-31 07:04:34 -07:00
parent 280feb0861
commit d115bfd72a

View File

@ -96,20 +96,18 @@ class LogCursorEventsPlugin(CreateEnginePlugin):
This reads "hists" from from a queue fed by _log_event() and This reads "hists" from from a queue fed by _log_event() and
writes (db,op)+=count stats to the database after ten seconds writes (db,op)+=count stats to the database after ten seconds
of no activity to avoid triggering a write for every SELECT of no activity to avoid triggering a write for every SELECT
call. Write no less often than every thirty seconds and/or 100 call. Write no less often than every sixty seconds to avoid being
pending hits to avoid being starved by constant activity. starved by constant activity.
""" """
LOG.debug('[%i] Writer thread running' % os.getpid()) LOG.debug('[%i] Writer thread running' % os.getpid())
while True: while True:
to_write = {} to_write = {}
total = 0
last = time.time() last = time.time()
while time.time() - last < 30 and total < 100: while time.time() - last < 60:
try: try:
item = self.queue.get(timeout=10) item = self.queue.get(timeout=10)
to_write.setdefault(item, 0) to_write.setdefault(item, 0)
to_write[item] += 1 to_write[item] += 1
total += 1
except queue.Empty: except queue.Empty:
break break