ironic/doc/source/contributor/osprofiler-support.rst

107 lines
3.5 KiB
ReStructuredText
Raw Normal View History

Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
.. _OSProfiler-support:
================
About OSProfiler
================
OSProfiler is an OpenStack cross-project profiling library. Its API
provides different ways to add a new trace point. Trace points contain
two messages (start and stop). Messages like below are sent to a collector::
{
"name": <point_name>-(start|stop),
"base_id": <uuid>,
"parent_id": <uuid>,
"trace_id": <uuid>,
"info": <dict>
}
The fields are defined as follows:
``base_id`` - <uuid> that is same for all trace points that belong to
one trace. This is used to simplify the process of retrieving all
trace points (related to one trace) from the collector.
``parent_id`` - <uuid> of parent trace point.
``trace_id`` - <uuid> of current trace point.
``info`` - the dictionary that contains user information passed when
calling profiler start() & stop() methods.
The profiler uses ceilometer as a centralized collector. Two other
alternatives for ceilometer are pure MongoDB driver and Elasticsearch.
A notifier is setup to send notifications to ceilometer using oslo.messaging
and ceilometer API is used to retrieve all messages related to one trace.
OSProfiler has entry point that allows the user to retrieve information
about traces and present it in HTML/JSON using CLI.
For more details see
:osprofiler-doc:`OSProfiler Cross-project profiling library <index.html>`.
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
How to Use OSProfiler with Ironic in Devstack
=============================================
To use or test OSProfiler in ironic, the user needs to setup Devstack
with OSProfiler and ceilometer. In addition to the setup described at
:ref:`deploy_devstack`, the user needs to do the following:
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
Add the following to ``localrc`` to enable OSProfiler and ceilometer::
enable_plugin panko https://opendev.org/openstack/panko
enable_plugin ceilometer https://opendev.org/openstack/ceilometer
enable_plugin osprofiler https://opendev.org/openstack/osprofiler
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
# Enable the following services
CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler
ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral
ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector
ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier
ENABLED_SERVICES+=,ceilometer-api
Run stack.sh.
Once Devstack environment is setup, edit ``ironic.conf`` to set the following
profiler options and restart ironic services::
[profiler]
enabled = True
hmac_keys = SECRET_KEY # default value used across several OpenStack projects
trace_sqlalchemy = True
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
In order to trace ironic using OSProfiler, use openstackclient to run
baremetal commands with ``--os-profile SECRET_KEY``.
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
For example, the following will cause a <trace-id> to be printed after node list::
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
$ openstack --os-profile SECRET_KEY baremetal node list
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
Output of the above command will include the following::
Trace ID: <trace-id>
Display trace with command:
osprofiler trace show --html <trace-id>
The trace results can be seen using this command::
$ osprofiler trace show --html <trace-id>
The trace results can be saved in a file with ``--out file-name`` option::
$ osprofiler trace show --html <trace-id> --out trace.html
The trace results show the time spent in ironic-api, ironic-conductor, and db
calls. More detailed db tracing is enabled if ``trace_sqlalchemy``
is set to true.
Enable OSProfiler support in Ironic This patch does the following: * Adds osprofiler wsgi middleware This middleware is used for 2 things: - It checks that person who wants to trace is trusted and knows secret HMAC key. - It starts tracing in case of proper trace headers and adds first wsgi trace point, with info about HTTP request. * Adds initialization of osprofiler at start of service - Initialize and set an oslo.messaging based notifier instance to osprofiler, which will be used to send notifications to Ceilometer. * Traces HTTP/RPC/DB API calls and SQL requests NOTE to test this patch: 1) Make the following changes in localrc to configure DevStack to enable OSProfiler: enable_plugin panko https://git.openstack.org/openstack/panko enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler # Enable the following services CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral ENABLED_SERVICES+=,ceilometer-anotification,ceilometer-collector ENABLED_SERVICES+=,ceilometer-alarm-evaluator,ceilometer-alarm-notifier ENABLED_SERVICES+=,ceilometer-api NOTE: the order of enabling plugins matters. 2) Run stack.sh. Once DevStack environment is setup, enable profiler options in ironic.conf and restart ironic services: [profiler] enabled = true hmac_keys = SECRET_KEY trace_sqlalchemy = true 3) Use openstackclient and run baremetal command with --os-profile SECRET_KEY [--profile can be used, but it is deprecated.] For example, the following will cause the <trace-id> to be printed after node list: $ openstack --os-profile SECRET_KEY baremetal node list ..... ..... Trace ID: <trace-id> Display trace with command: osprofiler trace show --html <trace-id> 4) The trace results can be saved using this command: $ osprofiler trace show --html <trace-id> --out trace.html OSprofiler spec: https://review.openstack.org/#/c/103825/ Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com> Co-Authored-By: Hieu LE <hieulq@vn.fujitsu.com> Partial-Bug: #1560704 Change-Id: Icd3d7c62cf7442de8a77fc67f119ae9b03725f02
2016-08-18 16:55:56 -07:00
References
==========
- :osprofiler-doc:`OSProfiler Cross-project profiling library <index.html>`
- :ref:`deploy_devstack`