With current version of rabbitmq-exporter,
unable to retrieve data sometimes,
failing with rabbitmq timeout issues.
Rabbitmq timeout threshold is set as 10 sec
and is not configurable with current version.
Updating the rabbitmq-exporter version to
kbudde/rabbitmq-exporter:v1.0.0-RC7.1
(Default "RABBITMQ_TIMEOUT" set as 30 sec)
to solve rabbitmq timeout issues.
Change-Id: Ia51f368a1bba2b0fd9195cf9991b55864cdebfc1
This PS adds RABBIT_TIMEOUT parameter as configurable
with kbudde/rabbitmq-exporter:v1.0.0-RC7.1 version
Change-Id: I8faf8cd706863f65afb5137d93a7627d421270e9
This patch set changes the source of the rabbitmq-exporter's admin user
credential to leverage the existing secret rather than the values in the
Values.yaml file.
Change-Id: I1ad48ade3984e455d07be3a8b8ee3d9b25b449a2
Signed-off-by: Tin Lam <tin@irrational.io>
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0
Change-Id: I15950b735b4f8566bc0018fe4f4ea9ba729235fc
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.
Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
When we use amarda to deploy openstack service for ipv6, rabbitmq
pod could not start listen on [::]:5672 and [::]:15672.
For ipv6, we need do some override as below.
conf:
rabbitmq:
management.listener.port: 15672
management.listener.ip: "::"
rabbitmq_env: |
SERVER_ADDITIONAL_ERL_ARGS="+A 128 -kernel inetrc '/etc/rabbitmq/erl_inetrc' -proto_dist inet6_tcp"
CTL_ERL_ARGS="-proto_dist inet6_tcp"
erl_inetrc: |
{inet6,true}.
We have test pass on both ipv4 and ipv6 setup for StarlingX project.
Signed-off-by: Zhipeng Liu <zhipengs.liu@intel.com>
Change-Id: I7af840ecd8960f9f1aa3f38d155c6e1bd822cb6e
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.
Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.
Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
This PS looks to make a few small tweaks to the rabbitmq probes so
that its health and readiness is more reflective of what is actually
happening inside the container. We were previously seeing instances
of the pod marked as ready before it actually was.
Change-Id: If48ec02d4050f7385e71c2e6fe0fff8f59667af4
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.
Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.
Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
By default erlang VM determines a number of scheduler threads equal to a
number of CPU cores it detects [0]. Running rabbitmq in container makes
Erlang VM to think it has all host CPU power, making extra scheduler
threads competing for CPU time and, depending on a difference between
a number host CPU cores and container limits, causing CPU throttling even
while idle.
This commit limits a number of schedulers to a value actually available
to container via k8s resource limits (min 1) emulating the default
behavior.
[0] https://www.rabbitmq.com/runtime.html#scheduling
Change-Id: If36f63173de4c8035daf7aac4014c027c579b58f
This adds three new variables:
- skip_queues is for ability to skip metrics for some queues
- include_queues is the opposite parameter for presice setup
- rabbit_exporters is for ability to enable/disable exporter modules
Change-Id: Ia81a9921be6c14ec2035009fd164aab4c912f328
Some infra charts still have old ocata xenial images as default. This
should bring them up to date with the OSH charts.
Change-Id: If8454b6d0fe52387bf6327501ee4ff87f56e87b8
Signed-off-by: Tin Lam <tin@irrational.io>
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
The currently defined RabbitMQ Exporter probes make a call to the
"/metrics" path of the exporter service, which downloads a huge file and
takes a very long time to download. An http probe should be based on a very
simple and short url response from the service. So this changes the
probes to just call the base path "/" of the url and set the timeout to
something reasonable like 5 seconds.
Change-Id: Ie106490b2fb8d61660663f39a992bf4dc1a61222
There is a duplicate network_policy: key (one for ingress and one for
egress). This patch set fixes the netpol override yaml so it is correct.
Change-Id: I0df65ce248c010b5cf6e54515cfa10206436fa6c
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set updates and tests the apiVersion for rbac.authorization.k8s.io
from v1beta1 to v1 in preparation for its removal in k8s 1.20.
Change-Id: I4e68db1f75ff72eee55ecec93bd59c68c179c627
Signed-off-by: Tin Lam <tin@irrational.io>
Currently using envsubst to perform substitution of value overrides in
the feature gate caused conflicts as gotpl gets templated into those
overrides. This adds in '%%%REPLACE_${var}%%%' and uses sed to perform
the substitution instead to address the issue.
Change-Id: I9d3d630b53a2f3d828866229a5072bb04440ae15
Signed-off-by: Tin Lam <tin@irrational.io>
The patch fixes typo in PUBLISH_PORT and adds quotes for
PUBLISH_PORT because of the fact that it is string values
otherwise it leads to the error below:
error updating the release: rpc error: code = Unknown desc = release
rabbitmq failed: Deployment in version "v1" cannot be handled as
a Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec:
v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.Value:
ReadString: expects " or n, but found 9, error found in #10 byte of ...|,"value":9095},{"nam|...,
bigger context ...|value":"no_sort"},{"name":"PUBLISH_PORT","value":9095},{"name":"LOG_LEVEL","value":"info"},{"name":"|...
Change-Id: I027c91ee48df8eb5b4b2bf3fd28036b8eca47238
This patch set places logic to generate kubernetes egress network policy
rule based on the dependencies specified in values.yaml. This also sets
up the necessary default network policy for the OSH gate.
Change-Id: I1ac649cc9debb5d1f4ea0a32f506dcda4d8b8536
Signed-off-by: Tin Lam <tin@irrational.io>
This change replaces direct references to the exporter port
in values.yaml with calls to helm-toolkit lookup functions.
The referenced port number under the network key is removed,
as the helm-toolkit function will return the port number under
the endpoints key.
Change-Id: Ib6f533c49af5a88fca377920d28d5468d7387892
Trivial change. This patch set cleans up a python script.
- Move the comment to a helm-template comment so the python comments do
not get rendered by helm.
- Remove an unused python module.
Change-Id: Id287ddae8904d2cfa88725277bb97cf027a942c3
Signed-off-by: Tin Lam <tin@irrational.io>
The patch adds Network Policy ingress rules for RabbitMQ
and Prometheus RabbitMQ exporter.
It also fixes name generation for network policies,
to make sure they do not contain a prohibited '_' symbol,
which may appear in some label names.
Change-Id: I9821983b61d90e73e62c5ac669eefeb4ba9999d2
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintained
Change-Id: I5bfdc156ae228ab16da57569ac6b05a9a125cb6a
Signed-off-by: Steve Wilkerson <sw5822@att.com>
This change adds network policy overrides for multiple infra
services for the openstack-helm network policy gate.
Change-Id: If051ec1749cb9ed1e289f0cf82a8876371e36531
This change adds egress rules to the following charts:
- ingress
- memcache
- libvirt
- rabbitmq
These rules will be tightend down in future changes
Change-Id: I6f297d50ca4c06234c7c79986a12cccf3beb5efb
This change makes rabbitmq container run with the rabbitmq user
instead of the root user. As the rabbitmq user doesn't have write
access to '/run' directory, the templates are updated to use the
'/tmp' directory instead which the rabbitmq user has write access
to.
Change-Id: Ia35c3f741fefe3172c93bb042bf8d26bf7672cfc
This PS udpated the reset node function to leave the assets generated
via init containers in place when resetting the node.
Change-Id: Iac52ca82e95bb372dbcbca0eeea3b262215e9c12
Signed-off-by: Pete Birley <pete@port.direct>
When upgrading/reconfiguring a rabbit cluster its possible that the nodes
will not return the cluster status for some time, this ps allows us to
cope with this much more gracefully than simply crashing a few times, before
proceeding.
Change-Id: Ibf525df9e3a9362282f70e5dbb136430734181fd
Signed-off-by: Pete Birley <pete@port.direct>
This PS updates the cluster wait job to prune any extra nodes from
the cluster if scaling down.
Change-Id: I58d22121a07cd99448add62502582a6873776622
Signed-off-by: Pete Birley <pete@port.direct>
This PS extends the rabbit startup locgic to ensure nodes have
actually joined the cluster on startup.
Change-Id: Ib876d9abd89209d0a7972983bdf4daacf5f8f582
Signed-off-by: Pete Birley <pete@port.direct>
This works well for python2, but things will become
messy when py3 will be the default.
This, at the same time, ensures the KeyErrors are
properly logged, with a way to debug them.
Change-Id: If5d8007bece9ccbff481187e757968e7d1b6f651