Since fb08d477eb7bf5e678b9cd99b44a435842a7dfbf was merged, storlets
invocation on proxy fails because the close method is called even if
iterator has list_iterator type which doesn't have the close method.
This change replaces the direct call of the close method by
close_if_possible, to avoid Exception if the response iterator is
list or tuple, which don't support the close method.
Change-Id: I851cbb17465b5693dda5206aeae0a3191c9fe4f4
Closes-Bug: #1901453
They finally got rid of Thread.isAlive, having added Thread.is_alive as
an alias back in 2.6.
array.tostring is also gone; use array.tofile instead, but only on py3.
Change-Id: I0b7e426c60118a5df879a8908729d7feb12dfc1b
Capture the on the wire status code for logging because we change the
logged status code sometimes.
Closes-Bug: #1896518
Change-Id: I27feabe923a6520e983637a9c68a19ec7174a0df
It's reasonably common that apps or middlewares may send back headers
via dict.items(), but WSGIContext users often expect the headers to be
a list. Convert it for them, to avoid errors like
AttributeError: 'dict_items' object has no attribute 'append'
Change-Id: I4d061fad4da370c1cbb77ab78a55133319ea2dd7
docs: Removing the use of NameVirtualHost from the apache examples
It's not used anymore. It's deprecated in fact: https://httpd.apache.org/docs/2.4/mod/core.html#namevirtualhost
Change-Id: I76999cfacc10a244024ee0cca66dda95a0169a67
docs: Added more spacing to the apache2 examples
They're easier to read and a bit less bloated.
Change-Id: I5e21a66018b7ef309918fbbde93f2494286d291e
docs: Switching to /srv/www to be more FHS 3.0 conformat
It's more modern and well supported to use /srv/www now in place of
/var/www.
Change-Id: Icd09ed4d5fb4e2b9b84ddead21313ea1c0a87c91
ref: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s17.html
docs: Added user, group and display name in WSGI examples
This properly sets the user and group for the wsgi processes in the
examples; as well as adding a display name for easier identification.
Change-Id: Ie5783081e4054e5b2fbf3a856716101a1aaf61b8
docs: Replace apachectl for systemctl commands
It's safe to asume that all modern distros; supported by OpenStack, will
have systemd implemented. It's better to favor systemctl in those cases.
Change-Id: Ic0d2e47c1ac53502ce638d6fc2424ab9df037262
docs: Emphasis to file paths and command options
I've enclosed configuration options or parameters in interpreted text
quotes.
Also, I've enclosed fiel paths with inline literal quotes.
Change-Id: Iec54b7758bce01fc8e8daff48498383cb70c62ce
docs: Fixed wording used to indicate the restart of apache
Just a little commit to make it clearer of what we're gonna do.
Change-Id: Id5ab3e94519bcfe1832b92e456a1d1fa81dd54e3
Add file to the reno documentation build to show release notes for
stable/victoria.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.
Change-Id: I229f74529a718be2dfd9627a9b60f677b67578c6
Sem-Ver: feature
When upgrading from liberasurecode<=1.5.0, you may want to continue
writing legacy CRCs until all nodes are upgraded and capabale of reading
fragments with zlib CRCs.
Starting in liberasurecode>=1.6.2, we can use the environment variable
LIBERASURECODE_WRITE_LEGACY_CRC to control whether we write zlib or
legacy CRCs, but for many operators it's easier to manage swift configs
than environment variables. Add a new option, write_legacy_ec_crc, to the
proxy-server app and object-reconstructor; if set to true, ensure legacy
frags are written.
Note that more daemons instantiate proxy-server apps than just the
proxy-server. The complete set of impacted daemons should be:
* proxy-server
* object-reconstructor
* container-reconciler
* any users of internal-client.conf
UpgradeImpact
=============
To ensure a smooth liberasurecode upgrade:
1. Determine whether your cluster writes legacy or zlib CRCs. Depending
on the order in which shared libraries are loaded, your servers may
already be reading and writing zlib CRCs, even with old
liberasurecode. In that case, no special action is required and
WRITING LEGACY CRCS DURING THE UPGRADE WILL CAUSE AN OUTAGE.
Just upgrade liberasurecode normally. See the closed bug for more
information and a script to determine which CRC is used.
2. On all nodes, ensure Swift is upgraded to a version that includes
write_legacy_ec_crc support and write_legacy_ec_crc is enabled on
all daemons.
3. On each node, upgrade liberasurecode and restart Swift services.
Because of (2), they will continue writing legacy CRCs which will
still be readable by nodes that have not yet upgraded.
4. Once all nodes are upgraded, remove the write_legacy_ec_crc option
from all configs across all nodes. After restarting daemons, they
will write zlib CRCs which will also be readable by all nodes.
Change-Id: Iff71069f808623453c0ff36b798559015e604c7d
Related-Bug: #1666320
Closes-Bug: #1886088
Depends-On: https://review.opendev.org/#/c/738959/
...and check for it *before* doing the MD5 check. We see this happen
ocassionally, but as best I can tell, it's always due to a
ChunkReadTimeout popping in the proxy that it can't recover from.
Change-Id: If238725bbec4fc3f6c8d000599c735a7c4972f7d
We're getting some blockage trying to feed backup requests in waterfall
EC because the pool_size was limited to the initial batch of requests.
This was (un?)fortunately working out in practice because there were
lots of initial primary fragment requests and some would inevitably be
quick enough to make room for the pending feeder requests. But when
enough of the initial requests were slow (network issue at the proxy?)
we wouldn't have the expected number of pending backup requests
in-flight. Since concurrent EC should never make extra requests to
non-primaries (at least not until an existing primary request
completes) ec_n_unique_fragments makes a reasonable cap for the pool.
Drive-bys:
* Don't make concurrent_ec_extra_requests unless you have enabled
concurrent_gets.
* Improved mock_http_connect extra requests tracking formatting
* FakeStatus __repr__'s w/ status code in AssertionErrors
Change-Id: Iec579ed874ef097c659dc80fff1ba326b6da05e9
This used to be the name of the py2 job, but now even that's gone;
I'm not sure how Zuul let us get away with this.
Change-Id: I1a2f22d592fc7245e40c645000026e4ba1fca528
Related-Change: Ia9ae0fc226dfc9b40157faebac100c10a9180c62
We fixed swift-dispersion-report already; -populate needed the same fix
or else it'd hit a "maximum recursion depth exceeded" error.
Change-Id: I2d22e087a88c9e8003621feb26460ab6e5ce2a57
Related-Change: I24f4bcc3d62dc37fd9559032bfd25f5b15f98745
Closes-Bug: #1895346
Related-Bug: #1863680
Otherwise, we miss out on transaction id and client IP information when
timeouts pop.
Closes-Bug: #1892421
Change-Id: I6dea3ccf780bcc703db8447a2ef13c33838ff12d
During a rebalance, it's expected that we may get a 404 for data that
does exist elsewhere in the cluster. Normally this isn't a problem; the
proxy sees the 404, keeps digging, and one of the other primaries will
serve the response.
Previously, if the other replicas were heavily loaded, the proxy would
see a bunch of timeouts and the fresh (empty) primary, treat the 404 as
good, and send that on to the client.
Now, have the proxy throw out that first 404 (provided it doesn't have a
timestamp); it will then return a 503 to the client, indicating that it
should try again.
Add a new (per-policy) proxy-server config option,
rebalance_missing_suppression_count; operators may use this to increase
the number of 404-no-timestamp responses to discard if their rebalances
are going faster than replication can keep up, or set it to zero to
return to the previous behavior.
Change-Id: If4bd39788642c00d66579b26144af8f116735b4d