This patch adds some more debug logs into agent processes, so that
we can observe their behavior more in detail.
Also, this patch removes some detailed informations like pids from
responses returned to gateway/runtime, because gateway/runtime doesn't
require the details of agent behavior.
Change-Id: I2fd9998966120c46414e607b94ce03bee884e11f
This patch is the final refactoring work to have client library
for sbus communication, and port current implementaion about
execute command from runtime to SBusClient.
This patch makes execute command rely on the same communication
as the other commands, which happens on SERVICE_OUT_FD, and remove
TASK_ID_OUT_FD which is used only for returning task id.
This allows us to avoid special handling for execute command.
Change-Id: I00dc7b3004512624f9aaf120420f910e451cac89
Use Python 3 to run storlet application when runtime version is not
specified.
Also fix py3 compatibility for some python storlet applications used
for functional tests.
Change-Id: Ic33bb339339e9ff446fd308663c692db834f2a81
This patch refactors package installations in storlets devstack
plugin, so that we can run storlets agents with the same python
version as the one we have in host to run storlets middleware.
This patch also bumps up openjdk version from 8 to 11, and python3
version from 3.5 to 3.6, so that we use the latest versin provided
in distro.
Change-Id: I6cb87976ff4fa6caaa58cf555685f24d95f214a8
Because python-swiftclient returns bytes response
body so the concat of the body iter should be defined
as bytes.
* bonus: remove unnecessary "hoge" print output
Change-Id: I0cd41055ba6c840b412d5f750ff2fc7e2c59f320
The execution result header in user metadata looks invalid
and it cuases the hang in the func CI tessting.
So let's remove the dangerous white space in the http
header key with this patch.
The skip test removed in this patch has been since
https://review.opendev.org/#/c/693549 . The skipping
purpose was to progress py3 migration (we'd like to find
any other issues than this at the time) so the test is still
worth, then this fix enables to pass the test.
Change-Id: Ib41735d8801f0c99ed0563213dcf414fefb5cbb2
...since we didn't provide any encoding. Also, open files in binary mode
when hashing to compare against ETags.
Change-Id: Ie8ac1d1ff0216d945b1b1602c944cb62234a84a8
Lots of str-needing-to-be-bytes fixes, mostly.
Seems to be an issue with tests around executing? Skipped the two that
would just hang, but test_deploy_storlet might be a good place to start
poking around.
Change-Id: I99a6beda9c3a8883d9f13a953cba13aa21d847bb
because the notebook is strictly tightened to python2.
Re-writing the python2 notebook in python3 will cause
an error or a failure in python2 functional test, so
add skipTest check for now.
In the future, we could switch the available version
or have a couple of tests for both versions.
Change-Id: I9ec5cc107459266de509d3c444c5294f965dd749
... as we don't expect long codes are included in __init__.py .
This patch migrates these implementations to new independent
files, with keeping alias so that we can import things with
the same path.
Change-Id: Iff0b60cd53281c999c930a4e789aaaab360b1c7a
Because swiftclient.put_object assumes it as bytes object.
Only ZipFile is not suppoting rb mode so leave it as it is.
It should be okay because Storlets use it just to pick up
the filnames that should be string instead of bytes.
Change-Id: I834b29886a18269de2745ffe584291491e80f9eb
These tests are skipped because we use swift codes which weren't
compatible with Py2, but now we are ready to use them in Py3.
Change-Id: I717686c7d63bb0b21d25f534ba439e611ea2f5bc
This patch bundiles a file descriptor and its metadata
into one class named SBusFileDescriptor, so that we can
keep the relationship between them easily.
Change-Id: Ib042f2a7bd4bd524e9b4b8d155ec192edaaca7f0
This patch normalizes method name for RunTimePaths, according
to the following policy, so that it is more easy to understand
the characteristics of each method functions
- Use @propery for the functions without any arguments
- Use get_foo for the functions which take arguments
- Use foo_dir for directory paths
This patch also fixes wrong path handling in storlet app log.
Change-Id: I9275a84a2881b29a91eb55efaa6bc5607f1e723c
Now we use very simple format using ':' as a separator return
task status and a message related to the task operation, but
that protocol makes it difficult to add more informations into
the response.
This patch changes its format to json, so that we can include
more information keys/values into the response easily.
Change-Id: If2d609fb2ba2187e121e9cadd281017af46a6661
This patch introduces basic implementation of the client to communicate
with remote process using sbus, to gather all codes which deeply depend
on the sbus protocol into the one place.
This patch also introduces sbus CLI, which enables operators to send
sbus command like halt easily.
Change-Id: Ie1721c016b10b6562b131eb39e3dc84b1d8d473f
Previously we place all lib files under /usr/local/lib/storlets, but
this makes it more difficult to have lang specific files there.
This patch creates java directory where we place jar files, so that
we can place libraries for other languages like python in the future.
Change-Id: Id384c72ab6e658c55312ad62499dfc2100ba3ca3
StorletRequest is used to parse basic options for storlet
excecutions. This patch makes language_version also a mamber
of the class, which is currently handled outside it.
Change-Id: Id887fb0eaab528005a946013f9faf6ddca82bf42
Previously, if we fail to start a storlet container using tenant
image, we retry to start it using default tenant image. However,
that sometimes causes unexpected retry, for example when there is
something wrong in tenant image which prevents daemon factory to
launch in time.
This patch makes docker runtime to check return code of container
restarting script, so that we only restart container when the
runtime fails in container operation using tenant image.
In addition, it lets docker runtime to log stdout/stderr output
generated by restart_docker_container, to help us to debug what
is happening about container management.
Change-Id: Icd81d28b9b3555bce365e7b279ada074d239fddc
That is just calling s2aio.sh for our functional gate but
just calling it at the playbook is easier so let's remove
the unnecessary bash shell.
Change-Id: Ic2820c681295e2e1b68bbf91d6fa70de266cdedc
