Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
[11:05:17 PDT] Transferring environment variable script to build VM
[11:05:17 PDT] Transferring kokoro_log_reader.py to build VM
[11:05:17 PDT] Transferring source code to build VM
[11:05:20 PDT] Executing build script on build VM



[ID: 2881275] Executing command via SSH:
export KOKORO_BUILD_NUMBER="548"
export KOKORO_JOB_NAME="grpc/core/master/linux/psm-csm-python"
source /tmpfs/kokoro-env_vars.sh; cd /tmpfs/src/ ; chmod 755 github/grpc/tools/internal_ci/linux/psm-interop-test-python.sh ; PYTHON_3_VERSION="$(pyenv which python3 2> /dev/null || which python3)" ; PYTHON_2_VERSION="$(pyenv which python2 2> /dev/null || which python2)" ; if "$PYTHON_3_VERSION" -c "import psutil" ; then KOKORO_PYTHON_COMMAND="$PYTHON_3_VERSION" ; else KOKORO_PYTHON_COMMAND="$PYTHON_2_VERSION" ; fi > /dev/null 2>&1 ; echo "export KOKORO_PYTHON_COMMAND="$KOKORO_PYTHON_COMMAND"" > "$HOME/.kokoro_python_vars" ; nohup bash -c "( rm -f /tmpfs/kokoro_build_exit_code ; github/grpc/tools/internal_ci/linux/psm-interop-test-python.sh ; echo \${PIPESTATUS[0]} > /tmpfs/kokoro_build_exit_code ) > /tmpfs/kokoro_build.log 2>&1" > /dev/null 2>&1 & echo $! > /tmpfs/kokoro_build.pid ; source "$HOME/.kokoro_python_vars" ; "$KOKORO_PYTHON_COMMAND" /tmpfs/kokoro_log_reader.py /tmpfs/kokoro_build.log /tmpfs/kokoro_build_exit_code /tmpfs/kokoro_build.pid /tmpfs/kokoro_log_reader.pid --start_byte 0

Sourcing test driver install script from: https://raw.githubusercontent.com/grpc/psm-interop/main/.kokoro/psm_interop_kokoro_lib.sh
Install script checksum:
d80fe84413d3c0a950fe90ffb8746024a37cee1fd3cec65f1e6ee50b9ebb3bed /tmp/tmp.zIwRIehjbI/psm_interop_kokoro_lib.sh
+ [18:05:21 UTC] Starting PSM Interop tests: csm
+ [18:05:21 UTC] Activated GKE cluster: GKE_CLUSTER_NAME=psm-interop-csm GKE_CLUSTER_REGION=us-central1
+ [18:05:21 UTC] Starting Kokoro provisioning
+ [18:05:21 UTC] Kokoro Ubuntu version:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.5 LTS
Release: 22.04
Codename: jammy
Python 3.10.12
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
+ [18:05:22 UTC] Sponge properties:
TESTS_FORMAT_VERSION,2
TESTGRID_EXCLUDE,0
GIT_ORIGIN_URL,https://github.com/XuanWang-Amos/grpc.git
GIT_COMMIT_SHORT,78f6c9348a
+ [18:05:22 UTC] Installing packages with apt, see install-apt.log
+ [18:06:09 UTC] Fetching GKE cluster credentials
Fetching cluster endpoint and auth data.
kubeconfig entry generated for psm-interop-csm.
+ [18:06:12 UTC] Cloning driver to https://github.com/grpc/psm-interop.git branch main to /tmp/tmp.AYSR3QX6TE/psm-interop
Cloning into '/tmp/tmp.AYSR3QX6TE/psm-interop'...
+ [18:06:13 UTC] Install python dependencies
+ [18:06:13 UTC] Creating python virtual environment: /tmp/tmp.AYSR3QX6TE/psm-interop/venv
+ [18:06:21 UTC] Installing Python packages with pip, see install-pip.log
+ [18:06:34 UTC] Generate python code from grpc.testing protos: protos/grpc/testing/test.proto protos/grpc/testing/messages.proto protos/grpc/testing/empty.proto protos/grpc/testing/xdsconfig/xdsconfig.proto
/tmp/tmp.AYSR3QX6TE/psm-interop/venv/lib/python3.10/site-packages/grpc_tools/protoc.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
+ [18:06:34 UTC] Generated files /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing:
e13416ba600677adc0b415ce3ec1cf89436a2b1d3200288202431c603e91d5db /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/xdsconfig/xdsconfig_pb2_grpc.py
ee91fba056e68ad9771e5d768b9aab8a2d5014ead3e82dd88f8b188f54ac43f0 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/xdsconfig/xdsconfig.proto
0ab3d5b39620603bd3cc10095df03c24a7fe6d99988f4d8ceb46a3674405d6ca /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/xdsconfig/xdsconfig_pb2.py
07f7894ac17798456c040acf1f558b5c6f3e7a690a20d9cd6693b2867b2d877f /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/messages.proto
bc10f5a4e285257916ed1327b8d17dac189ba5376b7dff97be18d5634b2e43c3 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/empty_pb2.py
fc0b620c26462272b1f1719fa46ca343d395cfdf6804d09e69fa0fb63d0e73ee /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/test_pb2_grpc.py
49ac67a669d49b2626b9e96bf3bffc38f3b6cbbfef544426f631c7599f0ab51f /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/test.proto
d686e804f171693117b7d030ec4023f205c70c234c8590f6557aa8702f65fe09 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/messages_pb2_grpc.py
81a8fd5745a37450f86af777c41636c8c9eba3244157eeb3c66fafc011d631ba /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/test_pb2.py
20d9f7b4cbdc299d82e8dec6019bbfc546534b51056824229777cdc9976f16a2 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/empty.proto
aee633ecdbef187b7d4cc485b4d76264a62506639366d0e2532f863254726300 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/messages_pb2.py
d686e804f171693117b7d030ec4023f205c70c234c8590f6557aa8702f65fe09 /tmp/tmp.AYSR3QX6TE/psm-interop/protos/grpc/testing/empty_pb2_grpc.py
+ [18:06:36 UTC] Client image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
Client image not found
+ [18:06:37 UTC] Server image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
Server image not found
+ [18:06:37 UTC] Building xDS interop test app Docker images
Adding credentials for: us-docker.pkg.dev
Docker configuration file updated.
/tmpfs/src/github/grpc /tmp/tmp.AYSR3QX6TE/psm-interop
+ [18:06:38 UTC] Building python xDS interop test client
+ [18:06:38 UTC] docker build -f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.client -t us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 .
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile.client
#1 transferring dockerfile: 1.20kB done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/python:3.9-slim-bookworm
#2 DONE 0.7s

#3 [internal] load .dockerignore
#3 transferring context: 89B done
#3 DONE 0.0s

#4 [stage-0 1/10] FROM docker.io/library/python:3.9-slim-bookworm@sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc
#4 resolve docker.io/library/python:3.9-slim-bookworm@sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc 0.0s done
#4 sha256:4c0965d3919510b506d8856ebc050a96e996c7dae96e4fb420882dbe7e037e67 0B / 3.51MB 0.1s
#4 sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 0B / 14.74MB 0.1s
#4 sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc 10.41kB / 10.41kB done
#4 sha256:93ab151da4e5310ea79c4ecf306ece628262b86a4d7a49cc601664f19fe44e36 1.75kB / 1.75kB done
#4 sha256:9d8cb7037cd8e90893e5f430ce4c048a872511e414580c7641675f2dad0a0351 5.20kB / 5.20kB done
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 0B / 29.13MB 0.1s
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 3.15MB / 29.13MB 0.2s
#4 ...

#5 [stage-0 9/10] ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
#5 DONE 0.3s

#4 [stage-0 1/10] FROM docker.io/library/python:3.9-slim-bookworm@sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc
#4 sha256:4c0965d3919510b506d8856ebc050a96e996c7dae96e4fb420882dbe7e037e67 2.10MB / 3.51MB 0.3s
#4 sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 3.15MB / 14.74MB 0.3s
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 10.49MB / 29.13MB 0.3s
#4 sha256:4c0965d3919510b506d8856ebc050a96e996c7dae96e4fb420882dbe7e037e67 3.51MB / 3.51MB 0.3s done
#4 sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 8.39MB / 14.74MB 0.4s
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 14.68MB / 29.13MB 0.4s
#4 sha256:62a08b8dd4f53ad5493dabf2af00ccde91abb3771fb2187040bcf2fe94a7ced7 0B / 248B 0.4s
#4 sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 14.74MB / 14.74MB 0.5s done
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 20.97MB / 29.13MB 0.5s
#4 sha256:62a08b8dd4f53ad5493dabf2af00ccde91abb3771fb2187040bcf2fe94a7ced7 248B / 248B 0.4s done
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 29.13MB / 29.13MB 0.6s
#4 sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 29.13MB / 29.13MB 0.6s done
#4 extracting sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab
#4 ...

#6 [internal] load build context
#6 transferring context: 96.36MB 2.0s done
#6 DONE 2.1s

#4 [stage-0 1/10] FROM docker.io/library/python:3.9-slim-bookworm@sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc
#4 extracting sha256:302e3ee498053a7b5332ac79e8efebec16e900289fc1ecd1c754ce8fa047fcab 1.7s done
#4 extracting sha256:4c0965d3919510b506d8856ebc050a96e996c7dae96e4fb420882dbe7e037e67 0.1s
#4 extracting sha256:4c0965d3919510b506d8856ebc050a96e996c7dae96e4fb420882dbe7e037e67 0.1s done
#4 extracting sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 0.1s
#4 extracting sha256:fdeeec85abbad3878f2008f9445f15a19a5a224d1b7e7715ac6b923072333e57 0.7s done
#4 extracting sha256:62a08b8dd4f53ad5493dabf2af00ccde91abb3771fb2187040bcf2fe94a7ced7 done
#4 DONE 3.4s

#7 [stage-0 2/10] RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
#7 0.287 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#7 0.322 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#7 0.335 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#7 0.411 Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8787 kB]
#7 0.519 Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [2468 B]
#7 0.550 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [188 kB]
#7 1.833 Fetched 9232 kB in 2s (5869 kB/s)
#7 1.833 Reading package lists...
#7 2.567 Reading package lists...
#7 3.297 Building dependency tree...
#7 3.473 Reading state information...
#7 3.521 Calculating upgrade...
#7 3.769 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#7 3.778 Reading package lists...
#7 4.478 Building dependency tree...
#7 4.651 Reading state information...
#7 4.894 The following additional packages will be installed:
#7 4.894 binfmt-support binutils binutils-common binutils-x86-64-linux-gnu bzip2
#7 4.894 clang-14 cpp cpp-12 dirmngr dpkg-dev fakeroot fontconfig-config
#7 4.894 fonts-dejavu-core g++ g++-12 gcc gcc-12 gnupg gnupg-l10n gnupg-utils gpg
#7 4.894 gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm icu-devtools
#7 4.894 lib32gcc-s1 lib32stdc++6 libabsl20220623 libalgorithm-diff-perl
#7 4.894 libalgorithm-diff-xs-perl libalgorithm-merge-perl libaom3 libasan8
#7 4.894 libassuan0 libatomic1 libavif15 libbinutils libbrotli1 libbsd0 libc-dev-bin
#7 4.894 libc-devtools libc6-dev libc6-i386 libcc1-0 libclang-common-14-dev
#7 4.894 libclang-cpp14 libclang-rt-14-dev libclang1-14 libcrypt-dev libctf-nobfd0
#7 4.894 libctf0 libcurl3-nss libcurl4 libdav1d6 libde265-0 libdeflate0 libdpkg-perl
#7 4.894 libedit2 libfakeroot libffi-dev libfile-fcntllock-perl libfontconfig1
#7 4.895 libfreetype6 libgav1-1 libgc1 libgcc-12-dev libgd3 libgdbm-compat4 libgomp1
#7 4.895 libgpm2 libgprofng0 libheif1 libicu-dev libicu72 libisl23 libitm1
#7 4.895 libjansson4 libjbig0 libjpeg62-turbo libksba8 libldap-2.5-0 libldap-common
#7 4.895 liblerc4 libllvm14 liblocale-gettext-perl liblsan0 libmpc3 libmpfr6
#7 4.895 libncurses-dev libncurses6 libnghttp2-14 libnpth0 libnsl-dev libnspr4
#7 4.895 libnss3 libnuma1 libobjc-12-dev libobjc4 libperl5.36 libpfm4 libpipeline1
#7 4.895 libpng16-16 libpsl5 libpython3-stdlib libpython3.11-minimal
#7 4.895 libpython3.11-stdlib libquadmath0 librav1e0 librtmp1 libsasl2-2
#7 4.895 libsasl2-modules libsasl2-modules-db libssh2-1 libstdc++-12-dev
#7 4.895 libsvtav1enc1 libtiff6 libtinfo-dev libtirpc-dev libtsan2 libubsan1 libwebp7
#7 4.895 libx11-6 libx11-data libx265-199 libxau6 libxcb1 libxdmcp6 libxml2
#7 4.895 libxml2-dev libxpm4 libyaml-0-2 libyuv0 libz3-4 libz3-dev linux-libc-dev
#7 4.895 llvm-14 llvm-14-dev llvm-14-linker-tools llvm-14-runtime llvm-14-tools make
#7 4.895 manpages manpages-dev media-types nss-plugin-pem patch perl
#7 4.896 perl-modules-5.36 pinentry-curses publicsuffix python3 python3-minimal
#7 4.896 python3-pkg-resources python3-pygments python3-yaml python3.11
#7 4.896 python3.11-minimal rpcsvc-proto xz-utils
#7 4.898 Suggested packages:
#7 4.898 binutils-doc bzip2-doc clang-14-doc wasi-libc cpp-doc gcc-12-locales
#7 4.898 cpp-12-doc dbus-user-session libpam-systemd pinentry-gnome3 tor
#7 4.898 debian-keyring g++-multilib g++-12-multilib gcc-12-doc gcc-multilib autoconf
#7 4.898 automake libtool flex bison gdb gcc-doc gcc-12-multilib parcimonie
#7 4.898 xloadimage scdaemon glibc-doc sensible-utils git bzr libgd-tools gpm icu-doc
#7 4.898 ncurses-doc libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal
#7 4.898 libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql
#7 4.898 libstdc++-12-doc pkg-config llvm-14-doc make-doc man-browser ed
#7 4.898 diffutils-doc perl-doc libterm-readline-gnu-perl
#7 4.898 | libterm-readline-perl-perl libtap-harness-archive-perl pinentry-doc
#7 4.898 python3-doc python3-tk python3-venv python3-setuptools python-pygments-doc
#7 4.898 ttf-bitstream-vera python3.11-venv python3.11-doc
#7 6.040 The following NEW packages will be installed:
#7 6.040 binfmt-support binutils binutils-common binutils-x86-64-linux-gnu
#7 6.040 build-essential bzip2 clang clang-14 cpp cpp-12 curl dirmngr dpkg-dev
#7 6.040 fakeroot fontconfig-config fonts-dejavu-core g++ g++-12 gcc gcc-12 gnupg
#7 6.040 gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf
#7 6.040 gpgsm icu-devtools lib32gcc-s1 lib32stdc++6 libabsl20220623
#7 6.040 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl
#7 6.040 libaom3 libasan8 libassuan0 libatomic1 libavif15 libbinutils libbrotli1
#7 6.040 libbsd0 libc-dev-bin libc-devtools libc6-dev libc6-i386 libcc1-0
#7 6.040 libclang-common-14-dev libclang-cpp14 libclang-rt-14-dev libclang1-14
#7 6.040 libcrypt-dev libctf-nobfd0 libctf0 libcurl3-nss libcurl4 libdav1d6
#7 6.040 libde265-0 libdeflate0 libdpkg-perl libedit2 libfakeroot libffi-dev
#7 6.040 libfile-fcntllock-perl libfontconfig1 libfreetype6 libgav1-1 libgc1
#7 6.041 libgcc-12-dev libgd3 libgdbm-compat4 libgomp1 libgpm2 libgprofng0 libheif1
#7 6.041 libicu-dev libicu72 libisl23 libitm1 libjansson4 libjbig0 libjpeg62-turbo
#7 6.041 libksba8 libldap-2.5-0 libldap-common liblerc4 libllvm14
#7 6.041 liblocale-gettext-perl liblsan0 libmpc3 libmpfr6 libncurses-dev libncurses6
#7 6.041 libnghttp2-14 libnpth0 libnsl-dev libnspr4 libnss3 libnuma1 libobjc-12-dev
#7 6.041 libobjc4 libperl5.36 libpfm4 libpipeline1 libpng16-16 libpsl5
#7 6.041 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib libquadmath0
#7 6.041 librav1e0 librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db libssh2-1
#7 6.041 libstdc++-12-dev libsvtav1enc1 libtiff6 libtinfo-dev libtirpc-dev libtsan2
#7 6.041 libubsan1 libwebp7 libx11-6 libx11-data libx265-199 libxau6 libxcb1
#7 6.041 libxdmcp6 libxml2 libxml2-dev libxpm4 libyaml-0-2 libyuv0 libz3-4 libz3-dev
#7 6.041 linux-libc-dev llvm-14 llvm-14-dev llvm-14-linker-tools llvm-14-runtime
#7 6.042 llvm-14-tools make manpages manpages-dev media-types nss-plugin-pem patch
#7 6.042 perl perl-modules-5.36 pinentry-curses publicsuffix python3 python3-minimal
#7 6.042 python3-pkg-resources python3-pygments python3-yaml python3.11
#7 6.042 python3.11-minimal rpcsvc-proto xz-utils
#7 6.093 0 upgraded, 164 newly installed, 0 to remove and 0 not upgraded.
#7 6.093 Need to get 233 MB of archives.
#7 6.093 After this operation, 1165 MB of additional disk space will be used.
#7 6.093 Get:1 http://deb.debian.org/debian bookworm/main amd64 perl-modules-5.36 all 5.36.0-7+deb12u1 [2815 kB]
#7 6.175 Get:2 http://deb.debian.org/debian bookworm/main amd64 libgdbm-compat4 amd64 1.23-3 [48.2 kB]
#7 6.175 Get:3 http://deb.debian.org/debian bookworm/main amd64 libperl5.36 amd64 5.36.0-7+deb12u1 [4218 kB]
#7 6.203 Get:4 http://deb.debian.org/debian bookworm/main amd64 perl amd64 5.36.0-7+deb12u1 [239 kB]
#7 6.205 Get:5 http://deb.debian.org/debian bookworm/main amd64 libpipeline1 amd64 1.5.7-1 [38.5 kB]
#7 6.206 Get:6 http://deb.debian.org/debian bookworm/main amd64 binfmt-support amd64 2.2.2-2 [64.0 kB]
#7 6.207 Get:7 http://deb.debian.org/debian bookworm/main amd64 liblocale-gettext-perl amd64 1.07-5 [15.4 kB]
#7 6.207 Get:8 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-minimal amd64 3.11.2-6+deb12u3 [816 kB]
#7 6.212 Get:9 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11-minimal amd64 3.11.2-6+deb12u3 [2067 kB]
#7 6.221 Get:10 http://deb.debian.org/debian bookworm/main amd64 python3-minimal amd64 3.11.2-1+b1 [26.3 kB]
#7 6.221 Get:11 http://deb.debian.org/debian bookworm/main amd64 media-types all 10.0.0 [26.1 kB]
#7 6.222 Get:12 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-stdlib amd64 3.11.2-6+deb12u3 [1797 kB]
#7 6.230 Get:13 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11 amd64 3.11.2-6+deb12u3 [573 kB]
#7 6.233 Get:14 http://deb.debian.org/debian bookworm/main amd64 libpython3-stdlib amd64 3.11.2-1+b1 [9312 B]
#7 6.233 Get:15 http://deb.debian.org/debian bookworm/main amd64 python3 amd64 3.11.2-1+b1 [26.3 kB]
#7 6.233 Get:16 http://deb.debian.org/debian bookworm/main amd64 bzip2 amd64 1.0.8-5+b1 [49.8 kB]
#7 6.234 Get:17 http://deb.debian.org/debian bookworm/main amd64 manpages all 6.03-2 [1332 kB]
#7 6.243 Get:18 http://deb.debian.org/debian bookworm/main amd64 xz-utils amd64 5.4.1-0.2 [471 kB]
#7 6.247 Get:19 http://deb.debian.org/debian bookworm/main amd64 binutils-common amd64 2.40-2 [2487 kB]
#7 6.263 Get:20 http://deb.debian.org/debian bookworm/main amd64 libbinutils amd64 2.40-2 [572 kB]
#7 6.267 Get:21 http://deb.debian.org/debian bookworm/main amd64 libctf-nobfd0 amd64 2.40-2 [153 kB]
#7 6.269 Get:22 http://deb.debian.org/debian bookworm/main amd64 libctf0 amd64 2.40-2 [89.8 kB]
#7 6.270 Get:23 http://deb.debian.org/debian bookworm/main amd64 libgprofng0 amd64 2.40-2 [812 kB]
#7 6.275 Get:24 http://deb.debian.org/debian bookworm/main amd64 libjansson4 amd64 2.14-2 [40.8 kB]
#7 6.276 Get:25 http://deb.debian.org/debian bookworm/main amd64 binutils-x86-64-linux-gnu amd64 2.40-2 [2246 kB]
#7 6.290 Get:26 http://deb.debian.org/debian bookworm/main amd64 binutils amd64 2.40-2 [65.0 kB]
#7 6.291 Get:27 http://deb.debian.org/debian bookworm/main amd64 libc-dev-bin amd64 2.36-9+deb12u8 [46.3 kB]
#7 6.292 Get:28 http://deb.debian.org/debian-security bookworm-security/main amd64 linux-libc-dev amd64 6.1.112-1 [2043 kB]
#7 6.301 Get:29 http://deb.debian.org/debian bookworm/main amd64 libcrypt-dev amd64 1:4.4.33-2 [118 kB]
#7 6.302 Get:30 http://deb.debian.org/debian bookworm/main amd64 libtirpc-dev amd64 1.3.3+ds-1 [191 kB]
#7 6.303 Get:31 http://deb.debian.org/debian bookworm/main amd64 libnsl-dev amd64 1.3.0-2 [66.4 kB]
#7 6.304 Get:32 http://deb.debian.org/debian bookworm/main amd64 rpcsvc-proto amd64 1.4.3-1 [63.3 kB]
#7 6.305 Get:33 http://deb.debian.org/debian bookworm/main amd64 libc6-dev amd64 2.36-9+deb12u8 [1900 kB]
#7 6.318 Get:34 http://deb.debian.org/debian bookworm/main amd64 libisl23 amd64 0.25-1.1 [683 kB]
#7 6.322 Get:35 http://deb.debian.org/debian bookworm/main amd64 libmpfr6 amd64 4.2.0-1 [701 kB]
#7 6.328 Get:36 http://deb.debian.org/debian bookworm/main amd64 libmpc3 amd64 1.3.1-1 [51.5 kB]
#7 6.328 Get:37 http://deb.debian.org/debian bookworm/main amd64 cpp-12 amd64 12.2.0-14 [9764 kB]
#7 6.392 Get:38 http://deb.debian.org/debian bookworm/main amd64 cpp amd64 4:12.2.0-3 [6836 B]
#7 6.392 Get:39 http://deb.debian.org/debian bookworm/main amd64 libcc1-0 amd64 12.2.0-14 [41.7 kB]
#7 6.393 Get:40 http://deb.debian.org/debian bookworm/main amd64 libgomp1 amd64 12.2.0-14 [116 kB]
#7 6.394 Get:41 http://deb.debian.org/debian bookworm/main amd64 libitm1 amd64 12.2.0-14 [26.1 kB]
#7 6.394 Get:42 http://deb.debian.org/debian bookworm/main amd64 libatomic1 amd64 12.2.0-14 [9328 B]
#7 6.395 Get:43 http://deb.debian.org/debian bookworm/main amd64 libasan8 amd64 12.2.0-14 [2195 kB]
#7 6.410 Get:44 http://deb.debian.org/debian bookworm/main amd64 liblsan0 amd64 12.2.0-14 [969 kB]
#7 6.416 Get:45 http://deb.debian.org/debian bookworm/main amd64 libtsan2 amd64 12.2.0-14 [2196 kB]
#7 6.431 Get:46 http://deb.debian.org/debian bookworm/main amd64 libubsan1 amd64 12.2.0-14 [883 kB]
#7 6.437 Get:47 http://deb.debian.org/debian bookworm/main amd64 libquadmath0 amd64 12.2.0-14 [144 kB]
#7 6.439 Get:48 http://deb.debian.org/debian bookworm/main amd64 libgcc-12-dev amd64 12.2.0-14 [2437 kB]
#7 6.455 Get:49 http://deb.debian.org/debian bookworm/main amd64 gcc-12 amd64 12.2.0-14 [19.3 MB]
#7 6.579 Get:50 http://deb.debian.org/debian bookworm/main amd64 gcc amd64 4:12.2.0-3 [5216 B]
#7 6.579 Get:51 http://deb.debian.org/debian bookworm/main amd64 libstdc++-12-dev amd64 12.2.0-14 [2046 kB]
#7 6.593 Get:52 http://deb.debian.org/debian bookworm/main amd64 g++-12 amd64 12.2.0-14 [10.7 MB]
#7 6.662 Get:53 http://deb.debian.org/debian bookworm/main amd64 g++ amd64 4:12.2.0-3 [1356 B]
#7 6.663 Get:54 http://deb.debian.org/debian bookworm/main amd64 make amd64 4.3-4.1 [396 kB]
#7 6.666 Get:55 http://deb.debian.org/debian bookworm/main amd64 libdpkg-perl all 1.21.22 [603 kB]
#7 6.670 Get:56 http://deb.debian.org/debian bookworm/main amd64 patch amd64 2.7.6-7 [128 kB]
#7 6.672 Get:57 http://deb.debian.org/debian bookworm/main amd64 dpkg-dev all 1.21.22 [1353 kB]
#7 6.681 Get:58 http://deb.debian.org/debian bookworm/main amd64 build-essential amd64 12.9 [7704 B]
#7 6.681 Get:59 http://deb.debian.org/debian bookworm/main amd64 libbsd0 amd64 0.11.7-2 [117 kB]
#7 6.682 Get:60 http://deb.debian.org/debian bookworm/main amd64 libedit2 amd64 3.1-20221030-2 [93.0 kB]
#7 6.683 Get:61 http://deb.debian.org/debian bookworm/main amd64 libicu72 amd64 72.1-3 [9376 kB]
#7 6.744 Get:62 http://deb.debian.org/debian bookworm/main amd64 libxml2 amd64 2.9.14+dfsg-1.3~deb12u1 [687 kB]
#7 6.749 Get:63 http://deb.debian.org/debian bookworm/main amd64 libz3-4 amd64 4.8.12-3.1 [7216 kB]
#7 6.796 Get:64 http://deb.debian.org/debian bookworm/main amd64 libllvm14 amd64 1:14.0.6-12 [21.8 MB]
#7 6.943 Get:65 http://deb.debian.org/debian bookworm/main amd64 libclang-cpp14 amd64 1:14.0.6-12 [11.1 MB]
#7 7.020 Get:66 http://deb.debian.org/debian bookworm/main amd64 libgc1 amd64 1:8.2.2-3 [245 kB]
#7 7.022 Get:67 http://deb.debian.org/debian bookworm/main amd64 libobjc4 amd64 12.2.0-14 [43.1 kB]
#7 7.023 Get:68 http://deb.debian.org/debian bookworm/main amd64 libobjc-12-dev amd64 12.2.0-14 [170 kB]
#7 7.024 Get:69 http://deb.debian.org/debian bookworm/main amd64 libclang-common-14-dev all 1:14.0.6-12 [890 kB]
#7 7.030 Get:70 http://deb.debian.org/debian bookworm/main amd64 llvm-14-linker-tools amd64 1:14.0.6-12 [1288 kB]
#7 7.039 Get:71 http://deb.debian.org/debian bookworm/main amd64 libclang1-14 amd64 1:14.0.6-12 [6157 kB]
#7 7.081 Get:72 http://deb.debian.org/debian bookworm/main amd64 clang-14 amd64 1:14.0.6-12 [102 kB]
#7 7.082 Get:73 http://deb.debian.org/debian bookworm/main amd64 clang amd64 1:14.0-55.7~deb12u1 [5144 B]
#7 7.082 Get:74 http://deb.debian.org/debian bookworm/main amd64 libbrotli1 amd64 1.0.9-2+b6 [275 kB]
#7 7.084 Get:75 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg-10 [20.3 kB]
#7 7.085 Get:76 http://deb.debian.org/debian bookworm/main amd64 libsasl2-2 amd64 2.1.28+dfsg-10 [59.7 kB]
#7 7.085 Get:77 http://deb.debian.org/debian bookworm/main amd64 libldap-2.5-0 amd64 2.5.13+dfsg-5 [183 kB]
#7 7.087 Get:78 http://deb.debian.org/debian bookworm/main amd64 libnghttp2-14 amd64 1.52.0-1+deb12u1 [72.4 kB]
#7 7.087 Get:79 http://deb.debian.org/debian bookworm/main amd64 libpsl5 amd64 0.21.2-1 [58.7 kB]
#7 7.088 Get:80 http://deb.debian.org/debian bookworm/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2+b2 [60.8 kB]
#7 7.093 Get:81 http://deb.debian.org/debian bookworm/main amd64 libssh2-1 amd64 1.10.0-3+b1 [179 kB]
#7 7.104 Get:82 http://deb.debian.org/debian bookworm/main amd64 libcurl4 amd64 7.88.1-10+deb12u7 [390 kB]
#7 7.108 Get:83 http://deb.debian.org/debian bookworm/main amd64 curl amd64 7.88.1-10+deb12u7 [315 kB]
#7 7.110 Get:84 http://deb.debian.org/debian bookworm/main amd64 libassuan0 amd64 2.5.5-5 [48.5 kB]
#7 7.111 Get:85 http://deb.debian.org/debian bookworm/main amd64 gpgconf amd64 2.2.40-1.1 [564 kB]
#7 7.115 Get:86 http://deb.debian.org/debian bookworm/main amd64 libksba8 amd64 1.6.3-2 [128 kB]
#7 7.116 Get:87 http://deb.debian.org/debian bookworm/main amd64 libnpth0 amd64 1.6-3 [19.0 kB]
#7 7.116 Get:88 http://deb.debian.org/debian bookworm/main amd64 dirmngr amd64 2.2.40-1.1 [792 kB]
#7 7.122 Get:89 http://deb.debian.org/debian bookworm/main amd64 libfakeroot amd64 1.31-1.2 [28.3 kB]
#7 7.122 Get:90 http://deb.debian.org/debian bookworm/main amd64 fakeroot amd64 1.31-1.2 [66.9 kB]
#7 7.123 Get:91 http://deb.debian.org/debian bookworm/main amd64 fonts-dejavu-core all 2.37-6 [1068 kB]
#7 7.131 Get:92 http://deb.debian.org/debian bookworm/main amd64 fontconfig-config amd64 2.14.1-4 [315 kB]
#7 7.133 Get:93 http://deb.debian.org/debian bookworm/main amd64 gnupg-l10n all 2.2.40-1.1 [1093 kB]
#7 7.141 Get:94 http://deb.debian.org/debian bookworm/main amd64 gnupg-utils amd64 2.2.40-1.1 [927 kB]
#7 7.147 Get:95 http://deb.debian.org/debian bookworm/main amd64 gpg amd64 2.2.40-1.1 [949 kB]
#7 7.153 Get:96 http://deb.debian.org/debian bookworm/main amd64 pinentry-curses amd64 1.2.1-1 [77.4 kB]
#7 7.154 Get:97 http://deb.debian.org/debian bookworm/main amd64 gpg-agent amd64 2.2.40-1.1 [695 kB]
#7 7.159 Get:98 http://deb.debian.org/debian bookworm/main amd64 gpg-wks-client amd64 2.2.40-1.1 [541 kB]
#7 7.163 Get:99 http://deb.debian.org/debian bookworm/main amd64 gpg-wks-server amd64 2.2.40-1.1 [531 kB]
#7 7.167 Get:100 http://deb.debian.org/debian bookworm/main amd64 gpgsm amd64 2.2.40-1.1 [671 kB]
#7 7.171 Get:101 http://deb.debian.org/debian bookworm/main amd64 gnupg all 2.2.40-1.1 [846 kB]
#7 7.177 Get:102 http://deb.debian.org/debian bookworm/main amd64 icu-devtools amd64 72.1-3 [206 kB]
#7 7.179 Get:103 http://deb.debian.org/debian bookworm/main amd64 libc6-i386 amd64 2.36-9+deb12u8 [2459 kB]
#7 7.195 Get:104 http://deb.debian.org/debian bookworm/main amd64 lib32gcc-s1 amd64 12.2.0-14 [59.7 kB]
#7 7.195 Get:105 http://deb.debian.org/debian bookworm/main amd64 lib32stdc++6 amd64 12.2.0-14 [644 kB]
#7 7.200 Get:106 http://deb.debian.org/debian bookworm/main amd64 libabsl20220623 amd64 20220623.1-1 [391 kB]
#7 7.203 Get:107 http://deb.debian.org/debian bookworm/main amd64 libalgorithm-diff-perl all 1.201-1 [43.3 kB]
#7 7.203 Get:108 http://deb.debian.org/debian bookworm/main amd64 libalgorithm-diff-xs-perl amd64 0.04-8+b1 [11.4 kB]
#7 7.204 Get:109 http://deb.debian.org/debian bookworm/main amd64 libalgorithm-merge-perl all 0.08-5 [11.8 kB]
#7 7.204 Get:110 http://deb.debian.org/debian bookworm/main amd64 libaom3 amd64 3.6.0-1+deb12u1 [1851 kB]
#7 7.217 Get:111 http://deb.debian.org/debian bookworm/main amd64 libdav1d6 amd64 1.0.0-2+deb12u1 [513 kB]
#7 7.220 Get:112 http://deb.debian.org/debian bookworm/main amd64 libgav1-1 amd64 0.18.0-1+b1 [332 kB]
#7 7.223 Get:113 http://deb.debian.org/debian bookworm/main amd64 librav1e0 amd64 0.5.1-6 [763 kB]
#7 7.228 Get:114 http://deb.debian.org/debian bookworm/main amd64 libsvtav1enc1 amd64 1.4.1+dfsg-1 [2121 kB]
#7 7.242 Get:115 http://deb.debian.org/debian bookworm/main amd64 libjpeg62-turbo amd64 1:2.1.5-2 [166 kB]
#7 7.243 Get:116 http://deb.debian.org/debian bookworm/main amd64 libyuv0 amd64 0.0~git20230123.b2528b0-1 [168 kB]
#7 7.244 Get:117 http://deb.debian.org/debian bookworm/main amd64 libavif15 amd64 0.11.1-1 [93.8 kB]
#7 7.245 Get:118 http://deb.debian.org/debian bookworm/main amd64 libpng16-16 amd64 1.6.39-2 [276 kB]
#7 7.247 Get:119 http://deb.debian.org/debian bookworm/main amd64 libfreetype6 amd64 2.12.1+dfsg-5+deb12u3 [398 kB]
#7 7.250 Get:120 http://deb.debian.org/debian bookworm/main amd64 libfontconfig1 amd64 2.14.1-4 [386 kB]
#7 7.253 Get:121 http://deb.debian.org/debian bookworm/main amd64 libde265-0 amd64 1.0.11-1+deb12u2 [185 kB]
#7 7.254 Get:122 http://deb.debian.org/debian bookworm/main amd64 libnuma1 amd64 2.0.16-1 [21.0 kB]
#7 7.255 Get:123 http://deb.debian.org/debian bookworm/main amd64 libx265-199 amd64 3.5-2+b1 [1150 kB]
#7 7.262 Get:124 http://deb.debian.org/debian bookworm/main amd64 libheif1 amd64 1.15.1-1 [215 kB]
#7 7.264 Get:125 http://deb.debian.org/debian bookworm/main amd64 libdeflate0 amd64 1.14-1 [61.4 kB]
#7 7.270 Get:126 http://deb.debian.org/debian bookworm/main amd64 libjbig0 amd64 2.1-6.1 [31.7 kB]
#7 7.271 Get:127 http://deb.debian.org/debian bookworm/main amd64 liblerc4 amd64 4.0.0+ds-2 [170 kB]
#7 7.273 Get:128 http://deb.debian.org/debian bookworm/main amd64 libwebp7 amd64 1.2.4-0.2+deb12u1 [286 kB]
#7 7.275 Get:129 http://deb.debian.org/debian bookworm/main amd64 libtiff6 amd64 4.5.0-6+deb12u1 [316 kB]
#7 7.278 Get:130 http://deb.debian.org/debian bookworm/main amd64 libxau6 amd64 1:1.0.9-1 [19.7 kB]
#7 7.278 Get:131 http://deb.debian.org/debian bookworm/main amd64 libxdmcp6 amd64 1:1.1.2-3 [26.3 kB]
#7 7.278 Get:132 http://deb.debian.org/debian bookworm/main amd64 libxcb1 amd64 1.15-1 [144 kB]
#7 7.280 Get:133 http://deb.debian.org/debian bookworm/main amd64 libx11-data all 2:1.8.4-2+deb12u2 [292 kB]
#7 7.282 Get:134 http://deb.debian.org/debian bookworm/main amd64 libx11-6 amd64 2:1.8.4-2+deb12u2 [760 kB]
#7 7.287 Get:135 http://deb.debian.org/debian bookworm/main amd64 libxpm4 amd64 1:3.5.12-1.1+deb12u1 [48.6 kB]
#7 7.291 Get:136 http://deb.debian.org/debian bookworm/main amd64 libgd3 amd64 2.3.3-9 [124 kB]
#7 7.292 Get:137 http://deb.debian.org/debian bookworm/main amd64 libc-devtools amd64 2.36-9+deb12u8 [53.9 kB]
#7 7.293 Get:138 http://deb.debian.org/debian bookworm/main amd64 libclang-rt-14-dev amd64 1:14.0.6-12 [3275 kB]
#7 7.314 Get:139 http://deb.debian.org/debian bookworm/main amd64 libnspr4 amd64 2:4.35-1 [113 kB]
#7 7.315 Get:140 http://deb.debian.org/debian bookworm/main amd64 libnss3 amd64 2:3.87.1-1 [1331 kB]
#7 7.324 Get:141 http://deb.debian.org/debian bookworm/main amd64 nss-plugin-pem amd64 1.0.8+1-1 [54.6 kB]
#7 7.325 Get:142 http://deb.debian.org/debian bookworm/main amd64 libcurl3-nss amd64 7.88.1-10+deb12u7 [394 kB]
#7 7.328 Get:143 http://deb.debian.org/debian bookworm/main amd64 libffi-dev amd64 3.4.4-1 [59.4 kB]
#7 7.328 Get:144 http://deb.debian.org/debian bookworm/main amd64 libfile-fcntllock-perl amd64 0.22-4+b1 [34.8 kB]
#7 7.329 Get:145 http://deb.debian.org/debian bookworm/main amd64 libgpm2 amd64 1.20.7-10+b1 [14.2 kB]
#7 7.329 Get:146 http://deb.debian.org/debian bookworm/main amd64 libicu-dev amd64 72.1-3 [10.3 MB]
#7 7.397 Get:147 http://deb.debian.org/debian bookworm/main amd64 libldap-common all 2.5.13+dfsg-5 [29.3 kB]
#7 7.397 Get:148 http://deb.debian.org/debian bookworm/main amd64 libncurses6 amd64 6.4-4 [103 kB]
#7 7.398 Get:149 http://deb.debian.org/debian bookworm/main amd64 libncurses-dev amd64 6.4-4 [349 kB]
#7 7.401 Get:150 http://deb.debian.org/debian bookworm/main amd64 libpfm4 amd64 4.13.0-1 [294 kB]
#7 7.403 Get:151 http://deb.debian.org/debian bookworm/main amd64 libsasl2-modules amd64 2.1.28+dfsg-10 [66.6 kB]
#7 7.403 Get:152 http://deb.debian.org/debian bookworm/main amd64 libtinfo-dev amd64 6.4-4 [924 B]
#7 7.403 Get:153 http://deb.debian.org/debian bookworm/main amd64 libxml2-dev amd64 2.9.14+dfsg-1.3~deb12u1 [782 kB]
#7 7.409 Get:154 http://deb.debian.org/debian bookworm/main amd64 libyaml-0-2 amd64 0.2.5-1 [53.6 kB]
#7 7.410 Get:155 http://deb.debian.org/debian bookworm/main amd64 libz3-dev amd64 4.8.12-3.1 [90.6 kB]
#7 7.411 Get:156 http://deb.debian.org/debian bookworm/main amd64 llvm-14-runtime amd64 1:14.0.6-12 [477 kB]
#7 7.422 Get:157 http://deb.debian.org/debian bookworm/main amd64 llvm-14 amd64 1:14.0.6-12 [11.7 MB]
#7 7.501 Get:158 http://deb.debian.org/debian bookworm/main amd64 python3-pkg-resources all 66.1.1-1 [296 kB]
#7 7.503 Get:159 http://deb.debian.org/debian bookworm/main amd64 python3-pygments all 2.14.0+dfsg-1 [783 kB]
#7 7.509 Get:160 http://deb.debian.org/debian bookworm/main amd64 python3-yaml amd64 6.0-3+b2 [119 kB]
#7 7.510 Get:161 http://deb.debian.org/debian bookworm/main amd64 llvm-14-tools amd64 1:14.0.6-12 [405 kB]
#7 7.513 Get:162 http://deb.debian.org/debian bookworm/main amd64 llvm-14-dev amd64 1:14.0.6-12 [33.9 MB]
#7 7.742 Get:163 http://deb.debian.org/debian bookworm/main amd64 manpages-dev all 6.03-2 [2030 kB]
#7 7.755 Get:164 http://deb.debian.org/debian bookworm/main amd64 publicsuffix all 20230209.2326-1 [126 kB]
#7 7.942 debconf: delaying package configuration, since apt-utils is not installed
#7 7.980 Fetched 233 MB in 2s (137 MB/s)
#7 8.005 Selecting previously unselected package perl-modules-5.36.
#7 8.005 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 6695 files and directories currently installed.)
#7 8.013 Preparing to unpack .../0-perl-modules-5.36_5.36.0-7+deb12u1_all.deb ...
#7 8.017 Unpacking perl-modules-5.36 (5.36.0-7+deb12u1) ...
#7 8.540 Selecting previously unselected package libgdbm-compat4:amd64.
#7 8.543 Preparing to unpack .../1-libgdbm-compat4_1.23-3_amd64.deb ...
#7 8.588 Unpacking libgdbm-compat4:amd64 (1.23-3) ...
#7 8.709 Selecting previously unselected package libperl5.36:amd64.
#7 8.712 Preparing to unpack .../2-libperl5.36_5.36.0-7+deb12u1_amd64.deb ...
#7 8.811 Unpacking libperl5.36:amd64 (5.36.0-7+deb12u1) ...
#7 ...

#8 [stage-1 2/5] RUN apt-get update -y && apt-get install -y python3 && apt-get -y autoremove && rm -rf /var/lib/apt/lists/*
#8 0.273 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#8 0.308 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#8 0.321 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#8 0.397 Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8787 kB]
#8 0.502 Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [2468 B]
#8 0.536 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [188 kB]
#8 1.815 Fetched 9232 kB in 2s (5862 kB/s)
#8 1.815 Reading package lists...
#8 2.541 Reading package lists...
#8 3.259 Building dependency tree...
#8 3.430 Reading state information...
#8 3.654 The following additional packages will be installed:
#8 3.654 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib media-types
#8 3.654 python3-minimal python3.11 python3.11-minimal
#8 3.656 Suggested packages:
#8 3.656 python3-doc python3-tk python3-venv python3.11-venv python3.11-doc binutils
#8 3.656 binfmt-support
#8 3.728 The following NEW packages will be installed:
#8 3.728 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib media-types
#8 3.729 python3 python3-minimal python3.11 python3.11-minimal
#8 3.789 0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
#8 3.789 Need to get 5340 kB of archives.
#8 3.789 After this operation, 21.8 MB of additional disk space will be used.
#8 3.789 Get:1 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-minimal amd64 3.11.2-6+deb12u3 [816 kB]
#8 3.847 Get:2 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11-minimal amd64 3.11.2-6+deb12u3 [2067 kB]
#8 3.868 Get:3 http://deb.debian.org/debian bookworm/main amd64 python3-minimal amd64 3.11.2-1+b1 [26.3 kB]
#8 3.869 Get:4 http://deb.debian.org/debian bookworm/main amd64 media-types all 10.0.0 [26.1 kB]
#8 3.869 Get:5 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-stdlib amd64 3.11.2-6+deb12u3 [1797 kB]
#8 3.878 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11 amd64 3.11.2-6+deb12u3 [573 kB]
#8 3.882 Get:7 http://deb.debian.org/debian bookworm/main amd64 libpython3-stdlib amd64 3.11.2-1+b1 [9312 B]
#8 3.882 Get:8 http://deb.debian.org/debian bookworm/main amd64 python3 amd64 3.11.2-1+b1 [26.3 kB]
#8 4.039 debconf: delaying package configuration, since apt-utils is not installed
#8 4.075 Fetched 5340 kB in 0s (37.3 MB/s)
#8 4.106 Selecting previously unselected package libpython3.11-minimal:amd64.
#8 4.106 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 6695 files and directories currently installed.)
#8 4.114 Preparing to unpack .../libpython3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#8 4.118 Unpacking libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#8 4.234 Selecting previously unselected package python3.11-minimal.
#8 4.237 Preparing to unpack .../python3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#8 4.245 Unpacking python3.11-minimal (3.11.2-6+deb12u3) ...
#8 4.480 Setting up libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#8 4.496 Setting up python3.11-minimal (3.11.2-6+deb12u3) ...
#8 5.360 Selecting previously unselected package python3-minimal.
#8 5.360 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 7002 files and directories currently installed.)
#8 5.368 Preparing to unpack .../python3-minimal_3.11.2-1+b1_amd64.deb ...
#8 5.373 Unpacking python3-minimal (3.11.2-1+b1) ...
#8 5.407 Selecting previously unselected package media-types.
#8 5.409 Preparing to unpack .../media-types_10.0.0_all.deb ...
#8 5.413 Unpacking media-types (10.0.0) ...
#8 5.447 Selecting previously unselected package libpython3.11-stdlib:amd64.
#8 5.449 Preparing to unpack .../libpython3.11-stdlib_3.11.2-6+deb12u3_amd64.deb ...
#8 5.453 Unpacking libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#8 5.657 Selecting previously unselected package python3.11.
#8 5.659 Preparing to unpack .../python3.11_3.11.2-6+deb12u3_amd64.deb ...
#8 5.663 Unpacking python3.11 (3.11.2-6+deb12u3) ...
#8 5.709 Selecting previously unselected package libpython3-stdlib:amd64.
#8 5.712 Preparing to unpack .../libpython3-stdlib_3.11.2-1+b1_amd64.deb ...
#8 5.715 Unpacking libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#8 5.747 Setting up python3-minimal (3.11.2-1+b1) ...
#8 5.935 Selecting previously unselected package python3.
#8 5.935 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 7412 files and directories currently installed.)
#8 5.943 Preparing to unpack .../python3_3.11.2-1+b1_amd64.deb ...
#8 5.952 Unpacking python3 (3.11.2-1+b1) ...
#8 5.989 Setting up media-types (10.0.0) ...
#8 6.003 Setting up libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#8 6.014 Setting up libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#8 6.024 Setting up python3.11 (3.11.2-6+deb12u3) ...
#8 6.933 Setting up python3 (3.11.2-1+b1) ...
#8 6.959 running python rtupdate hooks for python3.11...
#8 6.959 running python post-rtupdate hooks for python3.11...
#8 7.094 Reading package lists...
#8 7.807 Building dependency tree...
#8 7.978 Reading state information...
#8 8.216 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#8 DONE 9.4s

#7 [stage-0 2/10] RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
#7 9.400 Selecting previously unselected package perl.
#7 9.403 Preparing to unpack .../3-perl_5.36.0-7+deb12u1_amd64.deb ...
#7 9.413 Unpacking perl (5.36.0-7+deb12u1) ...
#7 9.478 Selecting previously unselected package libpipeline1:amd64.
#7 9.480 Preparing to unpack .../4-libpipeline1_1.5.7-1_amd64.deb ...
#7 9.483 Unpacking libpipeline1:amd64 (1.5.7-1) ...
#7 9.527 Selecting previously unselected package binfmt-support.
#7 ...

#9 [stage-1 3/5] RUN ln -s /usr/bin/python3 /usr/bin/python
#9 DONE 0.2s

#7 [stage-0 2/10] RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
#7 9.529 Preparing to unpack .../5-binfmt-support_2.2.2-2_amd64.deb ...
#7 9.534 Unpacking binfmt-support (2.2.2-2) ...
#7 9.569 Selecting previously unselected package liblocale-gettext-perl.
#7 9.571 Preparing to unpack .../6-liblocale-gettext-perl_1.07-5_amd64.deb ...
#7 9.574 Unpacking liblocale-gettext-perl (1.07-5) ...
#7 9.625 Selecting previously unselected package libpython3.11-minimal:amd64.
#7 9.626 Preparing to unpack .../7-libpython3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#7 9.630 Unpacking libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#7 9.748 Selecting previously unselected package python3.11-minimal.
#7 9.749 Preparing to unpack .../8-python3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#7 9.758 Unpacking python3.11-minimal (3.11.2-6+deb12u3) ...
#7 9.992 Setting up libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#7 10.01 Setting up python3.11-minimal (3.11.2-6+deb12u3) ...
#7 10.86 Selecting previously unselected package python3-minimal.
#7 10.86 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 9037 files and directories currently installed.)
#7 10.87 Preparing to unpack .../python3-minimal_3.11.2-1+b1_amd64.deb ...
#7 10.87 Unpacking python3-minimal (3.11.2-1+b1) ...
#7 10.91 Selecting previously unselected package media-types.
#7 10.91 Preparing to unpack .../media-types_10.0.0_all.deb ...
#7 10.92 Unpacking media-types (10.0.0) ...
#7 10.95 Selecting previously unselected package libpython3.11-stdlib:amd64.
#7 10.95 Preparing to unpack .../libpython3.11-stdlib_3.11.2-6+deb12u3_amd64.deb ...
#7 10.96 Unpacking libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#7 11.16 Selecting previously unselected package python3.11.
#7 11.16 Preparing to unpack .../python3.11_3.11.2-6+deb12u3_amd64.deb ...
#7 11.16 Unpacking python3.11 (3.11.2-6+deb12u3) ...
#7 11.21 Selecting previously unselected package libpython3-stdlib:amd64.
#7 11.21 Preparing to unpack .../libpython3-stdlib_3.11.2-1+b1_amd64.deb ...
#7 11.21 Unpacking libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#7 11.25 Setting up python3-minimal (3.11.2-1+b1) ...
#7 11.45 Selecting previously unselected package python3.
#7 11.45 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 9447 files and directories currently installed.)
#7 11.46 Preparing to unpack .../000-python3_3.11.2-1+b1_amd64.deb ...
#7 11.47 Unpacking python3 (3.11.2-1+b1) ...
#7 11.50 Selecting previously unselected package bzip2.
#7 11.50 Preparing to unpack .../001-bzip2_1.0.8-5+b1_amd64.deb ...
#7 11.51 Unpacking bzip2 (1.0.8-5+b1) ...
#7 11.55 Selecting previously unselected package manpages.
#7 11.55 Preparing to unpack .../002-manpages_6.03-2_all.deb ...
#7 11.55 Unpacking manpages (6.03-2) ...
#7 11.68 Selecting previously unselected package xz-utils.
#7 11.68 Preparing to unpack .../003-xz-utils_5.4.1-0.2_amd64.deb ...
#7 11.69 Unpacking xz-utils (5.4.1-0.2) ...
#7 11.77 Selecting previously unselected package binutils-common:amd64.
#7 11.77 Preparing to unpack .../004-binutils-common_2.40-2_amd64.deb ...
#7 11.77 Unpacking binutils-common:amd64 (2.40-2) ...
#7 12.05 Selecting previously unselected package libbinutils:amd64.
#7 12.05 Preparing to unpack .../005-libbinutils_2.40-2_amd64.deb ...
#7 12.06 Unpacking libbinutils:amd64 (2.40-2) ...
#7 12.15 Selecting previously unselected package libctf-nobfd0:amd64.
#7 12.15 Preparing to unpack .../006-libctf-nobfd0_2.40-2_amd64.deb ...
#7 12.16 Unpacking libctf-nobfd0:amd64 (2.40-2) ...
#7 12.21 Selecting previously unselected package libctf0:amd64.
#7 12.21 Preparing to unpack .../007-libctf0_2.40-2_amd64.deb ...
#7 12.21 Unpacking libctf0:amd64 (2.40-2) ...
#7 12.26 Selecting previously unselected package libgprofng0:amd64.
#7 12.26 Preparing to unpack .../008-libgprofng0_2.40-2_amd64.deb ...
#7 12.26 Unpacking libgprofng0:amd64 (2.40-2) ...
#7 12.38 Selecting previously unselected package libjansson4:amd64.
#7 12.38 Preparing to unpack .../009-libjansson4_2.14-2_amd64.deb ...
#7 12.38 Unpacking libjansson4:amd64 (2.14-2) ...
#7 12.42 Selecting previously unselected package binutils-x86-64-linux-gnu.
#7 12.42 Preparing to unpack .../010-binutils-x86-64-linux-gnu_2.40-2_amd64.deb ...
#7 12.42 Unpacking binutils-x86-64-linux-gnu (2.40-2) ...
#7 12.70 Selecting previously unselected package binutils.
#7 12.70 Preparing to unpack .../011-binutils_2.40-2_amd64.deb ...
#7 12.70 Unpacking binutils (2.40-2) ...
#7 12.74 Selecting previously unselected package libc-dev-bin.
#7 12.75 Preparing to unpack .../012-libc-dev-bin_2.36-9+deb12u8_amd64.deb ...
#7 12.75 Unpacking libc-dev-bin (2.36-9+deb12u8) ...
#7 12.79 Selecting previously unselected package linux-libc-dev:amd64.
#7 12.79 Preparing to unpack .../013-linux-libc-dev_6.1.112-1_amd64.deb ...
#7 12.79 Unpacking linux-libc-dev:amd64 (6.1.112-1) ...
#7 13.01 Selecting previously unselected package libcrypt-dev:amd64.
#7 13.01 Preparing to unpack .../014-libcrypt-dev_1%3a4.4.33-2_amd64.deb ...
#7 13.02 Unpacking libcrypt-dev:amd64 (1:4.4.33-2) ...
#7 13.06 Selecting previously unselected package libtirpc-dev:amd64.
#7 13.06 Preparing to unpack .../015-libtirpc-dev_1.3.3+ds-1_amd64.deb ...
#7 13.07 Unpacking libtirpc-dev:amd64 (1.3.3+ds-1) ...
#7 13.12 Selecting previously unselected package libnsl-dev:amd64.
#7 13.12 Preparing to unpack .../016-libnsl-dev_1.3.0-2_amd64.deb ...
#7 13.12 Unpacking libnsl-dev:amd64 (1.3.0-2) ...
#7 13.16 Selecting previously unselected package rpcsvc-proto.
#7 13.16 Preparing to unpack .../017-rpcsvc-proto_1.4.3-1_amd64.deb ...
#7 13.16 Unpacking rpcsvc-proto (1.4.3-1) ...
#7 13.20 Selecting previously unselected package libc6-dev:amd64.
#7 13.20 Preparing to unpack .../018-libc6-dev_2.36-9+deb12u8_amd64.deb ...
#7 13.21 Unpacking libc6-dev:amd64 (2.36-9+deb12u8) ...
#7 13.45 Selecting previously unselected package libisl23:amd64.
#7 13.45 Preparing to unpack .../019-libisl23_0.25-1.1_amd64.deb ...
#7 13.45 Unpacking libisl23:amd64 (0.25-1.1) ...
#7 13.55 Selecting previously unselected package libmpfr6:amd64.
#7 13.55 Preparing to unpack .../020-libmpfr6_4.2.0-1_amd64.deb ...
#7 13.55 Unpacking libmpfr6:amd64 (4.2.0-1) ...
#7 13.63 Selecting previously unselected package libmpc3:amd64.
#7 13.63 Preparing to unpack .../021-libmpc3_1.3.1-1_amd64.deb ...
#7 13.63 Unpacking libmpc3:amd64 (1.3.1-1) ...
#7 13.67 Selecting previously unselected package cpp-12.
#7 13.67 Preparing to unpack .../022-cpp-12_12.2.0-14_amd64.deb ...
#7 13.67 Unpacking cpp-12 (12.2.0-14) ...
#7 14.48 Selecting previously unselected package cpp.
#7 14.48 Preparing to unpack .../023-cpp_4%3a12.2.0-3_amd64.deb ...
#7 14.49 Unpacking cpp (4:12.2.0-3) ...
#7 14.52 Selecting previously unselected package libcc1-0:amd64.
#7 14.52 Preparing to unpack .../024-libcc1-0_12.2.0-14_amd64.deb ...
#7 14.53 Unpacking libcc1-0:amd64 (12.2.0-14) ...
#7 14.57 Selecting previously unselected package libgomp1:amd64.
#7 14.57 Preparing to unpack .../025-libgomp1_12.2.0-14_amd64.deb ...
#7 14.57 Unpacking libgomp1:amd64 (12.2.0-14) ...
#7 14.62 Selecting previously unselected package libitm1:amd64.
#7 14.62 Preparing to unpack .../026-libitm1_12.2.0-14_amd64.deb ...
#7 14.63 Unpacking libitm1:amd64 (12.2.0-14) ...
#7 14.67 Selecting previously unselected package libatomic1:amd64.
#7 14.67 Preparing to unpack .../027-libatomic1_12.2.0-14_amd64.deb ...
#7 14.67 Unpacking libatomic1:amd64 (12.2.0-14) ...
#7 14.70 Selecting previously unselected package libasan8:amd64.
#7 14.71 Preparing to unpack .../028-libasan8_12.2.0-14_amd64.deb ...
#7 14.71 Unpacking libasan8:amd64 (12.2.0-14) ...
#7 14.96 Selecting previously unselected package liblsan0:amd64.
#7 14.97 Preparing to unpack .../029-liblsan0_12.2.0-14_amd64.deb ...
#7 14.97 Unpacking liblsan0:amd64 (12.2.0-14) ...
#7 15.09 Selecting previously unselected package libtsan2:amd64.
#7 15.09 Preparing to unpack .../030-libtsan2_12.2.0-14_amd64.deb ...
#7 15.10 Unpacking libtsan2:amd64 (12.2.0-14) ...
#7 15.35 Selecting previously unselected package libubsan1:amd64.
#7 15.36 Preparing to unpack .../031-libubsan1_12.2.0-14_amd64.deb ...
#7 15.36 Unpacking libubsan1:amd64 (12.2.0-14) ...
#7 15.48 Selecting previously unselected package libquadmath0:amd64.
#7 15.48 Preparing to unpack .../032-libquadmath0_12.2.0-14_amd64.deb ...
#7 15.49 Unpacking libquadmath0:amd64 (12.2.0-14) ...
#7 15.53 Selecting previously unselected package libgcc-12-dev:amd64.
#7 15.53 Preparing to unpack .../033-libgcc-12-dev_12.2.0-14_amd64.deb ...
#7 15.54 Unpacking libgcc-12-dev:amd64 (12.2.0-14) ...
#7 15.78 Selecting previously unselected package gcc-12.
#7 15.78 Preparing to unpack .../034-gcc-12_12.2.0-14_amd64.deb ...
#7 15.79 Unpacking gcc-12 (12.2.0-14) ...
#7 16.69 Selecting previously unselected package gcc.
#7 16.69 Preparing to unpack .../035-gcc_4%3a12.2.0-3_amd64.deb ...
#7 16.69 Unpacking gcc (4:12.2.0-3) ...
#7 16.73 Selecting previously unselected package libstdc++-12-dev:amd64.
#7 16.73 Preparing to unpack .../036-libstdc++-12-dev_12.2.0-14_amd64.deb ...
#7 16.73 Unpacking libstdc++-12-dev:amd64 (12.2.0-14) ...
#7 16.99 Selecting previously unselected package g++-12.
#7 16.99 Preparing to unpack .../037-g++-12_12.2.0-14_amd64.deb ...
#7 16.99 Unpacking g++-12 (12.2.0-14) ...
#7 17.83 Selecting previously unselected package g++.
#7 17.83 Preparing to unpack .../038-g++_4%3a12.2.0-3_amd64.deb ...
#7 17.84 Unpacking g++ (4:12.2.0-3) ...
#7 17.86 Selecting previously unselected package make.
#7 17.87 Preparing to unpack .../039-make_4.3-4.1_amd64.deb ...
#7 17.87 Unpacking make (4.3-4.1) ...
#7 17.94 Selecting previously unselected package libdpkg-perl.
#7 17.94 Preparing to unpack .../040-libdpkg-perl_1.21.22_all.deb ...
#7 17.95 Unpacking libdpkg-perl (1.21.22) ...
#7 18.03 Selecting previously unselected package patch.
#7 18.03 Preparing to unpack .../041-patch_2.7.6-7_amd64.deb ...
#7 18.03 Unpacking patch (2.7.6-7) ...
#7 18.07 Selecting previously unselected package dpkg-dev.
#7 18.08 Preparing to unpack .../042-dpkg-dev_1.21.22_all.deb ...
#7 18.08 Unpacking dpkg-dev (1.21.22) ...
#7 18.22 Selecting previously unselected package build-essential.
#7 18.22 Preparing to unpack .../043-build-essential_12.9_amd64.deb ...
#7 18.23 Unpacking build-essential (12.9) ...
#7 18.26 Selecting previously unselected package libbsd0:amd64.
#7 18.27 Preparing to unpack .../044-libbsd0_0.11.7-2_amd64.deb ...
#7 18.27 Unpacking libbsd0:amd64 (0.11.7-2) ...
#7 18.32 Selecting previously unselected package libedit2:amd64.
#7 18.32 Preparing to unpack .../045-libedit2_3.1-20221030-2_amd64.deb ...
#7 18.32 Unpacking libedit2:amd64 (3.1-20221030-2) ...
#7 18.37 Selecting previously unselected package libicu72:amd64.
#7 18.37 Preparing to unpack .../046-libicu72_72.1-3_amd64.deb ...
#7 18.37 Unpacking libicu72:amd64 (72.1-3) ...
#7 19.07 Selecting previously unselected package libxml2:amd64.
#7 19.08 Preparing to unpack .../047-libxml2_2.9.14+dfsg-1.3~deb12u1_amd64.deb ...
#7 19.08 Unpacking libxml2:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
#7 19.18 Selecting previously unselected package libz3-4:amd64.
#7 19.18 Preparing to unpack .../048-libz3-4_4.8.12-3.1_amd64.deb ...
#7 19.19 Unpacking libz3-4:amd64 (4.8.12-3.1) ...
#7 19.89 Selecting previously unselected package libllvm14:amd64.
#7 19.89 Preparing to unpack .../049-libllvm14_1%3a14.0.6-12_amd64.deb ...
#7 19.89 Unpacking libllvm14:amd64 (1:14.0.6-12) ...
#7 20.84 Selecting previously unselected package libclang-cpp14.
#7 20.85 Preparing to unpack .../050-libclang-cpp14_1%3a14.0.6-12_amd64.deb ...
#7 20.85 Unpacking libclang-cpp14 (1:14.0.6-12) ...
#7 21.47 Selecting previously unselected package libgc1:amd64.
#7 21.47 Preparing to unpack .../051-libgc1_1%3a8.2.2-3_amd64.deb ...
#7 21.47 Unpacking libgc1:amd64 (1:8.2.2-3) ...
#7 21.53 Selecting previously unselected package libobjc4:amd64.
#7 21.53 Preparing to unpack .../052-libobjc4_12.2.0-14_amd64.deb ...
#7 21.53 Unpacking libobjc4:amd64 (12.2.0-14) ...
#7 21.57 Selecting previously unselected package libobjc-12-dev:amd64.
#7 21.57 Preparing to unpack .../053-libobjc-12-dev_12.2.0-14_amd64.deb ...
#7 21.57 Unpacking libobjc-12-dev:amd64 (12.2.0-14) ...
#7 21.62 Selecting previously unselected package libclang-common-14-dev.
#7 21.63 Preparing to unpack .../054-libclang-common-14-dev_1%3a14.0.6-12_all.deb ...
#7 21.63 Unpacking libclang-common-14-dev (1:14.0.6-12) ...
#7 21.77 Selecting previously unselected package llvm-14-linker-tools.
#7 21.77 Preparing to unpack .../055-llvm-14-linker-tools_1%3a14.0.6-12_amd64.deb ...
#7 21.77 Unpacking llvm-14-linker-tools (1:14.0.6-12) ...
#7 21.93 Selecting previously unselected package libclang1-14.
#7 21.93 Preparing to unpack .../056-libclang1-14_1%3a14.0.6-12_amd64.deb ...
#7 21.94 Unpacking libclang1-14 (1:14.0.6-12) ...
#7 22.51 Selecting previously unselected package clang-14.
#7 22.52 Preparing to unpack .../057-clang-14_1%3a14.0.6-12_amd64.deb ...
#7 22.52 Unpacking clang-14 (1:14.0.6-12) ...
#7 22.56 Selecting previously unselected package clang.
#7 22.57 Preparing to unpack .../058-clang_1%3a14.0-55.7~deb12u1_amd64.deb ...
#7 22.57 Unpacking clang (1:14.0-55.7~deb12u1) ...
#7 22.61 Selecting previously unselected package libbrotli1:amd64.
#7 22.61 Preparing to unpack .../059-libbrotli1_1.0.9-2+b6_amd64.deb ...
#7 22.61 Unpacking libbrotli1:amd64 (1.0.9-2+b6) ...
#7 22.67 Selecting previously unselected package libsasl2-modules-db:amd64.
#7 22.67 Preparing to unpack .../060-libsasl2-modules-db_2.1.28+dfsg-10_amd64.deb ...
#7 22.68 Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ...
#7 22.71 Selecting previously unselected package libsasl2-2:amd64.
#7 22.72 Preparing to unpack .../061-libsasl2-2_2.1.28+dfsg-10_amd64.deb ...
#7 22.72 Unpacking libsasl2-2:amd64 (2.1.28+dfsg-10) ...
#7 22.76 Selecting previously unselected package libldap-2.5-0:amd64.
#7 22.76 Preparing to unpack .../062-libldap-2.5-0_2.5.13+dfsg-5_amd64.deb ...
#7 22.77 Unpacking libldap-2.5-0:amd64 (2.5.13+dfsg-5) ...
#7 22.82 Selecting previously unselected package libnghttp2-14:amd64.
#7 22.82 Preparing to unpack .../063-libnghttp2-14_1.52.0-1+deb12u1_amd64.deb ...
#7 22.83 Unpacking libnghttp2-14:amd64 (1.52.0-1+deb12u1) ...
#7 22.87 Selecting previously unselected package libpsl5:amd64.
#7 22.87 Preparing to unpack .../064-libpsl5_0.21.2-1_amd64.deb ...
#7 22.87 Unpacking libpsl5:amd64 (0.21.2-1) ...
#7 22.92 Selecting previously unselected package librtmp1:amd64.
#7 22.92 Preparing to unpack .../065-librtmp1_2.4+20151223.gitfa8646d.1-2+b2_amd64.deb ...
#7 22.92 Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ...
#7 22.96 Selecting previously unselected package libssh2-1:amd64.
#7 22.96 Preparing to unpack .../066-libssh2-1_1.10.0-3+b1_amd64.deb ...
#7 22.97 Unpacking libssh2-1:amd64 (1.10.0-3+b1) ...
#7 23.02 Selecting previously unselected package libcurl4:amd64.
#7 23.02 Preparing to unpack .../067-libcurl4_7.88.1-10+deb12u7_amd64.deb ...
#7 23.03 Unpacking libcurl4:amd64 (7.88.1-10+deb12u7) ...
#7 23.09 Selecting previously unselected package curl.
#7 23.09 Preparing to unpack .../068-curl_7.88.1-10+deb12u7_amd64.deb ...
#7 23.09 Unpacking curl (7.88.1-10+deb12u7) ...
#7 23.15 Selecting previously unselected package libassuan0:amd64.
#7 23.16 Preparing to unpack .../069-libassuan0_2.5.5-5_amd64.deb ...
#7 23.16 Unpacking libassuan0:amd64 (2.5.5-5) ...
#7 23.19 Selecting previously unselected package gpgconf.
#7 23.20 Preparing to unpack .../070-gpgconf_2.2.40-1.1_amd64.deb ...
#7 23.20 Unpacking gpgconf (2.2.40-1.1) ...
#7 23.26 Selecting previously unselected package libksba8:amd64.
#7 23.26 Preparing to unpack .../071-libksba8_1.6.3-2_amd64.deb ...
#7 23.26 Unpacking libksba8:amd64 (1.6.3-2) ...
#7 23.31 Selecting previously unselected package libnpth0:amd64.
#7 23.31 Preparing to unpack .../072-libnpth0_1.6-3_amd64.deb ...
#7 23.32 Unpacking libnpth0:amd64 (1.6-3) ...
#7 23.36 Selecting previously unselected package dirmngr.
#7 23.36 Preparing to unpack .../073-dirmngr_2.2.40-1.1_amd64.deb ...
#7 23.38 Unpacking dirmngr (2.2.40-1.1) ...
#7 23.46 Selecting previously unselected package libfakeroot:amd64.
#7 23.46 Preparing to unpack .../074-libfakeroot_1.31-1.2_amd64.deb ...
#7 23.47 Unpacking libfakeroot:amd64 (1.31-1.2) ...
#7 23.51 Selecting previously unselected package fakeroot.
#7 23.51 Preparing to unpack .../075-fakeroot_1.31-1.2_amd64.deb ...
#7 23.51 Unpacking fakeroot (1.31-1.2) ...
#7 23.55 Selecting previously unselected package fonts-dejavu-core.
#7 23.56 Preparing to unpack .../076-fonts-dejavu-core_2.37-6_all.deb ...
#7 23.56 Unpacking fonts-dejavu-core (2.37-6) ...
#7 23.70 Selecting previously unselected package fontconfig-config.
#7 23.71 Preparing to unpack .../077-fontconfig-config_2.14.1-4_amd64.deb ...
#7 23.87 Unpacking fontconfig-config (2.14.1-4) ...
#7 23.91 Selecting previously unselected package gnupg-l10n.
#7 23.92 Preparing to unpack .../078-gnupg-l10n_2.2.40-1.1_all.deb ...
#7 23.92 Unpacking gnupg-l10n (2.2.40-1.1) ...
#7 24.03 Selecting previously unselected package gnupg-utils.
#7 24.03 Preparing to unpack .../079-gnupg-utils_2.2.40-1.1_amd64.deb ...
#7 24.03 Unpacking gnupg-utils (2.2.40-1.1) ...
#7 24.12 Selecting previously unselected package gpg.
#7 24.12 Preparing to unpack .../080-gpg_2.2.40-1.1_amd64.deb ...
#7 24.12 Unpacking gpg (2.2.40-1.1) ...
#7 24.21 Selecting previously unselected package pinentry-curses.
#7 24.21 Preparing to unpack .../081-pinentry-curses_1.2.1-1_amd64.deb ...
#7 24.21 Unpacking pinentry-curses (1.2.1-1) ...
#7 24.25 Selecting previously unselected package gpg-agent.
#7 24.26 Preparing to unpack .../082-gpg-agent_2.2.40-1.1_amd64.deb ...
#7 24.26 Unpacking gpg-agent (2.2.40-1.1) ...
#7 24.32 Selecting previously unselected package gpg-wks-client.
#7 24.32 Preparing to unpack .../083-gpg-wks-client_2.2.40-1.1_amd64.deb ...
#7 24.32 Unpacking gpg-wks-client (2.2.40-1.1) ...
#7 24.38 Selecting previously unselected package gpg-wks-server.
#7 24.39 Preparing to unpack .../084-gpg-wks-server_2.2.40-1.1_amd64.deb ...
#7 24.39 Unpacking gpg-wks-server (2.2.40-1.1) ...
#7 24.43 Selecting previously unselected package gpgsm.
#7 24.44 Preparing to unpack .../085-gpgsm_2.2.40-1.1_amd64.deb ...
#7 24.44 Unpacking gpgsm (2.2.40-1.1) ...
#7 24.50 Selecting previously unselected package gnupg.
#7 24.50 Preparing to unpack .../086-gnupg_2.2.40-1.1_all.deb ...
#7 24.51 Unpacking gnupg (2.2.40-1.1) ...
#7 24.58 Selecting previously unselected package icu-devtools.
#7 24.58 Preparing to unpack .../087-icu-devtools_72.1-3_amd64.deb ...
#7 24.58 Unpacking icu-devtools (72.1-3) ...
#7 24.65 Selecting previously unselected package libc6-i386.
#7 24.65 Preparing to unpack .../088-libc6-i386_2.36-9+deb12u8_amd64.deb ...
#7 24.66 Unpacking libc6-i386 (2.36-9+deb12u8) ...
#7 24.95 Selecting previously unselected package lib32gcc-s1.
#7 24.95 Preparing to unpack .../089-lib32gcc-s1_12.2.0-14_amd64.deb ...
#7 24.95 Unpacking lib32gcc-s1 (12.2.0-14) ...
#7 25.00 Selecting previously unselected package lib32stdc++6.
#7 25.00 Preparing to unpack .../090-lib32stdc++6_12.2.0-14_amd64.deb ...
#7 25.01 Unpacking lib32stdc++6 (12.2.0-14) ...
#7 25.11 Selecting previously unselected package libabsl20220623:amd64.
#7 25.11 Preparing to unpack .../091-libabsl20220623_20220623.1-1_amd64.deb ...
#7 25.11 Unpacking libabsl20220623:amd64 (20220623.1-1) ...
#7 25.19 Selecting previously unselected package libalgorithm-diff-perl.
#7 25.19 Preparing to unpack .../092-libalgorithm-diff-perl_1.201-1_all.deb ...
#7 25.20 Unpacking libalgorithm-diff-perl (1.201-1) ...
#7 25.23 Selecting previously unselected package libalgorithm-diff-xs-perl:amd64.
#7 25.24 Preparing to unpack .../093-libalgorithm-diff-xs-perl_0.04-8+b1_amd64.deb ...
#7 25.24 Unpacking libalgorithm-diff-xs-perl:amd64 (0.04-8+b1) ...
#7 25.28 Selecting previously unselected package libalgorithm-merge-perl.
#7 25.28 Preparing to unpack .../094-libalgorithm-merge-perl_0.08-5_all.deb ...
#7 25.28 Unpacking libalgorithm-merge-perl (0.08-5) ...
#7 25.32 Selecting previously unselected package libaom3:amd64.
#7 25.32 Preparing to unpack .../095-libaom3_3.6.0-1+deb12u1_amd64.deb ...
#7 25.33 Unpacking libaom3:amd64 (3.6.0-1+deb12u1) ...
#7 25.54 Selecting previously unselected package libdav1d6:amd64.
#7 25.54 Preparing to unpack .../096-libdav1d6_1.0.0-2+deb12u1_amd64.deb ...
#7 25.54 Unpacking libdav1d6:amd64 (1.0.0-2+deb12u1) ...
#7 25.63 Selecting previously unselected package libgav1-1:amd64.
#7 25.63 Preparing to unpack .../097-libgav1-1_0.18.0-1+b1_amd64.deb ...
#7 25.64 Unpacking libgav1-1:amd64 (0.18.0-1+b1) ...
#7 25.71 Selecting previously unselected package librav1e0:amd64.
#7 25.71 Preparing to unpack .../098-librav1e0_0.5.1-6_amd64.deb ...
#7 25.72 Unpacking librav1e0:amd64 (0.5.1-6) ...
#7 25.83 Selecting previously unselected package libsvtav1enc1:amd64.
#7 25.83 Preparing to unpack .../099-libsvtav1enc1_1.4.1+dfsg-1_amd64.deb ...
#7 25.83 Unpacking libsvtav1enc1:amd64 (1.4.1+dfsg-1) ...
#7 26.07 Selecting previously unselected package libjpeg62-turbo:amd64.
#7 26.07 Preparing to unpack .../100-libjpeg62-turbo_1%3a2.1.5-2_amd64.deb ...
#7 26.08 Unpacking libjpeg62-turbo:amd64 (1:2.1.5-2) ...
#7 26.13 Selecting previously unselected package libyuv0:amd64.
#7 26.14 Preparing to unpack .../101-libyuv0_0.0~git20230123.b2528b0-1_amd64.deb ...
#7 26.14 Unpacking libyuv0:amd64 (0.0~git20230123.b2528b0-1) ...
#7 26.19 Selecting previously unselected package libavif15:amd64.
#7 26.19 Preparing to unpack .../102-libavif15_0.11.1-1_amd64.deb ...
#7 26.20 Unpacking libavif15:amd64 (0.11.1-1) ...
#7 26.24 Selecting previously unselected package libpng16-16:amd64.
#7 26.25 Preparing to unpack .../103-libpng16-16_1.6.39-2_amd64.deb ...
#7 26.25 Unpacking libpng16-16:amd64 (1.6.39-2) ...
#7 26.31 Selecting previously unselected package libfreetype6:amd64.
#7 26.31 Preparing to unpack .../104-libfreetype6_2.12.1+dfsg-5+deb12u3_amd64.deb ...
#7 26.32 Unpacking libfreetype6:amd64 (2.12.1+dfsg-5+deb12u3) ...
#7 26.39 Selecting previously unselected package libfontconfig1:amd64.
#7 26.39 Preparing to unpack .../105-libfontconfig1_2.14.1-4_amd64.deb ...
#7 26.40 Unpacking libfontconfig1:amd64 (2.14.1-4) ...
#7 26.45 Selecting previously unselected package libde265-0:amd64.
#7 26.45 Preparing to unpack .../106-libde265-0_1.0.11-1+deb12u2_amd64.deb ...
#7 26.46 Unpacking libde265-0:amd64 (1.0.11-1+deb12u2) ...
#7 26.51 Selecting previously unselected package libnuma1:amd64.
#7 26.52 Preparing to unpack .../107-libnuma1_2.0.16-1_amd64.deb ...
#7 26.52 Unpacking libnuma1:amd64 (2.0.16-1) ...
#7 26.56 Selecting previously unselected package libx265-199:amd64.
#7 26.57 Preparing to unpack .../108-libx265-199_3.5-2+b1_amd64.deb ...
#7 26.57 Unpacking libx265-199:amd64 (3.5-2+b1) ...
#7 26.74 Selecting previously unselected package libheif1:amd64.
#7 26.74 Preparing to unpack .../109-libheif1_1.15.1-1_amd64.deb ...
#7 26.75 Unpacking libheif1:amd64 (1.15.1-1) ...
#7 26.81 Selecting previously unselected package libdeflate0:amd64.
#7 26.81 Preparing to unpack .../110-libdeflate0_1.14-1_amd64.deb ...
#7 26.81 Unpacking libdeflate0:amd64 (1.14-1) ...
#7 26.86 Selecting previously unselected package libjbig0:amd64.
#7 26.86 Preparing to unpack .../111-libjbig0_2.1-6.1_amd64.deb ...
#7 26.86 Unpacking libjbig0:amd64 (2.1-6.1) ...
#7 26.91 Selecting previously unselected package liblerc4:amd64.
#7 26.91 Preparing to unpack .../112-liblerc4_4.0.0+ds-2_amd64.deb ...
#7 26.91 Unpacking liblerc4:amd64 (4.0.0+ds-2) ...
#7 26.97 Selecting previously unselected package libwebp7:amd64.
#7 26.97 Preparing to unpack .../113-libwebp7_1.2.4-0.2+deb12u1_amd64.deb ...
#7 26.98 Unpacking libwebp7:amd64 (1.2.4-0.2+deb12u1) ...
#7 27.04 Selecting previously unselected package libtiff6:amd64.
#7 27.04 Preparing to unpack .../114-libtiff6_4.5.0-6+deb12u1_amd64.deb ...
#7 27.05 Unpacking libtiff6:amd64 (4.5.0-6+deb12u1) ...
#7 27.10 Selecting previously unselected package libxau6:amd64.
#7 27.10 Preparing to unpack .../115-libxau6_1%3a1.0.9-1_amd64.deb ...
#7 27.11 Unpacking libxau6:amd64 (1:1.0.9-1) ...
#7 27.14 Selecting previously unselected package libxdmcp6:amd64.
#7 27.15 Preparing to unpack .../116-libxdmcp6_1%3a1.1.2-3_amd64.deb ...
#7 27.15 Unpacking libxdmcp6:amd64 (1:1.1.2-3) ...
#7 27.19 Selecting previously unselected package libxcb1:amd64.
#7 27.19 Preparing to unpack .../117-libxcb1_1.15-1_amd64.deb ...
#7 27.20 Unpacking libxcb1:amd64 (1.15-1) ...
#7 27.24 Selecting previously unselected package libx11-data.
#7 27.24 Preparing to unpack .../118-libx11-data_2%3a1.8.4-2+deb12u2_all.deb ...
#7 27.24 Unpacking libx11-data (2:1.8.4-2+deb12u2) ...
#7 27.33 Selecting previously unselected package libx11-6:amd64.
#7 27.33 Preparing to unpack .../119-libx11-6_2%3a1.8.4-2+deb12u2_amd64.deb ...
#7 27.33 Unpacking libx11-6:amd64 (2:1.8.4-2+deb12u2) ...
#7 27.43 Selecting previously unselected package libxpm4:amd64.
#7 27.43 Preparing to unpack .../120-libxpm4_1%3a3.5.12-1.1+deb12u1_amd64.deb ...
#7 27.44 Unpacking libxpm4:amd64 (1:3.5.12-1.1+deb12u1) ...
#7 27.48 Selecting previously unselected package libgd3:amd64.
#7 27.48 Preparing to unpack .../121-libgd3_2.3.3-9_amd64.deb ...
#7 27.49 Unpacking libgd3:amd64 (2.3.3-9) ...
#7 27.53 Selecting previously unselected package libc-devtools.
#7 27.54 Preparing to unpack .../122-libc-devtools_2.36-9+deb12u8_amd64.deb ...
#7 27.54 Unpacking libc-devtools (2.36-9+deb12u8) ...
#7 27.58 Selecting previously unselected package libclang-rt-14-dev:amd64.
#7 27.58 Preparing to unpack .../123-libclang-rt-14-dev_1%3a14.0.6-12_amd64.deb ...
#7 27.58 Unpacking libclang-rt-14-dev:amd64 (1:14.0.6-12) ...
#7 27.91 Selecting previously unselected package libnspr4:amd64.
#7 27.91 Preparing to unpack .../124-libnspr4_2%3a4.35-1_amd64.deb ...
#7 27.91 Unpacking libnspr4:amd64 (2:4.35-1) ...
#7 27.96 Selecting previously unselected package libnss3:amd64.
#7 27.96 Preparing to unpack .../125-libnss3_2%3a3.87.1-1_amd64.deb ...
#7 27.97 Unpacking libnss3:amd64 (2:3.87.1-1) ...
#7 28.13 Selecting previously unselected package nss-plugin-pem:amd64.
#7 28.13 Preparing to unpack .../126-nss-plugin-pem_1.0.8+1-1_amd64.deb ...
#7 28.13 Unpacking nss-plugin-pem:amd64 (1.0.8+1-1) ...
#7 28.18 Selecting previously unselected package libcurl3-nss:amd64.
#7 28.18 Preparing to unpack .../127-libcurl3-nss_7.88.1-10+deb12u7_amd64.deb ...
#7 28.18 Unpacking libcurl3-nss:amd64 (7.88.1-10+deb12u7) ...
#7 28.24 Selecting previously unselected package libffi-dev:amd64.
#7 28.25 Preparing to unpack .../128-libffi-dev_3.4.4-1_amd64.deb ...
#7 28.25 Unpacking libffi-dev:amd64 (3.4.4-1) ...
#7 28.29 Selecting previously unselected package libfile-fcntllock-perl.
#7 28.29 Preparing to unpack .../129-libfile-fcntllock-perl_0.22-4+b1_amd64.deb ...
#7 28.30 Unpacking libfile-fcntllock-perl (0.22-4+b1) ...
#7 28.34 Selecting previously unselected package libgpm2:amd64.
#7 28.34 Preparing to unpack .../130-libgpm2_1.20.7-10+b1_amd64.deb ...
#7 28.35 Unpacking libgpm2:amd64 (1.20.7-10+b1) ...
#7 28.38 Selecting previously unselected package libicu-dev:amd64.
#7 28.38 Preparing to unpack .../131-libicu-dev_72.1-3_amd64.deb ...
#7 28.39 Unpacking libicu-dev:amd64 (72.1-3) ...
#7 29.07 Selecting previously unselected package libldap-common.
#7 29.08 Preparing to unpack .../132-libldap-common_2.5.13+dfsg-5_all.deb ...
#7 29.08 Unpacking libldap-common (2.5.13+dfsg-5) ...
#7 29.12 Selecting previously unselected package libncurses6:amd64.
#7 29.12 Preparing to unpack .../133-libncurses6_6.4-4_amd64.deb ...
#7 29.12 Unpacking libncurses6:amd64 (6.4-4) ...
#7 29.16 Selecting previously unselected package libncurses-dev:amd64.
#7 29.17 Preparing to unpack .../134-libncurses-dev_6.4-4_amd64.deb ...
#7 29.17 Unpacking libncurses-dev:amd64 (6.4-4) ...
#7 29.25 Selecting previously unselected package libpfm4:amd64.
#7 29.25 Preparing to unpack .../135-libpfm4_4.13.0-1_amd64.deb ...
#7 29.25 Unpacking libpfm4:amd64 (4.13.0-1) ...
#7 29.32 Selecting previously unselected package libsasl2-modules:amd64.
#7 29.33 Preparing to unpack .../136-libsasl2-modules_2.1.28+dfsg-10_amd64.deb ...
#7 29.34 Unpacking libsasl2-modules:amd64 (2.1.28+dfsg-10) ...
#7 29.38 Selecting previously unselected package libtinfo-dev:amd64.
#7 29.38 Preparing to unpack .../137-libtinfo-dev_6.4-4_amd64.deb ...
#7 29.38 Unpacking libtinfo-dev:amd64 (6.4-4) ...
#7 29.41 Selecting previously unselected package libxml2-dev:amd64.
#7 29.41 Preparing to unpack .../138-libxml2-dev_2.9.14+dfsg-1.3~deb12u1_amd64.deb ...
#7 29.42 Unpacking libxml2-dev:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
#7 29.53 Selecting previously unselected package libyaml-0-2:amd64.
#7 29.53 Preparing to unpack .../139-libyaml-0-2_0.2.5-1_amd64.deb ...
#7 29.53 Unpacking libyaml-0-2:amd64 (0.2.5-1) ...
#7 29.57 Selecting previously unselected package libz3-dev:amd64.
#7 29.57 Preparing to unpack .../140-libz3-dev_4.8.12-3.1_amd64.deb ...
#7 29.58 Unpacking libz3-dev:amd64 (4.8.12-3.1) ...
#7 29.62 Selecting previously unselected package llvm-14-runtime.
#7 29.62 Preparing to unpack .../141-llvm-14-runtime_1%3a14.0.6-12_amd64.deb ...
#7 29.63 Unpacking llvm-14-runtime (1:14.0.6-12) ...
#7 29.70 Selecting previously unselected package llvm-14.
#7 29.71 Preparing to unpack .../142-llvm-14_1%3a14.0.6-12_amd64.deb ...
#7 29.71 Unpacking llvm-14 (1:14.0.6-12) ...
#7 30.45 Selecting previously unselected package python3-pkg-resources.
#7 30.45 Preparing to unpack .../143-python3-pkg-resources_66.1.1-1_all.deb ...
#7 30.46 Unpacking python3-pkg-resources (66.1.1-1) ...
#7 30.52 Selecting previously unselected package python3-pygments.
#7 30.52 Preparing to unpack .../144-python3-pygments_2.14.0+dfsg-1_all.deb ...
#7 30.53 Unpacking python3-pygments (2.14.0+dfsg-1) ...
#7 30.65 Selecting previously unselected package python3-yaml.
#7 30.65 Preparing to unpack .../145-python3-yaml_6.0-3+b2_amd64.deb ...
#7 30.65 Unpacking python3-yaml (6.0-3+b2) ...
#7 30.70 Selecting previously unselected package llvm-14-tools.
#7 30.70 Preparing to unpack .../146-llvm-14-tools_1%3a14.0.6-12_amd64.deb ...
#7 30.70 Unpacking llvm-14-tools (1:14.0.6-12) ...
#7 30.84 Selecting previously unselected package llvm-14-dev.
#7 30.84 Preparing to unpack .../147-llvm-14-dev_1%3a14.0.6-12_amd64.deb ...
#7 30.85 Unpacking llvm-14-dev (1:14.0.6-12) ...
#7 32.20 Selecting previously unselected package manpages-dev.
#7 32.20 Preparing to unpack .../148-manpages-dev_6.03-2_all.deb ...
#7 32.20 Unpacking manpages-dev (6.03-2) ...
#7 32.48 Selecting previously unselected package publicsuffix.
#7 32.48 Preparing to unpack .../149-publicsuffix_20230209.2326-1_all.deb ...
#7 32.49 Unpacking publicsuffix (20230209.2326-1) ...
#7 32.55 Setting up libksba8:amd64 (1.6.3-2) ...
#7 32.56 Setting up media-types (10.0.0) ...
#7 32.57 Setting up libpipeline1:amd64 (1.5.7-1) ...
#7 32.58 Setting up libaom3:amd64 (3.6.0-1+deb12u1) ...
#7 32.59 Setting up libabsl20220623:amd64 (20220623.1-1) ...
#7 32.60 Setting up libxau6:amd64 (1:1.0.9-1) ...
#7 32.61 Setting up libpsl5:amd64 (0.21.2-1) ...
#7 32.62 Setting up libgpm2:amd64 (1.20.7-10+b1) ...
#7 32.64 Setting up libicu72:amd64 (72.1-3) ...
#7 32.65 Setting up liblerc4:amd64 (4.0.0+ds-2) ...
#7 32.66 Setting up libyaml-0-2:amd64 (0.2.5-1) ...
#7 32.67 Setting up manpages (6.03-2) ...
#7 32.68 Setting up libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#7 32.69 Setting up libbrotli1:amd64 (1.0.9-2+b6) ...
#7 32.70 Setting up libsasl2-modules:amd64 (2.1.28+dfsg-10) ...
#7 32.72 Setting up binutils-common:amd64 (2.40-2) ...
#7 32.73 Setting up libnghttp2-14:amd64 (1.52.0-1+deb12u1) ...
#7 32.74 Setting up libdeflate0:amd64 (1.14-1) ...
#7 32.75 Setting up linux-libc-dev:amd64 (6.1.112-1) ...
#7 32.76 Setting up libctf-nobfd0:amd64 (2.40-2) ...
#7 32.77 Setting up libnpth0:amd64 (1.6-3) ...
#7 32.78 Setting up libsvtav1enc1:amd64 (1.4.1+dfsg-1) ...
#7 32.79 Setting up libassuan0:amd64 (2.5.5-5) ...
#7 32.81 Setting up libgomp1:amd64 (12.2.0-14) ...
#7 32.82 Setting up bzip2 (1.0.8-5+b1) ...
#7 32.83 Setting up libffi-dev:amd64 (3.4.4-1) ...
#7 32.83 Setting up libldap-common (2.5.13+dfsg-5) ...
#7 32.85 Setting up libjbig0:amd64 (2.1-6.1) ...
#7 32.86 Setting up librav1e0:amd64 (0.5.1-6) ...
#7 32.87 Setting up libfakeroot:amd64 (1.31-1.2) ...
#7 32.88 Setting up libjansson4:amd64 (2.14-2) ...
#7 32.89 Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg-10) ...
#7 32.90 Setting up fakeroot (1.31-1.2) ...
#7 32.91 update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/man1/fakeroot.1.gz because associated file /usr/share/man/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/man1/faked.1.gz because associated file /usr/share/man/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/es/man1/fakeroot.1.gz because associated file /usr/share/man/es/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/es/man1/faked.1.gz because associated file /usr/share/man/es/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/fr/man1/fakeroot.1.gz because associated file /usr/share/man/fr/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/fr/man1/faked.1.gz because associated file /usr/share/man/fr/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/sv/man1/fakeroot.1.gz because associated file /usr/share/man/sv/man1/fakeroot-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.91 update-alternatives: warning: skip creation of /usr/share/man/sv/man1/faked.1.gz because associated file /usr/share/man/sv/man1/faked-sysv.1.gz (of link group fakeroot) doesn't exist
#7 32.92 Setting up perl-modules-5.36 (5.36.0-7+deb12u1) ...
#7 32.93 Setting up libz3-4:amd64 (4.8.12-3.1) ...
#7 32.94 Setting up libtirpc-dev:amd64 (1.3.3+ds-1) ...
#7 32.96 Setting up libpfm4:amd64 (4.13.0-1) ...
#7 32.97 Setting up rpcsvc-proto (1.4.3-1) ...
#7 32.98 Setting up libjpeg62-turbo:amd64 (1:2.1.5-2) ...
#7 32.99 Setting up libx11-data (2:1.8.4-2+deb12u2) ...
#7 33.00 Setting up make (4.3-4.1) ...
#7 33.01 Setting up libmpfr6:amd64 (4.2.0-1) ...
#7 33.02 Setting up libnspr4:amd64 (2:4.35-1) ...
#7 33.03 Setting up gnupg-l10n (2.2.40-1.1) ...
#7 33.04 Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ...
#7 33.05 Setting up libncurses6:amd64 (6.4-4) ...
#7 33.06 Setting up xz-utils (5.4.1-0.2) ...
#7 33.07 update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode
#7 33.07 update-alternatives: warning: skip creation of /usr/share/man/man1/lzma.1.gz because associated file /usr/share/man/man1/xz.1.gz (of link group lzma) doesn't exist
#7 33.07 update-alternatives: warning: skip creation of /usr/share/man/man1/unlzma.1.gz because associated file /usr/share/man/man1/unxz.1.gz (of link group lzma) doesn't exist
#7 33.07 update-alternatives: warning: skip creation of /usr/share/man/man1/lzcat.1.gz because associated file /usr/share/man/man1/xzcat.1.gz (of link group lzma) doesn't exist
#7 33.07 update-alternatives: warning: skip creation of /usr/share/man/man1/lzmore.1.gz because associated file /usr/share/man/man1/xzmore.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzless.1.gz because associated file /usr/share/man/man1/xzless.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzdiff.1.gz because associated file /usr/share/man/man1/xzdiff.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzcmp.1.gz because associated file /usr/share/man/man1/xzcmp.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzgrep.1.gz because associated file /usr/share/man/man1/xzgrep.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzegrep.1.gz because associated file /usr/share/man/man1/xzegrep.1.gz (of link group lzma) doesn't exist
#7 33.08 update-alternatives: warning: skip creation of /usr/share/man/man1/lzfgrep.1.gz because associated file /usr/share/man/man1/xzfgrep.1.gz (of link group lzma) doesn't exist
#7 33.08 Setting up libquadmath0:amd64 (12.2.0-14) ...
#7 33.10 Setting up libpng16-16:amd64 (1.6.39-2) ...
#7 33.11 Setting up libmpc3:amd64 (1.3.1-1) ...
#7 33.12 Setting up libatomic1:amd64 (12.2.0-14) ...
#7 33.13 Setting up patch (2.7.6-7) ...
#7 33.14 Setting up binfmt-support (2.2.2-2) ...
#7 33.17 invoke-rc.d: could not determine current runlevel
#7 33.17 invoke-rc.d: policy-rc.d denied execution of start.
#7 33.32 Setting up icu-devtools (72.1-3) ...
#7 33.33 Setting up fonts-dejavu-core (2.37-6) ...
#7 33.39 Setting up libgdbm-compat4:amd64 (1.23-3) ...
#7 33.40 Setting up libgav1-1:amd64 (0.18.0-1+b1) ...
#7 33.41 Setting up libgc1:amd64 (1:8.2.2-3) ...
#7 33.42 Setting up libdav1d6:amd64 (1.0.0-2+deb12u1) ...
#7 33.43 Setting up libsasl2-2:amd64 (2.1.28+dfsg-10) ...
#7 33.44 Setting up libwebp7:amd64 (1.2.4-0.2+deb12u1) ...
#7 33.44 Setting up libubsan1:amd64 (12.2.0-14) ...
#7 33.45 Setting up libnuma1:amd64 (2.0.16-1) ...
#7 33.47 Setting up libnsl-dev:amd64 (1.3.0-2) ...
#7 33.47 Setting up libcrypt-dev:amd64 (1:4.4.33-2) ...
#7 33.49 Setting up libtiff6:amd64 (4.5.0-6+deb12u1) ...
#7 33.50 Setting up libasan8:amd64 (12.2.0-14) ...
#7 33.51 Setting up gpgconf (2.2.40-1.1) ...
#7 33.52 Setting up libc6-i386 (2.36-9+deb12u8) ...
#7 33.54 Setting up libssh2-1:amd64 (1.10.0-3+b1) ...
#7 33.55 Setting up libtsan2:amd64 (12.2.0-14) ...
#7 33.56 Setting up libbinutils:amd64 (2.40-2) ...
#7 33.57 Setting up libisl23:amd64 (0.25-1.1) ...
#7 33.58 Setting up libde265-0:amd64 (1.0.11-1+deb12u2) ...
#7 33.59 Setting up libc-dev-bin (2.36-9+deb12u8) ...
#7 33.60 Setting up libbsd0:amd64 (0.11.7-2) ...
#7 33.61 Setting up libyuv0:amd64 (0.0~git20230123.b2528b0-1) ...
#7 33.62 Setting up publicsuffix (20230209.2326-1) ...
#7 33.65 Setting up libxml2:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
#7 33.66 Setting up libcc1-0:amd64 (12.2.0-14) ...
#7 33.67 Setting up libperl5.36:amd64 (5.36.0-7+deb12u1) ...
#7 33.68 Setting up liblocale-gettext-perl (1.07-5) ...
#7 33.69 Setting up gpg (2.2.40-1.1) ...
#7 33.70 Setting up liblsan0:amd64 (12.2.0-14) ...
#7 33.71 Setting up libitm1:amd64 (12.2.0-14) ...
#7 33.72 Setting up libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#7 33.73 Setting up gnupg-utils (2.2.40-1.1) ...
#7 33.74 Setting up libctf0:amd64 (2.40-2) ...
#7 33.75 Setting up pinentry-curses (1.2.1-1) ...
#7 33.77 Setting up libz3-dev:amd64 (4.8.12-3.1) ...
#7 33.78 Setting up manpages-dev (6.03-2) ...
#7 33.79 Setting up python3.11 (3.11.2-6+deb12u3) ...
#7 34.68 Setting up libxdmcp6:amd64 (1:1.1.2-3) ...
#7 34.69 Setting up cpp-12 (12.2.0-14) ...
#7 34.70 Setting up libxcb1:amd64 (1.15-1) ...
#7 34.71 Setting up gpg-agent (2.2.40-1.1) ...
#7 35.32 Setting up libavif15:amd64 (0.11.1-1) ...
#7 35.34 Setting up fontconfig-config (2.14.1-4) ...
#7 35.45 debconf: unable to initialize frontend: Dialog
#7 35.45 debconf: (TERM is not set, so the dialog frontend is not usable.)
#7 35.45 debconf: falling back to frontend: Readline
#7 35.46 debconf: unable to initialize frontend: Readline
#7 35.46 debconf: (This frontend requires a controlling tty.)
#7 35.46 debconf: falling back to frontend: Teletype
#7 35.66 Setting up libedit2:amd64 (3.1-20221030-2) ...
#7 35.67 Setting up gpgsm (2.2.40-1.1) ...
#7 35.68 Setting up libobjc4:amd64 (12.2.0-14) ...
#7 35.69 Setting up python3 (3.11.2-1+b1) ...
#7 35.71 running python rtupdate hooks for python3.11...
#7 35.71 running python post-rtupdate hooks for python3.11...
#7 35.82 Setting up libnss3:amd64 (2:3.87.1-1) ...
#7 35.83 Setting up libldap-2.5-0:amd64 (2.5.13+dfsg-5) ...
#7 35.84 Setting up dirmngr (2.2.40-1.1) ...
#7 36.01 Setting up perl (5.36.0-7+deb12u1) ...
#7 36.04 Setting up libgprofng0:amd64 (2.40-2) ...
#7 36.05 Setting up libfreetype6:amd64 (2.12.1+dfsg-5+deb12u3) ...
#7 36.06 Setting up lib32gcc-s1 (12.2.0-14) ...
#7 36.07 Setting up lib32stdc++6 (12.2.0-14) ...
#7 36.08 Setting up libgcc-12-dev:amd64 (12.2.0-14) ...
#7 36.09 Setting up libdpkg-perl (1.21.22) ...
#7 36.10 Setting up libx265-199:amd64 (3.5-2+b1) ...
#7 36.11 Setting up gpg-wks-server (2.2.40-1.1) ...
#7 36.12 Setting up libllvm14:amd64 (1:14.0.6-12) ...
#7 36.13 Setting up libobjc-12-dev:amd64 (12.2.0-14) ...
#7 36.14 Setting up cpp (4:12.2.0-3) ...
#7 36.16 Setting up libcurl4:amd64 (7.88.1-10+deb12u7) ...
#7 36.17 Setting up libc6-dev:amd64 (2.36-9+deb12u8) ...
#7 36.18 Setting up libx11-6:amd64 (2:1.8.4-2+deb12u2) ...
#7 36.19 Setting up curl (7.88.1-10+deb12u7) ...
#7 36.20 Setting up libfontconfig1:amd64 (2.14.1-4) ...
#7 36.21 Setting up llvm-14-linker-tools (1:14.0.6-12) ...
#7 36.22 Setting up libicu-dev:amd64 (72.1-3) ...
#7 36.23 Setting up binutils-x86-64-linux-gnu (2.40-2) ...
#7 36.24 Setting up python3-pkg-resources (66.1.1-1) ...
#7 36.55 Setting up libncurses-dev:amd64 (6.4-4) ...
#7 36.56 Setting up libxpm4:amd64 (1:3.5.12-1.1+deb12u1) ...
#7 36.57 Setting up gpg-wks-client (2.2.40-1.1) ...
#7 36.58 Setting up libstdc++-12-dev:amd64 (12.2.0-14) ...
#7 36.59 Setting up libfile-fcntllock-perl (0.22-4+b1) ...
#7 36.60 Setting up libalgorithm-diff-perl (1.201-1) ...
#7 36.61 Setting up libheif1:amd64 (1.15.1-1) ...
#7 36.62 Setting up nss-plugin-pem:amd64 (1.0.8+1-1) ...
#7 36.63 Setting up libclang1-14 (1:14.0.6-12) ...
#7 36.64 Setting up python3-yaml (6.0-3+b2) ...
#7 36.83 Setting up binutils (2.40-2) ...
#7 36.84 Setting up dpkg-dev (1.21.22) ...
#7 36.86 Setting up python3-pygments (2.14.0+dfsg-1) ...
#7 37.64 Setting up libxml2-dev:amd64 (2.9.14+dfsg-1.3~deb12u1) ...
#7 37.65 Setting up llvm-14-runtime (1:14.0.6-12) ...
#7 37.66 Setting up libclang-rt-14-dev:amd64 (1:14.0.6-12) ...
#7 37.67 Setting up gcc-12 (12.2.0-14) ...
#7 37.68 Setting up libgd3:amd64 (2.3.3-9) ...
#7 37.69 Setting up libclang-common-14-dev (1:14.0.6-12) ...
#7 37.70 Setting up libclang-cpp14 (1:14.0.6-12) ...
#7 37.71 Setting up libcurl3-nss:amd64 (7.88.1-10+deb12u7) ...
#7 37.72 Setting up gnupg (2.2.40-1.1) ...
#7 37.73 Setting up libalgorithm-diff-xs-perl:amd64 (0.04-8+b1) ...
#7 37.74 Setting up libc-devtools (2.36-9+deb12u8) ...
#7 37.75 Setting up libalgorithm-merge-perl (0.08-5) ...
#7 37.77 Setting up g++-12 (12.2.0-14) ...
#7 37.78 Setting up llvm-14 (1:14.0.6-12) ...
#7 37.79 Setting up llvm-14-tools (1:14.0.6-12) ...
#7 37.80 Setting up libtinfo-dev:amd64 (6.4-4) ...
#7 37.81 Setting up clang-14 (1:14.0.6-12) ...
#7 37.82 Setting up gcc (4:12.2.0-3) ...
#7 37.85 Setting up clang (1:14.0-55.7~deb12u1) ...
#7 37.88 Setting up g++ (4:12.2.0-3) ...
#7 37.89 update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
#7 37.91 Setting up build-essential (12.9) ...
#7 37.92 Setting up llvm-14-dev (1:14.0.6-12) ...
#7 37.93 Processing triggers for libc-bin (2.36-9+deb12u8) ...
#7 DONE 40.8s

#10 [stage-0 3/10] WORKDIR /workdir
#10 DONE 0.0s

#11 [stage-0 4/10] RUN ln -s /usr/bin/python3 /usr/bin/python
#11 DONE 0.2s

#12 [stage-0 5/10] RUN mkdir /artifacts
#12 DONE 0.2s

#13 [stage-0 6/10] COPY . .
#13 DONE 2.0s

#14 [stage-0 7/10] RUN tools/bazel build -c dbg //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client
#14 0.166 INFO: Running bazel wrapper (see //tools/bazel for details), bazel version 7.3.1 will be used instead of system-wide bazel installation.
#14 0.171 Downloading bazel, will try URLs: https://storage.googleapis.com/grpc-bazel-mirror/github.com/bazelbuild/bazel/releases/download/7.3.1/bazel-7.3.1-linux-x86_64 https://github.com/bazelbuild/bazel/releases/download/7.3.1/bazel-7.3.1-linux-x86_64
#14 0.181 % Total % Received % Xferd Average Speed Time Time Time Current
#14 0.181 Dload Upload Total Spent Left Speed
#14 0.181 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 14 68.2M 14 10.0M 0 0 69.9M 0 --:--:-- --:--:-- --:--:-- 69.6M100 68.2M 100 68.2M 0 0 139M 0 --:--:-- --:--:-- --:--:-- 139M
#14 0.680 Extracting Bazel installation...
#14 3.534 Starting local Bazel server and connecting to it...
#14 5.582 Computing main repo mapping:
#14 6.586 Computing main repo mapping:
#14 7.586 Computing main repo mapping:
#14 8.587 Computing main repo mapping:
#14 9.587 Computing main repo mapping:
#14 10.72 Loading:
#14 10.72 Loading: 1 packages loaded
#14 10.98 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (2 packages loaded, 0 targets configured)
#14 11.00 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (2 packages loaded, 0 targets configured)
#14 11.00 [0 / 1] checking cached actions
#14 12.28 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (52 packages loaded, 9 targets configured)
#14 12.28 [1 / 1] checking cached actions
#14 13.59 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (52 packages loaded, 9 targets configured)
#14 13.59 [1 / 1] checking cached actions
#14 15.41 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (52 packages loaded, 9 targets configured)
#14 15.41 [1 / 1] checking cached actions
#14 16.41 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (96 packages loaded, 2471 targets configured)
#14 16.41 [1 / 1] checking cached actions
#14 17.59 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (197 packages loaded, 4626 targets configured)
#14 17.59 [1 / 1] checking cached actions
#14 18.66 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (197 packages loaded, 4626 targets configured)
#14 18.66 [1 / 1] checking cached actions
#14 20.27 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (228 packages loaded, 7249 targets configured)
#14 20.27 [1 / 1] checking cached actions
#14 21.33 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (229 packages loaded, 7565 targets configured)
#14 21.33 [1 / 1] checking cached actions
#14 22.58 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (233 packages loaded, 7683 targets configured)
#14 22.58 [1 / 1] checking cached actions
#14 23.60 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (239 packages loaded, 7860 targets configured)
#14 23.60 [1 / 1] checking cached actions
#14 24.27 INFO: Analyzed target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client (245 packages loaded, 8021 targets configured).
#14 24.61 [700 / 3,386] Creating runfiles tree bazel-out/k8-dbg/bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_client.runfiles; 0s local ... (8 actions, 6 running)
#14 25.62 [1,581 / 4,110] Compiling absl/time/internal/cctz/src/time_zone_format.cc [for tool]; 0s processwrapper-sandbox ... (16 actions, 15 running)
#14 26.62 [1,606 / 4,110] Compiling absl/time/internal/cctz/src/time_zone_format.cc [for tool]; 1s processwrapper-sandbox ... (16 actions, 14 running)
#14 27.63 [1,655 / 4,110] Compiling absl/time/internal/cctz/src/time_zone_info.cc [for tool]; 1s processwrapper-sandbox ... (17 actions, 16 running)
#14 28.64 [1,702 / 4,110] Compiling absl/time/internal/cctz/src/time_zone_info.cc [for tool]; 2s processwrapper-sandbox ... (17 actions, 16 running)
#14 29.64 [1,746 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 2s processwrapper-sandbox ... (17 actions, 16 running)
#14 30.68 [1,785 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 31.76 [1,807 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 32.81 [1,844 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 33.24 INFO: From Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation:
#14 33.24 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/117/execroot/com_github_grpc_grpc/src/python/grpcio_observability/grpc_observability/_cyobservability.pxd
#14 33.24 tree = Parsing.p_module(s, pxd, full_module_name)
#14 33.35 INFO: From Executing genrule //src/python/grpcio_tests/tests/fork:native_debug.pyx_cython_translation:
#14 33.35 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/217/execroot/com_github_grpc_grpc/src/python/grpcio_tests/tests/fork/native_debug.pyx
#14 33.35 tree = Parsing.p_module(s, pxd, full_module_name)
#14 33.85 [1,863 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 34.98 [1,884 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 36.05 [1,902 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 37.08 [1,921 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 10s processwrapper-sandbox ... (16 actions, 15 running)
#14 38.08 [1,934 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 39.14 [1,950 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 12s processwrapper-sandbox ... (16 actions, 15 running)
#14 40.15 [1,964 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 13s processwrapper-sandbox ... (16 actions, 15 running)
#14 41.23 [1,983 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 14s processwrapper-sandbox ... (17 actions, 15 running)
#14 42.26 [2,000 / 4,110] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 15s processwrapper-sandbox ... (17 actions, 15 running)
#14 42.40 INFO: From Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation:
#14 42.40 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/61/execroot/com_github_grpc_grpc/src/python/grpcio/grpc/_cython/cygrpc.pxd
#14 42.40 tree = Parsing.p_module(s, pxd, full_module_name)
#14 42.40 warning: src/python/grpcio/grpc/_cython/cygrpc.pxd:40:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#14 42.40 warning: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pxd.pxi:19:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#14 42.40 warning: src/python/grpcio/grpc/_cython/cygrpc.pyx:64:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#14 42.40 warning: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:24:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pxd.pxi:24:22: No exception value declared for '_get_metadata' in pxd file.
#14 42.40 Users cimporting this function and calling it without the gil will always require an exception check.
#14 42.40 Suggest adding an explicit exception value.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi:80:5: Exception check on '_destroy' will always require the GIL to be acquired.
#14 42.40 Possible solutions:
#14 42.40 1. Declare '_destroy' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 2. Use an 'int' return type on '_destroy' to allow an error code to be returned.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi:104:5: Exception check on '_on_shutdown' will always require the GIL to be acquired.
#14 42.40 Possible solutions:
#14 42.40 1. Declare '_on_shutdown' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 2. Use an 'int' return type on '_on_shutdown' to allow an error code to be returned.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi:21:5: Exception check on '_copy_slice' will always require the GIL to be acquired. Declare '_copy_slice' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi:26:5: Exception check on '_slice_from_bytes' will always require the GIL to be acquired. Declare '_slice_from_bytes' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/security.pyx.pxi:20:5: Exception check on 'ssl_roots_override_callback' will always require the GIL to be acquired. Declare 'ssl_roots_override_callback' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:30:9: Exception check on '_unified_socket_write' will always require the GIL to be acquired.
#14 42.40 Possible solutions:
#14 42.40 1. Declare '_unified_socket_write' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 2. Use an 'int' return type on '_unified_socket_write' to allow an error code to be returned.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:116:41: Exception check after calling '_unified_socket_write' will always require the GIL to be acquired.
#14 42.40 Possible solutions:
#14 42.40 1. Declare '_unified_socket_write' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 2. Use an 'int' return type on '_unified_socket_write' to allow an error code to be returned.
#14 42.40 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:126:22: Exception check after calling '_poll' will always require the GIL to be acquired.
#14 42.40 Possible solutions:
#14 42.40 1. Declare '_poll' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#14 42.40 2. Use an 'int' return type on '_poll' to allow an error code to be returned.
#14 43.28 [2,039 / 4,110] Compiling src/ssl/dtls_record.cc; 1s processwrapper-sandbox ... (17 actions, 15 running)
#14 44.32 [2,085 / 4,110] Compiling src/core/util/dump_args.cc; 1s processwrapper-sandbox ... (16 actions, 15 running)
#14 45.34 [2,139 / 4,110] Compiling absl/log/internal/globals.cc [for tool]; 0s processwrapper-sandbox ... (17 actions, 15 running)
#14 46.35 [2,175 / 4,110] Compiling absl/strings/internal/cordz_info.cc; 1s processwrapper-sandbox ... (16 actions, 15 running)
#14 47.35 [2,226 / 4,110] Compiling absl/strings/internal/cord_rep_btree_navigator.cc; 0s processwrapper-sandbox ... (15 actions, 13 running)
#14 48.36 [2,283 / 4,110] Compiling absl/strings/cord.cc; 1s processwrapper-sandbox ... (16 actions, 15 running)
#14 49.39 [2,347 / 4,110] Compiling absl/strings/cord.cc; 2s processwrapper-sandbox ... (15 actions, 14 running)
#14 50.64 [2,386 / 4,110] Compiling absl/debugging/symbolize.cc [for tool]; 1s processwrapper-sandbox ... (17 actions, 16 running)
#14 51.64 [2,398 / 4,110] Compiling src/crypto/fipsmodule/bcm.c; 1s processwrapper-sandbox ... (17 actions, 16 running)
#14 52.65 [2,404 / 4,110] Compiling src/crypto/fipsmodule/bcm.c; 2s processwrapper-sandbox ... (17 actions, 16 running)
#14 53.84 [2,417 / 4,110] Compiling absl/strings/internal/str_format/arg.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#14 54.95 [2,427 / 4,110] Compiling absl/status/statusor.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 56.13 [2,438 / 4,110] Compiling absl/flags/reflection.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 57.18 [2,449 / 4,110] Compiling absl/flags/marshalling.cc [for tool]; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 58.23 [2,459 / 4,110] Compiling absl/strings/internal/str_format/float_conversion.cc [for tool]; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 59.42 [2,469 / 4,110] Compiling src/google/protobuf/arenastring.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 60.46 [2,475 / 4,110] Compiling src/google/protobuf/io/printer.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#14 61.51 [2,484 / 4,110] Compiling src/google/protobuf/io/printer.cc; 4s processwrapper-sandbox ... (17 actions, 15 running)
#14 62.58 [2,490 / 4,110] Compiling src/google/protobuf/io/printer.cc; 5s processwrapper-sandbox ... (17 actions, 15 running)
#14 63.58 [2,499 / 4,110] Compiling python/google/protobuf/pyext/descriptor.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 64.65 [2,506 / 4,110] Compiling src/google/protobuf/extension_set.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#14 65.65 [2,514 / 4,110] Compiling src/google/protobuf/extension_set.cc; 4s processwrapper-sandbox ... (17 actions, 16 running)
#14 66.65 [2,520 / 4,110] Compiling src/google/protobuf/extension_set.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#14 67.65 [2,532 / 4,110] Compiling src/core/lib/event_engine/thready_event_engine/thready_event_engine.cc; 3s processwrapper-sandbox ... (17 actions, 16 running)
#14 68.73 [2,535 / 4,110] Compiling python/google/protobuf/pyext/descriptor_containers.cc; 3s processwrapper-sandbox ... (17 actions, 16 running)
#14 69.89 [2,546 / 4,110] Compiling src/google/protobuf/generated_message_tctable_lite.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 70.96 [2,555 / 4,110] Compiling src/google/protobuf/generated_message_tctable_lite.cc; 5s processwrapper-sandbox ... (17 actions, 15 running)
#14 72.07 [2,561 / 4,110] Compiling python/google/protobuf/pyext/descriptor_database.cc; 3s processwrapper-sandbox ... (17 actions, 15 running)
#14 73.20 [2,573 / 4,110] Compiling python/google/protobuf/pyext/descriptor_database.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 74.61 [2,578 / 4,110] Compiling src/google/protobuf/wire_format.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 75.73 [2,588 / 4,110] Compiling absl/flags/parse.cc [for tool]; 3s processwrapper-sandbox ... (17 actions, 16 running)
#14 76.73 [2,594 / 4,110] Compiling absl/flags/parse.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 16 running)
#14 77.81 [2,605 / 4,110] Compiling python/google/protobuf/pyext/descriptor_pool.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#14 78.95 [2,615 / 4,110] Compiling python/google/protobuf/pyext/descriptor_pool.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 80.18 [2,627 / 4,110] Compiling src/google/protobuf/text_format.cc; 5s processwrapper-sandbox ... (15 actions, 14 running)
#14 81.32 [2,633 / 4,110] Compiling src/google/protobuf/text_format.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#14 82.73 [2,638 / 4,110] Compiling src/google/protobuf/text_format.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 83.73 [2,647 / 4,110] Compiling src/core/xds/grpc/xds_metadata.cc; 4s processwrapper-sandbox ... (17 actions, 16 running)
#14 84.75 [2,652 / 4,110] Compiling python/google/protobuf/pyext/extension_dict.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 86.17 [2,656 / 4,110] Compiling src/google/protobuf/io/printer.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 15 running)
#14 87.73 [2,662 / 4,110] Compiling src/google/protobuf/io/printer.cc [for tool]; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 88.73 [2,667 / 4,110] Compiling src/google/protobuf/io/printer.cc [for tool]; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 89.74 [2,674 / 4,110] Compiling src/google/protobuf/wire_format.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 16 running)
#14 90.74 [2,675 / 4,110] Compiling src/google/protobuf/wire_format.cc [for tool]; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 91.74 [2,680 / 4,110] Compiling src/google/protobuf/wire_format.cc [for tool]; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 92.74 [2,685 / 4,110] Compiling src/google/protobuf/wire_format.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 93.74 [2,689 / 4,110] Compiling src/google/protobuf/feature_resolver.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#14 94.75 [2,692 / 4,110] Compiling src/google/protobuf/feature_resolver.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 96.58 [2,698 / 4,110] Compiling src/google/protobuf/descriptor_database.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 97.75 [2,700 / 4,110] Compiling src/google/protobuf/descriptor_database.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 98.91 [2,705 / 4,110] Compiling src/google/protobuf/extension_set.cc [for tool]; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 100.3 [2,709 / 4,110] Compiling src/google/protobuf/descriptor.pb.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 101.6 [2,714 / 4,110] Compiling src/google/protobuf/descriptor.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 102.7 [2,714 / 4,110] Compiling src/google/protobuf/descriptor.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 103.8 [2,721 / 4,110] Compiling src/google/protobuf/descriptor.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#14 105.2 [2,725 / 4,110] Compiling src/google/protobuf/descriptor.cc; 12s processwrapper-sandbox ... (16 actions, 15 running)
#14 106.2 [2,728 / 4,110] Compiling src/google/protobuf/descriptor.cc; 13s processwrapper-sandbox ... (17 actions, 15 running)
#14 107.2 [2,733 / 4,110] Compiling src/google/protobuf/descriptor.cc; 14s processwrapper-sandbox ... (16 actions, 15 running)
#14 108.6 [2,736 / 4,110] Compiling src/google/protobuf/descriptor.cc; 15s processwrapper-sandbox ... (16 actions, 15 running)
#14 109.7 [2,738 / 4,110] Compiling src/google/protobuf/descriptor.cc; 16s processwrapper-sandbox ... (16 actions, 15 running)
#14 110.8 [2,740 / 4,110] Compiling src/google/protobuf/descriptor.cc; 17s processwrapper-sandbox ... (17 actions, 16 running)
#14 111.8 [2,741 / 4,110] Compiling src/google/protobuf/descriptor.cc; 18s processwrapper-sandbox ... (17 actions, 16 running)
#14 113.1 [2,747 / 4,110] Compiling src/google/protobuf/descriptor.cc; 20s processwrapper-sandbox ... (16 actions, 15 running)
#14 114.1 [2,748 / 4,110] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 15 running)
#14 115.7 [2,752 / 4,110] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 12s processwrapper-sandbox ... (16 actions, 15 running)
#14 116.8 [2,753 / 4,110] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 16 running)
#14 117.8 [2,757 / 4,110] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 14s processwrapper-sandbox ... (17 actions, 16 running)
#14 119.0 [2,761 / 4,110] Compiling src/google/protobuf/compiler/cpp/enum.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 15 running)
#14 120.2 [2,766 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 121.8 [2,768 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 122.8 [2,770 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 124.0 [2,773 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 13s processwrapper-sandbox ... (16 actions, 15 running)
#14 125.5 [2,777 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 14s processwrapper-sandbox ... (16 actions, 15 running)
#14 126.7 [2,778 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 15 running)
#14 127.8 [2,779 / 4,110] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 16 running)
#14 128.8 [2,781 / 4,110] Compiling src/google/protobuf/compiler/cpp/extension.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 129.8 [2,784 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 130.8 [2,787 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 131.8 [2,788 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 16 running)
#14 132.8 [2,789 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 16 running)
#14 133.8 [2,790 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 14s processwrapper-sandbox ... (17 actions, 16 running)
#14 134.8 [2,792 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 16 running)
#14 135.8 [2,793 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 16 running)
#14 136.8 [2,796 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 17s processwrapper-sandbox ... (17 actions, 16 running)
#14 137.8 [2,799 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 18s processwrapper-sandbox ... (17 actions, 16 running)
#14 138.8 [2,802 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 19s processwrapper-sandbox ... (17 actions, 16 running)
#14 139.8 [2,803 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 20s processwrapper-sandbox ... (17 actions, 15 running)
#14 141.1 [2,805 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 21s processwrapper-sandbox ... (16 actions, 15 running)
#14 142.3 [2,808 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 23s processwrapper-sandbox ... (16 actions, 15 running)
#14 143.8 [2,810 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 24s processwrapper-sandbox ... (17 actions, 16 running)
#14 144.8 [2,812 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 25s processwrapper-sandbox ... (17 actions, 16 running)
#14 146.0 [2,814 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 26s processwrapper-sandbox ... (17 actions, 15 running)
#14 147.7 [2,816 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 28s processwrapper-sandbox ... (17 actions, 15 running)
#14 148.8 [2,820 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 29s processwrapper-sandbox ... (17 actions, 16 running)
#14 149.9 [2,822 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 30s processwrapper-sandbox ... (16 actions, 15 running)
#14 151.1 [2,824 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 31s processwrapper-sandbox ... (16 actions, 15 running)
#14 152.8 [2,825 / 4,110] Compiling src/google/protobuf/descriptor.cc [for tool]; 33s processwrapper-sandbox ... (17 actions, 16 running)
#14 153.8 [2,829 / 4,110] Compiling src/google/protobuf/compiler/python/generator.cc [for tool]; 12s processwrapper-sandbox ... (16 actions running)
#14 155.0 [2,830 / 4,110] Compiling src/google/protobuf/compiler/objectivec/message.cc [for tool]; 10s processwrapper-sandbox ... (16 actions, 15 running)
#14 156.2 [2,838 / 4,110] Compiling src/google/protobuf/compiler/objectivec/field.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 157.8 [2,840 / 4,110] Compiling src/google/protobuf/compiler/objectivec/file.cc [for tool]; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 158.8 [2,841 / 4,110] Compiling src/google/protobuf/compiler/objectivec/file.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 159.8 [2,842 / 4,110] Compiling src/google/protobuf/compiler/objectivec/file.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 160.8 [2,846 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 15 running)
#14 162.6 [2,848 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 15 running)
#14 163.7 [2,852 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 14s processwrapper-sandbox ... (16 actions, 15 running)
#14 164.8 [2,854 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 16 running)
#14 165.8 [2,855 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 16 running)
#14 166.8 [2,856 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 17s processwrapper-sandbox ... (17 actions, 16 running)
#14 167.9 [2,859 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 18s processwrapper-sandbox ... (16 actions, 15 running)
#14 169.0 [2,861 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 19s processwrapper-sandbox ... (17 actions, 15 running)
#14 170.3 [2,862 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 20s processwrapper-sandbox ... (16 actions, 15 running)
#14 171.4 [2,863 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 21s processwrapper-sandbox ... (17 actions, 15 running)
#14 172.4 [2,865 / 4,110] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 22s processwrapper-sandbox ... (17 actions, 15 running)
#14 173.8 [2,870 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 18s processwrapper-sandbox ... (14 actions running)
#14 174.8 [2,872 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 19s processwrapper-sandbox ... (12 actions running)
#14 176.8 [2,878 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 21s processwrapper-sandbox ... (6 actions running)
#14 178.8 [2,881 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 23s processwrapper-sandbox ... (3 actions running)
#14 179.8 [2,882 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 24s processwrapper-sandbox ... (2 actions running)
#14 180.8 [2,883 / 4,110] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 25s processwrapper-sandbox
#14 183.6 [2,884 / 4,110] [Prepa] Linking external/com_google_protobuf/protoc [for tool]
#14 183.9 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/duration_pb2.py:
#14 183.9 external/com_google_protobuf/.: warning: directory does not exist.
#14 183.9 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/empty_pb2.py:
#14 183.9 external/com_google_protobuf/.: warning: directory does not exist.
#14 183.9 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/timestamp_pb2.py:
#14 183.9 external/com_google_protobuf/.: warning: directory does not exist.
#14 183.9 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/source_context_pb2.py:
#14 183.9 external/com_google_protobuf/.: warning: directory does not exist.
#14 183.9 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/wrappers_pb2.py:
#14 183.9 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/struct_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/api_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/any_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/type_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/field_mask_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/compiler/plugin_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.0 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/descriptor_pb2.py:
#14 184.0 external/com_google_protobuf/.: warning: directory does not exist.
#14 184.8 [2,921 / 4,110] Compiling external/com_google_protobuf/src/google/protobuf/_virtual_imports/timestamp_proto/google/protobuf/timestamp.pb.cc [for tool]; 0s processwrapper-sandbox ... (17 actions, 16 running)
#14 188.0 [2,922 / 4,110] Compiling external/com_google_protobuf/src/google/protobuf/_virtual_imports/timestamp_proto/google/protobuf/timestamp.pb.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 15 running)
#14 189.4 [2,931 / 4,110] Compiling src/google/protobuf/util/field_mask_util.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 15 running)
#14 190.8 [2,933 / 4,110] Compiling src/google/protobuf/util/field_mask_util.cc [for tool]; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 191.8 [2,934 / 4,110] Compiling src/google/protobuf/json/internal/parser.cc [for tool]; 7s processwrapper-sandbox ... (16 actions running)
#14 192.8 [2,936 / 4,110] Compiling src/google/protobuf/json/internal/parser.cc [for tool]; 8s processwrapper-sandbox ... (14 actions running)
#14 194.8 [2,939 / 4,110] Compiling src/google/protobuf/json/internal/parser.cc [for tool]; 10s processwrapper-sandbox ... (11 actions running)
#14 195.8 [2,941 / 4,110] Compiling src/google/protobuf/json/internal/parser.cc [for tool]; 11s processwrapper-sandbox ... (9 actions running)
#14 196.8 [2,945 / 4,110] Compiling src/google/protobuf/json/internal/parser.cc [for tool]; 12s processwrapper-sandbox ... (5 actions running)
#14 198.8 [2,974 / 4,110] Compiling src/compiler/python_generator.cc [for tool]; 10s processwrapper-sandbox ... (7 actions running)
#14 199.8 [2,979 / 4,110] Compiling upb_generator/file_layout.cc [for tool]; 2s processwrapper-sandbox ... (5 actions running)
#14 200.8 [2,981 / 4,110] Compiling upb_generator/names.cc [for tool]; 3s processwrapper-sandbox ... (3 actions running)
#14 201.8 [3,016 / 4,110] Compiling upb_generator/protoc-gen-upb.cc [for tool]; 4s processwrapper-sandbox ... (16 actions running)
#14 202.8 [3,109 / 4,110] Compiling upb_generator/protoc-gen-upb.cc [for tool]; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 203.9 [3,197 / 4,110] Compiling external/envoy_api/envoy/config/bootstrap/v3/bootstrap.upb_minitable.c; 0s processwrapper-sandbox ... (16 actions, 15 running)
#14 204.9 [3,303 / 4,110] Compiling src/core/tsi/alts/handshaker/transport_security_common_api.cc; 0s processwrapper-sandbox ... (16 actions, 15 running)
#14 205.9 [3,381 / 4,110] Compiling src/core/lib/security/certificate_provider/certificate_provider_registry.cc; 1s processwrapper-sandbox ... (17 actions, 15 running)
#14 206.9 [3,445 / 4,110] Compiling src/core/lib/security/certificate_provider/certificate_provider_registry.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#14 208.0 [3,461 / 4,110] Compiling upb_generator/file_layout.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#14 209.0 [3,476 / 4,110] Compiling upb_generator/file_layout.cc [for tool]; 3s processwrapper-sandbox ... (17 actions, 15 running)
#14 210.0 [3,494 / 4,110] Compiling src/core/lib/resource_quota/memory_quota.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 211.2 [3,505 / 4,110] Compiling src/core/lib/resource_quota/memory_quota.cc; 5s processwrapper-sandbox ... (17 actions, 15 running)
#14 212.2 [3,524 / 4,110] Compiling src/core/lib/resource_quota/resource_quota.cc; 3s processwrapper-sandbox ... (17 actions, 15 running)
#14 213.4 [3,530 / 4,110] Compiling src/core/lib/event_engine/posix_engine/posix_endpoint.cc; 3s processwrapper-sandbox ... (17 actions, 15 running)
#14 214.5 [3,535 / 4,110] Compiling src/core/lib/event_engine/posix_engine/posix_endpoint.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 215.5 [3,545 / 4,110] Compiling upb_generator/protoc-gen-upbdefs.cc [for tool]; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 216.6 [3,551 / 4,110] Compiling src/core/lib/event_engine/ares_resolver.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 217.7 [3,558 / 4,110] Compiling src/core/lib/event_engine/ares_resolver.cc; 5s processwrapper-sandbox ... (15 actions, 14 running)
#14 218.7 [3,567 / 4,110] Compiling src/core/channelz/channelz_registry.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#14 219.8 [3,572 / 4,110] Compiling src/core/telemetry/call_tracer.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 220.8 [3,580 / 4,110] Compiling src/core/resolver/polling_resolver.cc; 4s processwrapper-sandbox ... (16 actions running)
#14 221.8 [3,592 / 4,110] Compiling src/core/lib/event_engine/posix_engine/posix_engine.cc; 4s processwrapper-sandbox ... (16 actions running)
#14 222.8 [3,623 / 4,110] Compiling src/core/lib/event_engine/posix_engine/posix_engine.cc; 5s processwrapper-sandbox ... (16 actions running)
#14 223.8 [3,651 / 4,110] Compiling src/core/lib/event_engine/posix_engine/posix_engine.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 224.8 [3,661 / 4,110] Compiling src/core/channelz/channelz.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#14 225.8 [3,672 / 4,110] Compiling src/core/lib/transport/call_filters.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#14 226.8 [3,685 / 4,110] Compiling src/core/lib/transport/call_filters.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 227.8 [3,700 / 4,110] Compiling src/core/lib/transport/call_spine.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 228.8 [3,717 / 4,110] Compiling src/core/lib/transport/call_spine.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 229.9 [3,735 / 4,110] Compiling src/core/lib/transport/call_spine.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 230.9 [3,753 / 4,110] Compiling src/core/lib/surface/channel_init.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 231.9 [3,779 / 4,110] Compiling src/core/client_channel/client_channel_service_config.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 233.0 [3,796 / 4,110] Compiling src/core/service_config/service_config_impl.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#14 234.4 [3,804 / 4,110] Compiling src/core/service_config/service_config_impl.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 235.4 [3,808 / 4,110] Compiling src/core/service_config/service_config_impl.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 236.6 [3,814 / 4,110] Compiling src/core/resolver/sockaddr/sockaddr_resolver.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 237.9 [3,818 / 4,110] Compiling src/core/lib/event_engine/default_event_engine.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#14 238.9 [3,822 / 4,110] Compiling src/core/lib/iomgr/event_engine_shims/tcp_client.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 239.9 [3,824 / 4,110] Compiling src/core/server/server_config_selector_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 241.0 [3,827 / 4,110] Compiling src/core/server/server_config_selector_filter.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 242.0 [3,830 / 4,110] Compiling src/core/lib/security/transport/server_auth_filter.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 243.2 [3,832 / 4,110] Compiling src/core/lib/security/transport/server_auth_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 244.3 [3,835 / 4,110] Compiling src/core/lib/security/transport/client_auth_filter.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 244.5 INFO: From Compiling src/python/grpcio_observability/grpc_observability/sampler.cc:
#14 244.5 src/python/grpcio_observability/grpc_observability/sampler.cc:42:44: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#14 244.5 const std::string trace_id_bytes = absl::HexStringToBytes(trace_id);
#14 244.5 ^
#14 244.5 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#14 244.5 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#14 244.5 ^
#14 244.5 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#14 244.5 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#14 244.5 ^
#14 244.5 1 warning generated.
#14 245.3 [3,838 / 4,110] Compiling src/core/lib/security/transport/client_auth_filter.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 246.3 [3,843 / 4,110] Compiling src/core/lib/security/transport/client_auth_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 247.5 [3,849 / 4,110] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 248.8 [3,852 / 4,110] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 249.8 [3,855 / 4,110] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 250.9 [3,859 / 4,110] Compiling src/python/grpcio_observability/grpc_observability/metadata_exchange.cc; 5s processwrapper-sandbox ... (17 actions, 15 running)
#14 251.9 [3,862 / 4,110] Compiling src/python/grpcio_observability/grpc_observability/python_observability_context.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 252.3 INFO: From Compiling src/python/grpcio_observability/grpc_observability/python_observability_context.cc:
#14 252.3 src/python/grpcio_observability/grpc_observability/python_observability_context.cc:93:13: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#14 252.3 absl::HexStringToBytes(absl::string_view(ctx.GetSpanContext().TraceId()));
#14 252.3 ^
#14 252.3 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#14 252.3 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#14 252.3 ^
#14 252.3 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#14 252.3 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#14 252.3 ^
#14 252.3 src/python/grpcio_observability/grpc_observability/python_observability_context.cc:95:13: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#14 252.3 absl::HexStringToBytes(absl::string_view(ctx.GetSpanContext().SpanId()));
#14 252.3 ^
#14 252.3 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#14 252.3 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#14 252.3 ^
#14 252.3 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#14 252.3 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#14 252.3 ^
#14 252.3 2 warnings generated.
#14 252.9 [3,866 / 4,110] Compiling src/core/xds/grpc/xds_listener.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 253.9 [3,869 / 4,110] Compiling src/core/xds/grpc/xds_listener.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 254.9 [3,873 / 4,110] Compiling src/core/ext/filters/stateful_session/stateful_session_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 255.9 [3,877 / 4,110] Compiling src/core/ext/filters/stateful_session/stateful_session_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 256.9 [3,882 / 4,110] Compiling src/core/ext/filters/stateful_session/stateful_session_filter.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 258.1 [3,886 / 4,110] Compiling src/core/ext/filters/stateful_session/stateful_session_filter.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 259.1 [3,890 / 4,110] Compiling src/core/resolver/xds/xds_config.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#14 260.2 [3,892 / 4,110] Compiling src/core/resolver/xds/xds_config.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 261.9 [3,895 / 4,110] Compiling src/core/lib/channel/promise_based_filter.cc; 9s processwrapper-sandbox ... (17 actions, 15 running)
#14 262.9 [3,900 / 4,110] Compiling src/core/lib/channel/promise_based_filter.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#14 263.9 [3,904 / 4,110] Compiling src/core/lib/channel/promise_based_filter.cc; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 264.9 [3,906 / 4,110] Compiling src/core/lib/channel/promise_based_filter.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#14 265.9 [3,909 / 4,110] Compiling src/core/lib/channel/promise_based_filter.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#14 267.2 [3,916 / 4,110] Compiling src/core/lib/surface/client_call.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 268.4 [3,920 / 4,110] Compiling src/core/lib/surface/client_call.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 269.7 [3,921 / 4,110] Compiling src/core/lib/surface/client_call.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 270.9 [3,925 / 4,110] Compiling src/core/lib/surface/client_call.cc; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 272.8 [3,931 / 4,110] Compiling src/core/lib/security/authorization/grpc_server_authz_filter.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 273.9 [3,937 / 4,110] Compiling src/core/lib/security/authorization/grpc_server_authz_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 274.9 [3,940 / 4,110] Compiling src/core/lib/surface/filter_stack_call.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 276.3 [3,941 / 4,110] Compiling src/core/lib/surface/filter_stack_call.cc; 9s processwrapper-sandbox ... (17 actions, 15 running)
#14 277.9 [3,946 / 4,110] Compiling src/core/lib/surface/server_call.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 278.9 [3,948 / 4,110] Compiling src/core/lib/surface/server_call.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 280.3 [3,951 / 4,110] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#14 281.9 [3,956 / 4,110] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 282.9 [3,959 / 4,110] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#14 283.9 [3,961 / 4,110] Compiling src/core/ext/filters/channel_idle/legacy_channel_idle_filter.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 285.4 [3,967 / 4,110] Compiling src/core/lib/security/credentials/oauth2/oauth2_credentials.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 286.6 [3,970 / 4,110] Compiling src/core/resolver/dns/native/dns_resolver.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#14 287.6 [3,972 / 4,110] Compiling src/core/resolver/dns/native/dns_resolver.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 288.9 [3,973 / 4,110] Compiling src/core/resolver/dns/native/dns_resolver.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 290.6 [3,976 / 4,110] Compiling src/core/ext/filters/message_size/message_size_filter.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 291.8 [3,979 / 4,110] Compiling src/core/ext/filters/message_size/message_size_filter.cc; 9s processwrapper-sandbox ... (17 actions, 15 running)
#14 292.9 [3,983 / 4,110] Compiling src/core/ext/filters/http/client/http_client_filter.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 293.9 [3,985 / 4,110] Compiling src/core/client_channel/client_channel.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 294.9 [3,987 / 4,110] Compiling src/core/client_channel/client_channel.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 296.1 [3,989 / 4,110] Compiling src/core/client_channel/client_channel.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 297.9 [3,992 / 4,110] Compiling src/core/client_channel/client_channel.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#14 298.9 [3,994 / 4,110] Compiling src/core/client_channel/client_channel.cc; 14s processwrapper-sandbox ... (17 actions, 16 running)
#14 299.9 [3,997 / 4,110] Compiling src/core/lib/surface/legacy_channel.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 300.9 [4,000 / 4,110] Compiling src/core/ext/filters/http/message_compress/compression_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 301.9 [4,001 / 4,110] Compiling src/core/ext/filters/http/message_compress/compression_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 302.9 [4,004 / 4,110] Compiling src/core/ext/filters/gcp_authentication/gcp_authentication_filter.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 304.0 [4,005 / 4,110] Compiling src/core/ext/filters/gcp_authentication/gcp_authentication_filter.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 305.2 [4,009 / 4,110] Compiling src/core/ext/transport/chttp2/transport/writing.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 306.3 [4,014 / 4,110] Compiling src/core/lib/security/credentials/external/url_external_account_credentials.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#14 307.8 [4,017 / 4,110] Compiling src/core/server/server.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#14 308.9 [4,017 / 4,110] Compiling src/core/server/server.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 309.9 [4,018 / 4,110] Compiling src/core/server/server.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 310.9 [4,019 / 4,110] Compiling src/core/server/server.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 312.3 [4,021 / 4,110] Compiling src/core/server/server.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#14 313.4 [4,025 / 4,110] Compiling src/core/server/server.cc; 13s processwrapper-sandbox ... (17 actions, 15 running)
#14 314.9 [4,031 / 4,110] Compiling src/core/server/server.cc; 14s processwrapper-sandbox ... (16 actions, 15 running)
#14 315.9 [4,032 / 4,110] Compiling src/core/ext/transport/chttp2/transport/chttp2_transport.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 317.9 [4,034 / 4,110] Compiling src/core/ext/transport/chttp2/transport/chttp2_transport.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#14 319.7 [4,036 / 4,110] Compiling src/core/client_channel/client_channel_filter.cc; 12s processwrapper-sandbox ... (17 actions, 15 running)
#14 320.8 [4,040 / 4,110] Compiling src/core/client_channel/client_channel_filter.cc; 13s processwrapper-sandbox ... (16 actions, 15 running)
#14 321.9 [4,043 / 4,110] Compiling src/core/load_balancing/rls/rls.cc; 12s processwrapper-sandbox ... (16 actions, 15 running)
#14 322.9 [4,045 / 4,110] Compiling src/core/load_balancing/rls/rls.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#14 323.9 [4,048 / 4,110] Compiling src/core/load_balancing/priority/priority.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#14 325.3 [4,049 / 4,110] Compiling src/core/load_balancing/priority/priority.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#14 327.0 [4,051 / 4,110] Compiling src/core/load_balancing/grpclb/grpclb.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 328.6 [4,053 / 4,110] Compiling src/core/load_balancing/grpclb/grpclb.cc; 11s processwrapper-sandbox ... (17 actions, 15 running)
#14 329.8 [4,056 / 4,110] Compiling src/core/load_balancing/grpclb/grpclb.cc; 12s processwrapper-sandbox ... (17 actions, 15 running)
#14 331.0 [4,059 / 4,110] Compiling src/core/load_balancing/xds/xds_cluster_manager.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 332.0 [4,061 / 4,110] Compiling src/core/load_balancing/outlier_detection/outlier_detection.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 333.0 [4,062 / 4,110] Compiling src/core/load_balancing/outlier_detection/outlier_detection.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 334.0 [4,066 / 4,110] Compiling src/core/load_balancing/grpclb/client_load_reporting_filter.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#14 336.0 [4,067 / 4,110] Compiling src/core/client_channel/retry_filter_legacy_call_data.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#14 337.1 [4,069 / 4,110] Compiling src/core/client_channel/retry_filter_legacy_call_data.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 338.2 [4,072 / 4,110] Compiling src/core/client_channel/load_balanced_call_destination.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#14 339.7 [4,074 / 4,110] Compiling src/core/client_channel/load_balanced_call_destination.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#14 341.0 [4,075 / 4,110] Compiling src/core/xds/grpc/xds_transport_grpc.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#14 342.7 [4,077 / 4,110] Compiling src/core/xds/grpc/xds_transport_grpc.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#14 344.0 [4,079 / 4,110] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 11s processwrapper-sandbox ... (17 actions, 16 running)
#14 345.4 [4,080 / 4,110] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 12s processwrapper-sandbox ... (16 actions, 15 running)
#14 347.0 [4,080 / 4,110] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 14s processwrapper-sandbox ... (17 actions, 16 running)
#14 348.0 [4,084 / 4,110] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 15s processwrapper-sandbox ... (17 actions, 16 running)
#14 349.0 [4,086 / 4,110] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 16s processwrapper-sandbox ... (17 actions, 16 running)
#14 350.1 [4,091 / 4,110] Compiling src/core/resolver/xds/xds_resolver.cc; 16s processwrapper-sandbox ... (16 actions, 15 running)
#14 351.3 [4,092 / 4,110] Compiling src/core/server/xds_server_config_fetcher.cc; 14s processwrapper-sandbox ... (16 actions, 15 running)
#14 353.0 [4,095 / 4,110] Compiling src/python/grpcio/grpc/_cython/cygrpc.cpp; 15s processwrapper-sandbox ... (14 actions running)
#14 355.0 [4,098 / 4,110] Compiling src/python/grpcio/grpc/_cython/cygrpc.cpp; 17s processwrapper-sandbox ... (11 actions running)
#14 356.2 INFO: From Compiling src/python/grpcio/grpc/_cython/cygrpc.cpp:
#14 356.2 bazel-out/k8-dbg/bin/src/python/grpcio/grpc/_cython/cygrpc.cpp:186186:1: warning: unused function '__Pyx_PyAsyncGen_Fini' [-Wunused-function]
#14 356.2 __Pyx_PyAsyncGen_Fini(void)
#14 356.2 ^
#14 356.2 1 warning generated.
#14 357.0 [4,102 / 4,110] Compiling src/core/xds/grpc/xds_http_fault_filter.cc; 9s processwrapper-sandbox ... (7 actions running)
#14 358.0 [4,105 / 4,110] Compiling src/core/xds/grpc/xds_http_gcp_authn_filter.cc; 10s processwrapper-sandbox ... (4 actions running)
#14 359.0 [4,106 / 4,110] Compiling src/core/xds/grpc/xds_http_rbac_filter.cc; 10s processwrapper-sandbox ... (3 actions running)
#14 361.0 [4,108 / 4,110] Compiling src/core/xds/grpc/xds_route_config_parser.cc; 9s processwrapper-sandbox
#14 362.0 [4,109 / 4,110] Linking src/python/grpcio/grpc/_cython/cygrpc.so; 0s processwrapper-sandbox
#14 367.7 INFO: Found 1 target...
#14 367.7 Target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_client up-to-date:
#14 367.7 bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_client
#14 367.8 INFO: Elapsed time: 367.047s, Critical Path: 83.10s
#14 367.8 INFO: 4110 processes: 1611 internal, 2499 processwrapper-sandbox.
#14 367.8 INFO: Build completed successfully, 4110 total actions
#14 DONE 369.0s

#15 [stage-0 8/10] RUN cp -rL /workdir/bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_client* /artifacts/
#15 DONE 1.6s

#16 [stage-0 9/10] ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
#16 DONE 0.0s

#17 [stage-0 10/10] RUN chmod +x /tini
#17 DONE 0.2s

#18 [stage-1 4/5] COPY --from=0 /artifacts ./
#18 DONE 1.5s

#19 [stage-1 5/5] COPY --from=0 /tini /tini
#19 DONE 0.0s

#20 exporting to image
#20 exporting layers
#20 exporting layers 2.0s done
#20 writing image sha256:5edca09ef48fa9e0278cc427427022b9922f1fdced70da2ae335cf30e902737c done
#20 naming to us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 done
#20 DONE 2.0s

1 warning found (use docker --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 14)
+ [18:13:46 UTC] Cmd finished: docker
+ [18:13:46 UTC] docker push us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
The push refers to repository [us-docker.pkg.dev/grpc-testing/psm-interop/python-client]
1233efbfa2fb: Preparing
43238d137cad: Preparing
cbcff793d232: Preparing
4a63ba1347ba: Preparing
9e599118e168: Preparing
e228adf1886f: Preparing
fb5ccd0db472: Preparing
8d853c8add5d: Preparing
e228adf1886f: Waiting
fb5ccd0db472: Waiting
8d853c8add5d: Waiting
9e599118e168: Layer already exists
e228adf1886f: Layer already exists
fb5ccd0db472: Layer already exists
8d853c8add5d: Layer already exists
cbcff793d232: Pushed
1233efbfa2fb: Pushed
4a63ba1347ba: Pushed
43238d137cad: Pushed
78f6c9348a24f6c49f81cc67867973e14eb00ef4: digest: sha256:7c07151a9b7dbd50e743523a00aeea0d1602797f76e58788ee532f52e177683a size: 1998
+ [18:14:02 UTC] Cmd finished: docker
+ [18:14:02 UTC] Building python xDS interop test server
+ [18:14:02 UTC] docker build -f src/python/grpcio_tests/tests_py3_only/interop/Dockerfile.server -t us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 .
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile.server
#1 transferring dockerfile: 1.17kB done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/python:3.9-slim-bookworm
#2 DONE 0.3s

#3 [internal] load .dockerignore
#3 transferring context: 89B done
#3 DONE 0.0s

#4 [stage-0 1/10] FROM docker.io/library/python:3.9-slim-bookworm@sha256:49f94609e5a997dc16086a66ac9664591854031d48e375945a9dbf4d1d53abbc
#4 CACHED

#5 [internal] load build context
#5 ...

#6 [stage-0 9/10] ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
#6 DONE 0.2s

#5 [internal] load build context
#5 transferring context: 1.17MB 0.7s done
#5 DONE 0.7s

#7 [stage-0 5/10] RUN mkdir /artifacts
#7 CACHED

#8 [stage-0 3/10] WORKDIR /workdir
#8 CACHED

#9 [stage-0 2/10] RUN apt-get update -y && apt-get upgrade -y && apt-get install -y build-essential clang curl
#9 CACHED

#10 [stage-0 4/10] RUN ln -s /usr/bin/python3 /usr/bin/python
#10 CACHED

#11 [stage-0 6/10] COPY . .
#11 CACHED

#6 [stage-0 9/10] ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
#6 CACHED

#12 [stage-1 2/5] RUN apt-get update -y && apt-get install -y python3 && apt-get -y autoremove && rm -rf /var/lib/apt/lists/*
#12 0.233 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#12 0.269 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#12 0.282 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#12 0.365 Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8787 kB]
#12 0.473 Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [2468 B]
#12 0.508 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [188 kB]
#12 1.907 Fetched 9232 kB in 2s (5414 kB/s)
#12 1.907 Reading package lists...
#12 2.615 Reading package lists...
#12 3.357 Building dependency tree...
#12 3.552 Reading state information...
#12 3.833 The following additional packages will be installed:
#12 3.834 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib media-types
#12 3.834 python3-minimal python3.11 python3.11-minimal
#12 3.836 Suggested packages:
#12 3.836 python3-doc python3-tk python3-venv python3.11-venv python3.11-doc binutils
#12 3.836 binfmt-support
#12 3.912 The following NEW packages will be installed:
#12 3.913 libpython3-stdlib libpython3.11-minimal libpython3.11-stdlib media-types
#12 3.913 python3 python3-minimal python3.11 python3.11-minimal
#12 3.963 0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
#12 3.963 Need to get 5340 kB of archives.
#12 3.963 After this operation, 21.8 MB of additional disk space will be used.
#12 3.963 Get:1 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-minimal amd64 3.11.2-6+deb12u3 [816 kB]
#12 4.031 Get:2 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11-minimal amd64 3.11.2-6+deb12u3 [2067 kB]
#12 4.055 Get:3 http://deb.debian.org/debian bookworm/main amd64 python3-minimal amd64 3.11.2-1+b1 [26.3 kB]
#12 4.055 Get:4 http://deb.debian.org/debian bookworm/main amd64 media-types all 10.0.0 [26.1 kB]
#12 4.056 Get:5 http://deb.debian.org/debian-security bookworm-security/main amd64 libpython3.11-stdlib amd64 3.11.2-6+deb12u3 [1797 kB]
#12 4.067 Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 python3.11 amd64 3.11.2-6+deb12u3 [573 kB]
#12 4.070 Get:7 http://deb.debian.org/debian bookworm/main amd64 libpython3-stdlib amd64 3.11.2-1+b1 [9312 B]
#12 4.070 Get:8 http://deb.debian.org/debian bookworm/main amd64 python3 amd64 3.11.2-1+b1 [26.3 kB]
#12 4.555 debconf: delaying package configuration, since apt-utils is not installed
#12 4.592 Fetched 5340 kB in 0s (36.2 MB/s)
#12 4.625 Selecting previously unselected package libpython3.11-minimal:amd64.
#12 4.625 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 6695 files and directories currently installed.)
#12 4.634 Preparing to unpack .../libpython3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#12 4.638 Unpacking libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#12 4.764 Selecting previously unselected package python3.11-minimal.
#12 4.765 Preparing to unpack .../python3.11-minimal_3.11.2-6+deb12u3_amd64.deb ...
#12 4.774 Unpacking python3.11-minimal (3.11.2-6+deb12u3) ...
#12 5.033 Setting up libpython3.11-minimal:amd64 (3.11.2-6+deb12u3) ...
#12 5.049 Setting up python3.11-minimal (3.11.2-6+deb12u3) ...
#12 5.937 Selecting previously unselected package python3-minimal.
#12 5.937 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 7002 files and directories currently installed.)
#12 5.945 Preparing to unpack .../python3-minimal_3.11.2-1+b1_amd64.deb ...
#12 5.949 Unpacking python3-minimal (3.11.2-1+b1) ...
#12 5.988 Selecting previously unselected package media-types.
#12 5.989 Preparing to unpack .../media-types_10.0.0_all.deb ...
#12 5.994 Unpacking media-types (10.0.0) ...
#12 6.032 Selecting previously unselected package libpython3.11-stdlib:amd64.
#12 6.034 Preparing to unpack .../libpython3.11-stdlib_3.11.2-6+deb12u3_amd64.deb ...
#12 6.038 Unpacking libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#12 6.248 Selecting previously unselected package python3.11.
#12 6.249 Preparing to unpack .../python3.11_3.11.2-6+deb12u3_amd64.deb ...
#12 6.253 Unpacking python3.11 (3.11.2-6+deb12u3) ...
#12 6.301 Selecting previously unselected package libpython3-stdlib:amd64.
#12 6.303 Preparing to unpack .../libpython3-stdlib_3.11.2-1+b1_amd64.deb ...
#12 6.306 Unpacking libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#12 6.343 Setting up python3-minimal (3.11.2-1+b1) ...
#12 6.542 Selecting previously unselected package python3.
#12 6.542 (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 7412 files and directories currently installed.)
#12 6.550 Preparing to unpack .../python3_3.11.2-1+b1_amd64.deb ...
#12 6.559 Unpacking python3 (3.11.2-1+b1) ...
#12 6.599 Setting up media-types (10.0.0) ...
#12 6.615 Setting up libpython3.11-stdlib:amd64 (3.11.2-6+deb12u3) ...
#12 6.626 Setting up libpython3-stdlib:amd64 (3.11.2-1+b1) ...
#12 6.637 Setting up python3.11 (3.11.2-6+deb12u3) ...
#12 7.847 Setting up python3 (3.11.2-1+b1) ...
#12 7.869 running python rtupdate hooks for python3.11...
#12 7.870 running python post-rtupdate hooks for python3.11...
#12 8.093 Reading package lists...
#12 8.976 Building dependency tree...
#12 9.164 Reading state information...
#12 9.433 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
#12 ...

#13 [stage-0 7/10] RUN tools/bazel build -c dbg //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server
#13 0.190 INFO: Running bazel wrapper (see //tools/bazel for details), bazel version 7.3.1 will be used instead of system-wide bazel installation.
#13 0.194 Downloading bazel, will try URLs: https://storage.googleapis.com/grpc-bazel-mirror/github.com/bazelbuild/bazel/releases/download/7.3.1/bazel-7.3.1-linux-x86_64 https://github.com/bazelbuild/bazel/releases/download/7.3.1/bazel-7.3.1-linux-x86_64
#13 0.203 % Total % Received % Xferd Average Speed Time Time Time Current
#13 0.203 Dload Upload Total Spent Left Speed
#13 0.204 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 68.2M 100 68.2M 0 0 93.4M 0 --:--:-- --:--:-- --:--:-- 93.5M
#13 0.943 Extracting Bazel installation...
#13 3.809 Starting local Bazel server and connecting to it...
#13 5.950 Computing main repo mapping:
#13 6.954 Computing main repo mapping:
#13 7.955 Computing main repo mapping:
#13 8.955 Computing main repo mapping:
#13 9.956 Computing main repo mapping:
#13 ...

#12 [stage-1 2/5] RUN apt-get update -y && apt-get install -y python3 && apt-get -y autoremove && rm -rf /var/lib/apt/lists/*
#12 DONE 10.9s

#14 [stage-1 3/5] RUN ln -s /usr/bin/python3 /usr/bin/python
#14 DONE 0.2s

#13 [stage-0 7/10] RUN tools/bazel build -c dbg //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server
#13 10.96 Computing main repo mapping:
#13 11.54 Loading:
#13 11.54 Loading: 1 packages loaded
#13 11.79 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (2 packages loaded, 0 targets configured)
#13 11.83 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (2 packages loaded, 0 targets configured)
#13 11.83 [0 / 1] checking cached actions
#13 12.96 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (52 packages loaded, 9 targets configured)
#13 12.96 [1 / 1] checking cached actions
#13 13.96 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (52 packages loaded, 9 targets configured)
#13 13.96 [1 / 1] checking cached actions
#13 16.30 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (52 packages loaded, 9 targets configured)
#13 16.30 [1 / 1] checking cached actions
#13 17.30 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (98 packages loaded, 852 targets configured)
#13 17.30 [1 / 1] checking cached actions
#13 18.96 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (197 packages loaded, 2992 targets configured)
#13 18.96 [1 / 1] checking cached actions
#13 19.98 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (225 packages loaded, 4092 targets configured)
#13 19.98 [1 / 1] checking cached actions
#13 20.97 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (228 packages loaded, 5587 targets configured)
#13 20.97 [1 / 1] checking cached actions
#13 22.06 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (229 packages loaded, 5935 targets configured)
#13 22.06 [1 / 1] checking cached actions
#13 23.33 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (233 packages loaded, 6053 targets configured)
#13 23.33 [1 / 1] checking cached actions
#13 24.71 Analyzing: target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (239 packages loaded, 6230 targets configured)
#13 24.71 [1 / 1] checking cached actions
#13 25.07 INFO: Analyzed target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server (245 packages loaded, 6391 targets configured).
#13 25.71 [1,110 / 3,602] Compiling absl/base/internal/raw_logging.cc [for tool]; 0s processwrapper-sandbox ... (13 actions running)
#13 26.73 [1,575 / 4,120] Compiling absl/numeric/int128.cc [for tool]; 1s processwrapper-sandbox ... (16 actions, 15 running)
#13 27.81 [1,640 / 4,120] Compiling absl/base/internal/thread_identity.cc [for tool]; 0s processwrapper-sandbox ... (17 actions, 15 running)
#13 28.87 [1,661 / 4,120] Compiling absl/synchronization/internal/graphcycles.cc [for tool]; 1s processwrapper-sandbox ... (15 actions, 14 running)
#13 29.91 [1,680 / 4,120] Compiling absl/time/internal/cctz/src/time_zone_format.cc [for tool]; 1s processwrapper-sandbox ... (16 actions, 15 running)
#13 30.93 [1,698 / 4,120] Compiling absl/time/duration.cc [for tool]; 1s processwrapper-sandbox ... (16 actions, 15 running)
#13 31.95 [1,744 / 4,120] Compiling absl/debugging/internal/demangle.cc [for tool]; 1s processwrapper-sandbox ... (16 actions, 15 running)
#13 32.97 [1,761 / 4,120] Compiling absl/debugging/internal/demangle.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 33.98 [1,779 / 4,120] Compiling absl/time/civil_time.cc [for tool]; 1s processwrapper-sandbox ... (17 actions, 16 running)
#13 34.97 [1,794 / 4,120] Compiling absl/strings/escaping.cc [for tool]; 1s processwrapper-sandbox ... (16 actions running)
#13 35.97 [1,809 / 4,120] Compiling absl/strings/numbers.cc [for tool]; 1s processwrapper-sandbox ... (17 actions, 16 running)
#13 36.98 [1,823 / 4,120] Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation; 2s processwrapper-sandbox ... (17 actions, 16 running)
#13 37.98 [1,839 / 4,120] Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation; 3s processwrapper-sandbox ... (17 actions, 16 running)
#13 38.98 [1,857 / 4,120] Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation; 4s processwrapper-sandbox ... (17 actions, 16 running)
#13 39.98 [1,873 / 4,120] Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 40.33 INFO: From Executing genrule //src/python/grpcio_observability/grpc_observability:_cyobservability.pyx_cython_translation:
#13 40.33 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/201/execroot/com_github_grpc_grpc/src/python/grpcio_observability/grpc_observability/_cyobservability.pxd
#13 40.33 tree = Parsing.p_module(s, pxd, full_module_name)
#13 40.99 [1,896 / 4,120] Compiling absl/flags/marshalling.cc [for tool]; 2s processwrapper-sandbox ... (17 actions, 16 running)
#13 41.99 [1,917 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 2s processwrapper-sandbox ... (17 actions, 15 running)
#13 42.31 INFO: From Executing genrule //src/python/grpcio_tests/tests/fork:native_debug.pyx_cython_translation:
#13 42.31 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/297/execroot/com_github_grpc_grpc/src/python/grpcio_tests/tests/fork/native_debug.pyx
#13 42.31 tree = Parsing.p_module(s, pxd, full_module_name)
#13 42.99 [1,938 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 3s processwrapper-sandbox ... (16 actions running)
#13 43.99 [1,949 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 4s processwrapper-sandbox ... (17 actions, 16 running)
#13 44.99 [1,969 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 46.01 [1,985 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 47.01 [2,003 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 48.04 [2,030 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 49.03 [2,065 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 50.10 [2,091 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 11s processwrapper-sandbox ... (17 actions, 15 running)
#13 51.10 [2,117 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 52.18 [2,145 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 53.18 [2,168 / 4,120] Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation; 14s processwrapper-sandbox ... (17 actions, 16 running)
#13 53.86 INFO: From Executing genrule //src/python/grpcio/grpc/_cython:cygrpc.pyx_cython_translation:
#13 53.86 /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/external/cython/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /root/.cache/bazel/_bazel_root/ec321eb2cc2d0f8f91b676b6d4c66c29/sandbox/processwrapper-sandbox/271/execroot/com_github_grpc_grpc/src/python/grpcio/grpc/_cython/cygrpc.pxd
#13 53.86 tree = Parsing.p_module(s, pxd, full_module_name)
#13 53.86 warning: src/python/grpcio/grpc/_cython/cygrpc.pxd:40:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#13 53.86 warning: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pxd.pxi:19:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#13 53.86 warning: src/python/grpcio/grpc/_cython/cygrpc.pyx:64:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#13 53.86 warning: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:24:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pxd.pxi:24:22: No exception value declared for '_get_metadata' in pxd file.
#13 53.86 Users cimporting this function and calling it without the gil will always require an exception check.
#13 53.86 Suggest adding an explicit exception value.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi:80:5: Exception check on '_destroy' will always require the GIL to be acquired.
#13 53.86 Possible solutions:
#13 53.86 1. Declare '_destroy' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 2. Use an 'int' return type on '_destroy' to allow an error code to be returned.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi:104:5: Exception check on '_on_shutdown' will always require the GIL to be acquired.
#13 53.86 Possible solutions:
#13 53.86 1. Declare '_on_shutdown' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 2. Use an 'int' return type on '_on_shutdown' to allow an error code to be returned.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi:21:5: Exception check on '_copy_slice' will always require the GIL to be acquired. Declare '_copy_slice' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi:26:5: Exception check on '_slice_from_bytes' will always require the GIL to be acquired. Declare '_slice_from_bytes' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/security.pyx.pxi:20:5: Exception check on 'ssl_roots_override_callback' will always require the GIL to be acquired. Declare 'ssl_roots_override_callback' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:30:9: Exception check on '_unified_socket_write' will always require the GIL to be acquired.
#13 53.86 Possible solutions:
#13 53.86 1. Declare '_unified_socket_write' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 2. Use an 'int' return type on '_unified_socket_write' to allow an error code to be returned.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:116:41: Exception check after calling '_unified_socket_write' will always require the GIL to be acquired.
#13 53.86 Possible solutions:
#13 53.86 1. Declare '_unified_socket_write' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 2. Use an 'int' return type on '_unified_socket_write' to allow an error code to be returned.
#13 53.86 performance hint: src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi:126:22: Exception check after calling '_poll' will always require the GIL to be acquired.
#13 53.86 Possible solutions:
#13 53.86 1. Declare '_poll' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
#13 53.86 2. Use an 'int' return type on '_poll' to allow an error code to be returned.
#13 54.19 [2,188 / 4,120] Compiling absl/strings/cord.cc [for tool]; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 55.21 [2,217 / 4,120] Compiling upb_generator/file_layout.cc [for tool]; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 56.31 [2,250 / 4,120] Compiling absl/flags/reflection.cc [for tool]; 3s processwrapper-sandbox ... (17 actions, 15 running)
#13 57.30 [2,272 / 4,120] Compiling absl/flags/reflection.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 16 running)
#13 58.32 [2,298 / 4,120] Compiling absl/status/internal/status_internal.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 59.42 [2,327 / 4,120] Compiling absl/flags/parse.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 60.47 [2,368 / 4,120] Compiling absl/flags/parse.cc [for tool]; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 61.48 [2,410 / 4,120] Compiling absl/flags/parse.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 15 running)
#13 62.50 [2,445 / 4,120] Compiling src/core/lib/event_engine/posix_engine/native_posix_dns_resolver.cc; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 63.51 [2,494 / 4,120] Compiling src/core/xds/xds_client/xds_backend_metric_propagation.cc; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 64.85 [2,503 / 4,120] Compiling src/core/tsi/fake_transport_security.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#13 65.85 [2,515 / 4,120] Compiling src/crypto/fipsmodule/bcm.c; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 66.97 [2,525 / 4,120] Compiling src/google/protobuf/json/internal/writer.cc [for tool]; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 67.97 [2,530 / 4,120] Compiling src/core/lib/slice/slice.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#13 69.00 [2,539 / 4,120] Compiling src/core/ext/transport/chttp2/transport/ping_callbacks.cc; 3s processwrapper-sandbox ... (17 actions, 16 running)
#13 70.00 [2,549 / 4,120] Compiling src/google/protobuf/io/printer.cc [for tool]; 2s processwrapper-sandbox ... (17 actions, 16 running)
#13 71.00 [2,554 / 4,120] Compiling src/google/protobuf/io/printer.cc [for tool]; 3s processwrapper-sandbox ... (17 actions, 16 running)
#13 72.13 [2,565 / 4,120] Compiling src/google/protobuf/io/printer.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 15 running)
#13 73.26 [2,575 / 4,120] Compiling src/google/protobuf/io/printer.cc [for tool]; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 74.47 [2,584 / 4,120] Compiling src/google/protobuf/io/printer.cc [for tool]; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 75.49 [2,594 / 4,120] Compiling src/google/protobuf/extension_set.cc [for tool]; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 76.57 [2,602 / 4,120] Compiling src/google/protobuf/extension_set.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 15 running)
#13 77.59 [2,609 / 4,120] Compiling src/google/protobuf/extension_set.cc [for tool]; 6s processwrapper-sandbox ... (16 actions, 15 running)
#13 78.70 [2,613 / 4,120] Compiling src/google/protobuf/extension_set.cc [for tool]; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 79.88 [2,622 / 4,120] Compiling src/google/protobuf/extension_set.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 15 running)
#13 80.91 [2,629 / 4,120] Compiling src/google/protobuf/generated_message_tctable_lite.cc [for tool]; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 82.00 [2,632 / 4,120] Compiling src/google/protobuf/generated_message_tctable_lite.cc [for tool]; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 83.00 [2,638 / 4,120] Compiling src/google/protobuf/generated_message_tctable_lite.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#13 84.00 [2,647 / 4,120] Compiling src/google/protobuf/compiler/retention.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 16 running)
#13 85.06 [2,648 / 4,120] Compiling src/google/protobuf/compiler/retention.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 15 running)
#13 86.09 [2,655 / 4,120] Compiling src/google/protobuf/compiler/retention.cc [for tool]; 6s processwrapper-sandbox ... (16 actions, 15 running)
#13 87.38 [2,661 / 4,120] Compiling src/google/protobuf/io/printer.cc; 4s processwrapper-sandbox ... (17 actions, 15 running)
#13 88.44 [2,667 / 4,120] Compiling src/google/protobuf/io/printer.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 89.70 [2,673 / 4,120] Compiling src/google/protobuf/compiler/python/pyi_generator.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 15 running)
#13 91.00 [2,675 / 4,120] Compiling src/google/protobuf/compiler/python/pyi_generator.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 92.01 [2,681 / 4,120] Compiling src/google/protobuf/compiler/python/pyi_generator.cc [for tool]; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 93.12 [2,691 / 4,120] Compiling src/google/protobuf/compiler/python/pyi_generator.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 15 running)
#13 94.27 [2,694 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 95.43 [2,696 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 96.43 [2,699 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 97.60 [2,702 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 98.61 [2,706 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 99.63 [2,709 / 4,120] Compiling src/google/protobuf/text_format.cc [for tool]; 11s processwrapper-sandbox ... (16 actions, 15 running)
#13 100.8 [2,714 / 4,120] Compiling src/google/protobuf/compiler/python/generator.cc [for tool]; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 102.0 [2,719 / 4,120] Compiling src/google/protobuf/compiler/python/generator.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#13 103.0 [2,720 / 4,120] Compiling src/google/protobuf/compiler/python/generator.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 16 running)
#13 104.2 [2,723 / 4,120] Compiling src/google/protobuf/compiler/python/generator.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 105.5 [2,725 / 4,120] Compiling src/google/protobuf/compiler/java/helpers.cc [for tool]; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 107.0 [2,726 / 4,120] Compiling src/google/protobuf/compiler/java/helpers.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 108.0 [2,731 / 4,120] Compiling src/google/protobuf/util/message_differencer.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 109.0 [2,735 / 4,120] Compiling src/google/protobuf/util/message_differencer.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 110.8 [2,740 / 4,120] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 111.9 [2,743 / 4,120] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 113.0 [2,744 / 4,120] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 15 running)
#13 114.0 [2,746 / 4,120] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 14s processwrapper-sandbox ... (17 actions, 16 running)
#13 115.0 [2,748 / 4,120] Compiling src/google/protobuf/compiler/cpp/helpers.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 16 running)
#13 116.8 [2,752 / 4,120] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 15s processwrapper-sandbox ... (16 actions, 15 running)
#13 117.9 [2,755 / 4,120] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 15 running)
#13 119.0 [2,761 / 4,120] Compiling src/google/protobuf/compiler/cpp/enum.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 16 running)
#13 120.3 [2,764 / 4,120] Compiling src/google/protobuf/compiler/cpp/enum.cc [for tool]; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 121.7 [2,766 / 4,120] Compiling src/google/protobuf/compiler/php/php_generator.cc [for tool]; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 123.0 [2,768 / 4,120] Compiling src/google/protobuf/compiler/php/php_generator.cc [for tool]; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 124.0 [2,770 / 4,120] Compiling src/google/protobuf/compiler/php/php_generator.cc [for tool]; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 125.0 [2,772 / 4,120] Compiling src/google/protobuf/compiler/php/php_generator.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 16 running)
#13 126.5 [2,774 / 4,120] Compiling src/google/protobuf/compiler/php/php_generator.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 128.0 [2,778 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 129.0 [2,780 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 11s processwrapper-sandbox ... (17 actions, 16 running)
#13 130.0 [2,782 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 12s processwrapper-sandbox ... (17 actions, 16 running)
#13 131.0 [2,784 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 16 running)
#13 132.0 [2,786 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 14s processwrapper-sandbox ... (17 actions, 16 running)
#13 133.0 [2,787 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 16 running)
#13 134.0 [2,791 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 16 running)
#13 135.0 [2,792 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 17s processwrapper-sandbox ... (17 actions, 16 running)
#13 136.0 [2,793 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 18s processwrapper-sandbox ... (17 actions, 16 running)
#13 138.3 [2,794 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 20s processwrapper-sandbox ... (17 actions, 15 running)
#13 140.0 [2,797 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 22s processwrapper-sandbox ... (17 actions, 15 running)
#13 141.0 [2,804 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 23s processwrapper-sandbox ... (17 actions, 16 running)
#13 142.0 [2,805 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 24s processwrapper-sandbox ... (17 actions, 16 running)
#13 143.0 [2,806 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 25s processwrapper-sandbox ... (17 actions, 16 running)
#13 145.0 [2,808 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 27s processwrapper-sandbox ... (17 actions, 16 running)
#13 146.0 [2,810 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 28s processwrapper-sandbox ... (17 actions, 16 running)
#13 147.1 [2,812 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 29s processwrapper-sandbox ... (17 actions, 16 running)
#13 148.1 [2,815 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 30s processwrapper-sandbox ... (17 actions, 16 running)
#13 149.1 [2,816 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 31s processwrapper-sandbox ... (17 actions, 16 running)
#13 150.2 [2,818 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 32s processwrapper-sandbox ... (16 actions, 15 running)
#13 151.2 [2,821 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 33s processwrapper-sandbox ... (17 actions, 15 running)
#13 152.3 [2,824 / 4,120] Compiling src/google/protobuf/descriptor.cc [for tool]; 34s processwrapper-sandbox ... (16 actions, 15 running)
#13 153.7 [2,826 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 15 running)
#13 154.8 [2,829 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 14s processwrapper-sandbox ... (16 actions, 15 running)
#13 156.1 [2,831 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 16 running)
#13 157.1 [2,833 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 16s processwrapper-sandbox ... (17 actions, 16 running)
#13 158.8 [2,836 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 18s processwrapper-sandbox ... (17 actions, 15 running)
#13 160.1 [2,838 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 19s processwrapper-sandbox ... (17 actions, 16 running)
#13 161.1 [2,839 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 20s processwrapper-sandbox ... (17 actions, 16 running)
#13 162.1 [2,840 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 21s processwrapper-sandbox ... (16 actions, 15 running)
#13 163.5 [2,844 / 4,120] Compiling src/google/protobuf/compiler/cpp/file.cc [for tool]; 23s processwrapper-sandbox ... (17 actions, 15 running)
#13 164.6 [2,850 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 13s processwrapper-sandbox ... (17 actions, 15 running)
#13 166.0 [2,853 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 15s processwrapper-sandbox ... (17 actions, 15 running)
#13 167.1 [2,856 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 16s processwrapper-sandbox ... (16 actions running)
#13 168.1 [2,859 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 17s processwrapper-sandbox ... (16 actions, 15 running)
#13 170.1 [2,861 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 19s processwrapper-sandbox ... (17 actions, 16 running)
#13 171.1 [2,863 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 20s processwrapper-sandbox ... (17 actions, 16 running)
#13 172.1 [2,868 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 21s processwrapper-sandbox ... (17 actions, 16 running)
#13 173.3 [2,871 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 22s processwrapper-sandbox ... (17 actions, 15 running)
#13 174.7 [2,873 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 23s processwrapper-sandbox ... (16 actions, 15 running)
#13 176.1 [2,876 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 25s processwrapper-sandbox ... (14 actions running)
#13 177.1 [2,877 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 26s processwrapper-sandbox ... (13 actions running)
#13 178.1 [2,880 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 27s processwrapper-sandbox ... (10 actions running)
#13 180.1 [2,887 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 29s processwrapper-sandbox ... (3 actions running)
#13 181.1 [2,888 / 4,120] Compiling src/google/protobuf/compiler/cpp/message.cc [for tool]; 30s processwrapper-sandbox ... (2 actions running)
#13 182.1 [2,889 / 4,120] Compiling src/google/protobuf/compiler/java/full/message.cc [for tool]; 7s processwrapper-sandbox
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/field_mask_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/timestamp_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/api_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/type_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/source_context_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/struct_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/compiler/plugin_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/wrappers_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/any_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/duration_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/empty_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 182.8 INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/descriptor_pb2.py:
#13 182.8 external/com_google_protobuf/.: warning: directory does not exist.
#13 184.1 [2,929 / 4,120] Compiling external/com_google_protobuf/src/google/protobuf/_virtual_imports/duration_proto/google/protobuf/duration.pb.cc [for tool]; 1s processwrapper-sandbox ... (17 actions, 16 running)
#13 186.8 [2,930 / 4,120] Compiling external/com_google_protobuf/src/google/protobuf/_virtual_imports/duration_proto/google/protobuf/duration.pb.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 15 running)
#13 188.1 [2,938 / 4,120] Compiling src/google/protobuf/util/field_mask_util.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 189.1 [2,941 / 4,120] Compiling src/google/protobuf/util/field_mask_util.cc [for tool]; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 191.1 [2,942 / 4,120] Compiling src/google/protobuf/json/internal/untyped_message.cc [for tool]; 8s processwrapper-sandbox ... (16 actions running)
#13 192.1 [2,944 / 4,120] Compiling src/google/protobuf/json/internal/untyped_message.cc [for tool]; 9s processwrapper-sandbox ... (14 actions running)
#13 194.1 [2,948 / 4,120] Compiling src/google/protobuf/json/internal/untyped_message.cc [for tool]; 11s processwrapper-sandbox ... (10 actions running)
#13 195.1 [2,954 / 4,120] Compiling src/compiler/csharp_generator.cc [for tool]; 7s processwrapper-sandbox ... (8 actions, 6 running)
#13 196.1 [2,971 / 4,120] Compiling src/compiler/node_generator.cc [for tool]; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 198.1 [2,983 / 4,120] Compiling src/compiler/python_generator.cc [for tool]; 8s processwrapper-sandbox ... (6 actions running)
#13 199.1 [2,986 / 4,120] Compiling src/compiler/python_generator.cc [for tool]; 9s processwrapper-sandbox ... (3 actions running)
#13 200.1 [3,057 / 4,120] Compiling upb_generator/protoc-gen-upb.cc [for tool]; 4s processwrapper-sandbox ... (16 actions running)
#13 201.1 [3,150 / 4,120] Compiling upb_generator/protoc-gen-upb.cc [for tool]; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 202.2 [3,233 / 4,120] [Prepa] Generating upb protos ... (16 actions, 0 running)
#13 203.2 [3,368 / 4,120] Compiling src/core/lib/event_engine/posix_engine/lockfree_event.cc; 0s processwrapper-sandbox ... (16 actions, 15 running)
#13 204.2 [3,409 / 4,120] Compiling src/core/lib/event_engine/posix_engine/lockfree_event.cc; 1s processwrapper-sandbox ... (17 actions, 15 running)
#13 205.2 [3,432 / 4,120] Compiling src/core/lib/event_engine/posix_engine/lockfree_event.cc; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 206.3 [3,454 / 4,120] Compiling upb_generator/file_layout.cc [for tool]; 3s processwrapper-sandbox ... (17 actions, 15 running)
#13 207.3 [3,471 / 4,120] Compiling src/core/xds/grpc/xds_health_status.cc; 3s processwrapper-sandbox ... (17 actions, 15 running)
#13 208.3 [3,511 / 4,120] Compiling src/core/lib/event_engine/posix_engine/ev_epoll1_linux.cc; 2s processwrapper-sandbox ... (17 actions, 15 running)
#13 209.6 [3,540 / 4,120] Compiling upb_generator/protoc-gen-upbdefs.cc [for tool]; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 211.1 [3,544 / 4,120] Compiling upb_generator/protoc-gen-upbdefs.cc [for tool]; 4s processwrapper-sandbox ... (17 actions, 16 running)
#13 212.3 [3,549 / 4,120] Compiling src/core/lib/resource_quota/memory_quota.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 213.3 [3,561 / 4,120] Compiling src/core/lib/resource_quota/memory_quota.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#13 214.4 [3,579 / 4,120] Compiling src/core/lib/event_engine/ares_resolver.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 215.5 [3,588 / 4,120] Compiling src/core/lib/event_engine/ares_resolver.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#13 216.5 [3,604 / 4,120] Compiling src/core/lib/event_engine/posix_engine/posix_engine.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#13 217.7 [3,621 / 4,120] Compiling src/core/lib/transport/metadata.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#13 218.7 [3,632 / 4,120] Compiling src/core/lib/transport/metadata.cc; 5s processwrapper-sandbox ... (17 actions, 15 running)
#13 219.8 [3,648 / 4,120] Compiling src/core/tsi/alts/zero_copy_frame_protector/alts_grpc_record_protocol_common.cc; 2s processwrapper-sandbox ... (16 actions, 15 running)
#13 221.0 [3,662 / 4,120] Compiling src/core/lib/transport/call_filters.cc; 3s processwrapper-sandbox ... (17 actions, 15 running)
#13 222.1 [3,671 / 4,120] Compiling src/core/lib/transport/call_filters.cc; 4s processwrapper-sandbox ... (16 actions, 15 running)
#13 223.1 [3,679 / 4,120] Compiling src/core/lib/transport/call_filters.cc; 5s processwrapper-sandbox ... (16 actions, 15 running)
#13 224.1 [3,688 / 4,120] Compiling src/core/lib/transport/call_filters.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 225.1 [3,697 / 4,120] Compiling src/core/lib/transport/call_spine.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 226.2 [3,706 / 4,120] Compiling src/core/lib/transport/call_spine.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 227.2 [3,716 / 4,120] Compiling src/core/lib/surface/channel.cc; 6s processwrapper-sandbox ... (16 actions, 15 running)
#13 228.4 [3,735 / 4,120] Compiling src/core/lib/resource_quota/api.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 229.4 [3,750 / 4,120] Compiling src/core/lib/transport/interception_chain.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 230.4 [3,763 / 4,120] Compiling src/core/lib/iomgr/event_engine_shims/tcp_client.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 231.5 [3,773 / 4,120] Compiling src/core/lib/iomgr/event_engine_shims/tcp_client.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 232.6 [3,793 / 4,120] Compiling src/core/lib/iomgr/endpoint_pair_posix.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 233.6 [3,805 / 4,120] Compiling src/core/resolver/fake/fake_resolver.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 234.7 [3,818 / 4,120] Compiling src/core/lib/config/core_configuration.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 235.7 [3,835 / 4,120] Compiling src/core/resolver/binder/binder_resolver.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 237.0 [3,840 / 4,120] Compiling src/core/lib/iomgr/resolve_address_posix.cc; 3s processwrapper-sandbox ... (16 actions, 15 running)
#13 238.1 [3,844 / 4,120] Compiling src/core/lib/iomgr/resolve_address_posix.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 239.1 [3,858 / 4,120] Compiling src/core/lib/iomgr/resolve_address_posix.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 240.7 [3,865 / 4,120] Compiling src/core/service_config/service_config_impl.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 241.8 [3,871 / 4,120] Compiling src/core/service_config/service_config_impl.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 243.0 [3,882 / 4,120] Compiling src/core/lib/iomgr/tcp_posix.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 244.1 [3,884 / 4,120] Compiling src/core/lib/iomgr/tcp_posix.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 245.1 [3,885 / 4,120] Compiling src/core/server/server_call_tracer_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 246.1 [3,887 / 4,120] Compiling src/core/lib/iomgr/tcp_server_posix.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 247.1 [3,890 / 4,120] Compiling src/core/handshaker/http_connect/http_proxy_mapper.cc; 5s processwrapper-sandbox ... (17 actions, 16 running)
#13 248.9 [3,891 / 4,120] Compiling src/core/handshaker/http_connect/http_proxy_mapper.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 250.1 [3,894 / 4,120] Compiling src/core/xds/xds_client/xds_client.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 251.1 [3,895 / 4,120] Compiling src/core/xds/xds_client/xds_client.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 252.7 [3,899 / 4,120] Compiling src/core/lib/security/transport/server_auth_filter.cc; 10s processwrapper-sandbox ... (16 actions, 15 running)
#13 253.8 [3,903 / 4,120] Compiling src/core/lib/security/transport/client_auth_filter.cc; 9s processwrapper-sandbox ... (17 actions, 15 running)
#13 255.1 [3,907 / 4,120] Compiling src/core/util/gcp_metadata_query.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 256.8 [3,910 / 4,120] Compiling src/core/handshaker/http_connect/http_connect_handshaker.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 257.8 [3,912 / 4,120] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 258.9 [3,913 / 4,120] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 260.1 [3,916 / 4,120] Compiling src/core/lib/security/credentials/token_fetcher/token_fetcher_credentials.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 261.1 [3,918 / 4,120] Compiling src/core/xds/xds_client/lrs_client.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 262.1 [3,925 / 4,120] Compiling src/core/xds/grpc/xds_server_grpc.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 263.1 [3,927 / 4,120] Compiling src/core/lib/security/context/security_context.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 264.1 [3,928 / 4,120] Compiling src/core/xds/grpc/xds_route_config.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 265.1 [3,932 / 4,120] Compiling src/core/xds/grpc/xds_route_config.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 266.1 [3,933 / 4,120] Compiling src/core/handshaker/security/security_handshaker.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 267.1 [3,935 / 4,120] Compiling src/core/handshaker/security/secure_endpoint.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 268.1 [3,936 / 4,120] Compiling src/core/resolver/dns/native/dns_resolver.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 269.4 [3,940 / 4,120] Compiling src/core/lib/security/security_connector/ssl/ssl_security_connector.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 270.5 [3,944 / 4,120] Compiling src/core/ext/filters/http/server/http_server_filter.cc; 7s processwrapper-sandbox ... (16 actions, 15 running)
#13 272.1 [3,945 / 4,120] Compiling src/core/ext/filters/http/server/http_server_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 273.1 [3,948 / 4,120] Compiling src/core/ext/filters/message_size/message_size_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 274.1 [3,951 / 4,120] Compiling src/core/ext/filters/message_size/message_size_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 275.1 [3,955 / 4,120] Compiling src/core/ext/transport/chttp2/transport/writing.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 276.3 [3,957 / 4,120] Compiling src/core/ext/transport/chttp2/transport/writing.cc; 9s processwrapper-sandbox ... (17 actions, 15 running)
#13 278.1 [3,959 / 4,120] Compiling src/core/ext/filters/http/client/http_client_filter.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 279.1 [3,963 / 4,120] Compiling src/core/ext/filters/http/http_filters_plugin.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 281.0 [3,968 / 4,120] Compiling src/core/ext/filters/fault_injection/fault_injection_filter.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 282.1 [3,971 / 4,120] Compiling src/core/server/server_config_selector_filter.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 283.6 [3,973 / 4,120] Compiling src/core/server/xds_channel_stack_modifier.cc; 6s processwrapper-sandbox ... (17 actions, 15 running)
#13 285.1 [3,976 / 4,120] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 286.2 [3,977 / 4,120] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 287.5 [3,978 / 4,120] Compiling src/core/lib/security/credentials/tls/grpc_tls_crl_provider.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 288.7 INFO: From Compiling src/python/grpcio_observability/grpc_observability/sampler.cc:
#13 288.7 src/python/grpcio_observability/grpc_observability/sampler.cc:42:44: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#13 288.7 const std::string trace_id_bytes = absl::HexStringToBytes(trace_id);
#13 288.7 ^
#13 288.7 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#13 288.7 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#13 288.7 ^
#13 288.7 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#13 288.7 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#13 288.7 ^
#13 288.7 1 warning generated.
#13 288.7 [3,984 / 4,120] Compiling src/core/load_balancing/xds/xds_cluster_manager.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 290.2 [3,989 / 4,120] Compiling src/core/ext/transport/chttp2/transport/chttp2_transport.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 291.3 [3,990 / 4,120] Compiling src/core/ext/transport/chttp2/transport/chttp2_transport.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 293.2 [3,994 / 4,120] Compiling src/core/ext/transport/chttp2/transport/chttp2_transport.cc; 11s processwrapper-sandbox ... (17 actions, 16 running)
#13 294.2 [3,997 / 4,120] Compiling src/core/ext/transport/chttp2/transport/frame_data.cc; 6s processwrapper-sandbox ... (17 actions, 16 running)
#13 296.1 INFO: From Compiling src/python/grpcio_observability/grpc_observability/python_observability_context.cc:
#13 296.1 src/python/grpcio_observability/grpc_observability/python_observability_context.cc:93:13: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#13 296.1 absl::HexStringToBytes(absl::string_view(ctx.GetSpanContext().TraceId()));
#13 296.1 ^
#13 296.1 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#13 296.1 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#13 296.1 ^
#13 296.1 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#13 296.1 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#13 296.1 ^
#13 296.1 src/python/grpcio_observability/grpc_observability/python_observability_context.cc:95:13: warning: 'HexStringToBytes' is deprecated: Use the HexStringToBytes() that returns a bool [-Wdeprecated-declarations]
#13 296.1 absl::HexStringToBytes(absl::string_view(ctx.GetSpanContext().SpanId()));
#13 296.1 ^
#13 296.1 external/com_google_absl/absl/strings/escaping.h:174:1: note: 'HexStringToBytes' has been explicitly marked deprecated here
#13 296.1 ABSL_DEPRECATED("Use the HexStringToBytes() that returns a bool")
#13 296.1 ^
#13 296.1 external/com_google_absl/absl/base/attributes.h:686:49: note: expanded from macro 'ABSL_DEPRECATED'
#13 296.1 #define ABSL_DEPRECATED(message) __attribute__((deprecated(message)))
#13 296.1 ^
#13 296.1 2 warnings generated.
#13 296.1 [4,001 / 4,120] Compiling src/core/client_channel/client_channel.cc; 8s processwrapper-sandbox ... (16 actions, 15 running)
#13 297.2 [4,002 / 4,120] Compiling src/core/client_channel/client_channel.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 298.5 [4,004 / 4,120] Compiling src/core/client_channel/client_channel.cc; 10s processwrapper-sandbox ... (15 actions, 14 running)
#13 299.5 [4,009 / 4,120] Compiling src/core/client_channel/client_channel.cc; 11s processwrapper-sandbox ... (17 actions, 15 running)
#13 301.0 [4,011 / 4,120] Compiling src/core/client_channel/client_channel.cc; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 302.0 [4,014 / 4,120] Compiling src/core/client_channel/client_channel.cc; 14s processwrapper-sandbox ... (17 actions, 15 running)
#13 303.1 [4,019 / 4,120] Compiling src/core/lib/channel/promise_based_filter.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 304.2 [4,019 / 4,120] Compiling src/core/lib/channel/promise_based_filter.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#13 305.4 [4,022 / 4,120] Compiling src/core/lib/channel/promise_based_filter.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#13 307.2 [4,026 / 4,120] Compiling src/core/lib/channel/promise_based_filter.cc; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 308.2 [4,027 / 4,120] Compiling src/core/load_balancing/health_check_client.cc; 8s processwrapper-sandbox ... (17 actions, 16 running)
#13 309.2 [4,029 / 4,120] Compiling src/core/load_balancing/outlier_detection/outlier_detection.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 310.3 [4,031 / 4,120] Compiling src/core/load_balancing/outlier_detection/outlier_detection.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 311.4 [4,034 / 4,120] Compiling src/core/load_balancing/outlier_detection/outlier_detection.cc; 10s processwrapper-sandbox ... (17 actions, 15 running)
#13 313.2 [4,036 / 4,120] Compiling src/core/lib/surface/client_call.cc; 10s processwrapper-sandbox ... (17 actions, 16 running)
#13 314.5 [4,042 / 4,120] Compiling src/core/load_balancing/round_robin/round_robin.cc; 7s processwrapper-sandbox ... (17 actions, 15 running)
#13 315.7 [4,044 / 4,120] Compiling src/core/lib/security/credentials/external/aws_external_account_credentials.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#13 316.8 [4,047 / 4,120] Compiling src/core/lib/security/credentials/external/aws_external_account_credentials.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 318.2 [4,049 / 4,120] Compiling src/core/client_channel/client_channel_filter.cc; 7s processwrapper-sandbox ... (17 actions, 16 running)
#13 319.8 [4,050 / 4,120] Compiling src/core/client_channel/client_channel_filter.cc; 9s processwrapper-sandbox ... (16 actions, 15 running)
#13 321.0 [4,051 / 4,120] Compiling src/core/client_channel/client_channel_filter.cc; 10s processwrapper-sandbox ... (16 actions, 15 running)
#13 322.1 [4,058 / 4,120] Compiling src/core/client_channel/client_channel_filter.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#13 323.2 [4,059 / 4,120] Compiling src/core/client_channel/client_channel_filter.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#13 324.8 [4,060 / 4,120] Compiling src/core/server/server.cc; 11s processwrapper-sandbox ... (17 actions, 15 running)
#13 326.2 [4,063 / 4,120] Compiling src/core/server/server.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#13 327.5 [4,064 / 4,120] Compiling src/core/server/server.cc; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 328.8 [4,067 / 4,120] Compiling src/core/load_balancing/grpclb/grpclb.cc; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 330.1 [4,068 / 4,120] Compiling src/core/lib/security/credentials/external/external_account_credentials.cc; 8s processwrapper-sandbox ... (17 actions, 15 running)
#13 331.2 [4,071 / 4,120] Compiling src/core/load_balancing/rls/rls.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 332.7 [4,074 / 4,120] Compiling src/core/load_balancing/rls/rls.cc; 11s processwrapper-sandbox ... (16 actions, 15 running)
#13 334.1 [4,077 / 4,120] Compiling src/core/load_balancing/rls/rls.cc; 12s processwrapper-sandbox ... (17 actions, 15 running)
#13 335.2 [4,078 / 4,120] Compiling src/core/load_balancing/rls/rls.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#13 336.2 [4,081 / 4,120] Compiling src/core/load_balancing/xds/xds_cluster_impl.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#13 337.8 [4,084 / 4,120] Compiling src/core/client_channel/load_balanced_call_destination.cc; 10s processwrapper-sandbox ... (16 actions, 15 running)
#13 339.2 [4,087 / 4,120] Compiling src/core/load_balancing/xds/cds.cc; 9s processwrapper-sandbox ... (17 actions, 16 running)
#13 340.6 [4,089 / 4,120] Compiling src/core/load_balancing/xds/cds.cc; 10s processwrapper-sandbox ... (16 actions, 15 running)
#13 342.2 [4,090 / 4,120] Compiling src/core/load_balancing/xds/cds.cc; 12s processwrapper-sandbox ... (17 actions, 16 running)
#13 343.2 [4,091 / 4,120] Compiling src/core/load_balancing/xds/cds.cc; 13s processwrapper-sandbox ... (17 actions, 16 running)
#13 344.9 [4,092 / 4,120] Compiling src/core/load_balancing/xds/cds.cc; 14s processwrapper-sandbox ... (16 actions, 15 running)
#13 345.9 [4,095 / 4,120] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 13s processwrapper-sandbox ... (16 actions, 15 running)
#13 347.2 [4,096 / 4,120] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 14s processwrapper-sandbox ... (17 actions, 16 running)
#13 348.2 [4,098 / 4,120] Compiling src/core/resolver/xds/xds_dependency_manager.cc; 15s processwrapper-sandbox ... (17 actions, 16 running)
#13 349.3 [4,100 / 4,120] Compiling src/core/resolver/xds/xds_resolver.cc; 15s processwrapper-sandbox ... (16 actions, 15 running)
#13 351.2 [4,104 / 4,120] Compiling src/core/resolver/xds/xds_resolver.cc; 17s processwrapper-sandbox ... (12 actions running)
#13 353.2 [4,106 / 4,120] Compiling src/core/xds/grpc/xds_listener_parser.cc; 12s processwrapper-sandbox ... (10 actions running)
#13 354.2 [4,108 / 4,120] Compiling src/core/xds/grpc/xds_route_config_parser.cc; 9s processwrapper-sandbox ... (8 actions running)
#13 355.4 [4,111 / 4,120] Compiling src/core/xds/grpc/xds_route_config_parser.cc; 10s processwrapper-sandbox ... (6 actions, 5 running)
#13 357.2 [4,117 / 4,120] Compiling src/core/server/xds_server_config_fetcher.cc; 11s processwrapper-sandbox ... (2 actions running)
#13 359.2 [4,118 / 4,120] Compiling src/python/grpcio/grpc/_cython/cygrpc.cpp; 10s processwrapper-sandbox
#13 361.8 INFO: From Compiling src/python/grpcio/grpc/_cython/cygrpc.cpp:
#13 361.8 bazel-out/k8-dbg/bin/src/python/grpcio/grpc/_cython/cygrpc.cpp:186186:1: warning: unused function '__Pyx_PyAsyncGen_Fini' [-Wunused-function]
#13 361.8 __Pyx_PyAsyncGen_Fini(void)
#13 361.8 ^
#13 361.8 1 warning generated.
#13 361.8 [4,119 / 4,120] [Prepa] Linking src/python/grpcio/grpc/_cython/cygrpc.so
#13 363.2 [4,119 / 4,120] Linking src/python/grpcio/grpc/_cython/cygrpc.so; 1s processwrapper-sandbox
#13 367.6 INFO: Found 1 target...
#13 367.7 Target //src/python/grpcio_tests/tests_py3_only/interop:xds_interop_server up-to-date:
#13 367.7 bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_server
#13 367.8 INFO: Elapsed time: 366.703s, Critical Path: 83.83s
#13 367.8 INFO: 4120 processes: 1615 internal, 2505 processwrapper-sandbox.
#13 367.8 INFO: Build completed successfully, 4120 total actions
#13 DONE 368.9s

#15 [stage-0 8/10] RUN cp -rL /workdir/bazel-bin/src/python/grpcio_tests/tests_py3_only/interop/xds_interop_server* /artifacts/
#15 DONE 1.4s

#16 [stage-0 9/10] ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
#16 DONE 0.0s

#17 [stage-0 10/10] RUN chmod +x /tini
#17 DONE 0.2s

#18 [stage-1 4/5] COPY --from=0 /artifacts ./
#18 DONE 1.2s

#19 [stage-1 5/5] COPY --from=0 /tini /tini
#19 DONE 0.0s

#20 exporting to image
#20 exporting layers
#20 exporting layers 1.8s done
#20 writing image sha256:0a10a0ab7d3a1b56ec14d40f7027441a6a97543adce4a224df26294726628e9b done
#20 naming to us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 done
#20 DONE 1.8s

1 warning found (use docker --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 14)
+ [18:20:22 UTC] Cmd finished: docker
+ [18:20:22 UTC] docker push us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
The push refers to repository [us-docker.pkg.dev/grpc-testing/psm-interop/python-server]
1233efbfa2fb: Preparing
46b01639095c: Preparing
4ac7b27603d0: Preparing
dd40b161e097: Preparing
9e599118e168: Preparing
e228adf1886f: Preparing
fb5ccd0db472: Preparing
8d853c8add5d: Preparing
e228adf1886f: Waiting
fb5ccd0db472: Waiting
8d853c8add5d: Waiting
9e599118e168: Layer already exists
e228adf1886f: Layer already exists
1233efbfa2fb: Layer already exists
fb5ccd0db472: Layer already exists
8d853c8add5d: Layer already exists
4ac7b27603d0: Pushed
dd40b161e097: Pushed
46b01639095c: Pushed
78f6c9348a24f6c49f81cc67867973e14eb00ef4: digest: sha256:94406d6a2e5273e7cf032cc3da73fe961a46d972fdd4cc9238de5b0b0df1fc5c size: 1998
+ [18:20:37 UTC] Cmd finished: docker
/tmp/tmp.AYSR3QX6TE/psm-interop
+ [18:20:37 UTC] Finished xDS interop test app Docker images
+ [18:20:37 UTC] Tests in csm test suite:
- gamma.gamma_baseline_test
- gamma.gamma_grpcroute_test
- gamma.affinity_test
- gamma.affinity_session_drain_test
- gamma.csm_observability_test
- app_net_ssa_test
- app_net_csm_observability_test

+ [18:20:37 UTC] Running csm suite test: gamma.gamma_baseline_test
+ [18:20:37 UTC] Test driver flags for gamma.gamma_baseline_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_baseline_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_baseline_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [18:20:37 UTC] python -m tests.gamma.gamma_baseline_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_baseline_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_baseline_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 18:20:39.291397 124581092499456 xds_k8s_testcase.py:167] ----- Testing GammaBaselineTest -----
I1016 18:20:39.291570 124581092499456 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 18:20:39.291821 124581092499456 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 18:20:39.305092 124581092499456 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] GammaBaselineTest.test_ping_pong
I1016 18:20:39.349552 124581092499456 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1820-2nilg
I1016 18:20:39.744182 124581092499456 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_baseline_test
I1016 18:20:39.745450 124581092499456 xds_k8s_testcase.py:311] --- Starting subTest GammaBaselineTest.test_ping_pong.1_run_test_server ---
I1016 18:20:39.745576 124581092499456 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1820-2nilg: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=1
I1016 18:20:39.745895 124581092499456 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1820-2nilg%22;timeRange=2024-10-16T18:20:39.745648Z%2F2024-10-16T18:50:39.745648Z?project=grpc-testing
I1016 18:20:39.750956 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1820-2nilg
labels:
name: psm-csm-server-20241016-1820-2nilg
owner: xds-k8s-interop-test
...

I1016 18:20:39.796738 124581092499456 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1820-2nilg created
I1016 18:20:39.799474 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1820-2nilg
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:20:39.862843 124581092499456 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:20:39.864880 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1820-2nilg
namespace: psm-csm-server-20241016-1820-2nilg
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:20:39.907315 124581092499456 k8s_base_runner.py:371] Service psm-csm-20241016-1820-2nilg created
I1016 18:20:39.910114 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1820-2nilg
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1820-2nilg
namespace: psm-csm-server-20241016-1820-2nilg
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:20:39.964249 124581092499456 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:20:39.980754 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1820-2nilg
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-sd5qn6cc
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-sd5qn6cc
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-sd5qn6cc
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1820-2nilg-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 18:20:40.039661 124581092499456 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:20:40.039865 124581092499456 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I1016 18:20:41.109534 124581092499456 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-5489d8c679-b4hpq']
I1016 18:20:41.109753 124581092499456 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5489d8c679-b4hpq to start
I1016 18:20:49.353323 124581092499456 k8s_base_runner.py:1006] Pod psm-grpc-server-5489d8c679-b4hpq ready, IP: [{'ip': '10.56.5.76'}]
I1016 18:20:49.353630 124581092499456 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5489d8c679-b4hpq to gamma.gamma_baseline_test/test_app_logs/psm-csm-server-20241016-1820-2nilg_psm-grpc-server-5489d8c679-b4hpq.log
I1016 18:20:49.354091 124580993697344 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1820-2nilg] Starting log collection thread 124580993697344 for psm-grpc-server-5489d8c679-b4hpq
I1016 18:20:49.354297 124581092499456 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I1016 18:20:53.446773 124581092499456 k8s_base_runner.py:972] Deployment psm-grpc-server has 1 replicas available
I1016 18:20:53.447043 124581092499456 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1820-2nilg
I1016 18:20:53.492146 124581092499456 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1820-2nilg: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-4035422c, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:20:53.492337 124581092499456 xds_k8s_testcase.py:316] --- Finished subTest GammaBaselineTest.test_ping_pong.1_run_test_server ---
I1016 18:20:53.492418 124581092499456 xds_k8s_testcase.py:311] --- Starting subTest GammaBaselineTest.test_ping_pong.2_start_test_client ---
I1016 18:20:53.492498 124581092499456 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1820-2nilg: server_target=xds:///psm-csm-20241016-1820-2nilg.psm-csm-server-20241016-1820-2nilg.svc.cluster.local:8080 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 18:20:53.492703 124581092499456 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1820-2nilg%22;timeRange=2024-10-16T18:20:53.492579Z%2F2024-10-16T18:50:53.492579Z?project=grpc-testing
I1016 18:20:53.494590 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1820-2nilg
labels:
name: psm-csm-client-20241016-1820-2nilg
owner: xds-k8s-interop-test
...

I1016 18:20:53.511440 124581092499456 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1820-2nilg created
I1016 18:20:53.521478 124581092499456 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1820-2nilg
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ul1qnx9c
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ul1qnx9c
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ul1qnx9c
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1820-2nilg.psm-csm-server-20241016-1820-2nilg.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:20:53.577272 124581092499456 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:20:53.577494 124581092499456 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:20:54.646395 124581092499456 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-66767dbb9-vktk8']
I1016 18:20:54.646662 124581092499456 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-66767dbb9-vktk8 to start
I1016 18:21:01.806826 124581092499456 k8s_base_runner.py:1006] Pod psm-grpc-client-66767dbb9-vktk8 ready, IP: [{'ip': '10.56.6.67'}]
I1016 18:21:01.807099 124581092499456 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-66767dbb9-vktk8 to gamma.gamma_baseline_test/test_app_logs/psm-csm-client-20241016-1820-2nilg_psm-grpc-client-66767dbb9-vktk8.log
I1016 18:21:01.807543 124580983211584 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1820-2nilg] Starting log collection thread 124580983211584 for psm-grpc-client-66767dbb9-vktk8
I1016 18:21:01.807774 124581092499456 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:21:06.917912 124581092499456 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:21:06.918316 124581092499456 client_app.py:294] [psm-grpc-client-66767dbb9-vktk8] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:21:06.948544 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:21:16.967697 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:21:26.986795 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:21:37.006181 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:21:47.027024 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:22:03.051926 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:22:28.085847 124581092499456 client_app.py:322] [psm-grpc-client-66767dbb9-vktk8] xDS control plane channel:
I1016 18:22:30.094582 124581092499456 client_app.py:333] [psm-grpc-client-66767dbb9-vktk8] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 18:22:30.094838 124581092499456 client_app.py:304] [psm-grpc-client-66767dbb9-vktk8] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 18:22:30.095122 124581092499456 client_app.py:254] [psm-grpc-client-66767dbb9-vktk8] Waiting to report a READY channel to xds:///psm-csm-20241016-1820-2nilg.psm-csm-server-20241016-1820-2nilg.svc.cluster.local:8080
I1016 18:22:30.102576 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:22:30.102759 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:22:40.122925 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:22:40.123143 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:22:50.142901 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:22:50.143112 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:00.162997 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:00.163185 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:10.182487 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:10.182703 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:26.208202 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:26.208424 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:51.244246 124581092499456 client_app.py:381] [psm-grpc-client-66767dbb9-vktk8] Server channel:
I1016 18:23:51.248703 124581092499456 client_app.py:394] [psm-grpc-client-66767dbb9-vktk8] Found subchannel in state READY:
I1016 18:23:51.248956 124581092499456 client_app.py:265] [psm-grpc-client-66767dbb9-vktk8] Channel to xds:///psm-csm-20241016-1820-2nilg.psm-csm-server-20241016-1820-2nilg.svc.cluster.local:8080 transitioned to state READY:
I1016 18:23:51.249089 124581092499456 xds_k8s_testcase.py:316] --- Finished subTest GammaBaselineTest.test_ping_pong.2_start_test_client ---
I1016 18:23:51.249154 124581092499456 xds_k8s_testcase.py:311] --- Starting subTest GammaBaselineTest.test_ping_pong.3_test_server_received_rpcs_from_test_client ---
I1016 18:23:51.249385 124581092499456 grpc.py:75] [psm-grpc-client-66767dbb9-vktk8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I1016 18:23:55.329670 124581092499456 xds_k8s_testcase.py:683] [psm-grpc-client-66767dbb9-vktk8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5489d8c679-b4hpq: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-5489d8c679-b4hpq: 100

I1016 18:23:55.329891 124581092499456 xds_k8s_testcase.py:316] --- Finished subTest GammaBaselineTest.test_ping_pong.3_test_server_received_rpcs_from_test_client ---
I1016 18:23:55.329977 124581092499456 xds_k8s_testcase.py:848] ----- TestMethod GammaBaselineTest.test_ping_pong teardown -----
I1016 18:23:55.439971 124581092499456 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 18:24:05.590468 124581092499456 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 18:24:05.590730 124581092499456 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1820-2nilg
I1016 18:24:35.686397 124581092499456 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1820-2nilg deleted
I1016 18:24:35.686745 124581092499456 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 18:24:35.725741 124581092499456 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 18:24:35.758011 124581092499456 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 18:24:35.787581 124581092499456 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 18:24:36.876523 124581092499456 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 18:24:36.876767 124581092499456 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1820-2nilg
I1016 18:24:37.969499 124581092499456 k8s_base_runner.py:923] Service psm-csm-20241016-1820-2nilg deleted
I1016 18:24:37.969698 124581092499456 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 18:24:39.055185 124581092499456 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 18:24:39.055380 124581092499456 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 18:24:49.205047 124581092499456 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 18:24:49.205297 124581092499456 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1820-2nilg
I1016 18:26:19.396922 124581092499456 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1820-2nilg deleted
I1016 18:26:19.397303 124581092499456 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 18:26:19.397550 124581092499456 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-ul1qnx9c:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1820-2nilg%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-ul1qnx9c%22;timeRange=2024-10-16T18:20:53.492572Z%2F2024-10-16T18:24:35.686664Z;cursorTimestamp=2024-10-16T18:21:06.918110Z?project=grpc-testing
I1016 18:26:19.397755 124581092499456 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-sd5qn6cc:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1820-2nilg%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-sd5qn6cc%22;timeRange=2024-10-16T18:20:39.745642Z%2F2024-10-16T18:26:19.397125Z;cursorTimestamp=2024-10-16T18:20:53.446982Z?project=grpc-testing
[ OK ] GammaBaselineTest.test_ping_pong
I1016 18:26:19.398032 124581092499456 base_testcase.py:67] ----- PSM Test Case PASSED: GammaBaselineTest.test_ping_pong -----

----------------------------------------------------------------------
Ran 1 test in 340.107s

OK
+ [18:26:19 UTC] Cmd finished: python
+ [18:26:19 UTC] Finished csm suite test: gamma.gamma_baseline_test

+ [18:26:19 UTC] Running csm suite test: gamma.gamma_grpcroute_test
+ [18:26:19 UTC] Test driver flags for gamma.gamma_grpcroute_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_grpcroute_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_grpcroute_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [18:26:19 UTC] python -m tests.gamma.gamma_grpcroute_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_grpcroute_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.gamma_grpcroute_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 18:26:20.947727 139384990769152 xds_k8s_testcase.py:167] ----- Testing GammaGrpcRouteTest -----
I1016 18:26:20.947914 139384990769152 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 18:26:20.948179 139384990769152 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 18:26:20.961066 139384990769152 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] GammaGrpcRouteTest.test_ping_pong
[ SKIPPED ] GammaGrpcRouteTest.test_ping_pong
I1016 18:26:20.966964 139384990769152 base_testcase.py:60] ----- PSM Test Case SKIPPED: GammaGrpcRouteTest.test_ping_pong -----
Reason for skipping: Skipping test until cl/655674720 is merged
----------------------------------------------------------------------
Ran 1 test in 0.020s

OK (skipped=1)
+ [18:26:21 UTC] Cmd finished: python
+ [18:26:21 UTC] Finished csm suite test: gamma.gamma_grpcroute_test

+ [18:26:21 UTC] Running csm suite test: gamma.affinity_test
+ [18:26:21 UTC] Test driver flags for gamma.affinity_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [18:26:21 UTC] python -m tests.gamma.affinity_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 18:26:22.458863 131645990072320 xds_k8s_testcase.py:167] ----- Testing AffinityTest -----
I1016 18:26:22.459026 131645990072320 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 18:26:22.459287 131645990072320 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 18:26:22.471309 131645990072320 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] AffinityTest.test_session_affinity_filter
I1016 18:26:22.477323 131645990072320 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1826-6bdzn
I1016 18:26:22.850450 131645990072320 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_test
I1016 18:26:22.851805 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_filter.01_run_test_server ---
I1016 18:26:22.851945 131645990072320 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1826-6bdzn: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=3
I1016 18:26:22.852243 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1826-6bdzn%22;timeRange=2024-10-16T18:26:22.852086Z%2F2024-10-16T18:56:22.852086Z?project=grpc-testing
I1016 18:26:22.857283 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1826-6bdzn
labels:
name: psm-csm-server-20241016-1826-6bdzn
owner: xds-k8s-interop-test
...

I1016 18:26:22.910519 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1826-6bdzn created
I1016 18:26:22.913040 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1826-6bdzn
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:26:22.970125 131645990072320 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:26:22.972212 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1826-6bdzn
namespace: psm-csm-server-20241016-1826-6bdzn
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:26:23.002066 131645990072320 k8s_base_runner.py:371] Service psm-csm-20241016-1826-6bdzn created
I1016 18:26:23.005098 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http_ssafilter.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1826-6bdzn
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1826-6bdzn
namespace: psm-csm-server-20241016-1826-6bdzn
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
filters:
- type: ExtensionRef
extensionRef:
group: networking.gke.io
kind: GCPSessionAffinityFilter
name: ssa-filter
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:26:23.064158 131645990072320 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:26:23.081841 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1826-6bdzn
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jocb6bfd
owner: xds-k8s-interop-test
spec:
replicas: 3
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jocb6bfd
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jocb6bfd
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1826-6bdzn-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 18:26:23.141959 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:26:23.142163 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 3 pod(s)
I1016 18:26:24.378991 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 3 pod(s): ['psm-grpc-server-59fb4c9cc8-7k4jv', 'psm-grpc-server-59fb4c9cc8-nkrsq', 'psm-grpc-server-59fb4c9cc8-nrfng']
I1016 18:26:24.379358 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-59fb4c9cc8-7k4jv to start
I1016 18:26:31.620434 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-59fb4c9cc8-7k4jv ready, IP: [{'ip': '10.56.6.68'}]
I1016 18:26:31.620721 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-59fb4c9cc8-7k4jv to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1826-6bdzn_psm-grpc-server-59fb4c9cc8-7k4jv.log
I1016 18:26:31.621189 131645889840704 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1826-6bdzn] Starting log collection thread 131645889840704 for psm-grpc-server-59fb4c9cc8-7k4jv
I1016 18:26:31.621452 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-59fb4c9cc8-nkrsq to start
I1016 18:26:31.673378 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-59fb4c9cc8-nkrsq ready, IP: [{'ip': '10.56.5.77'}]
I1016 18:26:31.673678 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-59fb4c9cc8-nkrsq to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1826-6bdzn_psm-grpc-server-59fb4c9cc8-nkrsq.log
I1016 18:26:31.674134 131645879354944 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1826-6bdzn] Starting log collection thread 131645879354944 for psm-grpc-server-59fb4c9cc8-nkrsq
I1016 18:26:31.674346 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-59fb4c9cc8-nrfng to start
I1016 18:26:31.721309 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-59fb4c9cc8-nrfng ready, IP: [{'ip': '10.56.7.49'}]
I1016 18:26:31.721608 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-59fb4c9cc8-nrfng to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1826-6bdzn_psm-grpc-server-59fb4c9cc8-nrfng.log
I1016 18:26:31.722018 131645868869184 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1826-6bdzn] Starting log collection thread 131645868869184 for psm-grpc-server-59fb4c9cc8-nrfng
I1016 18:26:31.722209 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 3 available replica(s)
I1016 18:26:36.830861 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-server has 3 replicas available
I1016 18:26:36.831131 131645990072320 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1826-6bdzn
I1016 18:26:48.089999 131645990072320 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1826-6bdzn: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-5ae91aa5, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:26:48.090229 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_filter.01_run_test_server ---
I1016 18:26:48.090313 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_filter.02_create_ssa_filter ---
I1016 18:26:48.092244 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/session_affinity_filter.yaml:
---
apiVersion: networking.gke.io/v1
kind: GCPSessionAffinityFilter
metadata:
name: ssa-filter
namespace: psm-csm-server-20241016-1826-6bdzn
labels:
owner: xds-k8s-interop-test
spec:
statefulGeneratedCookie:
cookieTtlSeconds: 50
...

I1016 18:26:48.129227 131645990072320 k8s_base_runner.py:371] GCPSessionAffinityFilter ssa-filter created
I1016 18:26:48.129454 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_filter.02_create_ssa_filter ---
I1016 18:26:48.129524 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_filter.03_start_test_client ---
I1016 18:26:48.129615 131645990072320 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1826-6bdzn: server_target=xds:///psm-csm-20241016-1826-6bdzn.psm-csm-server-20241016-1826-6bdzn.svc.cluster.local:8080 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 18:26:48.129813 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1826-6bdzn%22;timeRange=2024-10-16T18:26:48.129699Z%2F2024-10-16T18:56:48.129699Z?project=grpc-testing
I1016 18:26:48.131688 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1826-6bdzn
labels:
name: psm-csm-client-20241016-1826-6bdzn
owner: xds-k8s-interop-test
...

I1016 18:26:48.162184 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1826-6bdzn created
I1016 18:26:48.172615 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1826-6bdzn
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-8ykqsuqu
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-8ykqsuqu
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-8ykqsuqu
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1826-6bdzn.psm-csm-server-20241016-1826-6bdzn.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:26:48.251383 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:26:48.251605 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:26:49.338886 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-b84745c65-wjf2g']
I1016 18:26:49.339112 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-b84745c65-wjf2g to start
I1016 18:26:56.559506 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-client-b84745c65-wjf2g ready, IP: [{'ip': '10.56.8.13'}]
I1016 18:26:56.559797 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-b84745c65-wjf2g to gamma.affinity_test/test_app_logs/psm-csm-client-20241016-1826-6bdzn_psm-grpc-client-b84745c65-wjf2g.log
I1016 18:26:56.560268 131645858383424 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1826-6bdzn] Starting log collection thread 131645858383424 for psm-grpc-client-b84745c65-wjf2g
I1016 18:26:56.560545 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:27:01.666162 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:27:01.666616 131645990072320 client_app.py:294] [psm-grpc-client-b84745c65-wjf2g] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:27:01.701258 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:27:11.721266 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:27:21.742872 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:27:31.763883 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:27:41.782660 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:27:57.811233 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:28:22.847215 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:28:47.882101 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:29:12.918648 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:29:37.955301 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:30:02.995255 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:30:28.021788 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:30:53.058461 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:31:18.095291 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:31:43.134518 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:32:08.171783 131645990072320 client_app.py:322] [psm-grpc-client-b84745c65-wjf2g] xDS control plane channel:
I1016 18:32:10.181175 131645990072320 client_app.py:333] [psm-grpc-client-b84745c65-wjf2g] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 18:32:10.181475 131645990072320 client_app.py:304] [psm-grpc-client-b84745c65-wjf2g] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 18:32:10.181795 131645990072320 client_app.py:254] [psm-grpc-client-b84745c65-wjf2g] Waiting to report a READY channel to xds:///psm-csm-20241016-1826-6bdzn.psm-csm-server-20241016-1826-6bdzn.svc.cluster.local:8080
I1016 18:32:10.194311 131645990072320 client_app.py:381] [psm-grpc-client-b84745c65-wjf2g] Server channel:
I1016 18:32:10.200299 131645990072320 client_app.py:394] [psm-grpc-client-b84745c65-wjf2g] Found subchannel in state READY:
I1016 18:32:10.200687 131645990072320 client_app.py:265] [psm-grpc-client-b84745c65-wjf2g] Channel to xds:///psm-csm-20241016-1826-6bdzn.psm-csm-server-20241016-1826-6bdzn.svc.cluster.local:8080 transitioned to state READY:
I1016 18:32:10.200833 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_filter.03_start_test_client ---
I1016 18:32:10.200899 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_filter.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:32:10.201374 131645990072320 grpc.py:75] [psm-grpc-client-b84745c65-wjf2g:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:32:10.307019 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-b84745c65-wjf2g] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-59fb4c9cc8-nrfng: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-59fb4c9cc8-nrfng: 1
metadatas_by_peer:
psm-grpc-server-59fb4c9cc8-nrfng:
hostname:
- psm-grpc-server-59fb4c9cc8-nrfng
set-cookie:
- GSSA=MTAuNTYuNy40OTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MTA1NDE0NjQ0MTg1MjgyODA5;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:32:10.307524 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_filter.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:32:10.307641 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_filter.05_send_RPCs_with_cookie ---
I1016 18:32:10.307889 131645990072320 grpc.py:75] [psm-grpc-client-b84745c65-wjf2g:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNy40OTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MTA1NDE0NjQ0MTg1MjgyODA5; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50'}]}), timeout=5, wait_for_ready=True)
I1016 18:32:10.312742 131645990072320 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-59fb4c9cc8-nrfng']
I1016 18:32:10.312993 131645990072320 grpc.py:75] [psm-grpc-client-b84745c65-wjf2g:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 10, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:32:10.789072 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-b84745c65-wjf2g] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-59fb4c9cc8-nrfng: 10
rpcs_by_method:
UnaryCall:
psm-grpc-server-59fb4c9cc8-nrfng: 10
metadatas_by_peer:
psm-grpc-server-59fb4c9cc8-nrfng:
hostname:
- psm-grpc-server-59fb4c9cc8-nrfng

I1016 18:32:10.789495 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_filter.05_send_RPCs_with_cookie ---
I1016 18:32:10.789617 131645990072320 xds_k8s_testcase.py:848] ----- TestMethod AffinityTest.test_session_affinity_filter teardown -----
I1016 18:32:10.917812 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 18:32:21.056031 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 18:32:21.056286 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1826-6bdzn
I1016 18:32:51.187120 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1826-6bdzn deleted
I1016 18:32:51.187360 131645990072320 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 18:32:51.216931 131645990072320 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 18:33:01.310940 131645990072320 k8s_base_runner.py:858] GCPSessionAffinityFilter ssa-filter deleted
I1016 18:33:01.311129 131645990072320 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 18:33:01.340228 131645990072320 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 18:33:02.425876 131645990072320 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 18:33:02.426086 131645990072320 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1826-6bdzn
I1016 18:33:03.503841 131645990072320 k8s_base_runner.py:923] Service psm-csm-20241016-1826-6bdzn deleted
I1016 18:33:03.504060 131645990072320 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 18:33:04.607927 131645990072320 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 18:33:04.608157 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 18:33:14.772073 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 18:33:14.772333 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1826-6bdzn
I1016 18:36:15.136495 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1826-6bdzn deleted
I1016 18:36:15.136946 131645990072320 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 18:36:15.137192 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-8ykqsuqu:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1826-6bdzn%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-8ykqsuqu%22;timeRange=2024-10-16T18:26:48.129691Z%2F2024-10-16T18:32:51.187324Z;cursorTimestamp=2024-10-16T18:27:01.666391Z?project=grpc-testing
I1016 18:36:15.137329 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-jocb6bfd:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1826-6bdzn%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-jocb6bfd%22;timeRange=2024-10-16T18:26:22.852080Z%2F2024-10-16T18:36:15.136738Z;cursorTimestamp=2024-10-16T18:26:36.831061Z?project=grpc-testing
[ OK ] AffinityTest.test_session_affinity_filter
I1016 18:36:15.137627 131645990072320 base_testcase.py:67] ----- PSM Test Case PASSED: AffinityTest.test_session_affinity_filter -----

[ RUN ] AffinityTest.test_session_affinity_policy_with_route_target
I1016 18:36:15.137872 131645990072320 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1836-f4gbe
I1016 18:36:15.138497 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_route_target.01_run_test_server ---
I1016 18:36:15.138602 131645990072320 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1836-f4gbe: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=3
I1016 18:36:15.138772 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1836-f4gbe%22;timeRange=2024-10-16T18:36:15.138677Z%2F2024-10-16T19:06:15.138677Z?project=grpc-testing
I1016 18:36:15.139342 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1836-f4gbe
labels:
name: psm-csm-server-20241016-1836-f4gbe
owner: xds-k8s-interop-test
...

I1016 18:36:15.182226 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1836-f4gbe created
I1016 18:36:15.183249 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1836-f4gbe
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:36:15.241894 131645990072320 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:36:15.242775 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1836-f4gbe
namespace: psm-csm-server-20241016-1836-f4gbe
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:36:15.271021 131645990072320 k8s_base_runner.py:371] Service psm-csm-20241016-1836-f4gbe created
I1016 18:36:15.273942 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1836-f4gbe
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1836-f4gbe
namespace: psm-csm-server-20241016-1836-f4gbe
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:36:15.313145 131645990072320 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:36:15.316545 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1836-f4gbe
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c7g0licq
owner: xds-k8s-interop-test
spec:
replicas: 3
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c7g0licq
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c7g0licq
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1836-f4gbe-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 18:36:15.388356 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:36:15.388596 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 3 pod(s)
I1016 18:36:16.477946 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 3 pod(s): ['psm-grpc-server-5df4f9bc7c-5676l', 'psm-grpc-server-5df4f9bc7c-57855', 'psm-grpc-server-5df4f9bc7c-p7xkc']
I1016 18:36:16.478335 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5df4f9bc7c-5676l to start
I1016 18:36:18.556684 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-5df4f9bc7c-5676l ready, IP: [{'ip': '10.56.7.50'}]
I1016 18:36:18.556959 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5df4f9bc7c-5676l to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1836-f4gbe_psm-grpc-server-5df4f9bc7c-5676l.log
I1016 18:36:18.557319 131645868869184 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1836-f4gbe] Starting log collection thread 131645868869184 for psm-grpc-server-5df4f9bc7c-5676l
I1016 18:36:18.557544 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5df4f9bc7c-57855 to start
I1016 18:36:18.605942 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-5df4f9bc7c-57855 ready, IP: [{'ip': '10.56.6.69'}]
I1016 18:36:18.606231 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5df4f9bc7c-57855 to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1836-f4gbe_psm-grpc-server-5df4f9bc7c-57855.log
I1016 18:36:18.606638 131645889840704 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1836-f4gbe] Starting log collection thread 131645889840704 for psm-grpc-server-5df4f9bc7c-57855
I1016 18:36:18.606862 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5df4f9bc7c-p7xkc to start
I1016 18:36:18.667701 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-5df4f9bc7c-p7xkc ready, IP: [{'ip': '10.56.5.78'}]
I1016 18:36:18.667986 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5df4f9bc7c-p7xkc to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1836-f4gbe_psm-grpc-server-5df4f9bc7c-p7xkc.log
I1016 18:36:18.668339 131645879354944 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1836-f4gbe] Starting log collection thread 131645879354944 for psm-grpc-server-5df4f9bc7c-p7xkc
I1016 18:36:18.668582 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 3 available replica(s)
I1016 18:36:22.772447 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-server has 3 replicas available
I1016 18:36:22.772730 131645990072320 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1836-f4gbe
I1016 18:36:22.811889 131645990072320 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1836-f4gbe: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-98dbfe25, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:36:22.812098 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_route_target.01_run_test_server ---
I1016 18:36:22.812173 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_route_target.02_create_ssa_policy ---
I1016 18:36:22.814632 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/session_affinity_policy_route.yaml:
---
apiVersion: networking.gke.io/v1
kind: GCPSessionAffinityPolicy
metadata:
name: ssa-policy
namespace: psm-csm-server-20241016-1836-f4gbe
labels:
owner: xds-k8s-interop-test
spec:
statefulGeneratedCookie:
cookieTtlSeconds: 50
targetRef:
name: route-httproute-psm-grpc-server
group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: psm-csm-server-20241016-1836-f4gbe
...

I1016 18:36:22.861286 131645990072320 k8s_base_runner.py:371] GCPSessionAffinityPolicy ssa-policy created
I1016 18:36:22.861539 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_route_target.02_create_ssa_policy ---
I1016 18:36:22.861617 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_route_target.03_start_test_client ---
I1016 18:36:22.861710 131645990072320 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1836-f4gbe: server_target=xds:///psm-csm-20241016-1836-f4gbe.psm-csm-server-20241016-1836-f4gbe.svc.cluster.local:8080 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 18:36:22.861900 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1836-f4gbe%22;timeRange=2024-10-16T18:36:22.861792Z%2F2024-10-16T19:06:22.861792Z?project=grpc-testing
I1016 18:36:22.862465 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1836-f4gbe
labels:
name: psm-csm-client-20241016-1836-f4gbe
owner: xds-k8s-interop-test
...

I1016 18:36:22.879784 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1836-f4gbe created
I1016 18:36:22.883344 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1836-f4gbe
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ug2o2opj
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ug2o2opj
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ug2o2opj
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1836-f4gbe.psm-csm-server-20241016-1836-f4gbe.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:36:22.945334 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:36:22.945560 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:36:24.026012 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-77d4bcc7f4-jnhc8']
I1016 18:36:24.026241 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-77d4bcc7f4-jnhc8 to start
I1016 18:36:26.118363 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-client-77d4bcc7f4-jnhc8 ready, IP: [{'ip': '10.56.8.14'}]
I1016 18:36:26.118673 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-77d4bcc7f4-jnhc8 to gamma.affinity_test/test_app_logs/psm-csm-client-20241016-1836-f4gbe_psm-grpc-client-77d4bcc7f4-jnhc8.log
I1016 18:36:26.119053 131645858383424 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1836-f4gbe] Starting log collection thread 131645858383424 for psm-grpc-client-77d4bcc7f4-jnhc8
I1016 18:36:26.119267 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:36:30.229239 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:36:30.229722 131645990072320 client_app.py:294] [psm-grpc-client-77d4bcc7f4-jnhc8] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:36:30.253101 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:36:40.272076 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:36:50.292660 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:37:00.314384 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:37:10.337678 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:37:26.363935 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:37:51.398869 131645990072320 client_app.py:322] [psm-grpc-client-77d4bcc7f4-jnhc8] xDS control plane channel:
I1016 18:37:53.407386 131645990072320 client_app.py:333] [psm-grpc-client-77d4bcc7f4-jnhc8] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 18:37:53.407695 131645990072320 client_app.py:304] [psm-grpc-client-77d4bcc7f4-jnhc8] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 18:37:53.407986 131645990072320 client_app.py:254] [psm-grpc-client-77d4bcc7f4-jnhc8] Waiting to report a READY channel to xds:///psm-csm-20241016-1836-f4gbe.psm-csm-server-20241016-1836-f4gbe.svc.cluster.local:8080
I1016 18:37:53.415556 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:37:53.415733 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:03.438103 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:03.438307 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:13.461900 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:13.462100 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:23.483657 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:23.483856 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:33.505000 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:33.505201 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:49.534195 131645990072320 client_app.py:381] [psm-grpc-client-77d4bcc7f4-jnhc8] Server channel:
I1016 18:38:49.539367 131645990072320 client_app.py:394] [psm-grpc-client-77d4bcc7f4-jnhc8] Found subchannel in state READY:
I1016 18:38:49.539679 131645990072320 client_app.py:265] [psm-grpc-client-77d4bcc7f4-jnhc8] Channel to xds:///psm-csm-20241016-1836-f4gbe.psm-csm-server-20241016-1836-f4gbe.svc.cluster.local:8080 transitioned to state READY:
I1016 18:38:49.539820 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_route_target.03_start_test_client ---
I1016 18:38:49.539888 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_route_target.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:38:49.540432 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:38:49.658997 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l

I1016 18:38:49.659645 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:38:59.670054 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:38:59.782808 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l

I1016 18:38:59.783170 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:39:09.793576 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:39:09.911534 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc:
hostname:
- psm-grpc-server-5df4f9bc7c-p7xkc

I1016 18:39:09.911895 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:39:19.922125 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:39:20.034535 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc:
hostname:
- psm-grpc-server-5df4f9bc7c-p7xkc

I1016 18:39:20.034879 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:39:30.045249 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:39:30.158078 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l

I1016 18:39:30.158441 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:39:40.168831 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:39:40.281006 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-p7xkc: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-p7xkc:
hostname:
- psm-grpc-server-5df4f9bc7c-p7xkc

I1016 18:39:40.281360 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:39:50.291779 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:39:50.403786 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-57855: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-57855: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-57855:
hostname:
- psm-grpc-server-5df4f9bc7c-57855

I1016 18:39:50.404158 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:40:00.414660 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:40:00.532084 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l

I1016 18:40:00.532466 131645990072320 retryers.py:213] Retrying framework.test_cases.session_affinity_util.assert_eventually_retrieve_cookie_and_server.._assert_retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: {} has length of 0, expected 1.
I1016 18:40:10.542905 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:40:10.657014 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 1
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l
set-cookie:
- GSSA=MTAuNTYuNy41MDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF8zMzQ2ODk5MTQ3MzY0ODc0NTQy;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:40:10.657363 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_route_target.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:40:10.657473 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_route_target.05_send_RPCs_with_cookie ---
I1016 18:40:10.657727 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNy41MDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF8zMzQ2ODk5MTQ3MzY0ODc0NTQy; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50'}]}), timeout=5, wait_for_ready=True)
I1016 18:40:10.661503 131645990072320 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-5df4f9bc7c-5676l']
I1016 18:40:10.661744 131645990072320 grpc.py:75] [psm-grpc-client-77d4bcc7f4-jnhc8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 10, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:40:11.138826 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-77d4bcc7f4-jnhc8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5df4f9bc7c-5676l: 10
rpcs_by_method:
UnaryCall:
psm-grpc-server-5df4f9bc7c-5676l: 10
metadatas_by_peer:
psm-grpc-server-5df4f9bc7c-5676l:
hostname:
- psm-grpc-server-5df4f9bc7c-5676l

I1016 18:40:11.139230 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_route_target.05_send_RPCs_with_cookie ---
I1016 18:40:11.139335 131645990072320 xds_k8s_testcase.py:848] ----- TestMethod AffinityTest.test_session_affinity_policy_with_route_target teardown -----
I1016 18:40:11.276600 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 18:40:21.439011 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 18:40:21.439264 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1836-f4gbe
I1016 18:40:51.568133 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1836-f4gbe deleted
I1016 18:40:51.568374 131645990072320 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 18:41:01.672389 131645990072320 k8s_base_runner.py:835] GCPSessionAffinityPolicy ssa-policy deleted
I1016 18:41:01.672602 131645990072320 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 18:41:01.704604 131645990072320 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 18:41:01.745357 131645990072320 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 18:41:01.830350 131645990072320 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 18:41:01.830570 131645990072320 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1836-f4gbe
I1016 18:41:01.916943 131645990072320 k8s_base_runner.py:923] Service psm-csm-20241016-1836-f4gbe deleted
I1016 18:41:01.917144 131645990072320 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 18:41:02.995929 131645990072320 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 18:41:02.996126 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 18:41:13.111251 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 18:41:13.111528 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1836-f4gbe
I1016 18:44:43.589533 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1836-f4gbe deleted
I1016 18:44:43.589978 131645990072320 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 18:44:43.590208 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-ug2o2opj:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1836-f4gbe%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-ug2o2opj%22;timeRange=2024-10-16T18:36:22.861784Z%2F2024-10-16T18:40:51.568340Z;cursorTimestamp=2024-10-16T18:36:30.229484Z?project=grpc-testing
I1016 18:44:43.590343 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-c7g0licq:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1836-f4gbe%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-c7g0licq%22;timeRange=2024-10-16T18:36:15.138672Z%2F2024-10-16T18:44:43.589759Z;cursorTimestamp=2024-10-16T18:36:22.772662Z?project=grpc-testing
[ OK ] AffinityTest.test_session_affinity_policy_with_route_target
I1016 18:44:43.590638 131645990072320 base_testcase.py:67] ----- PSM Test Case PASSED: AffinityTest.test_session_affinity_policy_with_route_target -----

[ RUN ] AffinityTest.test_session_affinity_policy_with_service_target
I1016 18:44:43.590899 131645990072320 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1844-e79dc
I1016 18:44:43.591535 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_service_target.01_run_test_server ---
I1016 18:44:43.591635 131645990072320 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1844-e79dc: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=3
I1016 18:44:43.591788 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1844-e79dc%22;timeRange=2024-10-16T18:44:43.591702Z%2F2024-10-16T19:14:43.591702Z?project=grpc-testing
I1016 18:44:43.592360 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1844-e79dc
labels:
name: psm-csm-server-20241016-1844-e79dc
owner: xds-k8s-interop-test
...

I1016 18:44:43.635775 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1844-e79dc created
I1016 18:44:43.636744 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1844-e79dc
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:44:43.694867 131645990072320 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:44:43.695709 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1844-e79dc
namespace: psm-csm-server-20241016-1844-e79dc
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:44:43.727428 131645990072320 k8s_base_runner.py:371] Service psm-csm-20241016-1844-e79dc created
I1016 18:44:43.728650 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1844-e79dc
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1844-e79dc
namespace: psm-csm-server-20241016-1844-e79dc
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:44:43.770563 131645990072320 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:44:43.773821 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1844-e79dc
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-v2cyzk8i
owner: xds-k8s-interop-test
spec:
replicas: 3
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-v2cyzk8i
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-v2cyzk8i
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1844-e79dc-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 18:44:43.830873 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:44:43.831069 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 3 pod(s)
I1016 18:44:44.909435 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 3 pod(s): ['psm-grpc-server-7d687f955b-957hl', 'psm-grpc-server-7d687f955b-9m79n', 'psm-grpc-server-7d687f955b-xcps4']
I1016 18:44:44.909799 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-7d687f955b-957hl to start
I1016 18:44:46.985713 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-7d687f955b-957hl ready, IP: [{'ip': '10.56.6.70'}]
I1016 18:44:46.986002 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-7d687f955b-957hl to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1844-e79dc_psm-grpc-server-7d687f955b-957hl.log
I1016 18:44:46.986346 131645889840704 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1844-e79dc] Starting log collection thread 131645889840704 for psm-grpc-server-7d687f955b-957hl
I1016 18:44:46.986585 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-7d687f955b-9m79n to start
I1016 18:44:47.033999 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-7d687f955b-9m79n ready, IP: [{'ip': '10.56.5.79'}]
I1016 18:44:47.034275 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-7d687f955b-9m79n to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1844-e79dc_psm-grpc-server-7d687f955b-9m79n.log
I1016 18:44:47.034665 131645868869184 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1844-e79dc] Starting log collection thread 131645868869184 for psm-grpc-server-7d687f955b-9m79n
I1016 18:44:47.034881 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-7d687f955b-xcps4 to start
I1016 18:44:47.092878 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-server-7d687f955b-xcps4 ready, IP: [{'ip': '10.56.7.51'}]
I1016 18:44:47.093157 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-7d687f955b-xcps4 to gamma.affinity_test/test_app_logs/psm-csm-server-20241016-1844-e79dc_psm-grpc-server-7d687f955b-xcps4.log
I1016 18:44:47.093943 131645879354944 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1844-e79dc] Starting log collection thread 131645879354944 for psm-grpc-server-7d687f955b-xcps4
I1016 18:44:47.094171 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 3 available replica(s)
I1016 18:44:51.188606 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-server has 3 replicas available
I1016 18:44:51.188869 131645990072320 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1844-e79dc
I1016 18:44:51.228926 131645990072320 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1844-e79dc: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-53747a26, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:44:51.229129 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_service_target.01_run_test_server ---
I1016 18:44:51.229202 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_service_target.02_create_ssa_policy ---
I1016 18:44:51.231627 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/session_affinity_policy_service.yaml:
---
apiVersion: networking.gke.io/v1
kind: GCPSessionAffinityPolicy
metadata:
name: ssa-policy
namespace: psm-csm-server-20241016-1844-e79dc
labels:
owner: xds-k8s-interop-test
spec:
statefulGeneratedCookie:
cookieTtlSeconds: 50
targetRef:
name: psm-grpc-server
kind: Service
namespace: psm-csm-server-20241016-1844-e79dc
group: ""
...

I1016 18:44:51.267057 131645990072320 k8s_base_runner.py:371] GCPSessionAffinityPolicy ssa-policy created
I1016 18:44:51.267260 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_service_target.02_create_ssa_policy ---
I1016 18:44:51.267330 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_service_target.03_start_test_client ---
I1016 18:44:51.267435 131645990072320 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1844-e79dc: server_target=xds:///psm-csm-20241016-1844-e79dc.psm-csm-server-20241016-1844-e79dc.svc.cluster.local:8080 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 18:44:51.267642 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1844-e79dc%22;timeRange=2024-10-16T18:44:51.267522Z%2F2024-10-16T19:14:51.267522Z?project=grpc-testing
I1016 18:44:51.268205 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1844-e79dc
labels:
name: psm-csm-client-20241016-1844-e79dc
owner: xds-k8s-interop-test
...

I1016 18:44:51.288203 131645990072320 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1844-e79dc created
I1016 18:44:51.291838 131645990072320 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1844-e79dc
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-z2i7g5jq
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-z2i7g5jq
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-z2i7g5jq
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1844-e79dc.psm-csm-server-20241016-1844-e79dc.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:44:51.351850 131645990072320 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:44:51.352059 131645990072320 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:44:52.418980 131645990072320 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-65bc8498cc-tx48v']
I1016 18:44:52.419201 131645990072320 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-65bc8498cc-tx48v to start
I1016 18:44:54.504174 131645990072320 k8s_base_runner.py:1006] Pod psm-grpc-client-65bc8498cc-tx48v ready, IP: [{'ip': '10.56.8.15'}]
I1016 18:44:54.504467 131645990072320 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-65bc8498cc-tx48v to gamma.affinity_test/test_app_logs/psm-csm-client-20241016-1844-e79dc_psm-grpc-client-65bc8498cc-tx48v.log
I1016 18:44:54.504824 131645858383424 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1844-e79dc] Starting log collection thread 131645858383424 for psm-grpc-client-65bc8498cc-tx48v
I1016 18:44:54.505032 131645990072320 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:44:58.608450 131645990072320 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:44:58.608899 131645990072320 client_app.py:294] [psm-grpc-client-65bc8498cc-tx48v] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:44:58.631857 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:45:08.653602 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:45:18.673230 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:45:28.692996 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:45:38.712317 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:45:54.739875 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:46:19.775682 131645990072320 client_app.py:322] [psm-grpc-client-65bc8498cc-tx48v] xDS control plane channel:
I1016 18:46:21.783981 131645990072320 client_app.py:333] [psm-grpc-client-65bc8498cc-tx48v] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 18:46:21.784243 131645990072320 client_app.py:304] [psm-grpc-client-65bc8498cc-tx48v] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 18:46:21.784523 131645990072320 client_app.py:254] [psm-grpc-client-65bc8498cc-tx48v] Waiting to report a READY channel to xds:///psm-csm-20241016-1844-e79dc.psm-csm-server-20241016-1844-e79dc.svc.cluster.local:8080
I1016 18:46:21.792597 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:21.792820 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:31.812353 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:31.812571 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:41.835778 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:41.835993 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:51.860603 131645990072320 client_app.py:381] [psm-grpc-client-65bc8498cc-tx48v] Server channel:
I1016 18:46:51.866173 131645990072320 client_app.py:394] [psm-grpc-client-65bc8498cc-tx48v] Found subchannel in state READY:
I1016 18:46:51.866479 131645990072320 client_app.py:265] [psm-grpc-client-65bc8498cc-tx48v] Channel to xds:///psm-csm-20241016-1844-e79dc.psm-csm-server-20241016-1844-e79dc.svc.cluster.local:8080 transitioned to state READY:
I1016 18:46:51.866622 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_service_target.03_start_test_client ---
I1016 18:46:51.866692 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_service_target.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:46:51.867211 131645990072320 grpc.py:75] [psm-grpc-client-65bc8498cc-tx48v:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:46:51.993261 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-65bc8498cc-tx48v] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-7d687f955b-xcps4: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-7d687f955b-xcps4: 1
metadatas_by_peer:
psm-grpc-server-7d687f955b-xcps4:
hostname:
- psm-grpc-server-7d687f955b-xcps4
set-cookie:
- GSSA=MTAuNTYuNy41MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF80NjY5NjE3MTE1OTU3NjcxNzY4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:46:51.993733 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_service_target.04_send_first_RPC_and_retrieve_cookie ---
I1016 18:46:51.993845 131645990072320 xds_k8s_testcase.py:311] --- Starting subTest AffinityTest.test_session_affinity_policy_with_service_target.05_send_RPCs_with_cookie ---
I1016 18:46:51.994089 131645990072320 grpc.py:75] [psm-grpc-client-65bc8498cc-tx48v:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNy41MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF80NjY5NjE3MTE1OTU3NjcxNzY4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50'}]}), timeout=5, wait_for_ready=True)
I1016 18:46:51.998269 131645990072320 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-7d687f955b-xcps4']
I1016 18:46:51.998524 131645990072320 grpc.py:75] [psm-grpc-client-65bc8498cc-tx48v:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 10, 'timeoutSec': 1200, 'metadataKeys': ['*']}), timeout=1200, wait_for_ready=True)
I1016 18:46:52.474611 131645990072320 xds_k8s_testcase.py:683] [psm-grpc-client-65bc8498cc-tx48v] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-7d687f955b-xcps4: 10
rpcs_by_method:
UnaryCall:
psm-grpc-server-7d687f955b-xcps4: 10
metadatas_by_peer:
psm-grpc-server-7d687f955b-xcps4:
hostname:
- psm-grpc-server-7d687f955b-xcps4

I1016 18:46:52.475021 131645990072320 xds_k8s_testcase.py:316] --- Finished subTest AffinityTest.test_session_affinity_policy_with_service_target.05_send_RPCs_with_cookie ---
I1016 18:46:52.475148 131645990072320 xds_k8s_testcase.py:848] ----- TestMethod AffinityTest.test_session_affinity_policy_with_service_target teardown -----
I1016 18:46:52.606728 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 18:47:02.763037 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 18:47:02.763301 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1844-e79dc
I1016 18:47:32.857004 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1844-e79dc deleted
I1016 18:47:32.857216 131645990072320 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 18:47:32.929357 131645990072320 k8s_base_runner.py:835] GCPSessionAffinityPolicy ssa-policy deleted
I1016 18:47:32.929564 131645990072320 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 18:47:32.961825 131645990072320 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 18:47:32.993110 131645990072320 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 18:47:34.092621 131645990072320 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 18:47:34.092809 131645990072320 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1844-e79dc
I1016 18:47:34.172198 131645990072320 k8s_base_runner.py:923] Service psm-csm-20241016-1844-e79dc deleted
I1016 18:47:34.172391 131645990072320 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 18:47:35.281786 131645990072320 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 18:47:35.281972 131645990072320 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 18:47:45.427710 131645990072320 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 18:47:45.427977 131645990072320 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1844-e79dc
I1016 18:50:45.837796 131645990072320 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1844-e79dc deleted
I1016 18:50:45.838220 131645990072320 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 18:50:45.838466 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-z2i7g5jq:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1844-e79dc%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-z2i7g5jq%22;timeRange=2024-10-16T18:44:51.267514Z%2F2024-10-16T18:47:32.857184Z;cursorTimestamp=2024-10-16T18:44:58.608694Z?project=grpc-testing
I1016 18:50:45.838604 131645990072320 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-v2cyzk8i:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1844-e79dc%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-v2cyzk8i%22;timeRange=2024-10-16T18:44:43.591697Z%2F2024-10-16T18:50:45.838036Z;cursorTimestamp=2024-10-16T18:44:51.188800Z?project=grpc-testing
[ OK ] AffinityTest.test_session_affinity_policy_with_service_target
I1016 18:50:45.838873 131645990072320 base_testcase.py:67] ----- PSM Test Case PASSED: AffinityTest.test_session_affinity_policy_with_service_target -----

----------------------------------------------------------------------
Ran 3 tests in 1463.380s

OK
+ [18:50:46 UTC] Cmd finished: python
+ [18:50:46 UTC] Finished csm suite test: gamma.affinity_test

+ [18:50:46 UTC] Running csm suite test: gamma.affinity_session_drain_test
+ [18:50:46 UTC] Test driver flags for gamma.affinity_session_drain_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_session_drain_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_session_drain_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [18:50:46 UTC] python -m tests.gamma.affinity_session_drain_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_session_drain_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_session_drain_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 18:50:47.430692 134733162897408 xds_k8s_testcase.py:167] ----- Testing AffinitySessionDrainTest -----
I1016 18:50:47.430854 134733162897408 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 18:50:47.431118 134733162897408 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 18:50:47.444100 134733162897408 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] AffinitySessionDrainTest.test_session_drain
I1016 18:50:47.449834 134733162897408 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1850-pd0gu
I1016 18:50:47.813769 134733162897408 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.affinity_session_drain_test
I1016 18:50:47.815147 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.01_run_test_server ---
I1016 18:50:47.815276 134733162897408 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1850-pd0gu: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=3
I1016 18:50:47.815601 134733162897408 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1850-pd0gu%22;timeRange=2024-10-16T18:50:47.815430Z%2F2024-10-16T19:20:47.815430Z?project=grpc-testing
I1016 18:50:47.820657 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1850-pd0gu
labels:
name: psm-csm-server-20241016-1850-pd0gu
owner: xds-k8s-interop-test
...

I1016 18:50:47.894180 134733162897408 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1850-pd0gu created
I1016 18:50:47.896690 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:50:47.952122 134733162897408 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:50:47.954119 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1850-pd0gu
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:50:47.983267 134733162897408 k8s_base_runner.py:371] Service psm-csm-20241016-1850-pd0gu created
I1016 18:50:47.986046 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1850-pd0gu
namespace: psm-csm-server-20241016-1850-pd0gu
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:50:48.028208 134733162897408 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:50:48.046778 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c76q7xu7
owner: xds-k8s-interop-test
spec:
replicas: 3
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c76q7xu7
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-c76q7xu7
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/cpp-server:v1.62.x
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000
lifecycle:
preStop:
exec:
command:
- "/tmp/prestop-hook/grpcurl"
- "-vv"
- "--plaintext"
- "--import-path=/tmp/prestop-hook"
- "--proto=/tmp/prestop-hook/protos/grpc/testing/test.proto"
- "localhost:8080"
- "grpc.testing.HookService/Hook"
args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
- name: prestop-hook-volume
mountPath: /tmp/prestop-hook
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1850-pd0gu-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
- name: prestop-hook-init
image: us-docker.pkg.dev/grpc-testing/psm-interop/prestop-hook:v0.0.6
imagePullPolicy: Always
args: ["/tmp/prestop-hook"]
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- name: prestop-hook-volume
mountPath: /tmp/prestop-hook
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
- name: prestop-hook-volume
emptyDir:
medium: Memory
...

I1016 18:50:48.114425 134733162897408 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:50:48.114662 134733162897408 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 3 pod(s)
I1016 18:50:49.220291 134733162897408 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 3 pod(s): ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:50:49.220689 134733162897408 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-599c4c74f-ckkfn to start
I1016 18:50:52.338746 134733162897408 k8s_base_runner.py:1006] Pod psm-grpc-server-599c4c74f-ckkfn ready, IP: [{'ip': '10.56.6.71'}]
I1016 18:50:52.339030 134733162897408 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-599c4c74f-ckkfn to gamma.affinity_session_drain_test/test_app_logs/psm-csm-server-20241016-1850-pd0gu_psm-grpc-server-599c4c74f-ckkfn.log
I1016 18:50:52.339482 134733063259712 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1850-pd0gu] Starting log collection thread 134733063259712 for psm-grpc-server-599c4c74f-ckkfn
I1016 18:50:52.339714 134733162897408 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-599c4c74f-dp28l to start
I1016 18:50:52.395196 134733162897408 k8s_base_runner.py:1006] Pod psm-grpc-server-599c4c74f-dp28l ready, IP: [{'ip': '10.56.5.80'}]
I1016 18:50:52.395480 134733162897408 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-599c4c74f-dp28l to gamma.affinity_session_drain_test/test_app_logs/psm-csm-server-20241016-1850-pd0gu_psm-grpc-server-599c4c74f-dp28l.log
I1016 18:50:52.395930 134733052773952 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1850-pd0gu] Starting log collection thread 134733052773952 for psm-grpc-server-599c4c74f-dp28l
I1016 18:50:52.396129 134733162897408 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-599c4c74f-hwxn7 to start
I1016 18:50:52.445691 134733162897408 k8s_base_runner.py:1006] Pod psm-grpc-server-599c4c74f-hwxn7 ready, IP: [{'ip': '10.56.7.52'}]
I1016 18:50:52.445976 134733162897408 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-599c4c74f-hwxn7 to gamma.affinity_session_drain_test/test_app_logs/psm-csm-server-20241016-1850-pd0gu_psm-grpc-server-599c4c74f-hwxn7.log
I1016 18:50:52.446438 134733042288192 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1850-pd0gu] Starting log collection thread 134733042288192 for psm-grpc-server-599c4c74f-hwxn7
I1016 18:50:52.446675 134733162897408 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 3 available replica(s)
I1016 18:50:55.540294 134733162897408 k8s_base_runner.py:972] Deployment psm-grpc-server has 3 replicas available
I1016 18:50:55.540587 134733162897408 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1850-pd0gu
I1016 18:51:18.880369 134733162897408 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1850-pd0gu: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-2dd3a919, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:51:18.880602 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.01_run_test_server ---
I1016 18:51:18.880681 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.02_create_ssa_policies ---
I1016 18:51:18.883095 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/session_affinity_policy_route.yaml:
---
apiVersion: networking.gke.io/v1
kind: GCPSessionAffinityPolicy
metadata:
name: ssa-policy
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
owner: xds-k8s-interop-test
spec:
statefulGeneratedCookie:
cookieTtlSeconds: 50
targetRef:
name: route-httproute-psm-grpc-server
group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: psm-csm-server-20241016-1850-pd0gu
...

I1016 18:51:18.921107 134733162897408 k8s_base_runner.py:371] GCPSessionAffinityPolicy ssa-policy created
I1016 18:51:18.923903 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/backend_policy.yaml:
---
kind: GCPBackendPolicy
apiVersion: networking.gke.io/v1
metadata:
name: backend-policy
namespace: psm-csm-server-20241016-1850-pd0gu
labels:
owner: xds-k8s-interop-test
spec:
targetRef:
group: ""
kind: Service
name: psm-grpc-server
default:
connectionDraining:
drainingTimeoutSec: 600
...

I1016 18:51:18.963754 134733162897408 k8s_base_runner.py:371] GCPBackendPolicy backend-policy created
I1016 18:51:18.963974 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.02_create_ssa_policies ---
I1016 18:51:18.964044 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.03_start_test_client ---
I1016 18:51:18.964132 134733162897408 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1850-pd0gu: server_target=xds:///psm-csm-20241016-1850-pd0gu.psm-csm-server-20241016-1850-pd0gu.svc.cluster.local:8080 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 18:51:18.964329 134733162897408 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1850-pd0gu%22;timeRange=2024-10-16T18:51:18.964214Z%2F2024-10-16T19:21:18.964214Z?project=grpc-testing
I1016 18:51:18.966187 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1850-pd0gu
labels:
name: psm-csm-client-20241016-1850-pd0gu
owner: xds-k8s-interop-test
...

I1016 18:51:18.981577 134733162897408 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1850-pd0gu created
I1016 18:51:18.991472 134733162897408 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1850-pd0gu
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hes0pljq
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hes0pljq
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hes0pljq
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1850-pd0gu.psm-csm-server-20241016-1850-pd0gu.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:51:19.075813 134733162897408 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:51:19.076025 134733162897408 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:51:20.149824 134733162897408 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-647b5466cf-h6bxw']
I1016 18:51:20.150060 134733162897408 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-647b5466cf-h6bxw to start
I1016 18:51:22.234636 134733162897408 k8s_base_runner.py:1006] Pod psm-grpc-client-647b5466cf-h6bxw ready, IP: [{'ip': '10.56.8.16'}]
I1016 18:51:22.234924 134733162897408 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-647b5466cf-h6bxw to gamma.affinity_session_drain_test/test_app_logs/psm-csm-client-20241016-1850-pd0gu_psm-grpc-client-647b5466cf-h6bxw.log
I1016 18:51:22.235381 134733031802432 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1850-pd0gu] Starting log collection thread 134733031802432 for psm-grpc-client-647b5466cf-h6bxw
I1016 18:51:22.235670 134733162897408 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:51:26.341233 134733162897408 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:51:26.341706 134733162897408 client_app.py:294] [psm-grpc-client-647b5466cf-h6bxw] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:51:26.372590 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:51:36.392350 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:51:46.414342 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:51:56.433799 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:52:06.453754 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:52:22.479627 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:52:47.515598 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:53:12.552767 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:53:37.588221 134733162897408 client_app.py:322] [psm-grpc-client-647b5466cf-h6bxw] xDS control plane channel:
I1016 18:53:39.597039 134733162897408 client_app.py:333] [psm-grpc-client-647b5466cf-h6bxw] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 18:53:39.597303 134733162897408 client_app.py:304] [psm-grpc-client-647b5466cf-h6bxw] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 18:53:39.597692 134733162897408 client_app.py:254] [psm-grpc-client-647b5466cf-h6bxw] Waiting to report a READY channel to xds:///psm-csm-20241016-1850-pd0gu.psm-csm-server-20241016-1850-pd0gu.svc.cluster.local:8080
I1016 18:53:39.606956 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:53:39.607143 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:53:49.627918 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:53:49.628104 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:53:59.650484 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:53:59.650747 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:54:09.671520 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:54:09.671734 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:54:19.693646 134733162897408 client_app.py:381] [psm-grpc-client-647b5466cf-h6bxw] Server channel:
I1016 18:54:19.698475 134733162897408 client_app.py:394] [psm-grpc-client-647b5466cf-h6bxw] Found subchannel in state READY:
I1016 18:54:19.698765 134733162897408 client_app.py:265] [psm-grpc-client-647b5466cf-h6bxw] Channel to xds:///psm-csm-20241016-1850-pd0gu.psm-csm-server-20241016-1850-pd0gu.svc.cluster.local:8080 transitioned to state READY:
I1016 18:54:19.698905 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.03_start_test_client ---
I1016 18:54:19.698979 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.04_confirm_all_servers_receive_traffic ---
I1016 18:54:19.699305 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:54:19.699606 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 120, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:54:24.587725 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 120
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 120

I1016 18:54:24.588280 134733162897408 retryers.py:213] Retrying framework.xds_k8s_testcase.XdsKubernetesBaseTestCase._assertRpcsEventuallyGoToGivenServers in 5.0 seconds as it raised AssertionError: 'psm-grpc-server-599c4c74f-ckkfn' not found in {'psm-grpc-server-599c4c74f-hwxn7': 120} : Server psm-grpc-server-599c4c74f-ckkfn did not receive RPCs
I1016 18:54:29.593624 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:54:29.593955 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 120, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:54:34.471826 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 120
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 120

I1016 18:54:34.472151 134733162897408 retryers.py:213] Retrying framework.xds_k8s_testcase.XdsKubernetesBaseTestCase._assertRpcsEventuallyGoToGivenServers in 5.0 seconds as it raised AssertionError: 'psm-grpc-server-599c4c74f-ckkfn' not found in {'psm-grpc-server-599c4c74f-hwxn7': 120} : Server psm-grpc-server-599c4c74f-ckkfn did not receive RPCs
I1016 18:54:39.477530 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:54:39.477901 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 120, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:54:44.356766 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 120
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 120

I1016 18:54:44.357071 134733162897408 retryers.py:213] Retrying framework.xds_k8s_testcase.XdsKubernetesBaseTestCase._assertRpcsEventuallyGoToGivenServers in 5.0 seconds as it raised AssertionError: 'psm-grpc-server-599c4c74f-ckkfn' not found in {'psm-grpc-server-599c4c74f-hwxn7': 120} : Server psm-grpc-server-599c4c74f-ckkfn did not receive RPCs
I1016 18:54:49.362487 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:54:49.362871 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 120, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:54:54.238720 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 111
psm-grpc-server-599c4c74f-dp28l: 4
psm-grpc-server-599c4c74f-ckkfn: 5
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 111
psm-grpc-server-599c4c74f-dp28l: 4
psm-grpc-server-599c4c74f-ckkfn: 5

I1016 18:54:54.239026 134733162897408 affinity_session_drain_test.py:131] Confirmed all servers received traffic: ['psm-grpc-server-599c4c74f-ckkfn', 'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-hwxn7']
I1016 18:54:54.239119 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.04_confirm_all_servers_receive_traffic ---
I1016 18:54:54.239180 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.05_retrieve_cookie ---
I1016 18:54:54.239523 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:54:54.357769 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-dp28l: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-dp28l: 1

I1016 18:54:54.358309 134733162897408 retryers.py:213] Retrying framework.test_cases.session_affinity_mixin.SessionAffinityMixin._retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: No cookie header found
I1016 18:55:04.368816 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:04.482111 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 1

I1016 18:55:04.482464 134733162897408 retryers.py:213] Retrying framework.test_cases.session_affinity_mixin.SessionAffinityMixin._retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: No cookie header found
I1016 18:55:14.492916 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:14.616192 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 1
metadatas_by_peer:
psm-grpc-server-599c4c74f-hwxn7:
set-cookie:
- GSSA=MTAuNTYuNy41Mjo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:55:14.616539 134733162897408 affinity_session_drain_test.py:142] Chosen server: psm-grpc-server-599c4c74f-hwxn7, cookie: GSSA=MTAuNTYuNy41Mjo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
I1016 18:55:14.616637 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.05_retrieve_cookie ---
I1016 18:55:14.616699 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.06_only_chosen_server_receives_rpcs_with_cookie ---
I1016 18:55:14.616746 134733162897408 affinity_session_drain_test.py:145] Configuring client to send cookie GSSA=MTAuNTYuNy41Mjo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
I1016 18:55:14.617013 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNy41Mjo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50'}]}), wait_for_ready=True, timeout=5)
I1016 18:55:14.620950 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-hwxn7']
I1016 18:55:14.621160 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:18.714642 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 100

I1016 18:55:18.715021 134733162897408 affinity_session_drain_test.py:157] Confirmed all RPCs went to psm-grpc-server-599c4c74f-hwxn7
I1016 18:55:18.715119 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.06_only_chosen_server_receives_rpcs_with_cookie ---
I1016 18:55:18.715179 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.07_stopping_chosen_server ---
I1016 18:55:18.715282 134733162897408 k8s_base_runner.py:533] [ns/psm-csm-server-20241016-1850-pd0gu] Deleting pod psm-grpc-server-599c4c74f-hwxn7 with 0:10:00 (h:mm:ss) grace period.
I1016 18:55:18.789254 134733162897408 k8s_base_runner.py:560] [ns/psm-csm-server-20241016-1850-pd0gu] Pod psm-grpc-server-599c4c74f-hwxn7 deleted with 0:10:00 (h:mm:ss) grace period.
I1016 18:55:18.789445 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.07_stopping_chosen_server ---
I1016 18:55:18.789516 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.08_test_client_csds_shows_chosen_server_draining ---
I1016 18:55:18.796689 134733162897408 xds_k8s_testcase.py:650] << Found EDS endpoints: HEALTHY: ['10.56.5.80:8080', '10.56.6.71:8080', '10.56.7.52:8080'], DRAINING: []
I1016 18:55:28.814020 134733162897408 xds_k8s_testcase.py:650] << Found EDS endpoints: HEALTHY: ['10.56.5.80:8080', '10.56.6.71:8080'], DRAINING: ['10.56.7.52:8080']
I1016 18:55:28.814313 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.08_test_client_csds_shows_chosen_server_draining ---
I1016 18:55:28.814387 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.09_pinned_traffic_stays_on_draining_server ---
I1016 18:55:28.814700 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-hwxn7']
I1016 18:55:28.814851 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:32.902470 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-hwxn7: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-hwxn7: 100

I1016 18:55:32.902856 134733162897408 affinity_session_drain_test.py:177] Confirmed all RPCs with Cookie header are still sent to the chosen server while it's in the DRAINING state: psm-grpc-server-599c4c74f-hwxn7
I1016 18:55:32.902955 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.09_pinned_traffic_stays_on_draining_server ---
I1016 18:55:32.903011 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.10_new_traffic_not_sent_to_draining_server ---
I1016 18:55:32.903056 134733162897408 affinity_session_drain_test.py:184] Configuring client to send no cookies
I1016 18:55:32.903166 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL']}), wait_for_ready=True, timeout=5)
I1016 18:55:32.906570 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 120, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:37.812375 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-ckkfn: 42
psm-grpc-server-599c4c74f-dp28l: 44
psm-grpc-server-599c4c74f-jkpfv: 34
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-ckkfn: 42
psm-grpc-server-599c4c74f-dp28l: 44
psm-grpc-server-599c4c74f-jkpfv: 34
metadatas_by_peer:
psm-grpc-server-599c4c74f-jkpfv:
set-cookie:
- GSSA=MTAuNTYuNy41Mzo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
psm-grpc-server-599c4c74f-dp28l:
set-cookie:
- GSSA=MTAuNTYuNS44MDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
psm-grpc-server-599c4c74f-ckkfn:
set-cookie:
- GSSA=MTAuNTYuNi43MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:55:37.812667 134733162897408 affinity_session_drain_test.py:200] Confirmed draining server psm-grpc-server-599c4c74f-hwxn7 received no new traffic, while {'psm-grpc-server-599c4c74f-dp28l', 'psm-grpc-server-599c4c74f-ckkfn'} did
I1016 18:55:37.812754 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.10_new_traffic_not_sent_to_draining_server ---
I1016 18:55:37.812817 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.11_chosen_server_release_prestop ---
I1016 18:55:37.812867 134733162897408 affinity_session_drain_test.py:208] Releasing prestop hook on, psm-grpc-server-599c4c74f-hwxn7
I1016 18:55:37.813237 134733162897408 grpc.py:75] [psm-grpc-server-599c4c74f-hwxn7:8080] >> RPC HookService.SetReturnStatus(request=SetReturnStatusRequest({}), timeout=10, wait_for_ready=True)
I1016 18:55:37.819620 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.11_chosen_server_release_prestop ---
I1016 18:55:37.819798 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.12_confirm_chosen_server_stopped ---
I1016 18:55:37.819864 134733162897408 affinity_session_drain_test.py:212] Waiting on the chosen server to be deleted
I1016 18:55:40.093280 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 30, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:41.383745 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-ckkfn: 11
psm-grpc-server-599c4c74f-dp28l: 11
psm-grpc-server-599c4c74f-jkpfv: 8
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-ckkfn: 11
psm-grpc-server-599c4c74f-dp28l: 11
psm-grpc-server-599c4c74f-jkpfv: 8
metadatas_by_peer:
psm-grpc-server-599c4c74f-ckkfn:
set-cookie:
- GSSA=MTAuNTYuNi43MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
psm-grpc-server-599c4c74f-dp28l:
set-cookie:
- GSSA=MTAuNTYuNS44MDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
psm-grpc-server-599c4c74f-jkpfv:
set-cookie:
- GSSA=MTAuNTYuNy41Mzo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:55:41.383985 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.12_confirm_chosen_server_stopped ---
I1016 18:55:41.384057 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.13_get_new_cookie_and_new_server ---
I1016 18:55:41.384419 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:41.500879 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-jkpfv: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-jkpfv: 1
metadatas_by_peer:
psm-grpc-server-599c4c74f-jkpfv:
set-cookie:
- GSSA=MTAuNTYuNy41Mzo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:55:41.501495 134733162897408 retryers.py:213] Retrying framework.test_cases.session_affinity_mixin.SessionAffinityMixin._retrieve_cookie_and_server in 10.0 seconds as it raised AssertionError: [] has length of 0, expected 1.
I1016 18:55:51.511944 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:51.626198 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-ckkfn: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-ckkfn: 1
metadatas_by_peer:
psm-grpc-server-599c4c74f-ckkfn:
set-cookie:
- GSSA=MTAuNTYuNi43MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4;
HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50

I1016 18:55:51.626524 134733162897408 affinity_session_drain_test.py:236] New chosen server: psm-grpc-server-599c4c74f-ckkfn, cookie: GSSA=MTAuNTYuNi43MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
I1016 18:55:51.626612 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.13_get_new_cookie_and_new_server ---
I1016 18:55:51.626678 134733162897408 xds_k8s_testcase.py:311] --- Starting subTest AffinitySessionDrainTest.test_session_drain.14_new_chosen_server_receives_rpcs_with_cookie ---
I1016 18:55:51.626730 134733162897408 affinity_session_drain_test.py:241] Configuring client to send cookie GSSA=MTAuNTYuNy41Mjo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50
I1016 18:55:51.626883 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNi43MTo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF81MDIxNjgxNzI3MDE0MDI0NjM4; HttpOnly; Path=/grpc.testing.TestService/UnaryCall; Max-Age=50'}]}), wait_for_ready=True, timeout=5)
I1016 18:55:51.632185 134733162897408 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-599c4c74f-ckkfn']
I1016 18:55:51.632442 134733162897408 grpc.py:75] [psm-grpc-client-647b5466cf-h6bxw:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200, 'metadataKeys': ['set-cookie']}), timeout=1200, wait_for_ready=True)
I1016 18:55:55.730887 134733162897408 xds_k8s_testcase.py:683] [psm-grpc-client-647b5466cf-h6bxw] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-599c4c74f-ckkfn: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-599c4c74f-ckkfn: 100

I1016 18:55:55.731275 134733162897408 affinity_session_drain_test.py:253] Confirmed all RPCs went to the new chosen server: psm-grpc-server-599c4c74f-ckkfn
I1016 18:55:55.731375 134733162897408 xds_k8s_testcase.py:316] --- Finished subTest AffinitySessionDrainTest.test_session_drain.14_new_chosen_server_receives_rpcs_with_cookie ---
I1016 18:55:55.731478 134733162897408 xds_k8s_testcase.py:848] ----- TestMethod AffinitySessionDrainTest.test_session_drain teardown -----
I1016 18:55:55.850993 134733162897408 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 18:56:06.020143 134733162897408 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 18:56:06.020428 134733162897408 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1850-pd0gu
I1016 18:56:36.210207 134733162897408 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1850-pd0gu deleted
I1016 18:56:36.210463 134733162897408 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 18:56:46.295799 134733162897408 k8s_base_runner.py:835] GCPSessionAffinityPolicy ssa-policy deleted
I1016 18:56:46.295989 134733162897408 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 18:56:46.326590 134733162897408 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 18:56:46.404618 134733162897408 k8s_base_runner.py:889] GCPBackendPolicy backend-policy deleted
I1016 18:56:46.404806 134733162897408 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 18:56:46.512071 134733162897408 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 18:56:46.512255 134733162897408 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1850-pd0gu
I1016 18:56:46.595248 134733162897408 k8s_base_runner.py:923] Service psm-csm-20241016-1850-pd0gu deleted
I1016 18:56:46.595459 134733162897408 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 18:56:47.681867 134733162897408 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 18:56:47.682048 134733162897408 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 18:56:47.682137 134733162897408 k8s_xds_server_runner.py:388] [ns/psm-csm-server-20241016-1850-pd0gu] Releasing pre-stop hook on known server pods of deployment psm-grpc-server, deployment_id=psm-grpc-server-c76q7xu7
I1016 18:56:47.682281 134733162897408 grpc.py:75] [psm-grpc-server-599c4c74f-hwxn7:8080] >> RPC HookService.SetReturnStatus(request=SetReturnStatusRequest({}), wait_for_ready=True, timeout=5)
I1016 18:56:52.683173 134733162897408 k8s_xds_server_runner.py:402] [ns/psm-csm-server-20241016-1850-pd0gu] Pre-stop hook release not executed on pod marked for deletion: psm-grpc-server-599c4c74f-hwxn7 (this is normal)
I1016 18:56:52.683772 134733162897408 grpc.py:75] [psm-grpc-server-599c4c74f-ckkfn:8080] >> RPC HookService.SetReturnStatus(request=SetReturnStatusRequest({}), timeout=10, wait_for_ready=True)
I1016 18:56:52.688039 134733162897408 grpc.py:75] [psm-grpc-server-599c4c74f-dp28l:8080] >> RPC HookService.SetReturnStatus(request=SetReturnStatusRequest({}), timeout=10, wait_for_ready=True)
I1016 18:57:32.949129 134733162897408 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 18:57:32.949389 134733162897408 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1850-pd0gu
I1016 18:59:03.207359 134733162897408 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1850-pd0gu deleted
I1016 18:59:03.207823 134733162897408 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 18:59:03.208062 134733162897408 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-hes0pljq:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1850-pd0gu%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-hes0pljq%22;timeRange=2024-10-16T18:51:18.964207Z%2F2024-10-16T18:56:36.210426Z;cursorTimestamp=2024-10-16T18:51:26.341488Z?project=grpc-testing
I1016 18:59:03.208212 134733162897408 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-c76q7xu7:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1850-pd0gu%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-c76q7xu7%22;timeRange=2024-10-16T18:50:47.815423Z%2F2024-10-16T18:59:03.207631Z;cursorTimestamp=2024-10-16T18:50:55.540521Z?project=grpc-testing
[ OK ] AffinitySessionDrainTest.test_session_drain
I1016 18:59:03.208523 134733162897408 base_testcase.py:67] ----- PSM Test Case PASSED: AffinitySessionDrainTest.test_session_drain -----

----------------------------------------------------------------------
Ran 1 test in 495.778s

OK
+ [18:59:03 UTC] Cmd finished: python
+ [18:59:03 UTC] Finished csm suite test: gamma.affinity_session_drain_test

+ [18:59:03 UTC] Running csm suite test: gamma.csm_observability_test
+ [18:59:03 UTC] Test driver flags for gamma.csm_observability_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.csm_observability_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.csm_observability_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [18:59:03 UTC] python -m tests.gamma.csm_observability_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.csm_observability_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.csm_observability_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 18:59:04.754864 124906666532864 xds_k8s_testcase.py:167] ----- Testing CsmObservabilityTest -----
I1016 18:59:04.755031 124906666532864 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 18:59:04.755304 124906666532864 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 18:59:04.767676 124906666532864 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] CsmObservabilityTest.test_csm_observability
I1016 18:59:04.795223 124906666532864 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1859-o0sc5
I1016 18:59:05.204696 124906666532864 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/gamma.csm_observability_test
I1016 18:59:05.205874 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.1_run_test_server ---
I1016 18:59:05.205999 124906666532864 gamma_server_runner.py:132] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1859-o0sc5: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=1
I1016 18:59:05.206219 124906666532864 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1859-o0sc5%22;timeRange=2024-10-16T18:59:05.206069Z%2F2024-10-16T19:29:05.206069Z?project=grpc-testing
I1016 18:59:05.211243 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1859-o0sc5
labels:
name: psm-csm-server-20241016-1859-o0sc5
owner: xds-k8s-interop-test
...

I1016 18:59:05.286316 124906666532864 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1859-o0sc5 created
I1016 18:59:05.288890 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1859-o0sc5
labels:
owner: xds-k8s-interop-test
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 18:59:05.348685 124906666532864 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 18:59:05.350723 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/frontend_service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-csm-20241016-1859-o0sc5
namespace: psm-csm-server-20241016-1859-o0sc5
labels:
owner: xds-k8s-interop-test
spec:
ports:
- port: 8080
targetPort: 8080

I1016 18:59:05.397744 124906666532864 k8s_base_runner.py:371] Service psm-csm-20241016-1859-o0sc5 created
I1016 18:59:05.400582 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/gamma/route_http.yaml:
---
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: route-httproute-psm-grpc-server
namespace: psm-csm-server-20241016-1859-o0sc5
labels:
owner: xds-k8s-interop-test
spec:
parentRefs:
- name: psm-csm-20241016-1859-o0sc5
namespace: psm-csm-server-20241016-1859-o0sc5
group: ""
kind: Service
rules:
- matches:
- path:
type: Exact
value: /grpc.testing.TestService/UnaryCall
backendRefs:
- name: psm-grpc-server
port: 8080
...

I1016 18:59:05.446166 124906666532864 k8s_base_runner.py:371] HTTPRoute route-httproute-psm-grpc-server created
I1016 18:59:05.462947 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1859-o0sc5
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-kueor9ce
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-kueor9ce
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-kueor9ce
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
- "--enable_csm_observability=true"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
- name: CSM_WORKLOAD_NAME
value: csm_workload_name_from_server
- name: CSM_CANONICAL_SERVICE_NAME
value: csm_canonical_service_name_from_server
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=$(NAMESPACE_NAME)
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1859-o0sc5-psm-grpc-server"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 18:59:05.526655 124906666532864 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 18:59:05.529031 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/csm/pod-monitoring.yaml:
---
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
namespace: psm-csm-server-20241016-1859-o0sc5
name: psm-grpc-server-kueor9ce-gmp
labels:
owner: xds-k8s-interop-test
spec:
selector:
matchLabels:
deployment_id: psm-grpc-server-kueor9ce
endpoints:
- port: 9464
interval: 10s

I1016 18:59:05.722627 124906666532864 k8s_base_runner.py:371] PodMonitoring psm-grpc-server-kueor9ce-gmp created
I1016 18:59:05.722835 124906666532864 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I1016 18:59:05.792563 124906666532864 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-568b55dfd9-649s7']
I1016 18:59:05.792809 124906666532864 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-568b55dfd9-649s7 to start
I1016 18:59:09.006770 124906666532864 k8s_base_runner.py:1006] Pod psm-grpc-server-568b55dfd9-649s7 ready, IP: [{'ip': '10.56.5.81'}]
I1016 18:59:09.007050 124906666532864 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-568b55dfd9-649s7 to gamma.csm_observability_test/test_app_logs/psm-csm-server-20241016-1859-o0sc5_psm-grpc-server-568b55dfd9-649s7.log
I1016 18:59:09.007474 124906004022848 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1859-o0sc5] Starting log collection thread 124906004022848 for psm-grpc-server-568b55dfd9-649s7
I1016 18:59:09.007661 124906666532864 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I1016 18:59:12.130434 124906666532864 k8s_base_runner.py:972] Deployment psm-grpc-server has 1 replicas available
I1016 18:59:12.130718 124906666532864 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1859-o0sc5
I1016 18:59:18.248172 124906666532864 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1859-o0sc5: neg_name=k8s1-58b57ac6-psm-csm-server-20241016-1-psm-grpc-se-80-8f168a25, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 18:59:18.248366 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.1_run_test_server ---
I1016 18:59:18.248452 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.2_start_test_client ---
I1016 18:59:18.248530 124906666532864 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1859-o0sc5: server_target=xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080 rpc=UnaryCall qps=1 metadata='' secure_mode=False print_response=False
I1016 18:59:18.248721 124906666532864 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1859-o0sc5%22;timeRange=2024-10-16T18:59:18.248613Z%2F2024-10-16T19:29:18.248613Z?project=grpc-testing
I1016 18:59:18.250585 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1859-o0sc5
labels:
name: psm-csm-client-20241016-1859-o0sc5
owner: xds-k8s-interop-test
...

I1016 18:59:18.277016 124906666532864 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1859-o0sc5 created
I1016 18:59:18.287634 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1859-o0sc5
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c60r0489
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c60r0489
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c60r0489
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080"
- "--stats_port=8079"
- "--qps=1"
- "--rpc=UnaryCall"
- "--metadata="
- "--request_payload_size=27182"
- "--response_payload_size=31415"
- "--print_response=False"
- "--enable_csm_observability=true"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
- name: CSM_WORKLOAD_NAME
value: csm_workload_name_from_client
- name: CSM_CANONICAL_SERVICE_NAME
value: csm_canonical_service_name_from_client
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=$(NAMESPACE_NAME)
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--generate-mesh-id-experimental"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 18:59:18.343231 124906666532864 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 18:59:18.345607 124906666532864 k8s_base_runner.py:351] Rendered template kubernetes-manifests/csm/pod-monitoring.yaml:
---
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
namespace: psm-csm-client-20241016-1859-o0sc5
name: psm-grpc-client-c60r0489-gmp
labels:
owner: xds-k8s-interop-test
spec:
selector:
matchLabels:
deployment_id: psm-grpc-client-c60r0489
endpoints:
- port: 9464
interval: 10s

I1016 18:59:18.449002 124906666532864 k8s_base_runner.py:371] PodMonitoring psm-grpc-client-c60r0489-gmp created
I1016 18:59:18.449215 124906666532864 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 18:59:19.537912 124906666532864 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-8cf7f8b7b-442w8']
I1016 18:59:19.538156 124906666532864 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-8cf7f8b7b-442w8 to start
I1016 18:59:21.625469 124906666532864 k8s_base_runner.py:1006] Pod psm-grpc-client-8cf7f8b7b-442w8 ready, IP: [{'ip': '10.56.8.17'}]
I1016 18:59:21.625801 124906666532864 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-8cf7f8b7b-442w8 to gamma.csm_observability_test/test_app_logs/psm-csm-client-20241016-1859-o0sc5_psm-grpc-client-8cf7f8b7b-442w8.log
I1016 18:59:21.626253 124905983051328 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1859-o0sc5] Starting log collection thread 124905983051328 for psm-grpc-client-8cf7f8b7b-442w8
I1016 18:59:21.626531 124906666532864 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 18:59:25.871219 124906666532864 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 18:59:25.871641 124906666532864 client_app.py:294] [psm-grpc-client-8cf7f8b7b-442w8] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 18:59:25.894885 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 18:59:35.917841 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 18:59:45.939583 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 18:59:55.960705 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 19:00:05.981328 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 19:00:22.008254 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 19:00:47.044252 124906666532864 client_app.py:322] [psm-grpc-client-8cf7f8b7b-442w8] xDS control plane channel:
I1016 19:00:49.053089 124906666532864 client_app.py:333] [psm-grpc-client-8cf7f8b7b-442w8] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 19:00:49.053358 124906666532864 client_app.py:304] [psm-grpc-client-8cf7f8b7b-442w8] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 19:00:49.053737 124906666532864 client_app.py:254] [psm-grpc-client-8cf7f8b7b-442w8] Waiting to report a READY channel to xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080
I1016 19:00:49.063054 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:00:49.063235 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:00:59.083575 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:00:59.083767 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:09.104517 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:09.104727 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:19.123529 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:19.123740 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:29.144828 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:29.145018 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:45.172978 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:45.173165 124906666532864 client_app.py:381] [psm-grpc-client-8cf7f8b7b-442w8] Server channel:
I1016 19:01:45.178750 124906666532864 client_app.py:394] [psm-grpc-client-8cf7f8b7b-442w8] Found subchannel in state READY:
I1016 19:01:45.178995 124906666532864 client_app.py:265] [psm-grpc-client-8cf7f8b7b-442w8] Channel to xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080 transitioned to state READY:
I1016 19:01:45.179128 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.2_start_test_client ---
I1016 19:01:45.179192 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.3_test_server_received_rpcs_from_test_client ---
I1016 19:01:45.179433 124906666532864 grpc.py:75] [psm-grpc-client-8cf7f8b7b-442w8:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), wait_for_ready=True, timeout=1200)
I1016 19:03:27.160688 124906666532864 xds_k8s_testcase.py:683] [psm-grpc-client-8cf7f8b7b-442w8] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-568b55dfd9-649s7: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-568b55dfd9-649s7: 100

I1016 19:03:27.160953 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.3_test_server_received_rpcs_from_test_client ---
I1016 19:03:27.161026 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.4_export_prometheus_metrics_data ---
I1016 19:03:27.161089 124906666532864 csm_observability_test.py:229] Letting test client run for 90 seconds to produce metric data
I1016 19:04:57.464837 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.4_export_prometheus_metrics_data ---
I1016 19:04:57.465062 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.5_query_cloud_monitoring_metrics ---
I1016 19:04:57.465276 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram
I1016 19:04:57.529741 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram:
name: prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-568b55dfd9-649s7
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-568b55dfd9-649s7:9464
job: psm-grpc-server-kueor9ce-gmp
location: us-central1
namespace: psm-csm-server-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.529913 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram
I1016 19:04:57.597172 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-568b55dfd9-649s7
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-568b55dfd9-649s7:9464
job: psm-grpc-server-kueor9ce-gmp
location: us-central1
namespace: psm-csm-server-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.597337 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram
I1016 19:04:57.656996 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-568b55dfd9-649s7
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-568b55dfd9-649s7:9464
job: psm-grpc-server-kueor9ce-gmp
location: us-central1
namespace: psm-csm-server-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.657191 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram
I1016 19:04:57.711471 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-grpc-server
csm_service_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080
pod: psm-grpc-client-8cf7f8b7b-442w8
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-8cf7f8b7b-442w8:9464
job: psm-grpc-client-c60r0489-gmp
location: us-central1
namespace: psm-csm-client-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.711641 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram
I1016 19:04:57.770946 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-grpc-server
csm_service_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080
pod: psm-grpc-client-8cf7f8b7b-442w8
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-8cf7f8b7b-442w8:9464
job: psm-grpc-client-c60r0489-gmp
location: us-central1
namespace: psm-csm-client-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.771115 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram
I1016 19:04:57.814042 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-grpc-server
csm_service_namespace_name: psm-csm-server-20241016-1859-o0sc5
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080
pod: psm-grpc-client-8cf7f8b7b-442w8
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-8cf7f8b7b-442w8:9464
job: psm-grpc-client-c60r0489-gmp
location: us-central1
namespace: psm-csm-client-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.814239 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_started_total/counter
I1016 19:04:57.880684 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_server_call_started_total/counter:
name: prometheus.googleapis.com/grpc_server_call_started_total/counter
resource_type: prometheus_target
metric_labels:
grpc_method: grpc.testing.TestService/UnaryCall
pod: psm-grpc-server-568b55dfd9-649s7
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-568b55dfd9-649s7:9464
job: psm-grpc-server-kueor9ce-gmp
location: us-central1
namespace: psm-csm-server-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.880883 124906666532864 csm_observability_test.py:518] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_started_total/counter
I1016 19:04:57.933841 124906666532864 csm_observability_test.py:538] Metric prometheus.googleapis.com/grpc_client_attempt_started_total/counter:
name: prometheus.googleapis.com/grpc_client_attempt_started_total/counter
resource_type: prometheus_target
metric_labels:
grpc_method: grpc.testing.TestService/UnaryCall
grpc_target: xds:///psm-csm-20241016-1859-o0sc5.psm-csm-server-20241016-1859-o0sc5.svc.cluster.local:8080
pod: psm-grpc-client-8cf7f8b7b-442w8
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-8cf7f8b7b-442w8:9464
job: psm-grpc-client-c60r0489-gmp
location: us-central1
namespace: psm-csm-client-20241016-1859-o0sc5
project_id: grpc-testing

I1016 19:04:57.934083 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.5_query_cloud_monitoring_metrics ---
I1016 19:04:57.934164 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.6_check_metrics_time_series ---
I1016 19:04:57.934226 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.6_check_metrics_time_series ---
I1016 19:04:57.934273 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.7_check_metrics_labels_histogram_client ---
I1016 19:04:57.934363 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.7_check_metrics_labels_histogram_client ---
I1016 19:04:57.934426 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.8_check_metrics_labels_histogram_server ---
I1016 19:04:57.934492 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.8_check_metrics_labels_histogram_server ---
I1016 19:04:57.934535 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.9_check_metrics_labels_counter_client ---
I1016 19:04:57.934588 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.9_check_metrics_labels_counter_client ---
I1016 19:04:57.934634 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.10_check_metrics_labels_counter_server ---
I1016 19:04:57.934679 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.10_check_metrics_labels_counter_server ---
I1016 19:04:57.934725 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.11_check_client_resource_labels_client ---
I1016 19:04:57.934799 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.11_check_client_resource_labels_client ---
I1016 19:04:57.934845 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.12_check_server_resource_labels_server ---
I1016 19:04:57.934906 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.12_check_server_resource_labels_server ---
I1016 19:04:57.934948 124906666532864 xds_k8s_testcase.py:311] --- Starting subTest CsmObservabilityTest.test_csm_observability.13_check_bytes_sent_vs_data_points ---
I1016 19:04:57.935111 124906666532864 xds_k8s_testcase.py:316] --- Finished subTest CsmObservabilityTest.test_csm_observability.13_check_bytes_sent_vs_data_points ---
I1016 19:04:57.935270 124906666532864 xds_k8s_testcase.py:848] ----- TestMethod CsmObservabilityTest.test_csm_observability teardown -----
I1016 19:04:58.043743 124906666532864 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 19:05:08.211549 124906666532864 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 19:05:08.211810 124906666532864 k8s_base_runner.py:861] Deleting PodMonitoring psm-grpc-client-c60r0489-gmp
I1016 19:05:08.264500 124906666532864 k8s_base_runner.py:872] PodMonitoring psm-grpc-client-c60r0489-gmp deleted
I1016 19:05:08.264763 124906666532864 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1859-o0sc5
I1016 19:05:38.440177 124906666532864 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1859-o0sc5 deleted
I1016 19:05:38.440387 124906666532864 k8s_base_runner.py:815] Deleting GCPSessionAffinityPolicy ssa-policy
I1016 19:05:38.469122 124906666532864 k8s_base_runner.py:838] Deleting GCPSessionAffinityFilter ssa-filter
I1016 19:05:38.528078 124906666532864 k8s_base_runner.py:875] Deleting GCPBackendPolicy backend-policy
I1016 19:05:38.607234 124906666532864 k8s_base_runner.py:800] Deleting HTTPRoute route-httproute-psm-grpc-server
I1016 19:05:40.104055 124906666532864 k8s_base_runner.py:812] HTTPRoute route-httproute-psm-grpc-server deleted
I1016 19:05:40.104237 124906666532864 k8s_base_runner.py:910] Deleting service psm-csm-20241016-1859-o0sc5
I1016 19:05:40.214374 124906666532864 k8s_base_runner.py:923] Service psm-csm-20241016-1859-o0sc5 deleted
I1016 19:05:40.214615 124906666532864 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 19:05:41.348718 124906666532864 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 19:05:41.348902 124906666532864 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 19:05:51.607096 124906666532864 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 19:05:51.607342 124906666532864 k8s_base_runner.py:861] Deleting PodMonitoring psm-grpc-server-kueor9ce-gmp
I1016 19:05:51.654106 124906666532864 k8s_base_runner.py:872] PodMonitoring psm-grpc-server-kueor9ce-gmp deleted
I1016 19:05:51.654300 124906666532864 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1859-o0sc5
I1016 19:07:21.871704 124906666532864 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1859-o0sc5 deleted
I1016 19:07:21.872111 124906666532864 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 19:07:21.872328 124906666532864 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-c60r0489:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1859-o0sc5%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-c60r0489%22;timeRange=2024-10-16T18:59:18.248606Z%2F2024-10-16T19:05:38.440357Z;cursorTimestamp=2024-10-16T18:59:25.871433Z?project=grpc-testing
I1016 19:07:21.872476 124906666532864 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-kueor9ce:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1859-o0sc5%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-kueor9ce%22;timeRange=2024-10-16T18:59:05.206063Z%2F2024-10-16T19:07:21.871924Z;cursorTimestamp=2024-10-16T18:59:12.130655Z?project=grpc-testing
[ OK ] CsmObservabilityTest.test_csm_observability
I1016 19:07:21.872744 124906666532864 base_testcase.py:67] ----- PSM Test Case PASSED: CsmObservabilityTest.test_csm_observability -----

----------------------------------------------------------------------
Ran 1 test in 497.119s

OK
+ [19:07:22 UTC] Cmd finished: python
+ [19:07:22 UTC] Finished csm suite test: gamma.csm_observability_test

+ [19:07:22 UTC] Running csm suite test: app_net_ssa_test
+ [19:07:22 UTC] Test driver flags for app_net_ssa_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_ssa_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_ssa_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [19:07:22 UTC] python -m tests.app_net_ssa_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_ssa_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_ssa_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 19:07:23.400389 123829059989504 xds_k8s_testcase.py:167] ----- Testing AppNetSsaTest -----
I1016 19:07:23.400572 123829059989504 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 19:07:23.400857 123829059989504 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 19:07:23.412939 123829059989504 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] AppNetSsaTest.test_session_affinity_policy
I1016 19:07:23.418683 123829059989504 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1907-o21di
I1016 19:07:23.907321 123829059989504 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_ssa_test
I1016 19:07:24.110959 123829059989504 xds_k8s_testcase.py:824] Found unused xds port: 46335
I1016 19:07:24.111299 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.0_create_health_check ---
I1016 19:07:24.111392 123829059989504 traffic_director.py:226] Creating GRPC Health Check "psm-csm-health-check-20241016-1907-o21di"
I1016 19:07:24.115580 123829059989504 compute.py:584] Creating compute resource:
---
grpcHealthCheck:
portSpecification: USE_SERVING_PORT
name: psm-csm-health-check-20241016-1907-o21di
type: GRPC
...

I1016 19:07:24.566905 123829059989504 compute.py:661] Waiting 600 sec for compute operation id: operation-1729105644186-6249cc7b36d25-11ce836b-e5f54129
I1016 19:07:26.890274 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.0_create_health_check ---
I1016 19:07:26.890478 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.1_create_backend_service ---
I1016 19:07:26.890570 123829059989504 traffic_director.py:253] Creating GRPC Backend Service "psm-csm-backend-service-20241016-1907-o21di"
I1016 19:07:26.900316 123829059989504 compute.py:584] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-csm-health-check-20241016-1907-o21di
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-csm-backend-service-20241016-1907-o21di
protocol: GRPC
...

I1016 19:07:27.329495 123829059989504 compute.py:661] Waiting 600 sec for compute operation id: operation-1729105646902-6249cc7dcdd54-b8516f72-99823056
I1016 19:07:42.606976 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.1_create_backend_service ---
I1016 19:07:42.607142 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.2_create_mesh ---
I1016 19:07:42.607211 123829059989504 traffic_director.py:947] Creating Mesh psm-csm-mesh-20241016-1907-o21di
I1016 19:07:42.608493 123829059989504 api.py:529] Creating networkservices resource:
--- {}
...

I1016 19:07:42.853113 123829059989504 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729105662719-6249cc8ce3502-8e950a8d-4c20a7d4
I1016 19:07:47.081238 123829059989504 api.py:551] Loaded projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1907-o21di:
---
createTime: '2024-10-16T19:07:42.755812708Z'
name: projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1907-o21di
selfLink: https://networkservices.googleapis.com/v1alpha1/projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1907-o21di
updateTime: '2024-10-16T19:07:46.145868012Z'
...

I1016 19:07:47.081490 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.2_create_mesh ---
I1016 19:07:47.081575 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.3_create_http_route ---
I1016 19:07:47.081645 123829059989504 traffic_director.py:994] Creating HttpRoute psm-csm-http-route-20241016-1907-o21di
I1016 19:07:47.085965 123829059989504 api.py:529] Creating networkservices resource:
---
hostnames:
- psm-grpc-server:46335
meshes:
- projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1907-o21di
rules:
- action:
destinations:
- serviceName: projects/grpc-testing/locations/global/backendServices/psm-csm-backend-service-20241016-1907-o21di
weight: 1
statefulSessionAffinity:
cookieTtl: 50s
...

I1016 19:07:47.453248 123829059989504 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729105667125-6249cc911724e-cf493949-11973f5e
I1016 19:07:53.787731 123829059989504 api.py:551] Loaded projects/grpc-testing/locations/global/httpRoutes/psm-csm-http-route-20241016-1907-o21di:
---
createTime: '2024-10-16T19:07:47.232718431Z'
hostnames:
- psm-grpc-server:46335
meshes:
- projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1907-o21di
name: projects/grpc-testing/locations/global/httpRoutes/psm-csm-http-route-20241016-1907-o21di
rules:
- action:
destinations:
- serviceName: projects/grpc-testing/locations/global/backendServices/psm-csm-backend-service-20241016-1907-o21di
weight: 1
statefulSessionAffinity:
cookieTtl: 50s
selfLink: https://networkservices.googleapis.com/v1alpha1/projects/grpc-testing/locations/global/httpRoutes/psm-csm-http-route-20241016-1907-o21di
updateTime: '2024-10-16T19:07:53.279013859Z'
...

I1016 19:07:53.787945 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.3_create_http_route ---
I1016 19:07:53.788012 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.4_run_test_server ---
I1016 19:07:53.788088 123829059989504 k8s_xds_server_runner.py:201] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1907-o21di: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=3
I1016 19:07:53.788332 123829059989504 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1907-o21di%22;timeRange=2024-10-16T19:07:53.788172Z%2F2024-10-16T19:37:53.788172Z?project=grpc-testing
I1016 19:07:53.793369 123829059989504 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1907-o21di
labels:
name: psm-csm-server-20241016-1907-o21di
owner: xds-k8s-interop-test
...

I1016 19:07:53.873603 123829059989504 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1907-o21di created
I1016 19:07:53.877035 123829059989504 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1907-o21di
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-csm-server-20241016-1907-o21di-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 19:07:53.939661 123829059989504 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 19:07:53.939835 123829059989504 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1907-o21di
I1016 19:07:54.071734 123829059989504 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1907-o21di: neg_name=psm-csm-server-20241016-1907-o21di-psm-grpc-server, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 19:07:54.087279 123829059989504 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1907-o21di
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jccram98
owner: xds-k8s-interop-test
spec:
replicas: 3
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jccram98
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jccram98
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1907-o21di-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 19:07:54.161229 123829059989504 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 19:07:54.161470 123829059989504 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 3 pod(s)
I1016 19:07:55.298567 123829059989504 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 3 pod(s): ['psm-grpc-server-5467bc46b4-56wgx', 'psm-grpc-server-5467bc46b4-bhnhf', 'psm-grpc-server-5467bc46b4-zb24s']
I1016 19:07:55.298949 123829059989504 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5467bc46b4-56wgx to start
I1016 19:07:57.414297 123829059989504 k8s_base_runner.py:1006] Pod psm-grpc-server-5467bc46b4-56wgx ready, IP: [{'ip': '10.56.5.82'}]
I1016 19:07:57.414609 123829059989504 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5467bc46b4-56wgx to app_net_ssa_test/test_app_logs/psm-csm-server-20241016-1907-o21di_psm-grpc-server-5467bc46b4-56wgx.log
I1016 19:07:57.415044 123828961281600 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1907-o21di] Starting log collection thread 123828961281600 for psm-grpc-server-5467bc46b4-56wgx
I1016 19:07:57.415247 123829059989504 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5467bc46b4-bhnhf to start
I1016 19:07:57.472901 123829059989504 k8s_base_runner.py:1006] Pod psm-grpc-server-5467bc46b4-bhnhf ready, IP: [{'ip': '10.56.7.54'}]
I1016 19:07:57.473162 123829059989504 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5467bc46b4-bhnhf to app_net_ssa_test/test_app_logs/psm-csm-server-20241016-1907-o21di_psm-grpc-server-5467bc46b4-bhnhf.log
I1016 19:07:57.473529 123828950795840 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1907-o21di] Starting log collection thread 123828950795840 for psm-grpc-server-5467bc46b4-bhnhf
I1016 19:07:57.473742 123829059989504 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-5467bc46b4-zb24s to start
I1016 19:07:58.548033 123829059989504 k8s_base_runner.py:1006] Pod psm-grpc-server-5467bc46b4-zb24s ready, IP: [{'ip': '10.56.6.72'}]
I1016 19:07:58.548309 123829059989504 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-5467bc46b4-zb24s to app_net_ssa_test/test_app_logs/psm-csm-server-20241016-1907-o21di_psm-grpc-server-5467bc46b4-zb24s.log
I1016 19:07:58.548767 123828871104064 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1907-o21di] Starting log collection thread 123828871104064 for psm-grpc-server-5467bc46b4-zb24s
I1016 19:07:58.548969 123829059989504 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 3 available replica(s)
I1016 19:08:09.061025 123829059989504 k8s_base_runner.py:972] Deployment psm-grpc-server has 3 replicas available
I1016 19:08:09.061315 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.4_run_test_server ---
I1016 19:08:09.061390 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.5_setup_server_backends ---
I1016 19:08:09.123434 123829059989504 traffic_director.py:298] Loading Network Endpoint Groups in zones ['us-central1-a', 'us-central1-c'].
I1016 19:08:09.230225 123829059989504 compute.py:386] Loaded NEG "psm-csm-server-20241016-1907-o21di-psm-grpc-server", zone=us-central1-a, size=2
I1016 19:08:09.308195 123829059989504 compute.py:386] Loaded NEG "psm-csm-server-20241016-1907-o21di-psm-grpc-server", zone=us-central1-c, size=1
I1016 19:08:09.308393 123829059989504 traffic_director.py:315] Adding backends to Backend Service psm-csm-backend-service-20241016-1907-o21di: {ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1907-o21di-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server', zone='us-central1-c', id='8358815605985227263', size=1, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1907-o21di","service-name":"psm-grpc-server","port":"8080"}'), ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1907-o21di-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server', zone='us-central1-a', id='5438999342406808037', size=2, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1907-o21di","service-name":"psm-grpc-server","port":"8080"}')}
I1016 19:08:09.314016 123829059989504 compute.py:591] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server
maxRatePerEndpoint: 5
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server
maxRatePerEndpoint: 5
...

I1016 19:08:11.075699 123829059989504 compute.py:661] Waiting 600 sec for compute operation id: operation-1729105689320-6249cca641ddf-12148dfb-fa7968bd
I1016 19:09:20.782551 123829059989504 traffic_director.py:335] Waiting for Backend Service psm-csm-backend-service-20241016-1907-o21di to report backends healthy: {ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1907-o21di-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server', zone='us-central1-c', id='8358815605985227263', size=1, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1907-o21di","service-name":"psm-grpc-server","port":"8080"}'), ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1907-o21di-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1907-o21di-psm-grpc-server', zone='us-central1-a', id='5438999342406808037', size=2, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1907-o21di","service-name":"psm-grpc-server","port":"8080"}')}
I1016 19:09:33.883388 123829059989504 compute.py:536] Backend psm-csm-server-20241016-1907-o21di-psm-grpc-server in zone us-central1-c reported healthy
I1016 19:09:38.019259 123829059989504 compute.py:536] Backend psm-csm-server-20241016-1907-o21di-psm-grpc-server in zone us-central1-a reported healthy
I1016 19:09:42.023986 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.5_setup_server_backends ---
I1016 19:09:42.024176 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.6_start_test_client ---
I1016 19:09:42.024272 123829059989504 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1907-o21di: server_target=xds:///psm-grpc-server:46335 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I1016 19:09:42.024555 123829059989504 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1907-o21di%22;timeRange=2024-10-16T19:09:42.024364Z%2F2024-10-16T19:39:42.024364Z?project=grpc-testing
I1016 19:09:42.026754 123829059989504 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1907-o21di
labels:
name: psm-csm-client-20241016-1907-o21di
owner: xds-k8s-interop-test
...

I1016 19:09:42.071244 123829059989504 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1907-o21di created
I1016 19:09:42.081294 123829059989504 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1907-o21di
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-twg4lcx0
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-twg4lcx0
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-twg4lcx0
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:46335"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--config-mesh-experimental=psm-csm-mesh-20241016-1907-o21di"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 19:09:42.159799 123829059989504 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 19:09:42.160002 123829059989504 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 19:09:43.225737 123829059989504 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-87585998c-5kkqf']
I1016 19:09:43.225951 123829059989504 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-87585998c-5kkqf to start
I1016 19:09:46.321213 123829059989504 k8s_base_runner.py:1006] Pod psm-grpc-client-87585998c-5kkqf ready, IP: [{'ip': '10.56.8.18'}]
I1016 19:09:46.321533 123829059989504 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-87585998c-5kkqf to app_net_ssa_test/test_app_logs/psm-csm-client-20241016-1907-o21di_psm-grpc-client-87585998c-5kkqf.log
I1016 19:09:46.321987 123828860618304 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1907-o21di] Starting log collection thread 123828860618304 for psm-grpc-client-87585998c-5kkqf
I1016 19:09:46.322234 123829059989504 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 19:09:49.402715 123829059989504 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 19:09:49.403131 123829059989504 client_app.py:294] [psm-grpc-client-87585998c-5kkqf] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 19:09:49.431904 123829059989504 client_app.py:322] [psm-grpc-client-87585998c-5kkqf] xDS control plane channel:
I1016 19:09:51.440587 123829059989504 client_app.py:333] [psm-grpc-client-87585998c-5kkqf] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 19:09:51.440952 123829059989504 client_app.py:304] [psm-grpc-client-87585998c-5kkqf] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 19:09:51.441156 123829059989504 client_app.py:254] [psm-grpc-client-87585998c-5kkqf] Waiting to report a READY channel to xds:///psm-grpc-server:46335
I1016 19:09:51.447920 123829059989504 client_app.py:381] [psm-grpc-client-87585998c-5kkqf] Server channel:
I1016 19:09:51.448114 123829059989504 client_app.py:381] [psm-grpc-client-87585998c-5kkqf] Server channel:
I1016 19:10:01.467560 123829059989504 client_app.py:381] [psm-grpc-client-87585998c-5kkqf] Server channel:
I1016 19:10:01.467790 123829059989504 client_app.py:381] [psm-grpc-client-87585998c-5kkqf] Server channel:
I1016 19:10:11.488795 123829059989504 client_app.py:381] [psm-grpc-client-87585998c-5kkqf] Server channel:
I1016 19:10:11.494290 123829059989504 client_app.py:394] [psm-grpc-client-87585998c-5kkqf] Found subchannel in state READY:
I1016 19:10:11.494565 123829059989504 client_app.py:265] [psm-grpc-client-87585998c-5kkqf] Channel to xds:///psm-grpc-server:46335 transitioned to state READY:
I1016 19:10:11.494683 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.6_start_test_client ---
I1016 19:10:11.494745 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.7_send_first_RPC_and_retrieve_cookie ---
I1016 19:10:11.495232 123829059989504 grpc.py:75] [psm-grpc-client-87585998c-5kkqf:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 1, 'timeoutSec': 1200, 'metadataKeys': ['*']}), wait_for_ready=True, timeout=1200)
I1016 19:10:11.594543 123829059989504 xds_k8s_testcase.py:683] [psm-grpc-client-87585998c-5kkqf] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5467bc46b4-bhnhf: 1
rpcs_by_method:
UnaryCall:
psm-grpc-server-5467bc46b4-bhnhf: 1
metadatas_by_peer:
psm-grpc-server-5467bc46b4-bhnhf:
hostname:
- psm-grpc-server-5467bc46b4-bhnhf
set-cookie:
- GSSA=MTAuNTYuNy41NDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF82Nzc5NjQ0NjA1NTUzNTAwMTY=;
HttpOnly; Path=/; Max-Age=50

I1016 19:10:11.594995 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.7_send_first_RPC_and_retrieve_cookie ---
I1016 19:10:11.595096 123829059989504 xds_k8s_testcase.py:311] --- Starting subTest AppNetSsaTest.test_session_affinity_policy.8_send_RPCs_with_cookie ---
I1016 19:10:11.595335 123829059989504 grpc.py:75] [psm-grpc-client-87585998c-5kkqf:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'cookie', 'value': 'GSSA=MTAuNTYuNy41NDo4MDgwO2Nsb3VkLWludGVybmFsLWlzdGlvOmNsb3VkX21wXzgzMDI5MzI2MzM4NF82Nzc5NjQ0NjA1NTUzNTAwMTY=; HttpOnly; Path=/; Max-Age=50'}]}), wait_for_ready=True, timeout=5)
I1016 19:10:11.599983 123829059989504 xds_k8s_testcase.py:520] Verifying RPCs go to servers ['psm-grpc-server-5467bc46b4-bhnhf']
I1016 19:10:11.600221 123829059989504 grpc.py:75] [psm-grpc-client-87585998c-5kkqf:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 10, 'timeoutSec': 1200, 'metadataKeys': ['*']}), wait_for_ready=True, timeout=1200)
I1016 19:10:12.075929 123829059989504 xds_k8s_testcase.py:683] [psm-grpc-client-87585998c-5kkqf] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-5467bc46b4-bhnhf: 10
rpcs_by_method:
UnaryCall:
psm-grpc-server-5467bc46b4-bhnhf: 10
metadatas_by_peer:
psm-grpc-server-5467bc46b4-bhnhf:
hostname:
- psm-grpc-server-5467bc46b4-bhnhf

I1016 19:10:12.076329 123829059989504 xds_k8s_testcase.py:316] --- Finished subTest AppNetSsaTest.test_session_affinity_policy.8_send_RPCs_with_cookie ---
I1016 19:10:12.076452 123829059989504 xds_k8s_testcase.py:848] ----- TestMethod AppNetSsaTest.test_session_affinity_policy teardown -----
I1016 19:10:12.187424 123829059989504 traffic_director.py:1018] Deleting HttpRoute psm-csm-http-route-20241016-1907-o21di
I1016 19:10:12.479084 123829059989504 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729105812247-6249cd1b7d201-2c71b6bc-24ebaa9f
I1016 19:10:41.165788 123829059989504 traffic_director.py:1007] Deleting GrpcRoute psm-csm-grpc-route-20241016-1907-o21di
I1016 19:10:41.246447 123829059989504 traffic_director.py:961] Deleting Mesh psm-csm-mesh-20241016-1907-o21di
I1016 19:10:41.433156 123829059989504 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729105841290-6249cd372fcae-f4b7f926-48ae16da
I1016 19:10:53.715570 123829059989504 traffic_director.py:768] Deleting Forwarding rule "psm-csm-forwarding-rule-20241016-1907-o21di"
I1016 19:10:53.822614 123829059989504 traffic_director.py:818] Deleting alternative Forwarding rule "psm-csm-forwarding-rule-alt-20241016-1907-o21di"
I1016 19:10:53.992583 123829059989504 traffic_director.py:668] Deleting HTTP Target proxy "psm-csm-target-proxy-20241016-1907-o21di"
I1016 19:10:54.282381 123829059989504 traffic_director.py:656] Deleting Target GRPC proxy "psm-csm-target-proxy-20241016-1907-o21di"
I1016 19:10:54.586472 123829059989504 traffic_director.py:708] Deleting alternative Target GRPC proxy "psm-csm-target-proxy-alt-20241016-1907-o21di"
I1016 19:10:55.014793 123829059989504 traffic_director.py:574] Deleting URL Map "psm-csm-url-map-20241016-1907-o21di"
I1016 19:10:55.187531 123829059989504 traffic_director.py:610] Deleting alternative URL Map "psm-csm-url-map-alt-20241016-1907-o21di"
I1016 19:10:55.503044 123829059989504 traffic_director.py:279] Deleting Backend Service "psm-csm-backend-service-20241016-1907-o21di"
I1016 19:10:55.829769 123829059989504 compute.py:661] Waiting 600 sec for compute operation id: operation-1729105855513-6249cd44c056a-d448c789-1e113694
I1016 19:11:26.078073 123829059989504 traffic_director.py:376] Deleting Alternative Backend Service "psm-csm-backend-service-alt-20241016-1907-o21di"
I1016 19:11:26.361854 123829059989504 traffic_director.py:455] Deleting Affinity Backend Service "psm-csm-backend-service-affinity-20241016-1907-o21di"
I1016 19:11:26.575746 123829059989504 traffic_director.py:237] Deleting Health Check "psm-csm-health-check-20241016-1907-o21di"
I1016 19:11:27.005200 123829059989504 compute.py:661] Waiting 600 sec for compute operation id: operation-1729105886655-6249cd62734cb-61dc150a-aeb5451d
I1016 19:11:31.557657 123829059989504 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 19:11:41.666652 123829059989504 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 19:11:41.666890 123829059989504 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1907-o21di
I1016 19:12:11.766191 123829059989504 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1907-o21di deleted
I1016 19:12:11.766451 123829059989504 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 19:12:21.931783 123829059989504 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 19:12:21.932027 123829059989504 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 19:12:23.013261 123829059989504 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 19:12:23.013481 123829059989504 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1907-o21di
I1016 19:13:23.137182 123829059989504 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1907-o21di deleted
I1016 19:13:23.137580 123829059989504 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 19:13:23.137835 123829059989504 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-twg4lcx0:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1907-o21di%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-twg4lcx0%22;timeRange=2024-10-16T19:09:42.024356Z%2F2024-10-16T19:12:11.766416Z;cursorTimestamp=2024-10-16T19:09:49.402919Z?project=grpc-testing
I1016 19:13:23.137986 123829059989504 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-jccram98:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1907-o21di%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-jccram98%22;timeRange=2024-10-16T19:07:53.788166Z%2F2024-10-16T19:13:23.137377Z;cursorTimestamp=2024-10-16T19:08:09.061229Z?project=grpc-testing
[ OK ] AppNetSsaTest.test_session_affinity_policy
I1016 19:13:23.138247 123829059989504 base_testcase.py:67] ----- PSM Test Case PASSED: AppNetSsaTest.test_session_affinity_policy -----

----------------------------------------------------------------------
Ran 1 test in 359.741s

OK
+ [19:13:23 UTC] Cmd finished: python
+ [19:13:23 UTC] Finished csm suite test: app_net_ssa_test

+ [19:13:23 UTC] Running csm suite test: app_net_csm_observability_test
+ [19:13:23 UTC] Test driver flags for app_net_csm_observability_test:
--flagfile=config/grpc-testing.cfg
--kube_context=gke_grpc-testing_us-central1_psm-interop-csm
--force_cleanup
--collect_app_logs
--log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_csm_observability_test
--xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_csm_observability_test/sponge_log.xml
--testing_version=dev-master
--client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
--flagfile=config/common-csm.cfg

+ [19:13:23 UTC] python -m tests.app_net_csm_observability_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1_psm-interop-csm --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_csm_observability_test --xml_output_file=/tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_csm_observability_test/sponge_log.xml --testing_version=dev-master --client_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --server_image=us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4 --flagfile=config/common-csm.cfg
Running tests under Python 3.10.12: /tmp/tmp.AYSR3QX6TE/psm-interop/venv/bin/python
I1016 19:13:24.720319 133395994099712 xds_k8s_testcase.py:167] ----- Testing AppNetCsmObservabilityTest -----
I1016 19:13:24.720504 133395994099712 xds_k8s_testcase.py:168] Logs timezone: UTC
I1016 19:13:24.720810 133395994099712 xds_k8s_testcase.py:112] Detected language and version: TestConfig(client_lang='python', server_lang='python', version='dev-master')
I1016 19:13:24.733246 133395994099712 k8s.py:264] Using kubernetes context "gke_grpc-testing_us-central1_psm-interop-csm", active host: https://34.29.103.132
[ RUN ] AppNetCsmObservabilityTest.test_csm_observability
I1016 19:13:24.761729 133395994099712 xds_k8s_testcase.py:829] Test run resource prefix: psm-csm, suffix: 20241016-1913-tjblu
I1016 19:13:25.262126 133395994099712 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/core/master/linux/psm-csm-python/app_net_csm_observability_test
I1016 19:13:25.586289 133395994099712 xds_k8s_testcase.py:824] Found unused xds port: 3765
I1016 19:13:25.586739 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.0_create_health_check ---
I1016 19:13:25.586841 133395994099712 traffic_director.py:226] Creating GRPC Health Check "psm-csm-health-check-20241016-1913-tjblu"
I1016 19:13:25.590108 133395994099712 compute.py:584] Creating compute resource:
---
grpcHealthCheck:
portSpecification: USE_SERVING_PORT
name: psm-csm-health-check-20241016-1913-tjblu
type: GRPC
...

I1016 19:13:26.053622 133395994099712 compute.py:661] Waiting 600 sec for compute operation id: operation-1729106005656-6249cdd3f0577-53d178fc-40523506
I1016 19:13:28.315199 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.0_create_health_check ---
I1016 19:13:28.315373 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.1_create_backend_service ---
I1016 19:13:28.315468 133395994099712 traffic_director.py:253] Creating GRPC Backend Service "psm-csm-backend-service-20241016-1913-tjblu"
I1016 19:13:28.325122 133395994099712 compute.py:584] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-csm-health-check-20241016-1913-tjblu
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-csm-backend-service-20241016-1913-tjblu
protocol: GRPC
...

I1016 19:13:28.748101 133395994099712 compute.py:661] Waiting 600 sec for compute operation id: operation-1729106008327-6249cdd67c5b2-7cde4332-98eea433
I1016 19:13:48.298314 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.1_create_backend_service ---
I1016 19:13:48.298529 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.2_create_mesh ---
I1016 19:13:48.298605 133395994099712 traffic_director.py:947] Creating Mesh psm-csm-mesh-20241016-1913-tjblu
I1016 19:13:48.299835 133395994099712 api.py:529] Creating networkservices resource:
--- {}
...

I1016 19:13:48.550146 133395994099712 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729106028412-6249cde9a3ef9-28f8fd33-5da567ef
I1016 19:13:52.775357 133395994099712 api.py:551] Loaded projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1913-tjblu:
---
createTime: '2024-10-16T19:13:48.451105275Z'
name: projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1913-tjblu
selfLink: https://networkservices.googleapis.com/v1alpha1/projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1913-tjblu
updateTime: '2024-10-16T19:13:51.847449235Z'
...

I1016 19:13:52.775605 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.2_create_mesh ---
I1016 19:13:52.775683 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.3_create_grpc_route ---
I1016 19:13:52.775757 133395994099712 traffic_director.py:978] Creating GrpcRoute psm-csm-grpc-route-20241016-1913-tjblu
I1016 19:13:52.778647 133395994099712 api.py:529] Creating networkservices resource:
---
hostnames: psm-grpc-server:3765
meshes:
- projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1913-tjblu
rules:
- action:
destinations:
- serviceName: projects/grpc-testing/locations/global/backendServices/psm-csm-backend-service-20241016-1913-tjblu
...

I1016 19:13:53.041841 133395994099712 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729106032820-6249cdedd81b5-e355cf61-6c8f0d48
I1016 19:13:59.324444 133395994099712 api.py:551] Loaded projects/grpc-testing/locations/global/grpcRoutes/psm-csm-grpc-route-20241016-1913-tjblu:
---
createTime: '2024-10-16T19:13:52.942627188Z'
hostnames:
- psm-grpc-server:3765
meshes:
- projects/grpc-testing/locations/global/meshes/psm-csm-mesh-20241016-1913-tjblu
name: projects/grpc-testing/locations/global/grpcRoutes/psm-csm-grpc-route-20241016-1913-tjblu
rules:
- action:
destinations:
- serviceName: projects/grpc-testing/locations/global/backendServices/psm-csm-backend-service-20241016-1913-tjblu
selfLink: https://networkservices.googleapis.com/v1alpha1/projects/grpc-testing/locations/global/grpcRoutes/psm-csm-grpc-route-20241016-1913-tjblu
updateTime: '2024-10-16T19:13:58.678713368Z'
...

I1016 19:13:59.324672 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.3_create_grpc_route ---
I1016 19:13:59.324743 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.1_run_test_server ---
I1016 19:13:59.324820 133395994099712 k8s_xds_server_runner.py:201] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-csm-server-20241016-1913-tjblu: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=1
I1016 19:13:59.325054 133395994099712 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1913-tjblu%22;timeRange=2024-10-16T19:13:59.324905Z%2F2024-10-16T19:43:59.324905Z?project=grpc-testing
I1016 19:13:59.330083 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-server-20241016-1913-tjblu
labels:
name: psm-csm-server-20241016-1913-tjblu
owner: xds-k8s-interop-test
...

I1016 19:13:59.406650 133395994099712 k8s_base_runner.py:371] Namespace psm-csm-server-20241016-1913-tjblu created
I1016 19:13:59.410125 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1913-tjblu
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-csm-server-20241016-1913-tjblu-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I1016 19:13:59.465603 133395994099712 k8s_base_runner.py:371] Service psm-grpc-server created
I1016 19:13:59.465781 133395994099712 k8s_base_runner.py:1073] Waiting for 'cloud.google.com/neg-status' annotation for a NEG at port 8080 to be assigned to Kubernetes Service psm-grpc-server in namespace psm-csm-server-20241016-1913-tjblu
I1016 19:13:59.509930 133395994099712 k8s_base_runner.py:1087] Detected 'cloud.google.com/neg-status' annotation for Kubernetes Service psm-grpc-server, namespace psm-csm-server-20241016-1913-tjblu: neg_name=psm-csm-server-20241016-1913-tjblu-psm-grpc-server, port=8080, zones=['us-central1-a', 'us-central1-c']
I1016 19:13:59.526963 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-csm-server-20241016-1913-tjblu
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jnkewiry
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jnkewiry
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jnkewiry
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-server
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-server:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000

args:
- "--port=8080"
- "--enable_csm_observability=true"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
- name: CSM_WORKLOAD_NAME
value: csm_workload_name_from_server
- name: CSM_CANONICAL_SERVICE_NAME
value: csm_canonical_service_name_from_server
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=$(NAMESPACE_NAME)
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true

resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-csm-server-20241016-1913-tjblu-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf

volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory

...

I1016 19:13:59.591089 133395994099712 k8s_base_runner.py:371] Deployment psm-grpc-server created
I1016 19:13:59.593506 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/csm/pod-monitoring.yaml:
---
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
namespace: psm-csm-server-20241016-1913-tjblu
name: psm-grpc-server-jnkewiry-gmp
labels:
owner: xds-k8s-interop-test
spec:
selector:
matchLabels:
deployment_id: psm-grpc-server-jnkewiry
endpoints:
- port: 9464
interval: 10s

I1016 19:13:59.762034 133395994099712 k8s_base_runner.py:371] PodMonitoring psm-grpc-server-jnkewiry-gmp created
I1016 19:13:59.762230 133395994099712 k8s_base_runner.py:981] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I1016 19:13:59.890693 133395994099712 k8s_base_runner.py:991] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-754bcb4bf7-4m6sg']
I1016 19:13:59.890954 133395994099712 k8s_base_runner.py:1001] Waiting for pod psm-grpc-server-754bcb4bf7-4m6sg to start
I1016 19:14:02.991897 133395994099712 k8s_base_runner.py:1006] Pod psm-grpc-server-754bcb4bf7-4m6sg ready, IP: [{'ip': '10.56.5.83'}]
I1016 19:14:02.992192 133395994099712 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-server-754bcb4bf7-4m6sg to app_net_csm_observability_test/test_app_logs/psm-csm-server-20241016-1913-tjblu_psm-grpc-server-754bcb4bf7-4m6sg.log
I1016 19:14:02.992692 133395321456192 k8s_log_collector.py:71] [ns/psm-csm-server-20241016-1913-tjblu] Starting log collection thread 133395321456192 for psm-grpc-server-754bcb4bf7-4m6sg
I1016 19:14:02.992913 133395994099712 k8s_base_runner.py:963] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I1016 19:14:14.228621 133395994099712 k8s_base_runner.py:972] Deployment psm-grpc-server has 1 replicas available
I1016 19:14:14.228935 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.1_run_test_server ---
I1016 19:14:14.229017 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.5_setup_server_backends ---
I1016 19:14:14.237067 133395994099712 traffic_director.py:298] Loading Network Endpoint Groups in zones ['us-central1-a', 'us-central1-c'].
I1016 19:14:14.334812 133395994099712 compute.py:386] Loaded NEG "psm-csm-server-20241016-1913-tjblu-psm-grpc-server", zone=us-central1-a, size=1
I1016 19:14:14.417289 133395994099712 compute.py:386] Loaded NEG "psm-csm-server-20241016-1913-tjblu-psm-grpc-server", zone=us-central1-c, size=0
I1016 19:14:14.417504 133395994099712 traffic_director.py:315] Adding backends to Backend Service psm-csm-backend-service-20241016-1913-tjblu: {ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1913-tjblu-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server', zone='us-central1-a', id='2222481184627994264', size=1, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1913-tjblu","service-name":"psm-grpc-server","port":"8080"}'), ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1913-tjblu-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server', zone='us-central1-c', id='1310599085828189842', size=0, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1913-tjblu","service-name":"psm-grpc-server","port":"8080"}')}
I1016 19:14:14.423227 133395994099712 compute.py:591] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server
maxRatePerEndpoint: 5
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server
maxRatePerEndpoint: 5
...

I1016 19:14:15.071555 133395994099712 compute.py:661] Waiting 600 sec for compute operation id: operation-1729106054426-6249ce0272dd4-64766ade-a1498252
I1016 19:15:11.500713 133395994099712 traffic_director.py:335] Waiting for Backend Service psm-csm-backend-service-20241016-1913-tjblu to report backends healthy: {ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1913-tjblu-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server', zone='us-central1-a', id='2222481184627994264', size=1, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1913-tjblu","service-name":"psm-grpc-server","port":"8080"}'), ComputeV1.NegGcpResource(name='psm-csm-server-20241016-1913-tjblu-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-c/networkEndpointGroups/psm-csm-server-20241016-1913-tjblu-psm-grpc-server', zone='us-central1-c', id='1310599085828189842', size=0, network_endpoint_type='GCE_VM_IP_PORT', description='{"cluster-uid":"b235ab9a-b166-4a2c-91b3-24fd66c7ddd3","namespace":"psm-csm-server-20241016-1913-tjblu","service-name":"psm-grpc-server","port":"8080"}')}
I1016 19:15:20.674589 133395994099712 compute.py:536] Backend psm-csm-server-20241016-1913-tjblu-psm-grpc-server in zone us-central1-a reported healthy
I1016 19:15:24.883358 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.5_setup_server_backends ---
I1016 19:15:24.883597 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.2_start_test_client ---
I1016 19:15:24.883722 133395994099712 k8s_xds_client_runner.py:135] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-csm-client-20241016-1913-tjblu: server_target=xds:///psm-grpc-server:3765 rpc=UnaryCall qps=1 metadata='' secure_mode=False print_response=False
I1016 19:15:24.883964 133395994099712 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1913-tjblu%22;timeRange=2024-10-16T19:15:24.883830Z%2F2024-10-16T19:45:24.883830Z?project=grpc-testing
I1016 19:15:24.886091 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-csm-client-20241016-1913-tjblu
labels:
name: psm-csm-client-20241016-1913-tjblu
owner: xds-k8s-interop-test
...

I1016 19:15:24.951734 133395994099712 k8s_base_runner.py:371] Namespace psm-csm-client-20241016-1913-tjblu created
I1016 19:15:24.963447 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-csm-client-20241016-1913-tjblu
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2q8lbqc6
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2q8lbqc6
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2q8lbqc6
owner: xds-k8s-interop-test
spec:
containers:
- name: psm-grpc-client
image: us-docker.pkg.dev/grpc-testing/psm-interop/python-client:78f6c9348a24f6c49f81cc67867973e14eb00ef4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:3765"
- "--stats_port=8079"
- "--qps=1"
- "--rpc=UnaryCall"
- "--metadata="
- "--request_payload_size=27182"
- "--response_payload_size=31415"
- "--print_response=False"
- "--enable_csm_observability=true"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_ENABLE_OVERRIDE_HOST
value: "true"
- name: CSM_WORKLOAD_NAME
value: csm_workload_name_from_client
- name: CSM_CANONICAL_SERVICE_NAME
value: csm_canonical_service_name_from_client
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.pod.name=$(POD_NAME),k8s.namespace.name=$(NAMESPACE_NAME)
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: us-docker.pkg.dev/grpc-testing/trafficdirector/td-grpc-bootstrap:4ba99d77963c72d4973486ef2a3055913fdc2392
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--config-mesh-experimental=psm-csm-mesh-20241016-1913-tjblu"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I1016 19:15:25.026929 133395994099712 k8s_base_runner.py:371] Deployment psm-grpc-client created
I1016 19:15:25.029280 133395994099712 k8s_base_runner.py:351] Rendered template kubernetes-manifests/csm/pod-monitoring.yaml:
---
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
namespace: psm-csm-client-20241016-1913-tjblu
name: psm-grpc-client-2q8lbqc6-gmp
labels:
owner: xds-k8s-interop-test
spec:
selector:
matchLabels:
deployment_id: psm-grpc-client-2q8lbqc6
endpoints:
- port: 9464
interval: 10s

I1016 19:15:25.119278 133395994099712 k8s_base_runner.py:371] PodMonitoring psm-grpc-client-2q8lbqc6-gmp created
I1016 19:15:25.119523 133395994099712 k8s_base_runner.py:981] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I1016 19:15:25.206735 133395994099712 k8s_base_runner.py:991] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-5f56d885bb-df8sv']
I1016 19:15:25.206979 133395994099712 k8s_base_runner.py:1001] Waiting for pod psm-grpc-client-5f56d885bb-df8sv to start
I1016 19:15:27.307389 133395994099712 k8s_base_runner.py:1006] Pod psm-grpc-client-5f56d885bb-df8sv ready, IP: [{'ip': '10.56.8.19'}]
I1016 19:15:27.307696 133395994099712 k8s_base_runner.py:1049] Enabling log collection from pod psm-grpc-client-5f56d885bb-df8sv to app_net_csm_observability_test/test_app_logs/psm-csm-client-20241016-1913-tjblu_psm-grpc-client-5f56d885bb-df8sv.log
I1016 19:15:27.308110 133395172558400 k8s_log_collector.py:71] [ns/psm-csm-client-20241016-1913-tjblu] Starting log collection thread 133395172558400 for psm-grpc-client-5f56d885bb-df8sv
I1016 19:15:27.308310 133395994099712 k8s_base_runner.py:963] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I1016 19:15:32.434649 133395994099712 k8s_base_runner.py:972] Deployment psm-grpc-client has 1 replicas available
I1016 19:15:32.435063 133395994099712 client_app.py:294] [psm-grpc-client-5f56d885bb-df8sv] ADS: Waiting for active calls to xDS control plane to trafficdirector.googleapis.com:443
I1016 19:15:32.458853 133395994099712 client_app.py:322] [psm-grpc-client-5f56d885bb-df8sv] xDS control plane channel:
I1016 19:15:34.467332 133395994099712 client_app.py:333] [psm-grpc-client-5f56d885bb-df8sv] Detected active calls to xDS control plane trafficdirector.googleapis.com:443, channel:
I1016 19:15:34.467762 133395994099712 client_app.py:304] [psm-grpc-client-5f56d885bb-df8sv] ADS: Detected active calls to xDS control plane trafficdirector.googleapis.com:443
I1016 19:15:34.468003 133395994099712 client_app.py:254] [psm-grpc-client-5f56d885bb-df8sv] Waiting to report a READY channel to xds:///psm-grpc-server:3765
I1016 19:15:34.474620 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:15:34.474807 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:15:44.494575 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:15:44.494767 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:15:54.519202 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:15:54.519391 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:16:04.539904 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:16:04.540114 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:16:14.561778 133395994099712 client_app.py:381] [psm-grpc-client-5f56d885bb-df8sv] Server channel:
I1016 19:16:14.567927 133395994099712 client_app.py:394] [psm-grpc-client-5f56d885bb-df8sv] Found subchannel in state READY:
I1016 19:16:14.568196 133395994099712 client_app.py:265] [psm-grpc-client-5f56d885bb-df8sv] Channel to xds:///psm-grpc-server:3765 transitioned to state READY:
I1016 19:16:14.568322 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.2_start_test_client ---
I1016 19:16:14.568388 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.7_assert_xds_config_exists ---
I1016 19:16:14.568624 133395994099712 grpc.py:75] [psm-grpc-client-5f56d885bb-df8sv:8079] >> RPC ClientStatusDiscoveryService.FetchClientStatus(request=ClientStatusRequest({}), wait_for_ready=True, timeout=30)
I1016 19:16:14.575906 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.7_assert_xds_config_exists ---
I1016 19:16:14.576078 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.3_test_server_received_rpcs_from_test_client ---
I1016 19:16:14.576281 133395994099712 grpc.py:75] [psm-grpc-client-5f56d885bb-df8sv:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), wait_for_ready=True, timeout=1200)
I1016 19:17:56.607253 133395994099712 xds_k8s_testcase.py:683] [psm-grpc-client-5f56d885bb-df8sv] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-754bcb4bf7-4m6sg: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-754bcb4bf7-4m6sg: 100

I1016 19:17:56.607514 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.3_test_server_received_rpcs_from_test_client ---
I1016 19:17:56.607590 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.4_export_prometheus_metrics_data ---
I1016 19:17:56.607666 133395994099712 app_net_csm_observability_test.py:250] Letting test client run for 90 seconds to produce metric data
I1016 19:19:26.939346 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.4_export_prometheus_metrics_data ---
I1016 19:19:26.939582 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.5_query_cloud_monitoring_metrics ---
I1016 19:19:26.939793 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram
I1016 19:19:27.002221 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram:
name: prometheus.googleapis.com/grpc_server_call_duration_seconds/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-754bcb4bf7-4m6sg
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-754bcb4bf7-4m6sg:9464
job: psm-grpc-server-jnkewiry-gmp
location: us-central1
namespace: psm-csm-server-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.002413 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram
I1016 19:19:27.058703 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_server_call_rcvd_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-754bcb4bf7-4m6sg
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-754bcb4bf7-4m6sg:9464
job: psm-grpc-server-jnkewiry-gmp
location: us-central1
namespace: psm-csm-server-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.058883 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram
I1016 19:19:27.115991 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_server_call_sent_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_client
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-c
csm_remote_workload_name: csm_workload_name_from_client
csm_remote_workload_namespace_name: psm-csm-client-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_workload_canonical_service: csm_canonical_service_name_from_server
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
pod: psm-grpc-server-754bcb4bf7-4m6sg
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-754bcb4bf7-4m6sg:9464
job: psm-grpc-server-jnkewiry-gmp
location: us-central1
namespace: psm-csm-server-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.116199 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram
I1016 19:19:27.158035 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_sent_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-csm-backend-service-20241016-1913-tjblu
csm_service_namespace_name: unknown
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-grpc-server:3765
pod: psm-grpc-client-5f56d885bb-df8sv
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-5f56d885bb-df8sv:9464
job: psm-grpc-client-2q8lbqc6-gmp
location: us-central1
namespace: psm-csm-client-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.158212 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram
I1016 19:19:27.204570 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_rcvd_total_compressed_message_size_bytes/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-csm-backend-service-20241016-1913-tjblu
csm_service_namespace_name: unknown
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-grpc-server:3765
pod: psm-grpc-client-5f56d885bb-df8sv
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-5f56d885bb-df8sv:9464
job: psm-grpc-client-2q8lbqc6-gmp
location: us-central1
namespace: psm-csm-client-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.204748 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram
I1016 19:19:27.255166 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram:
name: prometheus.googleapis.com/grpc_client_attempt_duration_seconds/histogram
resource_type: prometheus_target
metric_labels:
csm_mesh_id: unknown
csm_remote_workload_canonical_service: csm_canonical_service_name_from_server
csm_remote_workload_cluster_name: psm-interop-csm
csm_remote_workload_location: us-central1-a
csm_remote_workload_name: csm_workload_name_from_server
csm_remote_workload_namespace_name: psm-csm-server-20241016-1913-tjblu
csm_remote_workload_project_id: grpc-testing
csm_remote_workload_type: gcp_kubernetes_engine
csm_service_name: psm-csm-backend-service-20241016-1913-tjblu
csm_service_namespace_name: unknown
csm_workload_canonical_service: csm_canonical_service_name_from_client
grpc_method: grpc.testing.TestService/UnaryCall
grpc_status: OK
grpc_target: xds:///psm-grpc-server:3765
pod: psm-grpc-client-5f56d885bb-df8sv
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-5f56d885bb-df8sv:9464
job: psm-grpc-client-2q8lbqc6-gmp
location: us-central1
namespace: psm-csm-client-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.255372 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_server_call_started_total/counter
I1016 19:19:27.300701 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_server_call_started_total/counter:
name: prometheus.googleapis.com/grpc_server_call_started_total/counter
resource_type: prometheus_target
metric_labels:
grpc_method: grpc.testing.TestService/UnaryCall
pod: psm-grpc-server-754bcb4bf7-4m6sg
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-server-754bcb4bf7-4m6sg:9464
job: psm-grpc-server-jnkewiry-gmp
location: us-central1
namespace: psm-csm-server-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.300893 133395994099712 app_net_csm_observability_test.py:539] Requesting list_time_series for metric prometheus.googleapis.com/grpc_client_attempt_started_total/counter
I1016 19:19:27.348556 133395994099712 app_net_csm_observability_test.py:559] Metric prometheus.googleapis.com/grpc_client_attempt_started_total/counter:
name: prometheus.googleapis.com/grpc_client_attempt_started_total/counter
resource_type: prometheus_target
metric_labels:
grpc_method: grpc.testing.TestService/UnaryCall
grpc_target: xds:///psm-grpc-server:3765
pod: psm-grpc-client-5f56d885bb-df8sv
resource_labels:
cluster: psm-interop-csm
instance: psm-grpc-client-5f56d885bb-df8sv:9464
job: psm-grpc-client-2q8lbqc6-gmp
location: us-central1
namespace: psm-csm-client-20241016-1913-tjblu
project_id: grpc-testing

I1016 19:19:27.348772 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.5_query_cloud_monitoring_metrics ---
I1016 19:19:27.348850 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.6_check_metrics_time_series ---
I1016 19:19:27.348911 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.6_check_metrics_time_series ---
I1016 19:19:27.348960 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.7_check_metrics_labels_histogram_client ---
I1016 19:19:27.349053 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.7_check_metrics_labels_histogram_client ---
I1016 19:19:27.349103 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.8_check_metrics_labels_histogram_server ---
I1016 19:19:27.349168 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.8_check_metrics_labels_histogram_server ---
I1016 19:19:27.349213 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.9_check_metrics_labels_counter_client ---
I1016 19:19:27.349261 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.9_check_metrics_labels_counter_client ---
I1016 19:19:27.349303 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.10_check_metrics_labels_counter_server ---
I1016 19:19:27.349347 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.10_check_metrics_labels_counter_server ---
I1016 19:19:27.349386 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.11_check_client_resource_labels_client ---
I1016 19:19:27.349478 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.11_check_client_resource_labels_client ---
I1016 19:19:27.349524 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.12_check_server_resource_labels_server ---
I1016 19:19:27.349595 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.12_check_server_resource_labels_server ---
I1016 19:19:27.349638 133395994099712 xds_k8s_testcase.py:311] --- Starting subTest AppNetCsmObservabilityTest.test_csm_observability.13_check_bytes_sent_vs_data_points ---
I1016 19:19:27.349793 133395994099712 xds_k8s_testcase.py:316] --- Finished subTest AppNetCsmObservabilityTest.test_csm_observability.13_check_bytes_sent_vs_data_points ---
I1016 19:19:27.349953 133395994099712 xds_k8s_testcase.py:848] ----- TestMethod AppNetCsmObservabilityTest.test_csm_observability teardown -----
I1016 19:19:27.477628 133395994099712 traffic_director.py:1018] Deleting HttpRoute psm-csm-http-route-20241016-1913-tjblu
I1016 19:19:27.557943 133395994099712 traffic_director.py:1007] Deleting GrpcRoute psm-csm-grpc-route-20241016-1913-tjblu
I1016 19:19:27.711833 133395994099712 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729106367601-6249cf2d1dc5f-b244f047-69d970d8
I1016 19:19:56.334858 133395994099712 traffic_director.py:961] Deleting Mesh psm-csm-mesh-20241016-1913-tjblu
I1016 19:19:56.477050 133395994099712 api.py:585] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1729106396366-6249cf488c891-8076711e-1b4fb842
I1016 19:20:00.610383 133395994099712 traffic_director.py:768] Deleting Forwarding rule "psm-csm-forwarding-rule-20241016-1913-tjblu"
I1016 19:20:00.719436 133395994099712 traffic_director.py:818] Deleting alternative Forwarding rule "psm-csm-forwarding-rule-alt-20241016-1913-tjblu"
I1016 19:20:00.828780 133395994099712 traffic_director.py:668] Deleting HTTP Target proxy "psm-csm-target-proxy-20241016-1913-tjblu"
I1016 19:20:01.109622 133395994099712 traffic_director.py:656] Deleting Target GRPC proxy "psm-csm-target-proxy-20241016-1913-tjblu"
I1016 19:20:01.412090 133395994099712 traffic_director.py:708] Deleting alternative Target GRPC proxy "psm-csm-target-proxy-alt-20241016-1913-tjblu"
I1016 19:20:01.790386 133395994099712 traffic_director.py:574] Deleting URL Map "psm-csm-url-map-20241016-1913-tjblu"
I1016 19:20:02.070701 133395994099712 traffic_director.py:610] Deleting alternative URL Map "psm-csm-url-map-alt-20241016-1913-tjblu"
I1016 19:20:02.245760 133395994099712 traffic_director.py:279] Deleting Backend Service "psm-csm-backend-service-20241016-1913-tjblu"
I1016 19:20:02.605555 133395994099712 compute.py:661] Waiting 600 sec for compute operation id: operation-1729106402256-6249cf4e2a749-180f6bfa-523fdbd5
I1016 19:20:32.880140 133395994099712 traffic_director.py:376] Deleting Alternative Backend Service "psm-csm-backend-service-alt-20241016-1913-tjblu"
I1016 19:20:33.179347 133395994099712 traffic_director.py:455] Deleting Affinity Backend Service "psm-csm-backend-service-affinity-20241016-1913-tjblu"
I1016 19:20:33.356831 133395994099712 traffic_director.py:237] Deleting Health Check "psm-csm-health-check-20241016-1913-tjblu"
I1016 19:20:33.721877 133395994099712 compute.py:661] Waiting 600 sec for compute operation id: operation-1729106433456-6249cf6beb9eb-239831dd-bc77b587
I1016 19:20:38.028926 133395994099712 k8s_base_runner.py:892] Deleting deployment psm-grpc-client
I1016 19:20:48.185026 133395994099712 k8s_base_runner.py:907] Deployment psm-grpc-client deleted
I1016 19:20:48.185298 133395994099712 k8s_base_runner.py:861] Deleting PodMonitoring psm-grpc-client-2q8lbqc6-gmp
I1016 19:20:48.237970 133395994099712 k8s_base_runner.py:872] PodMonitoring psm-grpc-client-2q8lbqc6-gmp deleted
I1016 19:20:48.238153 133395994099712 k8s_base_runner.py:943] Deleting namespace psm-csm-client-20241016-1913-tjblu
I1016 19:21:18.356218 133395994099712 k8s_base_runner.py:960] Namespace psm-csm-client-20241016-1913-tjblu deleted
I1016 19:21:18.356443 133395994099712 k8s_base_runner.py:892] Deleting deployment psm-grpc-server
I1016 19:21:28.520385 133395994099712 k8s_base_runner.py:907] Deployment psm-grpc-server deleted
I1016 19:21:28.520656 133395994099712 k8s_base_runner.py:910] Deleting service psm-grpc-server
I1016 19:21:29.609954 133395994099712 k8s_base_runner.py:923] Service psm-grpc-server deleted
I1016 19:21:29.610145 133395994099712 k8s_base_runner.py:861] Deleting PodMonitoring psm-grpc-server-jnkewiry-gmp
I1016 19:21:29.647768 133395994099712 k8s_base_runner.py:872] PodMonitoring psm-grpc-server-jnkewiry-gmp deleted
I1016 19:21:29.647955 133395994099712 k8s_base_runner.py:943] Deleting namespace psm-csm-server-20241016-1913-tjblu
I1016 19:23:29.932837 133395994099712 k8s_base_runner.py:960] Namespace psm-csm-server-20241016-1913-tjblu deleted
I1016 19:23:29.933251 133395994099712 xds_k8s_testcase.py:872] ----- Test client/server logs -----
I1016 19:23:29.933516 133395994099712 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-client-2q8lbqc6:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-csm-client-20241016-1913-tjblu%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-2q8lbqc6%22;timeRange=2024-10-16T19:15:24.883821Z%2F2024-10-16T19:21:18.356395Z;cursorTimestamp=2024-10-16T19:15:32.434863Z?project=grpc-testing
I1016 19:23:29.933663 133395994099712 k8s_base_runner.py:1171] GCP Logs Explorer link to psm-grpc-server-jnkewiry:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-csm-server-20241016-1913-tjblu%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-jnkewiry%22;timeRange=2024-10-16T19:13:59.324898Z%2F2024-10-16T19:23:29.933067Z;cursorTimestamp=2024-10-16T19:14:14.228855Z?project=grpc-testing
[ OK ] AppNetCsmObservabilityTest.test_csm_observability
I1016 19:23:29.933958 133395994099712 base_testcase.py:67] ----- PSM Test Case PASSED: AppNetCsmObservabilityTest.test_csm_observability -----

----------------------------------------------------------------------
Ran 1 test in 605.217s

OK
+ [19:23:30 UTC] Cmd finished: python
+ [19:23:30 UTC] Finished csm suite test: app_net_csm_observability_test

+ [19:23:30 UTC] Failed test suites: 0
+ [19:23:30 UTC] PSM Interop tests completed: csm


[ID: 2881275] Command finished after 4689 secs, exit value: 0


Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
[12:23:30 PDT] Collecting build artifacts from build VM
[12:23:32 PDT] Kokoro builder finished