Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
[11:44:20] Transferring environment variable script to build VM
[11:44:21] Transferring kokoro_log_reader.py to build VM
[11:44:22] Transferring source code to build VM
[11:44:24] Executing build script on build VM



[ID: 9179954] Executing command via SSH:
export KOKORO_BUILD_NUMBER="644"
export KOKORO_JOB_NAME="grpc/go/master/branch/psm-security"
source /tmpfs/kokoro-env_vars.sh; cd /tmpfs/src/ ; chmod 755 github/grpc-go/test/kokoro/psm-security.sh ; PYTHON_3_VERSION="$(pyenv which python3 2> /dev/null || which python3)" ; PYTHON_2_VERSION="$(pyenv which python2 2> /dev/null || which python2)" ; if "$PYTHON_3_VERSION" -c "import psutil" ; then KOKORO_PYTHON_COMMAND="$PYTHON_3_VERSION" ; else KOKORO_PYTHON_COMMAND="$PYTHON_2_VERSION" ; fi > /dev/null 2>&1 ; echo "export KOKORO_PYTHON_COMMAND="$KOKORO_PYTHON_COMMAND"" > "$HOME/.kokoro_python_vars" ; nohup bash -c "( rm -f /tmpfs/kokoro_build_exit_code ; github/grpc-go/test/kokoro/psm-security.sh ; echo \${PIPESTATUS[0]} > /tmpfs/kokoro_build_exit_code ) > /tmpfs/kokoro_build.log 2>&1" > /dev/null 2>&1 & echo $! > /tmpfs/kokoro_build.pid ; source "$HOME/.kokoro_python_vars" ; "$KOKORO_PYTHON_COMMAND" /tmpfs/kokoro_log_reader.py /tmpfs/kokoro_build.log /tmpfs/kokoro_build_exit_code /tmpfs/kokoro_build.pid /tmpfs/kokoro_log_reader.pid --start_byte 0

Sourcing test driver install script from: https://raw.githubusercontent.com/grpc/grpc/master/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh
Activated GKE cluster: GKE_CLUSTER_NAME=psm-interop-security GKE_CLUSTER_ZONE=us-central1-a
+ [[ -n /tmpfs/src ]]
+ kokoro_setup_test_driver grpc-go
+ set +x
Kokoro Ubuntu version:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy
Python 3.10.6
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Sponge properties:
TESTS_FORMAT_VERSION,2
TESTGRID_EXCLUDE,1
GIT_ORIGIN_URL,https://github.com/dfawley/grpc-go.git
GIT_COMMIT_SHORT,0af3cad6
(Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 128497 files and directories currently installed.)
Removing needrestart (3.5-5ubuntu2.1) ...
Processing triggers for man-db (2.10.2-1) ...
W: https://packages.cloud.google.com/apt/dists/cloud-sdk/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://download.docker.com/linux/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
(Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 128442 files and directories currently installed.)
Removing libintl-xs-perl (1.26-3build2) ...
Removing libintl-perl (1.26-3build2) ...
Removing libmodule-find-perl (0.15-1) ...
Removing libmodule-scandeps-perl (1.31-1) ...
Removing libproc-processtable-perl:amd64 (0.634-1build1) ...
Removing libsort-naturally-perl (1.03-2) ...
Removing libterm-readkey-perl (2.38-1build4) ...
(Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 128042 files and directories currently installed.)
Preparing to unpack .../00-python3.10-dev_3.10.12-1~22.04.2_amd64.deb ...
Unpacking python3.10-dev (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../01-libpython3.10-dev_3.10.12-1~22.04.2_amd64.deb ...
Unpacking libpython3.10-dev:amd64 (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../02-libpython3.10_3.10.12-1~22.04.2_amd64.deb ...
Unpacking libpython3.10:amd64 (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../03-python3.10_3.10.12-1~22.04.2_amd64.deb ...
Unpacking python3.10 (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../04-libpython3.10-stdlib_3.10.12-1~22.04.2_amd64.deb ...
Unpacking libpython3.10-stdlib:amd64 (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../05-python3.10-minimal_3.10.12-1~22.04.2_amd64.deb ...
Unpacking python3.10-minimal (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Preparing to unpack .../06-libpython3.10-minimal_3.10.12-1~22.04.2_amd64.deb ...
Unpacking libpython3.10-minimal:amd64 (3.10.12-1~22.04.2) over (3.10.6-1~22.04.2ubuntu1) ...
Selecting previously unselected package google-cloud-sdk-gke-gcloud-auth-plugin.
Preparing to unpack .../07-google-cloud-sdk-gke-gcloud-auth-plugin_442.0.0-0_amd64.deb ...
Unpacking google-cloud-sdk-gke-gcloud-auth-plugin (442.0.0-0) ...
Selecting previously unselected package kubectl.
Preparing to unpack .../08-kubectl_1.27.4-00_amd64.deb ...
Unpacking kubectl (1.27.4-00) ...
Selecting previously unselected package python3-pip-whl.
Preparing to unpack .../09-python3-pip-whl_22.0.2+dfsg-1ubuntu0.3_all.deb ...
Unpacking python3-pip-whl (22.0.2+dfsg-1ubuntu0.3) ...
Selecting previously unselected package python3-setuptools-whl.
Preparing to unpack .../10-python3-setuptools-whl_59.6.0-1.2ubuntu0.22.04.1_all.deb ...
Unpacking python3-setuptools-whl (59.6.0-1.2ubuntu0.22.04.1) ...
Selecting previously unselected package python3.10-venv.
Preparing to unpack .../11-python3.10-venv_3.10.12-1~22.04.2_amd64.deb ...
Unpacking python3.10-venv (3.10.12-1~22.04.2) ...
Setting up python3-setuptools-whl (59.6.0-1.2ubuntu0.22.04.1) ...
Setting up python3-pip-whl (22.0.2+dfsg-1ubuntu0.3) ...
Setting up google-cloud-sdk-gke-gcloud-auth-plugin (442.0.0-0) ...
Setting up kubectl (1.27.4-00) ...
Setting up libpython3.10-minimal:amd64 (3.10.12-1~22.04.2) ...
Setting up python3.10-minimal (3.10.12-1~22.04.2) ...
Setting up libpython3.10-stdlib:amd64 (3.10.12-1~22.04.2) ...
Setting up libpython3.10:amd64 (3.10.12-1~22.04.2) ...
Setting up python3.10 (3.10.12-1~22.04.2) ...
Setting up libpython3.10-dev:amd64 (3.10.12-1~22.04.2) ...
Setting up python3.10-dev (3.10.12-1~22.04.2) ...
Setting up python3.10-venv (3.10.12-1~22.04.2) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
Fetching cluster endpoint and auth data.
kubeconfig entry generated for psm-interop-security.
Cloning driver to https://github.com/grpc/grpc.git branch master to /tmp/tmp.T3wY7IAC08/grpc
Cloning into '/tmp/tmp.T3wY7IAC08/grpc'...
Install python dependencies
Creating python virtual environment: /tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv
Collecting Mako==1.2.4 (from -r requirements.lock (line 1))
Downloading Mako-1.2.4-py3-none-any.whl (78 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.7/78.7 kB 2.0 MB/s eta 0:00:00
Collecting PyYAML==6.0 (from -r requirements.lock (line 2))
Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 14.5 MB/s eta 0:00:00
Collecting absl-py==0.15.0 (from -r requirements.lock (line 3))
Downloading absl_py-0.15.0-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.0/132.0 kB 25.3 MB/s eta 0:00:00
Collecting google-api-python-client==1.12.11 (from -r requirements.lock (line 4))
Downloading google_api_python_client-1.12.11-py2.py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.1/62.1 kB 13.3 MB/s eta 0:00:00
Collecting google-cloud-secret-manager==2.15.1 (from -r requirements.lock (line 5))
Downloading google_cloud_secret_manager-2.15.1-py2.py3-none-any.whl (100 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.4/100.4 kB 18.2 MB/s eta 0:00:00
Collecting grpcio==1.51.1 (from -r requirements.lock (line 6))
Downloading grpcio-1.51.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 74.4 MB/s eta 0:00:00
Collecting grpcio-health-checking==1.48.2 (from -r requirements.lock (line 7))
Downloading grpcio_health_checking-1.48.2-py3-none-any.whl (18 kB)
Collecting grpcio-tools==1.48.2 (from -r requirements.lock (line 8))
Downloading grpcio_tools-1.48.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 98.1 MB/s eta 0:00:00
Collecting grpcio-channelz==1.48.2 (from -r requirements.lock (line 9))
Downloading grpcio_channelz-1.48.2-py3-none-any.whl (21 kB)
Collecting kubernetes==27.2.0 (from -r requirements.lock (line 10))
Obtaining dependency information for kubernetes==27.2.0 from https://files.pythonhosted.org/packages/99/89/3ab0cb3069f49ae2eaf73f884c82164f18f70fcc598e0312edea71614ad7/kubernetes-27.2.0-py2.py3-none-any.whl.metadata
Downloading kubernetes-27.2.0-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting six==1.16.0 (from -r requirements.lock (line 11))
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tenacity==6.3.1 (from -r requirements.lock (line 12))
Downloading tenacity-6.3.1-py2.py3-none-any.whl (36 kB)
Collecting packaging==21.3 (from -r requirements.lock (line 13))
Downloading packaging-21.3-py3-none-any.whl (40 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 7.8 MB/s eta 0:00:00
Collecting Pygments==2.14.0 (from -r requirements.lock (line 14))
Downloading Pygments-2.14.0-py3-none-any.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 83.3 MB/s eta 0:00:00
Collecting python-dateutil==2.8.2 (from -r requirements.lock (line 15))
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 37.0 MB/s eta 0:00:00
Collecting protobuf==3.20.3 (from -r requirements.lock (line 16))
Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 74.1 MB/s eta 0:00:00
Collecting xds-protos==0.0.11 (from -r requirements.lock (line 17))
Downloading xds_protos-0.0.11-py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 93.0 MB/s eta 0:00:00
Collecting cachetools==5.3.0 (from -r requirements.lock (line 19))
Downloading cachetools-5.3.0-py3-none-any.whl (9.3 kB)
Collecting certifi==2022.12.7 (from -r requirements.lock (line 20))
Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 28.2 MB/s eta 0:00:00
Collecting charset-normalizer==3.0.1 (from -r requirements.lock (line 21))
Downloading charset_normalizer-3.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (198 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 198.8/198.8 kB 35.1 MB/s eta 0:00:00
Collecting google-api-core==2.11.0 (from -r requirements.lock (line 22))
Downloading google_api_core-2.11.0-py3-none-any.whl (120 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 120.3/120.3 kB 21.9 MB/s eta 0:00:00
Collecting google-auth==2.16.0 (from -r requirements.lock (line 23))
Downloading google_auth-2.16.0-py2.py3-none-any.whl (177 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.8/177.8 kB 30.5 MB/s eta 0:00:00
Collecting google-auth-httplib2==0.1.0 (from -r requirements.lock (line 24))
Downloading google_auth_httplib2-0.1.0-py2.py3-none-any.whl (9.3 kB)
Collecting googleapis-common-protos==1.58.0 (from -r requirements.lock (line 25))
Downloading googleapis_common_protos-1.58.0-py2.py3-none-any.whl (223 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 223.0/223.0 kB 36.9 MB/s eta 0:00:00
Collecting grpc-google-iam-v1==0.12.6 (from -r requirements.lock (line 26))
Downloading grpc_google_iam_v1-0.12.6-py2.py3-none-any.whl (26 kB)
Collecting grpcio-status==1.48.2 (from -r requirements.lock (line 27))
Downloading grpcio_status-1.48.2-py3-none-any.whl (14 kB)
Collecting httplib2==0.21.0 (from -r requirements.lock (line 28))
Downloading httplib2-0.21.0-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.8/96.8 kB 19.7 MB/s eta 0:00:00
Collecting idna==3.4 (from -r requirements.lock (line 29))
Downloading idna-3.4-py3-none-any.whl (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 12.0 MB/s eta 0:00:00
Collecting MarkupSafe==2.1.2 (from -r requirements.lock (line 30))
Downloading MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting oauthlib==3.2.2 (from -r requirements.lock (line 31))
Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 25.0 MB/s eta 0:00:00
Collecting proto-plus==1.22.2 (from -r requirements.lock (line 32))
Downloading proto_plus-1.22.2-py3-none-any.whl (47 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.9/47.9 kB 9.0 MB/s eta 0:00:00
Collecting pyasn1==0.4.8 (from -r requirements.lock (line 33))
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 14.8 MB/s eta 0:00:00
Collecting pyasn1-modules==0.2.8 (from -r requirements.lock (line 34))
Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 25.2 MB/s eta 0:00:00
Collecting pyparsing==3.0.9 (from -r requirements.lock (line 35))
Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 14.9 MB/s eta 0:00:00
Collecting requests==2.28.2 (from -r requirements.lock (line 36))
Downloading requests-2.28.2-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 12.0 MB/s eta 0:00:00
Collecting requests-oauthlib==1.3.1 (from -r requirements.lock (line 37))
Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting rsa==4.9 (from -r requirements.lock (line 38))
Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting uritemplate==3.0.1 (from -r requirements.lock (line 39))
Downloading uritemplate-3.0.1-py2.py3-none-any.whl (15 kB)
Collecting urllib3==1.26.14 (from -r requirements.lock (line 40))
Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 23.9 MB/s eta 0:00:00
Collecting websocket-client==1.5.1 (from -r requirements.lock (line 41))
Downloading websocket_client-1.5.1-py3-none-any.whl (55 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.9/55.9 kB 9.3 MB/s eta 0:00:00
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.0 (from google-cloud-secret-manager==2.15.1->-r requirements.lock (line 5))
Obtaining dependency information for google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.0 from https://files.pythonhosted.org/packages/6e/c4/c3cd048b6cbeba8d9ae50dd7643ac065b85237338aa7501b0efae91eb4d9/google_api_core-2.11.1-py3-none-any.whl.metadata
Downloading google_api_core-2.11.1-py3-none-any.whl.metadata (2.7 kB)
Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from grpcio-tools==1.48.2->-r requirements.lock (line 8)) (68.0.0)
Collecting googleapis-common-protos[grpc]<2.0.0dev,>=1.56.0 (from grpc-google-iam-v1==0.12.6->-r requirements.lock (line 26))
Obtaining dependency information for googleapis-common-protos[grpc]<2.0.0dev,>=1.56.0 from https://files.pythonhosted.org/packages/a7/bc/416a1ffeba4dcd072bc10523dac9ed97f2e7fc4b760580e2bdbdc1e2afdd/googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata
Downloading googleapis_common_protos-1.60.0-py2.py3-none-any.whl.metadata (1.5 kB)
INFO: pip is looking at multiple versions of google-api-core[grpc] to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of googleapis-common-protos[grpc] to determine which version is compatible with other requirements. This could take a while.
Obtaining dependency information for googleapis-common-protos[grpc]<2.0.0dev,>=1.56.0 from https://files.pythonhosted.org/packages/b3/b7/bbaa556e9ff0580f408c64ccf4db0c1414eec79e7151d33a10bc209ffb6d/googleapis_common_protos-1.59.1-py2.py3-none-any.whl.metadata
Downloading googleapis_common_protos-1.59.1-py2.py3-none-any.whl.metadata (1.5 kB)
Downloading googleapis_common_protos-1.59.0-py2.py3-none-any.whl (223 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 223.6/223.6 kB 31.9 MB/s eta 0:00:00
Downloading kubernetes-27.2.0-py2.py3-none-any.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 84.3 MB/s eta 0:00:00
Installing collected packages: pyasn1, charset-normalizer, websocket-client, urllib3, uritemplate, six, rsa, PyYAML, pyparsing, Pygments, pyasn1-modules, protobuf, oauthlib, MarkupSafe, idna, grpcio, certifi, cachetools, xds-protos, tenacity, requests, python-dateutil, proto-plus, packaging, Mako, httplib2, grpcio-tools, grpcio-health-checking, grpcio-channelz, googleapis-common-protos, google-auth, absl-py, requests-oauthlib, grpcio-status, google-auth-httplib2, google-api-core, kubernetes, grpc-google-iam-v1, google-api-python-client, google-cloud-secret-manager
Successfully installed Mako-1.2.4 MarkupSafe-2.1.2 PyYAML-6.0 Pygments-2.14.0 absl-py-0.15.0 cachetools-5.3.0 certifi-2022.12.7 charset-normalizer-3.0.1 google-api-core-2.11.0 google-api-python-client-1.12.11 google-auth-2.16.0 google-auth-httplib2-0.1.0 google-cloud-secret-manager-2.15.1 googleapis-common-protos-1.58.0 grpc-google-iam-v1-0.12.6 grpcio-1.51.1 grpcio-channelz-1.48.2 grpcio-health-checking-1.48.2 grpcio-status-1.48.2 grpcio-tools-1.48.2 httplib2-0.21.0 idna-3.4 kubernetes-27.2.0 oauthlib-3.2.2 packaging-21.3 proto-plus-1.22.2 protobuf-3.20.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-3.0.9 python-dateutil-2.8.2 requests-2.28.2 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tenacity-6.3.1 uritemplate-3.0.1 urllib3-1.26.14 websocket-client-1.5.1 xds-protos-0.0.11
Installed Python packages:
Package Version
--------------------------- ---------
absl-py 0.15.0
cachetools 5.3.0
certifi 2022.12.7
charset-normalizer 3.0.1
google-api-core 2.11.0
google-api-python-client 1.12.11
google-auth 2.16.0
google-auth-httplib2 0.1.0
google-cloud-secret-manager 2.15.1
googleapis-common-protos 1.58.0
grpc-google-iam-v1 0.12.6
grpcio 1.51.1
grpcio-channelz 1.48.2
grpcio-health-checking 1.48.2
grpcio-status 1.48.2
grpcio-tools 1.48.2
httplib2 0.21.0
idna 3.4
kubernetes 27.2.0
Mako 1.2.4
MarkupSafe 2.1.2
oauthlib 3.2.2
packaging 21.3
pip 23.2.1
proto-plus 1.22.2
protobuf 3.20.3
pyasn1 0.4.8
pyasn1-modules 0.2.8
Pygments 2.14.0
pyparsing 3.0.9
python-dateutil 2.8.2
PyYAML 6.0
requests 2.28.2
requests-oauthlib 1.3.1
rsa 4.9
setuptools 68.0.0
six 1.16.0
tenacity 6.3.1
uritemplate 3.0.1
urllib3 1.26.14
websocket-client 1.5.1
xds-protos 0.0.11
Generate python code from grpc.testing protos: src/proto/grpc/testing/test.proto src/proto/grpc/testing/messages.proto src/proto/grpc/testing/empty.proto
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/grpc_tools/protoc.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
Generated files /tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/src/proto/grpc/testing:
total 84
-rw-rw-r-- 1 kbuilder kbuilder 1163 Aug 11 18:45 empty_pb2.py
-rw-rw-r-- 1 kbuilder kbuilder 159 Aug 11 18:45 empty_pb2_grpc.py
-rw-rw-r-- 1 kbuilder kbuilder 32763 Aug 11 18:45 messages_pb2.py
-rw-rw-r-- 1 kbuilder kbuilder 159 Aug 11 18:45 messages_pb2_grpc.py
-rw-rw-r-- 1 kbuilder kbuilder 3934 Aug 11 18:45 test_pb2.py
-rw-rw-r-- 1 kbuilder kbuilder 34950 Aug 11 18:45 test_pb2_grpc.py
Server image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
Server image not found
Client image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
Client image not found
Building Go xDS interop test app Docker images
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.51kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/golang:1.21-alpine
#3 ...

#4 [internal] load metadata for docker.io/library/alpine:latest
#4 DONE 0.6s

#3 [internal] load metadata for docker.io/library/golang:1.21-alpine
#3 DONE 0.8s

#5 [build 1/4] FROM docker.io/library/golang:1.21-alpine@sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f
#5 resolve docker.io/library/golang:1.21-alpine@sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f 0.0s done
#5 sha256:c9e410d017b6e59e5c2c06f67b44cc13020b698b35b572edf9c94288220bc172 6.16kB / 6.16kB done
#5 sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 0B / 3.40MB 0.1s
#5 sha256:c4d48a809fc2256f8aa0aeee47998488d64409855adba00a7cb3007ab9f3286e 0B / 284.69kB 0.1s
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 0B / 66.88MB 0.1s
#5 sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f 1.65kB / 1.65kB done
#5 sha256:dd8888bb7f1b0b05e1e625aa29483f50f38a9b64073a4db00b04076cec52b71c 1.16kB / 1.16kB done
#5 sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 3.40MB / 3.40MB 0.2s done
#5 sha256:c4d48a809fc2256f8aa0aeee47998488d64409855adba00a7cb3007ab9f3286e 284.69kB / 284.69kB 0.2s
#5 extracting sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de
#5 sha256:ad3a456e5733f4fd4d5c3f67fcf931e1034a03ab86e308ca9e8cc62249ecf768 0B / 155B 0.2s
#5 sha256:c4d48a809fc2256f8aa0aeee47998488d64409855adba00a7cb3007ab9f3286e 284.69kB / 284.69kB 0.2s done
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 10.49MB / 66.88MB 0.3s
#5 sha256:ad3a456e5733f4fd4d5c3f67fcf931e1034a03ab86e308ca9e8cc62249ecf768 155B / 155B 0.2s done
#5 ...

#6 [stage-1 1/2] FROM docker.io/library/alpine@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a
#6 resolve docker.io/library/alpine@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a 0.0s done
#6 sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a 1.64kB / 1.64kB done
#6 sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33 528B / 528B done
#6 sha256:7e01a0d0a1dcd9e539f8e9bbd80106d59efbdf97293b3d38f5d7a34501526cdb 1.47kB / 1.47kB done
#6 sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 3.40MB / 3.40MB 0.2s done
#6 extracting sha256:7264a8db6415046d36d16ba98b79778e18accee6ffa71850405994cffa9be7de 0.1s done
#6 DONE 0.4s

#5 [build 1/4] FROM docker.io/library/golang:1.21-alpine@sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 27.26MB / 66.88MB 0.4s
#5 extracting sha256:c4d48a809fc2256f8aa0aeee47998488d64409855adba00a7cb3007ab9f3286e
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 38.65MB / 66.88MB 0.5s
#5 extracting sha256:c4d48a809fc2256f8aa0aeee47998488d64409855adba00a7cb3007ab9f3286e 0.1s done
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 53.48MB / 66.88MB 0.6s
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 66.88MB / 66.88MB 0.8s
#5 ...

#7 [internal] load build context
#7 transferring context: 76.82MB 0.7s done
#7 DONE 0.8s

#5 [build 1/4] FROM docker.io/library/golang:1.21-alpine@sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f
#5 sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 66.88MB / 66.88MB 0.8s done
#5 extracting sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d
#5 extracting sha256:4f1c88b9dad58987453186064cc54e131c5ec4b47f021c054e3d218e3e0f758d 2.3s done
#5 extracting sha256:ad3a456e5733f4fd4d5c3f67fcf931e1034a03ab86e308ca9e8cc62249ecf768 done
#5 DONE 3.5s

#8 [build 2/4] WORKDIR /go/src/grpc-go
#8 DONE 1.2s

#9 [build 3/4] COPY . .
#9 DONE 0.4s

#10 [build 4/4] RUN go build -tags osusergo,netgo interop/xds/client/client.go
#10 0.363 go: downloading google.golang.org/protobuf v1.31.0
#10 0.364 go: downloading github.com/golang/protobuf v1.5.3
#10 0.370 go: downloading github.com/envoyproxy/go-control-plane v0.11.1
#10 0.391 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98
#10 0.421 go: downloading golang.org/x/net v0.12.0
#10 0.440 go: downloading google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98
#10 0.481 go: downloading golang.org/x/sys v0.10.0
#10 0.859 go: downloading github.com/cespare/xxhash/v2 v2.2.0
#10 0.863 go: downloading github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4
#10 0.922 go: downloading github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe
#10 1.285 go: downloading github.com/envoyproxy/protoc-gen-validate v1.0.2
#10 1.285 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98
#10 1.286 go: downloading golang.org/x/oauth2 v0.10.0
#10 1.287 go: downloading golang.org/x/text v0.11.0
#10 1.287 go: downloading github.com/census-instrumentation/opencensus-proto v0.4.1
#10 1.290 go: downloading golang.org/x/sync v0.3.0
#10 1.366 go: downloading cloud.google.com/go/compute/metadata v0.2.3
#10 1.386 go: downloading cloud.google.com/go/compute v1.21.0
#10 DONE 18.5s

#11 [stage-1 2/2] COPY --from=build /go/src/grpc-go/client .
#11 DONE 0.1s

#12 exporting to image
#12 exporting layers
#12 exporting layers 0.2s done
#12 writing image sha256:d4ef674b5073a51789f99b5cc8d858124483ba3cb2fbe3d9b302e23bf55f1c20 done
#12 naming to gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4 done
#12 DONE 0.2s
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.51kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/golang:1.21-alpine
#3 DONE 0.1s

#4 [internal] load metadata for docker.io/library/alpine:latest
#4 DONE 0.2s

#5 [build 1/4] FROM docker.io/library/golang:1.21-alpine@sha256:445f34008a77b0b98bf1821bf7ef5e37bb63cc42d22ee7c21cc17041070d134f
#5 DONE 0.0s

#6 [stage-1 1/2] FROM docker.io/library/alpine@sha256:7144f7bab3d4c2648d7e59409f15ec52a18006a128c733fcff20d3a4a54ba44a
#6 CACHED

#7 [internal] load build context
#7 transferring context: 73.44kB 0.1s done
#7 DONE 0.1s

#8 [build 2/4] WORKDIR /go/src/grpc-go
#8 CACHED

#9 [build 3/4] COPY . .
#9 CACHED

#10 [build 4/4] RUN go build -tags osusergo,netgo interop/xds/server/server.go
#10 0.317 go: downloading google.golang.org/protobuf v1.31.0
#10 0.319 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98
#10 0.320 go: downloading github.com/envoyproxy/go-control-plane v0.11.1
#10 0.343 go: downloading github.com/golang/protobuf v1.5.3
#10 0.346 go: downloading golang.org/x/net v0.12.0
#10 0.411 go: downloading google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98
#10 0.736 go: downloading github.com/cespare/xxhash/v2 v2.2.0
#10 0.738 go: downloading github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4
#10 0.795 go: downloading github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe
#10 1.186 go: downloading golang.org/x/sys v0.10.0
#10 1.194 go: downloading github.com/envoyproxy/protoc-gen-validate v1.0.2
#10 1.194 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98
#10 1.198 go: downloading golang.org/x/text v0.11.0
#10 1.199 go: downloading golang.org/x/oauth2 v0.10.0
#10 1.199 go: downloading github.com/census-instrumentation/opencensus-proto v0.4.1
#10 1.203 go: downloading golang.org/x/sync v0.3.0
#10 1.283 go: downloading cloud.google.com/go/compute/metadata v0.2.3
#10 1.295 go: downloading cloud.google.com/go/compute v1.21.0
#10 DONE 18.5s

#11 [stage-1 2/2] COPY --from=build /go/src/grpc-go/server .
#11 DONE 0.1s

#12 exporting to image
#12 exporting layers
#12 exporting layers 0.2s done
#12 writing image sha256:acc8832691401293481234c4f8bcf5a402302ca26d0b48a85ab4070a6d4134ac done
#12 naming to gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4 done
#12 DONE 0.2s
Adding credentials for all GCR repositories.
WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
Docker configuration file updated.
The push refers to repository [gcr.io/grpc-testing/xds-interop/go-client]
880a862e47a1: Preparing
4693057ce236: Preparing
4693057ce236: Layer already exists
880a862e47a1: Pushed
0af3cad6ea5703485ec781cf0a15b4ae412010a4: digest: sha256:c0d896d78f4a6f536f99c49a6728b1b85dc956272b9a729e1ab933cf49f5a79a size: 740
The push refers to repository [gcr.io/grpc-testing/xds-interop/go-server]
a14cfc809c47: Preparing
4693057ce236: Preparing
4693057ce236: Layer already exists
a14cfc809c47: Pushed
0af3cad6ea5703485ec781cf0a15b4ae412010a4: digest: sha256:d1526cc07bf6dd4d984fdca7208b1226de37f4135a4cb8b17adecdc4ce3d13c5 size: 740
+ local out_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test
+ mkdir -pv /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test
mkdir: created directory '/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test'
+ python -m tests.baseline_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1-a_psm-interop-security --server_image=gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --client_image=gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --testing_version=dev-master --nocheck_local_certs --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test --xml_output_file=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test/sponge_log.xml
+ tee /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test/sponge_log.log
Running tests under Python 3.10.12: /tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/bin/python
I0811 18:46:16.280181 139724790939648 xds_k8s_testcase.py:130] ----- Testing BaselineTest -----
I0811 18:46:16.280323 139724790939648 xds_k8s_testcase.py:131] Logs timezone: UTC
I0811 18:46:16.280557 139724790939648 skips.py:124] Detected language and version: TestConfig(client_lang='go', server_lang='go', version='dev-master')
I0811 18:46:17.196259 139724790939648 k8s.py:130] Using kubernetes context "gke_grpc-testing_us-central1-a_psm-interop-security", active host: https://35.202.85.90
I0811 18:46:17.207187 139724790939648 k8s.py:130] Using kubernetes context "None", active host: https://35.202.85.90
[ RUN ] BaselineTest.test_traffic_director_grpc_setup
I0811 18:46:17.211619 139724790939648 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1846-5jjib
I0811 18:46:17.627263 139724790939648 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/baseline_test
I0811 18:46:17.932351 139724790939648 xds_k8s_testcase.py:625] Found unused xds port: 30342
I0811 18:46:17.932677 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.0_create_health_check ---
I0811 18:46:17.932760 139724790939648 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1846-5jjib"
I0811 18:46:17.935905 139724790939648 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
portSpecification: USE_SERVING_PORT
name: psm-interop-health-check-20230811-1846-5jjib
type: GRPC
...

I0811 18:46:18.603138 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779578013-602aa20c92792-ae02d039-b25e311a
I0811 18:46:21.073207 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.0_create_health_check ---
I0811 18:46:21.073372 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.1_create_backend_service ---
I0811 18:46:21.073440 139724790939648 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1846-5jjib"
I0811 18:46:21.080281 139724790939648 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1846-5jjib
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1846-5jjib
protocol: GRPC
...

I0811 18:46:21.915283 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779581082-602aa20f7fb3b-68fdba71-9d27d629
I0811 18:46:43.964860 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.1_create_backend_service ---
I0811 18:46:43.965064 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.2_create_url_map ---
I0811 18:46:43.965137 139724790939648 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1846-5jjib": psm-grpc-server:30342 -> psm-interop-backend-service-20230811-1846-5jjib
I0811 18:46:43.976026 139724790939648 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1846-5jjib
hostRules:
- hosts:
- psm-grpc-server:30342
pathMatcher: psm-interop-path-matcher-20230811-1846-5jjib
name: psm-interop-url-map-20230811-1846-5jjib
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1846-5jjib
name: psm-interop-path-matcher-20230811-1846-5jjib
...

I0811 18:46:44.855548 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779603978-602aa22555831-6b3ae85b-b2fff048
I0811 18:46:47.153437 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.2_create_url_map ---
I0811 18:46:47.153599 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.3_create_target_proxy ---
I0811 18:46:47.153666 139724790939648 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1846-5jjib proxy "GRPC" to URL map psm-interop-url-map-20230811-1846-5jjib
I0811 18:46:47.155541 139724790939648 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1846-5jjib
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1846-5jjib
validate_for_proxyless: true
...

I0811 18:46:48.200073 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779607157-602aa2285db17-32832570-8a3f5c72
I0811 18:46:50.447627 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.3_create_target_proxy ---
I0811 18:46:50.447842 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.4_create_forwarding_rule ---
I0811 18:46:50.447967 139724790939648 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1846-5jjib" in network "default-vpc": 0.0.0.0:30342 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1846-5jjib
I0811 18:46:50.450506 139724790939648 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1846-5jjib
network: global/networks/default-vpc
portRange: 30342
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1846-5jjib
...

I0811 18:46:51.105811 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779610452-602aa22b82251-8c087a47-63b8c5d2
I0811 18:47:04.320769 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.4_create_forwarding_rule ---
I0811 18:47:04.321001 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.5_start_test_server ---
I0811 18:47:04.321099 139724790939648 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1846-5jjib: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=1
I0811 18:47:04.321345 139724790939648 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1846-5jjib%22;timeRange=2023-08-11T18:47:04.321184Z%2F2023-08-11T19:17:04.321184Z?project=grpc-testing
I0811 18:47:04.326226 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1846-5jjib
labels:
name: psm-interop-server-20230811-1846-5jjib
owner: xds-k8s-interop-test
...

I0811 18:47:04.476648 139724790939648 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1846-5jjib created
I0811 18:47:04.479547 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1846-5jjib
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1846-5jjib-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 18:47:04.552677 139724790939648 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 18:47:04.552929 139724790939648 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 18:47:04.649452 139724790939648 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1846-5jjib-psm-grpc-server in zones=['us-central1-a']
I0811 18:47:04.649669 139724790939648 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1846-5jjib/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:47:05.582284 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1846-5jjib
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 18:47:05.641304 139724790939648 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 18:47:05.649361 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1846-5jjib
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-8xu9dvdm
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-8xu9dvdm
template:
metadata:
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-8xu9dvdm
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8080
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
ports:
- containerPort: 8080
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1846-5jjib-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 18:47:05.727804 139724790939648 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 18:47:05.728117 139724790939648 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 18:47:05.836108 139724790939648 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-6b996d75f9-6hpwj']
I0811 18:47:05.836364 139724790939648 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-6b996d75f9-6hpwj to start
I0811 18:47:09.076444 139724790939648 k8s_base_runner.py:554] Pod psm-grpc-server-6b996d75f9-6hpwj ready, IP: 10.12.17.7
I0811 18:47:09.076719 139724790939648 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-6b996d75f9-6hpwj to baseline_test/test_app_logs/psm-interop-server-20230811-1846-5jjib_psm-grpc-server-6b996d75f9-6hpwj.log
I0811 18:47:09.077182 139724700771904 k8s_log_collector.py:70] Starting log collection thread 139724700771904 for psm-grpc-server-6b996d75f9-6hpwj
I0811 18:47:09.077350 139724790939648 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 18:47:13.405508 139724790939648 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 18:47:13.405889 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.5_start_test_server ---
I0811 18:47:13.406029 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.6_add_server_backends_to_backend_service ---
I0811 18:47:13.442924 139724790939648 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 18:47:13.740764 139724790939648 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1846-5jjib-psm-grpc-server" in zone us-central1-a
I0811 18:47:13.740981 139724790939648 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1846-5jjib: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1846-5jjib-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1846-5jjib-psm-grpc-server', zone='us-central1-a')}
I0811 18:47:13.744958 139724790939648 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1846-5jjib-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 18:47:14.703753 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779633747-602aa241b9542-6866b31f-b3adf782
I0811 18:48:00.907604 139724790939648 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1846-5jjib to report all backends healthy: ['psm-interop-server-20230811-1846-5jjib-psm-grpc-server']
I0811 18:48:10.070815 139724790939648 compute.py:487] Backend psm-interop-server-20230811-1846-5jjib-psm-grpc-server in zone us-central1-a reported healthy
I0811 18:48:14.075472 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.6_add_server_backends_to_backend_service ---
I0811 18:48:14.075696 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.7_start_test_client ---
I0811 18:48:14.075798 139724790939648 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1846-5jjib: server_target=xds:///psm-grpc-server:30342 rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I0811 18:48:14.076072 139724790939648 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1846-5jjib%22;timeRange=2023-08-11T18:48:14.075885Z%2F2023-08-11T19:18:14.075885Z?project=grpc-testing
I0811 18:48:14.077999 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1846-5jjib
labels:
name: psm-interop-client-20230811-1846-5jjib
owner: xds-k8s-interop-test
...

I0811 18:48:14.121456 139724790939648 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1846-5jjib created
I0811 18:48:14.121688 139724790939648 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1846-5jjib/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:48:14.706018 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1846-5jjib
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 18:48:14.776395 139724790939648 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 18:48:14.784091 139724790939648 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1846-5jjib
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-krzw4phl
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-krzw4phl
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-krzw4phl
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:30342"
- "--stats_port=8079"
- "--qps=25"
- "--rpc=UnaryCall"
- "--metadata="
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
value: "true"
- name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
value: "true"
- name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 18:48:14.856174 139724790939648 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 18:48:14.856420 139724790939648 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 18:48:14.968658 139724790939648 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-78b77db78f-pkdwf']
I0811 18:48:14.968914 139724790939648 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-78b77db78f-pkdwf to start
I0811 18:48:18.207739 139724790939648 k8s_base_runner.py:554] Pod psm-grpc-client-78b77db78f-pkdwf ready, IP: 10.12.9.208
I0811 18:48:18.208070 139724790939648 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-78b77db78f-pkdwf to baseline_test/test_app_logs/psm-interop-client-20230811-1846-5jjib_psm-grpc-client-78b77db78f-pkdwf.log
I0811 18:48:18.208541 139724692379200 k8s_log_collector.py:70] Starting log collection thread 139724692379200 for psm-grpc-client-78b77db78f-pkdwf
I0811 18:48:18.208702 139724790939648 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 18:48:19.423576 139724790939648 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 18:48:19.424191 139724790939648 client_app.py:200] [psm-grpc-client-78b77db78f-pkdwf] Waiting to report a READY channel to xds:///psm-grpc-server:30342
I0811 18:48:19.528154 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:48:29.573035 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:48:39.618393 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:48:49.663479 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:48:59.708385 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:49:15.758246 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:49:40.818427 139724790939648 client_app.py:236] [psm-grpc-client-78b77db78f-pkdwf] Server channel:
I0811 18:49:40.851533 139724790939648 client_app.py:249] [psm-grpc-client-78b77db78f-pkdwf] Found subchannel in state READY:
I0811 18:49:40.851836 139724790939648 client_app.py:211] [psm-grpc-client-78b77db78f-pkdwf] Channel to xds:///psm-grpc-server:30342 transitioned to state READY:
I0811 18:49:40.851967 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.7_start_test_client ---
I0811 18:49:40.852076 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.8_test_client_xds_config_exists ---
I0811 18:49:40.852291 139724790939648 grpc.py:74] [psm-grpc-client-78b77db78f-pkdwf:8079] >> RPC ClientStatusDiscoveryService.FetchClientStatus(request=ClientStatusRequest({}), timeout=90, wait_for_ready=True)
I0811 18:49:40.887564 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.8_test_client_xds_config_exists ---
I0811 18:49:40.887786 139724790939648 xds_k8s_testcase.py:239] --- Starting subTest __main__.BaselineTest.test_traffic_director_grpc_setup.9_test_server_received_rpcs_from_test_client ---
I0811 18:49:40.888006 139724790939648 grpc.py:74] [psm-grpc-client-78b77db78f-pkdwf:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 18:49:44.919685 139724790939648 xds_k8s_testcase.py:558] [psm-grpc-client-78b77db78f-pkdwf] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-6b996d75f9-6hpwj: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-6b996d75f9-6hpwj: 100

I0811 18:49:44.919959 139724790939648 xds_k8s_testcase.py:244] --- Finished subTest __main__.BaselineTest.test_traffic_director_grpc_setup.9_test_server_received_rpcs_from_test_client ---
I0811 18:49:44.920107 139724790939648 xds_k8s_testcase.py:640] ----- TestMethod __main__.BaselineTest.test_traffic_director_grpc_setup teardown -----
I0811 18:49:45.085015 139724790939648 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1846-5jjib"
I0811 18:49:46.081815 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779785089-602aa2d20e14d-7ed3889e-cc25b06b
I0811 18:50:15.132241 139724790939648 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1846-5jjib"
I0811 18:50:15.300675 139724790939648 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:15.300904 139724790939648 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1846-5jjib"
I0811 18:50:15.792369 139724790939648 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:15.792583 139724790939648 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1846-5jjib"
I0811 18:50:16.446944 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779815796-602aa2ef56f33-941ad8e2-3d72c2ff
I0811 18:50:18.782871 139724790939648 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1846-5jjib"
I0811 18:50:19.066696 139724790939648 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:19.066938 139724790939648 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1846-5jjib"
I0811 18:50:19.885019 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779819073-602aa2f276f45-eed80d7d-2a834830
I0811 18:50:22.240393 139724790939648 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1846-5jjib"
I0811 18:50:22.706796 139724790939648 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:22.707046 139724790939648 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1846-5jjib"
I0811 18:50:23.626308 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779822712-602aa2f5ef56b-9d8a998a-e7f0cfdd
I0811 18:50:52.042784 139724790939648 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1846-5jjib"
I0811 18:50:52.445677 139724790939648 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:52.445899 139724790939648 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1846-5jjib"
I0811 18:50:52.817123 139724790939648 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1846-5jjib" not deleted since it does not exist
I0811 18:50:52.817337 139724790939648 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1846-5jjib"
I0811 18:50:53.373138 139724790939648 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779852891-602aa312b750d-cb27940e-764c0f99
I0811 18:50:57.917170 139724790939648 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/kubernetes/client/rest.py:44: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
return self.urllib3_response.getheaders()
I0811 18:51:08.095300 139724790939648 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1846-5jjib/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:51:08.719910 139724790939648 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 18:51:08.841026 139724790939648 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1846-5jjib
I0811 18:51:39.031232 139724790939648 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 18:51:49.262717 139724790939648 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 18:51:49.384111 139724790939648 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1846-5jjib/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:51:50.130478 139724790939648 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 18:51:50.247204 139724790939648 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1846-5jjib
I0811 18:52:20.404576 139724790939648 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 18:52:20.404951 139724790939648 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-krzw4phl:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1846-5jjib%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-krzw4phl%22;timeRange=2023-08-11T18:48:14.075876Z%2F2023-08-11T18:51:39.027537Z?project=grpc-testing
I0811 18:52:20.405147 139724790939648 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-8xu9dvdm:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1846-5jjib%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-8xu9dvdm%22;timeRange=2023-08-11T18:47:04.321177Z%2F2023-08-11T18:52:20.404423Z?project=grpc-testing
[ OK ] BaselineTest.test_traffic_director_grpc_setup
----------------------------------------------------------------------
Ran 1 test in 364.128s

OK
+ for test in "${test_suites[@]}"
+ run_test security_test
+ local test_name=security_test
+ set -x
+ local out_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test
+ mkdir -pv /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test
mkdir: created directory '/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test'
+ python -m tests.security_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1-a_psm-interop-security --server_image=gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --client_image=gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --testing_version=dev-master --nocheck_local_certs --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test --xml_output_file=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test/sponge_log.xml
+ tee /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test/sponge_log.log
Running tests under Python 3.10.12: /tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/bin/python
I0811 18:52:21.397061 140493315874816 xds_k8s_testcase.py:130] ----- Testing SecurityTest -----
I0811 18:52:21.397199 140493315874816 xds_k8s_testcase.py:131] Logs timezone: UTC
I0811 18:52:21.397480 140493315874816 skips.py:124] Detected language and version: TestConfig(client_lang='go', server_lang='go', version='dev-master')
I0811 18:52:21.409445 140493315874816 k8s.py:130] Using kubernetes context "gke_grpc-testing_us-central1-a_psm-interop-security", active host: https://35.202.85.90
I0811 18:52:21.420345 140493315874816 k8s.py:130] Using kubernetes context "None", active host: https://35.202.85.90
[ RUN ] SecurityTest.test_mtls
I0811 18:52:21.424768 140493315874816 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1852-bojwv
I0811 18:52:22.087465 140493315874816 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/security_test
I0811 18:52:22.420914 140493315874816 xds_k8s_testcase.py:625] Found unused xds port: 30413
I0811 18:52:22.421261 140493315874816 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1852-bojwv"
I0811 18:52:22.424495 140493315874816 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1852-bojwv
type: GRPC
...

I0811 18:52:23.024074 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779942486-602aa3682925b-8f19fda9-a79a72db
I0811 18:52:25.449578 140493315874816 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1852-bojwv"
I0811 18:52:25.456800 140493315874816 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1852-bojwv
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1852-bojwv
protocol: GRPC
...

I0811 18:52:26.382465 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779945459-602aa36afee33-d435ff7d-322da3d1
I0811 18:52:48.356985 140493315874816 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1852-bojwv": psm-grpc-server:30413 -> psm-interop-backend-service-20230811-1852-bojwv
I0811 18:52:48.368375 140493315874816 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1852-bojwv
hostRules:
- hosts:
- psm-grpc-server:30413
pathMatcher: psm-interop-path-matcher-20230811-1852-bojwv
name: psm-interop-url-map-20230811-1852-bojwv
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1852-bojwv
name: psm-interop-path-matcher-20230811-1852-bojwv
...

I0811 18:52:49.271773 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779968374-602aa380d98b4-3cf79535-88cb6002
I0811 18:52:51.613183 140493315874816 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1852-bojwv proxy "GRPC" to URL map psm-interop-url-map-20230811-1852-bojwv
I0811 18:52:51.615207 140493315874816 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1852-bojwv
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1852-bojwv
validate_for_proxyless: true
...

I0811 18:52:52.306019 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779971617-602aa383f12dc-6fc3fc9d-74b18099
I0811 18:52:54.696635 140493315874816 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1852-bojwv" in network "default-vpc": 0.0.0.0:30413 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1852-bojwv
I0811 18:52:54.699341 140493315874816 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1852-bojwv
network: global/networks/default-vpc
portRange: 30413
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1852-bojwv
...

I0811 18:52:55.600802 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779974701-602aa386e21ef-0be49be4-a46daab6
I0811 18:53:08.955157 140493315874816 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1852-bojwv
I0811 18:53:08.957780 140493315874816 api.py:468] Creating networksecurity resource:
---
clientCertificate: &id001
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
serverValidationCa:
- *id001
...

I0811 18:53:09.555658 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691779989142-602aa394a7dd7-c81d890f-ccc820a2
I0811 18:53:13.960015 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1852-bojwv:
---
clientCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
createTime: '2023-08-11T18:53:09.235078130Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1852-bojwv
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T18:53:12.949889530Z'
...

I0811 18:53:13.960344 140493315874816 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1852-bojwv: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1852-bojwv, server spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1852-bojwv/sa/psm-grpc-server
I0811 18:53:13.964229 140493315874816 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1852-bojwv
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1852-bojwv/sa/psm-grpc-server
...

I0811 18:53:14.589549 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691779993968-602aa39941fc0-7f101798-aba6fcec
I0811 18:53:34.442834 140493315874816 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1852-bojwv
I0811 18:53:34.445292 140493315874816 api.py:468] Creating networksecurity resource:
---
mtlsPolicy:
clientValidationCa:
- &id001
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
serverCertificate: *id001
...

I0811 18:53:34.774790 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780014517-602aa3acdaeab-f681e63a-b812b8e0
I0811 18:53:55.934389 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1852-bojwv:
---
createTime: '2023-08-11T18:53:34.596827188Z'
mtlsPolicy:
clientValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1852-bojwv
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T18:53:55.010034583Z'
...

I0811 18:53:55.934650 140493315874816 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1852-bojwv
I0811 18:53:55.938169 140493315874816 api.py:468] Creating networkservices resource:
---
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1852-bojwv-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1852-bojwv
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 18:53:56.426016 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691780036093-602aa3c16e610-75df3556-89b7a874
I0811 18:54:21.391686 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1852-bojwv:
---
createTime: '2023-08-11T18:53:56.156774378Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1852-bojwv-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1852-bojwv
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1852-bojwv
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T18:54:19.348675362Z'
...

I0811 18:54:21.391982 140493315874816 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1852-bojwv: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 18:54:21.392274 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1852-bojwv%22;timeRange=2023-08-11T18:54:21.392120Z%2F2023-08-11T19:24:21.392120Z?project=grpc-testing
I0811 18:54:21.397405 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1852-bojwv
labels:
name: psm-interop-server-20230811-1852-bojwv
owner: xds-k8s-interop-test
...

I0811 18:54:21.548766 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1852-bojwv created
I0811 18:54:21.551840 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1852-bojwv
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1852-bojwv-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 18:54:21.616295 140493315874816 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 18:54:21.616513 140493315874816 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 18:54:21.709953 140493315874816 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1852-bojwv-psm-grpc-server in zones=['us-central1-a']
I0811 18:54:21.710169 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1852-bojwv/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:54:22.463898 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1852-bojwv
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 18:54:22.533317 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 18:54:22.541582 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1852-bojwv
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-6jxrhzwc
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-6jxrhzwc
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-6jxrhzwc
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1852-bojwv-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 18:54:22.620605 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 18:54:22.620831 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 18:54:22.729843 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-84f5588f6c-zj6g6']
I0811 18:54:22.730129 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-84f5588f6c-zj6g6 to start
I0811 18:54:30.151324 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-server-84f5588f6c-zj6g6 ready, IP: 10.12.15.164
I0811 18:54:30.151595 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-84f5588f6c-zj6g6 to security_test/test_app_logs/psm-interop-server-20230811-1852-bojwv_psm-grpc-server-84f5588f6c-zj6g6.log
I0811 18:54:30.152036 140493224142400 k8s_log_collector.py:70] Starting log collection thread 140493224142400 for psm-grpc-server-84f5588f6c-zj6g6
I0811 18:54:30.152245 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 18:54:33.457675 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 18:54:33.495034 140493315874816 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 18:54:33.607755 140493315874816 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1852-bojwv-psm-grpc-server" in zone us-central1-a
I0811 18:54:33.608006 140493315874816 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1852-bojwv: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1852-bojwv-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1852-bojwv-psm-grpc-server', zone='us-central1-a')}
I0811 18:54:33.611925 140493315874816 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1852-bojwv-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 18:54:34.769191 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780073615-602aa3e536f5f-c69d44b1-ff495b11
I0811 18:55:20.640320 140493315874816 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1852-bojwv to report all backends healthy: ['psm-interop-server-20230811-1852-bojwv-psm-grpc-server']
I0811 18:55:25.161970 140493315874816 compute.py:487] Backend psm-interop-server-20230811-1852-bojwv-psm-grpc-server in zone us-central1-a reported healthy
I0811 18:55:29.166641 140493315874816 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1852-bojwv: server_target=xds:///psm-grpc-server:30413 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 18:55:29.167034 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1852-bojwv%22;timeRange=2023-08-11T18:55:29.166879Z%2F2023-08-11T19:25:29.166879Z?project=grpc-testing
I0811 18:55:29.169002 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1852-bojwv
labels:
name: psm-interop-client-20230811-1852-bojwv
owner: xds-k8s-interop-test
...

I0811 18:55:29.211281 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1852-bojwv created
I0811 18:55:29.211524 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1852-bojwv/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:55:29.979572 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1852-bojwv
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 18:55:30.038735 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 18:55:30.046213 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1852-bojwv
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ukmk9wob
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ukmk9wob
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ukmk9wob
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:30413"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 18:55:30.118850 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 18:55:30.119091 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 18:55:30.227700 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-767dc4555c-kzxjs']
I0811 18:55:30.228004 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-767dc4555c-kzxjs to start
I0811 18:55:34.513287 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-client-767dc4555c-kzxjs ready, IP: 10.12.10.10
I0811 18:55:34.513572 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-767dc4555c-kzxjs to security_test/test_app_logs/psm-interop-client-20230811-1852-bojwv_psm-grpc-client-767dc4555c-kzxjs.log
I0811 18:55:34.514001 140493144979008 k8s_log_collector.py:70] Starting log collection thread 140493144979008 for psm-grpc-client-767dc4555c-kzxjs
I0811 18:55:34.514158 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 18:55:35.728430 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 18:55:35.729009 140493315874816 client_app.py:200] [psm-grpc-client-767dc4555c-kzxjs] Waiting to report a READY channel to xds:///psm-grpc-server:30413
I0811 18:55:35.831588 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:55:45.877225 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:55:55.922548 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:05.968147 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:16.012866 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:32.062168 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:57.122597 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:57.155171 140493315874816 client_app.py:249] [psm-grpc-client-767dc4555c-kzxjs] Found subchannel in state READY:
I0811 18:56:57.155495 140493315874816 client_app.py:211] [psm-grpc-client-767dc4555c-kzxjs] Channel to xds:///psm-grpc-server:30413 transitioned to state READY:
I0811 18:56:57.189010 140493315874816 client_app.py:236] [psm-grpc-client-767dc4555c-kzxjs] Server channel:
I0811 18:56:57.221670 140493315874816 client_app.py:249] [psm-grpc-client-767dc4555c-kzxjs] Found subchannel in state READY:
I0811 18:56:57.480966 140493315874816 server_app.py:174] [psm-grpc-server-84f5588f6c-zj6g6] Found matching socket pair: server(local=10.12.15.164:8080, remote=10.12.10.10:55062) <-> client(local=10.12.10.10:55062, remote=10.12.15.164:8080)
I0811 18:56:57.481261 140493315874816 xds_k8s_testcase.py:922] Server certs: local: , remote:
I0811 18:56:57.481369 140493315874816 xds_k8s_testcase.py:923] Client certs: local: , remote:
I0811 18:56:57.481621 140493315874816 grpc.py:74] [psm-grpc-client-767dc4555c-kzxjs:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 18:57:01.516307 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-767dc4555c-kzxjs] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-84f5588f6c-zj6g6: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-84f5588f6c-zj6g6: 100

I0811 18:57:01.516527 140493315874816 security_test.py:62] [SUCCESS] mTLS security mode confirmed.
I0811 18:57:01.516620 140493315874816 xds_k8s_testcase.py:640] ----- TestMethod __main__.SecurityTest.test_mtls teardown -----
I0811 18:57:01.662581 140493315874816 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1852-bojwv"
I0811 18:57:02.471314 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780221667-602aa47268766-221aa1c2-7acb95b1
I0811 18:57:37.142281 140493315874816 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1852-bojwv"
I0811 18:57:37.301288 140493315874816 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:57:37.301500 140493315874816 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1852-bojwv"
I0811 18:57:37.739377 140493315874816 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:57:37.739612 140493315874816 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1852-bojwv"
I0811 18:57:38.277667 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780257743-602aa494d0272-23258306-78163ad5
I0811 18:57:40.682765 140493315874816 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1852-bojwv"
I0811 18:57:41.186861 140493315874816 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:57:41.187081 140493315874816 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1852-bojwv"
I0811 18:57:41.904161 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780261193-602aa4981a56a-4b500d8c-e218995c
I0811 18:57:44.320131 140493315874816 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1852-bojwv"
I0811 18:57:44.715006 140493315874816 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:57:44.715217 140493315874816 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1852-bojwv"
I0811 18:57:45.350388 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780264720-602aa49b776b9-1050b5fe-9ea4c010
I0811 18:58:13.373146 140493315874816 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1852-bojwv"
I0811 18:58:13.724757 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:58:13.724976 140493315874816 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1852-bojwv"
I0811 18:58:14.192086 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1852-bojwv" not deleted since it does not exist
I0811 18:58:14.192332 140493315874816 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1852-bojwv"
I0811 18:58:14.712814 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780294275-602aa4b7a72ae-2092f7a7-b2ef91e9
I0811 18:58:19.197279 140493315874816 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1852-bojwv
I0811 18:58:19.594665 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691780299283-602aa4bc6dcd7-fc169413-f8bdb87c
I0811 18:58:23.825019 140493315874816 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1852-bojwv
I0811 18:58:24.119485 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780303915-602aa4c0d8b53-57a28c0b-09ee9b2b
I0811 18:58:28.398556 140493315874816 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1852-bojwv
I0811 18:58:28.677543 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780308472-602aa4c531211-caa27d30-f3c02181
I0811 18:58:32.945798 140493315874816 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1852-bojwv
I0811 18:58:33.016937 140493315874816 api.py:504] projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1852-bojwv not deleted since it does not exist
I0811 18:58:33.020197 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/kubernetes/client/rest.py:44: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
return self.urllib3_response.getheaders()
I0811 18:58:43.259703 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1852-bojwv/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:58:44.146593 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 18:58:44.270862 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1852-bojwv
I0811 18:59:14.494405 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 18:59:24.661457 140493315874816 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 18:59:24.786669 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1852-bojwv/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 18:59:25.531399 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 18:59:25.651064 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1852-bojwv
I0811 19:00:55.965785 140493315874816 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:00:55.966193 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-ukmk9wob:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1852-bojwv%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-ukmk9wob%22;timeRange=2023-08-11T18:55:29.166869Z%2F2023-08-11T18:59:14.490456Z?project=grpc-testing
I0811 19:00:55.966361 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-6jxrhzwc:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1852-bojwv%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-6jxrhzwc%22;timeRange=2023-08-11T18:54:21.392111Z%2F2023-08-11T19:00:55.965625Z?project=grpc-testing
[ OK ] SecurityTest.test_mtls
[ RUN ] SecurityTest.test_mtls_error
I0811 19:00:55.966790 140493315874816 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1900-o8lpb
I0811 19:00:56.233906 140493315874816 xds_k8s_testcase.py:625] Found unused xds port: 11741
I0811 19:00:56.234185 140493315874816 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1900-o8lpb"
I0811 19:00:56.236694 140493315874816 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1900-o8lpb
type: GRPC
...

I0811 19:00:56.930369 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780456295-602aa5522ac01-2942123a-8e034709
I0811 19:00:59.276915 140493315874816 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1900-o8lpb"
I0811 19:00:59.281040 140493315874816 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1900-o8lpb
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1900-o8lpb
protocol: GRPC
...

I0811 19:01:00.122771 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780459283-602aa55504329-c178c095-39248318
I0811 19:01:15.489463 140493315874816 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1900-o8lpb: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:01:15.489806 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1900-o8lpb%22;timeRange=2023-08-11T19:01:15.489686Z%2F2023-08-11T19:31:15.489686Z?project=grpc-testing
I0811 19:01:15.491860 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1900-o8lpb
labels:
name: psm-interop-server-20230811-1900-o8lpb
owner: xds-k8s-interop-test
...

I0811 19:01:15.569468 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1900-o8lpb created
I0811 19:01:15.572356 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1900-o8lpb
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1900-o8lpb-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:01:15.645934 140493315874816 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:01:15.646164 140493315874816 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:01:15.744729 140493315874816 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1900-o8lpb-psm-grpc-server in zones=['us-central1-a']
I0811 19:01:15.744940 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1900-o8lpb/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:01:16.417188 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1900-o8lpb
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:01:16.479732 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:01:16.487519 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1900-o8lpb
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-52eg8ct5
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-52eg8ct5
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-52eg8ct5
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1900-o8lpb-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:01:16.566389 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:01:16.566630 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:01:16.667094 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-84bdf54f5f-fsrfg']
I0811 19:01:16.667362 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-84bdf54f5f-fsrfg to start
I0811 19:01:21.994907 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-server-84bdf54f5f-fsrfg ready, IP: 10.12.14.27
I0811 19:01:21.995196 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-84bdf54f5f-fsrfg to security_test/test_app_logs/psm-interop-server-20230811-1900-o8lpb_psm-grpc-server-84bdf54f5f-fsrfg.log
I0811 19:01:21.995545 140493224142400 k8s_log_collector.py:70] Starting log collection thread 140493224142400 for psm-grpc-server-84bdf54f5f-fsrfg
I0811 19:01:21.995684 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:01:26.336565 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:01:26.372947 140493315874816 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:01:26.641576 140493315874816 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1900-o8lpb-psm-grpc-server" in zone us-central1-a
I0811 19:01:26.641813 140493315874816 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1900-o8lpb: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1900-o8lpb-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1900-o8lpb-psm-grpc-server', zone='us-central1-a')}
I0811 19:01:26.646011 140493315874816 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1900-o8lpb-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:01:27.666181 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780486648-602aa56f1d244-16ae111e-eb6b5c10
I0811 19:02:20.581743 140493315874816 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1900-o8lpb
I0811 19:02:20.583877 140493315874816 api.py:468] Creating networksecurity resource:
---
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:02:20.961327 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780540664-602aa5a2a0be9-af98216d-73632422
I0811 19:02:39.887089 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1900-o8lpb:
---
createTime: '2023-08-11T19:02:20.751082461Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1900-o8lpb
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:02:37.883658614Z'
...

I0811 19:02:39.887407 140493315874816 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1900-o8lpb: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1900-o8lpb, server spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1900-o8lpb/sa/psm-grpc-server
I0811 19:02:39.891729 140493315874816 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1900-o8lpb
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1900-o8lpb/sa/psm-grpc-server
...

I0811 19:02:40.848718 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780559895-602aa5b4f7a47-12406172-c69f424c
I0811 19:03:00.712682 140493315874816 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1900-o8lpb
I0811 19:03:00.714910 140493315874816 api.py:468] Creating networksecurity resource:
---
mtlsPolicy:
clientValidationCa:
- &id001
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
serverCertificate: *id001
...

I0811 19:03:01.056413 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780580793-602aa5c8e5d51-14e51cfc-5ad6a5ce
I0811 19:03:05.555200 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1900-o8lpb:
---
createTime: '2023-08-11T19:03:00.857616136Z'
mtlsPolicy:
clientValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1900-o8lpb
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:03:04.493181457Z'
...

I0811 19:03:05.555476 140493315874816 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1900-o8lpb
I0811 19:03:05.558140 140493315874816 api.py:468] Creating networkservices resource:
---
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1900-o8lpb-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1900-o8lpb
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:03:05.890146 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691780585635-602aa5cd83d65-a1d063f0-2c1c439e
I0811 19:03:12.261595 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1900-o8lpb:
---
createTime: '2023-08-11T19:03:05.702617874Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1900-o8lpb-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1900-o8lpb
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1900-o8lpb
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:03:09.956971438Z'
...

I0811 19:03:12.261863 140493315874816 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1900-o8lpb": psm-grpc-server:11741 -> psm-interop-backend-service-20230811-1900-o8lpb
I0811 19:03:12.266977 140493315874816 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1900-o8lpb
hostRules:
- hosts:
- psm-grpc-server:11741
pathMatcher: psm-interop-path-matcher-20230811-1900-o8lpb
name: psm-interop-url-map-20230811-1900-o8lpb
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1900-o8lpb
name: psm-interop-path-matcher-20230811-1900-o8lpb
...

I0811 19:03:13.126316 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780592269-602aa5d3d795b-16b3ec91-6ef65425
I0811 19:03:15.639810 140493315874816 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1900-o8lpb proxy "GRPC" to URL map psm-interop-url-map-20230811-1900-o8lpb
I0811 19:03:15.641837 140493315874816 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1900-o8lpb
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1900-o8lpb
validate_for_proxyless: true
...

I0811 19:03:16.144706 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780595644-602aa5d70f811-158bf67e-fb52c3e4
I0811 19:03:18.575403 140493315874816 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1900-o8lpb" in network "default-vpc": 0.0.0.0:11741 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1900-o8lpb
I0811 19:03:18.578147 140493315874816 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1900-o8lpb
network: global/networks/default-vpc
portRange: 11741
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1900-o8lpb
...

I0811 19:03:19.492029 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780598580-602aa5d9dc4b0-2625cbdf-97965b82
I0811 19:03:32.944523 140493315874816 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1900-o8lpb to report all backends healthy: ['psm-interop-server-20230811-1900-o8lpb-psm-grpc-server']
I0811 19:03:33.224685 140493315874816 compute.py:487] Backend psm-interop-server-20230811-1900-o8lpb-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:03:37.229626 140493315874816 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1900-o8lpb: server_target=xds:///psm-grpc-server:11741 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:03:37.230114 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1900-o8lpb%22;timeRange=2023-08-11T19:03:37.229894Z%2F2023-08-11T19:33:37.229894Z?project=grpc-testing
I0811 19:03:37.232227 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1900-o8lpb
labels:
name: psm-interop-client-20230811-1900-o8lpb
owner: xds-k8s-interop-test
...

I0811 19:03:37.374545 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1900-o8lpb created
I0811 19:03:37.374769 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1900-o8lpb/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:03:38.024554 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1900-o8lpb
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:03:38.081498 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:03:38.089224 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1900-o8lpb
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ung51ljt
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ung51ljt
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ung51ljt
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:11741"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:03:38.165977 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:03:38.166213 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:03:38.266418 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-58f645ff75-c9s4v']
I0811 19:03:38.266712 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-58f645ff75-c9s4v to start
I0811 19:03:42.547816 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-client-58f645ff75-c9s4v ready, IP: 10.12.9.211
I0811 19:03:42.548136 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-58f645ff75-c9s4v to security_test/test_app_logs/psm-interop-client-20230811-1900-o8lpb_psm-grpc-client-58f645ff75-c9s4v.log
I0811 19:03:42.548479 140493144979008 k8s_log_collector.py:70] Starting log collection thread 140493144979008 for psm-grpc-client-58f645ff75-c9s4v
I0811 19:03:42.548660 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:03:43.690825 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:03:43.691422 140493315874816 client_app.py:200] [psm-grpc-client-58f645ff75-c9s4v] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:11741
I0811 19:03:43.790761 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:03:43.791052 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:03:53.836562 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:03:53.836828 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:04:03.882077 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:04:03.882364 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:04:13.928262 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:04:13.928523 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:04:23.971039 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:04:23.971334 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:04:40.020069 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:04:40.020370 140493315874816 client_app.py:257] [psm-grpc-client-58f645ff75-c9s4v] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:05:05.080312 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:05:05.112545 140493315874816 client_app.py:249] [psm-grpc-client-58f645ff75-c9s4v] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:05:05.112873 140493315874816 client_app.py:211] [psm-grpc-client-58f645ff75-c9s4v] Channel to xds:///psm-grpc-server:11741 transitioned to state TRANSIENT_FAILURE:
I0811 19:05:05.145215 140493315874816 grpc.py:74] [psm-grpc-client-58f645ff75-c9s4v:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:05:09.151118 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-58f645ff75-c9s4v] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:05:09.151374 140493315874816 xds_k8s_testcase.py:1095] Check 1 passed, waiting 0:00:10 before the next check
I0811 19:05:19.161922 140493315874816 client_app.py:200] [psm-grpc-client-58f645ff75-c9s4v] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:11741
I0811 19:05:19.197942 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:05:19.230339 140493315874816 client_app.py:249] [psm-grpc-client-58f645ff75-c9s4v] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:05:19.230792 140493315874816 client_app.py:211] [psm-grpc-client-58f645ff75-c9s4v] Channel to xds:///psm-grpc-server:11741 transitioned to state TRANSIENT_FAILURE:
I0811 19:05:19.263092 140493315874816 grpc.py:74] [psm-grpc-client-58f645ff75-c9s4v:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:05:23.271171 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-58f645ff75-c9s4v] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:05:23.271433 140493315874816 xds_k8s_testcase.py:1095] Check 2 passed, waiting 0:00:10 before the next check
I0811 19:05:33.281880 140493315874816 client_app.py:200] [psm-grpc-client-58f645ff75-c9s4v] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:11741
I0811 19:05:33.316979 140493315874816 client_app.py:236] [psm-grpc-client-58f645ff75-c9s4v] Server channel:
I0811 19:05:33.349393 140493315874816 client_app.py:249] [psm-grpc-client-58f645ff75-c9s4v] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:05:33.349804 140493315874816 client_app.py:211] [psm-grpc-client-58f645ff75-c9s4v] Channel to xds:///psm-grpc-server:11741 transitioned to state TRANSIENT_FAILURE:
I0811 19:05:33.381927 140493315874816 grpc.py:74] [psm-grpc-client-58f645ff75-c9s4v:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:05:37.390530 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-58f645ff75-c9s4v] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:05:37.390794 140493315874816 security_test.py:160] [SUCCESS] Client's connectivity state is consistent with a mTLS error caused by not presenting mTLS certificate to the server.
I0811 19:05:37.390918 140493315874816 xds_k8s_testcase.py:640] ----- TestMethod __main__.SecurityTest.test_mtls_error teardown -----
I0811 19:05:37.605305 140493315874816 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1900-o8lpb"
I0811 19:05:38.236150 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780737609-602aa65e72ead-aab79668-85f3b8b7
I0811 19:05:51.543628 140493315874816 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1900-o8lpb"
I0811 19:05:51.736118 140493315874816 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:05:51.736344 140493315874816 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1900-o8lpb"
I0811 19:05:52.222874 140493315874816 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:05:52.223125 140493315874816 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1900-o8lpb"
I0811 19:05:52.985759 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780752226-602aa66c63a26-947bfb44-86f22319
I0811 19:05:55.346898 140493315874816 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1900-o8lpb"
I0811 19:05:55.584504 140493315874816 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:05:55.584884 140493315874816 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1900-o8lpb"
I0811 19:05:56.382196 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780755590-602aa66f98cfb-2977b062-d38f538b
I0811 19:06:00.891283 140493315874816 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1900-o8lpb"
I0811 19:06:01.292090 140493315874816 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:06:01.292461 140493315874816 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1900-o8lpb"
I0811 19:06:02.051883 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780761297-602aa6750a227-1196effb-626e52cf
I0811 19:06:30.647094 140493315874816 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1900-o8lpb"
I0811 19:06:31.111936 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:06:31.112256 140493315874816 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1900-o8lpb"
I0811 19:06:31.589234 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1900-o8lpb" not deleted since it does not exist
I0811 19:06:31.589447 140493315874816 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1900-o8lpb"
I0811 19:06:32.085127 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780791681-602aa692042c1-cb298a27-c16cfa32
I0811 19:06:36.793336 140493315874816 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1900-o8lpb
I0811 19:06:37.088464 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691780796878-602aa696f902c-d30998eb-842828e8
I0811 19:06:49.705569 140493315874816 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1900-o8lpb
I0811 19:06:50.053782 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780809811-602aa6a34e7a2-ef829317-c2ebe45d
I0811 19:06:52.185928 140493315874816 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1900-o8lpb
I0811 19:06:52.564095 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691780812258-602aa6a5a3cf3-bacaa5d9-4675a87d
I0811 19:06:56.801899 140493315874816 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1900-o8lpb
I0811 19:06:56.875841 140493315874816 api.py:504] projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1900-o8lpb not deleted since it does not exist
I0811 19:06:56.880125 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 19:07:07.057758 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1900-o8lpb/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:07:07.921884 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:07:08.052454 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1900-o8lpb
I0811 19:07:38.281725 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:07:48.454585 140493315874816 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:07:48.579646 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1900-o8lpb/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:07:49.216430 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:07:49.340202 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1900-o8lpb
I0811 19:09:19.657535 140493315874816 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:09:19.657962 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-ung51ljt:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1900-o8lpb%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-ung51ljt%22;timeRange=2023-08-11T19:03:37.229878Z%2F2023-08-11T19:07:38.278572Z?project=grpc-testing
I0811 19:09:19.658135 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-52eg8ct5:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1900-o8lpb%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-52eg8ct5%22;timeRange=2023-08-11T19:01:15.489678Z%2F2023-08-11T19:09:19.657359Z?project=grpc-testing
[ OK ] SecurityTest.test_mtls_error
[ RUN ] SecurityTest.test_plaintext_fallback
I0811 19:09:19.658570 140493315874816 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1909-7mz72
I0811 19:09:19.837198 140493315874816 xds_k8s_testcase.py:625] Found unused xds port: 64848
I0811 19:09:19.837464 140493315874816 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1909-7mz72"
I0811 19:09:19.840689 140493315874816 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1909-7mz72
type: GRPC
...

I0811 19:09:20.522674 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780959903-602aa732720d1-c284ca0e-0a9af9a5
I0811 19:09:23.017322 140493315874816 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1909-7mz72"
I0811 19:09:23.021572 140493315874816 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1909-7mz72
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1909-7mz72
protocol: GRPC
...

I0811 19:09:23.825989 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780963023-602aa7356bbee-4bcd9d41-80dfd0bf
I0811 19:09:56.733935 140493315874816 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1909-7mz72": psm-grpc-server:64848 -> psm-interop-backend-service-20230811-1909-7mz72
I0811 19:09:56.739182 140493315874816 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1909-7mz72
hostRules:
- hosts:
- psm-grpc-server:64848
pathMatcher: psm-interop-path-matcher-20230811-1909-7mz72
name: psm-interop-url-map-20230811-1909-7mz72
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1909-7mz72
name: psm-interop-path-matcher-20230811-1909-7mz72
...

I0811 19:09:57.298529 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780996744-602aa755943ae-5b691098-95651a63
I0811 19:09:59.612596 140493315874816 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1909-7mz72 proxy "GRPC" to URL map psm-interop-url-map-20230811-1909-7mz72
I0811 19:09:59.614568 140493315874816 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1909-7mz72
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1909-7mz72
validate_for_proxyless: true
...

I0811 19:10:00.054319 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691780999616-602aa758518ea-7804b6e8-f0604cb4
I0811 19:10:02.434964 140493315874816 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1909-7mz72" in network "default-vpc": 0.0.0.0:64848 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1909-7mz72
I0811 19:10:02.437802 140493315874816 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1909-7mz72
network: global/networks/default-vpc
portRange: 64848
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1909-7mz72
...

I0811 19:10:03.198052 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781002440-602aa75b03030-7bb4f73e-333151eb
I0811 19:10:14.040648 140493315874816 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1909-7mz72
W0811 19:10:14.040869 140493315874816 traffic_director.py:1019] Client TLS Policy psm-interop-client-tls-policy-20230811-1909-7mz72 neither TLS, nor mTLS policy. Skipping creation
W0811 19:10:14.040943 140493315874816 traffic_director.py:1056] Client TLS policy not created, skipping attaching to Backend Service psm-interop-backend-service-20230811-1909-7mz72
I0811 19:10:14.041008 140493315874816 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1909-7mz72
W0811 19:10:14.041098 140493315874816 traffic_director.py:908] Server TLS Policy psm-interop-server-tls-policy-20230811-1909-7mz72 neither TLS, nor mTLS policy. Skipping creation
I0811 19:10:14.041161 140493315874816 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1909-7mz72
W0811 19:10:14.041213 140493315874816 traffic_director.py:990] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1909-7mz72 with no Server TLS policy attached
I0811 19:10:14.043674 140493315874816 api.py:468] Creating networkservices resource:
---
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1909-7mz72-psm-grpc-server
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:10:14.400904 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691781014098-602aa766211f8-d2aa3ed0-a35c36f3
I0811 19:10:18.742464 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1909-7mz72:
---
createTime: '2023-08-11T19:10:14.165748142Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1909-7mz72-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1909-7mz72
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:10:17.821655241Z'
...

I0811 19:10:18.742771 140493315874816 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1909-7mz72: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:10:18.743034 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1909-7mz72%22;timeRange=2023-08-11T19:10:18.742867Z%2F2023-08-11T19:40:18.742867Z?project=grpc-testing
I0811 19:10:18.744943 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1909-7mz72
labels:
name: psm-interop-server-20230811-1909-7mz72
owner: xds-k8s-interop-test
...

I0811 19:10:18.825007 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1909-7mz72 created
I0811 19:10:18.827901 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1909-7mz72
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1909-7mz72-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:10:18.894280 140493315874816 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:10:18.894513 140493315874816 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:10:18.998212 140493315874816 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1909-7mz72-psm-grpc-server in zones=['us-central1-a']
I0811 19:10:18.998447 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1909-7mz72/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:10:19.702341 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1909-7mz72
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:10:19.763323 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:10:19.771139 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1909-7mz72
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-dco3lcme
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-dco3lcme
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-dco3lcme
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1909-7mz72-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:10:19.849137 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:10:19.849397 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:10:19.957914 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-75589d4f75-5zr8h']
I0811 19:10:19.958262 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-75589d4f75-5zr8h to start
I0811 19:10:26.337893 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-server-75589d4f75-5zr8h ready, IP: 10.12.15.165
I0811 19:10:26.338182 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-75589d4f75-5zr8h to security_test/test_app_logs/psm-interop-server-20230811-1909-7mz72_psm-grpc-server-75589d4f75-5zr8h.log
I0811 19:10:26.338571 140493224142400 k8s_log_collector.py:70] Starting log collection thread 140493224142400 for psm-grpc-server-75589d4f75-5zr8h
I0811 19:10:26.338714 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:10:28.602025 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:10:28.639780 140493315874816 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:10:28.922650 140493315874816 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1909-7mz72-psm-grpc-server" in zone us-central1-a
I0811 19:10:28.922879 140493315874816 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1909-7mz72: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1909-7mz72-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1909-7mz72-psm-grpc-server', zone='us-central1-a')}
I0811 19:10:28.926969 140493315874816 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1909-7mz72-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:10:29.871840 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781028930-602aa774462ce-6a471e57-55a8f90c
I0811 19:11:16.295079 140493315874816 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1909-7mz72 to report all backends healthy: ['psm-interop-server-20230811-1909-7mz72-psm-grpc-server']
I0811 19:11:33.766177 140493315874816 compute.py:487] Backend psm-interop-server-20230811-1909-7mz72-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:11:37.770900 140493315874816 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1909-7mz72: server_target=xds:///psm-grpc-server:64848 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:11:37.771310 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1909-7mz72%22;timeRange=2023-08-11T19:11:37.771170Z%2F2023-08-11T19:41:37.771170Z?project=grpc-testing
I0811 19:11:37.773271 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1909-7mz72
labels:
name: psm-interop-client-20230811-1909-7mz72
owner: xds-k8s-interop-test
...

I0811 19:11:37.814875 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1909-7mz72 created
I0811 19:11:37.815248 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1909-7mz72/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:11:38.531531 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1909-7mz72
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:11:38.592910 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:11:38.600610 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1909-7mz72
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-3rw1co4a
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-3rw1co4a
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-3rw1co4a
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:64848"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:11:38.680566 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:11:38.680813 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:11:38.796311 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-95bdbb849-5hmn9']
I0811 19:11:38.796616 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-95bdbb849-5hmn9 to start
I0811 19:11:45.168543 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-client-95bdbb849-5hmn9 ready, IP: 10.12.14.28
I0811 19:11:45.168811 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-95bdbb849-5hmn9 to security_test/test_app_logs/psm-interop-client-20230811-1909-7mz72_psm-grpc-client-95bdbb849-5hmn9.log
I0811 19:11:45.169191 140493144979008 k8s_log_collector.py:70] Starting log collection thread 140493144979008 for psm-grpc-client-95bdbb849-5hmn9
I0811 19:11:45.169372 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:11:46.383119 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:11:46.383723 140493315874816 client_app.py:200] [psm-grpc-client-95bdbb849-5hmn9] Waiting to report a READY channel to xds:///psm-grpc-server:64848
I0811 19:11:46.483267 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:11:56.528988 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:12:06.574429 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:12:16.619596 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:12:26.664685 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:12:42.714089 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:13:07.773611 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:13:07.806237 140493315874816 client_app.py:249] [psm-grpc-client-95bdbb849-5hmn9] Found subchannel in state READY:
I0811 19:13:07.806549 140493315874816 client_app.py:211] [psm-grpc-client-95bdbb849-5hmn9] Channel to xds:///psm-grpc-server:64848 transitioned to state READY:
I0811 19:13:07.839107 140493315874816 client_app.py:236] [psm-grpc-client-95bdbb849-5hmn9] Server channel:
I0811 19:13:07.871631 140493315874816 client_app.py:249] [psm-grpc-client-95bdbb849-5hmn9] Found subchannel in state READY:
I0811 19:13:08.131125 140493315874816 server_app.py:174] [psm-grpc-server-75589d4f75-5zr8h] Found matching socket pair: server(local=10.12.15.165:8080, remote=10.12.14.28:43680) <-> client(local=10.12.14.28:43680, remote=10.12.15.165:8080)
I0811 19:13:08.131371 140493315874816 xds_k8s_testcase.py:922] Server certs: local: , remote:
I0811 19:13:08.131479 140493315874816 xds_k8s_testcase.py:923] Client certs: local: , remote:
I0811 19:13:08.131691 140493315874816 grpc.py:74] [psm-grpc-client-95bdbb849-5hmn9:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:13:12.137174 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-95bdbb849-5hmn9] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-75589d4f75-5zr8h: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-75589d4f75-5zr8h: 100

I0811 19:13:12.137431 140493315874816 security_test.py:107] [SUCCESS] Plaintext security mode confirmed.
I0811 19:13:12.137561 140493315874816 xds_k8s_testcase.py:640] ----- TestMethod __main__.SecurityTest.test_plaintext_fallback teardown -----
I0811 19:13:12.290791 140493315874816 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1909-7mz72"
I0811 19:13:12.922351 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781192294-602aa8101219a-b33baef1-680596a0
I0811 19:13:21.769088 140493315874816 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1909-7mz72"
I0811 19:13:21.972949 140493315874816 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:13:21.973179 140493315874816 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1909-7mz72"
I0811 19:13:22.498687 140493315874816 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:13:22.498916 140493315874816 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1909-7mz72"
I0811 19:13:23.128214 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781202502-602aa819ce342-50f39736-a63c93ea
I0811 19:13:25.464594 140493315874816 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1909-7mz72"
I0811 19:13:25.752635 140493315874816 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:13:25.752842 140493315874816 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1909-7mz72"
I0811 19:13:26.456344 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781205757-602aa81ce8f0d-dfd7c857-acfa6db3
I0811 19:13:28.933187 140493315874816 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1909-7mz72"
I0811 19:13:29.406643 140493315874816 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:13:29.406879 140493315874816 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1909-7mz72"
I0811 19:13:30.150525 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781209412-602aa8206520f-ef43ee57-c94f0328
I0811 19:14:02.872027 140493315874816 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1909-7mz72"
I0811 19:14:03.311396 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:14:03.311632 140493315874816 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1909-7mz72"
I0811 19:14:03.721267 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1909-7mz72" not deleted since it does not exist
I0811 19:14:03.721479 140493315874816 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1909-7mz72"
I0811 19:14:04.336878 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781243817-602aa84134ca4-f24cf3dc-00e7f784
I0811 19:14:08.995925 140493315874816 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1909-7mz72
I0811 19:14:09.286130 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691781249059-602aa84634a35-62b8b9f2-f11da8da
I0811 19:14:25.952411 140493315874816 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1909-7mz72
I0811 19:14:26.090747 140493315874816 api.py:504] projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1909-7mz72 not deleted since it does not exist
I0811 19:14:26.090989 140493315874816 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1909-7mz72
I0811 19:14:26.169161 140493315874816 api.py:504] projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1909-7mz72 not deleted since it does not exist
I0811 19:14:26.169384 140493315874816 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1909-7mz72
I0811 19:14:26.240664 140493315874816 api.py:504] projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1909-7mz72 not deleted since it does not exist
I0811 19:14:26.244126 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 19:14:36.413988 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1909-7mz72/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:14:37.024761 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:14:37.155854 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1909-7mz72
I0811 19:15:07.355521 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:15:17.577126 140493315874816 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:15:17.695091 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1909-7mz72/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:15:18.332256 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:15:18.445764 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1909-7mz72
I0811 19:17:18.874527 140493315874816 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:17:18.874932 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-3rw1co4a:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1909-7mz72%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-3rw1co4a%22;timeRange=2023-08-11T19:11:37.771162Z%2F2023-08-11T19:15:07.352953Z?project=grpc-testing
I0811 19:17:18.875100 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-dco3lcme:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1909-7mz72%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-dco3lcme%22;timeRange=2023-08-11T19:10:18.742859Z%2F2023-08-11T19:17:18.874341Z?project=grpc-testing
[ OK ] SecurityTest.test_plaintext_fallback
[ RUN ] SecurityTest.test_server_authz_error
I0811 19:17:18.875525 140493315874816 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1917-gdkde
I0811 19:17:19.013504 140493315874816 xds_k8s_testcase.py:625] Found unused xds port: 2094
I0811 19:17:19.013802 140493315874816 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1917-gdkde"
I0811 19:17:19.016444 140493315874816 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1917-gdkde
type: GRPC
...

I0811 19:17:19.718981 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781439086-602aa8fb6df18-d40af092-09ec8837
I0811 19:17:22.198903 140493315874816 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1917-gdkde"
I0811 19:17:22.203459 140493315874816 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1917-gdkde
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1917-gdkde
protocol: GRPC
...

I0811 19:17:23.065660 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781442205-602aa8fe677f4-2aebb8e1-b33ec90b
I0811 19:17:42.699861 140493315874816 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1917-gdkde: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:17:42.700289 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1917-gdkde%22;timeRange=2023-08-11T19:17:42.700130Z%2F2023-08-11T19:47:42.700130Z?project=grpc-testing
I0811 19:17:42.702479 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1917-gdkde
labels:
name: psm-interop-server-20230811-1917-gdkde
owner: xds-k8s-interop-test
...

I0811 19:17:42.753511 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1917-gdkde created
I0811 19:17:42.756393 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1917-gdkde
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1917-gdkde-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:17:42.824502 140493315874816 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:17:42.824724 140493315874816 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:17:42.920653 140493315874816 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1917-gdkde-psm-grpc-server in zones=['us-central1-a']
I0811 19:17:42.920862 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1917-gdkde/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:17:43.636841 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1917-gdkde
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:17:43.694931 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:17:43.702645 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1917-gdkde
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-euirndle
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-euirndle
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-euirndle
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1917-gdkde-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:17:43.777206 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:17:43.777443 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:17:43.889528 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-7668b6bb4d-wfdnl']
I0811 19:17:43.889818 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-7668b6bb4d-wfdnl to start
I0811 19:17:48.172995 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-server-7668b6bb4d-wfdnl ready, IP: 10.12.15.166
I0811 19:17:48.173313 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-7668b6bb4d-wfdnl to security_test/test_app_logs/psm-interop-server-20230811-1917-gdkde_psm-grpc-server-7668b6bb4d-wfdnl.log
I0811 19:17:48.173681 140493224142400 k8s_log_collector.py:70] Starting log collection thread 140493224142400 for psm-grpc-server-7668b6bb4d-wfdnl
I0811 19:17:48.174050 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:17:51.400174 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:17:51.436836 140493315874816 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:17:51.750662 140493315874816 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1917-gdkde-psm-grpc-server" in zone us-central1-a
I0811 19:17:51.750880 140493315874816 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1917-gdkde: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1917-gdkde-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1917-gdkde-psm-grpc-server', zone='us-central1-a')}
I0811 19:17:51.754872 140493315874816 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1917-gdkde-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:17:52.723530 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781471757-602aa91a96448-97b25269-e118a4d6
I0811 19:18:34.105016 140493315874816 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1917-gdkde
I0811 19:18:34.107013 140493315874816 api.py:468] Creating networksecurity resource:
---
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:18:34.511413 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691781514194-602aa9430ecec-98cb87a7-0b906bb5
I0811 19:18:39.016258 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1917-gdkde:
---
createTime: '2023-08-11T19:18:34.264362972Z'
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1917-gdkde
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:18:37.952903737Z'
...

I0811 19:18:39.016505 140493315874816 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1917-gdkde
I0811 19:18:39.019264 140493315874816 api.py:468] Creating networkservices resource:
---
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1917-gdkde-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1917-gdkde
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:18:39.550338 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691781519105-602aa947bde5c-cb64fda3-17053711
I0811 19:18:43.888482 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1917-gdkde:
---
createTime: '2023-08-11T19:18:39.172383206Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1917-gdkde-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1917-gdkde
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1917-gdkde
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:18:42.906329898Z'
...

I0811 19:18:43.888788 140493315874816 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1917-gdkde
I0811 19:18:43.890583 140493315874816 api.py:468] Creating networksecurity resource:
---
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:18:44.233119 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691781523967-602aa94c60da1-76475f46-3b4f2e71
I0811 19:18:48.628908 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1917-gdkde:
---
createTime: '2023-08-11T19:18:44.055379142Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1917-gdkde
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:18:47.615893478Z'
...

I0811 19:18:48.629183 140493315874816 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1917-gdkde: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1917-gdkde, server spiffe://grpc-testing.svc.id.goog/ns/incorrect-namespace-VLeeb8tN/sa/psm-grpc-server
I0811 19:18:48.633049 140493315874816 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1917-gdkde
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/incorrect-namespace-VLeeb8tN/sa/psm-grpc-server
...

I0811 19:18:49.694354 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781528635-602aa950d492f-c2c5e1c4-b9671540
I0811 19:19:24.650136 140493315874816 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1917-gdkde": psm-grpc-server:2094 -> psm-interop-backend-service-20230811-1917-gdkde
I0811 19:19:24.655172 140493315874816 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1917-gdkde
hostRules:
- hosts:
- psm-grpc-server:2094
pathMatcher: psm-interop-path-matcher-20230811-1917-gdkde
name: psm-interop-url-map-20230811-1917-gdkde
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1917-gdkde
name: psm-interop-path-matcher-20230811-1917-gdkde
...

I0811 19:19:25.529308 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781564658-602aa9732f246-23a310cd-574aaa74
I0811 19:19:27.938720 140493315874816 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1917-gdkde proxy "GRPC" to URL map psm-interop-url-map-20230811-1917-gdkde
I0811 19:19:27.940711 140493315874816 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1917-gdkde
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1917-gdkde
validate_for_proxyless: true
...

I0811 19:19:28.355973 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781567943-602aa97651265-59497665-1a9c6f9f
I0811 19:19:30.724007 140493315874816 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1917-gdkde" in network "default-vpc": 0.0.0.0:2094 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1917-gdkde
I0811 19:19:30.726962 140493315874816 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1917-gdkde
network: global/networks/default-vpc
portRange: 2094
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1917-gdkde
...

I0811 19:19:31.471428 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781570729-602aa978f9689-e784beba-a1781119
I0811 19:19:44.433355 140493315874816 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1917-gdkde to report all backends healthy: ['psm-interop-server-20230811-1917-gdkde-psm-grpc-server']
I0811 19:19:44.735082 140493315874816 compute.py:487] Backend psm-interop-server-20230811-1917-gdkde-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:19:48.740028 140493315874816 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1917-gdkde: server_target=xds:///psm-grpc-server:2094 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:19:48.740476 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1917-gdkde%22;timeRange=2023-08-11T19:19:48.740288Z%2F2023-08-11T19:49:48.740288Z?project=grpc-testing
I0811 19:19:48.742430 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1917-gdkde
labels:
name: psm-interop-client-20230811-1917-gdkde
owner: xds-k8s-interop-test
...

I0811 19:19:48.889889 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1917-gdkde created
I0811 19:19:48.890169 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1917-gdkde/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:19:49.624498 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1917-gdkde
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:19:49.689825 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:19:49.697180 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1917-gdkde
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-mgidz665
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-mgidz665
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-mgidz665
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:2094"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:19:49.768545 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:19:49.768783 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:19:49.878922 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-579c9bc658-8s8cs']
I0811 19:19:49.879240 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-579c9bc658-8s8cs to start
I0811 19:19:54.160520 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-client-579c9bc658-8s8cs ready, IP: 10.12.13.12
I0811 19:19:54.160808 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-579c9bc658-8s8cs to security_test/test_app_logs/psm-interop-client-20230811-1917-gdkde_psm-grpc-client-579c9bc658-8s8cs.log
I0811 19:19:54.161167 140493144979008 k8s_log_collector.py:70] Starting log collection thread 140493144979008 for psm-grpc-client-579c9bc658-8s8cs
I0811 19:19:54.161313 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:19:55.398140 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:19:55.398776 140493315874816 client_app.py:200] [psm-grpc-client-579c9bc658-8s8cs] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:2094
I0811 19:19:55.499175 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:19:55.499439 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:20:05.544719 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:20:05.544985 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:20:15.590245 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:20:15.590497 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:20:25.636210 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:20:25.636480 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:20:35.681473 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:20:35.681747 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:20:51.731183 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:20:51.731450 140493315874816 client_app.py:257] [psm-grpc-client-579c9bc658-8s8cs] Not found a TRANSIENT_FAILURE subchannel for channel_id 2
I0811 19:21:16.791778 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:21:16.824654 140493315874816 client_app.py:249] [psm-grpc-client-579c9bc658-8s8cs] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:21:16.824980 140493315874816 client_app.py:211] [psm-grpc-client-579c9bc658-8s8cs] Channel to xds:///psm-grpc-server:2094 transitioned to state TRANSIENT_FAILURE:
I0811 19:21:16.858946 140493315874816 grpc.py:74] [psm-grpc-client-579c9bc658-8s8cs:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:21:20.867702 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-579c9bc658-8s8cs] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:21:20.867896 140493315874816 xds_k8s_testcase.py:1095] Check 1 passed, waiting 0:00:10 before the next check
I0811 19:21:30.878410 140493315874816 client_app.py:200] [psm-grpc-client-579c9bc658-8s8cs] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:2094
I0811 19:21:30.913561 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:21:30.946981 140493315874816 client_app.py:249] [psm-grpc-client-579c9bc658-8s8cs] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:21:30.947462 140493315874816 client_app.py:211] [psm-grpc-client-579c9bc658-8s8cs] Channel to xds:///psm-grpc-server:2094 transitioned to state TRANSIENT_FAILURE:
I0811 19:21:30.980166 140493315874816 grpc.py:74] [psm-grpc-client-579c9bc658-8s8cs:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:21:34.986781 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-579c9bc658-8s8cs] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:21:34.987079 140493315874816 xds_k8s_testcase.py:1095] Check 2 passed, waiting 0:00:10 before the next check
I0811 19:21:44.997546 140493315874816 client_app.py:200] [psm-grpc-client-579c9bc658-8s8cs] Waiting to report a TRANSIENT_FAILURE channel to xds:///psm-grpc-server:2094
I0811 19:21:45.033249 140493315874816 client_app.py:236] [psm-grpc-client-579c9bc658-8s8cs] Server channel:
I0811 19:21:45.066348 140493315874816 client_app.py:249] [psm-grpc-client-579c9bc658-8s8cs] Found subchannel in state TRANSIENT_FAILURE:
I0811 19:21:45.066774 140493315874816 client_app.py:211] [psm-grpc-client-579c9bc658-8s8cs] Channel to xds:///psm-grpc-server:2094 transitioned to state TRANSIENT_FAILURE:
I0811 19:21:45.099707 140493315874816 grpc.py:74] [psm-grpc-client-579c9bc658-8s8cs:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:21:49.107256 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-579c9bc658-8s8cs] << Received LoadBalancerStatsResponse:
num_failures: 100
rpcs_by_peer: {}
rpcs_by_method: {}

I0811 19:21:49.107584 140493315874816 security_test.py:212] [SUCCESS] Client's connectivity state is consistent with AuthZ error caused by server presenting incorrect SAN.
I0811 19:21:49.107731 140493315874816 xds_k8s_testcase.py:640] ----- TestMethod __main__.SecurityTest.test_server_authz_error teardown -----
I0811 19:21:49.338094 140493315874816 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1917-gdkde"
I0811 19:21:49.998549 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781709343-602aa9fd2a93e-f989d675-c0359f4b
I0811 19:22:16.537980 140493315874816 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1917-gdkde"
I0811 19:22:16.705501 140493315874816 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:16.705715 140493315874816 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1917-gdkde"
I0811 19:22:17.207351 140493315874816 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:17.207566 140493315874816 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1917-gdkde"
I0811 19:22:17.640472 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781737210-602aaa17be39e-ac7f4e8a-f082efc1
I0811 19:22:19.846224 140493315874816 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1917-gdkde"
I0811 19:22:20.096087 140493315874816 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:20.096301 140493315874816 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1917-gdkde"
I0811 19:22:20.850416 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781740101-602aaa1a7fec4-19bb6bee-264c3f7b
I0811 19:22:23.193095 140493315874816 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1917-gdkde"
I0811 19:22:23.510566 140493315874816 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:23.510793 140493315874816 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1917-gdkde"
I0811 19:22:24.217150 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781743515-602aaa1dc1848-53a50857-4c729d54
I0811 19:22:52.331158 140493315874816 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1917-gdkde"
I0811 19:22:52.751118 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:52.751345 140493315874816 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1917-gdkde"
I0811 19:22:53.168488 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1917-gdkde" not deleted since it does not exist
I0811 19:22:53.168714 140493315874816 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1917-gdkde"
I0811 19:22:53.710681 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781773264-602aaa3a204cf-ae08c232-3169634f
I0811 19:22:58.214775 140493315874816 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1917-gdkde
I0811 19:22:58.544553 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691781778303-602aaa3eee8bd-8301e482-867fadd8
I0811 19:23:15.137764 140493315874816 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1917-gdkde
I0811 19:23:15.490195 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691781795269-602aaa4f1cc4b-86e40fe8-a0cc44cc
I0811 19:23:40.925209 140493315874816 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1917-gdkde
I0811 19:23:41.341341 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691781821029-602aaa67ade07-6df1e30c-c284e2d9
I0811 19:23:45.624151 140493315874816 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1917-gdkde
I0811 19:23:45.694227 140493315874816 api.py:504] projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1917-gdkde not deleted since it does not exist
I0811 19:23:45.697390 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 19:23:55.981152 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1917-gdkde/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:23:56.774674 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:23:56.895547 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1917-gdkde
I0811 19:24:27.081661 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:24:37.249388 140493315874816 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:24:37.372616 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1917-gdkde/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:24:38.237672 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:24:38.358078 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1917-gdkde
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/kubernetes/client/rest.py:44: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
return self.urllib3_response.getheaders()
I0811 19:25:38.623078 140493315874816 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:25:38.623431 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-mgidz665:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1917-gdkde%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-mgidz665%22;timeRange=2023-08-11T19:19:48.740279Z%2F2023-08-11T19:24:27.078633Z?project=grpc-testing
I0811 19:25:38.623581 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-euirndle:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1917-gdkde%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-euirndle%22;timeRange=2023-08-11T19:17:42.700121Z%2F2023-08-11T19:25:38.622877Z?project=grpc-testing
[ OK ] SecurityTest.test_server_authz_error
[ RUN ] SecurityTest.test_tls
I0811 19:25:38.623962 140493315874816 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1925-e8fqh
I0811 19:25:38.720614 140493315874816 xds_k8s_testcase.py:625] Found unused xds port: 44929
I0811 19:25:38.720882 140493315874816 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1925-e8fqh"
I0811 19:25:38.723403 140493315874816 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1925-e8fqh
type: GRPC
...

I0811 19:25:39.358404 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781938786-602aaad7fafc1-4b18b3c0-17f5b621
I0811 19:25:41.699087 140493315874816 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1925-e8fqh"
I0811 19:25:41.703560 140493315874816 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1925-e8fqh
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1925-e8fqh
protocol: GRPC
...

I0811 19:25:42.443718 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691781941706-602aaadac3dc6-7a6c7f48-55557f61
I0811 19:26:41.946797 140493315874816 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1925-e8fqh": psm-grpc-server:44929 -> psm-interop-backend-service-20230811-1925-e8fqh
I0811 19:26:41.953296 140493315874816 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1925-e8fqh
hostRules:
- hosts:
- psm-grpc-server:44929
pathMatcher: psm-interop-path-matcher-20230811-1925-e8fqh
name: psm-interop-url-map-20230811-1925-e8fqh
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1925-e8fqh
name: psm-interop-path-matcher-20230811-1925-e8fqh
...

I0811 19:26:42.851383 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782001955-602aab1439486-2ae91b61-5132287c
I0811 19:26:45.275884 140493315874816 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1925-e8fqh proxy "GRPC" to URL map psm-interop-url-map-20230811-1925-e8fqh
I0811 19:26:45.277846 140493315874816 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1925-e8fqh
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1925-e8fqh
validate_for_proxyless: true
...

I0811 19:26:46.109395 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782005279-602aab1764cd8-2d9360a3-e8829d7a
I0811 19:26:48.573672 140493315874816 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1925-e8fqh" in network "default-vpc": 0.0.0.0:44929 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1925-e8fqh
I0811 19:26:48.576527 140493315874816 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1925-e8fqh
network: global/networks/default-vpc
portRange: 44929
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1925-e8fqh
...

I0811 19:26:49.273652 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782008579-602aab1a8a463-f35ce738-a11ab950
I0811 19:27:02.787871 140493315874816 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1925-e8fqh
I0811 19:27:02.789872 140493315874816 api.py:468] Creating networksecurity resource:
---
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:27:03.139792 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782022870-602aab282b6be-6b3d420c-6519d856
I0811 19:27:22.164433 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1925-e8fqh:
---
createTime: '2023-08-11T19:27:02.949334089Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1925-e8fqh
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:27:20.740098181Z'
...

I0811 19:27:22.164718 140493315874816 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1925-e8fqh: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1925-e8fqh, server spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1925-e8fqh/sa/psm-grpc-server
I0811 19:27:22.168847 140493315874816 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1925-e8fqh
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1925-e8fqh/sa/psm-grpc-server
...

I0811 19:27:22.994738 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782042171-602aab3a93a3e-04d877e6-ed796000
I0811 19:28:00.092714 140493315874816 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1925-e8fqh
I0811 19:28:00.094711 140493315874816 api.py:468] Creating networksecurity resource:
---
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:28:00.554296 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782080163-602aab5eced08-7416e077-6adea324
I0811 19:28:05.095191 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1925-e8fqh:
---
createTime: '2023-08-11T19:28:00.245641404Z'
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1925-e8fqh
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:28:04.044923673Z'
...

I0811 19:28:05.095480 140493315874816 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1925-e8fqh
I0811 19:28:05.098077 140493315874816 api.py:468] Creating networkservices resource:
---
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1925-e8fqh-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1925-e8fqh
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:28:05.491390 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691782085225-602aab63a2ad2-5715ddca-d5a0744e
I0811 19:28:09.776773 140493315874816 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1925-e8fqh:
---
createTime: '2023-08-11T19:28:05.295379113Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1925-e8fqh-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1925-e8fqh
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1925-e8fqh
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:28:08.841675556Z'
...

I0811 19:28:09.777081 140493315874816 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1925-e8fqh: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:28:09.777333 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1925-e8fqh%22;timeRange=2023-08-11T19:28:09.777181Z%2F2023-08-11T19:58:09.777181Z?project=grpc-testing
I0811 19:28:09.779212 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1925-e8fqh
labels:
name: psm-interop-server-20230811-1925-e8fqh
owner: xds-k8s-interop-test
...

I0811 19:28:09.927355 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1925-e8fqh created
I0811 19:28:09.930321 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1925-e8fqh
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1925-e8fqh-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:28:10.005548 140493315874816 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:28:10.005775 140493315874816 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:28:10.113006 140493315874816 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1925-e8fqh-psm-grpc-server in zones=['us-central1-a']
I0811 19:28:10.113316 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1925-e8fqh/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:28:10.823043 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1925-e8fqh
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:28:10.888747 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:28:10.896273 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1925-e8fqh
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-rql1gfg3
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-rql1gfg3
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-rql1gfg3
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1925-e8fqh-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:28:10.983585 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:28:10.983843 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:28:11.102618 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-864cd957f7-8hpms']
I0811 19:28:11.102936 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-864cd957f7-8hpms to start
I0811 19:28:18.568489 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-server-864cd957f7-8hpms ready, IP: 10.12.14.30
I0811 19:28:18.568752 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-864cd957f7-8hpms to security_test/test_app_logs/psm-interop-server-20230811-1925-e8fqh_psm-grpc-server-864cd957f7-8hpms.log
I0811 19:28:18.569085 140493224142400 k8s_log_collector.py:70] Starting log collection thread 140493224142400 for psm-grpc-server-864cd957f7-8hpms
I0811 19:28:18.569330 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:28:21.893936 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:28:21.936260 140493315874816 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:28:22.197328 140493315874816 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1925-e8fqh-psm-grpc-server" in zone us-central1-a
I0811 19:28:22.197553 140493315874816 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1925-e8fqh: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1925-e8fqh-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1925-e8fqh-psm-grpc-server', zone='us-central1-a')}
I0811 19:28:22.201585 140493315874816 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1925-e8fqh-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:28:23.101633 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782102203-602aab73d3d54-059cb6f0-63ad292f
I0811 19:29:24.731129 140493315874816 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1925-e8fqh to report all backends healthy: ['psm-interop-server-20230811-1925-e8fqh-psm-grpc-server']
I0811 19:29:24.972998 140493315874816 compute.py:487] Backend psm-interop-server-20230811-1925-e8fqh-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:29:28.977864 140493315874816 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1925-e8fqh: server_target=xds:///psm-grpc-server:44929 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:29:28.978327 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1925-e8fqh%22;timeRange=2023-08-11T19:29:28.978151Z%2F2023-08-11T19:59:28.978151Z?project=grpc-testing
I0811 19:29:28.980248 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1925-e8fqh
labels:
name: psm-interop-client-20230811-1925-e8fqh
owner: xds-k8s-interop-test
...

I0811 19:29:29.029080 140493315874816 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1925-e8fqh created
I0811 19:29:29.029398 140493315874816 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1925-e8fqh/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
W0811 19:29:29.030448 140493315874816 http.py:163] Sleeping 1.17 seconds before retry 1 of 5 for request: POST https://iam.googleapis.com/v1/projects/grpc-testing/serviceAccounts/xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com:getIamPolicy?options.requestedPolicyVersion=3&alt=json, after [Errno 32] Broken pipe
I0811 19:29:30.866330 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1925-e8fqh
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:29:30.943937 140493315874816 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:29:30.951533 140493315874816 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1925-e8fqh
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2wbtz9j8
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2wbtz9j8
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-2wbtz9j8
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:44929"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:29:31.054121 140493315874816 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:29:31.054370 140493315874816 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:29:31.171004 140493315874816 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-57666bdc87-zgf4d']
I0811 19:29:31.171284 140493315874816 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-57666bdc87-zgf4d to start
I0811 19:29:36.433038 140493315874816 k8s_base_runner.py:554] Pod psm-grpc-client-57666bdc87-zgf4d ready, IP: 10.12.15.167
I0811 19:29:36.433307 140493315874816 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-57666bdc87-zgf4d to security_test/test_app_logs/psm-interop-client-20230811-1925-e8fqh_psm-grpc-client-57666bdc87-zgf4d.log
I0811 19:29:36.433666 140493144979008 k8s_log_collector.py:70] Starting log collection thread 140493144979008 for psm-grpc-client-57666bdc87-zgf4d
I0811 19:29:36.433802 140493315874816 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:29:37.650993 140493315874816 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:29:37.651583 140493315874816 client_app.py:200] [psm-grpc-client-57666bdc87-zgf4d] Waiting to report a READY channel to xds:///psm-grpc-server:44929
I0811 19:29:37.751161 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:29:47.796402 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:29:57.842006 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:07.887037 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:17.932123 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:33.981577 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:59.041697 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:59.074370 140493315874816 client_app.py:249] [psm-grpc-client-57666bdc87-zgf4d] Found subchannel in state READY:
I0811 19:30:59.074699 140493315874816 client_app.py:211] [psm-grpc-client-57666bdc87-zgf4d] Channel to xds:///psm-grpc-server:44929 transitioned to state READY:
I0811 19:30:59.107344 140493315874816 client_app.py:236] [psm-grpc-client-57666bdc87-zgf4d] Server channel:
I0811 19:30:59.140192 140493315874816 client_app.py:249] [psm-grpc-client-57666bdc87-zgf4d] Found subchannel in state READY:
I0811 19:30:59.401360 140493315874816 server_app.py:174] [psm-grpc-server-864cd957f7-8hpms] Found matching socket pair: server(local=10.12.14.30:8080, remote=10.12.15.167:49578) <-> client(local=10.12.15.167:49578, remote=10.12.14.30:8080)
I0811 19:30:59.401616 140493315874816 xds_k8s_testcase.py:922] Server certs: local: , remote:
I0811 19:30:59.401744 140493315874816 xds_k8s_testcase.py:923] Client certs: local: , remote:
I0811 19:30:59.401976 140493315874816 grpc.py:74] [psm-grpc-client-57666bdc87-zgf4d:8079] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), timeout=1200, wait_for_ready=True)
I0811 19:31:03.422485 140493315874816 xds_k8s_testcase.py:558] [psm-grpc-client-57666bdc87-zgf4d] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
psm-grpc-server-864cd957f7-8hpms: 100
rpcs_by_method:
UnaryCall:
psm-grpc-server-864cd957f7-8hpms: 100

I0811 19:31:03.422740 140493315874816 security_test.py:83] [SUCCESS] TLS security mode confirmed.
I0811 19:31:03.422852 140493315874816 xds_k8s_testcase.py:640] ----- TestMethod __main__.SecurityTest.test_tls teardown -----
I0811 19:31:03.565139 140493315874816 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1925-e8fqh"
I0811 19:31:04.318558 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782263569-602aac0db7b51-810c9bf0-ee27dd41
I0811 19:31:13.538947 140493315874816 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1925-e8fqh"
I0811 19:31:13.629984 140493315874816 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:13.630232 140493315874816 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1925-e8fqh"
I0811 19:31:14.061012 140493315874816 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:14.061248 140493315874816 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1925-e8fqh"
I0811 19:31:14.534372 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782274065-602aac17ba2b8-6cde8150-de04ae27
I0811 19:31:17.024387 140493315874816 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1925-e8fqh"
I0811 19:31:17.342476 140493315874816 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:17.342710 140493315874816 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1925-e8fqh"
I0811 19:31:18.284717 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782277349-602aac1adc269-0a63b41f-2395d01b
I0811 19:31:20.760339 140493315874816 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1925-e8fqh"
I0811 19:31:21.305035 140493315874816 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:21.305258 140493315874816 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1925-e8fqh"
I0811 19:31:21.990720 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782281310-602aac1ea307e-c0843eb5-693b39c0
I0811 19:31:48.201974 140493315874816 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1925-e8fqh"
I0811 19:31:48.640250 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:48.640477 140493315874816 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1925-e8fqh"
I0811 19:31:49.106221 140493315874816 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1925-e8fqh" not deleted since it does not exist
I0811 19:31:49.106449 140493315874816 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1925-e8fqh"
I0811 19:31:49.734439 140493315874816 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782309174-602aac3935bb7-8b856929-ec642930
I0811 19:31:54.258296 140493315874816 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1925-e8fqh
W0811 19:31:54.260334 140493315874816 http.py:163] Sleeping 1.23 seconds before retry 1 of 5 for request: DELETE https://networkservices.googleapis.com/v1beta1/projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1925-e8fqh?alt=json, after [Errno 32] Broken pipe
I0811 19:31:55.811702 140493315874816 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691782315568-602aac3f4ed1e-12a2543f-80776cab
I0811 19:32:00.106394 140493315874816 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1925-e8fqh
W0811 19:32:00.108346 140493315874816 http.py:163] Sleeping 0.56 seconds before retry 1 of 5 for request: DELETE https://networksecurity.googleapis.com/v1beta1/projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1925-e8fqh?alt=json, after [Errno 32] Broken pipe
I0811 19:32:01.003982 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782320779-602aac4446fd5-bdfe3985-1ebdd0fb
I0811 19:32:05.286733 140493315874816 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1925-e8fqh
I0811 19:32:05.701392 140493315874816 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782325364-602aac48a687d-9f8eb8f6-bc654e22
I0811 19:32:10.039826 140493315874816 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1925-e8fqh
I0811 19:32:10.113318 140493315874816 api.py:504] projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1925-e8fqh not deleted since it does not exist
I0811 19:32:10.117372 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/kubernetes/client/rest.py:44: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
return self.urllib3_response.getheaders()
I0811 19:32:20.318031 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1925-e8fqh/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:32:21.007369 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:32:21.138529 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1925-e8fqh
I0811 19:32:51.328240 140493315874816 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:33:01.498964 140493315874816 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:33:01.630516 140493315874816 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1925-e8fqh/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:33:02.327815 140493315874816 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:33:02.460859 140493315874816 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1925-e8fqh
I0811 19:34:02.733788 140493315874816 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:34:02.734249 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-2wbtz9j8:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1925-e8fqh%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-2wbtz9j8%22;timeRange=2023-08-11T19:29:28.978143Z%2F2023-08-11T19:32:51.325083Z?project=grpc-testing
I0811 19:34:02.734433 140493315874816 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-rql1gfg3:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1925-e8fqh%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-rql1gfg3%22;timeRange=2023-08-11T19:28:09.777173Z%2F2023-08-11T19:34:02.733620Z?project=grpc-testing
[ OK ] SecurityTest.test_tls
----------------------------------------------------------------------
Ran 5 tests in 2501.342s

OK
+ for test in "${test_suites[@]}"
+ run_test authz_test
+ local test_name=authz_test
+ set -x
+ local out_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test
+ mkdir -pv /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test
mkdir: created directory '/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test'
+ python -m tests.authz_test --flagfile=config/grpc-testing.cfg --kube_context=gke_grpc-testing_us-central1-a_psm-interop-security --server_image=gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --client_image=gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4 --testing_version=dev-master --nocheck_local_certs --force_cleanup --collect_app_logs --log_dir=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test --xml_output_file=/tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test/sponge_log.xml
+ tee /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test/sponge_log.log
Running tests under Python 3.10.12: /tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/bin/python
I0811 19:34:03.771678 140212882345984 xds_k8s_testcase.py:130] ----- Testing AuthzTest -----
I0811 19:34:03.771827 140212882345984 xds_k8s_testcase.py:131] Logs timezone: UTC
I0811 19:34:03.772265 140212882345984 skips.py:124] Detected language and version: TestConfig(client_lang='go', server_lang='go', version='dev-master')
I0811 19:34:03.784893 140212882345984 k8s.py:130] Using kubernetes context "gke_grpc-testing_us-central1-a_psm-interop-security", active host: https://35.202.85.90
I0811 19:34:03.795752 140212882345984 k8s.py:130] Using kubernetes context "None", active host: https://35.202.85.90
[ RUN ] AuthzTest.test_mtls_allow
I0811 19:34:03.800145 140212882345984 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1934-qygkj
I0811 19:34:04.420266 140212882345984 logs.py:31] Log root dir: /tmpfs/src/artifacts/grpc/go/master/branch/psm-security/authz_test
I0811 19:34:04.890591 140212882345984 xds_k8s_testcase.py:625] Found unused xds port: 4914
I0811 19:34:04.890986 140212882345984 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1934-qygkj"
I0811 19:34:04.894296 140212882345984 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1934-qygkj
type: GRPC
...

I0811 19:34:05.499982 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782444970-602aacbab73c6-0296dcc5-f0b1eea1
I0811 19:34:07.852828 140212882345984 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1934-qygkj"
I0811 19:34:07.860186 140212882345984 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1934-qygkj
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1934-qygkj
protocol: GRPC
...

I0811 19:34:08.623347 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782447862-602aacbd792ac-6e14dd21-eb3914bf
I0811 19:35:05.308948 140212882345984 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1934-qygkj": psm-grpc-server:4914 -> psm-interop-backend-service-20230811-1934-qygkj
I0811 19:35:05.320178 140212882345984 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1934-qygkj
hostRules:
- hosts:
- psm-grpc-server:4914
pathMatcher: psm-interop-path-matcher-20230811-1934-qygkj
name: psm-interop-url-map-20230811-1934-qygkj
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1934-qygkj
name: psm-interop-path-matcher-20230811-1934-qygkj
...

I0811 19:35:06.093628 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782505322-602aacf445900-6a4a6df6-f421fa71
I0811 19:35:08.359504 140212882345984 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1934-qygkj proxy "GRPC" to URL map psm-interop-url-map-20230811-1934-qygkj
I0811 19:35:08.361621 140212882345984 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1934-qygkj
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1934-qygkj
validate_for_proxyless: true
...

I0811 19:35:09.147462 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782508364-602aacf72c2de-2cf0e6d8-c801ade1
I0811 19:35:11.432157 140212882345984 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1934-qygkj" in network "default-vpc": 0.0.0.0:4914 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1934-qygkj
I0811 19:35:11.434923 140212882345984 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1934-qygkj
network: global/networks/default-vpc
portRange: 4914
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1934-qygkj
...

I0811 19:35:12.385259 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782511437-602aacfa1a556-8d581946-a03062de
I0811 19:35:25.739049 140212882345984 traffic_director.py:943] Creating Authz Policy psm-interop-authz-policy-20230811-1934-qygkj
I0811 19:35:25.748073 140212882345984 api.py:468] Creating networksecurity resource:
---
action: ALLOW
rules:
- destinations:
hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:4914
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:4914
- psm-grpc-server:4914
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
hosts:
- not-the-host:4914
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1934-qygkj/sa/psm-grpc-client
- destinations:
hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
...

I0811 19:35:26.419042 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782525921-602aad07ea8c7-6e7591af-fc5bf2b3
I0811 19:35:30.743058 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1934-qygkj:
---
action: ALLOW
createTime: '2023-08-11T19:35:26.022991071Z'
name: projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1934-qygkj
rules:
- destinations:
- hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
- hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:4914
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:4914
- psm-grpc-server:4914
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
- hosts:
- not-the-host:4914
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
- hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
- hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1934-qygkj/sa/psm-grpc-client
- destinations:
- hosts:
- '*:4914'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
updateTime: '2023-08-11T19:35:29.676091628Z'
...

I0811 19:35:30.743417 140212882345984 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1934-qygkj
I0811 19:35:30.745519 140212882345984 api.py:468] Creating networksecurity resource:
---
clientCertificate: &id001
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
serverValidationCa:
- *id001
...

I0811 19:35:31.088754 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782530820-602aad0c96886-f43f703c-2f1bd998
I0811 19:35:35.503456 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1934-qygkj:
---
clientCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
createTime: '2023-08-11T19:35:30.909510289Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1934-qygkj
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:35:34.515272228Z'
...

I0811 19:35:35.503759 140212882345984 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1934-qygkj: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1934-qygkj, server spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1934-qygkj/sa/psm-grpc-server
I0811 19:35:35.507821 140212882345984 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1934-qygkj
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1934-qygkj/sa/psm-grpc-server
...

I0811 19:35:36.470502 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782535510-602aad110fa0b-6afe15f8-298e37a0
I0811 19:36:09.420182 140212882345984 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1934-qygkj
I0811 19:36:09.422548 140212882345984 api.py:468] Creating networksecurity resource:
---
mtlsPolicy:
clientValidationCa:
- &id001
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
serverCertificate: *id001
...

I0811 19:36:09.752544 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782569497-602aad31791fb-f87330f3-49291704
I0811 19:36:14.164507 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1934-qygkj:
---
createTime: '2023-08-11T19:36:09.574950162Z'
mtlsPolicy:
clientValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1934-qygkj
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:36:13.237564723Z'
...

I0811 19:36:14.164762 140212882345984 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1934-qygkj
I0811 19:36:14.168005 140212882345984 api.py:468] Creating networkservices resource:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1934-qygkj
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1934-qygkj-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1934-qygkj
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:36:14.569001 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691782574301-602aad360e313-6a5e7775-79bb0931
I0811 19:36:18.802015 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1934-qygkj:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1934-qygkj
createTime: '2023-08-11T19:36:14.367008500Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1934-qygkj-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1934-qygkj
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1934-qygkj
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:36:17.941127999Z'
...

I0811 19:36:18.802291 140212882345984 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1934-qygkj: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:36:18.802538 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1934-qygkj%22;timeRange=2023-08-11T19:36:18.802387Z%2F2023-08-11T20:06:18.802387Z?project=grpc-testing
I0811 19:36:18.807374 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1934-qygkj
labels:
name: psm-interop-server-20230811-1934-qygkj
owner: xds-k8s-interop-test
...

I0811 19:36:18.965554 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1934-qygkj created
I0811 19:36:18.968561 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1934-qygkj
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1934-qygkj-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:36:19.042082 140212882345984 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:36:19.042326 140212882345984 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:36:19.141849 140212882345984 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1934-qygkj-psm-grpc-server in zones=['us-central1-a']
I0811 19:36:19.142051 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1934-qygkj/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:36:19.855479 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1934-qygkj
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:36:19.918321 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:36:19.926363 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1934-qygkj
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jytl0zsu
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jytl0zsu
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-jytl0zsu
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1934-qygkj-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:36:19.998502 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:36:19.998736 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:36:20.106206 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-78fcb9c6c-znhhx']
I0811 19:36:20.106485 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-78fcb9c6c-znhhx to start
I0811 19:36:24.399606 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-server-78fcb9c6c-znhhx ready, IP: 10.12.17.14
I0811 19:36:24.399858 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-78fcb9c6c-znhhx to authz_test/test_app_logs/psm-interop-server-20230811-1934-qygkj_psm-grpc-server-78fcb9c6c-znhhx.log
I0811 19:36:24.400303 140212790879808 k8s_log_collector.py:70] Starting log collection thread 140212790879808 for psm-grpc-server-78fcb9c6c-znhhx
I0811 19:36:24.400470 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:36:28.747098 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:36:28.786585 140212882345984 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:36:28.917641 140212882345984 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1934-qygkj-psm-grpc-server" in zone us-central1-a
I0811 19:36:28.917860 140212882345984 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1934-qygkj: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1934-qygkj-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1934-qygkj-psm-grpc-server', zone='us-central1-a')}
I0811 19:36:28.921770 140212882345984 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1934-qygkj-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:36:29.979600 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782588923-602aad43fff8b-1836cd33-e504171e
I0811 19:37:22.239205 140212882345984 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1934-qygkj to report all backends healthy: ['psm-interop-server-20230811-1934-qygkj-psm-grpc-server']
I0811 19:37:35.301404 140212882345984 compute.py:487] Backend psm-interop-server-20230811-1934-qygkj-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:37:39.306125 140212882345984 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1934-qygkj: server_target=xds:///psm-grpc-server:4914 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:37:39.306563 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1934-qygkj%22;timeRange=2023-08-11T19:37:39.306391Z%2F2023-08-11T20:07:39.306391Z?project=grpc-testing
I0811 19:37:39.308585 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1934-qygkj
labels:
name: psm-interop-client-20230811-1934-qygkj
owner: xds-k8s-interop-test
...

I0811 19:37:39.350723 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1934-qygkj created
I0811 19:37:39.350979 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1934-qygkj/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:37:40.026351 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1934-qygkj
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:37:40.092013 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:37:40.099487 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1934-qygkj
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ipdytv0i
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ipdytv0i
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-ipdytv0i
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:4914"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:37:40.183939 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:37:40.184249 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:37:40.298856 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-bbb9749bd-krs6g']
I0811 19:37:40.299139 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-bbb9749bd-krs6g to start
I0811 19:37:48.760004 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-client-bbb9749bd-krs6g ready, IP: 10.12.9.216
I0811 19:37:48.760302 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-bbb9749bd-krs6g to authz_test/test_app_logs/psm-interop-client-20230811-1934-qygkj_psm-grpc-client-bbb9749bd-krs6g.log
I0811 19:37:48.760727 140212782487104 k8s_log_collector.py:70] Starting log collection thread 140212782487104 for psm-grpc-client-bbb9749bd-krs6g
I0811 19:37:48.760874 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:37:49.975726 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:37:49.976529 140212882345984 client_app.py:200] [psm-grpc-client-bbb9749bd-krs6g] Waiting to report a READY channel to xds:///psm-grpc-server:4914
I0811 19:37:50.077546 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:38:00.122636 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:38:10.168208 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:38:20.213015 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:38:30.257917 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:38:46.307300 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:39:11.367624 140212882345984 client_app.py:236] [psm-grpc-client-bbb9749bd-krs6g] Server channel:
I0811 19:39:11.400713 140212882345984 client_app.py:249] [psm-grpc-client-bbb9749bd-krs6g] Found subchannel in state READY:
I0811 19:39:11.401027 140212882345984 client_app.py:211] [psm-grpc-client-bbb9749bd-krs6g] Channel to xds:///psm-grpc-server:4914 transitioned to state READY:
I0811 19:39:16.406375 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_mtls_allow.01_host_wildcard ---
I0811 19:39:16.406737 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:16.439761 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'host-wildcard'}]}), timeout=5, wait_for_ready=True)
I0811 19:39:16.472414 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:17.007211 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:17.042192 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-bbb9749bd-krs6g] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 14
result:
(0, OK): 13

I0811 19:39:17.043060 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_mtls_allow.01_host_wildcard ---
I0811 19:39:17.043210 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_mtls_allow.02_no_match ---
I0811 19:39:17.043369 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL']}), timeout=5, wait_for_ready=True)
I0811 19:39:17.075984 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:17.610653 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:17.645626 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-bbb9749bd-krs6g] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 14
result:
(7, PERMISSION_DENIED): 14

I0811 19:39:17.646392 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_mtls_allow.02_no_match ---
I0811 19:39:17.646523 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_mtls_allow.04_match_principal ---
I0811 19:39:17.646709 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'match-principal'}]}), timeout=5, wait_for_ready=True)
I0811 19:39:17.679565 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:18.214336 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:18.249316 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-bbb9749bd-krs6g] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 14
result:
(0, OK): 14

I0811 19:39:18.250008 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_mtls_allow.04_match_principal ---
I0811 19:39:18.250144 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_mtls_allow.05_never_match_principal ---
I0811 19:39:18.250339 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'never-match-principal'}]}), timeout=5, wait_for_ready=True)
I0811 19:39:18.282845 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:18.817534 140212882345984 grpc.py:74] [psm-grpc-client-bbb9749bd-krs6g:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:39:18.852710 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-bbb9749bd-krs6g] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 14
result:
(7, PERMISSION_DENIED): 14

I0811 19:39:18.853430 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_mtls_allow.05_never_match_principal ---
I0811 19:39:18.853580 140212882345984 xds_k8s_testcase.py:640] ----- TestMethod __main__.AuthzTest.test_mtls_allow teardown -----
I0811 19:39:19.005300 140212882345984 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1934-qygkj"
I0811 19:39:19.674782 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782759009-602aade634bff-3a508774-fe9b982d
I0811 19:39:43.574570 140212882345984 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1934-qygkj"
I0811 19:39:43.763349 140212882345984 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:39:43.763558 140212882345984 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1934-qygkj"
I0811 19:39:44.297926 140212882345984 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:39:44.298177 140212882345984 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1934-qygkj"
I0811 19:39:44.895655 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782784301-602aadfe53bac-368ff97c-717a45cc
I0811 19:39:47.331897 140212882345984 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1934-qygkj"
I0811 19:39:47.858673 140212882345984 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:39:47.858878 140212882345984 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1934-qygkj"
I0811 19:39:48.420619 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782787864-602aae01b96e6-2ee7b8d7-0b95df9f
I0811 19:39:50.780935 140212882345984 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1934-qygkj"
I0811 19:39:51.246707 140212882345984 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:39:51.246937 140212882345984 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1934-qygkj"
I0811 19:39:52.047535 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782791252-602aae04f49e7-d0d582b6-c2c17955
I0811 19:40:21.007928 140212882345984 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1934-qygkj"
I0811 19:40:21.501384 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:40:21.501596 140212882345984 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1934-qygkj"
I0811 19:40:21.870867 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1934-qygkj" not deleted since it does not exist
I0811 19:40:21.871115 140212882345984 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1934-qygkj"
I0811 19:40:22.407709 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782821952-602aae223bc5c-238a670c-a1fdfd43
I0811 19:40:26.974189 140212882345984 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1934-qygkj
I0811 19:40:27.372522 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691782827099-602aae2724717-84e9776a-bf3752c3
I0811 19:40:31.613534 140212882345984 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1934-qygkj
I0811 19:40:32.094204 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782831743-602aae2b92078-bbf959ef-daaa17b1
I0811 19:40:36.328468 140212882345984 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1934-qygkj
I0811 19:40:36.840521 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782836443-602aae300d971-0f1e116b-cf5fe690
I0811 19:40:41.141376 140212882345984 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1934-qygkj
I0811 19:40:41.506568 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691782841217-602aae349b000-06a197f5-983454c0
I0811 19:40:45.766139 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
/tmp/tmp.T3wY7IAC08/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.10/site-packages/kubernetes/client/rest.py:44: DeprecationWarning: HTTPResponse.getheaders() is deprecated and will be removed in urllib3 v2.1.0. Instead access HTTPResponse.headers directly.
return self.urllib3_response.getheaders()
I0811 19:40:55.946485 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1934-qygkj/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:40:56.502529 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:40:56.627666 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1934-qygkj
I0811 19:41:26.865070 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:41:37.043241 140212882345984 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:41:37.169955 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1934-qygkj/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:41:37.912899 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:41:38.032756 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1934-qygkj
I0811 19:42:08.217041 140212882345984 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:42:08.217424 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-ipdytv0i:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1934-qygkj%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-ipdytv0i%22;timeRange=2023-08-11T19:37:39.306382Z%2F2023-08-11T19:41:26.861501Z?project=grpc-testing
I0811 19:42:08.217600 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-jytl0zsu:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1934-qygkj%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-jytl0zsu%22;timeRange=2023-08-11T19:36:18.802380Z%2F2023-08-11T19:42:08.216877Z?project=grpc-testing
[ OK ] AuthzTest.test_mtls_allow
[ RUN ] AuthzTest.test_plaintext_allow
I0811 19:42:08.217989 140212882345984 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1942-kwyfw
I0811 19:42:08.330320 140212882345984 xds_k8s_testcase.py:625] Found unused xds port: 36633
I0811 19:42:08.330579 140212882345984 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1942-kwyfw"
I0811 19:42:08.333759 140212882345984 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1942-kwyfw
type: GRPC
...

I0811 19:42:09.108876 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782928398-602aae87bf89f-b61c4a66-ad10aa86
I0811 19:42:11.590652 140212882345984 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1942-kwyfw"
I0811 19:42:11.594804 140212882345984 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1942-kwyfw
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1942-kwyfw
protocol: GRPC
...

I0811 19:42:12.650476 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782931597-602aae8acc937-dc7ccb29-421fd711
I0811 19:42:54.665573 140212882345984 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1942-kwyfw": psm-grpc-server:36633 -> psm-interop-backend-service-20230811-1942-kwyfw
I0811 19:42:54.670683 140212882345984 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1942-kwyfw
hostRules:
- hosts:
- psm-grpc-server:36633
pathMatcher: psm-interop-path-matcher-20230811-1942-kwyfw
name: psm-interop-url-map-20230811-1942-kwyfw
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1942-kwyfw
name: psm-interop-path-matcher-20230811-1942-kwyfw
...

I0811 19:42:55.273886 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782974672-602aaeb3e0fde-111fa787-6ab3ec4e
I0811 19:42:57.629132 140212882345984 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1942-kwyfw proxy "GRPC" to URL map psm-interop-url-map-20230811-1942-kwyfw
I0811 19:42:57.631124 140212882345984 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1942-kwyfw
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1942-kwyfw
validate_for_proxyless: true
...

I0811 19:42:58.041260 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782977633-602aaeb6b3c3b-10f1483e-0f4cfa10
I0811 19:43:00.375108 140212882345984 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1942-kwyfw" in network "default-vpc": 0.0.0.0:36633 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1942-kwyfw
I0811 19:43:00.378088 140212882345984 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1942-kwyfw
network: global/networks/default-vpc
portRange: 36633
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1942-kwyfw
...

I0811 19:43:01.322953 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691782980380-602aaeb9527ad-fe08cd84-dd19c133
I0811 19:43:45.417272 140212882345984 traffic_director.py:943] Creating Authz Policy psm-interop-authz-policy-20230811-1942-kwyfw
I0811 19:43:45.425547 140212882345984 api.py:468] Creating networksecurity resource:
---
action: ALLOW
rules:
- destinations:
hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:36633
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:36633
- psm-grpc-server:36633
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
hosts:
- not-the-host:36633
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1942-kwyfw/sa/psm-grpc-client
- destinations:
hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
...

I0811 19:43:45.719393 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783025497-602aaee459751-b0ff7c1f-95be07dd
I0811 19:44:08.880571 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1942-kwyfw:
---
action: ALLOW
createTime: '2023-08-11T19:43:45.593999195Z'
name: projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1942-kwyfw
rules:
- destinations:
- hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
- hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:36633
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:36633
- psm-grpc-server:36633
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
- hosts:
- not-the-host:36633
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
- hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
- hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1942-kwyfw/sa/psm-grpc-client
- destinations:
- hosts:
- '*:36633'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
updateTime: '2023-08-11T19:44:07.200493547Z'
...

I0811 19:44:08.880930 140212882345984 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1942-kwyfw
W0811 19:44:08.881035 140212882345984 traffic_director.py:1019] Client TLS Policy psm-interop-client-tls-policy-20230811-1942-kwyfw neither TLS, nor mTLS policy. Skipping creation
W0811 19:44:08.881099 140212882345984 traffic_director.py:1056] Client TLS policy not created, skipping attaching to Backend Service psm-interop-backend-service-20230811-1942-kwyfw
I0811 19:44:08.881159 140212882345984 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1942-kwyfw
W0811 19:44:08.881209 140212882345984 traffic_director.py:908] Server TLS Policy psm-interop-server-tls-policy-20230811-1942-kwyfw neither TLS, nor mTLS policy. Skipping creation
I0811 19:44:08.881259 140212882345984 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1942-kwyfw
W0811 19:44:08.881310 140212882345984 traffic_director.py:990] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1942-kwyfw with no Server TLS policy attached
I0811 19:44:08.883793 140212882345984 api.py:468] Creating networkservices resource:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1942-kwyfw
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1942-kwyfw-psm-grpc-server
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:44:09.339649 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691783048952-602aaefab7a01-917756b4-83869584
I0811 19:44:13.624660 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1942-kwyfw:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1942-kwyfw
createTime: '2023-08-11T19:44:09.017477528Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1942-kwyfw-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1942-kwyfw
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:44:12.670931639Z'
...

I0811 19:44:13.624939 140212882345984 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1942-kwyfw: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:44:13.625212 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1942-kwyfw%22;timeRange=2023-08-11T19:44:13.625053Z%2F2023-08-11T20:14:13.625053Z?project=grpc-testing
I0811 19:44:13.627283 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1942-kwyfw
labels:
name: psm-interop-server-20230811-1942-kwyfw
owner: xds-k8s-interop-test
...

I0811 19:44:13.780339 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1942-kwyfw created
I0811 19:44:13.783192 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1942-kwyfw
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1942-kwyfw-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:44:13.855461 140212882345984 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:44:13.855671 140212882345984 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:44:13.951648 140212882345984 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1942-kwyfw-psm-grpc-server in zones=['us-central1-a']
I0811 19:44:13.951872 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1942-kwyfw/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:44:14.701631 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1942-kwyfw
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:44:14.764396 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:44:14.772157 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1942-kwyfw
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-vf5illhp
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-vf5illhp
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-vf5illhp
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1942-kwyfw-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:44:14.847022 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:44:14.847257 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:44:14.962424 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-67689bbfb9-xw77d']
I0811 19:44:14.962731 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-67689bbfb9-xw77d to start
I0811 19:44:19.309355 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-server-67689bbfb9-xw77d ready, IP: 10.12.14.31
I0811 19:44:19.309641 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-67689bbfb9-xw77d to authz_test/test_app_logs/psm-interop-server-20230811-1942-kwyfw_psm-grpc-server-67689bbfb9-xw77d.log
I0811 19:44:19.309983 140212790879808 k8s_log_collector.py:70] Starting log collection thread 140212790879808 for psm-grpc-server-67689bbfb9-xw77d
I0811 19:44:19.310171 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:44:23.587282 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:44:23.624317 140212882345984 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:44:23.911290 140212882345984 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1942-kwyfw-psm-grpc-server" in zone us-central1-a
I0811 19:44:23.911512 140212882345984 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1942-kwyfw: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1942-kwyfw-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1942-kwyfw-psm-grpc-server', zone='us-central1-a')}
I0811 19:44:23.915585 140212882345984 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1942-kwyfw-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:44:24.944964 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783063918-602aaf08fd608-a0360bb0-9244f450
I0811 19:45:06.419670 140212882345984 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1942-kwyfw to report all backends healthy: ['psm-interop-server-20230811-1942-kwyfw-psm-grpc-server']
I0811 19:45:27.832926 140212882345984 compute.py:487] Backend psm-interop-server-20230811-1942-kwyfw-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:45:31.837656 140212882345984 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1942-kwyfw: server_target=xds:///psm-grpc-server:36633 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:45:31.838034 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1942-kwyfw%22;timeRange=2023-08-11T19:45:31.837882Z%2F2023-08-11T20:15:31.837882Z?project=grpc-testing
I0811 19:45:31.839961 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1942-kwyfw
labels:
name: psm-interop-client-20230811-1942-kwyfw
owner: xds-k8s-interop-test
...

I0811 19:45:31.882911 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1942-kwyfw created
I0811 19:45:31.883181 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1942-kwyfw/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:45:32.636383 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1942-kwyfw
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:45:32.700538 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:45:32.707987 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1942-kwyfw
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c9xn4q48
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c9xn4q48
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-c9xn4q48
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:36633"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:45:32.788469 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:45:32.788723 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:45:32.898376 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-6c878fc6dc-lnq8t']
I0811 19:45:32.898674 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-6c878fc6dc-lnq8t to start
I0811 19:45:36.148259 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-client-6c878fc6dc-lnq8t ready, IP: 10.12.9.218
I0811 19:45:36.148526 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-6c878fc6dc-lnq8t to authz_test/test_app_logs/psm-interop-client-20230811-1942-kwyfw_psm-grpc-client-6c878fc6dc-lnq8t.log
I0811 19:45:36.148889 140212782487104 k8s_log_collector.py:70] Starting log collection thread 140212782487104 for psm-grpc-client-6c878fc6dc-lnq8t
I0811 19:45:36.149066 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:45:38.336463 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:45:38.337081 140212882345984 client_app.py:200] [psm-grpc-client-6c878fc6dc-lnq8t] Waiting to report a READY channel to xds:///psm-grpc-server:36633
I0811 19:45:38.436522 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:45:48.482084 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:45:58.527295 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:46:08.573157 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:46:18.618623 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:46:34.668195 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:46:59.728515 140212882345984 client_app.py:236] [psm-grpc-client-6c878fc6dc-lnq8t] Server channel:
I0811 19:46:59.761445 140212882345984 client_app.py:249] [psm-grpc-client-6c878fc6dc-lnq8t] Found subchannel in state READY:
I0811 19:46:59.761703 140212882345984 client_app.py:211] [psm-grpc-client-6c878fc6dc-lnq8t] Channel to xds:///psm-grpc-server:36633 transitioned to state READY:
I0811 19:47:04.767186 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.01_host_wildcard ---
I0811 19:47:04.767541 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:04.800534 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'host-wildcard'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:04.833101 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:05.367869 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:05.402777 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(0, OK): 13

I0811 19:47:05.403516 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.01_host_wildcard ---
I0811 19:47:05.403654 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.02_no_match ---
I0811 19:47:05.403851 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'no-such-rule'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:05.436358 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:05.971204 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:06.006189 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:47:06.006993 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL']}), timeout=5, wait_for_ready=True)
I0811 19:47:06.039900 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:06.575019 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:06.610687 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:47:06.611540 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.02_no_match ---
I0811 19:47:06.611695 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.03_header_regex ---
I0811 19:47:06.611924 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'header-regex-a'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:06.644456 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:07.179301 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:07.214399 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(0, OK): 13

I0811 19:47:07.215217 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'header-regex-aa'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:07.247817 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:07.782813 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:07.817957 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(0, OK): 13

I0811 19:47:07.818814 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'header-regex-'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:07.851636 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:08.386630 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:08.421931 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:47:08.422934 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'header-regex-ab'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:08.455501 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:08.990491 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:09.025629 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 14
result:
(7, PERMISSION_DENIED): 14

I0811 19:47:09.026683 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'aheader-regex-a'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:09.059340 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:09.594378 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:09.629792 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 14
result:
(7, PERMISSION_DENIED): 14

I0811 19:47:09.630570 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.03_header_regex ---
I0811 19:47:09.630712 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.04_host_match ---
I0811 19:47:09.630908 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'host-match1'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:09.663389 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:10.198379 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:10.233661 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 14
result:
(0, OK): 14

I0811 19:47:10.234489 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'host-match2'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:10.267134 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:10.802302 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:10.837366 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 14
result:
(0, OK): 14

I0811 19:47:10.838076 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.04_host_match ---
I0811 19:47:10.838208 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.05_never_match_host ---
I0811 19:47:10.838398 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'never-match-host'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:10.870986 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:11.405968 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:11.440940 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:47:11.441712 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.05_never_match_host ---
I0811 19:47:11.441853 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_allow.06_never_match_port ---
I0811 19:47:11.442054 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL'], 'metadata': [{'type': 'UNARY_CALL', 'key': 'test', 'value': 'never-match-port'}]}), timeout=5, wait_for_ready=True)
I0811 19:47:11.474532 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:12.009497 140212882345984 grpc.py:74] [psm-grpc-client-6c878fc6dc-lnq8t:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:47:12.044633 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6c878fc6dc-lnq8t] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:47:12.045361 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_allow.06_never_match_port ---
I0811 19:47:12.045520 140212882345984 xds_k8s_testcase.py:640] ----- TestMethod __main__.AuthzTest.test_plaintext_allow teardown -----
I0811 19:47:12.262283 140212882345984 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1942-kwyfw"
I0811 19:47:13.031472 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783232266-602aafa989fa7-ebe12042-21d6b52d
I0811 19:47:32.736235 140212882345984 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1942-kwyfw"
I0811 19:47:32.971474 140212882345984 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:47:32.971713 140212882345984 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1942-kwyfw"
I0811 19:47:33.404408 140212882345984 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:47:33.404633 140212882345984 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1942-kwyfw"
I0811 19:47:34.073812 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783253407-602aafbdb38a3-7ffec75b-1d5800c9
I0811 19:47:36.512696 140212882345984 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1942-kwyfw"
I0811 19:47:36.823929 140212882345984 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:47:36.824275 140212882345984 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1942-kwyfw"
I0811 19:47:37.566175 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783256830-602aafc0f7132-8b350ac4-ad0ba47c
I0811 19:47:42.167098 140212882345984 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1942-kwyfw"
I0811 19:47:42.707301 140212882345984 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:47:42.707554 140212882345984 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1942-kwyfw"
I0811 19:47:43.392691 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783262713-602aafc69364f-95716a85-88f2995d
I0811 19:48:16.312125 140212882345984 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1942-kwyfw"
I0811 19:48:16.725252 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:48:16.725471 140212882345984 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1942-kwyfw"
I0811 19:48:17.139758 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1942-kwyfw" not deleted since it does not exist
I0811 19:48:17.139991 140212882345984 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1942-kwyfw"
I0811 19:48:17.681008 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783297225-602aafe77d193-1dd83be9-066a69e2
I0811 19:48:22.026699 140212882345984 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1942-kwyfw
I0811 19:48:22.498514 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691783302147-602aafec2ece9-0c0cc70d-7c2625eb
I0811 19:48:26.699382 140212882345984 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1942-kwyfw
I0811 19:48:26.837876 140212882345984 api.py:504] projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1942-kwyfw not deleted since it does not exist
I0811 19:48:26.838112 140212882345984 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1942-kwyfw
I0811 19:48:26.941161 140212882345984 api.py:504] projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1942-kwyfw not deleted since it does not exist
I0811 19:48:26.941403 140212882345984 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1942-kwyfw
I0811 19:48:27.202432 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783307008-602aaff0d188c-a9766f19-346a12a9
I0811 19:48:31.422052 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 19:48:41.599251 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1942-kwyfw/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:48:42.302868 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:48:42.416226 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1942-kwyfw
I0811 19:49:12.645303 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:49:22.817740 140212882345984 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:49:22.933526 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1942-kwyfw/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:49:23.639152 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:49:23.763862 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1942-kwyfw
I0811 19:50:24.022167 140212882345984 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:50:24.022587 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-c9xn4q48:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1942-kwyfw%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-c9xn4q48%22;timeRange=2023-08-11T19:45:31.837874Z%2F2023-08-11T19:49:12.642355Z?project=grpc-testing
I0811 19:50:24.022754 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-vf5illhp:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1942-kwyfw%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-vf5illhp%22;timeRange=2023-08-11T19:44:13.625046Z%2F2023-08-11T19:50:24.021951Z?project=grpc-testing
[ OK ] AuthzTest.test_plaintext_allow
[ RUN ] AuthzTest.test_plaintext_deny
I0811 19:50:24.023175 140212882345984 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1950-iwqqo
I0811 19:50:24.228363 140212882345984 xds_k8s_testcase.py:625] Found unused xds port: 61901
I0811 19:50:24.228661 140212882345984 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1950-iwqqo"
I0811 19:50:24.231110 140212882345984 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1950-iwqqo
type: GRPC
...

I0811 19:50:24.901458 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783424303-602ab060ae079-0dffd80d-6a1db572
I0811 19:50:27.401649 140212882345984 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1950-iwqqo"
I0811 19:50:27.405697 140212882345984 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1950-iwqqo
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1950-iwqqo
protocol: GRPC
...

I0811 19:50:28.132860 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783427407-602ab063a403e-0102d7c1-c8a31010
I0811 19:50:50.360000 140212882345984 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1950-iwqqo": psm-grpc-server:61901 -> psm-interop-backend-service-20230811-1950-iwqqo
I0811 19:50:50.365173 140212882345984 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1950-iwqqo
hostRules:
- hosts:
- psm-grpc-server:61901
pathMatcher: psm-interop-path-matcher-20230811-1950-iwqqo
name: psm-interop-url-map-20230811-1950-iwqqo
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1950-iwqqo
name: psm-interop-path-matcher-20230811-1950-iwqqo
...

I0811 19:50:51.093971 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783450367-602ab07989671-700fdadc-0318e103
I0811 19:50:53.388265 140212882345984 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1950-iwqqo proxy "GRPC" to URL map psm-interop-url-map-20230811-1950-iwqqo
I0811 19:50:53.390193 140212882345984 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1950-iwqqo
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1950-iwqqo
validate_for_proxyless: true
...

I0811 19:50:54.114232 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783453392-602ab07c6be00-3acfc8e7-58d5c17c
I0811 19:50:56.448344 140212882345984 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1950-iwqqo" in network "default-vpc": 0.0.0.0:61901 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1950-iwqqo
I0811 19:50:56.451031 140212882345984 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1950-iwqqo
network: global/networks/default-vpc
portRange: 61901
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1950-iwqqo
...

I0811 19:50:57.193305 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783456454-602ab07f57635-82cc96d6-29e0ff33
I0811 19:51:10.298644 140212882345984 traffic_director.py:943] Creating Authz Policy psm-interop-authz-policy-20230811-1950-iwqqo
I0811 19:51:10.306669 140212882345984 api.py:468] Creating networksecurity resource:
---
action: DENY
rules:
- destinations:
hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:61901
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:61901
- psm-grpc-server:61901
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
hosts:
- not-the-host:61901
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1950-iwqqo/sa/psm-grpc-client
- destinations:
hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
...

I0811 19:51:10.585928 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783470372-602ab08c9d5df-17cb4a29-743100a1
I0811 19:51:14.915425 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1950-iwqqo:
---
action: DENY
createTime: '2023-08-11T19:51:10.458778045Z'
name: projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1950-iwqqo
rules:
- destinations:
- hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
- hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:61901
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:61901
- psm-grpc-server:61901
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
- hosts:
- not-the-host:61901
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
- hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
- hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1950-iwqqo/sa/psm-grpc-client
- destinations:
- hosts:
- '*:61901'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
updateTime: '2023-08-11T19:51:13.849133772Z'
...

I0811 19:51:14.915794 140212882345984 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1950-iwqqo
W0811 19:51:14.915902 140212882345984 traffic_director.py:1019] Client TLS Policy psm-interop-client-tls-policy-20230811-1950-iwqqo neither TLS, nor mTLS policy. Skipping creation
W0811 19:51:14.915965 140212882345984 traffic_director.py:1056] Client TLS policy not created, skipping attaching to Backend Service psm-interop-backend-service-20230811-1950-iwqqo
I0811 19:51:14.916023 140212882345984 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1950-iwqqo
W0811 19:51:14.916105 140212882345984 traffic_director.py:908] Server TLS Policy psm-interop-server-tls-policy-20230811-1950-iwqqo neither TLS, nor mTLS policy. Skipping creation
I0811 19:51:14.916160 140212882345984 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1950-iwqqo
W0811 19:51:14.916213 140212882345984 traffic_director.py:990] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1950-iwqqo with no Server TLS policy attached
I0811 19:51:14.918703 140212882345984 api.py:468] Creating networkservices resource:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1950-iwqqo
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1950-iwqqo-psm-grpc-server
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 19:51:15.390794 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691783474997-602ab091068bc-ded66b39-334f26a1
I0811 19:51:19.793675 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1950-iwqqo:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1950-iwqqo
createTime: '2023-08-11T19:51:15.061496462Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1950-iwqqo-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1950-iwqqo
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T19:51:18.800135315Z'
...

I0811 19:51:19.794003 140212882345984 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1950-iwqqo: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 19:51:19.794310 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1950-iwqqo%22;timeRange=2023-08-11T19:51:19.794143Z%2F2023-08-11T20:21:19.794143Z?project=grpc-testing
I0811 19:51:19.796255 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1950-iwqqo
labels:
name: psm-interop-server-20230811-1950-iwqqo
owner: xds-k8s-interop-test
...

I0811 19:51:19.879281 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1950-iwqqo created
I0811 19:51:19.882689 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1950-iwqqo
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1950-iwqqo-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 19:51:19.951134 140212882345984 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 19:51:19.951374 140212882345984 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 19:51:20.050349 140212882345984 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1950-iwqqo-psm-grpc-server in zones=['us-central1-a']
I0811 19:51:20.050560 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1950-iwqqo/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:51:20.742438 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1950-iwqqo
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:51:20.806669 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 19:51:20.814674 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1950-iwqqo
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-s1gu2l4l
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-s1gu2l4l
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-s1gu2l4l
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1950-iwqqo-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:51:20.891837 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 19:51:20.892122 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 19:51:20.998753 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-6bcbbd87c5-2dwpn']
I0811 19:51:20.999231 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-6bcbbd87c5-2dwpn to start
I0811 19:51:25.294520 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-server-6bcbbd87c5-2dwpn ready, IP: 10.12.13.16
I0811 19:51:25.294817 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-6bcbbd87c5-2dwpn to authz_test/test_app_logs/psm-interop-server-20230811-1950-iwqqo_psm-grpc-server-6bcbbd87c5-2dwpn.log
I0811 19:51:25.295204 140212790879808 k8s_log_collector.py:70] Starting log collection thread 140212790879808 for psm-grpc-server-6bcbbd87c5-2dwpn
I0811 19:51:25.295399 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 19:51:30.618829 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 19:51:30.655210 140212882345984 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 19:51:30.962554 140212882345984 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1950-iwqqo-psm-grpc-server" in zone us-central1-a
I0811 19:51:30.962765 140212882345984 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1950-iwqqo: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1950-iwqqo-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1950-iwqqo-psm-grpc-server', zone='us-central1-a')}
I0811 19:51:30.966769 140212882345984 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1950-iwqqo-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 19:51:31.991776 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783490968-602ab0a041c60-32102052-48c26eec
I0811 19:52:30.811622 140212882345984 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1950-iwqqo to report all backends healthy: ['psm-interop-server-20230811-1950-iwqqo-psm-grpc-server']
I0811 19:52:35.319729 140212882345984 compute.py:487] Backend psm-interop-server-20230811-1950-iwqqo-psm-grpc-server in zone us-central1-a reported healthy
I0811 19:52:39.324410 140212882345984 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1950-iwqqo: server_target=xds:///psm-grpc-server:61901 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 19:52:39.324815 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1950-iwqqo%22;timeRange=2023-08-11T19:52:39.324668Z%2F2023-08-11T20:22:39.324668Z?project=grpc-testing
I0811 19:52:39.327064 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1950-iwqqo
labels:
name: psm-interop-client-20230811-1950-iwqqo
owner: xds-k8s-interop-test
...

I0811 19:52:39.372772 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1950-iwqqo created
I0811 19:52:39.373100 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1950-iwqqo/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:52:41.818017 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1950-iwqqo
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 19:52:41.884385 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 19:52:41.892154 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1950-iwqqo
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-cp7byi4w
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-cp7byi4w
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-cp7byi4w
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:61901"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 19:52:41.965191 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 19:52:41.965434 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 19:52:42.072824 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-7b9dd84d59-6vp6v']
I0811 19:52:42.073090 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-7b9dd84d59-6vp6v to start
I0811 19:52:46.357105 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-client-7b9dd84d59-6vp6v ready, IP: 10.12.9.220
I0811 19:52:46.357383 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-7b9dd84d59-6vp6v to authz_test/test_app_logs/psm-interop-client-20230811-1950-iwqqo_psm-grpc-client-7b9dd84d59-6vp6v.log
I0811 19:52:46.357762 140212782487104 k8s_log_collector.py:70] Starting log collection thread 140212782487104 for psm-grpc-client-7b9dd84d59-6vp6v
I0811 19:52:46.357918 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 19:52:47.574650 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 19:52:47.575264 140212882345984 client_app.py:200] [psm-grpc-client-7b9dd84d59-6vp6v] Waiting to report a READY channel to xds:///psm-grpc-server:61901
I0811 19:52:47.674110 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:52:57.719233 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:53:07.764773 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:53:17.809772 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:53:27.854933 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:53:43.904456 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:54:08.964658 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:54:34.024814 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:54:59.084719 140212882345984 client_app.py:236] [psm-grpc-client-7b9dd84d59-6vp6v] Server channel:
I0811 19:54:59.117391 140212882345984 client_app.py:249] [psm-grpc-client-7b9dd84d59-6vp6v] Found subchannel in state READY:
I0811 19:54:59.117723 140212882345984 client_app.py:211] [psm-grpc-client-7b9dd84d59-6vp6v] Channel to xds:///psm-grpc-server:61901 transitioned to state READY:
I0811 19:55:04.123079 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_deny.01_host_wildcard ---
I0811 19:55:04.123531 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:55:04.156731 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'host-wildcard'}]}), timeout=5, wait_for_ready=True)
I0811 19:55:04.189398 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:55:04.724319 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:55:04.759209 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-7b9dd84d59-6vp6v] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 19:55:04.759938 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_deny.01_host_wildcard ---
I0811 19:55:04.760122 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_plaintext_deny.02_no_match ---
I0811 19:55:04.760295 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL']}), timeout=5, wait_for_ready=True)
I0811 19:55:04.793016 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:55:05.327765 140212882345984 grpc.py:74] [psm-grpc-client-7b9dd84d59-6vp6v:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 19:55:05.362831 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-7b9dd84d59-6vp6v] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(0, OK): 13

I0811 19:55:05.363606 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_plaintext_deny.02_no_match ---
I0811 19:55:05.363785 140212882345984 xds_k8s_testcase.py:640] ----- TestMethod __main__.AuthzTest.test_plaintext_deny teardown -----
I0811 19:55:05.595154 140212882345984 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1950-iwqqo"
I0811 19:55:06.334479 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783705599-602ab16cf1dd1-2697d6dc-f4cbdbf2
I0811 19:55:19.634846 140212882345984 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1950-iwqqo"
I0811 19:55:19.821659 140212882345984 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:19.821886 140212882345984 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1950-iwqqo"
I0811 19:55:20.234766 140212882345984 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:20.234983 140212882345984 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1950-iwqqo"
I0811 19:55:20.663841 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783720237-602ab17ae7b79-0a51b609-7072db9f
I0811 19:55:23.009429 140212882345984 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1950-iwqqo"
I0811 19:55:23.675723 140212882345984 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:23.676014 140212882345984 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1950-iwqqo"
I0811 19:55:24.306671 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783723681-602ab17e3050b-3b402f8d-a990206a
I0811 19:55:28.912495 140212882345984 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1950-iwqqo"
I0811 19:55:29.381318 140212882345984 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:29.381534 140212882345984 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1950-iwqqo"
I0811 19:55:30.197069 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783729386-602ab183a14b1-c67a9813-45c9135d
I0811 19:55:58.666417 140212882345984 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1950-iwqqo"
I0811 19:55:59.024591 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:59.024808 140212882345984 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1950-iwqqo"
I0811 19:55:59.448800 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1950-iwqqo" not deleted since it does not exist
I0811 19:55:59.449023 140212882345984 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1950-iwqqo"
I0811 19:56:00.009377 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783759535-602ab1a061be6-44267948-60cd9331
I0811 19:56:04.466579 140212882345984 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1950-iwqqo
I0811 19:56:05.028639 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691783764584-602ab1a532714-f25c940e-36255b35
I0811 19:56:21.916725 140212882345984 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1950-iwqqo
I0811 19:56:22.037106 140212882345984 api.py:504] projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1950-iwqqo not deleted since it does not exist
I0811 19:56:22.037331 140212882345984 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1950-iwqqo
I0811 19:56:22.109586 140212882345984 api.py:504] projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1950-iwqqo not deleted since it does not exist
I0811 19:56:22.109819 140212882345984 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1950-iwqqo
I0811 19:56:22.316770 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783782184-602ab1b5fb540-53dddbcc-34209cea
I0811 19:56:41.254762 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 19:56:51.520029 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1950-iwqqo/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:56:52.326230 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 19:56:52.446080 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1950-iwqqo
I0811 19:57:22.655442 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 19:57:32.826802 140212882345984 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 19:57:32.947523 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1950-iwqqo/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 19:57:33.690874 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 19:57:33.807213 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1950-iwqqo
I0811 19:58:34.055907 140212882345984 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 19:58:34.056361 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-cp7byi4w:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1950-iwqqo%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-cp7byi4w%22;timeRange=2023-08-11T19:52:39.324659Z%2F2023-08-11T19:57:22.652087Z?project=grpc-testing
I0811 19:58:34.056556 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-s1gu2l4l:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1950-iwqqo%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-s1gu2l4l%22;timeRange=2023-08-11T19:51:19.794135Z%2F2023-08-11T19:58:34.055733Z?project=grpc-testing
[ OK ] AuthzTest.test_plaintext_deny
[ RUN ] AuthzTest.test_tls_allow
I0811 19:58:34.056951 140212882345984 xds_k8s_testcase.py:590] Test run resource prefix: psm-interop, suffix: 20230811-1958-dzff9
I0811 19:58:34.153830 140212882345984 xds_k8s_testcase.py:625] Found unused xds port: 22988
I0811 19:58:34.154133 140212882345984 traffic_director.py:187] Creating GRPC Health Check "psm-interop-health-check-20230811-1958-dzff9"
I0811 19:58:34.156659 140212882345984 compute.py:530] Creating compute resource:
---
grpcHealthCheck:
port: 8081
portSpecification: USE_FIXED_PORT
name: psm-interop-health-check-20230811-1958-dzff9
type: GRPC
...

I0811 19:58:34.831789 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783914216-602ab233e5c89-aa26664d-cc5a486c
I0811 19:58:37.235579 140212882345984 traffic_director.py:214] Creating GRPC Backend Service "psm-interop-backend-service-20230811-1958-dzff9"
I0811 19:58:37.239755 140212882345984 compute.py:530] Creating compute resource:
---
healthChecks:
- https://www.googleapis.com/compute/v1/projects/grpc-testing/global/healthChecks/psm-interop-health-check-20230811-1958-dzff9
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-backend-service-20230811-1958-dzff9
protocol: GRPC
...

I0811 19:58:38.183485 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783917242-602ab236c878e-85bb75c1-bc67d9df
I0811 19:58:58.241276 140212882345984 traffic_director.py:475] Creating URL map "psm-interop-url-map-20230811-1958-dzff9": psm-grpc-server:22988 -> psm-interop-backend-service-20230811-1958-dzff9
I0811 19:58:58.247420 140212882345984 compute.py:530] Creating compute resource:
---
defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1958-dzff9
hostRules:
- hosts:
- psm-grpc-server:22988
pathMatcher: psm-interop-path-matcher-20230811-1958-dzff9
name: psm-interop-url-map-20230811-1958-dzff9
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/backendServices/psm-interop-backend-service-20230811-1958-dzff9
name: psm-interop-path-matcher-20230811-1958-dzff9
...

I0811 19:58:59.083671 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783938250-602ab24ad156c-5bcf7d52-105cace1
I0811 19:59:01.508020 140212882345984 traffic_director.py:574] Creating target psm-interop-target-proxy-20230811-1958-dzff9 proxy "GRPC" to URL map psm-interop-url-map-20230811-1958-dzff9
I0811 19:59:01.509918 140212882345984 compute.py:530] Creating compute resource:
---
name: psm-interop-target-proxy-20230811-1958-dzff9
url_map: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/urlMaps/psm-interop-url-map-20230811-1958-dzff9
validate_for_proxyless: true
...

I0811 19:59:02.223135 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783941512-602ab24dedb65-43528847-cc8abdee
I0811 19:59:04.631273 140212882345984 traffic_director.py:650] Creating forwarding rule "psm-interop-forwarding-rule-20230811-1958-dzff9" in network "default-vpc": 0.0.0.0:22988 -> https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1958-dzff9
I0811 19:59:04.634026 140212882345984 compute.py:530] Creating compute resource:
---
IPAddress: 0.0.0.0
loadBalancingScheme: INTERNAL_SELF_MANAGED
name: psm-interop-forwarding-rule-20230811-1958-dzff9
network: global/networks/default-vpc
portRange: 22988
target: https://www.googleapis.com/compute/v1/projects/grpc-testing/global/targetGrpcProxies/psm-interop-target-proxy-20230811-1958-dzff9
...

I0811 19:59:05.824691 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783944636-602ab250e86fb-61759e89-801d7bd2
I0811 19:59:21.157593 140212882345984 traffic_director.py:943] Creating Authz Policy psm-interop-authz-policy-20230811-1958-dzff9
I0811 19:59:21.165555 140212882345984 api.py:468] Creating networksecurity resource:
---
action: ALLOW
rules:
- destinations:
hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:22988
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:22988
- psm-grpc-server:22988
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
hosts:
- not-the-host:22988
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1958-dzff9/sa/psm-grpc-client
- destinations:
hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
...

I0811 19:59:21.454674 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783961244-602ab260bf2a4-392e61cb-bf1519be
I0811 19:59:25.872295 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1958-dzff9:
---
action: ALLOW
createTime: '2023-08-11T19:59:21.331998850Z'
name: projects/grpc-testing/locations/global/authorizationPolicies/psm-interop-authz-policy-20230811-1958-dzff9
rules:
- destinations:
- hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: host-wildcard
ports:
- 8080
- destinations:
- hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: header-regex-a+
ports:
- 8080
- destinations:
- hosts:
- psm-grpc-server:22988
httpHeaderMatch:
headerName: test
regexMatch: host-match1
ports:
- 8080
- hosts:
- a-not-it.com:22988
- psm-grpc-server:22988
- z-not-it.com:1
httpHeaderMatch:
headerName: test
regexMatch: host-match2
ports:
- 1
- 8080
- 65535
- destinations:
- hosts:
- not-the-host:22988
- not-the-host
httpHeaderMatch:
headerName: test
regexMatch: never-match-host
ports:
- 8080
- destinations:
- hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: never-match-port
ports:
- 1
- destinations:
- hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-client-20230811-1958-dzff9/sa/psm-grpc-client
- destinations:
- hosts:
- '*:22988'
httpHeaderMatch:
headerName: test
regexMatch: never-match-principal
ports:
- 8080
sources:
- principals:
- spiffe://grpc-testing.svc.id.goog/not/the/client
updateTime: '2023-08-11T19:59:24.629349476Z'
...

I0811 19:59:25.872634 140212882345984 traffic_director.py:1017] Creating Client TLS Policy psm-interop-client-tls-policy-20230811-1958-dzff9
I0811 19:59:25.874391 140212882345984 api.py:468] Creating networksecurity resource:
---
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:59:26.282062 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783965952-602ab2653cba2-8d7f8cd4-71b2934c
I0811 19:59:30.610407 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1958-dzff9:
---
createTime: '2023-08-11T19:59:26.030816874Z'
name: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1958-dzff9
serverValidationCa:
- certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:59:29.682438520Z'
...

I0811 19:59:30.610652 140212882345984 traffic_director.py:1069] Adding Client TLS Policy to Backend Service psm-interop-backend-service-20230811-1958-dzff9: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1958-dzff9, server spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1958-dzff9/sa/psm-grpc-server
I0811 19:59:30.614957 140212882345984 compute.py:537] Patching compute resource:
---
securitySettings:
clientTlsPolicy: projects/grpc-testing/locations/global/clientTlsPolicies/psm-interop-client-tls-policy-20230811-1958-dzff9
subjectAltNames:
- spiffe://grpc-testing.svc.id.goog/ns/psm-interop-server-20230811-1958-dzff9/sa/psm-grpc-server
...

I0811 19:59:31.576551 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691783970618-602ab269afb4d-bb77b762-b55f765e
I0811 19:59:55.890382 140212882345984 traffic_director.py:906] Creating Server TLS Policy psm-interop-server-tls-policy-20230811-1958-dzff9
I0811 19:59:55.892406 140212882345984 api.py:468] Creating networksecurity resource:
---
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
...

I0811 19:59:56.237669 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691783995959-602ab281daa65-e379eaef-7e681f31
I0811 20:00:00.631459 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1958-dzff9:
---
createTime: '2023-08-11T19:59:56.045846232Z'
name: projects/grpc-testing/locations/global/serverTlsPolicies/psm-interop-server-tls-policy-20230811-1958-dzff9
serverCertificate:
certificateProviderInstance:
pluginInstance: google_cloud_private_spiffe
updateTime: '2023-08-11T19:59:59.714078557Z'
...

I0811 20:00:00.631724 140212882345984 traffic_director.py:968] Creating Endpoint Policy psm-interop-endpoint-policy-20230811-1958-dzff9
I0811 20:00:00.634405 140212882345984 api.py:468] Creating networkservices resource:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1958-dzff9
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1958-dzff9-psm-grpc-server
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1958-dzff9
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
...

I0811 20:00:00.963200 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691784000691-602ab2865dea2-396300c4-3c2de51e
I0811 20:00:05.265222 140212882345984 api.py:490] Loaded projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1958-dzff9:
---
authorizationPolicy: psm-interop-authz-policy-20230811-1958-dzff9
createTime: '2023-08-11T20:00:00.754236106Z'
endpointMatcher:
metadataLabelMatcher:
metadataLabelMatchCriteria: MATCH_ALL
metadataLabels:
- labelName: app
labelValue: psm-interop-server-20230811-1958-dzff9-psm-grpc-server
name: projects/grpc-testing/locations/global/endpointPolicies/psm-interop-endpoint-policy-20230811-1958-dzff9
serverTlsPolicy: psm-interop-server-tls-policy-20230811-1958-dzff9
trafficPortSelector:
ports:
- '8080'
type: GRPC_SERVER
updateTime: '2023-08-11T20:00:04.366247353Z'
...

I0811 20:00:05.265510 140212882345984 k8s_xds_server_runner.py:145] Deploying xDS test server "psm-grpc-server" to k8s namespace psm-interop-server-20230811-1958-dzff9: test_port=8080 maintenance_port=8081 secure_mode=True replica_count=1
I0811 20:00:05.265760 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1958-dzff9%22;timeRange=2023-08-11T20:00:05.265609Z%2F2023-08-11T20:30:05.265609Z?project=grpc-testing
I0811 20:00:05.267616 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-server-20230811-1958-dzff9
labels:
name: psm-interop-server-20230811-1958-dzff9
owner: xds-k8s-interop-test
...

I0811 20:00:05.426038 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-server-20230811-1958-dzff9 created
I0811 20:00:05.429035 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server.service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1958-dzff9
labels:
owner: xds-k8s-interop-test
annotations:
cloud.google.com/neg: '{"exposed_ports": {"8080":{"name":"psm-interop-server-20230811-1958-dzff9-psm-grpc-server"}}}'
spec:
type: ClusterIP
selector:
app: psm-grpc-server
ports:
- port: 8080
protocol: TCP
targetPort: 8080
...

I0811 20:00:05.501365 140212882345984 k8s_base_runner.py:280] Service psm-grpc-server created
I0811 20:00:05.501612 140212882345984 k8s_base_runner.py:596] Waiting for NEG for service psm-grpc-server
I0811 20:00:05.595916 140212882345984 k8s_base_runner.py:601] Service psm-grpc-server: detected NEG=psm-interop-server-20230811-1958-dzff9-psm-grpc-server in zones=['us-central1-a']
I0811 20:00:05.596188 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1958-dzff9/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 20:00:06.366589 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1958-dzff9
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 20:00:06.427321 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-server created
I0811 20:00:06.435293 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/server-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-server
namespace: psm-interop-server-20230811-1958-dzff9
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-y9yqreie
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-server
deployment_id: psm-grpc-server-y9yqreie
template:
metadata:
annotations:
security.cloud.google.com/use-workload-certificates: ""
labels:
app: psm-grpc-server
deployment_id: psm-grpc-server-y9yqreie
owner: xds-k8s-interop-test
spec:
serviceAccountName: psm-grpc-server
containers:
- name: psm-grpc-server
image: gcr.io/grpc-testing/xds-interop/go-server:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8081
periodSeconds: 3
failureThreshold: 1000
args:
- "--port=8080"
- "--maintenance_port=8081"
- "--secure_mode=True"
ports:
- containerPort: 8080
- containerPort: 8081
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_NEW_SERVER_API
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_RBAC
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
- "--node-metadata=app=psm-interop-server-20230811-1958-dzff9-psm-grpc-server"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 20:00:06.513105 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-server created
I0811 20:00:06.513345 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0811 20:00:06.622735 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-587ccb864b-66xsf']
I0811 20:00:06.623077 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-server-587ccb864b-66xsf to start
I0811 20:00:10.257917 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-server-587ccb864b-66xsf ready, IP: 10.12.10.18
I0811 20:00:10.258196 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-server-587ccb864b-66xsf to authz_test/test_app_logs/psm-interop-server-20230811-1958-dzff9_psm-grpc-server-587ccb864b-66xsf.log
I0811 20:00:10.258555 140212790879808 k8s_log_collector.py:70] Starting log collection thread 140212790879808 for psm-grpc-server-587ccb864b-66xsf
I0811 20:00:10.258721 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0811 20:00:18.771010 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-server has 1 replicas available
I0811 20:00:18.808145 140212882345984 traffic_director.py:252] Waiting for Network Endpoint Groups to load endpoints.
I0811 20:00:19.071223 140212882345984 traffic_director.py:255] Loaded NEG "psm-interop-server-20230811-1958-dzff9-psm-grpc-server" in zone us-central1-a
I0811 20:00:19.071439 140212882345984 traffic_director.py:273] Adding backends to Backend Service psm-interop-backend-service-20230811-1958-dzff9: {ComputeV1.ZonalGcpResource(name='psm-interop-server-20230811-1958-dzff9-psm-grpc-server', url='https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1958-dzff9-psm-grpc-server', zone='us-central1-a')}
I0811 20:00:19.075417 140212882345984 compute.py:537] Patching compute resource:
---
backends:
- balancingMode: RATE
group: https://www.googleapis.com/compute/v1/projects/grpc-testing/zones/us-central1-a/networkEndpointGroups/psm-interop-server-20230811-1958-dzff9-psm-grpc-server
maxRatePerEndpoint: 5
...

I0811 20:00:20.208732 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784019080-602ab297e7596-c711163c-4a215e5e
I0811 20:01:01.888524 140212882345984 traffic_director.py:290] Waiting for Backend Service psm-interop-backend-service-20230811-1958-dzff9 to report all backends healthy: ['psm-interop-server-20230811-1958-dzff9-psm-grpc-server']
I0811 20:01:10.471355 140212882345984 compute.py:487] Backend psm-interop-server-20230811-1958-dzff9-psm-grpc-server in zone us-central1-a reported healthy
I0811 20:01:14.476122 140212882345984 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace psm-interop-client-20230811-1958-dzff9: server_target=xds:///psm-grpc-server:22988 rpc=UnaryCall qps=25 metadata='' secure_mode=True print_response=False
I0811 20:01:14.476509 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1958-dzff9%22;timeRange=2023-08-11T20:01:14.476342Z%2F2023-08-11T20:31:14.476342Z?project=grpc-testing
I0811 20:01:14.478431 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
name: psm-interop-client-20230811-1958-dzff9
labels:
name: psm-interop-client-20230811-1958-dzff9
owner: xds-k8s-interop-test
...

I0811 20:01:14.520336 140212882345984 k8s_base_runner.py:280] Namespace psm-interop-client-20230811-1958-dzff9 created
I0811 20:01:14.520561 140212882345984 k8s_base_runner.py:337] Granting roles/iam.workloadIdentityUser to serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1958-dzff9/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 20:01:15.216307 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1958-dzff9
labels:
owner: xds-k8s-interop-test
annotations:
iam.gke.io/gcp-service-account: xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
...

I0811 20:01:15.283313 140212882345984 k8s_base_runner.py:280] ServiceAccount psm-grpc-client created
I0811 20:01:15.291060 140212882345984 k8s_base_runner.py:258] Rendered template kubernetes-manifests/client-secure.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: psm-grpc-client
namespace: psm-interop-client-20230811-1958-dzff9
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hjm6tf63
owner: xds-k8s-interop-test
spec:
replicas: 1
selector:
matchLabels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hjm6tf63
template:
metadata:
labels:
app: psm-grpc-client
deployment_id: psm-grpc-client-hjm6tf63
owner: xds-k8s-interop-test
annotations:
security.cloud.google.com/use-workload-certificates: ""
spec:
serviceAccountName: psm-grpc-client
containers:
- name: psm-grpc-client
image: gcr.io/grpc-testing/xds-interop/go-client:0af3cad6ea5703485ec781cf0a15b4ae412010a4
imagePullPolicy: Always
startupProbe:
tcpSocket:
port: 8079
periodSeconds: 3
failureThreshold: 1000
args:
- "--server=xds:///psm-grpc-server:22988"
- "--stats_port=8079"
- "--secure_mode=True"
- "--qps=25"
- "--rpc=UnaryCall"
- "--print_response=False"
ports:
- containerPort: 8079
env:
- name: GRPC_XDS_BOOTSTRAP
value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
- name: GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT
value: "true"
- name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
value: "true"
volumeMounts:
- mountPath: /tmp/grpc-xds/
name: grpc-td-conf
readOnly: true
resources:
limits:
cpu: 800m
memory: 512Mi
requests:
cpu: 100m
memory: 512Mi
initContainers:
- name: grpc-td-init
image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
imagePullPolicy: Always
args:
- "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
- "--vpc-network-name=default-vpc"
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 10m
memory: 100Mi
volumeMounts:
- mountPath: /tmp/bootstrap/
name: grpc-td-conf
volumes:
- name: grpc-td-conf
emptyDir:
medium: Memory
...

I0811 20:01:15.364629 140212882345984 k8s_base_runner.py:280] Deployment psm-grpc-client created
I0811 20:01:15.364862 140212882345984 k8s_base_runner.py:531] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0811 20:01:15.474196 140212882345984 k8s_base_runner.py:541] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-6fff6df9d6-z4j65']
I0811 20:01:15.474487 140212882345984 k8s_base_runner.py:551] Waiting for pod psm-grpc-client-6fff6df9d6-z4j65 to start
I0811 20:01:21.845635 140212882345984 k8s_base_runner.py:554] Pod psm-grpc-client-6fff6df9d6-z4j65 ready, IP: 10.12.17.18
I0811 20:01:21.845908 140212882345984 k8s_base_runner.py:577] Enabling log collection from pod psm-grpc-client-6fff6df9d6-z4j65 to authz_test/test_app_logs/psm-interop-client-20230811-1958-dzff9_psm-grpc-client-6fff6df9d6-z4j65.log
I0811 20:01:21.846284 140212782487104 k8s_log_collector.py:70] Starting log collection thread 140212782487104 for psm-grpc-client-6fff6df9d6-z4j65
I0811 20:01:21.846426 140212882345984 k8s_base_runner.py:513] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0811 20:01:25.081721 140212882345984 k8s_base_runner.py:522] Deployment psm-grpc-client has 1 replicas available
I0811 20:01:25.082341 140212882345984 client_app.py:200] [psm-grpc-client-6fff6df9d6-z4j65] Waiting to report a READY channel to xds:///psm-grpc-server:22988
I0811 20:01:25.180706 140212882345984 client_app.py:236] [psm-grpc-client-6fff6df9d6-z4j65] Server channel:
I0811 20:01:35.226263 140212882345984 client_app.py:236] [psm-grpc-client-6fff6df9d6-z4j65] Server channel:
I0811 20:01:45.271456 140212882345984 client_app.py:236] [psm-grpc-client-6fff6df9d6-z4j65] Server channel:
I0811 20:01:55.317235 140212882345984 client_app.py:236] [psm-grpc-client-6fff6df9d6-z4j65] Server channel:
I0811 20:02:05.362455 140212882345984 client_app.py:236] [psm-grpc-client-6fff6df9d6-z4j65] Server channel:
I0811 20:02:05.394746 140212882345984 client_app.py:249] [psm-grpc-client-6fff6df9d6-z4j65] Found subchannel in state READY:
I0811 20:02:05.395063 140212882345984 client_app.py:211] [psm-grpc-client-6fff6df9d6-z4j65] Channel to xds:///psm-grpc-server:22988 transitioned to state READY:
I0811 20:02:10.400373 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_tls_allow.01_host_wildcard ---
I0811 20:02:10.400710 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 20:02:10.433182 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['EMPTY_CALL'], 'metadata': [{'key': 'test', 'value': 'host-wildcard'}]}), timeout=5, wait_for_ready=True)
I0811 20:02:10.465281 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 20:02:10.999503 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 20:02:11.033986 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6fff6df9d6-z4j65] << Received accumulated stats difference. Expecting RPCs with status (0, OK) for method EMPTY_CALL:
- method: EMPTY_CALL
rpcs_started: 13
result:
(0, OK): 13

I0811 20:02:11.034717 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_tls_allow.01_host_wildcard ---
I0811 20:02:11.034852 140212882345984 xds_k8s_testcase.py:239] --- Starting subTest __main__.AuthzTest.test_tls_allow.02_no_match ---
I0811 20:02:11.035009 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC XdsUpdateClientConfigureService.Configure(request=ClientConfigureRequest({'types': ['UNARY_CALL']}), timeout=5, wait_for_ready=True)
I0811 20:02:11.066993 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 20:02:11.601219 140212882345984 grpc.py:74] [psm-grpc-client-6fff6df9d6-z4j65:8079] >> RPC LoadBalancerStatsService.GetClientAccumulatedStats(request=LoadBalancerAccumulatedStatsRequest({}), timeout=600, wait_for_ready=True)
I0811 20:02:11.635664 140212882345984 xds_k8s_testcase.py:361] [psm-grpc-client-6fff6df9d6-z4j65] << Received accumulated stats difference. Expecting RPCs with status (7, PERMISSION_DENIED) for method UNARY_CALL:
- method: UNARY_CALL
rpcs_started: 13
result:
(7, PERMISSION_DENIED): 13

I0811 20:02:11.636434 140212882345984 xds_k8s_testcase.py:244] --- Finished subTest __main__.AuthzTest.test_tls_allow.02_no_match ---
I0811 20:02:11.636636 140212882345984 xds_k8s_testcase.py:640] ----- TestMethod __main__.AuthzTest.test_tls_allow teardown -----
I0811 20:02:11.783079 140212882345984 traffic_director.py:670] Deleting Forwarding rule "psm-interop-forwarding-rule-20230811-1958-dzff9"
I0811 20:02:12.624552 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784131787-602ab30363b0a-7bc0f90e-052a9e9a
I0811 20:02:40.946672 140212882345984 traffic_director.py:709] Deleting alternative Forwarding rule "psm-interop-forwarding-rule-alt-20230811-1958-dzff9"
I0811 20:02:41.125676 140212882345984 compute.py:561] Resource forwardingRule "psm-interop-forwarding-rule-alt-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:02:41.125887 140212882345984 traffic_director.py:601] Deleting HTTP Target proxy "psm-interop-target-proxy-20230811-1958-dzff9"
I0811 20:02:41.553579 140212882345984 compute.py:561] Resource targetHttpProxy "psm-interop-target-proxy-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:02:41.553800 140212882345984 traffic_director.py:589] Deleting Target GRPC proxy "psm-interop-target-proxy-20230811-1958-dzff9"
I0811 20:02:42.329020 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784161556-602ab31fc79e7-0616318d-a90c1cd2
I0811 20:02:44.659154 140212882345984 traffic_director.py:629] Deleting alternative Target GRPC proxy "psm-interop-target-proxy-alt-20230811-1958-dzff9"
I0811 20:02:45.098410 140212882345984 compute.py:561] Resource targetGrpcProxy "psm-interop-target-proxy-alt-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:02:45.098622 140212882345984 traffic_director.py:521] Deleting URL Map "psm-interop-url-map-20230811-1958-dzff9"
I0811 20:02:45.845001 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784165103-602ab3232990e-9995c7bb-dfbd05c5
I0811 20:02:48.218373 140212882345984 traffic_director.py:557] Deleting alternative URL Map "psm-interop-url-map-alt-20230811-1958-dzff9"
I0811 20:02:48.689065 140212882345984 compute.py:561] Resource urlMap "psm-interop-url-map-alt-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:02:48.689294 140212882345984 traffic_director.py:239] Deleting Backend Service "psm-interop-backend-service-20230811-1958-dzff9"
I0811 20:02:49.338075 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784168694-602ab3269629d-c53d5dd0-8b92809d
I0811 20:03:22.229069 140212882345984 traffic_director.py:328] Deleting Alternative Backend Service "psm-interop-backend-service-alt-20230811-1958-dzff9"
I0811 20:03:22.710068 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-alt-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:03:22.710294 140212882345984 traffic_director.py:401] Deleting Affinity Backend Service "psm-interop-backend-service-affinity-20230811-1958-dzff9"
I0811 20:03:23.202496 140212882345984 compute.py:561] Resource backendService "psm-interop-backend-service-affinity-20230811-1958-dzff9" not deleted since it does not exist
I0811 20:03:23.202733 140212882345984 traffic_director.py:198] Deleting Health Check "psm-interop-health-check-20230811-1958-dzff9"
I0811 20:03:23.805688 140212882345984 compute.py:589] Waiting 600 sec for compute operation id: operation-1691784203289-602ab3479443e-b134b9a4-b5cf2b71
I0811 20:03:28.350843 140212882345984 traffic_director.py:1011] Deleting Endpoint Policy psm-interop-endpoint-policy-20230811-1958-dzff9
I0811 20:03:28.639182 140212882345984 api.py:524] Waiting 600 sec for networkservices operation id: projects/grpc-testing/locations/global/operations/operation-1691784208420-602ab34c78cce-c2957fd9-6cee40ee
I0811 20:03:49.558756 140212882345984 traffic_director.py:937] Deleting Server TLS Policy psm-interop-server-tls-policy-20230811-1958-dzff9
I0811 20:03:49.898795 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691784229668-602ab360bc4fb-92ef4784-a213b77b
I0811 20:04:08.889247 140212882345984 traffic_director.py:1046] Deleting Client TLS Policy psm-interop-client-tls-policy-20230811-1958-dzff9
I0811 20:04:09.230314 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691784248995-602ab3732afcd-458f368a-f680f8ee
I0811 20:04:21.833804 140212882345984 traffic_director.py:960] Deleting Authz Policy psm-interop-authz-policy-20230811-1958-dzff9
I0811 20:04:22.071675 140212882345984 api.py:524] Waiting 600 sec for networksecurity operation id: projects/grpc-testing/locations/global/operations/operation-1691784261897-602ab37f78d7c-73c34828-09846178
I0811 20:04:26.287102 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-client
I0811 20:04:36.555059 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-client-20230811-1958-dzff9/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 20:04:37.067702 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-client
I0811 20:04:37.189754 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-client-20230811-1958-dzff9
I0811 20:05:07.374345 140212882345984 k8s_base_runner.py:462] Deleting deployment psm-grpc-server
I0811 20:05:17.561477 140212882345984 k8s_base_runner.py:474] Deleting service psm-grpc-server
I0811 20:05:17.680183 140212882345984 k8s_base_runner.py:356] Revoking roles/iam.workloadIdentityUser from serviceAccount:grpc-testing.svc.id.goog[psm-interop-server-20230811-1958-dzff9/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@grpc-testing.iam.gserviceaccount.com
I0811 20:05:18.484270 140212882345984 k8s_base_runner.py:487] Deleting service account psm-grpc-server
I0811 20:05:18.607720 140212882345984 k8s_base_runner.py:499] Deleting namespace psm-interop-server-20230811-1958-dzff9
I0811 20:06:48.973701 140212882345984 xds_k8s_testcase.py:664] ----- Test client/server logs -----
I0811 20:06:48.974116 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-client-hjm6tf63:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22psm-interop-client-20230811-1958-dzff9%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-hjm6tf63%22;timeRange=2023-08-11T20:01:14.476334Z%2F2023-08-11T20:05:07.371044Z?project=grpc-testing
I0811 20:06:48.974308 140212882345984 k8s_base_runner.py:672] GCP Logs Explorer link to psm-grpc-server-y9yqreie:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22grpc-testing%22%0Aresource.labels.container_name%3D%22psm-grpc-server%22%0Aresource.labels.namespace_name%3D%22psm-interop-server-20230811-1958-dzff9%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-server-y9yqreie%22;timeRange=2023-08-11T20:00:05.265600Z%2F2023-08-11T20:06:48.973517Z?project=grpc-testing
[ OK ] AuthzTest.test_tls_allow
----------------------------------------------------------------------
Ran 4 tests in 1965.208s

OK
+ echo 'Failed test suites: 0'
Failed test suites: 0


[ID: 9179954] Command finished after 4945 secs, exit value: 0


Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.
[13:06:50] Collecting build artifacts from build VM
[13:06:51] Kokoro builder finished