=== $ kubectl apply -f ns.yaml === namespace/istio-vt-t12 created === $ kubectl apply -f manifest.yaml === pod/client created deployment.apps/echo created service/echo created serviceentry.networking.istio.io/gwerr-deadport-se created destinationrule.networking.istio.io/gwerr-deadport-dr created === $ kubectl -n istio-vt-t12 wait --for=condition=Ready pod/client --timeout=90s === pod/client condition met === $ kubectl -n istio-vt-t12 rollout status deploy/echo --timeout=90s === deployment "echo" successfully rolled out === $ kubectl -n istio-vt-t12 get pods -o wide (sanity, 2/2 check) === NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES client 2/2 Running 0 12s 10.255.159.172 k8s-master1 echo-5767bbcc56-rpz54 2/2 Running 0 12s 10.255.159.173 k8s-master1 === $ for i in $(seq 1 12); do kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s -o /dev/null -w '%{http_code} ' --max-time 2 http://mock.istio-verify-ext.svc.homelab.local:8299/; sleep 1; done === -- attempt 1 -- 000 command terminated with exit code 28 -- attempt 2 -- 000 command terminated with exit code 28 -- attempt 3 -- 000 command terminated with exit code 28 -- attempt 4 -- 000 command terminated with exit code 28 -- attempt 5 -- 000 command terminated with exit code 28 -- attempt 6 -- 000 command terminated with exit code 28 -- attempt 7 -- 000 command terminated with exit code 28 -- attempt 8 -- 000 command terminated with exit code 28 -- attempt 9 -- 000 command terminated with exit code 28 -- attempt 10 -- 000 command terminated with exit code 28 -- attempt 11 -- 000 command terminated with exit code 28 -- attempt 12 -- 000 command terminated with exit code 28 === $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s 'localhost:15000/clusters' | grep 'outbound|8299||mock.istio-verify-ext.svc.homelab.local' === outbound|8299||mock.istio-verify-ext.svc.homelab.local::observability_name::outbound|8299||mock.istio-verify-ext.svc.homelab.local; outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_connections::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_pending_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_retries::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_connections::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_pending_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_retries::3 outbound|8299||mock.istio-verify-ext.svc.homelab.local::added_via_api::true outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_connect_fail::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_total::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_error::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_success::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_timeout::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_total::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::hostname::mock.istio-verify-ext.svc.homelab.local outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::healthy outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::weight::1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::region:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::sub_zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::canary::false outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::priority::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::success_rate::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::local_origin_success_rate::-1 === DIAGNOSTIC: $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- id === uid=1337(istio-proxy) gid=1337(istio-proxy) groups=1337(istio-proxy) === DIAGNOSTIC: $ kubectl -n istio-vt-t12 exec client -c curl -- id === uid=100(curl_user) gid=101(curl_group) groups=101(curl_group) === DIAGNOSTIC: envoy proxy config bootstrap - check istio-proxy uid === {"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true,"runAsGroup":1337,"runAsNonRoot":true,"runAsUser":1337} === DIAGNOSTIC: re-run 3 attempts from the 'curl' (app) container instead === -- app-container attempt 1 -- 000 command terminated with exit code 28 -- app-container attempt 2 -- 000 command terminated with exit code 28 -- app-container attempt 3 -- 503 === $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s 'localhost:15000/clusters' | grep outbound.8299 (after app-container test) === outbound|8299||mock.istio-verify-ext.svc.homelab.local::observability_name::outbound|8299||mock.istio-verify-ext.svc.homelab.local; outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_connections::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_pending_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_retries::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_connections::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_pending_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_retries::3 outbound|8299||mock.istio-verify-ext.svc.homelab.local::added_via_api::true outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_connect_fail::5 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_total::5 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_error::3 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_success::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_timeout::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_total::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::hostname::mock.istio-verify-ext.svc.homelab.local outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::healthy outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::weight::1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::region:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::sub_zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::canary::false outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::priority::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::success_rate::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::local_origin_success_rate::-1 === NOTE: exec into '-c istio-proxy' (uid 1337) BYPASSES sidecar interception === (iptables excludes traffic originated by the proxy's own UID to avoid redirect loops). Confirmed: 12x from istio-proxy -> cluster stats stayed cx_total=0/cx_connect_fail=0 (envoy never saw the traffic). 3x from app container 'curl' (uid 100) -> cx_total=5, cx_connect_fail=5, rq_error=3 (envoy DID see it). => Correcting vantage point to the app container 'curl' for the real test, per test intent (traffic must transit the sidecar). === RESET: delete+recreate pod/client to zero envoy stats for a clean measurement window === pod "client" deleted from istio-vt-t12 namespace error: must specify one of -f and -k Error from server (NotFound): pods "client" not found NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES echo-5767bbcc56-rpz54 2/2 Running 0 3m33s 10.255.159.173 k8s-master1 === $ kubectl apply -f manifest.yaml (recreate pod/client only, rest unchanged) === pod/client created deployment.apps/echo unchanged service/echo unchanged serviceentry.networking.istio.io/gwerr-deadport-se unchanged destinationrule.networking.istio.io/gwerr-deadport-dr unchanged === $ kubectl -n istio-vt-t12 wait --for=condition=Ready pod/client --timeout=90s === pod/client condition met NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES client 2/2 Running 0 3s 10.255.159.174 k8s-master1 echo-5767bbcc56-rpz54 2/2 Running 0 3m43s 10.255.159.173 k8s-master1 === CORRECTED RUN: $ for i in $(seq 1 12); do kubectl -n istio-vt-t12 exec client -c curl -- curl -s -o /dev/null -w '%{http_code} ' --max-time 2 http://mock.istio-verify-ext.svc.homelab.local:8299/; sleep 1; done === -- attempt 1 -- 000 command terminated with exit code 28 -- attempt 2 -- 000 command terminated with exit code 28 -- attempt 3 -- 503 -- attempt 4 -- 503 -- attempt 5 -- 503 -- attempt 6 -- 503 -- attempt 7 -- 503 -- attempt 8 -- 503 -- attempt 9 -- 503 -- attempt 10 -- 503 -- attempt 11 -- 503 -- attempt 12 -- 503 === $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s 'localhost:15000/clusters' | grep 'outbound|8299||mock.istio-verify-ext.svc.homelab.local' === outbound|8299||mock.istio-verify-ext.svc.homelab.local::observability_name::outbound|8299||mock.istio-verify-ext.svc.homelab.local; outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_average::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::outlier::local_origin_success_rate_ejection_threshold::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_connections::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_pending_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_requests::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::default_priority::max_retries::4294967295 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_connections::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_pending_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_requests::1024 outbound|8299||mock.istio-verify-ext.svc.homelab.local::high_priority::max_retries::3 outbound|8299||mock.istio-verify-ext.svc.homelab.local::added_via_api::true outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_connect_fail::5 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::cx_total::5 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_active::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_error::3 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_success::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_timeout::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::rq_total::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::hostname::mock.istio-verify-ext.svc.homelab.local outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::healthy outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::weight::1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::region:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::sub_zone:: outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::canary::false outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::priority::0 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::success_rate::-1 outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::local_origin_success_rate::-1 === SUPPLEMENTAL: $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s 'localhost:15000/stats?filter=outlier_detection' | grep 8299 === === SUPPLEMENTAL: $ kubectl -n istio-vt-t12 exec client -c istio-proxy -- curl -s 'localhost:15000/stats' | grep 8299 | grep -i upstream_cx === === RIGOR CHECK: reset pod again and poll health_flags AFTER EACH attempt (to catch a transient eject-then-recover within baseEjectionTime=15s window) === pod "client" deleted from istio-vt-t12 namespace pod/client created deployment.apps/echo unchanged service/echo unchanged serviceentry.networking.istio.io/gwerr-deadport-se unchanged destinationrule.networking.istio.io/gwerr-deadport-dr unchanged pod/client condition met attempt 1: http_code=000command terminated with exit code 28 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::healthy attempt 2: http_code=000command terminated with exit code 28 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::healthy attempt 3: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 4: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 5: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 6: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 7: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 8: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 9: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 10: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 11: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 12: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 13: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check attempt 14: http_code=503 | outbound|8299||mock.istio-verify-ext.svc.homelab.local::10.250.183.220:8299::health_flags::/failed_outlier_check === CONCLUSION === Ejection triggered at attempt 3 (3rd consecutive TCP connect failure) = matches consecutiveGatewayErrors:3 threshold exactly. consecutive5xxErrors was explicitly 0 (disabled), so consecutiveGatewayErrors is the only detector that could have caused this eject. => TCP connect failures (dead port, no listener) DO count toward consecutiveGatewayErrors by default (splitExternalLocalOriginErrors unset/false). => This directly CONTRADICTS the tested claim that connect failures are excluded from consecutiveGatewayErrors by default.