During testing of the downstream health endpoints, I found that some kong and hub requests could cause stalls in responses since the hub-python-client and kong-admin-client both use synchronous calls while the HA has them defined as async. Slow hub responses only had ~5s freezes, but kong has no timeout and degraded the whole api. These endpoints need to be synchronous and shared deps need to run in the same threadpool.
This was not apparent during development, but bombarding the API revealed these issues and they should be patched ASAP
During testing of the downstream health endpoints, I found that some kong and hub requests could cause stalls in responses since the
hub-python-clientandkong-admin-clientboth use synchronous calls while the HA has them defined as async. Slow hub responses only had ~5s freezes, but kong has no timeout and degraded the whole api. These endpoints need to be synchronous and shared deps need to run in the same threadpool.This was not apparent during development, but bombarding the API revealed these issues and they should be patched ASAP