[ENTRYPOINT] Validating Runtime configuration... [CONFIG] validation PASS streams=0 active_learning=False [GPU] Selected GPU ID: 0 [ENTRYPOINT] Starting Triton Inference Server... [ENTRYPOINT] Triton PID: 74 [ENTRYPOINT] Waiting for Triton to be ready... I0911 11:16:49.581463 74 pinned_memory_manager.cc:277] "Pinned memory pool is created at '0x6ffffc000000' with size 268435456" I0911 11:16:49.581817 74 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 0 with size 67108864" I0911 11:16:49.586027 74 model_lifecycle.cc:473] "loading: RFDETRMedium:2" I0911 11:16:49.720112 74 tensorrt.cc:65] "TRITONBACKEND_Initialize: tensorrt" I0911 11:16:49.720178 74 tensorrt.cc:75] "Triton TRITONBACKEND API version: 1.19" I0911 11:16:49.720188 74 tensorrt.cc:81] "'tensorrt' TRITONBACKEND API version: 1.19" I0911 11:16:49.720197 74 tensorrt.cc:105] "backend configuration:\n{\"cmdline\":{\"auto-complete-config\":\"true\",\"backend-directory\":\"/opt/tritonserver/backends\",\"min-compute-capability\":\"6.000000\",\"default-max-batch-size\":\"4\"}}" I0911 11:16:49.720776 74 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: RFDETRMedium (version 2)" I0911 11:16:49.831607 74 logging.cc:46] "Loaded engine size: 62 MiB" I0911 11:16:49.945468 74 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: RFDETRMedium_0 (GPU device 0)" I0911 11:16:50.047412 74 logging.cc:46] "Loaded engine size: 62 MiB" I0911 11:16:50.095491 74 logging.cc:46] "[MS] Running engine with multi stream info" I0911 11:16:50.095534 74 logging.cc:46] "[MS] Number of aux streams is 1" I0911 11:16:50.095542 74 logging.cc:46] "[MS] Number of total worker streams is 2" I0911 11:16:50.095550 74 logging.cc:46] "[MS] The main stream provided by execute/enqueue calls is the first worker stream" I0911 11:16:50.325952 74 logging.cc:46] "[MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +18, now: CPU 0, GPU 75 (MiB)" I0911 11:16:50.326674 74 instance_state.cc:186] "Created instance RFDETRMedium_0 on GPU 0 with stream priority 0 and optimization profile default[0];" I0911 11:16:50.327207 74 model_lifecycle.cc:849] "successfully loaded 'RFDETRMedium'" I0911 11:16:50.327408 74 server.cc:620] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+ I0911 11:16:50.327459 74 server.cc:647] +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Backend | Path | Config | +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tensorrt | /opt/tritonserver/backends/tensorrt/libtriton_tensorrt.so | {"cmdline":{"auto-complete-config":"true","backend-directory":"/opt/tritonserver/backends","min-compute-capability":"6.000000","default-max-batch-size":"4"}} | +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ I0911 11:16:50.327509 74 server.cc:690] +--------------+---------+--------+ | Model | Version | Status | +--------------+---------+--------+ | RFDETRMedium | 2 | READY | +--------------+---------+--------+ I0911 11:16:50.385074 74 metrics.cc:889] "Collecting metrics for GPU 0: NVIDIA RTX A6000" I0911 11:16:50.390403 74 metrics.cc:782] "Collecting CPU metrics" I0911 11:16:50.390705 74 tritonserver.cc:2598] +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Option | Value | +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | server_id | triton | | server_version | 2.67.0 | | server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging | | model_repository_path[0] | /opt/contextra/models/triton | | model_control_mode | MODE_NONE | | strict_model_config | 0 | | model_config_name | | | rate_limit | OFF | | pinned_memory_pool_byte_size | 268435456 | | cuda_memory_pool_byte_size{0} | 67108864 | | min_supported_compute_capability | 6.0 | | strict_readiness | 0 | | exit_timeout | 30 | | cache_enabled | 0 | +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I0911 11:16:50.397753 74 grpc_server.cc:2562] "Started GRPCInferenceService at 0.0.0.0:8001" I0911 11:16:50.398215 74 http_server.cc:4823] "Started HTTPService at 0.0.0.0:8000" I0911 11:16:50.440783 74 http_server.cc:358] "Started Metrics Service at 0.0.0.0:8002" [ENTRYPOINT] Triton not ready... retry (1/30) [ENTRYPOINT] Triton is ready at localhost:8001 [ENTRYPOINT] Starting FastAPI application... [ENTRYPOINT] Supervisor loop (watchdog every 10s): triton=74 fastapi=198 [RedisPublisher] Connected [FastAPI] Starting uvicorn on port 5000 ... INFO: Started server process [198] INFO: Waiting for application startup. (python3:198): GStreamer-WARNING **: 11:16:54.947: External plugin loader failed. This most likely means that the plugin loader helper binary was not found or could not be run. You might need to set the GST_PLUGIN_SCANNER environment variable if your setup is unusual. This should normally not be required though. (python3:198): GStreamer-WARNING **: 11:16:55.200: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory (python3:198): GStreamer-WARNING **: 11:16:55.226: Failed to load plugin '/opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory (python3:198): GStreamer-WARNING **: 11:16:56.592: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory [STREAM-MGR] loaded 1 streams (base=/etc/contextra/streams.yaml, overlay=/var/lib/contextra/runtime/streams_runtime.yaml) INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit) INFO: 127.0.0.1:38098 - "GET /ready HTTP/1.1" 200 OK