[ENTRYPOINT] Validating Runtime configuration... [CONFIG] validation PASS streams=0 active_learning=False [GPU] Selected GPU ID: 0 [ENTRYPOINT] Starting Triton Inference Server... [ENTRYPOINT] Triton PID: 73 [ENTRYPOINT] Waiting for Triton to be ready... I0914 02:27:48.420279 73 pinned_memory_manager.cc:277] "Pinned memory pool is created at '0x7daf34000000' with size 268435456" I0914 02:27:48.420444 73 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 0 with size 67108864" I0914 02:27:48.423801 73 model_lifecycle.cc:473] "loading: RFDETRMedium:2" I0914 02:27:48.423873 73 model_lifecycle.cc:473] "loading: RFDETRKeypointPreview:2" I0914 02:27:48.423922 73 model_lifecycle.cc:473] "loading: RFDETRFireSmoke:2" I0914 02:27:48.540886 73 tensorrt.cc:65] "TRITONBACKEND_Initialize: tensorrt" I0914 02:27:48.540954 73 tensorrt.cc:75] "Triton TRITONBACKEND API version: 1.19" I0914 02:27:48.540966 73 tensorrt.cc:81] "'tensorrt' TRITONBACKEND API version: 1.19" I0914 02:27:48.540975 73 tensorrt.cc:105] "backend configuration:\n{\"cmdline\":{\"auto-complete-config\":\"true\",\"backend-directory\":\"/opt/tritonserver/backends\",\"min-compute-capability\":\"6.000000\",\"default-max-batch-size\":\"4\"}}" I0914 02:27:48.541963 73 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: RFDETRMedium (version 2)" I0914 02:27:48.541983 73 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: RFDETRFireSmoke (version 2)" I0914 02:27:48.542178 73 tensorrt.cc:231] "TRITONBACKEND_ModelInitialize: RFDETRKeypointPreview (version 2)" W0914 02:27:48.544298 73 logging.cc:43] "WARNING The logger passed into createInferRuntime differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead." W0914 02:27:48.544430 73 logging.cc:43] "WARNING The logger passed into createInferRuntime differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead." I0914 02:27:48.776667 73 logging.cc:46] "Loaded engine size: 61 MiB" I0914 02:27:48.830176 73 logging.cc:46] "Loaded engine size: 62 MiB" I0914 02:27:48.847357 73 logging.cc:46] "Loaded engine size: 72 MiB" I0914 02:27:49.024729 73 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: RFDETRKeypointPreview_0 (GPU device 0)" W0914 02:27:49.025079 73 logging.cc:43] "WARNING The logger passed into createInferRuntime differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead." I0914 02:27:49.038321 73 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: RFDETRMedium_0 (GPU device 0)" I0914 02:27:49.056246 73 tensorrt.cc:297] "TRITONBACKEND_ModelInstanceInitialize: RFDETRFireSmoke_0 (GPU device 0)" I0914 02:27:49.244059 73 logging.cc:46] "Loaded engine size: 72 MiB" I0914 02:27:49.325629 73 logging.cc:46] "[MS] Running engine with multi stream info" I0914 02:27:49.325722 73 logging.cc:46] "[MS] Number of aux streams is 4" I0914 02:27:49.325731 73 logging.cc:46] "[MS] Number of total worker streams is 5" I0914 02:27:49.325741 73 logging.cc:46] "[MS] The main stream provided by execute/enqueue calls is the first worker stream" I0914 02:27:49.509380 73 logging.cc:46] "Loaded engine size: 62 MiB" W0914 02:27:49.591208 73 logging.cc:43] "WARNING The logger passed into createInferRuntime differs from one already registered for an existing builder, runtime, or refitter. So the current new logger is ignored, and TensorRT will use the existing one which is returned by nvinfer1::getLogger() instead." I0914 02:27:49.591249 73 logging.cc:46] "[MS] Running engine with multi stream info" I0914 02:27:49.591372 73 logging.cc:46] "[MS] Number of aux streams is 1" I0914 02:27:49.591387 73 logging.cc:46] "[MS] Number of total worker streams is 2" I0914 02:27:49.591398 73 logging.cc:46] "[MS] The main stream provided by execute/enqueue calls is the first worker stream" I0914 02:27:49.638381 73 logging.cc:46] "[MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +127, now: CPU 0, GPU 195 (MiB)" I0914 02:27:49.639555 73 instance_state.cc:186] "Created instance RFDETRKeypointPreview_0 on GPU 0 with stream priority 0 and optimization profile default[0];" I0914 02:27:49.640414 73 model_lifecycle.cc:849] "successfully loaded 'RFDETRKeypointPreview'" I0914 02:27:49.686422 73 logging.cc:46] "Loaded engine size: 61 MiB" I0914 02:27:49.739346 73 logging.cc:46] "[MS] Running engine with multi stream info" I0914 02:27:49.739392 73 logging.cc:46] "[MS] Number of aux streams is 1" I0914 02:27:49.739401 73 logging.cc:46] "[MS] Number of total worker streams is 2" I0914 02:27:49.739410 73 logging.cc:46] "[MS] The main stream provided by execute/enqueue calls is the first worker stream" [ENTRYPOINT] Triton not ready... retry (1/30) I0914 02:27:50.159993 73 logging.cc:46] "[MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +17, now: CPU 0, GPU 270 (MiB)" I0914 02:27:50.161719 73 instance_state.cc:186] "Created instance RFDETRFireSmoke_0 on GPU 0 with stream priority 0 and optimization profile default[0];" I0914 02:27:50.162416 73 model_lifecycle.cc:849] "successfully loaded 'RFDETRFireSmoke'" I0914 02:27:50.247770 73 logging.cc:46] "[MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +92, now: CPU 0, GPU 270 (MiB)" I0914 02:27:50.248856 73 instance_state.cc:186] "Created instance RFDETRMedium_0 on GPU 0 with stream priority 0 and optimization profile default[0];" I0914 02:27:50.249481 73 model_lifecycle.cc:849] "successfully loaded 'RFDETRMedium'" I0914 02:27:50.249805 73 server.cc:620] +------------------+------+ | Repository Agent | Path | +------------------+------+ +------------------+------+ I0914 02:27:50.249939 73 server.cc:647] +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Backend | Path | Config | +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tensorrt | /opt/tritonserver/backends/tensorrt/libtriton_tensorrt.so | {"cmdline":{"auto-complete-config":"true","backend-directory":"/opt/tritonserver/backends","min-compute-capability":"6.000000","default-max-batch-size":"4"}} | +----------+-----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+ I0914 02:27:50.250044 73 server.cc:690] +-----------------------+---------+--------+ | Model | Version | Status | +-----------------------+---------+--------+ | RFDETRFireSmoke | 2 | READY | | RFDETRKeypointPreview | 2 | READY | | RFDETRMedium | 2 | READY | +-----------------------+---------+--------+ I0914 02:27:50.324341 73 metrics.cc:889] "Collecting metrics for GPU 0: NVIDIA RTX A6000" I0914 02:27:50.331288 73 metrics.cc:782] "Collecting CPU metrics" I0914 02:27:50.331567 73 tritonserver.cc:2598] +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Option | Value | +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | server_id | triton | | server_version | 2.67.0 | | server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data parameters statistics trace logging | | model_repository_path[0] | /opt/contextra/models/triton | | model_control_mode | MODE_NONE | | strict_model_config | 0 | | model_config_name | | | rate_limit | OFF | | pinned_memory_pool_byte_size | 268435456 | | cuda_memory_pool_byte_size{0} | 67108864 | | min_supported_compute_capability | 6.0 | | strict_readiness | 0 | | exit_timeout | 30 | | cache_enabled | 0 | +----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I0914 02:27:50.340567 73 grpc_server.cc:2562] "Started GRPCInferenceService at 0.0.0.0:8001" I0914 02:27:50.341073 73 http_server.cc:4823] "Started HTTPService at 0.0.0.0:8000" I0914 02:27:50.384554 73 http_server.cc:358] "Started Metrics Service at 0.0.0.0:8002" [ENTRYPOINT] Triton is ready at localhost:8001 [ENTRYPOINT] Starting FastAPI application... [ENTRYPOINT] Supervisor loop (watchdog every 10s): triton=73 fastapi=201 [RedisPublisher] Connected [FastAPI] Starting uvicorn on port 5000 ... INFO: Started server process [201] INFO: Waiting for application startup. (python3:201): GStreamer-WARNING **: 02:27:54.068: External plugin loader failed. This most likely means that the plugin loader helper binary was not found or could not be run. You might need to set the GST_PLUGIN_SCANNER environment variable if your setup is unusual. This should normally not be required though. (python3:201): GStreamer-WARNING **: 02:27:54.390: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory Plugin contextra_rtp_time initialized (python3:201): GStreamer-WARNING **: 02:27:54.419: Failed to load plugin '/opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory (python3:201): GStreamer-WARNING **: 02:27:55.953: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.1: cannot open shared object file: No such file or directory [STREAM-MGR] loaded 1 streams (base=/etc/contextra/streams.yaml, overlay=/var/lib/contextra/runtime/streams_runtime.yaml) INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit) INFO: 127.0.0.1:56572 - "GET /ready HTTP/1.1" 200 OK INFO: 172.20.0.1:49500 - "GET /ready HTTP/1.1" 200 OK INFO: 172.20.0.1:49510 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49516 - "GET /api/v1/streams/CCV-01577/settings HTTP/1.1" 200 OK INFO: 172.20.0.1:49528 - "PATCH /api/v1/streams/CCV-01577/settings HTTP/1.1" 200 OK INFO: 172.20.0.1:49532 - "POST /api/v1/streams/CCV-01577/start HTTP/1.1" 202 Accepted INFO: 172.20.0.1:49542 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49558 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:49574 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49590 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:49594 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49602 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:49608 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49624 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK [STREAM-MGR] CCV-01577: STARTING type=file generation=1 Initializing GStreamer Backend...! Add Element ... batch_capture-mux-1 Add Element ... batch_capture-source-0_0 LINKING: Source: batch_capture-source-0_0 Target: batch_capture-mux-1 Add Element ... preprocess-preprocess-0 Add Element ... preprocess-queue-1 LINKING: Source: batch_capture-mux-1 Target: preprocess-preprocess-0 LINKING: Source: preprocess-preprocess-0 Target: preprocess-queue-1 Add Element ... infer-infer-0 LINKING: Source: preprocess-queue-1 Target: infer-infer-0 Add Element ... retrieve-converter-0 Add Element ... retrieve-appsink-2 Add Element ... retrieve-capsfilter-1 LINKING: Source: retrieve-converter-0 Target: retrieve-capsfilter-1 LINKING: Source: retrieve-capsfilter-1 Target: retrieve-appsink-2 LINKING: Source: infer-infer-0 Target: retrieve-converter-0 INFO: 172.20.0.1:49628 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49642 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: TritonGrpcBackend id:1430340086 initialized for model: RFDETRFireSmoke INFO: 172.20.0.1:49656 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49668 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK WARNING: unsupported tensor order for dims to image-info, retry as kLinear [rfdetr-adapter] Fire processor loaded Event Thread Enabled... Main Loop Running... Opening in BLOCKING MODE End of file reached on stdin. *** Inside cb_newpad name=video/x-raw INFO: 172.20.0.1:49676 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49688 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:49692 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49694 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK [rfdetr-adapter] stage=native_enter count=1 [rfdetr-adapter] stage=native_complete count=1 /opt/contextra/app/byte_tracking.py:62: FutureWarning: The `ByteTrack` was deprecated since v0.28.0. It will be removed in v0.31.0. self._trackers[class_id] = self.sv.ByteTrack( INFO: 172.20.0.1:49698 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:49708 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32872 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32880 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32894 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32906 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32922 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32926 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK [NATIVE_BRIDGE_SAMPLE] {"source": [1920, 1080], "surface": [1920, 1080], "detections": [{"class_id": 0, "score": 0.5820159912109375, "bbox_xyxy": [0.36627197265625, 0.2078857421875, 0.49652099609375, 0.4927978515625]}]} INFO: 172.20.0.1:32940 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32946 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32952 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32954 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32970 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32972 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:32986 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:32998 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK INFO: 172.20.0.1:33002 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:33014 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK Posted EOS to stop pipeline INFO: 172.20.0.1:33020 - "POST /api/v1/streams/CCV-01577/stop HTTP/1.1" 202 Accepted INFO: 172.20.0.1:33028 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:33044 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK nvstreammux: Successfully handled EOS for source_id=0 End of stream Main Loop Exited... sys:1: Warning: Source ID 3 was not found when attempting to remove it Pipeline det stopped INFO: 172.20.0.1:33048 - "GET /api/v1/streams/CCV-01577 HTTP/1.1" 200 OK INFO: 172.20.0.1:33050 - "GET /api/v1/metadata/status HTTP/1.1" 200 OK