{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "MxFBtHQ4ooZh" }, "source": [ "\"Open" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "ff6iCPqqooZp" }, "source": [ "# Spatio-temporal action detection with MMAction2\n", "Welcome to MMAction2! This is a tutorial on how to use MMAction2 for spatio-temporal action detection. In this tutorial, we will use the MultiSports dataset as an example, and provide a complete step-by-step guide for spatio-temporal action detection, including\n", "- Prepare spatio-temporal action detection dataset\n", "- Train detection model\n", "- Prepare AVA format dataset\n", "- Train spatio-temporal action detection model\n" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "xQlffdn7ooZq" }, "source": [ "## 0. Install MMAction2 and MMDetection" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "4vWjBJI-ooZr", "outputId": "1c852c24-eb40-407d-e1c4-72d4b43385a3" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Collecting openmim\n", " Downloading openmim-0.3.7-py2.py3-none-any.whl (51 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m51.3/51.3 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: Click in /usr/local/lib/python3.10/dist-packages (from openmim) (8.1.3)\n", "Collecting colorama (from openmim)\n", " Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n", "Collecting model-index (from openmim)\n", " Downloading model_index-0.1.11-py3-none-any.whl (34 kB)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from openmim) (1.5.3)\n", "Requirement already satisfied: pip>=19.3 in /usr/local/lib/python3.10/dist-packages (from openmim) (23.1.2)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from openmim) (2.27.1)\n", "Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from openmim) (13.3.4)\n", "Requirement already satisfied: tabulate in /usr/local/lib/python3.10/dist-packages (from openmim) (0.8.10)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from model-index->openmim) (6.0)\n", "Requirement already satisfied: markdown in /usr/local/lib/python3.10/dist-packages (from model-index->openmim) (3.4.3)\n", "Collecting ordered-set (from model-index->openmim)\n", " Downloading ordered_set-4.1.0-py3-none-any.whl (7.6 kB)\n", "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->openmim) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->openmim) (2022.7.1)\n", "Requirement already satisfied: numpy>=1.21.0 in /usr/local/lib/python3.10/dist-packages (from pandas->openmim) (1.22.4)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->openmim) (1.26.15)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->openmim) (2022.12.7)\n", "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->openmim) (2.0.12)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->openmim) (3.4)\n", "Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->openmim) (2.2.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->openmim) (2.14.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->openmim) (0.1.2)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->openmim) (1.16.0)\n", "Installing collected packages: ordered-set, colorama, model-index, openmim\n", "Successfully installed colorama-0.4.6 model-index-0.1.11 openmim-0.3.7 ordered-set-4.1.0\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Looking in links: https://download.openmmlab.com/mmcv/dist/cu118/torch2.0.0/index.html\n", "Collecting mmengine\n", " Downloading mmengine-0.7.4-py3-none-any.whl (374 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m374.3/374.3 kB\u001b[0m \u001b[31m17.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting addict (from mmengine)\n", " Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)\n", "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from mmengine) (3.7.1)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from mmengine) (1.22.4)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from mmengine) (6.0)\n", "Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from mmengine) (13.3.4)\n", "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from mmengine) (2.3.0)\n", "Collecting yapf (from mmengine)\n", " Downloading yapf-0.40.0-py3-none-any.whl (250 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m250.3/250.3 kB\u001b[0m \u001b[31m28.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: opencv-python>=3 in /usr/local/lib/python3.10/dist-packages (from mmengine) (4.7.0.72)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (1.0.7)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (0.11.0)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (4.39.3)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (1.4.4)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (23.1)\n", "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (8.4.0)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (3.0.9)\n", "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine) (2.8.2)\n", "Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine) (2.2.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine) (2.14.0)\n", "Collecting importlib-metadata>=6.6.0 (from yapf->mmengine)\n", " Downloading importlib_metadata-6.6.0-py3-none-any.whl (22 kB)\n", "Collecting platformdirs>=3.5.1 (from yapf->mmengine)\n", " Downloading platformdirs-3.5.3-py3-none-any.whl (15 kB)\n", "Requirement already satisfied: tomli>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from yapf->mmengine) (2.0.1)\n", "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata>=6.6.0->yapf->mmengine) (3.15.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->mmengine) (0.1.2)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->mmengine) (1.16.0)\n", "Installing collected packages: addict, platformdirs, importlib-metadata, yapf, mmengine\n", " Attempting uninstall: platformdirs\n", " Found existing installation: platformdirs 3.3.0\n", " Uninstalling platformdirs-3.3.0:\n", " Successfully uninstalled platformdirs-3.3.0\n", "Successfully installed addict-2.4.0 importlib-metadata-6.6.0 mmengine-0.7.4 platformdirs-3.5.3 yapf-0.40.0\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Looking in links: https://download.openmmlab.com/mmcv/dist/cu118/torch2.0.0/index.html\n", "Collecting mmcv\n", " Downloading https://download.openmmlab.com/mmcv/dist/cu118/torch2.0.0/mmcv-2.0.0-cp310-cp310-manylinux1_x86_64.whl (74.4 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.4/74.4 MB\u001b[0m \u001b[31m8.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: addict in /usr/local/lib/python3.10/dist-packages (from mmcv) (2.4.0)\n", "Requirement already satisfied: mmengine>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from mmcv) (0.7.4)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from mmcv) (1.22.4)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from mmcv) (23.1)\n", "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from mmcv) (8.4.0)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from mmcv) (6.0)\n", "Requirement already satisfied: yapf in /usr/local/lib/python3.10/dist-packages (from mmcv) (0.40.0)\n", "Requirement already satisfied: opencv-python>=3 in /usr/local/lib/python3.10/dist-packages (from mmcv) (4.7.0.72)\n", "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from mmengine>=0.2.0->mmcv) (3.7.1)\n", "Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from mmengine>=0.2.0->mmcv) (13.3.4)\n", "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from mmengine>=0.2.0->mmcv) (2.3.0)\n", "Requirement already satisfied: importlib-metadata>=6.6.0 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv) (6.6.0)\n", "Requirement already satisfied: platformdirs>=3.5.1 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv) (3.5.3)\n", "Requirement already satisfied: tomli>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv) (2.0.1)\n", "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata>=6.6.0->yapf->mmcv) (3.15.0)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (1.0.7)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (0.11.0)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (4.39.3)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (1.4.4)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (3.0.9)\n", "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmengine>=0.2.0->mmcv) (2.8.2)\n", "Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine>=0.2.0->mmcv) (2.2.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine>=0.2.0->mmcv) (2.14.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->mmengine>=0.2.0->mmcv) (0.1.2)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->mmengine>=0.2.0->mmcv) (1.16.0)\n", "Installing collected packages: mmcv\n", "Successfully installed mmcv-2.0.0\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Looking in links: https://download.openmmlab.com/mmcv/dist/cu118/torch2.0.0/index.html\n", "Collecting mmdet\n", " Downloading mmdet-3.0.0-py3-none-any.whl (1.7 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from mmdet) (3.7.1)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from mmdet) (1.22.4)\n", "Requirement already satisfied: pycocotools in /usr/local/lib/python3.10/dist-packages (from mmdet) (2.0.6)\n", "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from mmdet) (1.10.1)\n", "Requirement already satisfied: shapely in /usr/local/lib/python3.10/dist-packages (from mmdet) (2.0.1)\n", "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from mmdet) (1.16.0)\n", "Collecting terminaltables (from mmdet)\n", " Downloading terminaltables-3.1.10-py2.py3-none-any.whl (15 kB)\n", "Requirement already satisfied: mmcv<2.1.0,>=2.0.0rc4 in /usr/local/lib/python3.10/dist-packages (from mmdet) (2.0.0)\n", "Requirement already satisfied: mmengine<1.0.0,>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from mmdet) (0.7.4)\n", "Requirement already satisfied: addict in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (2.4.0)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (23.1)\n", "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (8.4.0)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (6.0)\n", "Requirement already satisfied: yapf in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (0.40.0)\n", "Requirement already satisfied: opencv-python>=3 in /usr/local/lib/python3.10/dist-packages (from mmcv<2.1.0,>=2.0.0rc4->mmdet) (4.7.0.72)\n", "Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from mmengine<1.0.0,>=0.7.1->mmdet) (13.3.4)\n", "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from mmengine<1.0.0,>=0.7.1->mmdet) (2.3.0)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (1.0.7)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (0.11.0)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (4.39.3)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (1.4.4)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (3.0.9)\n", "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmdet) (2.8.2)\n", "Requirement already satisfied: markdown-it-py<3.0.0,>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine<1.0.0,>=0.7.1->mmdet) (2.2.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->mmengine<1.0.0,>=0.7.1->mmdet) (2.14.0)\n", "Requirement already satisfied: importlib-metadata>=6.6.0 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv<2.1.0,>=2.0.0rc4->mmdet) (6.6.0)\n", "Requirement already satisfied: platformdirs>=3.5.1 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv<2.1.0,>=2.0.0rc4->mmdet) (3.5.3)\n", "Requirement already satisfied: tomli>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from yapf->mmcv<2.1.0,>=2.0.0rc4->mmdet) (2.0.1)\n", "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata>=6.6.0->yapf->mmcv<2.1.0,>=2.0.0rc4->mmdet) (3.15.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py<3.0.0,>=2.2.0->rich->mmengine<1.0.0,>=0.7.1->mmdet) (0.1.2)\n", "Installing collected packages: terminaltables, mmdet\n", "Successfully installed mmdet-3.0.0 terminaltables-3.1.10\n", "Cloning into 'mmaction2'...\n", "remote: Enumerating objects: 22869, done.\u001b[K\n", "remote: Counting objects: 100% (1491/1491), done.\u001b[K\n", "remote: Compressing objects: 100% (800/800), done.\u001b[K\n", "remote: Total 22869 (delta 855), reused 1176 (delta 686), pack-reused 21378\u001b[K\n", "Receiving objects: 100% (22869/22869), 82.81 MiB | 15.42 MiB/s, done.\n", "Resolving deltas: 100% (15954/15954), done.\n", "/content/mmaction2\n", "Using pip 23.1.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Obtaining file:///content/mmaction2\n", " Running command python setup.py egg_info\n", " running egg_info\n", " creating /tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info\n", " writing /tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/PKG-INFO\n", " writing dependency_links to /tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/dependency_links.txt\n", " writing requirements to /tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/requires.txt\n", " writing top-level names to /tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/top_level.txt\n", " writing manifest file '/tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/SOURCES.txt'\n", " reading manifest file '/tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/SOURCES.txt'\n", " reading manifest template 'MANIFEST.in'\n", " warning: no files found matching 'mmaction/.mim/model-index.yml'\n", " warning: no files found matching '*.py' under directory 'mmaction/.mim/configs'\n", " warning: no files found matching '*.yml' under directory 'mmaction/.mim/configs'\n", " warning: no files found matching '*.sh' under directory 'mmaction/.mim/tools'\n", " warning: no files found matching '*.py' under directory 'mmaction/.mim/tools'\n", " adding license file 'LICENSE'\n", " writing manifest file '/tmp/pip-pip-egg-info-d1y_zlo6/mmaction2.egg-info/SOURCES.txt'\n", " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Collecting decord>=0.4.1 (from mmaction2==1.0.0)\n", " Downloading decord-0.6.0-py3-none-manylinux2010_x86_64.whl (13.6 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.6/13.6 MB\u001b[0m \u001b[31m71.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting einops (from mmaction2==1.0.0)\n", " Downloading einops-0.6.1-py3-none-any.whl (42 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (3.7.1)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (1.22.4)\n", "Requirement already satisfied: opencv-contrib-python in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (4.7.0.72)\n", "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (8.4.0)\n", "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (1.10.1)\n", "Requirement already satisfied: torch>=1.3 in /usr/local/lib/python3.10/dist-packages (from mmaction2==1.0.0) (2.0.1+cu118)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (3.12.0)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (4.5.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (1.11.1)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (3.1)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (3.1.2)\n", "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.3->mmaction2==1.0.0) (2.0.0)\n", "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.3->mmaction2==1.0.0) (3.25.2)\n", "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.3->mmaction2==1.0.0) (16.0.5)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (1.0.7)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (0.11.0)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (4.39.3)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (1.4.4)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (23.1)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (3.0.9)\n", "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->mmaction2==1.0.0) (2.8.2)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->mmaction2==1.0.0) (1.16.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.3->mmaction2==1.0.0) (2.1.2)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.3->mmaction2==1.0.0) (1.3.0)\n", "Installing collected packages: einops, decord, mmaction2\n", " Running setup.py develop for mmaction2\n", " Running command python setup.py develop\n", " running develop\n", " /usr/local/lib/python3.10/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.\n", " !!\n", "\n", " ********************************************************************************\n", " Please avoid running ``setup.py`` and ``easy_install``.\n", " Instead, use pypa/build, pypa/installer, pypa/build or\n", " other standards-based tools.\n", "\n", " See https://github.com/pypa/setuptools/issues/917 for details.\n", " ********************************************************************************\n", "\n", " !!\n", " easy_install.initialize_options(self)\n", " /usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.\n", " !!\n", "\n", " ********************************************************************************\n", " Please avoid running ``setup.py`` directly.\n", " Instead, use pypa/build, pypa/installer, pypa/build or\n", " other standards-based tools.\n", "\n", " See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.\n", " ********************************************************************************\n", "\n", " !!\n", " self.initialize_options()\n", " running egg_info\n", " creating mmaction2.egg-info\n", " writing mmaction2.egg-info/PKG-INFO\n", " writing dependency_links to mmaction2.egg-info/dependency_links.txt\n", " writing requirements to mmaction2.egg-info/requires.txt\n", " writing top-level names to mmaction2.egg-info/top_level.txt\n", " writing manifest file 'mmaction2.egg-info/SOURCES.txt'\n", " reading manifest file 'mmaction2.egg-info/SOURCES.txt'\n", " reading manifest template 'MANIFEST.in'\n", " adding license file 'LICENSE'\n", " writing manifest file 'mmaction2.egg-info/SOURCES.txt'\n", " running build_ext\n", " Creating /usr/local/lib/python3.10/dist-packages/mmaction2.egg-link (link to .)\n", " Adding mmaction2 1.0.0 to easy-install.pth file\n", "\n", " Installed /content/mmaction2\n", "Successfully installed decord-0.6.0 einops-0.6.1 mmaction2-1.0.0\n", "/content/mmaction2/projects/stad_tutorial\n" ] } ], "source": [ "%pip install -U openmim\n", "!mim install mmengine\n", "!mim install mmcv\n", "!mim install mmdet\n", "\n", "!git clone https://github.com/open-mmlab/mmaction2.git\n", "\n", "%cd mmaction2\n", "%pip install -v -e .\n", "%cd projects/stad_tutorial" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "Ox0TM64FooZt" }, "source": [ "## 1. Prepare spatio-temporal action detection dataset\n", "\n", "Similar to detection tasks that require bounding box annotations, spatio-temporal action detection tasks require temporal and spatial localization, so more complex tube annotations are required. Taking the MultiSports dataset as an example, the `gttubes` field provides all the target action annotations in the video, and the following is an annotation fragment:\n", "\n", "```\n", " 'gttubes': {\n", " 'aerobic_gymnastics/v_aqMgwPExjD0_c001': # video_key\n", " {\n", " 10: # label index\n", " [\n", " array([[ 377., 904., 316., 1016., 584.], # 1st tube of class 10\n", " [ 378., 882., 315., 1016., 579.], # shape (n, 5): n frames，each annotation includes (frame idx，x1，y1, x2, y2)\n", " ...\n", " [ 398., 861., 304., 954., 549.]], dtype=float32)，\n", "\n", " array([[ 399., 881., 308., 955., 542.], # 2nd tube of class 10\n", " [ 400., 862., 303., 988., 539.],\n", " [ 401., 853., 292., 1000., 535.],\n", " ...])\n", " ...\n", "\n", " ] ,\n", " 9: # label index\n", " [\n", " array(...), # 1st tube of class 9\n", " array(...), # 2nd tube of class 9\n", " ...\n", " ]\n", " ...\n", " }\n", " }\n", "```\n", "\n", "The annotation file also needs to provide other field information, and the complete ground truth file includes the following information:\n", "\n", "```\n", "{\n", " 'labels': # label list\n", " ['aerobic push up', 'aerobic explosive push up', ...],\n", " 'train_videos': # training video list\n", " [\n", " [\n", " 'aerobic_gymnastics/v_aqMgwPExjD0_c001',\n", " 'aerobic_gymnastics/v_yaKOumdXwbU_c019',\n", " ...\n", " ]\n", " ]\n", " 'test_videos': # test video list\n", " [\n", " [\n", " 'aerobic_gymnastics/v_crsi07chcV8_c004',\n", " 'aerobic_gymnastics/v_dFYr67eNMwA_c005',\n", " ...\n", " ]\n", " ]\n", " 'n_frames': # dict provides frame number of each video\n", " {\n", " 'aerobic_gymnastics/v_crsi07chcV8_c004': 725,\n", " 'aerobic_gymnastics/v_dFYr67eNMwA_c005': 750,\n", " ...\n", " }\n", " 'resolution': # dict provides resolution of each video\n", " {\n", " 'aerobic_gymnastics/v_crsi07chcV8_c004': (720, 1280),\n", " 'aerobic_gymnastics/v_dFYr67eNMwA_c005': (720, 1280),\n", " ...\n", " }\n", " 'gt_tubes': # dict provides bouding boxes of each tube\n", " {\n", " ... # refer to above description\n", " }\n", "}\n", "```\n", "\n", "The subsequent experiments are based on MultiSports-tiny, we extracted a small number of videos from MultiSports for demonstration purposes." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "n5AzsRvdooZv", "outputId": "a6cad83b-4613-43cc-8c09-86ac79242656" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--2023-06-15 06:00:15-- https://download.openmmlab.com/mmaction/v1.0/projects/stad_tutorial/multisports-tiny.tar\n", "Resolving download.openmmlab.com (download.openmmlab.com)... 163.181.82.215, 163.181.82.216, 163.181.82.218, ...\n", "Connecting to download.openmmlab.com (download.openmmlab.com)|163.181.82.215|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 82780160 (79M) [application/x-tar]\n", "Saving to: ‘data/multisports-tiny.tar’\n", "\n", "multisports-tiny.ta 100%[===================>] 78.95M 13.3MB/s in 44s \n", "\n", "2023-06-15 06:01:00 (1.78 MB/s) - ‘data/multisports-tiny.tar’ saved [82780160/82780160]\n", "\n", "multisports-tiny/multisports/\n", "multisports-tiny/multisports/test/\n", "multisports-tiny/multisports/test/aerobic_gymnastics/\n", "multisports-tiny/multisports/test/aerobic_gymnastics/v_7G_IpU0FxLU_c001.mp4\n", "multisports-tiny/multisports/annotations/\n", "multisports-tiny/multisports/annotations/multisports_GT.pkl\n", "multisports-tiny/multisports/trainval/\n", "multisports-tiny/multisports/trainval/aerobic_gymnastics/\n", "multisports-tiny/multisports/trainval/aerobic_gymnastics/v__wAgwttPYaQ_c001.mp4\n", "multisports-tiny/multisports/trainval/aerobic_gymnastics/v__wAgwttPYaQ_c003.mp4\n", "multisports-tiny/multisports/trainval/aerobic_gymnastics/v__wAgwttPYaQ_c002.mp4\n", "Reading package lists...\n", "Building dependency tree...\n", "Reading state information...\n", "The following NEW packages will be installed:\n", " tree\n", "0 upgraded, 1 newly installed, 0 to remove and 46 not upgraded.\n", "Need to get 43.0 kB of archives.\n", "After this operation, 115 kB of additional disk space will be used.\n", "Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 tree amd64 1.8.0-1 [43.0 kB]\n", "Fetched 43.0 kB in 1s (43.0 kB/s)\n", "Selecting previously unselected package tree.\n", "(Reading database ... 122541 files and directories currently installed.)\n", "Preparing to unpack .../tree_1.8.0-1_amd64.deb ...\n", "Unpacking tree (1.8.0-1) ...\n", "Setting up tree (1.8.0-1) ...\n", "Processing triggers for man-db (2.9.1-1) ...\n", "\u001b[01;34mdata\u001b[00m\n", "├── \u001b[01;34mmultisports\u001b[00m\n", "│   ├── \u001b[01;34mannotations\u001b[00m\n", "│   │   └── \u001b[01;32mmultisports_GT.pkl\u001b[00m\n", "│   ├── \u001b[01;34mtest\u001b[00m\n", "│   │   └── \u001b[01;34maerobic_gymnastics\u001b[00m\n", "│   │   └── \u001b[01;32mv_7G_IpU0FxLU_c001.mp4\u001b[00m\n", "│   └── \u001b[01;34mtrainval\u001b[00m\n", "│   └── \u001b[01;34maerobic_gymnastics\u001b[00m\n", "│   ├── \u001b[01;32mv__wAgwttPYaQ_c001.mp4\u001b[00m\n", "│   ├── \u001b[01;32mv__wAgwttPYaQ_c002.mp4\u001b[00m\n", "│   └── \u001b[01;32mv__wAgwttPYaQ_c003.mp4\u001b[00m\n", "└── \u001b[01;31mmultisports-tiny.tar\u001b[00m\n", "\n", "6 directories, 6 files\n" ] } ], "source": [ "# Download dataset\n", "!wget -P data -c https://download.openmmlab.com/mmaction/v1.0/projects/stad_tutorial/multisports-tiny.tar\n", "!tar -xvf data/multisports-tiny.tar --strip 1 -C data\n", "!apt-get -q install tree\n", "!tree data" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "_u69LHscooZw" }, "source": [ "## 2. Train detection model\n", "\n", "In the SlowOnly + Det paradigm, we need to train a human detector first, and then predict actions based on the detection results. In this section, we train a detection model based on the annotation format in the previous section and the MMDetection algorithm library.\n", "\n", "### 2.1 Build detection dataset annotation (COCO format)\n", "\n", "Based on the annotation information of the spatio-temporal action detection dataset, we can build a COCO format detection dataset for training the detection model. We provide a script to convert the MultiSports format annotation, if you need to convert from other formats, you can refer to the [custom dataset](https://mmdetection.readthedocs.io/zh_CN/latest/advanced_guides/customize_dataset.html) document provided by MMDetection." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e8fu9VtRooZw", "outputId": "3e7a7053-a08d-4c32-9d66-a362b3de164d" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[01;34mdata/multisports/annotations\u001b[00m\n", "├── multisports_det_anno_train.json\n", "├── multisports_det_anno_val.json\n", "└── \u001b[01;32mmultisports_GT.pkl\u001b[00m\n", "\n", "0 directories, 3 files\n" ] } ], "source": [ "!python tools/generate_mmdet_anno.py data/multisports/annotations/multisports_GT.pkl data/multisports/annotations/multisports_det_anno.json\n", "!tree data/multisports/annotations" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "HJAb8EwwooZx", "outputId": "1c82387c-c731-484c-a4cc-8c255b3f2e62" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Will generate 3 rgb dir for aerobic_gymnastics.\n", "Generate v__wAgwttPYaQ_c003 rgb dir successfully.\n", "Generate v__wAgwttPYaQ_c002 rgb dir successfully.\n", "Generate v__wAgwttPYaQ_c001 rgb dir successfully.\n" ] } ], "source": [ "!python tools/generate_rgb.py" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "9xIOk_XkooZx" }, "source": [ "### 2.2 Modify config file\n", "\n", "We use faster-rcnn_x101-64x4d_fpn_1x_coco as the base configuration, and make the following modifications to train on the MultiSports dataset. The following parts need to be modified:\n", "- Number of model categories\n", "- Learning rate adjustment strategy\n", "- Optimizer configuration\n", "- Dataset/annotation file path\n", "- Evaluator configuration\n", "- Pre-trained model\n", "\n", "For more detailed tutorials, please refer to the [prepare configuration file](https://mmdetection.readthedocs.io/zh_CN/latest/user_guides/train.html#id9) document provided by MMDetection." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Ad1QLNM8ooZy", "outputId": "55f95e91-8fdf-40fa-dd08-5fa980444b6f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# Copyright (c) OpenMMLab. All rights reserved.\n", "_base_ = './faster-rcnn_r50-caffe_fpn_ms-1x_coco.py'\n", "model = dict(roi_head=dict(bbox_head=dict(num_classes=1)))\n", "\n", "# take 2 epochs as an example\n", "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2, val_interval=1)\n", "\n", "# learning rate\n", "param_scheduler = [\n", " dict(type='ConstantLR', factor=1.0, by_epoch=False, begin=0, end=500)\n", "]\n", "\n", "# optimizer\n", "optim_wrapper = dict(\n", " type='OptimWrapper',\n", " optimizer=dict(type='SGD', lr=0.0050, momentum=0.9, weight_decay=0.0001))\n", "\n", "dataset_type = 'CocoDataset'\n", "# modify metainfo\n", "metainfo = {\n", " 'classes': ('person', ),\n", " 'palette': [\n", " (220, 20, 60),\n", " ]\n", "}\n", "\n", "# specify metainfo, dataset path\n", "data_root = 'data/multisports/'\n", "\n", "train_dataloader = dict(\n", " dataset=dict(\n", " data_root=data_root,\n", " ann_file='annotations/multisports_det_anno_train.json',\n", " data_prefix=dict(img='rawframes/'),\n", " metainfo=metainfo))\n", "\n", "val_dataloader = dict(\n", " dataset=dict(\n", " data_root=data_root,\n", " ann_file='annotations/multisports_det_anno_val.json',\n", " data_prefix=dict(img='rawframes/'),\n", " metainfo=metainfo))\n", "\n", "test_dataloader = dict(\n", " dataset=dict(\n", " data_root=data_root,\n", " ann_file='annotations/ms_infer_anno.json',\n", " data_prefix=dict(img='rawframes/'),\n", " metainfo=metainfo))\n", "\n", "# specify annotaition file path, modify metric items\n", "val_evaluator = dict(\n", " ann_file='data/multisports/annotations/multisports_det_anno_val.json',\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5],\n", ")\n", "\n", "test_evaluator = dict(\n", " ann_file='data/multisports/annotations/ms_infer_anno.json',\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5],\n", ")\n", "\n", "# specify pretrain checkpoint\n", "load_from = 'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth' # noqa: E501\n" ] } ], "source": [ "!cat configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "W40JO80nooZ0" }, "source": [ "### 2.3 Train detection model" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "Oc1LWr4AooZ0" }, "source": [ "By using MIM, you can directly train MMDetection models in the current directory. Here is the simplest example of training on a single GPU. For more training commands, please refer to the MIM [tutorial](https://github.com/open-mmlab/mim#command)." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "QpxCbvr2ooZ0", "outputId": "ffe7b420-c359-4e5a-a1b1-3a75e923046d" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training command is /usr/bin/python3 /usr/local/lib/python3.10/dist-packages/mmdet/.mim/tools/train.py configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py --launcher none --work-dir work_dirs/det_model. \n", "06/15 06:02:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - \n", "------------------------------------------------------------\n", "System environment:\n", " sys.platform: linux\n", " Python: 3.10.12 (main, Jun 7 2023, 12:45:35) [GCC 9.4.0]\n", " CUDA available: True\n", " numpy_random_seed: 503128501\n", " GPU 0: Tesla T4\n", " CUDA_HOME: /usr/local/cuda\n", " NVCC: Cuda compilation tools, release 11.8, V11.8.89\n", " GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0\n", " PyTorch: 2.0.1+cu118\n", " PyTorch compiling details: PyTorch built with:\n", " - GCC 9.3\n", " - C++ Version: 201703\n", " - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications\n", " - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n", " - OpenMP 201511 (a.k.a. OpenMP 4.5)\n", " - LAPACK is enabled (usually provided by MKL)\n", " - NNPACK is enabled\n", " - CPU capability usage: AVX2\n", " - CUDA Runtime 11.8\n", " - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n", " - CuDNN 8.7\n", " - Magma 2.6.1\n", " - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n", "\n", " TorchVision: 0.15.2+cu118\n", " OpenCV: 4.7.0\n", " MMEngine: 0.7.4\n", "\n", "Runtime environment:\n", " cudnn_benchmark: False\n", " mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n", " dist_cfg: {'backend': 'nccl'}\n", " seed: 503128501\n", " Distributed launcher: none\n", " Distributed training: False\n", " GPU number: 1\n", "------------------------------------------------------------\n", "\n", "06/15 06:02:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Config:\n", "model = dict(\n", " type='FasterRCNN',\n", " data_preprocessor=dict(\n", " type='DetDataPreprocessor',\n", " mean=[103.53, 116.28, 123.675],\n", " std=[1.0, 1.0, 1.0],\n", " bgr_to_rgb=False,\n", " pad_size_divisor=32),\n", " backbone=dict(\n", " type='ResNet',\n", " depth=50,\n", " num_stages=4,\n", " out_indices=(0, 1, 2, 3),\n", " frozen_stages=1,\n", " norm_cfg=dict(type='BN', requires_grad=False),\n", " norm_eval=True,\n", " style='caffe',\n", " init_cfg=dict(\n", " type='Pretrained',\n", " checkpoint='open-mmlab://detectron2/resnet50_caffe')),\n", " neck=dict(\n", " type='FPN',\n", " in_channels=[256, 512, 1024, 2048],\n", " out_channels=256,\n", " num_outs=5),\n", " rpn_head=dict(\n", " type='RPNHead',\n", " in_channels=256,\n", " feat_channels=256,\n", " anchor_generator=dict(\n", " type='AnchorGenerator',\n", " scales=[8],\n", " ratios=[0.5, 1.0, 2.0],\n", " strides=[4, 8, 16, 32, 64]),\n", " bbox_coder=dict(\n", " type='DeltaXYWHBBoxCoder',\n", " target_means=[0.0, 0.0, 0.0, 0.0],\n", " target_stds=[1.0, 1.0, 1.0, 1.0]),\n", " loss_cls=dict(\n", " type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),\n", " loss_bbox=dict(type='L1Loss', loss_weight=1.0)),\n", " roi_head=dict(\n", " type='StandardRoIHead',\n", " bbox_roi_extractor=dict(\n", " type='SingleRoIExtractor',\n", " roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),\n", " out_channels=256,\n", " featmap_strides=[4, 8, 16, 32]),\n", " bbox_head=dict(\n", " type='Shared2FCBBoxHead',\n", " in_channels=256,\n", " fc_out_channels=1024,\n", " roi_feat_size=7,\n", " num_classes=1,\n", " bbox_coder=dict(\n", " type='DeltaXYWHBBoxCoder',\n", " target_means=[0.0, 0.0, 0.0, 0.0],\n", " target_stds=[0.1, 0.1, 0.2, 0.2]),\n", " reg_class_agnostic=False,\n", " loss_cls=dict(\n", " type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),\n", " loss_bbox=dict(type='L1Loss', loss_weight=1.0))),\n", " train_cfg=dict(\n", " rpn=dict(\n", " assigner=dict(\n", " type='MaxIoUAssigner',\n", " pos_iou_thr=0.7,\n", " neg_iou_thr=0.3,\n", " min_pos_iou=0.3,\n", " match_low_quality=True,\n", " ignore_iof_thr=-1),\n", " sampler=dict(\n", " type='RandomSampler',\n", " num=256,\n", " pos_fraction=0.5,\n", " neg_pos_ub=-1,\n", " add_gt_as_proposals=False),\n", " allowed_border=-1,\n", " pos_weight=-1,\n", " debug=False),\n", " rpn_proposal=dict(\n", " nms_pre=2000,\n", " max_per_img=1000,\n", " nms=dict(type='nms', iou_threshold=0.7),\n", " min_bbox_size=0),\n", " rcnn=dict(\n", " assigner=dict(\n", " type='MaxIoUAssigner',\n", " pos_iou_thr=0.5,\n", " neg_iou_thr=0.5,\n", " min_pos_iou=0.5,\n", " match_low_quality=False,\n", " ignore_iof_thr=-1),\n", " sampler=dict(\n", " type='RandomSampler',\n", " num=512,\n", " pos_fraction=0.25,\n", " neg_pos_ub=-1,\n", " add_gt_as_proposals=True),\n", " pos_weight=-1,\n", " debug=False)),\n", " test_cfg=dict(\n", " rpn=dict(\n", " nms_pre=1000,\n", " max_per_img=1000,\n", " nms=dict(type='nms', iou_threshold=0.7),\n", " min_bbox_size=0),\n", " rcnn=dict(\n", " score_thr=0.05,\n", " nms=dict(type='nms', iou_threshold=0.5),\n", " max_per_img=100)))\n", "dataset_type = 'CocoDataset'\n", "data_root = 'data/multisports/'\n", "backend_args = None\n", "train_pipeline = [\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='RandomChoiceResize',\n", " scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n", " (1333, 768), (1333, 800)],\n", " keep_ratio=True),\n", " dict(type='RandomFlip', prob=0.5),\n", " dict(type='PackDetInputs')\n", "]\n", "test_pipeline = [\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", "]\n", "train_dataloader = dict(\n", " batch_size=2,\n", " num_workers=2,\n", " persistent_workers=True,\n", " sampler=dict(type='DefaultSampler', shuffle=True),\n", " batch_sampler=dict(type='AspectRatioBatchSampler'),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/multisports_det_anno_train.json',\n", " data_prefix=dict(img='rawframes/'),\n", " filter_cfg=dict(filter_empty_gt=True, min_size=32),\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='RandomChoiceResize',\n", " scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n", " (1333, 768), (1333, 800)],\n", " keep_ratio=True),\n", " dict(type='RandomFlip', prob=0.5),\n", " dict(type='PackDetInputs')\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "val_dataloader = dict(\n", " batch_size=1,\n", " num_workers=2,\n", " persistent_workers=True,\n", " drop_last=False,\n", " sampler=dict(type='DefaultSampler', shuffle=False),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/multisports_det_anno_val.json',\n", " data_prefix=dict(img='rawframes/'),\n", " test_mode=True,\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "test_dataloader = dict(\n", " batch_size=1,\n", " num_workers=2,\n", " persistent_workers=True,\n", " drop_last=False,\n", " sampler=dict(type='DefaultSampler', shuffle=False),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/ms_infer_anno.json',\n", " data_prefix=dict(img='rawframes/'),\n", " test_mode=True,\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "val_evaluator = dict(\n", " type='CocoMetric',\n", " ann_file='data/multisports/annotations/multisports_det_anno_val.json',\n", " metric='bbox',\n", " format_only=False,\n", " backend_args=None,\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5])\n", "test_evaluator = dict(\n", " type='CocoMetric',\n", " ann_file='data/multisports/annotations/ms_infer_anno.json',\n", " metric='bbox',\n", " format_only=False,\n", " backend_args=None,\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5])\n", "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2, val_interval=1)\n", "val_cfg = dict(type='ValLoop')\n", "test_cfg = dict(type='TestLoop')\n", "param_scheduler = [\n", " dict(type='ConstantLR', factor=1.0, by_epoch=False, begin=0, end=500)\n", "]\n", "optim_wrapper = dict(\n", " type='OptimWrapper',\n", " optimizer=dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001))\n", "auto_scale_lr = dict(enable=False, base_batch_size=16)\n", "default_scope = 'mmdet'\n", "default_hooks = dict(\n", " timer=dict(type='IterTimerHook'),\n", " logger=dict(type='LoggerHook', interval=50),\n", " param_scheduler=dict(type='ParamSchedulerHook'),\n", " checkpoint=dict(type='CheckpointHook', interval=1),\n", " sampler_seed=dict(type='DistSamplerSeedHook'),\n", " visualization=dict(type='DetVisualizationHook'))\n", "env_cfg = dict(\n", " cudnn_benchmark=False,\n", " mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n", " dist_cfg=dict(backend='nccl'))\n", "vis_backends = [dict(type='LocalVisBackend')]\n", "visualizer = dict(\n", " type='DetLocalVisualizer',\n", " vis_backends=[dict(type='LocalVisBackend')],\n", " name='visualizer')\n", "log_processor = dict(type='LogProcessor', window_size=50, by_epoch=True)\n", "log_level = 'INFO'\n", "load_from = 'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth'\n", "resume = False\n", "metainfo = dict(classes=('person', ), palette=[(220, 20, 60)])\n", "launcher = 'none'\n", "work_dir = 'work_dirs/det_model'\n", "\n", "06/15 06:02:17 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n", "06/15 06:02:17 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Hooks will be executed in the following order:\n", "before_run:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "before_train:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_train_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DistSamplerSeedHook \n", " -------------------- \n", "before_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train_epoch:\n", "(NORMAL ) IterTimerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_val_epoch:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "before_val_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_val_iter:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DetVisualizationHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_val_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train:\n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_test_epoch:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "before_test_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_test_iter:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DetVisualizationHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_test_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_run:\n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "loading annotations into memory...\n", "Done (t=0.01s)\n", "creating index...\n", "index created!\n", "loading annotations into memory...\n", "Done (t=0.00s)\n", "creating index...\n", "index created!\n", "loading annotations into memory...\n", "Done (t=0.00s)\n", "creating index...\n", "index created!\n", "06/15 06:02:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - load model from: open-mmlab://detectron2/resnet50_caffe\n", "06/15 06:02:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Loads checkpoint by openmmlab backend from path: open-mmlab://detectron2/resnet50_caffe\n", "Downloading: \"https://download.openmmlab.com/pretrain/third_party/resnet50_msra-5891d200.pth\" to /root/.cache/torch/hub/checkpoints/resnet50_msra-5891d200.pth\n", "100% 89.9M/89.9M [00:02<00:00, 34.8MB/s]\n", "06/15 06:02:21 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - The model and loaded state dict do not match exactly\n", "\n", "unexpected key in source state_dict: conv1.bias\n", "\n", "Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth\n", "Downloading: \"https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth\" to /root/.cache/torch/hub/checkpoints/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth\n", "100% 158M/158M [00:04<00:00, 37.4MB/s]\n", "06/15 06:02:26 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Load checkpoint from https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth\n", "06/15 06:02:26 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - \"FileClient\" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io\n", "06/15 06:02:26 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - \"HardDiskBackend\" is the alias of \"LocalBackend\" and the former will be deprecated in future.\n", "06/15 06:02:26 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Checkpoints will be saved to /content/mmaction2/projects/stad_tutorial/work_dirs/det_model.\n", "06/15 06:02:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][ 50/118] lr: 5.0000e-03 eta: 0:01:56 time: 0.6273 data_time: 0.0111 memory: 3414 loss: 0.5456 loss_rpn_cls: 0.0070 loss_rpn_bbox: 0.0167 loss_cls: 0.1887 acc: 93.2617 loss_bbox: 0.3332\n", "06/15 06:03:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][100/118] lr: 5.0000e-03 eta: 0:01:16 time: 0.5041 data_time: 0.0078 memory: 3414 loss: 0.4017 loss_rpn_cls: 0.0027 loss_rpn_bbox: 0.0130 loss_cls: 0.1313 acc: 94.8242 loss_bbox: 0.2547\n", "06/15 06:03:31 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person_20230615_060208\n", "06/15 06:03:31 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 1 epochs\n", "06/15 06:03:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][ 50/120] eta: 0:00:08 time: 0.1196 data_time: 0.0059 memory: 3414 \n", "06/15 06:03:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][100/120] eta: 0:00:02 time: 0.1234 data_time: 0.0082 memory: 679 \n", "06/15 06:03:47 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Evaluating bbox...\n", "Loading and preparing results...\n", "DONE (t=0.00s)\n", "creating index...\n", "index created!\n", "Running per image evaluation...\n", "Evaluate annotation type *bbox*\n", "DONE (t=0.05s).\n", "Accumulating evaluation results...\n", "DONE (t=0.01s).\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.872\n", " Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = 0.709\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = 0.886\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.964\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=300 ] = 0.964\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=1000 ] = 0.964\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = 1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = 0.963\n", "06/15 06:03:47 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - bbox_mAP_copypaste: 0.872 -1.000 -1.000 -1.000 0.709 0.886\n", "06/15 06:03:47 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][120/120] coco/bbox_mAP_50: -1.0000 coco/bbox_AR@100: 0.9640 data_time: 0.0067 time: 0.1212\n", "06/15 06:04:14 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][ 50/118] lr: 5.0000e-03 eta: 0:00:37 time: 0.5316 data_time: 0.0094 memory: 3414 loss: 0.3385 loss_rpn_cls: 0.0012 loss_rpn_bbox: 0.0111 loss_cls: 0.1119 acc: 95.4102 loss_bbox: 0.2143\n", "06/15 06:04:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][100/118] lr: 5.0000e-03 eta: 0:00:09 time: 0.5152 data_time: 0.0078 memory: 3414 loss: 0.3152 loss_rpn_cls: 0.0017 loss_rpn_bbox: 0.0109 loss_cls: 0.1050 acc: 94.7266 loss_bbox: 0.1977\n", "06/15 06:04:49 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person_20230615_060208\n", "06/15 06:04:49 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 2 epochs\n", "06/15 06:04:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][ 50/120] eta: 0:00:08 time: 0.1237 data_time: 0.0080 memory: 3414 \n", "06/15 06:05:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][100/120] eta: 0:00:02 time: 0.1202 data_time: 0.0062 memory: 679 \n", "06/15 06:05:06 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Evaluating bbox...\n", "Loading and preparing results...\n", "DONE (t=0.00s)\n", "creating index...\n", "index created!\n", "Running per image evaluation...\n", "Evaluate annotation type *bbox*\n", "DONE (t=0.04s).\n", "Accumulating evaluation results...\n", "DONE (t=0.01s).\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.907\n", " Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = 0.762\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = 0.910\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.960\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=300 ] = 0.960\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=1000 ] = 0.960\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = 1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = 0.960\n", "06/15 06:05:06 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - bbox_mAP_copypaste: 0.907 -1.000 -1.000 -1.000 0.762 0.910\n", "06/15 06:05:06 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][120/120] coco/bbox_mAP_50: -1.0000 coco/bbox_AR@100: 0.9600 data_time: 0.0066 time: 0.1214\n", "\u001b[32mTraining finished successfully. \u001b[0m\n" ] } ], "source": [ "!mim train mmdet configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py \\\n", " --work-dir work_dirs/det_model" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "IxlO927KooZ1" }, "source": [ "### 2.4 Generating Proposal BBoxes\n", "\n", "During the training of the spatiotemporal action detection model, we need to rely on proposals generated by the detection model, rather than annotated detection boxes. Therefore, we need to use a trained detection model to perform inference on the entire dataset and convert the resulting proposals into the required format for subsequent training.\n", "\n", "#### 2.4.1 Converting the Dataset to Coco Format\n", "\n", "We provide a script to convert the MultiSports dataset into an annotation format without ground truth, which is used for inference." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e6C7D2DSooZ1", "outputId": "878015d1-0fc7-4eb6-af77-4f61aefcf2b2" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[>>] 2350/2350, 2053.0 task/s, elapsed: 1s, ETA: 0s\n", "save json file: data/multisports/rawframes/../annotations/ms_infer_anno.json\n" ] } ], "source": [ "!echo 'person' > data/multisports/annotations/label_map.txt\n", "!python tools/images2coco.py \\\n", " data/multisports/rawframes \\\n", " data/multisports/annotations/label_map.txt \\\n", " ms_infer_anno.json" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "fGL3t4MEooZ1" }, "source": [ "#### 2.4.2 Inference for Generating Proposal Files\n" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "gerYk6q6ooZ1" }, "source": [ "The inference of MMDetection models is also based on MIM. For more testing commands, please refer to the MIM [tutorial](GitHub - open-mmlab/mim: MIM Installs OpenMMLab Packages).\n", "\n", "After the inference is completed, the results will be saved in 'data/multisports/ms_proposals.pkl'." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "lutiaqzpooZ1", "outputId": "b05db6e8-04de-4e1e-8d99-32f4c952d633" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Testing command is /usr/bin/python3 /usr/local/lib/python3.10/dist-packages/mmdet/.mim/tools/test.py configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py work_dirs/det_model/epoch_2.pth --launcher none --out data/multisports/annotations/ms_det_proposals.pkl. \n", "06/15 06:05:16 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - \n", "------------------------------------------------------------\n", "System environment:\n", " sys.platform: linux\n", " Python: 3.10.12 (main, Jun 7 2023, 12:45:35) [GCC 9.4.0]\n", " CUDA available: True\n", " numpy_random_seed: 1289054678\n", " GPU 0: Tesla T4\n", " CUDA_HOME: /usr/local/cuda\n", " NVCC: Cuda compilation tools, release 11.8, V11.8.89\n", " GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0\n", " PyTorch: 2.0.1+cu118\n", " PyTorch compiling details: PyTorch built with:\n", " - GCC 9.3\n", " - C++ Version: 201703\n", " - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications\n", " - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n", " - OpenMP 201511 (a.k.a. OpenMP 4.5)\n", " - LAPACK is enabled (usually provided by MKL)\n", " - NNPACK is enabled\n", " - CPU capability usage: AVX2\n", " - CUDA Runtime 11.8\n", " - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n", " - CuDNN 8.7\n", " - Magma 2.6.1\n", " - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n", "\n", " TorchVision: 0.15.2+cu118\n", " OpenCV: 4.7.0\n", " MMEngine: 0.7.4\n", "\n", "Runtime environment:\n", " cudnn_benchmark: False\n", " mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n", " dist_cfg: {'backend': 'nccl'}\n", " seed: 1289054678\n", " Distributed launcher: none\n", " Distributed training: False\n", " GPU number: 1\n", "------------------------------------------------------------\n", "\n", "06/15 06:05:16 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Config:\n", "model = dict(\n", " type='FasterRCNN',\n", " data_preprocessor=dict(\n", " type='DetDataPreprocessor',\n", " mean=[103.53, 116.28, 123.675],\n", " std=[1.0, 1.0, 1.0],\n", " bgr_to_rgb=False,\n", " pad_size_divisor=32),\n", " backbone=dict(\n", " type='ResNet',\n", " depth=50,\n", " num_stages=4,\n", " out_indices=(0, 1, 2, 3),\n", " frozen_stages=1,\n", " norm_cfg=dict(type='BN', requires_grad=False),\n", " norm_eval=True,\n", " style='caffe',\n", " init_cfg=dict(\n", " type='Pretrained',\n", " checkpoint='open-mmlab://detectron2/resnet50_caffe')),\n", " neck=dict(\n", " type='FPN',\n", " in_channels=[256, 512, 1024, 2048],\n", " out_channels=256,\n", " num_outs=5),\n", " rpn_head=dict(\n", " type='RPNHead',\n", " in_channels=256,\n", " feat_channels=256,\n", " anchor_generator=dict(\n", " type='AnchorGenerator',\n", " scales=[8],\n", " ratios=[0.5, 1.0, 2.0],\n", " strides=[4, 8, 16, 32, 64]),\n", " bbox_coder=dict(\n", " type='DeltaXYWHBBoxCoder',\n", " target_means=[0.0, 0.0, 0.0, 0.0],\n", " target_stds=[1.0, 1.0, 1.0, 1.0]),\n", " loss_cls=dict(\n", " type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),\n", " loss_bbox=dict(type='L1Loss', loss_weight=1.0)),\n", " roi_head=dict(\n", " type='StandardRoIHead',\n", " bbox_roi_extractor=dict(\n", " type='SingleRoIExtractor',\n", " roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),\n", " out_channels=256,\n", " featmap_strides=[4, 8, 16, 32]),\n", " bbox_head=dict(\n", " type='Shared2FCBBoxHead',\n", " in_channels=256,\n", " fc_out_channels=1024,\n", " roi_feat_size=7,\n", " num_classes=1,\n", " bbox_coder=dict(\n", " type='DeltaXYWHBBoxCoder',\n", " target_means=[0.0, 0.0, 0.0, 0.0],\n", " target_stds=[0.1, 0.1, 0.2, 0.2]),\n", " reg_class_agnostic=False,\n", " loss_cls=dict(\n", " type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),\n", " loss_bbox=dict(type='L1Loss', loss_weight=1.0))),\n", " train_cfg=dict(\n", " rpn=dict(\n", " assigner=dict(\n", " type='MaxIoUAssigner',\n", " pos_iou_thr=0.7,\n", " neg_iou_thr=0.3,\n", " min_pos_iou=0.3,\n", " match_low_quality=True,\n", " ignore_iof_thr=-1),\n", " sampler=dict(\n", " type='RandomSampler',\n", " num=256,\n", " pos_fraction=0.5,\n", " neg_pos_ub=-1,\n", " add_gt_as_proposals=False),\n", " allowed_border=-1,\n", " pos_weight=-1,\n", " debug=False),\n", " rpn_proposal=dict(\n", " nms_pre=2000,\n", " max_per_img=1000,\n", " nms=dict(type='nms', iou_threshold=0.7),\n", " min_bbox_size=0),\n", " rcnn=dict(\n", " assigner=dict(\n", " type='MaxIoUAssigner',\n", " pos_iou_thr=0.5,\n", " neg_iou_thr=0.5,\n", " min_pos_iou=0.5,\n", " match_low_quality=False,\n", " ignore_iof_thr=-1),\n", " sampler=dict(\n", " type='RandomSampler',\n", " num=512,\n", " pos_fraction=0.25,\n", " neg_pos_ub=-1,\n", " add_gt_as_proposals=True),\n", " pos_weight=-1,\n", " debug=False)),\n", " test_cfg=dict(\n", " rpn=dict(\n", " nms_pre=1000,\n", " max_per_img=1000,\n", " nms=dict(type='nms', iou_threshold=0.7),\n", " min_bbox_size=0),\n", " rcnn=dict(\n", " score_thr=0.05,\n", " nms=dict(type='nms', iou_threshold=0.5),\n", " max_per_img=100)))\n", "dataset_type = 'CocoDataset'\n", "data_root = 'data/multisports/'\n", "backend_args = None\n", "train_pipeline = [\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='RandomChoiceResize',\n", " scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n", " (1333, 768), (1333, 800)],\n", " keep_ratio=True),\n", " dict(type='RandomFlip', prob=0.5),\n", " dict(type='PackDetInputs')\n", "]\n", "test_pipeline = [\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", "]\n", "train_dataloader = dict(\n", " batch_size=2,\n", " num_workers=2,\n", " persistent_workers=True,\n", " sampler=dict(type='DefaultSampler', shuffle=True),\n", " batch_sampler=dict(type='AspectRatioBatchSampler'),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/multisports_det_anno_train.json',\n", " data_prefix=dict(img='rawframes/'),\n", " filter_cfg=dict(filter_empty_gt=True, min_size=32),\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='RandomChoiceResize',\n", " scales=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),\n", " (1333, 768), (1333, 800)],\n", " keep_ratio=True),\n", " dict(type='RandomFlip', prob=0.5),\n", " dict(type='PackDetInputs')\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "val_dataloader = dict(\n", " batch_size=1,\n", " num_workers=2,\n", " persistent_workers=True,\n", " drop_last=False,\n", " sampler=dict(type='DefaultSampler', shuffle=False),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/multisports_det_anno_val.json',\n", " data_prefix=dict(img='rawframes/'),\n", " test_mode=True,\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "test_dataloader = dict(\n", " batch_size=1,\n", " num_workers=2,\n", " persistent_workers=True,\n", " drop_last=False,\n", " sampler=dict(type='DefaultSampler', shuffle=False),\n", " dataset=dict(\n", " type='CocoDataset',\n", " data_root='data/multisports/',\n", " ann_file='annotations/ms_infer_anno.json',\n", " data_prefix=dict(img='rawframes/'),\n", " test_mode=True,\n", " pipeline=[\n", " dict(type='LoadImageFromFile', backend_args=None),\n", " dict(type='Resize', scale=(1333, 800), keep_ratio=True),\n", " dict(type='LoadAnnotations', with_bbox=True),\n", " dict(\n", " type='PackDetInputs',\n", " meta_keys=('img_id', 'img_path', 'ori_shape', 'img_shape',\n", " 'scale_factor'))\n", " ],\n", " backend_args=None,\n", " metainfo=dict(classes=('person', ), palette=[(220, 20, 60)])))\n", "val_evaluator = dict(\n", " type='CocoMetric',\n", " ann_file='data/multisports/annotations/multisports_det_anno_val.json',\n", " metric='bbox',\n", " format_only=False,\n", " backend_args=None,\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5])\n", "test_evaluator = dict(\n", " type='CocoMetric',\n", " ann_file='data/multisports/annotations/ms_infer_anno.json',\n", " metric='bbox',\n", " format_only=False,\n", " backend_args=None,\n", " metric_items=['mAP_50', 'AR@100'],\n", " iou_thrs=[0.5])\n", "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2, val_interval=1)\n", "val_cfg = dict(type='ValLoop')\n", "test_cfg = dict(type='TestLoop')\n", "param_scheduler = [\n", " dict(type='ConstantLR', factor=1.0, by_epoch=False, begin=0, end=500)\n", "]\n", "optim_wrapper = dict(\n", " type='OptimWrapper',\n", " optimizer=dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001))\n", "auto_scale_lr = dict(enable=False, base_batch_size=16)\n", "default_scope = 'mmdet'\n", "default_hooks = dict(\n", " timer=dict(type='IterTimerHook'),\n", " logger=dict(type='LoggerHook', interval=50),\n", " param_scheduler=dict(type='ParamSchedulerHook'),\n", " checkpoint=dict(type='CheckpointHook', interval=1),\n", " sampler_seed=dict(type='DistSamplerSeedHook'),\n", " visualization=dict(type='DetVisualizationHook'))\n", "env_cfg = dict(\n", " cudnn_benchmark=False,\n", " mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n", " dist_cfg=dict(backend='nccl'))\n", "vis_backends = [dict(type='LocalVisBackend')]\n", "visualizer = dict(\n", " type='DetLocalVisualizer',\n", " vis_backends=[dict(type='LocalVisBackend')],\n", " name='visualizer')\n", "log_processor = dict(type='LogProcessor', window_size=50, by_epoch=True)\n", "log_level = 'INFO'\n", "load_from = 'work_dirs/det_model/epoch_2.pth'\n", "resume = False\n", "metainfo = dict(classes=('person', ), palette=[(220, 20, 60)])\n", "launcher = 'none'\n", "work_dir = './work_dirs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person'\n", "\n", "06/15 06:05:19 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n", "06/15 06:05:19 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Hooks will be executed in the following order:\n", "before_run:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "before_train:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_train_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DistSamplerSeedHook \n", " -------------------- \n", "before_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train_epoch:\n", "(NORMAL ) IterTimerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_val_epoch:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "before_val_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_val_iter:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DetVisualizationHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_val_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train:\n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_test_epoch:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "before_test_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_test_iter:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DetVisualizationHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_test_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_run:\n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "loading annotations into memory...\n", "Done (t=0.00s)\n", "creating index...\n", "index created!\n", "loading annotations into memory...\n", "Done (t=0.00s)\n", "creating index...\n", "index created!\n", "06/15 06:05:20 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - The prefix is not set in metric class DumpDetResults.\n", "Loads checkpoint by local backend from path: work_dirs/det_model/epoch_2.pth\n", "06/15 06:05:20 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Load checkpoint from work_dirs/det_model/epoch_2.pth\n", "06/15 06:05:28 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 50/2350] eta: 0:05:50 time: 0.1523 data_time: 0.0084 memory: 512 \n", "06/15 06:05:34 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 100/2350] eta: 0:05:05 time: 0.1191 data_time: 0.0042 memory: 512 \n", "06/15 06:05:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 150/2350] eta: 0:04:45 time: 0.1178 data_time: 0.0023 memory: 512 \n", "06/15 06:05:46 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 200/2350] eta: 0:04:36 time: 0.1255 data_time: 0.0074 memory: 512 \n", "06/15 06:05:52 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 250/2350] eta: 0:04:26 time: 0.1205 data_time: 0.0031 memory: 512 \n", "06/15 06:05:58 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 300/2350] eta: 0:04:19 time: 0.1238 data_time: 0.0063 memory: 512 \n", "06/15 06:06:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 350/2350] eta: 0:04:11 time: 0.1206 data_time: 0.0046 memory: 512 \n", "06/15 06:06:10 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 400/2350] eta: 0:04:03 time: 0.1178 data_time: 0.0030 memory: 512 \n", "06/15 06:06:16 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 450/2350] eta: 0:03:56 time: 0.1212 data_time: 0.0058 memory: 512 \n", "06/15 06:06:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 500/2350] eta: 0:03:48 time: 0.1165 data_time: 0.0031 memory: 512 \n", "06/15 06:06:28 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 550/2350] eta: 0:03:41 time: 0.1202 data_time: 0.0061 memory: 512 \n", "06/15 06:06:34 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 600/2350] eta: 0:03:34 time: 0.1179 data_time: 0.0044 memory: 512 \n", "06/15 06:06:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 650/2350] eta: 0:03:27 time: 0.1156 data_time: 0.0024 memory: 512 \n", "06/15 06:06:46 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 700/2350] eta: 0:03:21 time: 0.1212 data_time: 0.0058 memory: 512 \n", "06/15 06:06:52 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 750/2350] eta: 0:03:14 time: 0.1161 data_time: 0.0025 memory: 512 \n", "06/15 06:06:58 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 800/2350] eta: 0:03:08 time: 0.1200 data_time: 0.0058 memory: 512 \n", "06/15 06:07:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 850/2350] eta: 0:03:02 time: 0.1203 data_time: 0.0053 memory: 512 \n", "06/15 06:07:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 900/2350] eta: 0:02:55 time: 0.1177 data_time: 0.0030 memory: 512 \n", "06/15 06:07:16 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [ 950/2350] eta: 0:02:50 time: 0.1233 data_time: 0.0076 memory: 512 \n", "06/15 06:07:21 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1000/2350] eta: 0:02:43 time: 0.1172 data_time: 0.0025 memory: 512 \n", "06/15 06:07:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1050/2350] eta: 0:02:37 time: 0.1202 data_time: 0.0053 memory: 512 \n", "06/15 06:07:34 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1100/2350] eta: 0:02:31 time: 0.1208 data_time: 0.0059 memory: 512 \n", "06/15 06:07:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1150/2350] eta: 0:02:25 time: 0.1167 data_time: 0.0030 memory: 512 \n", "06/15 06:07:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1200/2350] eta: 0:02:19 time: 0.1212 data_time: 0.0053 memory: 512 \n", "06/15 06:07:51 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1250/2350] eta: 0:02:12 time: 0.1163 data_time: 0.0027 memory: 512 \n", "06/15 06:07:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1300/2350] eta: 0:02:06 time: 0.1188 data_time: 0.0046 memory: 512 \n", "06/15 06:08:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1350/2350] eta: 0:02:00 time: 0.1201 data_time: 0.0056 memory: 512 \n", "06/15 06:08:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1400/2350] eta: 0:01:54 time: 0.1161 data_time: 0.0024 memory: 512 \n", "06/15 06:08:15 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1450/2350] eta: 0:01:48 time: 0.1234 data_time: 0.0079 memory: 512 \n", "06/15 06:08:21 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1500/2350] eta: 0:01:42 time: 0.1165 data_time: 0.0024 memory: 512 \n", "06/15 06:08:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1550/2350] eta: 0:01:36 time: 0.1191 data_time: 0.0043 memory: 512 \n", "06/15 06:08:33 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1600/2350] eta: 0:01:30 time: 0.1219 data_time: 0.0071 memory: 512 \n", "06/15 06:08:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1650/2350] eta: 0:01:24 time: 0.1166 data_time: 0.0026 memory: 512 \n", "06/15 06:08:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1700/2350] eta: 0:01:18 time: 0.1224 data_time: 0.0067 memory: 512 \n", "06/15 06:08:51 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1750/2350] eta: 0:01:12 time: 0.1175 data_time: 0.0032 memory: 512 \n", "06/15 06:08:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1800/2350] eta: 0:01:06 time: 0.1186 data_time: 0.0041 memory: 512 \n", "06/15 06:09:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1850/2350] eta: 0:01:00 time: 0.1227 data_time: 0.0067 memory: 512 \n", "06/15 06:09:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1900/2350] eta: 0:00:54 time: 0.1220 data_time: 0.0070 memory: 512 \n", "06/15 06:09:15 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [1950/2350] eta: 0:00:48 time: 0.1229 data_time: 0.0081 memory: 512 \n", "06/15 06:09:21 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2000/2350] eta: 0:00:42 time: 0.1173 data_time: 0.0029 memory: 512 \n", "06/15 06:09:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2050/2350] eta: 0:00:36 time: 0.1184 data_time: 0.0037 memory: 512 \n", "06/15 06:09:33 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2100/2350] eta: 0:00:30 time: 0.1216 data_time: 0.0066 memory: 512 \n", "06/15 06:09:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2150/2350] eta: 0:00:24 time: 0.1166 data_time: 0.0026 memory: 512 \n", "06/15 06:09:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2200/2350] eta: 0:00:18 time: 0.1213 data_time: 0.0052 memory: 512 \n", "06/15 06:09:51 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2250/2350] eta: 0:00:12 time: 0.1180 data_time: 0.0033 memory: 512 \n", "06/15 06:09:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2300/2350] eta: 0:00:06 time: 0.1173 data_time: 0.0032 memory: 512 \n", "06/15 06:10:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2350/2350] eta: 0:00:00 time: 0.1203 data_time: 0.0048 memory: 512 \n", "06/15 06:10:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Evaluating bbox...\n", "Loading and preparing results...\n", "DONE (t=0.01s)\n", "creating index...\n", "index created!\n", "Running per image evaluation...\n", "Evaluate annotation type *bbox*\n", "DONE (t=0.36s).\n", "Accumulating evaluation results...\n", "DONE (t=0.28s).\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = -1.000\n", " Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=300 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=1000 ] = -1.000\n", " Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=1000 ] = -1.000\n", "06/15 06:10:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - bbox_mAP_copypaste: -1.000 -1.000 -1.000 -1.000 -1.000 -1.000\n", "06/15 06:10:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Results has been saved to data/multisports/annotations/ms_det_proposals.pkl.\n", "06/15 06:10:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(test) [2350/2350] coco/bbox_mAP_50: -1.0000 coco/bbox_AR@100: -1.0000 data_time: 0.0047 time: 0.1202\n", "\u001b[32mTesting finished successfully.\u001b[0m\n" ] } ], "source": [ "!mim test mmdet configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py \\\n", " --checkpoint work_dirs/det_model/epoch_2.pth \\\n", " --out data/multisports/annotations/ms_det_proposals.pkl" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "jzWhc7ClooZ1" }, "source": [ "## 3. Training the Spatio-temporal Action Detection Model\n", "The provided annotation files and the proposal files generated by MMDetection need to be converted to the required format for training the spatiotemporal action detection model. We have provided relevant script to generate the specified format." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "W3slJsWHooZ2", "outputId": "42a4b7be-91f8-4443-b693-ab40b743a14f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "loading test result...\n", "[>>] 2350/2350, 3799.7 task/s, elapsed: 1s, ETA: 0s\n", "\u001b[01;34mdata/multisports/annotations\u001b[00m\n", "├── label_map.txt\n", "├── ms_det_proposals.pkl\n", "├── ms_infer_anno.json\n", "├── multisports_det_anno_train.json\n", "├── multisports_det_anno_val.json\n", "├── \u001b[01;32mmultisports_GT.pkl\u001b[00m\n", "├── multisports_proposals_train.pkl\n", "├── multisports_proposals_val.pkl\n", "├── multisports_train.csv\n", "└── multisports_val.csv\n", "\n", "0 directories, 10 files\n" ] } ], "source": [ "# Convert annotation files\n", "!python ../../tools/data/multisports/parse_anno.py\n", "\n", "# Convert proposal files\n", "!python tools/convert_proposals.py\n", "\n", "!tree data/multisports/annotations" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "yRSSHmw0ooZ2" }, "source": [ "### 3.2 Training the Spatio-temporal Action Detection Model\n", "\n", "MMAction2 already supports training on the MultiSports dataset. You just need to modify the path to the proposal file. For detailed configurations, please refer to the [config](configs/slowonly_k400_multisports.py) file. Since the training data is limited, the configuration uses a pre-trained model trained on the complete MultiSports dataset. When training with a custom dataset, you don't need to specify the `load_from` configuration." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "vwaay7NvooZ2", "outputId": "add60ddd-2a40-4356-b120-1e7940043778" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training command is /usr/bin/python3 /content/mmaction2/mmaction/.mim/tools/train.py configs/slowonly_k400_multisports.py --launcher none --work-dir work_dirs/stad_model/. \n", "06/15 06:10:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - \n", "------------------------------------------------------------\n", "System environment:\n", " sys.platform: linux\n", " Python: 3.10.12 (main, Jun 7 2023, 12:45:35) [GCC 9.4.0]\n", " CUDA available: True\n", " numpy_random_seed: 1735696538\n", " GPU 0: Tesla T4\n", " CUDA_HOME: /usr/local/cuda\n", " NVCC: Cuda compilation tools, release 11.8, V11.8.89\n", " GCC: x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0\n", " PyTorch: 2.0.1+cu118\n", " PyTorch compiling details: PyTorch built with:\n", " - GCC 9.3\n", " - C++ Version: 201703\n", " - Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications\n", " - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n", " - OpenMP 201511 (a.k.a. OpenMP 4.5)\n", " - LAPACK is enabled (usually provided by MKL)\n", " - NNPACK is enabled\n", " - CPU capability usage: AVX2\n", " - CUDA Runtime 11.8\n", " - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n", " - CuDNN 8.7\n", " - Magma 2.6.1\n", " - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n", "\n", " TorchVision: 0.15.2+cu118\n", " OpenCV: 4.7.0\n", " MMEngine: 0.7.4\n", "\n", "Runtime environment:\n", " cudnn_benchmark: False\n", " mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n", " dist_cfg: {'backend': 'nccl'}\n", " seed: 1735696538\n", " diff_rank_seed: False\n", " deterministic: False\n", " Distributed launcher: none\n", " Distributed training: False\n", " GPU number: 1\n", "------------------------------------------------------------\n", "\n", "06/15 06:10:19 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Config:\n", "default_scope = 'mmaction'\n", "default_hooks = dict(\n", " runtime_info=dict(type='RuntimeInfoHook', _scope_='mmaction'),\n", " timer=dict(type='IterTimerHook', _scope_='mmaction'),\n", " logger=dict(\n", " type='LoggerHook', interval=20, ignore_last=False, _scope_='mmaction'),\n", " param_scheduler=dict(type='ParamSchedulerHook', _scope_='mmaction'),\n", " checkpoint=dict(\n", " type='CheckpointHook',\n", " interval=1,\n", " save_best='auto',\n", " _scope_='mmaction'),\n", " sampler_seed=dict(type='DistSamplerSeedHook', _scope_='mmaction'),\n", " sync_buffers=dict(type='SyncBuffersHook', _scope_='mmaction'))\n", "env_cfg = dict(\n", " cudnn_benchmark=False,\n", " mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n", " dist_cfg=dict(backend='nccl'))\n", "log_processor = dict(\n", " type='LogProcessor', window_size=20, by_epoch=True, _scope_='mmaction')\n", "vis_backends = [dict(type='LocalVisBackend', _scope_='mmaction')]\n", "visualizer = dict(\n", " type='ActionVisualizer',\n", " vis_backends=[dict(type='LocalVisBackend')],\n", " _scope_='mmaction')\n", "log_level = 'INFO'\n", "load_from = 'https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth'\n", "resume = False\n", "url = 'https://download.openmmlab.com/mmaction/v1.0/recognition/slowonly/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth'\n", "num_classes = 66\n", "model = dict(\n", " type='FastRCNN',\n", " _scope_='mmdet',\n", " init_cfg=dict(\n", " type='Pretrained',\n", " checkpoint=\n", " 'https://download.openmmlab.com/mmaction/v1.0/recognition/slowonly/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth'\n", " ),\n", " backbone=dict(\n", " type='mmaction.ResNet3dSlowOnly',\n", " depth=50,\n", " pretrained=None,\n", " pretrained2d=False,\n", " lateral=False,\n", " num_stages=4,\n", " conv1_kernel=(1, 7, 7),\n", " conv1_stride_t=1,\n", " pool1_stride_t=1,\n", " spatial_strides=(1, 2, 2, 1)),\n", " roi_head=dict(\n", " type='AVARoIHead',\n", " bbox_roi_extractor=dict(\n", " type='SingleRoIExtractor3D',\n", " roi_layer_type='RoIAlign',\n", " output_size=8,\n", " with_temporal_pool=True),\n", " bbox_head=dict(\n", " type='BBoxHeadAVA',\n", " in_channels=2048,\n", " num_classes=66,\n", " multilabel=False,\n", " dropout_ratio=0.5)),\n", " data_preprocessor=dict(\n", " type='mmaction.ActionDataPreprocessor',\n", " mean=[123.675, 116.28, 103.53],\n", " std=[58.395, 57.12, 57.375],\n", " format_shape='NCTHW'),\n", " train_cfg=dict(\n", " rcnn=dict(\n", " assigner=dict(\n", " type='MaxIoUAssignerAVA',\n", " pos_iou_thr=0.9,\n", " neg_iou_thr=0.9,\n", " min_pos_iou=0.9),\n", " sampler=dict(\n", " type='RandomSampler',\n", " num=32,\n", " pos_fraction=1,\n", " neg_pos_ub=-1,\n", " add_gt_as_proposals=True),\n", " pos_weight=1.0)),\n", " test_cfg=dict(rcnn=None))\n", "dataset_type = 'AVADataset'\n", "data_root = 'data/multisports/trainval'\n", "anno_root = 'data/multisports/annotations'\n", "ann_file_train = 'data/multisports/annotations/multisports_train.csv'\n", "ann_file_val = 'data/multisports/annotations/multisports_val.csv'\n", "gt_file = 'data/multisports/annotations/multisports_GT.pkl'\n", "proposal_file_train = 'data/multisports/annotations/multisports_proposals_train.pkl'\n", "proposal_file_val = 'data/multisports/annotations/multisports_proposals_val.pkl'\n", "file_client_args = dict(io_backend='disk')\n", "train_pipeline = [\n", " dict(type='DecordInit', io_backend='disk', _scope_='mmaction'),\n", " dict(\n", " type='SampleAVAFrames',\n", " clip_len=4,\n", " frame_interval=16,\n", " _scope_='mmaction'),\n", " dict(type='DecordDecode', _scope_='mmaction'),\n", " dict(type='RandomRescale', scale_range=(256, 320), _scope_='mmaction'),\n", " dict(type='RandomCrop', size=256, _scope_='mmaction'),\n", " dict(type='Flip', flip_ratio=0.5, _scope_='mmaction'),\n", " dict(\n", " type='FormatShape',\n", " input_format='NCTHW',\n", " collapse=True,\n", " _scope_='mmaction'),\n", " dict(type='PackActionInputs', _scope_='mmaction')\n", "]\n", "val_pipeline = [\n", " dict(type='DecordInit', io_backend='disk', _scope_='mmaction'),\n", " dict(\n", " type='SampleAVAFrames',\n", " clip_len=4,\n", " frame_interval=16,\n", " test_mode=True,\n", " _scope_='mmaction'),\n", " dict(type='DecordDecode', _scope_='mmaction'),\n", " dict(type='Resize', scale=(-1, 256), _scope_='mmaction'),\n", " dict(\n", " type='FormatShape',\n", " input_format='NCTHW',\n", " collapse=True,\n", " _scope_='mmaction'),\n", " dict(type='PackActionInputs', _scope_='mmaction')\n", "]\n", "train_dataloader = dict(\n", " batch_size=2,\n", " num_workers=2,\n", " persistent_workers=True,\n", " sampler=dict(type='DefaultSampler', shuffle=True, _scope_='mmaction'),\n", " dataset=dict(\n", " type='AVADataset',\n", " ann_file='data/multisports/annotations/multisports_train.csv',\n", " pipeline=[\n", " dict(type='DecordInit', io_backend='disk'),\n", " dict(type='SampleAVAFrames', clip_len=4, frame_interval=16),\n", " dict(type='DecordDecode'),\n", " dict(type='RandomRescale', scale_range=(256, 320)),\n", " dict(type='RandomCrop', size=256),\n", " dict(type='Flip', flip_ratio=0.5),\n", " dict(type='FormatShape', input_format='NCTHW', collapse=True),\n", " dict(type='PackActionInputs')\n", " ],\n", " num_classes=66,\n", " proposal_file=\n", " 'data/multisports/annotations/multisports_proposals_train.pkl',\n", " data_prefix=dict(img='data/multisports/trainval'),\n", " timestamp_start=1,\n", " start_index=0,\n", " use_frames=False,\n", " fps=1,\n", " _scope_='mmaction'))\n", "val_dataloader = dict(\n", " batch_size=1,\n", " num_workers=2,\n", " persistent_workers=True,\n", " sampler=dict(type='DefaultSampler', shuffle=False, _scope_='mmaction'),\n", " dataset=dict(\n", " type='AVADataset',\n", " ann_file='data/multisports/annotations/multisports_val.csv',\n", " pipeline=[\n", " dict(type='DecordInit', io_backend='disk'),\n", " dict(\n", " type='SampleAVAFrames',\n", " clip_len=4,\n", " frame_interval=16,\n", " test_mode=True),\n", " dict(type='DecordDecode'),\n", " dict(type='Resize', scale=(-1, 256)),\n", " dict(type='FormatShape', input_format='NCTHW', collapse=True),\n", " dict(type='PackActionInputs')\n", " ],\n", " num_classes=66,\n", " proposal_file=\n", " 'data/multisports/annotations/multisports_proposals_val.pkl',\n", " data_prefix=dict(img='data/multisports/trainval'),\n", " test_mode=True,\n", " timestamp_start=1,\n", " start_index=0,\n", " use_frames=False,\n", " fps=1,\n", " _scope_='mmaction'))\n", "test_dataloader = dict(\n", " batch_size=1,\n", " num_workers=8,\n", " persistent_workers=True,\n", " sampler=dict(type='DefaultSampler', shuffle=False, _scope_='mmaction'),\n", " dataset=dict(\n", " type='AVADataset',\n", " ann_file='data/multisports/annotations/multisports_val.csv',\n", " pipeline=[\n", " dict(type='DecordInit', io_backend='disk'),\n", " dict(\n", " type='SampleAVAFrames',\n", " clip_len=4,\n", " frame_interval=16,\n", " test_mode=True),\n", " dict(type='DecordDecode'),\n", " dict(type='Resize', scale=(-1, 256)),\n", " dict(type='FormatShape', input_format='NCTHW', collapse=True),\n", " dict(type='PackActionInputs')\n", " ],\n", " num_classes=66,\n", " proposal_file=\n", " 'data/multisports/annotations/multisports_dense_proposals_val.recall_96.13.pkl',\n", " data_prefix=dict(img='data/multisports/trainval'),\n", " test_mode=True,\n", " timestamp_start=1,\n", " start_index=0,\n", " use_frames=False,\n", " fps=1,\n", " _scope_='mmaction'))\n", "val_evaluator = dict(\n", " type='MultiSportsMetric',\n", " ann_file='data/multisports/annotations/multisports_GT.pkl',\n", " _scope_='mmaction')\n", "test_evaluator = dict(\n", " type='MultiSportsMetric',\n", " ann_file='data/multisports/annotations/multisports_GT.pkl',\n", " _scope_='mmaction')\n", "train_cfg = dict(\n", " type='EpochBasedTrainLoop',\n", " max_epochs=8,\n", " val_begin=1,\n", " val_interval=1,\n", " _scope_='mmaction')\n", "val_cfg = dict(type='ValLoop', _scope_='mmaction')\n", "test_cfg = dict(type='TestLoop', _scope_='mmaction')\n", "param_scheduler = [\n", " dict(\n", " type='LinearLR',\n", " start_factor=0.1,\n", " by_epoch=True,\n", " begin=0,\n", " end=5,\n", " _scope_='mmaction'),\n", " dict(\n", " type='MultiStepLR',\n", " begin=0,\n", " end=8,\n", " by_epoch=True,\n", " milestones=[6, 7],\n", " gamma=0.1,\n", " _scope_='mmaction')\n", "]\n", "optim_wrapper = dict(\n", " optimizer=dict(\n", " type='SGD',\n", " lr=0.01,\n", " momentum=0.9,\n", " weight_decay=1e-05,\n", " _scope_='mmaction'),\n", " clip_grad=dict(max_norm=5, norm_type=2))\n", "launcher = 'none'\n", "work_dir = 'work_dirs/stad_model/'\n", "randomness = dict(seed=None, diff_rank_seed=False, deterministic=False)\n", "\n", "06/15 06:10:23 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n", "06/15 06:10:23 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Hooks will be executed in the following order:\n", "before_run:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "before_train:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_train_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) DistSamplerSeedHook \n", " -------------------- \n", "before_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_train_iter:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train_epoch:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) SyncBuffersHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_val_epoch:\n", "(NORMAL ) IterTimerHook \n", "(NORMAL ) SyncBuffersHook \n", " -------------------- \n", "before_val_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_val_iter:\n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_val_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", "(LOW ) ParamSchedulerHook \n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "after_train:\n", "(VERY_LOW ) CheckpointHook \n", " -------------------- \n", "before_test_epoch:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "before_test_iter:\n", "(NORMAL ) IterTimerHook \n", " -------------------- \n", "after_test_iter:\n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_test_epoch:\n", "(VERY_HIGH ) RuntimeInfoHook \n", "(NORMAL ) IterTimerHook \n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "after_run:\n", "(BELOW_NORMAL) LoggerHook \n", " -------------------- \n", "06/15 06:10:24 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - 236 out of 236 frames are valid.\n", "06/15 06:10:24 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - 120 out of 120 frames are valid.\n", "06/15 06:10:25 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - load model from: https://download.openmmlab.com/mmaction/v1.0/recognition/slowonly/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth\n", "06/15 06:10:25 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Loads checkpoint by http backend from path: https://download.openmmlab.com/mmaction/v1.0/recognition/slowonly/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth\n", "Downloading: \"https://download.openmmlab.com/mmaction/v1.0/recognition/slowonly/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth\" to /root/.cache/torch/hub/checkpoints/slowonly_imagenet-pretrained-r50_8xb16-4x16x1-steplr-150e_kinetics400-rgb_20220901-e7b65fad.pth\n", "100% 124M/124M [00:01<00:00, 103MB/s]\n", "06/15 06:10:28 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - The model and loaded state dict do not match exactly\n", "\n", "unexpected key in source state_dict: cls_head.fc_cls.weight, cls_head.fc_cls.bias\n", "\n", "missing keys in source state_dict: roi_head.bbox_head.fc_cls.weight, roi_head.bbox_head.fc_cls.bias\n", "\n", "Loads checkpoint by http backend from path: https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth\n", "Downloading: \"https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth\" to /root/.cache/torch/hub/checkpoints/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth\n", "100% 122M/122M [00:03<00:00, 36.1MB/s]\n", "06/15 06:10:32 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Load checkpoint from https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth\n", "06/15 06:10:32 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - \"FileClient\" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io\n", "06/15 06:10:32 - mmengine - \u001b[5m\u001b[4m\u001b[33mWARNING\u001b[0m - \"HardDiskBackend\" is the alias of \"LocalBackend\" and the former will be deprecated in future.\n", "06/15 06:10:32 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Checkpoints will be saved to /content/mmaction2/projects/stad_tutorial/work_dirs/stad_model.\n", "06/15 06:10:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][ 20/118] lr: 1.0000e-03 eta: 0:06:07 time: 0.3982 data_time: 0.0431 memory: 1383 grad_norm: 13.0844 loss: 1.3834 recall@thr=0.5: 0.5385 prec@thr=0.5: 0.5385 recall@top3: 0.8462 prec@top3: 0.2821 recall@top5: 0.8462 prec@top5: 0.1692 loss_action_cls: 1.3834\n", "06/15 06:10:46 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][ 40/118] lr: 1.0000e-03 eta: 0:05:32 time: 0.3383 data_time: 0.0732 memory: 1383 grad_norm: 4.6786 loss: 0.6001 recall@thr=0.5: 0.9444 prec@thr=0.5: 0.9444 recall@top3: 0.9444 prec@top3: 0.3148 recall@top5: 0.9444 prec@top5: 0.1889 loss_action_cls: 0.6001\n", "06/15 06:10:52 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][ 60/118] lr: 1.0000e-03 eta: 0:04:59 time: 0.2784 data_time: 0.0300 memory: 1383 grad_norm: 2.9446 loss: 0.5144 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.5144\n", "06/15 06:10:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][ 80/118] lr: 1.0000e-03 eta: 0:04:36 time: 0.2646 data_time: 0.0144 memory: 1383 grad_norm: 1.7695 loss: 0.4988 recall@thr=0.5: 0.6923 prec@thr=0.5: 0.6923 recall@top3: 0.6923 prec@top3: 0.2308 recall@top5: 0.6923 prec@top5: 0.1385 loss_action_cls: 0.4988\n", "06/15 06:11:04 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][100/118] lr: 1.0000e-03 eta: 0:04:35 time: 0.3502 data_time: 0.0839 memory: 1383 grad_norm: 2.4095 loss: 0.3218 recall@thr=0.5: 0.9333 prec@thr=0.5: 0.9333 recall@top3: 0.9333 prec@top3: 0.3111 recall@top5: 0.9333 prec@top5: 0.1867 loss_action_cls: 0.3218\n", "06/15 06:11:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:11:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [1][118/118] lr: 1.0000e-03 eta: 0:04:20 time: 0.2563 data_time: 0.0102 memory: 1383 grad_norm: 1.8156 loss: 0.3895 recall@thr=0.5: 0.8125 prec@thr=0.5: 0.8125 recall@top3: 0.9375 prec@top3: 0.3125 recall@top5: 0.9375 prec@top5: 0.1875 loss_action_cls: 0.3895\n", "06/15 06:11:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 1 epochs\n", "06/15 06:11:14 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][ 20/120] eta: 0:00:16 time: 0.1669 data_time: 0.1073 memory: 466 \n", "06/15 06:11:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][ 40/120] eta: 0:00:13 time: 0.1698 data_time: 0.1145 memory: 466 \n", "06/15 06:11:20 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][ 60/120] eta: 0:00:09 time: 0.1428 data_time: 0.0896 memory: 466 \n", "06/15 06:11:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][ 80/120] eta: 0:00:05 time: 0.0998 data_time: 0.0504 memory: 466 \n", "06/15 06:11:25 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][100/120] eta: 0:00:02 time: 0.1122 data_time: 0.0612 memory: 466 \n", "06/15 06:11:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][120/120] eta: 0:00:00 time: 0.1031 data_time: 0.0528 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 59.66\n", "aerobic split jump 30.80\n", "aerobic scissors leap 88.34\n", "aerobic turn 98.48\n", "mAP 69.32\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 25.00\n", "aerobic split jump 20.00\n", "aerobic scissors leap 80.00\n", "aerobic turn 100.00\n", "mAP 56.25\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 25.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 50.00\n", "aerobic turn 100.00\n", "mAP 43.75\n", "06/15 06:11:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [1][120/120] mAP/frameAP: 69.3181 mAP/v_map@0.2: 56.2500 mAP/v_map@0.5: 43.7500 mAP/v_map_0.05:0.45: 55.1389 mAP/v_map_0.10:0.90: 41.2500 mAP/v_map_0.50:0.95: 28.1750 data_time: 0.0793 time: 0.1324\n", "06/15 06:11:29 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - The best checkpoint with 69.3181 mAP/frameAP at 1 epoch is saved to best_mAP_frameAP_epoch_1.pth.\n", "06/15 06:11:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][ 20/118] lr: 3.2500e-03 eta: 0:04:10 time: 0.2884 data_time: 0.0401 memory: 1383 grad_norm: 1.3823 loss: 0.3596 recall@thr=0.5: 0.6923 prec@thr=0.5: 0.6923 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3596\n", "06/15 06:11:46 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][ 40/118] lr: 3.2500e-03 eta: 0:04:00 time: 0.2728 data_time: 0.0204 memory: 1383 grad_norm: 1.2185 loss: 0.5274 recall@thr=0.5: 0.9333 prec@thr=0.5: 0.9333 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.5274\n", "06/15 06:11:52 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][ 60/118] lr: 3.2500e-03 eta: 0:03:56 time: 0.3296 data_time: 0.0699 memory: 1383 grad_norm: 1.7120 loss: 0.3599 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3599\n", "06/15 06:11:57 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][ 80/118] lr: 3.2500e-03 eta: 0:03:46 time: 0.2584 data_time: 0.0120 memory: 1383 grad_norm: 1.7462 loss: 0.2598 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2598\n", "06/15 06:12:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][100/118] lr: 3.2500e-03 eta: 0:03:39 time: 0.2858 data_time: 0.0263 memory: 1383 grad_norm: 0.8975 loss: 0.3959 recall@thr=0.5: 0.7692 prec@thr=0.5: 0.7692 recall@top3: 0.9231 prec@top3: 0.3077 recall@top5: 0.9231 prec@top5: 0.1846 loss_action_cls: 0.3959\n", "06/15 06:12:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:12:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [2][118/118] lr: 3.2500e-03 eta: 0:03:35 time: 0.3381 data_time: 0.0807 memory: 1383 grad_norm: 0.5466 loss: 0.4871 recall@thr=0.5: 0.8333 prec@thr=0.5: 0.8333 recall@top3: 0.8333 prec@top3: 0.2778 recall@top5: 0.8333 prec@top5: 0.1667 loss_action_cls: 0.4871\n", "06/15 06:12:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 2 epochs\n", "06/15 06:12:13 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][ 20/120] eta: 0:00:12 time: 0.1230 data_time: 0.0693 memory: 466 \n", "06/15 06:12:15 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][ 40/120] eta: 0:00:09 time: 0.1138 data_time: 0.0632 memory: 466 \n", "06/15 06:12:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][ 60/120] eta: 0:00:07 time: 0.1214 data_time: 0.0672 memory: 466 \n", "06/15 06:12:21 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][ 80/120] eta: 0:00:05 time: 0.1539 data_time: 0.1001 memory: 466 \n", "06/15 06:12:24 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][100/120] eta: 0:00:02 time: 0.1488 data_time: 0.0936 memory: 466 \n", "06/15 06:12:26 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][120/120] eta: 0:00:00 time: 0.1030 data_time: 0.0539 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 39.91\n", "aerobic split jump 29.66\n", "aerobic scissors leap 90.70\n", "aerobic turn 96.92\n", "mAP 64.30\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 20.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 100.00\n", "mAP 55.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 36.00\n", "aerobic turn 100.00\n", "mAP 34.00\n", "06/15 06:12:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [2][120/120] mAP/frameAP: 64.2982 mAP/v_map@0.2: 55.0000 mAP/v_map@0.5: 34.0000 mAP/v_map_0.05:0.45: 53.8889 mAP/v_map_0.10:0.90: 34.5833 mAP/v_map_0.50:0.95: 19.1250 data_time: 0.0744 time: 0.1270\n", "06/15 06:12:32 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][ 20/118] lr: 5.5000e-03 eta: 0:03:28 time: 0.2786 data_time: 0.0358 memory: 1383 grad_norm: 1.0935 loss: 0.3780 recall@thr=0.5: 0.8667 prec@thr=0.5: 0.8667 recall@top3: 0.8667 prec@top3: 0.2889 recall@top5: 0.8667 prec@top5: 0.1733 loss_action_cls: 0.3780\n", "06/15 06:12:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][ 40/118] lr: 5.5000e-03 eta: 0:03:22 time: 0.3217 data_time: 0.0573 memory: 1383 grad_norm: 1.4278 loss: 0.3261 recall@thr=0.5: 0.8750 prec@thr=0.5: 0.8750 recall@top3: 0.9375 prec@top3: 0.3125 recall@top5: 0.9375 prec@top5: 0.1875 loss_action_cls: 0.3261\n", "06/15 06:12:44 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][ 60/118] lr: 5.5000e-03 eta: 0:03:15 time: 0.2823 data_time: 0.0358 memory: 1383 grad_norm: 0.6230 loss: 0.4514 recall@thr=0.5: 0.9286 prec@thr=0.5: 0.9286 recall@top3: 0.9286 prec@top3: 0.3095 recall@top5: 0.9286 prec@top5: 0.1857 loss_action_cls: 0.4514\n", "06/15 06:12:49 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][ 80/118] lr: 5.5000e-03 eta: 0:03:08 time: 0.2561 data_time: 0.0115 memory: 1383 grad_norm: 0.1768 loss: 0.3241 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3241\n", "06/15 06:12:56 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][100/118] lr: 5.5000e-03 eta: 0:03:02 time: 0.3094 data_time: 0.0422 memory: 1383 grad_norm: 0.4979 loss: 0.4081 recall@thr=0.5: 0.8333 prec@thr=0.5: 0.8333 recall@top3: 0.8333 prec@top3: 0.2778 recall@top5: 0.8333 prec@top5: 0.1667 loss_action_cls: 0.4081\n", "06/15 06:13:01 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:13:01 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [3][118/118] lr: 5.5000e-03 eta: 0:02:56 time: 0.2776 data_time: 0.0266 memory: 1383 grad_norm: 0.7488 loss: 0.4131 recall@thr=0.5: 0.6667 prec@thr=0.5: 0.6667 recall@top3: 0.6667 prec@top3: 0.2222 recall@top5: 0.6667 prec@top5: 0.1333 loss_action_cls: 0.4131\n", "06/15 06:13:01 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 3 epochs\n", "06/15 06:13:05 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][ 20/120] eta: 0:00:11 time: 0.1182 data_time: 0.0691 memory: 466 \n", "06/15 06:13:07 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][ 40/120] eta: 0:00:09 time: 0.1132 data_time: 0.0628 memory: 466 \n", "06/15 06:13:10 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][ 60/120] eta: 0:00:07 time: 0.1542 data_time: 0.0996 memory: 466 \n", "06/15 06:13:13 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][ 80/120] eta: 0:00:05 time: 0.1479 data_time: 0.0937 memory: 466 \n", "06/15 06:13:15 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][100/120] eta: 0:00:02 time: 0.1232 data_time: 0.0726 memory: 466 \n", "06/15 06:13:17 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][120/120] eta: 0:00:00 time: 0.1029 data_time: 0.0529 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 29.65\n", "aerobic split jump 20.83\n", "aerobic scissors leap 90.63\n", "aerobic turn 97.10\n", "mAP 59.55\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 100.00\n", "mAP 50.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 36.00\n", "aerobic turn 100.00\n", "mAP 34.00\n", "06/15 06:13:18 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [3][120/120] mAP/frameAP: 59.5538 mAP/v_map@0.2: 50.0000 mAP/v_map@0.5: 34.0000 mAP/v_map_0.05:0.45: 50.0000 mAP/v_map_0.10:0.90: 32.9167 mAP/v_map_0.50:0.95: 19.1250 data_time: 0.0750 time: 0.1264\n", "06/15 06:13:24 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][ 20/118] lr: 7.7500e-03 eta: 0:02:50 time: 0.3089 data_time: 0.0514 memory: 1383 grad_norm: 0.2046 loss: 0.3238 recall@thr=0.5: 0.9091 prec@thr=0.5: 0.9091 recall@top3: 0.9091 prec@top3: 0.3030 recall@top5: 0.9091 prec@top5: 0.1818 loss_action_cls: 0.3238\n", "06/15 06:13:32 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][ 40/118] lr: 7.7500e-03 eta: 0:02:46 time: 0.3790 data_time: 0.0937 memory: 1383 grad_norm: 0.7468 loss: 0.4123 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.4123\n", "06/15 06:13:37 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][ 60/118] lr: 7.7500e-03 eta: 0:02:39 time: 0.2685 data_time: 0.0171 memory: 1383 grad_norm: 0.1904 loss: 0.4407 recall@thr=0.5: 0.6667 prec@thr=0.5: 0.6667 recall@top3: 0.6667 prec@top3: 0.2222 recall@top5: 0.6667 prec@top5: 0.1333 loss_action_cls: 0.4407\n", "06/15 06:13:42 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][ 80/118] lr: 7.7500e-03 eta: 0:02:32 time: 0.2546 data_time: 0.0100 memory: 1383 grad_norm: 0.1966 loss: 0.4266 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.4266\n", "06/15 06:13:49 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][100/118] lr: 7.7500e-03 eta: 0:02:27 time: 0.3283 data_time: 0.0548 memory: 1383 grad_norm: 0.3165 loss: 0.3308 recall@thr=0.5: 0.8000 prec@thr=0.5: 0.8000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3308\n", "06/15 06:13:53 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:13:53 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [4][118/118] lr: 7.7500e-03 eta: 0:02:21 time: 0.2671 data_time: 0.0151 memory: 1383 grad_norm: 0.1487 loss: 0.3003 recall@thr=0.5: 0.8333 prec@thr=0.5: 0.8333 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3003\n", "06/15 06:13:53 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 4 epochs\n", "06/15 06:13:58 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][ 20/120] eta: 0:00:12 time: 0.1273 data_time: 0.0729 memory: 466 \n", "06/15 06:14:00 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][ 40/120] eta: 0:00:10 time: 0.1306 data_time: 0.0797 memory: 466 \n", "06/15 06:14:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][ 60/120] eta: 0:00:08 time: 0.1539 data_time: 0.0979 memory: 466 \n", "06/15 06:14:06 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][ 80/120] eta: 0:00:05 time: 0.1355 data_time: 0.0815 memory: 466 \n", "06/15 06:14:08 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][100/120] eta: 0:00:02 time: 0.1132 data_time: 0.0646 memory: 466 \n", "06/15 06:14:10 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][120/120] eta: 0:00:00 time: 0.1050 data_time: 0.0553 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 23.92\n", "aerobic split jump 19.60\n", "aerobic scissors leap 91.02\n", "aerobic turn 96.05\n", "mAP 57.64\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 100.00\n", "mAP 50.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 36.00\n", "aerobic turn 100.00\n", "mAP 34.00\n", "06/15 06:14:11 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [4][120/120] mAP/frameAP: 57.6444 mAP/v_map@0.2: 50.0000 mAP/v_map@0.5: 34.0000 mAP/v_map_0.05:0.45: 50.0000 mAP/v_map_0.10:0.90: 32.9167 mAP/v_map_0.50:0.95: 18.3250 data_time: 0.0753 time: 0.1274\n", "06/15 06:14:17 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][ 20/118] lr: 1.0000e-02 eta: 0:02:14 time: 0.2810 data_time: 0.0329 memory: 1383 grad_norm: 0.6113 loss: 0.4312 recall@thr=0.5: 0.8182 prec@thr=0.5: 0.8182 recall@top3: 0.8182 prec@top3: 0.2727 recall@top5: 0.8182 prec@top5: 0.1636 loss_action_cls: 0.4312\n", "06/15 06:14:23 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][ 40/118] lr: 1.0000e-02 eta: 0:02:09 time: 0.3316 data_time: 0.0732 memory: 1383 grad_norm: 0.2282 loss: 0.3932 recall@thr=0.5: 0.8182 prec@thr=0.5: 0.8182 recall@top3: 0.8182 prec@top3: 0.2727 recall@top5: 0.8182 prec@top5: 0.1636 loss_action_cls: 0.3932\n", "06/15 06:14:29 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][ 60/118] lr: 1.0000e-02 eta: 0:02:03 time: 0.2738 data_time: 0.0286 memory: 1383 grad_norm: 0.2938 loss: 0.3828 recall@thr=0.5: 0.8571 prec@thr=0.5: 0.8571 recall@top3: 0.8571 prec@top3: 0.2857 recall@top5: 0.8571 prec@top5: 0.1714 loss_action_cls: 0.3828\n", "06/15 06:14:34 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][ 80/118] lr: 1.0000e-02 eta: 0:01:56 time: 0.2756 data_time: 0.0192 memory: 1383 grad_norm: 0.1112 loss: 0.3722 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3722\n", "06/15 06:14:41 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][100/118] lr: 1.0000e-02 eta: 0:01:51 time: 0.3193 data_time: 0.0573 memory: 1383 grad_norm: 0.6399 loss: 0.4427 recall@thr=0.5: 0.8000 prec@thr=0.5: 0.8000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.4427\n", "06/15 06:14:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:14:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [5][118/118] lr: 1.0000e-02 eta: 0:01:45 time: 0.2535 data_time: 0.0093 memory: 1383 grad_norm: 0.0985 loss: 0.2719 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2719\n", "06/15 06:14:45 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 5 epochs\n", "06/15 06:14:50 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][ 20/120] eta: 0:00:13 time: 0.1329 data_time: 0.0774 memory: 466 \n", "06/15 06:14:53 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][ 40/120] eta: 0:00:12 time: 0.1787 data_time: 0.1259 memory: 466 \n", "06/15 06:14:56 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][ 60/120] eta: 0:00:08 time: 0.1363 data_time: 0.0829 memory: 466 \n", "06/15 06:14:58 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][ 80/120] eta: 0:00:05 time: 0.1012 data_time: 0.0513 memory: 466 \n", "06/15 06:15:00 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][100/120] eta: 0:00:02 time: 0.1095 data_time: 0.0593 memory: 466 \n", "06/15 06:15:02 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][120/120] eta: 0:00:00 time: 0.1033 data_time: 0.0536 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 14.21\n", "aerobic split jump 15.37\n", "aerobic scissors leap 91.25\n", "aerobic turn 91.43\n", "mAP 53.06\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 80.00\n", "mAP 45.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 36.00\n", "aerobic turn 20.00\n", "mAP 14.00\n", "06/15 06:15:03 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [5][120/120] mAP/frameAP: 53.0627 mAP/v_map@0.2: 45.0000 mAP/v_map@0.5: 14.0000 mAP/v_map_0.05:0.45: 40.0000 mAP/v_map_0.10:0.90: 22.4444 mAP/v_map_0.50:0.95: 7.0250 data_time: 0.0749 time: 0.1267\n", "06/15 06:15:09 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][ 20/118] lr: 1.0000e-02 eta: 0:01:39 time: 0.3193 data_time: 0.0634 memory: 1383 grad_norm: 0.5229 loss: 0.3929 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3929\n", "06/15 06:15:15 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][ 40/118] lr: 1.0000e-02 eta: 0:01:33 time: 0.2972 data_time: 0.0439 memory: 1383 grad_norm: 0.4621 loss: 0.2891 recall@thr=0.5: 0.7692 prec@thr=0.5: 0.7692 recall@top3: 0.9231 prec@top3: 0.3077 recall@top5: 0.9231 prec@top5: 0.1846 loss_action_cls: 0.2891\n", "06/15 06:15:20 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][ 60/118] lr: 1.0000e-02 eta: 0:01:27 time: 0.2567 data_time: 0.0127 memory: 1383 grad_norm: 0.2534 loss: 0.3438 recall@thr=0.5: 0.9333 prec@thr=0.5: 0.9333 recall@top3: 0.9333 prec@top3: 0.3111 recall@top5: 0.9333 prec@top5: 0.1867 loss_action_cls: 0.3438\n", "06/15 06:15:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][ 80/118] lr: 1.0000e-02 eta: 0:01:21 time: 0.3277 data_time: 0.0645 memory: 1383 grad_norm: 0.0856 loss: 0.1859 recall@thr=0.5: 0.8571 prec@thr=0.5: 0.8571 recall@top3: 0.8571 prec@top3: 0.2857 recall@top5: 0.8571 prec@top5: 0.1714 loss_action_cls: 0.1859\n", "06/15 06:15:33 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][100/118] lr: 1.0000e-02 eta: 0:01:15 time: 0.2995 data_time: 0.0503 memory: 1383 grad_norm: 0.3619 loss: 0.3205 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3205\n", "06/15 06:15:37 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:15:37 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [6][118/118] lr: 1.0000e-02 eta: 0:01:10 time: 0.2619 data_time: 0.0190 memory: 1383 grad_norm: 0.3812 loss: 0.3911 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3911\n", "06/15 06:15:37 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 6 epochs\n", "06/15 06:15:43 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][ 20/120] eta: 0:00:17 time: 0.1739 data_time: 0.1178 memory: 466 \n", "06/15 06:15:46 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][ 40/120] eta: 0:00:13 time: 0.1519 data_time: 0.1032 memory: 466 \n", "06/15 06:15:48 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][ 60/120] eta: 0:00:08 time: 0.1031 data_time: 0.0536 memory: 466 \n", "06/15 06:15:50 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][ 80/120] eta: 0:00:05 time: 0.0998 data_time: 0.0505 memory: 466 \n", "06/15 06:15:52 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][100/120] eta: 0:00:02 time: 0.1126 data_time: 0.0620 memory: 466 \n", "06/15 06:15:54 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][120/120] eta: 0:00:00 time: 0.0995 data_time: 0.0506 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 10.49\n", "aerobic split jump 14.53\n", "aerobic scissors leap 90.24\n", "aerobic turn 87.53\n", "mAP 50.70\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 40.00\n", "mAP 35.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 36.00\n", "aerobic turn 40.00\n", "mAP 19.00\n", "06/15 06:15:55 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [6][120/120] mAP/frameAP: 50.6970 mAP/v_map@0.2: 35.0000 mAP/v_map@0.5: 19.0000 mAP/v_map_0.05:0.45: 35.0000 mAP/v_map_0.10:0.90: 20.7778 mAP/v_map_0.50:0.95: 8.4000 data_time: 0.0724 time: 0.1229\n", "06/15 06:16:02 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][ 20/118] lr: 1.0000e-03 eta: 0:01:04 time: 0.3578 data_time: 0.0847 memory: 1383 grad_norm: 0.5369 loss: 0.3628 recall@thr=0.5: 0.9167 prec@thr=0.5: 0.9167 recall@top3: 0.9167 prec@top3: 0.3056 recall@top5: 0.9167 prec@top5: 0.1833 loss_action_cls: 0.3628\n", "06/15 06:16:07 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][ 40/118] lr: 1.0000e-03 eta: 0:00:58 time: 0.2652 data_time: 0.0202 memory: 1383 grad_norm: 0.1603 loss: 0.2293 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2293\n", "06/15 06:16:13 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][ 60/118] lr: 1.0000e-03 eta: 0:00:52 time: 0.2710 data_time: 0.0178 memory: 1383 grad_norm: 0.3857 loss: 0.2737 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2737\n", "06/15 06:16:20 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][ 80/118] lr: 1.0000e-03 eta: 0:00:46 time: 0.3420 data_time: 0.0698 memory: 1383 grad_norm: 0.1271 loss: 0.2149 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2149\n", "06/15 06:16:25 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][100/118] lr: 1.0000e-03 eta: 0:00:40 time: 0.2673 data_time: 0.0232 memory: 1383 grad_norm: 0.0990 loss: 0.2749 recall@thr=0.5: 0.8571 prec@thr=0.5: 0.8571 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2749\n", "06/15 06:16:30 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:16:30 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [7][118/118] lr: 1.0000e-03 eta: 0:00:34 time: 0.2612 data_time: 0.0156 memory: 1383 grad_norm: 0.1387 loss: 0.3211 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3211\n", "06/15 06:16:30 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 7 epochs\n", "06/15 06:16:35 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][ 20/120] eta: 0:00:16 time: 0.1657 data_time: 0.1063 memory: 466 \n", "06/15 06:16:38 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][ 40/120] eta: 0:00:11 time: 0.1164 data_time: 0.0654 memory: 466 \n", "06/15 06:16:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][ 60/120] eta: 0:00:07 time: 0.1053 data_time: 0.0546 memory: 466 \n", "06/15 06:16:42 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][ 80/120] eta: 0:00:04 time: 0.1005 data_time: 0.0511 memory: 466 \n", "06/15 06:16:44 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][100/120] eta: 0:00:02 time: 0.1035 data_time: 0.0533 memory: 466 \n", "06/15 06:16:47 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][120/120] eta: 0:00:00 time: 0.1382 data_time: 0.0850 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 11.65\n", "aerobic split jump 15.62\n", "aerobic scissors leap 89.83\n", "aerobic turn 93.96\n", "mAP 52.77\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 80.00\n", "mAP 45.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 38.67\n", "aerobic turn 20.00\n", "mAP 14.67\n", "06/15 06:16:48 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [7][120/120] mAP/frameAP: 52.7652 mAP/v_map@0.2: 45.0000 mAP/v_map@0.5: 14.6667 mAP/v_map_0.05:0.45: 40.6944 mAP/v_map_0.10:0.90: 22.6389 mAP/v_map_0.50:0.95: 6.6833 data_time: 0.0691 time: 0.1213\n", "06/15 06:16:54 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][ 20/118] lr: 1.0000e-04 eta: 0:00:29 time: 0.3243 data_time: 0.0649 memory: 1383 grad_norm: 0.1808 loss: 0.3648 recall@thr=0.5: 0.8571 prec@thr=0.5: 0.8571 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3648\n", "06/15 06:16:59 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][ 40/118] lr: 1.0000e-04 eta: 0:00:23 time: 0.2578 data_time: 0.0117 memory: 1383 grad_norm: 0.0784 loss: 0.2355 recall@thr=0.5: 1.0000 prec@thr=0.5: 1.0000 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2355\n", "06/15 06:17:06 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][ 60/118] lr: 1.0000e-04 eta: 0:00:17 time: 0.3075 data_time: 0.0490 memory: 1383 grad_norm: 0.1707 loss: 0.3776 recall@thr=0.5: 0.9333 prec@thr=0.5: 0.9333 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3776\n", "06/15 06:17:12 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][ 80/118] lr: 1.0000e-04 eta: 0:00:11 time: 0.3092 data_time: 0.0576 memory: 1383 grad_norm: 0.1387 loss: 0.3873 recall@thr=0.5: 0.8182 prec@thr=0.5: 0.8182 recall@top3: 0.8182 prec@top3: 0.2727 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3873\n", "06/15 06:17:17 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][100/118] lr: 1.0000e-04 eta: 0:00:05 time: 0.2578 data_time: 0.0100 memory: 1383 grad_norm: 0.2137 loss: 0.3337 recall@thr=0.5: 0.8462 prec@thr=0.5: 0.8462 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.3337\n", "06/15 06:17:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Exp name: slowonly_k400_multisports_20230615_061017\n", "06/15 06:17:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(train) [8][118/118] lr: 1.0000e-04 eta: 0:00:00 time: 0.2755 data_time: 0.0148 memory: 1383 grad_norm: 0.0712 loss: 0.2038 recall@thr=0.5: 0.9091 prec@thr=0.5: 0.9091 recall@top3: 1.0000 prec@top3: 0.3333 recall@top5: 1.0000 prec@top5: 0.2000 loss_action_cls: 0.2038\n", "06/15 06:17:22 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Saving checkpoint at 8 epochs\n", "06/15 06:17:27 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][ 20/120] eta: 0:00:11 time: 0.1180 data_time: 0.0649 memory: 466 \n", "06/15 06:17:29 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][ 40/120] eta: 0:00:09 time: 0.1168 data_time: 0.0667 memory: 466 \n", "06/15 06:17:31 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][ 60/120] eta: 0:00:06 time: 0.1026 data_time: 0.0535 memory: 466 \n", "06/15 06:17:33 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][ 80/120] eta: 0:00:04 time: 0.1017 data_time: 0.0533 memory: 466 \n", "06/15 06:17:36 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][100/120] eta: 0:00:02 time: 0.1444 data_time: 0.0915 memory: 466 \n", "06/15 06:17:39 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][120/120] eta: 0:00:00 time: 0.1496 data_time: 0.0962 memory: 466 \n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluate aerobic kick jump\n", "do not evaluate aerobic off axis jump\n", "do not evaluate aerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluate aerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluate basketball save\n", "do not evaluate basketball jump ball\n", "frameAP_0.5\n", "\n", "aerobic straight jump 11.34\n", "aerobic split jump 12.82\n", "aerobic scissors leap 90.68\n", "aerobic turn 90.47\n", "mAP 51.33\n", "\u001b[2Klinking tubes... \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m0:00:00\u001b[0m\n", "\u001b[?25hno such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.2\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 100.00\n", "aerobic turn 80.00\n", "mAP 45.00\n", "no such label 0 aerobic push up\n", "no such label 1 aerobic explosive push up\n", "no such label 2 aerobic explosive support\n", "no such label 3 aerobic leg circle\n", "no such label 4 aerobic helicopter\n", "no such label 5 aerobic support\n", "no such label 6 aerobic v support\n", "no such label 7 aerobic horizontal support\n", "no such label 9 aerobic illusion\n", "no such label 10 aerobic bent leg(s) jump\n", "no such label 11 aerobic pike jump\n", "no such label 12 aerobic straddle jump\n", "do not evaluateaerobic kick jump\n", "do not evaluateaerobic off axis jump\n", "do not evaluateaerobic butterfly jump\n", "no such label 18 aerobic split\n", "do not evaluateaerobic balance turn\n", "no such label 21 volleyball serve\n", "no such label 22 volleyball block\n", "no such label 23 volleyball first pass\n", "no such label 24 volleyball defend\n", "no such label 25 volleyball protect\n", "no such label 26 volleyball second pass\n", "no such label 27 volleyball adjust\n", "no such label 28 volleyball save\n", "no such label 29 volleyball second attack\n", "no such label 30 volleyball spike\n", "no such label 31 volleyball dink\n", "no such label 32 volleyball no offensive attack\n", "no such label 33 football shoot\n", "no such label 34 football long pass\n", "no such label 35 football short pass\n", "no such label 36 football through pass\n", "no such label 37 football cross\n", "no such label 38 football dribble\n", "no such label 39 football trap\n", "no such label 40 football throw\n", "no such label 41 football diving\n", "no such label 42 football tackle\n", "no such label 43 football steal\n", "no such label 44 football clearance\n", "no such label 45 football block\n", "no such label 46 football press\n", "no such label 47 football aerial duels\n", "no such label 48 basketball pass\n", "no such label 49 basketball drive\n", "no such label 50 basketball dribble\n", "no such label 51 basketball 3-point shot\n", "no such label 52 basketball 2-point shot\n", "no such label 53 basketball free throw\n", "no such label 54 basketball block\n", "no such label 55 basketball offensive rebound\n", "no such label 56 basketball defensive rebound\n", "no such label 57 basketball pass steal\n", "no such label 58 basketball dribble steal\n", "no such label 59 basketball interfere shot\n", "no such label 60 basketball pick-and-roll defensive\n", "no such label 61 basketball sag\n", "no such label 62 basketball screen\n", "no such label 63 basketball pass-inbound\n", "do not evaluatebasketball save\n", "do not evaluatebasketball jump ball\n", "VideoAP_0.5\n", "\n", "aerobic straight jump 0.00\n", "aerobic split jump 0.00\n", "aerobic scissors leap 72.00\n", "aerobic turn 20.00\n", "mAP 23.00\n", "06/15 06:17:40 - mmengine - \u001b[4m\u001b[97mINFO\u001b[0m - Epoch(val) [8][120/120] mAP/frameAP: 51.3281 mAP/v_map@0.2: 45.0000 mAP/v_map@0.5: 23.0000 mAP/v_map_0.05:0.45: 40.0000 mAP/v_map_0.10:0.90: 24.4444 mAP/v_map_0.50:0.95: 9.7250 data_time: 0.0704 time: 0.1216\n", "\u001b[32mTraining finished successfully. \u001b[0m\n" ] } ], "source": [ "# Train the model using MIM\n", "!mim train mmaction2 configs/slowonly_k400_multisports.py \\\n", " --work-dir work_dirs/stad_model/" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "id": "yVjHqupPooZ2" }, "source": [ "## 4. Inferring the Spatiotemporal Action Detection Model\n", "\n", "After training the detection model and the spatiotemporal action detection model, we can use the spatiotemporal action detection demo for inference and visualize the model's performance.\n", "\n", "Since the tutorial uses a limited training dataset, the model's performance is not optimal, so a pre-trained model is used for visualization." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "NQF1yrEhooZ3", "outputId": "5331fbb6-7075-415c-f6f0-ec41c4b584a4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ALSA lib confmisc.c:767:(parse_card) cannot find card '0'\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory\n", "ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory\n", "ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory\n", "ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such file or directory\n", "ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default\n", "ALSA lib confmisc.c:767:(parse_card) cannot find card '0'\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory\n", "ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory\n", "ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name\n", "ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory\n", "ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such file or directory\n", "ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default\n", "Loads checkpoint by local backend from path: work_dirs/det_model/epoch_2.pth\n", "Performing Human Detection for each frame\n", "[>>] 99/99, 7.0 task/s, elapsed: 14s, ETA: 0s\n", "Loads checkpoint by http backend from path: https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth\n", "Performing SpatioTemporal Action Detection for each clip\n", "[>>] 99/99, 17.1 task/s, elapsed: 6s, ETA: 0sPerforming visualization\n", "Moviepy - Building video data/demo_spatiotemporal_det.mp4.\n", "Moviepy - Writing video data/demo_spatiotemporal_det.mp4\n", "\n", "Moviepy - Done !\n", "Moviepy - video ready data/demo_spatiotemporal_det.mp4\n" ] } ], "source": [ "!python ../../demo/demo_spatiotemporal_det.py \\\n", " data/multisports/test/aerobic_gymnastics/v_7G_IpU0FxLU_c001.mp4 \\\n", " data/demo_spatiotemporal_det.mp4 \\\n", " --config configs/slowonly_k400_multisports.py \\\n", " --checkpoint https://download.openmmlab.com/mmaction/v1.0/detection/slowonly/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowonly_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-a1ca5e76.pth \\\n", " --det-config configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco_ms_person.py \\\n", " --det-checkpoint work_dirs/det_model/epoch_2.pth \\\n", " --det-score-thr 0.85 \\\n", " --action-score-thr 0.8 \\\n", " --label-map ../../tools/data/multisports/label_map.txt \\\n", " --predict-stepsize 8 \\\n", " --output-stepsize 1 \\\n", " --output-fps 24" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 741 }, "id": "9JmeIkh5ooZ3", "outputId": "7fc38469-d8c4-4a02-81e7-ff93b88a62b2" }, "outputs": [], "source": [ "# Show Video\n", "import moviepy.editor\n", "moviepy.editor.ipython_display(\"data/demo_spatiotemporal_det.mp4\")" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "ipy_stad", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 0 }