Skip to main content

04.6.9.1 detr-resnet-50 (preview)

Node Description​

detr-resnet-50 (preview) - action type node needed to detect objects in images.

note

Model ID: @cf/facebook/detr-resnet-50. This model takes an image as input and returns a list of detected objects, each containing a label, a probability estimate, and the coordinates of its surrounding rectangle. DEtection TRansformer (DETR) model trained on COCO 2017 end-to-end object detection (118k annotated images).

AI Object Detection is a technology related to computer vision and image processing that consists of detecting objects of certain classes in digital images. Here are examples of how the technology is being used:

  1. Security and video surveillance: detecting suspicious objects in public places or monitoring traffic. Recognizing pedestrians, cars, etc.
  1. Industry and logistics: inspection of production lines for quality control or tracking goods and optimizing warehouse operations.
  1. Retail: analyzing customer behavior and preferences.
  1. Medicine: analyzing medical images to diagnose diseases.
  1. Smart Home and Robotics: recognizing people, objects and actions to automate household tasks.

Node Configuration​

The required Image field is required to configure the detr-resnet-50 (preview) node.

Image​

Field for entering image file content in the format 1.body.files.[0].content.

caution

Content is one of the output parameters of nodes that work with files. A file can also be transferred to the Trigger on Webhook node address.

Example of launching a node​

To process the image and get the list of objects, let's add two nodes to the scenario:

  1. Trigger on Webhook node to start the scenario and transfer the file to the scenario;
  1. Node detr-resnet-50 (preview) to detect objects in the image.

The result of the node execution is JSON:

  • with action execution status "success": true;
  • with detected objects "success", where "box" is the coordinates of the segment with the image, "score" is the confidence level of the model that the detected object is really present in the image, "label" is the name of the object.
JSON (abridged version)
{
  "result": {
    "errors": [],
    "messages": [],
    "result": [
      {
        "box": {
          "xmax": 174,
          "xmin": 53,
          "ymax": 179,
          "ymin": 122
        },
        "label": "car",
        "score": 0.9977849721908569
      },
      {
        "box": {
          "xmax": 27,
          "xmin": 0,
          "ymax": 124,
          "ymin": 96
        },
        "label": "car",
        "score": 0.9821611642837524
      },
      {
        "box": {
          "xmax": 271,
          "xmin": 249,
          "ymax": 159,
          "ymin": 96
        },
        "label": "person",
        "score": 0.9475067853927612
      },
      {
        "box": {
          "xmax": 271,
          "xmin": 250,
          "ymax": 173,
          "ymin": 101
        },
        "label": "person",
        "score": 0.9413467049598694
      },
      {
        "box": {
          "xmax": 198,
          "xmin": 155,
          "ymax": 128,
          "ymin": 42
        },
        "label": "traffic light",
        "score": 0.9313042163848877
      },
      {
        "box": {
          "xmax": 201,
          "xmin": 155,
          "ymax": 142,
          "ymin": 42
        },
        "label": "traffic light",
        "score": 0.9081672430038452
      },
      {
        "box": {
          "xmax": 72,
          "xmin": 33,
          "ymax": 121,
          "ymin": 96
        },
        "label": "car",
        "score": 0.8949986100196838
      },
				...
      {
        "box": {
          "xmax": 22,
          "xmin": 0,
          "ymax": 103,
          "ymin": 46
        },
        "label": "truck",
        "score": 0.01586713269352913
      }
    ],
    "success": true
  }
}