04.6.8.1 whisper (preview)

Node Description

whisper (preview) - action type node needed to generate text based on audio file.

note

Model ID: @cf/openai/whisper. Automatic Speech Recognition (ASR) system trained on 680,000 hours of supervised multilingual and multitasking data.

Automatic Speech Recognition (ASR) is a technology that converts audio speech into text. This technology is used in various applications and scenarios including:

1. Voice Assistants and Voice Control. ASR allows users to interact with devices and programs using their voice, instead of a keyboard or screen. Examples include Siri, Google Assistant, Alexa.

2. Audio and video transcription. ASR helps you automatically create text transcripts of interviews, lectures, meetings, and other audiovisual materials.

3. Subtitling and assisting people with hearing impairments. ASR is used to automatically caption video and audio content in real time.

4. Voice control in systems. ASR allows you to control various systems, devices and applications using your voice, such as controlling a smart home, car, etc.

5. Implementation in contact centers. ASR is used to automatically transcribe conversations with customers, which helps in improving the quality of service.

Node Configuration

The required Audio field is required to configure the whisper (preview) node.

Audio

Field to enter the content of the audio file in the format 1.body.files.[0].content.

caution

Content is one of the output parameters of nodes that work with files. A file can also be transferred to the Trigger on Webhook node address.

Example of launching a node

To get the text generated from the audio file, let's add two nodes to the scenario:

The output of the scenario is JSON:

• with the text "text" generated on the basis of the audio file;

• with status "success": true;

• transcript of audio in WebVTT (Web Video Text Tracks) format, which is used to add text tracks to video or audio files.

00.120 --> 02.480 - A time interval indicating that the text following this time interval should be displayed from 00.120 seconds to 02.480 seconds.

Weather. You can watch the - text that should be displayed at the specified time. In this case it is "Weather. You can watch the".

• with the number of words in the audio file "word_count": 452;

• with an array of words with start and end time stamps for each word.

JSON (abridged version)

{
  "result": {
    "errors": [],
    "messages": [],
    "result": {
      "text": "Weather. You can watch the weatherman on TV to find out what the weather will be like. It might be a nice clear day with no clouds in the sky. The sun might be shining. It could be a cloudy day. Sometimes cloudy days are just dull. On some cloudy days it begins to rain or snow. Some days are rainy. You need a raincoat. I'm Braille and Boots. On a rainy day. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm a Rainy. I'm days are rainy. You need a raincoat. I'm braille and boots on a rainy day. Rain makes the flowers and grass grow. The weather forecast might say that it will be windy. You could have a gentle breeze. It might be very gusty so that the wind pushes you. It is dangerous if the wind is very strong. A hurricane or tornado is very dangerous. Once in a while, the weatherman says, there will be hail. Hailstones are hard, cold palettes of ice that fall from the sky. Sometimes the weatherman will say that there will be snowflaries. Sometimes there is just a light dusting of snow. Sometimes there is a blizzard or a snowstorm. It can be dangerous driving through a blizzard. If there is a lot of snow, the streets have to be plowed. You need a hat, coat, mittens, and boots on a very cold day. Sometimes the weather forecast is wrong. The weatherman might say that it will be a sunny day, but then the clouds come in and it rains. That is not good if you are planning a picnic. I prefer sunny days that are warm but not too hot. I like to feel a gentle breeze to cool me down.",
      "vtt": "WEBVTT\n\n00.120 --> 02.480\nWeather. You can watch the\n\n02.480 --> 04.640\nweatherman on TV to find\n\n04.640 --> 05.840\nout what the weather will\n\n05.840 --> 07.780\nbe like. It might be\n\n07.780 --> 09.200\na nice clear day with\n\n09.200 --> 11.500\nno clouds in the sky.\n\n11.500 --> 13.560\nThe sun might be shining.\n\n13.560 --> 14.540\nIt could be a cloudy\n\n14.540 --> 17.920\nday. Sometimes cloudy days are\n\n17.920 --> 20.420\njust dull. On some cloudy\n\n20.420 --> 22.160\ndays it begins to rain\n\n22.160 --> 24.660\nor snow. Some days are\n\n24.660 --> 27.580\nrainy. You need a raincoat.\n\n27.580 --> 29.680\nI'm Braille and Boots. On\n\n29.680 --> 29.980\na rainy day. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n29.980 --> 29.980\na Rainy. I'm a Rainy.\n\n29.980 --> 29.980\nI'm a Rainy. I'm a\n\n29.980 --> 29.980\nRainy. I'm a Rainy. I'm\n\n24.100 --> 26.440\ndays are rainy. You need\n\n26.440 --> 28.600\na raincoat. I'm braille and\n\n28.600 --> 31.440\nboots on a rainy day.\n\n31.440 --> 33.020\nRain makes the flowers and\n\n33.020 --> 35.980\ngrass grow. The weather forecast\n\n36.420 --> 37.480\nmight say that it will\n\n37.480 --> 40.180\nbe windy. You could have\n\n40.180 --> 42.920\na gentle breeze. It might\n\n42.920 --> 45.000\nbe very gusty so that\n\n45.000 --> 47.900\nthe wind pushes you. It\n\n47.900 --> 49.780\nis dangerous if the wind\n\n49.780 --> 52.600\nis very strong. A hurricane\n\n52.600 --> 56.300\nor tornado is very dangerous.\n\n56.300 --> 57.660\nOnce in a while, the\n\n57.900 --> 59.900\nweatherman says, there will be\n\n59.900 --> 63.080\nhail. Hailstones are hard, cold\n\n63.080 --> 65.220\npalettes of ice that fall\n\n65.220 --> 68.000\nfrom the sky. Sometimes the\n\n68.000 --> 69.460\nweatherman will say that there\n\n69.460 --> 73.220\nwill be snowflaries. Sometimes there\n\n73.220 --> 74.880\nis just a light dusting\n\n74.880 --> 78.240\nof snow. Sometimes there is\n\n78.240 --> 81.300\na blizzard or a snowstorm.\n\n81.300 --> 83.140\nIt can be dangerous driving\n\n83.140 --> 85.460\nthrough a blizzard. If there\n\n85.460 --> 86.760\nis a lot of snow,\n\n86.760 --> 88.300\nthe streets have to be\n\n88.300 --> 91.680\nplowed. You need a hat,\n\n91.680 --> 94.740\ncoat, mittens, and boots on\n\n94.740 --> 97.480\na very cold day. Sometimes\n\n97.480 --> 100.460\nthe weather forecast is wrong.\n\n100.460 --> 102.400\nThe weatherman might say that\n\n102.400 --> 103.320\nit will be a sunny\n\n103.320 --> 104.960\nday, but then the clouds\n\n104.960 --> 107.360\ncome in and it rains.\n\n107.360 --> 108.880\nThat is not good if\n\n108.880 --> 111.180\nyou are planning a picnic.\n\n111.180 --> 113.000\nI prefer sunny days that\n\n113.000 --> 114.380\nare warm but not too\n\n114.380 --> 116.720\nhot. I like to feel\n\n116.720 --> 118.300\na gentle breeze to cool\n\n118.300 --> 119.300\nme down.",
      "word_count": 452,
      "words": [
        {
          "end": 1.2400000095367432,
          "start": 0.11999999731779099,
          "word": "Weather."
        },
        {
          "end": 1.7799999713897705,
          "start": 1.2400000095367432,
          "word": "You"
        },
        {
          "end": 1.9199999570846558,
          "start": 1.7799999713897705,
          "word": "can"
        },
        {
          "end": 2.2200000286102295,
          "start": 1.9199999570846558,
          "word": "watch"
        },
        {
          "end": 2.4800000190734863,
          "start": 2.2200000286102295,
          "word": "the"
        },
        {
          "end": 2.9600000381469727,
          "start": 2.4800000190734863,
          "word": "weatherman"
        },
        {
          "end": 3.299999952316284,
          "start": 2.9600000381469727,
          "word": "on"
        },
        {
          "end": 3.680000066757202,
          "start": 3.299999952316284,
          "word": "TV"
        },
        {
          "end": 4.360000133514404,
          "start": 3.680000066757202,
          "word": "to"
        },
        {
          "end": 4.639999866485596,
          "start": 4.360000133514404,
          "word": "find"
        },
        {
					...
        {
          "end": 119.30000305175781,
          "start": 118.5199966430664,
          "word": "down."
        }
      ]
    },
    "success": true
  }
}