STT longRunningRecognize in Cloud function

​https://gigazine.net/gsc_news/en/20180824-speech-to-text-gcp-cloud-mojiokoshi/​
Based on what I found in the docs, see below, it seems like it could be possible to start the  STT operation via a cloud function, and then check on the results later.
Asynchronous speech recognition starts a long running audio processing operation. Use asynchronous speech recognition to recognize audio that is longer than a minute. For shorter audio, Synchronous Speech Recognition is faster and simpler.
You can retrieve the results of the operation via the google.longrunning.Operations interface. Results remain available for retrieval for 5 days (120 hours). Audio content can be sent directly to Cloud Speech-to-Text or it can process audio content that already resides in Google Cloud Storage. See also the audio limits for asynchronous speech recognition requests.
from https://cloud.google.com/speech-to-text/docs/async-recognize​
This resource represents a long-running operation that is the result of a network API call.
1
{
2
  "name": string,
3
  "metadata": {
4
    "@type": string,
5
    field1: ...,
6
    ...
7
  },
8
  "done": boolean,
9
​
10
  // Union field result can be only one of the following:
11
  "error": {
12
    object (Status)
13
  },
14
  "response": {
15
    "@type": string,
16
    field1: ...,
17
    ...
18
  }
19
  // End of list of possible types for union field result.
20
}
Copied!
done - boolean
If the value is false, it means the operation is still in progress. If true, the operation is completed, and either error or response is available.
Union field result. The operation result, which can be either an error or a valid response. If done == false, neither error nor response is set. If done == true, exactly one of error or response is set. result can be only one of the following:

from https://cloud.google.com/speech-to-text/docs/reference/rest/v1/operations#resource-operation​
1
// Imports the Google Cloud client library
2
const speech = require('@google-cloud/speech');
3
​
4
// Creates a client
5
const client = new speech.SpeechClient();
6
​
7
/**
8
 * TODO(developer): Uncomment the following lines before running the sample.
9
 */
10
// const gcsUri = 'gs://my-bucket/audio.raw';
11
// const encoding = 'Encoding of the audio file, e.g. LINEAR16';
12
// const sampleRateHertz = 16000;
13
// const languageCode = 'BCP-47 language code, e.g. en-US';
14
​
15
const config = {
16
  encoding: encoding,
17
  sampleRateHertz: sampleRateHertz,
18
  languageCode: languageCode,
19
};
20
​
21
const audio = {
22
  uri: gcsUri,
23
};
24
​
25
const request = {
26
  config: config,
27
  audio: audio,
28
};
29
​
30
// Detects speech in the audio file. This creates a recognition job that you
31
// can wait for now, or get its result later.
32
const [operation] = await client.longRunningRecognize(request);
33
// Get a Promise representation of the final result of the job
34
const [response] = await operation.promise();
35
const transcription = response.results
36
  .map(result => result.alternatives[0].transcript)
37
  .join('\n');
38
console.log(`Transcription: ${transcription}`);
Copied!
From this example using await it seems like there isn't a way to get response before the result is ready. Which would not work in a cloud function.
But in in the SDK Client Refernece, SpeechClient, code example https://googleapis.dev/nodejs/speech/latest/v1p1beta1.SpeechClient.html#longRunningRecognize-examples​
SDK
Setup
1
const speech = require('@google-cloud/speech');
2
​
3
const client = new speech.v1p1beta1.SpeechClient({
4
  // optional auth parameters.
5
});
6
​
7
const encoding = 'FLAC';
8
const sampleRateHertz = 44100;
9
const languageCode = 'en-US';
10
const config = {
11
  encoding: encoding,
12
  sampleRateHertz: sampleRateHertz,
13
  languageCode: languageCode,
14
};
15
const uri = 'gs://bucket_name/file_name.flac';
16
const audio = {
17
  uri: uri,
18
};
19
const request = {
20
  config: config,
21
  audio: audio,
22
};
Copied!
The either
1
​
2
// Handle the operation using the promise pattern.
3
client.longRunningRecognize(request)
4
  .then(responses => {
5
    const [operation, initialApiResponse] = responses;
6
​
7
    // Operation#promise starts polling for the completion of the LRO.
8
    return operation.promise();
9
  })
10
  .then(responses => {
11
    const result = responses[0];
12
    const metadata = responses[1];
13
    const finalApiResponse = responses[2];
14
  })
15
  .catch(err => {
16
    console.error(err);
17
  });
18
​
Copied!
responses example, from second then .I've shorten the data in Buffer attributes for brevity.
first element of the response array is the results of GCP STT
second element of the response array is the metadata
third element of the response array is the final api response
1
[
2
  {
3
    "results": [
4
      {
5
        "alternatives": [
6
          {
7
            "transcript": "everyone welcome to the Apple Incorporated fourth quarter of fiscal year 2019 earnings conference call to this call is being recorded at this time for opening remarks and introductions I'd like to turn the call over to Nancy teksten can you collect your industrial Nations please go ahead and look at my history and after that we'll open the call to questions from analysts",
8
            "confidence": 0.9489607214927673
9
          }
10
        ]
11
      },
12
      {
13
        "alternatives": [
14
          {
15
            "transcript": " please note that some of the information you'll hear during our discussion today will consist of forward-looking statements including without limitation those regarding Revenue gross margin operating expenses other income and expense taxes Capital allocation and future business Outlook actual results are trans could differ materially from our forecast for more information please refer to the risk factors discussed in Apple's most recently filed periodic reports on form 10-K and form 10-q and a form 8-k filed with the SEC today along with the Associated Press release Apple assumes no obligation to update any forward-looking statements or information which speak as of their respective dates I'd now like to turn the call over to Tim for introductory remarks good afternoon and thanks to all of you for joining us especially those of you listening in on our new noise cancelling airpods Pro which are available beginning today",
16
            "confidence": 0.9777160882949829
17
          }
18
        ]
19
      },
20
      {
21
        "alternatives": [
22
          {
23
            "transcript": " this was Apple's highest revenue in a September quarter ever and I want to take you through some of the highlights before we get into greater detail on the conclusion of a remarkable physical 2019 for Apple",
24
            "confidence": 0.9561960101127625
25
          }
26
        ]
27
      }
28
    ]
29
  },
30
  {
31
    "progressPercent": 100,
32
    "startTime": { "seconds": "1574440034", "nanos": 902020000 },
33
    "lastUpdateTime": { "seconds": "1574440082", "nanos": 625018000 }
34
  },
35
  {
36
    "name": "59308473068549040",
37
    "metadata": {
38
      "type_url": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata",
39
      "value": {
40
        "type": "Buffer",
41
        "data": [
42
          8,
43
          100,
44
          ...
45
          2
46
        ]
47
      }
48
    },
49
    "done": true,
50
    "response": {
51
      "type_url": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeResponse",
52
      "value": {
53
        "type": "Buffer",
54
        "data": [
55
          18,
56
          ...
57
          63
58
        ]
59
      }
60
    },
61
    "result": "response"
62
  }
63
]
64
​
65
​
Copied!
Or 
1
​
2
// Handle the operation using the event emitter pattern.
3
client.longRunningRecognize(request)
4
  .then(responses => {
5
    const [operation, initialApiResponse] = responses;
6
​
7
    // Adding a listener for the "complete" event starts polling for the
8
    // completion of the operation.
9
    operation.on('complete', (result, metadata, finalApiResponse) => {
10
      // doSomethingWith(result);
11
    });
12
​
13
    // Adding a listener for the "progress" event causes the callback to be
14
    // called on any change in metadata when the operation is polled.
15
    operation.on('progress', (metadata, apiResponse) => {
16
      // doSomethingWith(metadata)
17
    });
18
​
19
    // Adding a listener for the "error" event handles any errors found during polling.
20
    operation.on('error', err => {
21
      // throw(err);
22
    });
23
  })
24
  .catch(err => {
25
    console.error(err);
26
  });
27
​
Copied!
​
initialApiResponseexample
1
{ 
2
    name: '8700551237405696231', 
3
    metadata: null, 
4
    done: false 
5
}
Copied!
Or
1
​
2
// Handle the operation using the await pattern.
3
const [operation] = await client.longRunningRecognize(request);
4
​
5
const [response] = await operation.promise();
Copied!
operations.get
operations.get 

Gets the latest state of a long-running operation. Clients can use this method to poll the operation result at intervals as recommended by the API service.
1
GET https://speech.googleapis.com/v1/operations/{name=**}
Copied!
in the example below, operationNameis the name that you get in initialApiResponse.  while firebaseApiKey you can find it in google cloud console. You do not need this API key if calling this end point within a firebase function.
1
https://speech.googleapis.com/v1/operations/${operationName}?key=${firebaseApiKey}
Copied!

from https://cloud.google.com/speech-to-text/docs/reference/rest/v1/operations/get​
Where name is a string and is 
The name of the operation resource.
Running STT long recognise in cloud function
If the audio takes to long to transcribe and the cloud function times out, from the name attribute of the initialApiResponse it would be possible to do pooling and gets the latest state of a long-running operation. Including result when it's done. via the operations.get.
​
for the firebase API key project settings in console, under web api key
Firebase - What is the API Key
Stack Overflow
​

enableWordTimeOffsets

STT + Cloud Function

Last modified 1yr ago

Copy link

Contents

SDK

operations.get

Running STT long recognise in cloud function