> ## Documentation Index
> Fetch the complete documentation index at: https://podonos.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Naturalness

> Evaluate the naturalness of your speeches/audios.

## Intro

One of the popular measuress in the synthesized speech is the naturalness: measuring how natural the synthesized speech is. One of the most popular naturalness evaluation methods for speech/audio is [mean opinion score](https://en.wikipedia.org/wiki/Mean_opinion_score) (MOS). Its scale typically ranges from 1 (lowest naturalness like old robot) to 5 (highest naturalness like human) with 1 granularity (which is called five-point [Likert Scale](https://en.wikipedia.org/wiki/Likert_scale)). Through [podonos](https://pypi.org/project/podonos/), you will evaluate the naturalness of your speech/audio in a fully managed way.

![naturalness](https://static-public.podonos.com/sdk/usecase/0_usecase_naturalness.png)

## Example

Our first example uses [AWS Polly](https://aws.amazon.com/polly/) to generate synthesized human voice
and uses [podonos](https://pypi.org/project/podonos/) for evaluation. Of course, you can use your own TTS (text-to-speech) model, or even your own voice. Here is a code example that you can immediately execute.

```python python theme={null}
import podonos
from podonos import *
import boto3


polly_client = boto3.Session().client('polly')
client = podonos.init()
etor = client.create_evaluator(
    name='nmos_polly',
    desc='naturalness of Polly TTS',
    type='NMOS',
    num_eval=10,
)

scripts = [
    'But in less than five minutes',
    'The two doctors therefore entered the room alone'
]

for script in scripts:
    # Generate the synthesized speech.
    response = polly_client.synthesize_speech(
        VoiceId='Brian', OutputFormat='mp3',
        Text=script, Engine='neural'
    )

    filename = 'my_synthesized_speech.mp3'
    file = open(filename, 'wb')
    file.write(response['AudioStream'].read())
    file.close()
    etor.add_file(File(path=filename, model_tag='AWS-Polly',
                       tags=['polly', 'brian']))

etor.close()
```

Ok, let's go line by line.

<Steps>
  <Step title="Create a Client">
    Let's first create a new instance of `Client`.

    ```python python theme={null}
    client = podonos.init()
    ```
  </Step>

  <Step title="Create an Evaluator">
    Then, you create a new instance of `Evaluator`:

    ```python python theme={null}
    etor = client.create_evaluator(name='nmos_polly',
        desc='naturalness of Polly TTS', type='NMOS', num_eval=10)
    ```
  </Step>

  <Step title="Add files">
    Now, you add every synthesized speech files to the evaluator.

    ```python python theme={null}
    etor.add_file(File(path=filename, model_tag='AWS-Polly',
                       tags=['polly', 'brian']))
    ```
  </Step>

  <Step title="Close">
    Finally, close the `Evaluator` object.

    ```python python theme={null}
    etor.close()
    ```
  </Step>
</Steps>

With this, you can evaluate the naturalness of two synthesized human voices via
[podonos](https://pypi.org/project/podonos/) in 5-point MOS scale from real human evaluators.
By default, 10 humans will evaluate the naturalness of each audio, their results are analyzed.
Once these steps finish, you can check the status in your [Workspace](https://workspace.podonos.com).