Sesame and ElevenLabs's Naturalness and Quality

Overall

Models

By Tag

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

Tags

Answers

5. Excellent

4. Good

3. Fair

2. Poor

1. Bad

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

120 (48.00%)

110 (44.00%)

17 (6.80%)

3 (1.20%)

0 (0.00%)

70 (28.00%)

119 (47.60%)

47 (18.80%)

13 (5.20%)

1 (0.40%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

120 (48.00%)

70 (28.00%)

Deep analysis

5. Excellent

4. Good

3. Fair

2. Poor

1. Bad

		Tags	Statistics
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	4.40 0.68 0.24
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	4.00 0.00 0.00
00:00 --:-- sesame/speech_3.wav	Sesame (csm-1b)	proper_noun	3.60 0.68 0.24
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	3.60 1.11 0.40
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	3.80 1.36 0.49
00:00 --:-- sesame/speech_6.wav	Sesame (csm-1b)	proper_noun	4.60 0.68 0.24
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	4.20 0.56 0.20
00:00 --:-- sesame/speech_8.wav	Sesame (csm-1b)	proper_noun	4.60 0.68 0.24

Answers

A1. Inaccurate pronunciation

A2. Some words are not spoken

A3. The speaking tone isn't well aligned with the context

A4. The speech is too slow or too fast

A5. Sudden changes in emotion

A6. None of the above

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

18 (7.06%)

3 (1.18%)

12 (4.71%)

20 (7.84%)

19 (7.45%)

183 (71.76%)

37 (12.59%)

22 (7.48%)

43 (14.63%)

50 (17.01%)

23 (7.82%)

119 (40.48%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

18 (7.06%)

37 (12.59%)

Deep analysis

A1. Inaccurate pronunciation

A2. Some words are not spoken

A3. The speaking tone isn't well aligned with the context

A4. The speech is too slow or too fast

A5. Sudden changes in emotion

A6. None of the above

Answers

		Tags
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	None of the above The speech is too slow or too fast
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	None of the above Some words are not spoken Inaccurate pronunciation 2 more...
00:00 --:-- sesame/speech_3.wav	Sesame (csm-1b)	proper_noun	The speech is too slow or too fast None of the above Some words are not spoken
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	The speaking tone isn't well aligned with the context None of the above The speech is too slow or too fast
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion The speaking tone isn't well aligned with the context
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion The speech is too slow or too fast 1 more...
00:00 --:-- sesame/speech_8.wav	Sesame (csm-1b)	proper_noun	Inaccurate pronunciation None of the above
00:00 --:-- sesame/speech_9.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion

Answers

A1. Background noise

A2. Inconsistent volume (e.g., too loud or too soft)

A3. Cut out or drop abruptly at times

A4. Distorted audio affecting quality (hoarse, metallic, or crackling voice)

A5. Unrelated noise (clicks, beeps, unnatural lip noises)

A6. None of the above

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

2 (0.80%)

17 (6.77%)

12 (4.78%)

6 (2.39%)

5 (1.99%)

209 (83.27%)

31 (11.19%)

30 (10.83%)

24 (8.66%)

21 (7.58%)

16 (5.78%)

155 (55.96%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

2 (0.80%)

31 (11.19%)

Deep analysis

A1. Background noise

A2. Inconsistent volume (e.g., too loud or too soft)

A3. Cut out or drop abruptly at times

A4. Distorted audio affecting quality (hoarse, metallic, or crackling voice)

A5. Unrelated noise (clicks, beeps, unnatural lip noises)

A6. None of the above

Answers

Background noise

Inconsistent volume (e.g., too loud or too soft)

Cut out or drop abruptly at times

Distorted audio affecting quality (hoarse, metallic, or crackling voice)

Unrelated noise (clicks, beeps, unnatural lip noises)

None of the above

		Tags
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	None of the above Background noise
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	None of the above Inconsistent volume (e.g., too loud or too soft)
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	Background noise Cut out or drop abruptly at times None of the above 1 more...
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	Background noise None of the above Unrelated noise (clicks, beeps, unnatural lip noises)
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	None of the above Inconsistent volume (e.g., too loud or too soft)
00:00 --:-- sesame/speech_10.wav	Sesame (csm-1b)	proper_noun	Background noise Unrelated noise (clicks, beeps, unnatural lip noises) None of the above
00:00 --:-- sesame/speech_11.wav	Sesame (csm-1b)	number	None of the above Distorted audio affecting quality (hoarse, metallic, or crackling voice) Cut out or drop abruptly at times
00:00 --:-- sesame/speech_12.wav	Sesame (csm-1b)	number	None of the above Background noise Cut out or drop abruptly at times

Overall

Models

By Tag

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

Tags

Answers

5. Excellent

4. Good

3. Fair

2. Poor

1. Bad

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

120 (48.00%)

110 (44.00%)

17 (6.80%)

3 (1.20%)

0 (0.00%)

70 (28.00%)

119 (47.60%)

47 (18.80%)

13 (5.20%)

1 (0.40%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

120 (48.00%)

70 (28.00%)

Deep analysis

5. Excellent

4. Good

3. Fair

2. Poor

1. Bad

		Tags	Statistics
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	4.40 0.68 0.24
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	4.00 0.00 0.00
00:00 --:-- sesame/speech_3.wav	Sesame (csm-1b)	proper_noun	3.60 0.68 0.24
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	3.60 1.11 0.40
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	3.80 1.36 0.49
00:00 --:-- sesame/speech_6.wav	Sesame (csm-1b)	proper_noun	4.60 0.68 0.24
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	4.20 0.56 0.20
00:00 --:-- sesame/speech_8.wav	Sesame (csm-1b)	proper_noun	4.60 0.68 0.24

Answers

A1. Inaccurate pronunciation

A2. Some words are not spoken

A3. The speaking tone isn't well aligned with the context

A4. The speech is too slow or too fast

A5. Sudden changes in emotion

A6. None of the above

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

18 (7.06%)

3 (1.18%)

12 (4.71%)

20 (7.84%)

19 (7.45%)

183 (71.76%)

37 (12.59%)

22 (7.48%)

43 (14.63%)

50 (17.01%)

23 (7.82%)

119 (40.48%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

18 (7.06%)

37 (12.59%)

Deep analysis

A1. Inaccurate pronunciation

A2. Some words are not spoken

A3. The speaking tone isn't well aligned with the context

A4. The speech is too slow or too fast

A5. Sudden changes in emotion

A6. None of the above

Answers

		Tags
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	None of the above The speech is too slow or too fast
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	None of the above Some words are not spoken Inaccurate pronunciation 2 more...
00:00 --:-- sesame/speech_3.wav	Sesame (csm-1b)	proper_noun	The speech is too slow or too fast None of the above Some words are not spoken
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	The speaking tone isn't well aligned with the context None of the above The speech is too slow or too fast
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion The speaking tone isn't well aligned with the context
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion The speech is too slow or too fast 1 more...
00:00 --:-- sesame/speech_8.wav	Sesame (csm-1b)	proper_noun	Inaccurate pronunciation None of the above
00:00 --:-- sesame/speech_9.wav	Sesame (csm-1b)	proper_noun	None of the above Sudden changes in emotion

Answers

A1. Background noise

A2. Inconsistent volume (e.g., too loud or too soft)

A3. Cut out or drop abruptly at times

A4. Distorted audio affecting quality (hoarse, metallic, or crackling voice)

A5. Unrelated noise (clicks, beeps, unnatural lip noises)

A6. None of the above

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

2 (0.80%)

17 (6.77%)

12 (4.78%)

6 (2.39%)

5 (1.99%)

209 (83.27%)

31 (11.19%)

30 (10.83%)

24 (8.66%)

21 (7.58%)

16 (5.78%)

155 (55.96%)

Tags

All tags

Eleven Labs (multilingual_v2)

Sesame (csm-1b)

2 (0.80%)

31 (11.19%)

Deep analysis

A1. Background noise

A2. Inconsistent volume (e.g., too loud or too soft)

A3. Cut out or drop abruptly at times

A4. Distorted audio affecting quality (hoarse, metallic, or crackling voice)

A5. Unrelated noise (clicks, beeps, unnatural lip noises)

A6. None of the above

Answers

Background noise

Inconsistent volume (e.g., too loud or too soft)

Cut out or drop abruptly at times

Distorted audio affecting quality (hoarse, metallic, or crackling voice)

Unrelated noise (clicks, beeps, unnatural lip noises)

None of the above

		Tags
00:00 --:-- sesame/speech_1.wav	Sesame (csm-1b)	proper_noun	None of the above Background noise
00:00 --:-- sesame/speech_2.wav	Sesame (csm-1b)	proper_noun	None of the above Inconsistent volume (e.g., too loud or too soft)
00:00 --:-- sesame/speech_4.wav	Sesame (csm-1b)	proper_noun	Background noise Cut out or drop abruptly at times None of the above 1 more...
00:00 --:-- sesame/speech_5.wav	Sesame (csm-1b)	proper_noun	Background noise None of the above Unrelated noise (clicks, beeps, unnatural lip noises)
00:00 --:-- sesame/speech_7.wav	Sesame (csm-1b)	proper_noun	None of the above Inconsistent volume (e.g., too loud or too soft)
00:00 --:-- sesame/speech_10.wav	Sesame (csm-1b)	proper_noun	Background noise Unrelated noise (clicks, beeps, unnatural lip noises) None of the above
00:00 --:-- sesame/speech_11.wav	Sesame (csm-1b)	number	None of the above Distorted audio affecting quality (hoarse, metallic, or crackling voice) Cut out or drop abruptly at times
00:00 --:-- sesame/speech_12.wav	Sesame (csm-1b)	number	None of the above Background noise Cut out or drop abruptly at times