V4 0079 using annotations for captions

recipe/0074-multiple-language-captions/index.md

@@ -17,33 +17,29 @@ Captions and subtitles may be available in multiple languages for video content
 
 ## Implementation notes
 
-This recipe builds on the pattern for [Using Caption and Subtitle Files with Video Content][0219], extending it to represent the availability of captions and subtitles in multiple languages.
+This recipe builds on the pattern for [Using Caption Files with Video Content][0219], extending it to represent the availability of captions and subtitles in multiple languages.
 
-Similarly to offering a single caption/subtitle file, the multiple subtitle/caption files are provided as a `supplementing` Annotation on the Canvas that contains the video file. In the case of multiple captions/subtitles, however, the Annotation provides all the available languages as choices. IIIF viewers are then able to offer to end-users the choice for displaying subtitles/captions in one of the available languages during the video playback.
-
-In addition to this implementation, one might consider to provide the captions/subtitles also as multiple timed annotations. Although redundant, providing both implementations will enable more IIIF viewers to display the captions/subtitles, since they may use whichever implementation they support. See [Using Annotations for Timed Text][0079].
+Similarly to offering a single caption file, the multiple subtitle/caption files are provided as `supplementing` Annotations on the Canvas that contains the video file, which also contain the `provides` property with the value `subtitles` or `closedCaptions`. One Annotation is used for each of the available captions/subtitles and languages. IIIF viewers  are then able to identify which annotations contain captions/subtitles by checking the value of the `provides` property, and offer to end-users the choice for displaying subtitles/captions in one of the available languages during the video playback.
 
 ## Restrictions
 
-Formats other than WebVTT (Web Video Text Tracks) are supported by the [Presentation API 3.0](https://iiif.io/api/presentation/3.0/), but current IIIF viewers do not support the display of captions/subtitles. Nevertheless, WebVTT is the format most likely to be implemented.
-
-When using segmented WebVTT with HLS (HTTP Live Streaming), see [Serving HLS Files][0257].
+Formats other than WebVTT (Web Video Text Tracks) are supported by the Presentation API 4.0, but WebVTT is the format most likely to be implemented in current IIIF viewers.
 
 ## Example
 
-In this example we represent a video with subtitles in two languages: English and Italian.
-For expressing the availability of the subtitles in the two languages, we use an Annotation with a Choice body that contains two items, one for each language. The language of each subtitle file is expressed with a `language` property  containing a [BCP 47](https://tools.ietf.org/html/bcp47) language code as specified by the [Presentation API 3.0](https://iiif.io/api/presentation/3.0/#language-of-property-values).
+In this example we represent a video with captions in Italian and subtitles in English. 
+For expressing the availability of captions and subtitles in the two languages, we use two Annotations, one for each language. The language of each caption/subtitle file is expressed with a `language` property containing a [BCP 47](https://tools.ietf.org/html/bcp47) language code as specified by the Presentation API 4.0.
 
-In this example we use subtitle files in the WebVTT format. Other format options include SRT (SubRip Text) or TTML (Timed Text Markup Language), but these are less likely to be supported in IIIF viewers.
+In this example we use files in the WebVTT format. Other format options include SRT (SubRip Text) or TTML (Timed Text Markup Language), but these are less likely to be supported in IIIF viewers.
 
 {% include manifest_links.html viewers="Ramp, Theseus" manifest="manifest.json" %}
 
-{% include jsonviewer.html src="manifest.json" config='data-line="60-92"'%}
+{% include jsonviewer.html src="manifest.json" config='data-line="60-95"'%}
 
 # Related recipes
 
 - [Simplest Manifest - Video][0003]
-- [Using Caption and Subtitle Files with Video Content][0219]
+- [Using Caption Files with Video Content][0219]
 - [Transcripts, Captions, and Subtitles - General Considerations][0231]
 - [Using Annotations for Timed Text][0079]
 - [Providing Access to Transcript Files of A/V Content][0017]

recipe/0074-multiple-language-captions/manifest.json

@@ -45,32 +45,37 @@
                     "id": "{{ id.url }}/anno/page/1",
                     "type": "AnnotationPage",
                     "items": [{
-                            "id": "{{ id.url }}/subtitles_captions-files-vtt",
+                            "id": "{{ id.url }}/subtitles_captions-files-vtt-en",
                             "type": "Annotation",
                             "motivation": "supplementing",
+							"provides": [ "subtitles" ],
                             "body": {
-                                "type": "Choice",
-                                "items": [{
-                                        "id": "{{ id.path }}/Per_voi_signore_Modelli_francesi_en.vtt",
-                                        "type": "Text",
-                                        "format": "text/vtt",
-                                        "label": {
-                                            "en": ["Captions in WebVTT format"]
-                                        },
-                                        "language": "en"
-                                    }, {
-                                        "id": "{{ id.path }}/Per_voi_signore_Modelli_francesi_it.vtt",
-                                        "type": "Text",
-                                        "format": "text/vtt",
-                                        "label": {
-                                            "it": ["Sottotitoli in formato WebVTT"]
-                                        },
-                                        "language": "it"
-                                    }
-                                ]
+                                "id": "{{ id.path }}/Per_voi_signore_Modelli_francesi_en.vtt",
+                                "type": "Text",
+                                "format": "text/vtt",
+                                "label": {
+                                    "en": ["English subtitles in WebVTT format"]
+                                },
+                                "language": "en"
                             },
                             "target": "{{ id.path }}/canvas"
-                        }
+                        },                        
+                        {
+                            "id": "{{ id.url }}/subtitles_captions-files-vtt-it",
+                            "type": "Annotation",
+                            "motivation": "supplementing",
+							"provides": [ "closedCaptions" ],
+                            "body": {
+                                "id": "{{ id.path }}/Per_voi_signore_Modelli_francesi_it.vtt",
+                                "type": "Text",
+                                "format": "text/vtt",
+                                "label": {
+                                    "en": ["Italian captions in WebVTT format"]
+                                },
+                                "language": "it"
+                            },
+                            "target": "{{ id.path }}/canvas"
+                        } 
                     ]
                 }
             ]

recipe/0079-annotations-for-captions/index.md

@@ -0,0 +1,31 @@
+---
+title: Using Annotations for Timed Text
+id: 79
+layout: recipe
+tags: [video, caption, subtitle, presentation]
+summary: "Providing captions, represented as annotations, to a video resource."
+viewers:
+ - Ramp
+ - Theseus
+v4-viewers:  
+ - Mirador
+topic: 
+ - AV
+code:
+ - iiif-prezi3
+top_tabs:
+  - label: Version 4
+    content: "{% capture my_include %}{%- include_relative v4/recipe.md version='4' -%}{% endcapture %}{{ my_include | markdownify }}"
+---
+
+{{ theme.block-center-start }}
+
+{% include blocks/tabs.html  tabs=page.top_tabs %}
+
+{{ theme.block-end }}
+<script>
+  if (!window.location.hash) {
+    let el = document.getElementById("version-3-heading");
+    el.className += " is-active";
+  }  
+</script>

recipe/0079-annotations-for-captions/v4/lunchroom_manners-captions-en.json

@@ -0,0 +1,124 @@
+{
+    "@context": "http://iiif.io/api/presentation/4/context.json",
+    "id": "{{ id.url }}",
+    "type": "AnnotationPage",
+    "items": [
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a1",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "[music]"
+            },
+            "target": "{{ id.path }}/canvas#t=1.200,21.000"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a2",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "Just before lunch one day, a puppet show \nwas put on at school."
+            },
+            "target": "{{ id.path }}/canvas#t=22.200,26.600"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a3",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "It was called \"Mister Bungle Goes to Lunch\"."
+            },
+            "target": "{{ id.path }}/canvas#t=26.700,31.500"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a4",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "It was fun to watch."
+            },
+            "target": "{{ id.path }}/canvas#t=31.600,34.500"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a5",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "In the puppet show, Mr. Bungle came to the \nboys' room on his way to lunch."
+            },
+            "target": "{{ id.path }}/canvas#t=36.100,41.300"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a6",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "He looked at his hands. His hands were dirty \nand his hair was messy."
+            },
+            "target": "{{ id.path }}/canvas#t=41.400,46.200"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a7",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "But Mr. Bungle didn't stop to wash his hands \nor comb his hair."
+            },
+            "target": "{{ id.path }}/canvas#t=46.300,51.100"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a8",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "He went right to lunch."
+            },
+            "target": "{{ id.path }}/canvas#t=51.200,54.900"
+        },
+        {
+            "id": "https://data.europeana.eu/annotation/2051943/data_euscreenXL_EUS_2BAA81D23D6F41578CB2C2071D0325FC/en/a9",
+            "type": "Annotation",
+            "motivation": "supplementing",
+            "provides": [ "closedCaptions" ],
+            "body": {
+                "type": "TextualBody",
+                "format": "text/plain",
+                "language": "en",
+                "value": "Then, instead of getting into line at the \nlunchroom, Mr. Bungle pushed everyone aside \nand went right to the front."
+            },
+            "target": "{{ id.path }}/canvas#t=57.900,65.700"
+        }
+    ]
+}

recipe/0079-annotations-for-captions/v4/manifest.json

@@ -0,0 +1,47 @@
+{
+  "@context": "http://iiif.io/api/presentation/4/context.json",
+  "id": "{{ id.url }}",
+  "type": "Manifest",
+  "label": {
+    "en": [
+      "Lunchroom Manners"
+    ]
+  },
+  "items": [
+    {
+      "id": "{{ id.path }}/canvas",
+      "type": "Canvas",
+      "height": 360,
+      "width": 480,
+      "duration": 572.034,
+      "items": [
+        {
+          "id": "{{ id.path }}/canvas/page",
+          "type": "AnnotationPage",
+          "items": [
+            {
+              "id": "{{ id.path }}/canvas/page/annotation1",
+              "type": "Annotation",
+              "motivation": "painting",
+              "body": {
+                "id": "https://fixtures.iiif.io/video/indiana/lunchroom_manners/high/lunchroom_manners_1024kb.mp4",
+                "type": "Video",
+                "height": 360,
+                "width": 480,
+                "duration": 572.034,
+                "format": "video/mp4"
+              },
+              "target": "{{ id.path }}/canvas"
+            }
+          ]
+        }
+      ],
+      "annotations": [
+        {
+		  "id": "{{ id.path }}/lunchroom_manners-captions-en.json",
+		  "type": "AnnotationPage"
+	    }
+      ]
+    }
+  ]
+}

recipe/0079-annotations-for-captions/v4/recipe.md

@@ -0,0 +1,38 @@
+## Use Case
+
+You want to represent captions for your video content as timed annotations.
+
+## Implementation notes
+
+This recipe follows the pattern for [Using Caption Files with Video Content][0219], but shows an alternative form of representing a caption file using annotations.
+
+In this recipe, annotations are used to mark up video content shown on a canvas of the manifest. The time alignment of the captions with the video content is achieved by annotations that target the canvas by using Media Fragments URIs with a [temporal dimenstion](https://www.w3.org/TR/media-frags/#naming-time). The `provides` property of the Annotation is used by the IIIF client to identify the annotations containing captions and be used as such by the media player. 
+
+## Restrictions
+
+
+
+## Example
+
+In this example we show the same captions and video from [Using Caption Files with Video Content][0219]. but the `annotations` section of the manifest links to a `AnnotationPage` instead of the WebVTT file. For readability, the example contains only the annotations that concern the captions of the first minute of the video. 
+
+The `AnnotationPage` contains one annotation per individual caption entry, and these are provided as `supplementing` annotations on the Canvas that contains the video file. The references to the canvas uses Media Fragment URIs with a begin time and an end time. The annotations also contain the `provides` property with the value `closedCaptions` to inform the IIIF viewer that these annotations are representing captions. The language of the captions is expressed with a `language` property containing a [BCP 47](https://tools.ietf.org/html/bcp47) language code as specified by the Presentation API 4.0.
+
+Note: This recipe may also be applied for subtitles. In such cases, the `provides` property of the annotations must have the value `subtitles`. 
+
+{% include manifest_links.html viewers="Clover, Ramp, Aviary, Theseus" manifest="v4/manifest.json" %}
+
+{% include jsonviewer.html src="v4/manifest.json" config='data-line="39-44"'%}
+
+{% include jsonviewer.html src="v4/lunchroom_manners-captions-en.json" config='data-line="6-18"'%}
+
+# Related recipes
+
+- [Simplest Manifest - Video][0003]
+- [Using Caption Files with Video Content][0219]
+- [Using Caption and Subtitle Files in Multiple Languages with Video Content][0074]
+- [Using Transcript Files with Audio or Video Content][0253]
+- [Providing Access to Transcripts of A/V Content][0017]
+
+{% include acronyms.md %}
+{% include links.md %}