add clips using annotations

[wykhuh]

@adamjarling @mathewjordan Here's a pull request to add clips to videos and audio using annotations. I also added a /demo page and a link to a sample manifest with clips.

Things to discuss

1. Is there a better way to distinguish between VTT captions versus clips? I'm currently using json format for clips.
2. Are there more uses for video annotations besides VTT captions and clips?

Todo:

• add the ability to set the position of the clips relative to the current media (e.g. left, right, top, bottom).

[adamjarling]

@wykhuh Thanks for pushing up. Taking at look at this now.

[adamjarling]

@wykhuh Question, in your clips file (https://seagate.whirl-i-gig.com/admin/service/IIIF/cliplist/representation:253), is this modeled after a standard or spec, or custom to your app/back-end?

[{
"identifier": "3",
"text": "Test clip 1",
"start": "00:00:01.100",
"end": "00:00:02.400"
},
{
"identifier": "4",
"text": "Test 2",
"start": "00:00:20.300",
"end": "00:00:34.200"
}]

This could also be represented by something like?

WEBVTT

00:01.100 --> 00:02.400
- Test clip 1

00:20.300 --> 00:34.200
- Test 2

[wykhuh]

@adamjarling I based the clips json format on the output of parse() from node-webvtt so that I could re-use the existing Menu component. I didn't use VTT since I didn't want the clips data to be used as captions for the video player.

[adamjarling]

@wykhuh Got it. So by default defining "navigational structure" as webvtt, Clover's Viewer automatically syncs the cue to highlighted text in the InformationPanel tab. And this isn't what you're looking for. But this text doesn't necessarily have to be a caption though right? Like it could be arbitrary text? For example...

NOTE 
A cue which is not a caption.  Is this ok?

00:01.100 --> 00:02.400
- I am just good navigation text.  I'm not a caption

Just trying to think through this... 😀

[adamjarling]

@wykhuh Yesterday you were asking about a "more IIIF way" of representing something like navigation for a video, or segments, etc. Another option may be Structures and Ranges (which Clover's Viewer doesn't currently support, but is in our sights). For general info, maybe this is interesting or useful at some point?

https://iiif.io/api/presentation/3.0/#54-range

https://iiif.io/api/presentation/3.0/#53-canvas

Here's an example of a manifest which uses IIIF Structures and Ranges, which a viewer should be able to parse out into navigation links:

https://github.com/samvera-labs/ramp/blob/main/public/manifests/dev/lunchroom_manners.json

Again, not super helpful at the current moment on either of our ends, but while I was thinking about it...

[wykhuh]

@adamjarling The difference between clips and captions is that captions text represent what is being said, while the clips text is added by the people who are inputting the data. It's up to the person inputting the data to determine how many clips they want to have and the text for each clip.

For website visitors, clips have similar functionality as captions in the information panel. As the video plays, the current clip is highlighted. When users click on a clip, then the video will jump to particular timestamp.

I have no preference to whether to use annotations or structures & ranges for clips. We can change our manifest to fit either.

[adamjarling]

@wykhuh I have an idea I'm playing around with. I'll push up a branch tomorrow and maybe we can discuss.