1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
4 <title> HTML Timed Media Elements and CSS Open Issues </title>
6 <link href="./mediaelement.css" rel="stylesheet" type="text/css">
8 <style type="text/css">
9 h4 + .element { margin-top: -2.5em; padding-top: 2em; }
10 h4 + p + .element { margin-top: -5em; padding-top: 4em; }
11 .element { background: #EFE; color: #000; margin: 0 0 1em -1em; padding: 0 1em 0.25em 0.75em; border-left: solid #9F9 0.25em; -padding: 0; /* that last decl is for IE6. Try removing it, it's hilarious! */ }
12 .proposal { border: blue solid; padding: 1em; }
13 table.matrix, table.matrix td { border: none; text-align: right; }
14 table.matrix { margin-left: 2em; }
16 .history table { width: 100%; }
17 .history, .history td, .history th { border: solid thin; font-size: x-small }
18 td.hauthor, td.hdate { width: 10%; }
19 td.hversion { width: 5%; text-align: center; }
20 td.hchange { width: 100%; }
22 /* needed to override wiki CSS */
23 a, a:link { text-decoration: underline;}
32 <h1> Timed Media Elements - Open Issues </h1> <h2 class="no-num no-toc" id="working"> 19 March 2007 </h2>
33 <p class="copyright">© Copyright 2007 Apple Inc. All rights reserved.</p>
37 <h2 class=no-num id=issues>HTML Open Issues</h2>
40 <li>Should there be advisory markup attributes in order to describe
41 content even more precisely, e.g. dataRate?
42 It would be an optional attribute that, if present, would specify the minimum network connection
43 speed (in bits per second) required to use the media resource. The mechanism used to measure
44 bandwidth could determined by the user agent; so some implementations might wish to use a static setting
45 controlled by the user, and some might wish to use the average network throughput to the server hosting the media
46 resource. It would only be used for static fallback.
49 <li class=no-num>Retrieving a HTMLImageElement for the frame at a specific time from a
50 video would be very useful. Would this be API, e.g.
52 <pre>HTMLImageElement getFrameImage(DOMString time)?</pre>
55 or a special URI scheme?
58 <li class=no-num>It may be desirable to sync multiple video and audio elements, and have
59 them keep time with each other.
61 <li class=no-num>Wherever times appear in APIs or CSS properties, it may be convenient to get and set
62 times in other time formats. Formats that may be of particular interest include:
64 <li>SMIL <a href="http://www.w3.org/TR/2005/REC-SMIL2-20050107/smil-timing.html#Timing-ClockValueSyntax"><clock-value></a>
65 <li>"HH:MM:SS.fraction" with hours and the fractional seconds optional
66 <li>frame count index relative to start chapter name and/or index
69 <li class=no-num>In CSS, time formats could have their own function syntax (such as
70 clock("HH:MM:SS.fraction"), etc. In APIs, it would be possible to just accept strings, but
71 that doesn't let you get times in your format of choice. A possible solution is a MediaTime
72 interface which can convert to or from various time formats, and which is passed to and
73 returned from API methods that deal in times.
75 <li class=no-num>What does playbackRate do for audio? Should it be pitch-preserving? Should it
76 skip sections if you are going fast enough?
78 <li class=no-num>A movie is just a description of the multimedia presentation, the actual media sample
79 data is in tracks (or streams). Each track represents a sequence of renderable elements of a
80 uniform type, e.g. audio, video, text, etc. We should expose track information via the API
81 so developers can determine exactly what a media file contains and so they can enable and
82 disable individual tracks to control which take part in the presentation. Tracks contain
83 both metadata (eg. sample rate, sample size, bit depth, etc) and annotations (see notes on
84 media annotations below). We haven't designed an API for this yet. Perhaps track info and
85 metadata should be specified separately.
87 <li class=no-num>API to provide access to embedded metadata. In addition to metadata about media
88 characteristic (frame rate, data size, bit rate, etc), we should provide access to
89 annotation metadata. Annotations in MPEG-4 files can be in several different formats (eg.
90 3GPP, iTunes, QuickTime, ID3v2, etc), need not have unique names, and can be tagged with
91 language, so an API must allow all of them to be specified. Another form of interesting
92 metadata is chapter names and time ranges.
94 <li class=no-num>Should there be policies expressed on how to deal with graceful degradation if an
95 implementation is unable to support some aspect of a feature (e.g., negative playback
96 rates)? This might be a characteristic of the implementation as a whole or the
97 characteristic only when dealing with a particular media format (e.g., negative playback on
98 a RTP stream doesn't work even though for a .mov it could). Or should this (in)ability be
99 discoverable? Some clients that implement video decoding in hardware may not be able to
102 <li class=no-num>How can a client discover the capabilities of the user agent (e.g., what codecs and
103 profiles/levels are supported)?
105 <li class=no-num>It might be useful in the API to get the current frame index, or go to a specific frame
106 (see time format discussion).
110 <h2 id="open-issues"><span class="secno"></span>CSS Open Issues</h2>
113 <li><a href="http://www.w3.org/TR/REC-CSS2/syndata.html#value-def-time">
114 <time></a> values only have second and millisecond unit identifiers. This will
115 make it inconvenient to represent time values in long duration media files because all
116 times must be converted to seconds. It may make more sense to use a time value
117 with a syntax like the SMIL
118 <a href="http://www.w3.org/TR/2005/REC-SMIL2-20051213/smil-timing.html#Timing-ClockValueSyntax">
119 <clock-value></a> type, which allows units of hours, minutes, seconds, and milliseconds,
120 so longer times can be expressed without conversion.</p>
122 <li>It would be useful to extend CSS <a href="http://www.w3.org/TR/css3-mediaqueries/">Media
123 Queries</a> with a new media query feature to make
124 rules that are matched based on bandwidth. This will allow a content author to build
125 pages that select different CSS files based on the user's bandwidth as well as existing
126 media query features like screen size.