Audio / Video 2.0 – codename “Disco Fries”
Some history:
I wanted to do a Make post on my wants for Audio / Video in 3.9 to solicit feedback and spark some discussion about what the community wants / needs / doesn’t want / doesn’t need. Adding audio / video in 3.6 was a great first step, but there are some things we can do to continue to modernize Media and give our huge user base even more ways to display and manage their content. We can also make some changes that help developers navigate the new world of MediaElement.js, Backbone, and Underscore.
First Things First: New Icons
#26650 Replace media file type icons with Dashicons
There are some lingering icons in the admin that don’t look as pretty as their MP6ify’d brethren
Document the “new” Media code introduced in 3.5
In 3.5, we overhauled Media. @Koop produced some beautiful code, and a LOT of it. Raise your hand if you’ve ever dived in and tried to program against it? Raise you hand if you understand how all of it works? Me neither. As a community, we need to help each other learn what it is and what it does. Documentation will go a long way in getting us all on the same page. Do we have a documentation standard for JS? We need one. While this isn’t the easiest place to start, it is a necessary one. I would be happy to spend time on this, as I have spent many hours recently reading the code and learning how it works. The main files: media-editor.js, media-views.js, media-models.js
Support subtitles for Video
#26628 Use the content of a video shortcode when provided.
This ticket speaks for itself, and already has a patch.
Generate audio/video metadata on-demand
#26825 Add ability to generate metadata for audio / video files on-demand
Add “Playlist” and “Video Playlist” shortcodes
#26631 Add a “playlist” shortcode
Adding inline players for audio and video was a great first step. How do I add music to my site? Just upload an MP3 and drop the URL on a line by itself. Done. Or use the audio shortcode. This works most of the time, but can be a little clunky if you want to share an album of your tunes. MediaElement doesn’t “support” playlists out of the box, but MediaElement is JavaScript, and with JavaScript and little UI elbow grease, we can EASILY support playlists.
My ticket already contains a patch, but is still considered a work in progress. I think the playlist shortcode should produce markup that does the following:
- Works out of the box with any existing theme: the HTML should be semi-bulletproof. Many of the Player libraries make heavy use of DIVs instead of items that might be overridden easily with CSS: LIs and the like.
- Gives the developer total control if they want to write their own implementation
- Exposes enough data to the page so the themer/dev can make their own decision regarding display of album cover, track meta, captions, etc.
My current implementation drops data onto the page for each playlist inline. A wrapper div “.wp-playlist” will have a script tag in it with type=”application/json”. I do this so that if ‘wp-playlist.js’ is unenqueue’d, the developer still has the data necessary to write their own implementation. The data is reachable in JS like so:
var data = $.parseJSON( el.find('script').html() );
My current UI for playlist is a basic one, and uses Backbone Views to render the tracklist on load and update the “current” playing track view. There are 2 camps of people when it comes to “JS on the frontend” – one who doesn’t like it (others) and one who says “who cares” (me). One of the reasons I am posting this at the beginning is so we can flesh out issues like this.
Abstract Gallery logic into “Collection” logic that Galleries, Playlists, etc can use with minimal registration code
I have already done a first pass at this in the playlist shortcode patch. It goes like this: a “gallery” is really a “collection” of attachments of type “image.” A “playlist” is really a “collection” of attachments of type “audio.” So they should extend or be instances of a “collection”-type class. Currently, the Gallery code has to be dupe’d. By abstracting this code, Gallery, Playlist, Video Playlist, + any other “collection” of media type can be registered minimally.
Other Ideas
- In our playlist JS code, emit events that others can hook into – maybe a video playlist is: News clip, ad, news clip, ad, etc. When emitting events before / after an ad, the dev could disable next/prev buttons
- Make a playlist embeddable on other sites via an iframe or embedded markup
- Register an endpoint for audio / video that will expose the “embed code” via oEmbed
Manny Fleurmond, 
Closing out tickets that shouldn’t be open any longer is always a fine idea! The keywords “close”, “2nd-opinion”, “reporter-feedback” are all good for this. Some kind of combination of 
Joe Dolson 7:00 pm on January 13, 2014 Permalink | Log in to Reply
Definite thumbs up from the Accessibility team for adding captions support. In addition to that, I’d like to suggest making it possible to enable keyboard accessibility more easily than it is right now. MediaElement.js includes settings which enable this — one is enabled by default, which enables keyboard access to the controls, but without ‘alwaysShowControls’ enabled, it’s not possible for a keyboard dependent user to move focus onto the player, so they can’t take advantage of any of those controls.
I’d like to see a localization variable so that the MediaElement settings can be adjusted via wp_localize_script, but for keyboard accessibility it may be valuable to make this an option, either via shortcode or settings, so it’s possible for non-programmers to enable keyboard accessibility.
I’ve got a plug-in, Accessible Video Library that hacks in support for alwaysShowControls by deregistering the default wp-mediaelement script, but that’s not an ideal solution.
It would also be nice if the caption selector within MediaElementjs could be made keyboard accessible; though that’s probably something that should be handled as a patch to MediaElementjs itself.
The Add Media Panel, in addition to new documentation, is in desperate need of accessibility work. See #23560. And several other tickets; there are quite a few accessibility tickets on the Add Media Panel.
Aaron Jorbin 7:03 pm on January 13, 2014 Permalink | Log in to Reply
RE: JavaScript Docs.
the JSDoc standard is pretty close to the phpdoc standard that we use for PHP and thus it makes the most sense in my eyes. There was a passing IRC conversation between members of the team that did the jshint work and the inline documentation that a future project could focus on improving the documentation of our JS. The media files seem like a great place to prototype this.
http://usejsdoc.org/ is the standard.
Eric Andrew Lewis 8:02 pm on January 13, 2014 Permalink | Log in to Reply
Yes to inline docs. However, I think we’re going to need an alternate form of documentation completely separate from inline to give an easily understandable overview of how media works.
Sam Sidler 8:37 pm on January 13, 2014 Permalink | Log in to Reply
What did you have in mind?
The inline docs will soon (ish) be present on developer.wordpress.org, which will help the visibility of them. Do you mean something more like a walkthrough on exactly how it works? I’m trying to think of where the best place for that would be. Definitely on the developer hub, but probably not part of the theme/plugin handbooks.
Eric Andrew Lewis 10:24 pm on January 13, 2014 Permalink | Log in to Reply
Not sure to be honest. I think there are limits to inline documentation. Inline is always so contextual to the code it surrounds; separate documentation can speak of overarching design principles and secondary information that doesn’t pertain to any code block in particular.
WP-API’s documentation for example is quite verbose, and covers a range of topics including the API’s philosophy, tutorials, schema details, etc. None of this could fit easily into inline documentation.
Full disclosure: I’ve only used the media library cursorily, and am not even sure what secondary documentation for it might look like, so I may be off-base here.
Aaron Jorbin 8:45 pm on January 13, 2014 Permalink | Log in to Reply
I agree that some good tutorials and foundation docs are important, but I think that inline docs can help move us forward.
Eric Andrew Lewis 10:29 pm on January 13, 2014 Permalink | Log in to Reply
Definitely. I’m in for an inline doc-athon anytime after this week.
two7s_clash 7:17 pm on January 13, 2014 Permalink | Log in to Reply
I wanted to point out that for a few years the flash player did support playlists: https://plugins.trac.wordpress.org/ticket/1869/ See also: http://jamesfishwick.com/3-6-audio-so-what-happens-to-blogs-using-the-old-jetpack-shortcode-for-generating-playlists/ Furthermore, I had a popular plug-in that did many of these things you list here called Jetpack Easy Playlists: http://jamesfishwick.com/software/jetpack-easy-playlists/ It worked on your “gallery” paradigm. The playlist still works if you hack Jetpack’s shortcode module and disable its check. If you want this to work with 3.6 and up, edit this file: /wp-content/plugins/jetpack/modules/shortcodes.php
Change line 51 to: “if ( version_compare( $wp_version, ’3.9-z′, ‘>=’ ) && stristr( $include, ‘audio.php’ ) )”
@ubernaut 7:58 pm on January 13, 2014 Permalink | Log in to Reply
i think this all sounds awesome.
s3w47m88 8:02 pm on January 13, 2014 Permalink | Log in to Reply
We provide custom Theme WordPress websites to our customers and the primary challenge we face, regarding media, is that customers frequently are embedding video from sources beside the big guys (YouTube, Vimeo, etc…) so they have HTML or something. But that requires them to have access to the HTML (insert blood curdling scream here). What would be a potential solution is a simple widget on the right of Pages/Posts that allows them to paste whatever code they have, then see a rendered version without reloading the page, then the ability to drag that rendered video sample into the Visual Editor.
As much as I like shortcode, customers still don’t get it for some reason. Anything they hear the word “code” they freak.
Sam Sidler 9:16 pm on January 13, 2014 Permalink | Log in to Reply
What you’re talking about sounds a lot like the CEUX project (currently on hiatus).
Scott Taylor 9:18 pm on January 13, 2014 Permalink | Log in to Reply
the shortcodes get inserted automatically from the Media Library, don’t have to be written by hand – they are just the easiest way to save the most minimal data necessary in the post content without hardcoding markup or URLs
Tom Auger 9:31 pm on January 13, 2014 Permalink | Log in to Reply
Sounds like a pretty wide scope of things. I’m primarily interested in extending the Media Editor, and have prepared a feature-as-plugin to this effect as a proof-of-concept. As I missed the deadline for 3.9, I was planning on waiting until the next round to bring this forward, but if work is being done in media-editor.js and friends, perhaps this is a good time to look at the challenges that the current architecture of the Media Editor presents, and my proposed solutions to it.
Manny Fleurmond 4:35 am on January 14, 2014 Permalink | Log in to Reply
I was planning on tackling the media editor as well, specifically making it more modular and using Backbone. Would love to see what you have so far
mttktz 2:44 am on January 14, 2014 Permalink | Log in to Reply
Half of what’s important in WP is making sure that it is easy to create great stuff and publish it on the web.
The other half is people finding that great stuff. Right now other platforms do a better job of sharing and redistributing great stuff. Better or more sophisticated oembed would probably mean more WP posts that look “right” when someone shares them on another site or tries to link to them.
That sounds really interesting to me – but I’m not sure what you are thinking about there.