Tag Archives: phase 3

Anthony Burchell 4:51 pm on January 17, 2024
Tags: , , , , phase 3   

Phase 3 Media Meeting: February 7, 2024

Date and time: February 7, 2024 at 24:00 UTC
Location: #core-media
Duration: 1:00 hr

The Media Component team is coordinating a meeting with the Editor team and other interested stakeholders to work on planning for the proposed Phase 3 Media Library. There are a few issues already open about the Media Library proposal, but this major development depends on close coordination between the Media and Editor teams to work effectively together.

In a recent Media Component meeting it was suggested that there should be a dedicated meeting time to kick off a round table discussion about some of the opportunities and challenges ahead in the development of Phase 3. The Media Component Maintainers hope to find areas of focus where contributors can lean in to align efforts with folks working on Phase 3.

In previous meetings the team has gone through the backlog of over 600 issues and categorized tickets as phase-3-media-triage for discussion around how Phase 3 could potentially solve these tickets alongside a rebuild of the Media Library. Some of these tickets are an opportunity to change lower level issues like db structure, but the redesign is a good time to address these possibilities.

Relevant Pull Requests, Issues, and Posts:

Phase 3: Collaboration > Media Library

PR Dataview: media view

Media Library

Proposed meeting agenda:

  1. Introductions
  2. Brief presentation on the goals and vision of Phase 3 redesign for the Media Library
  3. Discussion on alignment of CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. Media focus to help the process
  4. Agree on communication channels and checkpoints for regular updates
  5. Review of Phase-3-Triage Tickets

A more detailed agenda will be published prior to the meeting.

If you would like to join this meeting please join the #core-media channel February 7, 2024 at 24:00 UTC. This time seems to align with most of the world as a decent hour to meet for stakeholders and folks active in Media.

This meeting happens in the #core-media channel. To join the meeting, you’ll need an account on the Make WordPress Slack. Props @joedolson for helping prepare and proofread this post.

#agenda, #media, #media-library, #meeting, #phase-3

Simon Dickson 10:50 am on January 8, 2024
Tags: , phase 3,   

Phase 3 Agency Outreach Recap, December 2023

Phase 3’s focus on collaboration, revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. and workflow takes GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ further into enterprise territory. Inspired by @annezazu’s earlier outreach exercise, and following up on my own proposal, I took the opportunity of availability during December 2023 to reach out to WordPress agencies specialising in enterprise projects, inviting them to a series of informal show-and-tell sessions.

The aim was to gain an insight into enterprise clients’ requirements, and how agencies currently address them. Sessions were offered on a semi-confidential ‘Chatham House Rule’ basis, to avoid any concerns around client confidentiality; but I committed to publishing a summary of what we saw and learned.

Take-up was disappointing.

Invites were sent via several different outreach channels, chosen primarily based on the personal connections of the people involved: WordPress VIP’s partner networknetwork (versus site, blog), the Enterprise channel in CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/., and the News Nerdery community. This ought to have touched around 100 enterprise-centric agencies. I scheduled 9 one-hour sessions over a 10-day period, at times to suit all timezones. There was capacity for around 40 agencies to participate; but in the end, fewer than 5 agencies signed up. A few others indicated an interest, but did not get round to signing up.

There are many possible explanations. Perhaps December was simply poor timing. Our call for presentations was kept deliberately open and vague; perhaps it should have been more specific. Informal walkthroughs were requested, to minimise the burden of planning and preparation, but perhaps folks would have been more comfortable giving formal presentations.

What is clear is that Core currently lacks warm and clear communication channels with the agency ecosystem – at a time when such channels would be particularly useful for both parties.

It also follows that the following observations are based on a smaller snapshot of enterprise usage than was hoped for. 

Enterprise development practice moves slowly.

Agencies and their clients now seem comfortable with blocks, as delivered in Phase 1: this is largely backed up by data in the recent SOEWP survey.

But we saw no examples yet of Phase 2-style ‘site customisation’: agencies are still sticking with familiar models based on site metadata and Settings pages, even when Phase 2 functionality would probably deliver a better outcome.

An enterprise may only revisit its tech stack every 3 or 4 years, with no reason to fix what isn’t broken. Agencies are probably working on multiple enterprise contracts simultaneously, theoretically giving them more opportunities to embrace new possibilities; but with a decade of pre-Gutenberg ‘muscle memory’, they have a commercial incentive to stick with what they know until proven otherwise.

Collaboration will inevitably happen outside WordPress.

Enterprise employees spend their entire day in office suites like Google Docs or Microsoft 365; social media planning solutions; general collaboration platforms like Jira or Asana; industry specific tools like Desk-Net for newsrooms; technical platforms like GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/.

If businesses already have these tools, integrated into their company-wide workflow, for tasks beyond website management, it’s a bigger challenge for WordPress to force its way into the picture.

Collaboration also happens before WordPress arrives on the scene, in the definition of a ‘house style’, or page templates, which are then hard-coded into the WordPress workflow. Web content creators get limited flexibility, or perhaps none at all, simply filling in boxes. This may seem like the antithesis of user empowerment. But:

  • The business gets predictable, consistent results, always on-brand, delivered efficiently.
  • Employees’ main concern is to get the job done quickly, and move on to their next task.
  • Agencies can tailor UIUI User interface and UXUX User experience to very specific outcomes, minimising the support burden.

Real-time content collaboration is not a current user priority.

We saw no ‘cowpaths to be paved’ when it comes to real-time collaboration on normal content.

The closest we have come has been from an agency working with large news organisations: they described multiple people swarming on a single piece of content, but working on different aspects of it. There would only be one author writing in ‘the content area’: but they would be supported by a picture editor, an SEO manager, a headline manager, a ‘homepage manager’ deciding where it will appear. (This is broadly in line with @annezazu’s earlier findings.)

But some content types are, by definition, real-time and multi-author. One agency brought up the example of ‘live blogs’. If a news site runs ‘live blogs’, typically for major ‘breaking news’ events or sports coverage, these will usually be among their most popular articles of the day. (Source: Press Gazette, 2023.)

Publishers dislike ‘live blogblog (versus network, site)’ solutions embedded from third-party platforms, which do not deliver search-engine benefits. We saw one custom WP-based solution running in the back end, which suffered from the current inability to have multiple authors on the same edit screen. We also know of front-end based, Gutenberg-compatible liveblogs which allow for multiple authors and more creative content entry methods.

We certainly heard a desire for multiple users to collaborate around a single content item. But ‘Google Docs in the editor component’ is not the only possible meaning of collaborative creation… and may not even be the main one.

Revisions need work.

The current implementation of Revisions predates blocks, and is barely usable given the amount of code now held in post_content.

One agency demonstrated work they have done for a major European newspaper, refined over several years, which offers blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience.-compatible WYSIWYGWhat You See Is What You Get What You See Is What You Get. Most commonly used in relation to editors, where changes made in edit mode reflect exactly as they will translate to the published page. comparison, with red and green highlighting of changes. They are open to the idea of sharing their code with Core; and are investigating how this might happen.

It was noted that collaborative editing may have unintended consequences for the current Revisions model. Right now, when only one user can edit and save at any one time, we know exactly who made each change, and when. This can be important for corporations in regulated industries. We lose that simple attribution to an individual, if multiple users are editing simultaneously. (Also noted in @leogermani‘s GitHub discussion thread.)

In conclusion

  • Core currently lacks channels for communicating and engaging with the agency ecosystem.
  • Collaboration on aspects of the website often happens away from the website, and involves people who never touch the website.
  • WordPress cannot realistically be the sole collaboration space for many enterprises and publishers, even as relates to a WordPress-based publishing workflow.
  • The ability to collaborate across different components of a post is currently a greater priority than collaborating within the post_content itself.
  • Beware of unintended consequences – for example, collaborative editing breaking our current Revisions model.

Where next?

Outside of immediate steps I can take, like supporting the agency mentioned to package its work on revisions for Core or arranging future sessions with agencies who expressed interest, we wanted to open the floor to hear from others around ways to improve uptake for enterprise agencies.

  • What do you think would help?
  • What should we try next to spark these conversations and bring feedback into this next phase?

My thanks to @annezazu and various members of the Core team for participating in this initiative; and to enterprise agency Big Bite for sponsoring my time in organising and running these sessions.

#feedback, #phase-3, #summary

annezazu 8:11 am on October 23, 2023
Tags: , phase 3,   

Summary and Insights of Phase 3 related conversations

Over the last few months, I’ve had 20+ conversations with various folks across the WordPress ecosystem, from developers implementing WordPress at scale to folks working in large newsrooms to one-person operations. After leaving comments on the various Phase 3 posts, it felt advantageous to write a quick summary of the feedback to help inform our future work, akin to the efforts of the FSE Outreach program, and encourage others who might be having similar conversations to continue to share for the entire project to benefit from. 

Overview

Most conversations lasted about 45-60 minutes, with very few running into the two-hour range.  The conversations were open ended with folks broadly sharing their content process from ideation to completion and ended talking about the various phase 3 related posts with encouragement to engage. Of the 24 conversations, I have roughly grouped them across the following categories:

  • 3 individual developers with either experience implementing WordPress at scale or building collaborative tooling.
  • 1 person representing higher education.
  • 2 people running their own individual projects/sites.
  • 10 larger newsrooms or organizations.
  • 8 smaller newsrooms or organizations. 

These were sourced from a variety of connections including: GitHubGitHub GitHub is a website that offers online implementation of git repositories that can easily be shared, copied and modified by other developers. Public repositories are free to host, private repositories require a paid subscription. GitHub introduced the concept of the ‘pull request’ where code changes done in branches by contributors can be reviewed and discussed before being merged be the repository owner. https://github.com/ issue creators, WordPress VIP intros, Newspack clients, “cold emailing”, folks from the FSE Outreach Program, and more. In full disclosure, I am sponsored by Automattic to do this work and leveraged these connections to reach out to a broader range of folks. For many, they are using a suite of tools to get the job done, often without deep integration, and usually landing in WordPress when the copy is nearly complete. Phase 3 would/could shift that experience, allowing folks to land and work within WordPress sooner and with that comes a list of features and requirements. Read on!

Collaborative editing & asynchronous collaboration 

At a high level, I repeatedly heard from folks in larger organizations more of an interest in collaboration in the form of the ability for multiple people to be in different parts of an article over true collaborative editing. Especially for larger to medium-sized publishers, there’s less of a need for real-time collaboration and more of a need for different sets of folks in an article simultaneously with blocks locked individually. The same applies to the experience of editing the details of a post, like a categoryCategory The 'category' taxonomy lets you group posts / content together that share a common bond. Categories are pre-defined and broad ranging. or featured imageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts. while not having access to edit the entire post itself. For example, someone from the photo team can place images without having the ability to edit the content blocks, or a handful of folks being able to edit tags directly without getting in the way of the writers. 

  • Multiple people editing a doc at once.
  • Role-specific permissions allowing some folks to accept/deny various edits.
  • More granular ability to edit specific parts of a post/page at once without having full access to everything, including blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. leveling locking.
  • Commenting functionality, including the ability to @ folks.
  • Public preview link for folks, regardless of whether they have access to the site. 

Workflows

No one I spoke with manages everything in WordPress with a wide range of additional tools and workflows in use from extraordinarily complex and manual to some setups with deeper automatic integration (status changes in X which automatically updates Y at the next step) to more manual organizing before adding anything into WordPress (adding images to a folder on a desktop before uploading). A need for a dashboard/task manager is clear as well as a need for integration with other tooling. 

  • Support for tasks that can be customized depending on the post type and have a built-in “severityseverity The seriousness of the ticket in the eyes of the reporter. Generally, severity is a judgment of how bad a bug is, while priority is its relationship to other bugs.” type (required vs nice to have). For example, a task might be more of a nice to have than a necessity, but both are worth calling out for the person to act on. 
  • Seeing changes that have occurred since you last opened a document (related to my note on revisions).
  • Tie notifications into different tasks and the status of articles to reduce the need to pingPing The act of sending a very small amount of data to an end point. Ping is used in computer science to illicit a response from a target server to test it’s connection. Ping is also a term used by Slack users to @ someone or send them a direct message (DM). Users might say something along the lines of “Ping me when the meeting starts.” someone in a separate system. For example, perhaps legal needs to be emailed about an article, or the media team needs to be notified that a post is ready for images.
  • Ability to see open tasks/open notifications with ease (task manager), along with an at-a-glance view of where something might stand (editorial calendar).
  • Section level locking for different parts of a post, like the post content vs. post title vs. featured image, tied into task management with perhaps reviews needed for each before publishing.
  • Solid integration options with other tools, like SlackSlack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/./TrelloTrello Project management system using the concepts of boards and cards to organize tasks in a sane way. This is what the make.wordpress.com/marketing team uses for example: https://trello.com/b/8UGHVBu8/wp-marketing./Asana, for notifications and state changes (draft to pending review, reminding folks about assignments, etc).
  • Easy draft sharing for varying permission levels including stakeholders without access to the site itself.
  • Ability to publish with multiple authors.

RevisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision.

The most commonly requested items proved to be more visual revisions and the ability to track more than just post content changing. These stood out amongst the rest as the most broadly useful items. 

  • More visual revision history, perhaps with the option to toggle between code views.
  • Ability to track more than just post content, including changes to tags, a featured image, SEO, and other custom fields (like managing multiple headlines).
  • Ability to split out a revision to develop it separately from the main draft and keep track of how the current version continues with some form of track changes (this is likely more in an async workflow, but it relates).
  • Ability to see a notice of an audit trail when coming into a post after a few revisions, including who made which changes. For example, if an editor went through and made a number of changes to a post so the writer can see the difference.
  • A way to focus on a particular part of an article to see how it might have changed, rather than the entire piece.
  • Improved management of posts with numerous revision history events (think 50+). It’s currently pretty painful to try to go through revisions at scale.
  • Ability to readily schedule mass changes across the site. 

Tied to this, I’ll note from the FSE Outreach Program that feedback has come in very positively around the Style Revision experience both in terms of the visual nature and easy to understand timeline with a request to have that in place for templates and template parts.

Block library

The ability to configure more granular control over blocks continues to come up as a pain point, especially as more support is added by default in different ways that cause folks to feel a level of uncertainty and scrambling to keep up. Anything we can do to consolidate the methods for controlling block access and support would go a long way. For example, even if you use theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. to lock down all support and settings for coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress./media-text, block settings still appear in the block sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme. due to that being defined in core/media-text’s component code and inserted in a custom panel.

Tied to this, folks want the ability to see where patterns are used across a site (“this pattern is used on x number of posts”) as a better way to gauge the impact of changes, particularly for synced patterns.

Finally, the ability to have theme.json partials on groups of blocks would be a huge step forward to provide a locked-down experience for folks who are in the position of simply trying to put content into a specific layout.

Media Library

Media management, from the ability to add folders to controlling copyright information, are big concerns for folks, especially in larger organizations where many images are uploaded per day.

  • Way to manage and view image attribution and copyright information. This was repeated often as a high priority item and major pain currently. 
  • Improved alternative text management. 
  • Ability to add folders.
  • Improved searchability. 
  • Ability to upload photos designed for a specific post and having those photos surfaced when working on a post. 

Adminadmin (and super admin) redesign

There was a lack of overall feedback here as folks mainly focused on wanting the experience to be “more modern” without a lot of specific feature requests. 

  • The ability to reorder, hide, and favorite menu items for greater customization.
  • Desire for a redesign to modernize the experience. This was nonspecific but repeated feedback that ties into Data Views work.

Going forward

As more conversations occur and work progresses, I’ll continue to bring feedback into appropriate GitHub issues and/or create new ones. For more open ended feedback, like the above, I’ll do recap posts as appropriate. If you’re chatting with folks about Phase 3 efforts, please do the same! Similar to Phase 2, we’ll need folks across the WordPress community to have conversations, educate, and bring others along.

Want to chat about phase 3 in the future? Comment below.

If you’re interested in joining future hallway hangouts on the topic, please leave a comment below and I’ll @ your username for any future hallway hangouts I run going forward. 

Want to get involved today?

These conversations centered around the already public post on each topic so keep chiming in on those: Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Thank you to @cbringmann @chanthaboune for reviewing this post. 

#feedback, #phase-3, #summary

Riad Benguella 10:12 am on July 13, 2023
Tags: , phase 3   

Real-time Collaboration: architecture

The goal of this issue is to lay down the technical architecture to implement real-time collaboration in WP Adminadmin (and super admin) (Post and Site editor). See the following bootstrap post for more information on the scope and features of the project.

There are different use-cases we want to enable as outlined on the post but all of them come down to these two fundamental aspects:

  • Data needs to be shared and synchronized between users connected to the same WP-Admin.
  • If the networknetwork (versus site, blog) is disconnected, the data is persisted locally and synchronized back to the server the network is restored.

In addition to that I’d like to add another guiding principle to the technical solution we try to conceptualize: 

Ideally, developers working on the UIUI User interface of WP-Admin/editors shouldn’t have to think about whether the data is local/remote/synchronized/merged… Developers declare their “data requirements” and these requirements are fulfilled for them by a separate and automated layer. 

This principle is important for multiple reasons:

  • It frees developers from thinking about the synchronization and the collaboration for every new feature and piece of UI that they add. Additional data will be collaborative by default and offline-ready.
  • It is also important to ensure the backward compatibility of our existing public facing APIs to access and manipulate WordPress Data.

This is also the same guiding principle that we put in place when we initially developed the @wordpress/data package to address the local and remote data needs.

Current architecture

The following schema represents how the data flows in the site and post editor code-bases.

The architecture is separated into two sections: 

  • The UI layer: A developer working on this is typically writing components, declaring its data needs using selectors (like the `getEntityRecord` selector in the example above and is notified of data changes automatically. Developers can also make use of actions to perform mutations. (Editing posts, saving posts…).
  • The CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. Data layer: Responsible for addressing the data needs, retrieving the data from the server or locally, caching data if needed and notifying the consumers (UI layer) of any changes.

Proposal

So the goal here is to introduce data synchronization between remote peers (collaborators) while also persisting the data locally, without impacting the UI layer. To do so we can introduce a sync engine.

Prior art: Take a look at this resource if you want to read more about offline sync engines in SPAs and prior art (Figma, Linear…)

To understand better how such a sync engine would work, let’s take a look at a small example. First thing to note is that all the data that is rendered / shared / persisted can be represented as a simple list of documents / objects. So the role of the sync engine is to fetch / retrieve / persist locally any changes happening to these objects separately.

  • The user (UI components) asks for the post with id 1 by calling the `getEntityRecord` APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. of our core-data package. 
  • Internally, Core Data asks the sync engine to bootstrap a document of type post and its identifier is 1.
  • First, the sync engine creates a “document” in memory that will represent the source of truth.
  • The sync engine tries to load that document from the local database. If not found, it creates an empty database to persist any future changes of that particular document.
  • The sync engine then performs what we call the handshake where it connects to the remote peers, all the collaborators working on the same document (type post, identifier 1) and merges the local copy with the remote peers copy.
  • The sync engine also asynchronously triggers a fetch call to retrieve the document from the WordPress backend and refreshes the local copy or initializes it if there were no local copy or remote peers connected.
  • Finally, any changes that happen to the local document, regardless of the source of the change (user triggered change, loaded from the local database, change triggered by a remote peer) all trigger the same change handler in the core data package.
  • Once the change handler is called, the UI component is going to be notified (re-rendered).

Introducing Yjs

We can split the implementation of the proposed sync engine into the following steps:

  1. Introduce the observable documents objects: in-memory document objects with an API to make an update and to subscribe to changes.
  2. Support merging changes from multiple sources into the observable documents. Note that there are essentially two ways to synchronize changes: a conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved.-free replicated data type (CRDT) and operational transformation (OT). Previous explorations have shown that OT is too complex to implement on top of our existing architecture. 
  3. Loading and persisting document changes to a local database.
  4. A communication layer between all the users connected to the same WordPress Admin.  

Fortunately, there are some open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. solutions that can help us implement the proposed sync engine and address most of these requirements. The most promising one that has been used in the previous explorations is Yjs.

Yjs is an implementation of CRDT. You can think of it as a data structure that you can use to represent the objects being synchronized. Once you use that special representation, Yjs offers adapters to address all the requirements above: observe changes, merge changes from different sources, persist into a local database and potentially communicate with other peers.

Q&A

While this library solves most of our requirements, explorations have shown that the devil is in the details. Writing sync engines is a challenging project and there are a lot of questions that need to be addressed.

What about the performance impact of the added observable objects and the back and forth transformations from our regular objects into their equivalent CRDT format?

The sync engine is most likely going to have a small performance impact even on local changes, it remains to be seen whether this impact is going to be a blockerblocker A bug which is so severe that it blocks a release. or not. We do have the tools in place (performance metrics) to help assess the question once everything is in place.

Yjs allows using WebRTC or WebSockets by default to synchronize documents between peers, what communication layer is the most adapted to WordPress?

As mentioned in the bootstrap post, this is one of the biggest challenges for us. The most performant transport layers rely on a centralized web socket server and a lot of PHPPHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 5.6.20 or higher hosting providers don’t have support for restful communication using web sockets. 

This means that we need an alternative communication layer that can work with any WordPress install by default while allowing the communication layer to be replaceable by plugins/hosts.

Can we use WebRTC as the default communication layer as it’s P2P and should work with all WordPress installs?

While WebRTC is indeed P2P, most existing implementations of WebRTC, including the default Yjs WebRTC adapter rely on a centralized signaling server. This is a very light server used to perform the handshake (for the peers to discover each other) and in general rely on web sockets.

Since we can’t rely on web sockets, there are three possibilities that we can explore:

  • Build a public signaling server (on .org infrastructure for instance).
  • Implement WebRTC signaling using long polling instead of web-sockets (supported in all WordPress instances) and a public stun server (there are existing public stun servers and .org infrastructure can also ship one if needed). This also involves providing a custom Yjs adapter to support the polling based signaling server.
  • Avoid WebRTC by default and use long polling to perform both the handshake and then data changes. In this case, no public server is needed but performance can suffer.

What should we also consider as part of project

  • Performance impact.
  • Memory footprint of the shared documents.
  • Storage footprint of the local database. (yjs stores the history of changes with some optimizations).
  • Security: Prevent access to peers without the necessary rights.
  • Undo/Redo.

Get involved!

We’re just getting started in this journey, there’s a number of open questions and If you are interested in the challenges or want to leave any feedback on the project, please chime in.

#gutenberg, #phase-3

Matias Ventura 9:11 am on July 12, 2023
Tags: , phase 3   

Admin Design

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

About a year ago, some early notions of how we could evolve the admin experience were shared. The site editor, and the foundation set by its fluid browsing and editing flows, provides a pathway for revitalising the adminadmin (and super admin) experience. Given all the workflows and collaboration requirements through the upcoming phase 3 projects, it’s time to look more in depth at these ideas and where they might lead.

There are multiple goals to account for with this effort. The state of the art and user expectations for digital software are constantly changing and there’s a tangible need to revitalize the wp-admin design, improve its visual clarity, reduce cognitive weight, better support user workflows, and expand the personalization of the interface. WordPress thrives within its flexibility and its pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party ecosystem. The counter part to that can often be a complicated information density, particularly at the top level navigation, which can negatively affect the user experience — we’ve all seen admin dashboards with very long and daunting sidebars!

As WordPress turns twenty years old, the overall aim of this work is to improve upon this experience at a foundational design level, giving plugins and users more control over the navigation while ensuring each WordPress experience is recognizable, intuitive, accessible, and delightful.

Mockup of site editor admin with a light colored admin theme in place.

So how can we design the overall admin in such a way that it can be shaped to any need, regardless of their simplicity or sophistication? How can we then manage complexity without being complicated? How can we make designing and building great interfaces on the platform easier and more expressive to usher it through the next two decades? This balance cannot just be achieved with good practices but needs to be reflected in the semantics and the design of the software itself. For example, offering a way to customize the most important menu items at the highest level might allow both a blogger and a commerce store manager to have the best possible experiences, without trading compromises. Shaping WordPress to the particular needs of each person and project can be a large part of ensuring its continued long term success as the operating system of the web, democratizing access, and championing a diversity of experiences. The challenge is doing it without sacrificing the important aspect of good defaults and the “it just works” ethos.

In order to achieve this we’d want to clarify some of the main extension points as semantically as possible. What structures and surfaces does WordPress provide so that plugins and platforms can make use of them to achieve their visions? What does it mean to register settings, sections, or to add blocks? What elements of the interface are relevant? How can they be navigated? This is crucial to establish not just for ease of development but to ensure a usable and accessible experience for all.

Mockup showing a plugin section in the admin. The navigation is focused on the plugin tools in the sidebar.

This effort is also an opportunity to formalize the design primitives and interaction paradigms that are part of the UIUI User interface component system begun in wordpress/components. A crucial aspect is to ensure WordPress itself is built with the same pieces and APIs that plugin authors can use. Aside from color themes, our set of primitive components also need to work in dense environments like the editor, as well as environments that need more breathing room and focus like admin sections. Density, clarity, usability, and accessibilityAccessibility Accessibility (commonly shortened to a11y) refers to the design of products, devices, services, or environments for people with disabilities. The concept of accessible design ensures both “direct access” (i.e. unassisted) and “indirect access” meaning compatibility with a person’s assistive technology (for example, computer screen readers). (https://en.wikipedia.org/wiki/Accessibility) are paramount. A related discussion can be found here. As part of leveraging the components across the admin interface, we need to address functional gaps (like table and list views, bulk editing operations, etc) and assist plugin needs for anything that might not be already addressed that should be addressed. Ultimately, the design library needs to be showcased in the wordpress.orgWordPress.org The community site where WordPress code is created and shared by the users. This is where you can download the source code for WordPress core, plugins and themes as well as the central location for community conversations and organization. https://wordpress.org/ website as a clear resource for people building upon WordPress. A great developer experience should naturally guide people to accomplish great and consistent user experiences.

Mockup of the block editor sidebar showing several UI components.

Another primary goal is to achieve a cohesive experience between all editing and management activities. It goes without saying, but navigating through your tasks should feel seamless, whether editing with blocks or managing settings; going through default WordPress sections or plugin areas; or whether the user is operating as an author or an administrator. Considering the space afforded by a malleable browsing experience, combined with the collaboration work, it’s also a good opportunity to look at how offline support might work and its implication for handling optimistic interactive updates to user data. There are a lot of great efforts — from Playground to SQLite — that could also align to provide a fantastic platform experience with all the modern expectations for speed, access, and performance.

Scope

There remain a lot of open questions to work through before going into further technical details but the following should offer an outline of some concrete aspects to consider as more in depth design explorations are done:

  • Define the structural pieces of the admin experience, from a design and instrumental perspective. Formalize its semantic extensibility. What structures and surfaces does WordPress provide so that plugins and platforms can make use of them to extend and reduce what’s possible?
  • Restructure admin menu design to support a drill-down interactivity model. Leverage and augment existing APIs for maximum backwards compatibility. Explore using the frame as the encapsulated target destination for existing admin views.
  • Look into user personalization with the ability to reorganize top level admin menus. It could work in a similar fashion to “pinning” plugin sidebars in the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. editor, where both plugins and users are in control. Also consider coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. sections on the same level so that they could be toggled on or off based on needs.
  • Deep accessibility support for navigating regions, tuning global preferences (like button text labels over icons across the UI, and other affordances) as part of further refining each user experience according to people’s needs and preferences.
  • Look at wider deployment of Command Palette across admin sections and formalize its APIs.
  • Introduce more semantic placement of quick actions: for example, organize all “add new” options and hooksHooks In WordPress theme and development, hooks are functions that can be applied to an action or a Filter in WordPress. Actions are functions performed when a certain event occurs in WordPress. Filters allow you to modify certain functions. Arguments used to hook both filters and actions look the same.; place notifications, user profile, other top level extension markers, and so on.
  • Work on completing and evolving the design system gaps. Develop set of components to handle table and list views. Introduce full theme color support. Consider what sort of requirements for SSR are relevant.
  • Revise the design of general settings views by applying and structuring these components.
  • Look at the current role of the dashboard home and perhaps embrace admin blocks for improving its widgetWidget A WordPress Widget is a small block that performs a specific function. You can add these widgets in sidebars also known as widget-ready areas on your web page. WordPress widgets were originally created to provide a simple and easy-to-use way of giving design and structure control of the WordPress theme to the user.-like personalization. Improve built-ins like quick posts, post formats, and title-less posts as outlined in the workflows project.
  • Connect with notifications project and a path for reducing ad hoc notices across the admin.
  • Look into the implications for multi-site networks and its navigation patterns.
  • Address the state of front-end admin bar or equivalent.
  • Contemplate solutions that maintain as much backwards compatibility as possible for admin sections that may never be updated by the original authors.

Get Involved!

Please, share any feedback, ideas, or requests you might have as we embark on this road. Every voice in the WordPress ecosystem should be reflected in this work as we balance all the different needs and expectations.

#gutenberg, #phase-3

Matias Ventura 4:40 pm on July 10, 2023
Tags: , phase 3   

Block Library

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

Now that blocks are able to model and express an entire site, it’s important to improve the way they can be organized, listed, and installed by users. The overall goal is to improve how blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. management works outside the editors — for example, by allowing the disabling of blocks globally across a site, not just as user preferences.

Wall of block icons

We should connect functionalities that might already exist but are not fully developed as user features yet. For example, the relationship between single blocks, post formats, and custom post types. There’s code built in that tries to coordinate what initial block to show on the editor canvas by checking what’s the default post format (paragraph for standard posts); if the default format was Image a user would see an image block placeholder when creating a new post instead of an empty text prompt. This kind of customization is currently a bit obscure and could be more powerful if it allowed to more easily configure default block type across post types. It should also connect with other workflows like “quick post” interfaces in the dashboard.

It also means introducing more robust permission handling across the various capabilities (block registration, locking, etc) so administrators can define what blocks are available for different user roles (or even what capabilities of individual blocks are to be exposed). The fact that theme.json files are inherently composable should allow for more granular handlings of capabilities in a systematic way. For example, consider a fictional author.theme.json that sets, disables, and controls what tools and settings are available for authors over the root config file.

There are a couple important block API items to list that can have consequences on user flows as well. One example is to introduce an auto-insert mechanism so that plugins can better interact with the site document (and block trees) in a way that respects user control.

Furthermore, as part of the ongoing work on patterns, it’s time to start formalizing a connection between blocks and meta fields to empower building flexibility while retaining user clarity. For example, it should be possible for an administrator to set up a custom post typeCustom Post Type WordPress can hold and display many different types of content. A single item of such a content is generally called a post, although post is also a specific post type. Custom Post Types gives your site the ability to have templated posts, to simplify the concept., powered by a block pattern, where each individual block has content attributes connected to a specific metaMeta Meta is a term that refers to the inside workings of a group. For us, this is the team that works on internal WordPress sites like WordCamp Central and Make WordPress. field rather than post content. All could be accomplished within the block editor interface.

Finally, it’s also important to improve how installing blocks from the directory works, how we handle missing block types and reinstallation, how themes and patterns could reference third party blocks, etc. This also implies better visualizations of blocks introduced by larger plugins.

Scope

This is a summary of the broad tasks we need to look into:

  • Add a section next to, or under, plugins to manage blocks globally. This means expanding the current block manager to allow enabling and disabling blocks across a site.
  • Introduce more robust permission handling so administrators can define what blocks are available for different user roles.
  • Create auto-insert hooksHooks In WordPress theme and development, hooks are functions that can be applied to an action or a Filter in WordPress. Actions are functions performed when a certain event occurs in WordPress. Filters allow you to modify certain functions. Arguments used to hook both filters and actions look the same. and APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. flows for block types. This is a developer tool with user flow implications.
  • Improve the mechanics of the block directory with better visualization of blocks bundled in plugins and better flows for themes and patterns using third party blocks on templates.
  • Develop API integrations for mapping attributes to custom fields using the editor interface. The idea being blocks can provide a native UIUI User interface for managing and introducing meta data through its attributes layer.
  • Allow managing categories for saved patterns. Expose other pattern APIs, like block type or parent dependencies in the user flows.
  • Develop the concept of “partially synced patterns”. These allows the design layout and the style properties to remain global while the content can be locally customized.
  • Extensive application of theme.jsonJSON JSON, or JavaScript Object Notation, is a minimal, readable format for structuring data. It is used primarily to transmit data between a server and web application, as an alternative to XML. partials on groups of blocks (control settings and styles on specific templates, parts, patterns). Explore full theme.json settings objects on a per user role.

Get Involved!

Join the conversation if you have thoughts or feedback about ways to improve the workflows around blocks.

#gutenberg, #phase-3

Matias Ventura 2:47 pm on July 7, 2023
Tags: , phase 3   

Media Library

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

The media library has been an area of slow iteration during the first phases of the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. editor. It’s time to put more focus on it. The main goals are to expand the media management capabilities, unify the block edit and single media interfaces, and improve upon the major media flows.

Screenshot of the media inserter panel in the editor.

On the management side, it’s time to look at categorization and tagging, better handling of attached media, design improvements to the library views and filtering. Several important functionalities are already present but not presented in the best way. For example, media attachment functionality (items anchored to a specific post) is often hard to discover, the interface is not the most intuitive, and the connection doesn’t necessarily carry through other editing flows in a clear way. For larger teams, media ingestion can often be an entirely separate process from writing posts. The ability to bulk attach media files and connect them to the relevant content pieces is thus an important part of the collaborative process.

We should also revamp the image editing interface and align with the current block editor tools. Cropping tools in particular need to be unified and further improved. Iterate on how users can visualize and toggle between different thumbnails, while reducing the burden on picking the right file size (WordPress should determine that for you at runtime). Review default thumbnail sizes to accommodate current screen sizes distribution. It should also be possible to set a non-destructive duotone effect directly in the library. It’d be nice to bring UIUI User interface tools like focus point control to the library flows, so that media objects can store that information and be leveraged automatically when inserting on a cover block, or when using an image on a cropped featured media container.

When managing the media library, it should be easy to check and keep track of attribution. As we look into expanding the presence and touch points of Openverse, it’d be interesting to see how contributions to the commons could work directly from a user’s WordPress install. Another area to look at is improving handling and presentation of other media types (audio, video, files) and their connection with blocks and the block APIs. We should resurface work on a native Playlist block, ideally powered by the Interactivity API.

Finally, we should take a look at the publishing flows for media and format driven posts (where only one block type is present) to allow more expressive freedom.

Scope

  • Tackle design improvements of the media library elements and see about a path to migrate to wordpress/data and newer UI components. Make it even easier to reuse and connect the media experience anywhere on the adminadmin (and super admin) and blocks plugins.
  • Look into adding categories and tags support. Explore better browsing organization (for example, date headings and other improvements to the filtering UI).
  • Improve inline and freeform cropping across block editor and standalone image editing. Connect cropping variations with defining custom aspect ratios across the block UI.
  • Explore how patterns could be incorporated into media flows. For example, being able to preview a selection of media (or all attached media) within different media patterns quickly.
  • Connect with Workflows: for example, to require certain steps, such as adding a caption, attribution, alt text, or prevent things like publishing with external images.
  • Media flows and block integration should be excellent. For example, make blocks aware of attached media so that inserting an image block could automatically display unused media or allow an author to cycle through those without opening the full media library or browsing through an entire collection.
  • Expose relevant media tools and actions in the command center.
  • Improve presentation of media in other editing contexts, like showing small media previews in the block list view.
  • Incorporate featured imageFeatured image A featured image is the main image used on your blog archive page and is pulled when the post or page is shared on social media. The image can be used to display in widget areas on your site or in a summary list of posts. in the canvas of the post editor when the single template uses a featured image block (this is a lingering task from the site and post editor interface work).
  • Look at the possibility of supporting other featured media types beyond images.
  • Improve upon bulk editing operations, including attaching already uploaded media to a post type.
  • Explore treating focal point as image data and make it available to plugins and blocks.
  • Allow operations like drag to upload media anywhere on the editor — and then maybe anywhere on the admin.
  • Follow through on updates to the attachment theme template and single image expand functionality.
  • Review outstanding tasks on main media blocks (image, gallery, cover, video, audio, file, etc) for enhancements and bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority. fixes.

Get Involved!

What other improvements would you like to see around the media library experience?

#gutenberg, #phase-3

Matias Ventura 9:36 am on July 5, 2023
Tags: , phase 3   

Revisions

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

Collaboration workflows require revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. and edit history to be clear, usable, and performant. The design of the traditional post revisions interface needs to evolve to become fully aware of blocks. It should offer better visual comparison capabilities beyond the classic HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. code diffing, which is getting less and less adequate for illustrating blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. modifications. It should provide easy mechanisms for spotting changes regardless of the content type being shown.

Mockup showing a paragraph block with three words highlighted in green as "additions".

As part of improving the overall experience, we should also go beyond document level history and explore how the interface could let users browse through single block changes and offer the ability to restore them individually rather than requiring full post restores. For global styles, we should evolve the revisions panel to allow comparing two revisions side by side. For synced patterns, we could allow browsing edit history with side by side and overlay comparison tools.

There’s also a semantic overlap between conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved. resolution in real-time collaboration, offline reconciliation, and restoring or managing edit history. At the same time, internal revisions in WordPress are not a replacement for version controlversion control A version control system keeps track of the source code and revisions to the source code. WordPress uses Subversion (SVN) for version control, with Git mirrors for most repositories. systems, so there has to be points of contact to ensure developers can lift from or deployDeploy Launching code from a local development environment to the production web server, so that it's available to visitors. changesets down to the filesystem as needed. Plugins should also be able to extend the capabilities and develop more advanced integrations on top of what CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. provides. For example, consider the case of using all the site editor tools but ensuring user modifications are saved back into files so they can be managed by git or svn workflows, including wp-cli integrations. Exploring some of these problems should also help outline what architectural considerations multi-lingual would present in Phase 4 when managing content over all possible site objects.

Finally, we should investigate if we can adapt customize_changeset to orchestrate revision grouping across all new entities and requirements — such as grouping multiple revisions together, reference and orchestrate changes to entities, etc.

Scope

This is a summary of the broad tasks we need to look into:

  • Design a revisions interface that can better highlight visual differences beyond markup diffing. Align presentation of added & removed content with how edit suggestions might be presented in the other collaborative workflows.
  • Integrate with blocks. Allow selecting a block (like an image) and see all changes that have occurred to it. It should work out of the box with nested contexts, so if you select a parent it tracks and shows changes across all its children as well. Users should be able to focus at any level within a document.
  • Make it easy to restore changes on a document or per block basis. For example, reverting some design changes to an embedded pattern but keeping the rest of the post as is, even if the pattern is not synchronized separately. The flows for restoring should be connected with the flows for publishing, so a revision can be made the currently published view.
  • Explore ways in which branching could work for multi-entity history. For example, site title changes won’t be naturally covered in the edit history of a template that contains a site title block, but it could be referenced through a customizerCustomizer Tool built into WordPress core that hooks into most modern themes. You can use it to preview and modify many of your site’s appearance settings. changeset that connects template revision and discrete entity values. The multi-entity saving flow should evolve to take this into account.
  • Evaluate any possible shortcomings in storage and performance of the various revision systems if they are to be used broadly (for example, everything going through posts tables).
  • Improve the visual comparison tools so a user can check their content side by side or using overlays to spot differences. This can be rehearsed on the global styles revisions history and focused pattern views.
  • Develop more immediate clarity over what properties were changed on certain revisions. For example, on style revisions list if there were changes to color palettes, to typography, etc, to improve the browsing experience.
  • Explore viability of naming or grouping revisions, particularly for staging changes in the future. Connect “save” area in site editor flows with customize_changeset or equivalent. Keep track and list entity modifications.
  • Review the revisions extended pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party for scheduling updates to already published posts.

Get Involved!

If you are interested in helping improve the revision interfaces and architecture, please join in.

#gutenberg, #phase-3

Matias Ventura 8:08 am on July 4, 2023
Tags: , phase 3   

Workflows

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

Provide seamless collaboration during the entire editorial process, from draft to publication. Allow users to add comments, suggest edits, and tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.) other users for peer review. Ensure the interface remains focused on a smooth experience for writers and editors.

Mockup showing a new cursor / tool option in the editor for adding "comments" to blocks.

Improve the publishing flow by customizing the review process, establishing what needs to be done before a publication is ready. For example, an author could leave empty media blocks in a story they are writing and mark them to be completed by another team member, ensuring the post cannot be published while empty placeholders are still there. This could also include other types of requirements, like word count, fields to be completed, and so on.

Make it straightforward to share different types of content, from posts to design changes, while controlling access through permissions. Connect with the adminadmin (and super admin) notifications project to capture comment reviews and mentions. Build upon improvements to the post revisionsRevisions The WordPress revisions system stores a record of each saved draft or published update. The revision system allows you to see what changes were made in each revision by dragging a slider (or using the Next/Previous buttons). The display indicates what has changed in each revision. interface to provide clarity over edit history.

The tools and infrastructure developed need to support simpler use cases (one author sharing previews with friends for feedback) all the way to larger editorial teams, managing deadlines, handoffs, and more sophisticated review processes. Plugins should be able to take it further.

While a lot of this work naturally aligns with unpublished content, it’s also important to consider workflows around already published content and pages.

Scope

This is a summary of the broad tasks we need to look into:

  • Introduce inline comments on blocks within the editor experience. Explore using comment types to store them. Allow marking comments as resolved. Status of comments also need to fold within individual revisions, so that it’s easy to see what specific edit state a comment refers to. Possible connection with “pending review” functionality.
  • Explore introducing support for “tasks” in publish flow. These would allow highlighting missing actions before a post is to be published. It can be connected to various post statuses (such as going from “pending review” to “publish” readiness, ability to have pending review status over already published content, or other custom statuses). Individual actions should be highly configurable by users and extensibleExtensible This is the ability to add additional functionality to the code. Plugins extend the WordPress core software. by plugins. Tasks can also go beyond publishing and be relevant for other plugins (like marking fulfilled orders in WooCommerce data structures).
  • Ability to share draft links with permission controls and clear revision browsing. This also extends to previewing and sharing design changes across the entire collection of features of the site editor.
  • Introduce extension points for in-app previewing. For example, a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party might want to show how a post looks for subscribers and non-subscribers; with or without ad units; on an RSS feedRSS Feed RSS is an acronym for Real Simple Syndication which is a type of web feed which allows users to access updates to online content in a standardized, computer-readable format. This is the feed. or an email; etc. CoreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. should provide good mechanics for plugins to hook, control, and modify these views across editors in a way that integrates seamlessly with the editing flows.
  • Improve multi-entity saving to allow scheduling design changes on blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. themes that can be managed through the various revisions systems (styles, templates, patterns, pages, posts). Possibly allow naming future revisions to better manage and orchestrate changes throughout a site. Preview snapshots of a site before they go live.
  • Explore hook points for version controlversion control A version control system keeps track of the source code and revisions to the source code. WordPress uses Subversion (SVN) for version control, with Git mirrors for most repositories. systems to smoothly take over internal revision systems if desired.
  • Control access with granular permissions for patterns and templates rather than general locking. For example, lock patterns to “content only” for author roles but leave it open for admins.
  • Consider multi-author support on posts or improve how it can be represented as a side effect of real-time collaboration and revision authorship.

Get Involved!

There’s been a lot of interest from users, developers, agencies, etc, about these set of features. Many have already reached out over the last year and months to share experiences, insights, or existing plugin work to reference. Let’s capture and highlight feedback to ensure all use cases are represented.

#gutenberg, #phase-3

Matias Ventura 4:49 pm on July 3, 2023
Tags: , phase 3   

Real-Time Collaboration

This is part of the Phase 3: Collaboration roadmap. The main projects are Real-Time Collaboration, Workflows, Revisions, Media Library, Block Library, and Admin Design.

Introduction

The primary aim of real-time collaboration is to build functionality into the blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. editors so that concurrent collaboration, shared edits, and online presence of peers are possible. Supporting these workflows is not just about concurrency, though, but also about lifting restrictions that have been present in WordPress for a long time, such as locking a post when two people try to edit at the same time. There are various technical layers underpinning this functionality, so let’s quickly recap what they are and what we need to take into account to get started.

Animated mockup showing multiple users moving their cursors on a shared editor screen.

First, these capabilities should be available to the widest possible audience. That means using technologies that don’t rely on sophisticated server setups that would restrict the ability for people to collaborate within their WordPress sites, regardless of hosting infrastructure. This likely puts us on the path of building on top of open web standards like WebRTC where we can ensure deployment of these features without any special burdens on the backend. WordPress itself could provide an easy to scale signalling server, likely over REST endpoints, for authentication handshake.

However, we also want to ensure the system is flexible enough to extend with other server implementations for more advanced needs — such as a WebSockets service — in order to scale beyond current limits for peer-to-peer simultaneous browser connections, or for cases where peer connections cannot be established. This would naturally be pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party territory.

Outside of the peer sharing setup, we also need to establish the primitives for conflictconflict A conflict occurs when a patch changes code that was modified after the patch was created. These patches are considered stale, and will require a refresh of the changes before it can be applied, or the conflicts will need to be resolved. resolution that are going to be able to reason through our block data structures and orchestrate edits over time. For this purpose, it’s likely Yjs will come in handy, which is an amazing open sourceOpen Source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. library that implements conflict-free replicated data types (aka CRDTs) presented as shared types. This library has already been put to good success in most of the past explorations within the Gutenberg repo and related projects. The author of the library has also engaged directly in some of these iterations with suggestions and feedback. While it has worked pretty well with block data, it has not been paired with multi-entity documents yet. Furthermore, ongoing SQLite explorations might uncover other ideas and opportunities for wider offline-first experiences where CRDT needs to be handled at a different abstraction layer.

Scope

This is a summary of the broad tasks we need to look into:

  • Review all current explorations, their lessons and tradeoffs.
  • Outline the set of hooksHooks In WordPress theme and development, hooks are functions that can be applied to an action or a Filter in WordPress. Actions are functions performed when a certain event occurs in WordPress. Filters allow you to modify certain functions. Arguments used to hook both filters and actions look the same. necessary to encapsulate collaboration features over block editor providers, like sharing edits and peer state across browsers.
  • Ensure capabilities build on top of the block APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. so that block authors don’t need to make modifications for their blocks to work in collaborative environments.
  • Design and represent “presence” of connected peers in the interface. Integrate selection state with other useful parts of the interface, like the block list sidebarSidebar A sidebar in WordPress is referred to a widget-ready area used by WordPress themes to display information that is not a part of the main content. It is not always a vertical column on the side. It can be a horizontal rectangle below or above the content area, footer, header, or any where in the theme..
  • Lift concurrent editing restrictions. Right now WordPress locks a post to the current user that is editing it. Another user can take over, but it doesn’t allow two users working on the same post at the same time.
  • Review undo / redo stacks and resolution. It’s likely some modifications are needed here.
  • If employing CRDT, ensure its designed to work with multi-entity documents, which is a newer requirement from Phase 2 compared to just Phase 1.
  • Employ peer caret and selection primitives that work across block types and design their semantic and visual representation.
  • Consider offline a first-class use case of collaboration in orchestrating edits and transactions.
  • Explore the possibility of “following a user” as they work through a document.
  • Define authentication and messaging semantics. For example, the characteristics of a WebRTC handshake with a plain HTTPHTTP HTTP is an acronym for Hyper Text Transfer Protocol. HTTP is the underlying protocol used by the World Wide Web and this protocol defines how messages are formatted and transmitted, and what actions Web servers and browsers should take in response to various commands. REST APIREST API The REST API is an acronym for the RESTful Application Program Interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data. It is how the front end of an application (think “phone app” or “website”) can communicate with the data store (think “database” or “file system”) https://developer.wordpress.org/rest-api/. request to the backend. Provide answers to questions like read only sessions over peer-to-peer connections and cases where peer-connections cannot be established.
  • Structure WebRTC implementation as an adapter that could be swapped. The architecture should be pluggable so extenders can provide other solutions.
  • Outline error scenarios and functional overlaps with offline-first capabilities. For example, peers losing a connection while still making edits. We’d want to employ this for same-user resolution should you be editing across different devices as a single user. Handle saving and revision allocation.

Get involved!

If you are interested in the challenges of real time collaboration or want to leave any feedback on the project, please chime in.

#gutenberg, #phase-3