Drupal Commerce 2.x Roadmap

Introduction

Development of Drupal Commerce 2.x began in Drupal core through contributions to core initiatives and new core modules. As a result, the next version of Drupal Commerce will feature a shrunken codebase thanks to updates to Drupal's core Entity API and the Entity Reference module, which alone deprecates a significant portion of the Product Reference, Line Item, and Customer modules.

The roadmap lists out the changes we'll be making to the architecture of the core Commerce modules in response to the changes already present in Drupal 8. It also features a variety of topically organized tasks we've lined up to mitigate the configuration and development challenges our users have faced using Drupal Commerce 1.x.

Timeline

Drupal Commerce 2.x Timeline

Our original goal was to have a beta release by the time Drupal 8 hit a full release. However, in an attempt to avoid repeating the turbulent experience of developing Drupal Commerce 1.x on an unstable Drupal 7 / Entity API / Views, we postponed active development of Commerce 2.x until Drupal core settled down.

Development of Commerce 2.x will truly begin in earnest once Drupal 8 has an alpha or beta release. In the meantime, our focus is on ensuring Drupal 8 itself is a success and on testing implementations of proposed features on the new Drupal 8 APIs as they develop.

If Drupal 8 truly has a full release around DrupalCon Prague, there's little chance Drupal Commerce 2.x will be beta ready. We'll be sure to update the forecast as the date draws nearer.

Development Emphases

While many of the items in the roadmap task list are grouped according to the subsystem affected, others are more abstract in scope and represent an attempt to apply a general principle or development emphasis more broadly to the codebase. The emphases listed here are described below, providing you with the context necessary to understand the reasoning behind these broader changes:

Our task list certainly does not catch every area in Commerce where a principle should be applied. Contributors should feel free to propose changes not on the list that reflect one or more of these emphases, and we will discuss as a community which of these changes should become a part of the roadmap. All such discussion should take place in the issue queue as tasks or feature requests.

Improve Drupal Commerce as a conduit vs. a container

To move Drupal Commerce forward on Drupal 8, we need to look backward and recover one of our core founding principles: an eCommerce site is a data conduit, not a container. It should be easy to move data between a Drupal Commerce site and the various web services the site uses to power eCommerce at a much broader scale than is possible through the website alone.

  • Drupal Commerce 1.0 was the initial implementation of eCommerce native to Drupal 7, architecting around the Entity API, Field API, Views, and Rules to build a truly flexible eCommerce framework.
  • Commerce Kickstart 2.0 focused on the simplification of the launch and administration of a feature rich store built with Drupal Commerce.
  • Drupal Commerce 2.0 brings a renewed focus on integration with third party web services that make eCommerce possible and profitable.

This effort revolves around a refactoring on Drupal 8 that takes advantage of core REST functionality along with the hypermedia principles being pursued in Commerce Services on behalf of Commerce Guys' mobile app and client requirements.

Serve the majority use case

There are a few clear instances where we added premature "power features" or pursued a local solution to a task better managed by a third party service. This should be addressed in Drupal Commerce 2.0 to better serve the majority use case:

  • Shopping cart refresh - we refresh the cart on every single load of the order to ensure up to date pricing, but this load is unnecessary on most sites. We should instead provide configuration options to govern the refresh of an order, perhaps depending on an "on load" Rules event instead.
  • Product pricing rules - we force all price calculations to go through Rules in order to facilitate a largely unused feature, sell price pre-calculation. If pre-calculation is disabled, there's no harm in allowing modules to directly hook into the product pricing system, but we should provide a more robust API that doesn't depend on Rules actions to get the price components right.
  • Shopping cart revisions - by default we create new revisions of shopping cart orders on every cart operation to track shopping history and cart abandonment, but these analytics are better monitored through a third party service like our entry level freemium partner service Jirafe.
  • Checkout form validation / submission - similarly, we validate and submit every checkout pane on a checkout page to capture as much data as possible in spite of errors on the page. This makes dependencies between checkout panes impossible without custom logic in the checkout pane code itself. We could switch this to be a checkout page setting.

Note that this emphasis does not mean we'll remove any feature that is inconvenient to the average user. For example, many people are surprised by customer profile duplication, but this is an essential feature of Commerce for preserving the integrity of historical order data.

Use explicit configuration instead of implicit behavior

We did a fair job of stamping out "magic" configurations in Commerce 1.x - cases where a setting or combination of settings results in a secondary feature that isn't explicitly mentioned. For example, Commerce 1.x originally depended on a combination of settings for fields to be considered product attribute fields, but we eventually removed this implicit behavior with an explicit settings form built into the field edit form.

Build and use robust internal APIs

Commerce 1.x served as proof of concept for many of the Drupal 7 core and contributed APIs, most notably the Entity / Fields APIs. However, we did not do a great job developing and using APIs within our own modules to facilitate development tasks like creating orders, manipulating prices, and processing payments. This emphasis ties back into the first, as we should develop our systems with REST application in mind.

In Commerce 2.x, we should ensure that our modules define robust internal APIs that are then used by forms, Rules actions, etc. as opposed to embedding functionality into these systems' related callbacks. Most notably, we should seek to make price manipulation and price component management much easier as we open up the sell price calculation process to direct manipulation in module code (as opposed to strictly using Rules) when sell price pre-calculation is disabled.

Roadmap Task List

Each of these tasks will ultimately link to an issue on drupal.org where the change will be discussed and patched. Stricken tasks indicate completion of the core task, though there may be related issues still present in the issue queue.

    Integration with Core Drupal 8 Modules
  • Replace the Product Reference field with Entity Reference
  • Replace the Line Item Reference field with Entity Reference
  • Replace the Customer Profile Reference field with Entity Reference
  • Update all of our Views handlers for D8 Views
  • Update and re-export all of our default Views
    If exported default Views are stored in CMI configuration files, then we need to research default View alteration to facilitate the Cart module's alteration of the Order admin View.
  • Convert the cart block / form to use Views entirely
  • Convert our configurable data structures to use CMI
  • Convert Commerce conditions to the new condition plugin format
  • Convert Commerce actions to the action plugin format
  • Convert all of our theme functions and template files to Twig
  • Convert checkout panes to the plugin system
    Plugin type = checkout pane, Plugins = checkout panes
  • Convert payment methods to the plugin system
    Plugin type = payment method, Plugins = payment methods offered by the gateway, Derivatives = payment method instances
  • Convert taxes to the plugin system
    Plugin type = tax, Plugins = tax rates (no more tax types)
  • Remove the concept of tax types
    Rounding and display inclusiveness become tax rate settings; rounding settings also include the rounding precision.
  • Extend #ajax to use hidden buttons that are clicked to trigger updates (to facilitate Cart / Checkout form degradability)
    Use a "button_value" property on #ajax arrays using the onchange event and automatically add the hidden button / click it to trigger the refresh instead of depending on the onchange directly.
  • Deprecate Drupal Commerce's hidden product reference field widget.
    Drupal Commerce Entity / Plugin Updates
  • Update entity controllers to the D8 Entity API
  • Replace use of entity_metadata_wrapper() with the D8 property API
  • Add a new Store entity type
  • Add abstract support for entity deletion prevention
  • Propose treating all entities as "append only" with read only past revisions
  • Propose entity revision tagging
  • Anything defining a title should define a title callback instead
    Products / Product Displays
  • Implement the new product data model
    This involves product hierarchies, field inheritance, product entities as displays, and the custom inline edit form for child products.
  • Support unpurchasable products (i.e. parts of a kit)
  • Add default flat and multi-level product hierarchies
  • Add some Commerce Fancy Attributes support to core
    Cart
  • Enable configurable shopping cart refresh conditions
  • Allow shopping cart refresh on order save, not just load
  • Make the Add to Cart form JS degradable
  • Add Commerce Customizable Products to core
    But ensure we can map line item types to product types and individual products, and allow field / option availability to be filtered at either level as well.
  • Refresh the Add to Cart form when line item fields are updated
    Checkout
  • Spec out the implementation of complex checkout flows
    We need to support dynamic single page via AJAX, multivariate testing of alternate checkout flows, conditional pages and panes, independent vs. dependent checkout panes.
  • Enable configurable translatable checkout page titles and button values
  • Add a checkout progress wizard block to core
  • Make the checkout form JS degradable
  • Add Address Book to core
  • Add a "buy it now" display formatter that creates an order and proceeds to checkout without requiring Cart
  • Evaluate checkout form caching for PCI compliance
    Orders / Line Items
  • Use Message to make visible, e-mailable comments on orders
  • Implement a Kickstart 2.x style order history View to orders
  • Implement the complete design for order view pages
  • Rename order states to order status groups
  • Add UI / Cart + Checkout module support for multiple order types
  • Re-implement line items to be saved / updated through orders
    $order->line_items[0]->product = $product; $order->save();
  • Implement line item revisions
  • Propose replacing order line item reference field w/ a computed list property
  • Add a generic line item type to be used for order level discounts / fees
    Payment
  • Instantiate payment methods outside of Rules and simply enable them
  • Add split payment support to the Payment checkout pane
    Contributed Modules
  • Create a feature specification area where we publish module blueprints
  • Add a blueprint for Commerce Product Kit
  • Add the blueprint for the next Commerce Stock

Comments

Ryan Ryan on May 21, 2013

Yes, we'll just be targeting Drupal 8. Some of the items on the roadmap are already being developed as solutions for Commerce 1.x, so the roadmap will have an impact on Drupal 7 users, but the bulk of our effort will be spent preparing to take advantage of Drupal 8.

Ryan Ryan on May 21, 2013

There are definitely things for us to do in the core of Commerce, but a lot does depend on the theme and configuration of the site. Some items on the roadmap address the appearance and usability of interfaces like the order view page and product catalog management. While I haven't highlighted it as one of the primary emphases, we're definitely open to ideas for improving this at the core where possible.

Sierk on May 23, 2013

Very exiting. And a lot of work. I hope a lot of people will help out in the issue. I know I will.

One thing I'm especially curious about (because it sounds interesting, and because I probably don't understand completely), is what "Propose entity subtype plugins" is about. Is an "entity subtype" what was called a "bundle" in D7, or is it a "bundle type"? And is a "default field" what is called a "property" in D7, or does it refer to the locked fields Commerce uses (like the price and line item field)? Or is this about something else completely? And is there an issue on d.o that relates to this?

Another thing I'm curious about is Customer profiles. These are only mentioned as something that will not get changed. I totally agree that integrity of historical order data is essential. And if I remember correctly, customer profile duplication was necessary because reference fields weren't able to reference a single revision. Is that still the case? Or would there be a way in D8 to use revisions to handle this? It might not be, but the reason I'm asking is because (in my case) it would useful to have a real customer entity that would "own" can be managed more long term. Currently the user entity is the owner of the order but that doesn't really work (in my case) for anonymous owners (uid == 0 for every customer). The D7 customer profile doesn't work either, because it changes when the profile is duplicated. I realize that this might not be possible in D8 either, but a revisioned customer profile would make things definitely easier for me because I would have to maintain one less entity in D8.

Thanks for the great work!

Ryan Ryan on June 4, 2013

Yeah, the idea behind "entity subtype plugins" was to add per-bundle classes as opposed to just per-entity type classes. In Drupal 8, there are reasons it would be nice to have a "NodeArticle" class instead of just the generic "Node" class. In our use case, we would want to take advantage of the architecture to do things like ensure our locked fields are there - exactly as you've guessed. The code we have in place to create and manage these default fields has proven to be one of the most brittle parts of Commerce thanks to inconsistencies in the various ways Drupal installs modules.

Regarding customer profiles, there will not be a way to use customer profile revisions to accomplish what we wanted. We won't have entity revision referencing, just the same entity referencing, and if you start using revisions I can quickly imagine a scenario where we start having to support "branching" within revisions... and there be dragons. : )

perisho on May 29, 2013

"Drupal Commerce Kickstart 2.0" is not to be confused with "Drupal Commerce 2.0". DCK 2.0 is built on DC 1.x. Both of which are designed to run on D7.

Unless you are doing some fancy downloading of a dev version from somewhere you are probably using Drupal Commerce 1.x- I think.

I was confused by this too until it was explained to me.

Ryan Ryan on June 18, 2013

In thinking through a multi-currency issue, I just realized we should also update our price code to retain currency conversion rates when a price is converted from one to another. We need to be able to determine whether or not an order has been paid in full even if a currency conversion rate changes from one day to another.

Sem on June 19, 2013

Not sure why you need to retain the currency. I would say, just flag the order being paid in full at the moment it is and from there on lock any changes in price etc. For store admin information purposes you could print and save the base currency, the currency in which the order is paid plus the exchange rate on that moment. I do not see any benefit for users placing the order in either case.

Printing and saving base, order currency price plus exchange rate at that moment might also give you some nice report options such as knowing how much you lost or won on exchange rates. Or the currency risk your store is exposed to. One of the issues I am dealing with is that the current modules do not allow to set a markup of % on the exchange rate meaning that I loose % on each order being paid in a foreign currency.

Summarizing on multicurrency
- lock the order as soon as it is paid in full. No difficult history tables that might introduce problems in the future
- save the base price, order currency paid plus exchange rate used for administrator reports (exposure to exchange rate risks)
- make it possible to set a markup of % on exchange rates such that costs are covered. (perhaps adding the % markup to the information saved at step two)

Ryan Ryan on July 23, 2013

It just came up in an issue that the way we allow checkout pages to set their checkout button values isn't sufficient. When we re-implement the checkout form to support multiple checkout workflows, we should look at simplifying button naming and ensuring both the "Go back" and "Cancel" button texts are independently configurable.

In fact, perhaps we shouldn't think in terms of pages at all but rather in terms of multiple checkout flows each having their own steps that can be configured per flow (as opposed to per page), allowing checkout panes to be instantiated in any number of checkout flows.

miaoulafrite on September 30, 2013

Are you on track with the timeline?
can i get a dev version, or even the beta (as it is mentionned to be released in august) to start playing with it?

tititi on November 2, 2013

Hi, I also would like to play the beta release if it is possible.
Is there a chance to do it soon?

jetwong98 on December 3, 2013

Hi,
As per issue raised at https://drupal.org/node/2148075, reposted here as a formal request.

In Magento, we can select the items from an order for generating invoices for payment. In short, an order can have many invoices (unique invoice no) attached to it. This is also a very common rule in any billing and payment systems.

If an order is to be paid via offline payments like cheque / cash on delivery during checkout, the order status is Pending, invoice will be manually generated by the shop owner when the payment is received later. There can be multiple invoices attached to an order if partial payment is made at any point of time.

If an order is paid via Paypal or any online payment methods during checkout, the order status is Processing, invoice can be automatically generated during the time the online payment was captured by the processor.

The way of how Magento handles Order, Payment, Partial Payment, Credit Memo, Return of Goods, Invoice & Shipping are more clear and legally endorsed with traceability.

Unfortunately, from all the current non-collaborative invoicing modules https://drupal.org/node/1535498, none of the said truly fits the real-world practices, Commerce Billy even replaced the Order ID with its own generated Invoice ID when the order status is changed to Invoiced. This way of handling invoice is very odd and somehow considered tampering transactions in my sense, not legally accepted. Thanks

sirtet on February 13, 2014

I guess the lack of updates here (mainly the outdated timeline), negatively affects some decision-makers...
As someone checking by from time to time, i'd say the main info is around
"...will truly begin in earnest once Drupal 8 has an alpha or beta release..."
and
"targeting D8 only"

I think that should be updated and put in the summary of this page.
Also, why not put these crucial facts on the d.o. project page, where it currently only links here?