Could not create new data type (operation status: InvalidConfiguration)

[bjarnef]

In a clean Umbraco 16.2 project on Umbraco Cloud and Deploy 16.0.1, I am trying if I can update schemas from an Umbraco 13 project, which some very basic content types and datatypes configured instead of manually creating these.

However we have a special button picker property editor, which I need to adjust, but much simiar to color picker configuration (I compared with how the "items" in Appoved Color Picker on save is generated:
https://github.com/umbraco/Umbraco-CMS/blob/main/src/Umbraco.Core/PropertyEditors/ColorPickerConfiguration.cs#L11-L12

However when I click "Update Umbraco schema" button I get this in deploy.log:

EXCEPTION
Umbraco.Deploy.Core.Exceptions.ProcessArtifactException: Process pass #0 failed for artifact umb://data-type/01dd7a05575c42dba853421a6aa85642.
 ---> System.InvalidOperationException: Could not create new data type (operation status: InvalidConfiguration).
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.DataTypeConnector.ProcessAsync(ArtifactDeployState`2 state, IDeployContext context, Int32 pass, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.ServiceConnectorBase`4.Umbraco.Cms.Core.Deploy.IServiceConnector.ProcessAsync(ArtifactDeployState state, IDeployContext context, Int32 pass, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.ProcessArtifactsAsync(IEnumerable`1 artifactsWithOptions, String ownerEmail, IDeployContext context, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.ProcessArtifactsAsync(IEnumerable`1 artifactsWithOptions, String ownerEmail, IDeployContext context, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.ProcessPackageAsync(Guid sessionId, Package package, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.DiskReadWorkItem.ProceedAsync(List`1 artifacts, Manifest manifest, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.DiskReadWorkItem.ExecuteAsyncSub(IWorkContext context, List`1 resume, CancellationToken cancellationToken)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.DiskReadWorkItem.ExecuteAsync(IWorkContext context, CancellationToken cancellationToken)

The error in dashboard isn't very useful (and should probably ensure it doesn't overflow viewport, but has scroll inside wrapper).

The configuration of umb://data-type/01dd7a05575c42dba853421a6aa85642.

I tried adding "EditorUiAlias" (although the UI doesn't exist yet) and removed the old "Alias" property (which was just blank).

{
  "Name": "Vertical Alignment",
  "EditorAlias": "MCB.Umbraco.ButtonPicker",
  "EditorUiAlias": "MCB.PropertyEditorUi.ButtonPicker",
  "DatabaseType": "Nvarchar",
  "Configuration": {
    "multiple": false,
    "defaultValue": "Top",
    "buttons": [
      {
        "value": "Top"
      },
      {
        "value": "Center"
      },
      {
        "value": "Bottom"
      }
    ],
    "size": "medium",
    "showIcons": false
  },
  "Parent": "umb://data-type-container/ea846f023db349f8a3cb4bb1e108fd78",
  "Udi": "umb://data-type/01dd7a05575c42dba853421a6aa85642",
  "Dependencies": [
    {
      "Udi": "umb://data-type-container/ea846f023db349f8a3cb4bb1e108fd78",
      "Ordering": true
    }
  ],
  "__type": "Umbraco.Deploy.Infrastructure,Umbraco.Deploy.Infrastructure.Artifacts.DataTypeArtifact",
  "__version": "13.4.0"
}

The configuration of the property editor:

public class ButtonPickerConfiguration
{
    [ConfigurationField("multiple")]
    public bool Multiple { get; set; }

    [ConfigurationField("defaultValue")]
    public string? DefaultValue { get; set; }

    [ConfigurationField("buttons")]
    public List<ValueListItem> Buttons { get; set; } = new();

    [ConfigurationField("size")]
    public string? Size { get; set; }

    [ConfigurationField("showIcons")]
    public bool ShowIcons { get; set; }

    public class ValueListItem
    {
        public required string Value { get; set; }

        public string? Label { get; set; }
    }
}

[bjarnef]

I think this originally happened because of

protected override IDataValueEditor CreateValueEditor() =>
    DataValueEditorFactory.Create<MultipleValueEditor>(Attribute!);

which expected it to be a string array:
https://github.com/umbraco/Umbraco-CMS/blob/aba82dd5228ef83778adf343eab9feb8916577a2/src/Umbraco.Infrastructure/PropertyEditors/MultipleValueEditor.cs#L45

but instead I changed it to a custom expecting an array of object like color picker:
https://github.com/umbraco/Umbraco-CMS/blob/aba82dd5228ef83778adf343eab9feb8916577a2/src/Umbraco.Infrastructure/PropertyEditors/ColorPickerPropertyEditor.cs#L83-L91

protected override IDataValueEditor CreateValueEditor() =>
    DataValueEditorFactory.Create<ButtonPickerPropertyValueEditor>(Attribute!);

Besides that I updated the schemas/revisions via search & replace:

• Removed all Alias: "" which seemed to be empty for most entities (perhaps used for templates, but I didn't have any)
• Added EditorUiAlias to datatypes revisions: https://docs.umbraco.com/umbraco-cms/fundamentals/backoffice/property-editors/built-in-umbraco-property-editors
• For Umbraco.MediaPicker3 Configuration > filter property has changed from comma delimited string or media type aliases to media type guids.
• In multivalues configuration for dropdown list, radiobutton list, checkbox list id property has been removed and Configuration > items property is an array of string values.
• In color picker configuration Configuration > items property is an array of objects, where each object no longer has id property, but label and value properties.

Furthermore in color picker configuration value is no longer stringified json:

"items": [
  {
    "id": 1,
    "value": "{\"value\":\"ffffff\",\"label\":\"White\"}"
  },
  {
    "id": 2,
    "value": "{\"value\":\"000000\",\"label\":\"Black\"}"
  },
  {
    "id": 3,
    "value": "{\"value\":\"cccccc\",\"label\":\"Silver\"}"
  }
]`

But instead an object:

"items": [
  {
    "value": "ffffff",
    "label": "White"
  },
  {
    "value": "000000",
    "label": "Black"
  },
  {
    "value": "cccccc",
    "label": "Silver"
  }
]

Afterwards I could update Umbraco schema from data files successfully - and later export schema to data files just to update version number.

[ronaldbarendse]

This is an interesting way to setup new data types @bjarnef 😄 Looks like you've already figured it out, but I'll try to explain/give some feedback below...

I think this originally happened because of

protected override IDataValueEditor CreateValueEditor() =>
    DataValueEditorFactory.Create<MultipleValueEditor>(Attribute!);

which expected it to be a string array: https://github.com/umbraco/Umbraco-CMS/blob/aba82dd5228ef83778adf343eab9feb8916577a2/src/Umbraco.Infrastructure/PropertyEditors/MultipleValueEditor.cs#L45

but instead I changed it to a custom expecting an array of object like color picker: https://github.com/umbraco/Umbraco-CMS/blob/aba82dd5228ef83778adf343eab9feb8916577a2/src/Umbraco.Infrastructure/PropertyEditors/ColorPickerPropertyEditor.cs#L83-L91

protected override IDataValueEditor CreateValueEditor() =>
    DataValueEditorFactory.Create<ButtonPickerPropertyValueEditor>(Attribute!);

The data type configuration in the artifact uses an IDataTypeConfigurationConnector to map the JSON into a dictionary and the default registration/implementation uses the IDataEditor.GetConfigurationEditor().FromDatabase() method. I'm not sure why changing the value editor worked though, but maybe you changed both or they are somehow linked?

Besides that I updated the schemas/revisions via search & replace:

• Removed all Alias: "" which seemed to be empty for most entities (perhaps used for templates, but I didn't have any)

All Deploy artifacts inherit from a base class that contains this Alias property, but not all artifacts actually require it (data types don't). I've made it nullable in v14 (see umbraco/Umbraco-CMS#15965), so that's why you don't see it serialized to JSON anymore.

• Added EditorUiAlias to datatypes revisions: https://docs.umbraco.com/umbraco-cms/fundamentals/backoffice/property-editors/built-in-umbraco-property-editors

This is indeed required from version 14 onwards ✅ Deploy actually registered a default migrator to automatically add missing ones when importing artifacts...

• For Umbraco.MediaPicker3 Configuration > filter property has changed from comma delimited string or media type aliases to media type guids.

This was indeed changed in CMS v14 (see MigrateDataTypeConfigurations).

• In multivalues configuration for dropdown list, radiobutton list, checkbox list id property has been removed and Configuration > items property is an array of string values.
• In color picker configuration Configuration > items property is an array of objects, where each object no longer has id property, but label and value properties.
  Furthermore in color picker configuration value is no longer stringified json:

"items": [
  {
    "id": 1,
    "value": "{\"value\":\"ffffff\",\"label\":\"White\"}"
  },
  {
    "id": 2,
    "value": "{\"value\":\"000000\",\"label\":\"Black\"}"
  },
  {
    "id": 3,
    "value": "{\"value\":\"cccccc\",\"label\":\"Silver\"}"
  }
]`

But instead an object:

"items": [
  {
    "value": "ffffff",
    "label": "White"
  },
  {
    "value": "000000",
    "label": "Black"
  },
  {
    "value": "cccccc",
    "label": "Silver"
  }
]

Same reason as above!

Afterwards I could update Umbraco schema from data files successfully - and later export schema to data files just to update version number.

Some (if not all) of these manual changes are already done in Deploy using artifact migrators when doing an import 😉 Great to see you managed to get everything working in the end though! 💪🏻

[bjarnef]

@ronaldbarendse thanks for the explanation 😁
It works as expected now, but in makes a few manual steps was required as the project wasn't upgraded, so e.g. Media Picker 3 migration wasn't handled 😅
https://github.com/umbraco/Umbraco-CMS/blob/e5c942b45900e18e73e20b0770b14574ba6b78b5/src/Umbraco.Infrastructure/Migrations/Upgrade/V_14_0_0/MigrateDataTypeConfigurations.cs#L419-L422

So a clean new v16.2.0 project and imported Deploy revisions from v13 almost worked 😅

We have a custom Button Picker property editor (actually a bit similar to the old one which existed in Umbraco Uno 😜 ).
It is a kind of a combination between (approved) color picker and multi value editor like dropdown, radiobutton list and checkbox list.

My hope is that someday this will supported in Umbraco core: umbraco/Umbraco-CMS#11084

For now we need a custom property editor or a package to handle this scenario.

[ronaldbarendse]

No problem, happy to help and curious to find any cases that are currently unsupported!

Deploy 14 registers DefaultLegacyDataTypeConfigurationArtifactMigrator that does the same migrations as the linked CMS one, but it does require importing the data type artifact that was exported in a version before v14 to actually run (and an import operation, not simply putting the UDA file in umbraco\Deploy\Revision). If this didn't work for you and you can share the v13 UDA file, I can have a look...

For custom property editors, you can write your own IArtifactMigrator and IPropertyTypeMigrator to do these migrations on import (see docs). If you need to do similar data type configuration migrations as in the v14 one, you can even inherit from Umbraco.Deploy.Infrastructure.Migrators.LegacyDataTypeConfigurationArtifactMigratorBase and implement the abstract MigrateConfiguration() method, using any of the static helper methods like ReplaceUdiWithGuid(), ReplaceValueListArrayWithStringArray() or ReplaceAliasesWithKeys().

Adding more generic property editors to the CMS has always been a balance between compatibility, maintenance and priorities. The discussion you linked is a great use-case, especially now the CMS decoupled the editor and editor UI.

[bjarnef]

Ahh okay, actually I didn't try export/import (I didn't need any content and media) - just the datatypes, content types etc.
I copied the UDA files into umbraco\Deploy\Revision, and imported updated from schema files. I failed a few times until I manually adjusted to revisions and modified the custom Button Picker to v16, so it was registered.

So mainly the manually changed here:
#282 (comment)

Perhaps if importing content I should have IArtifactMigrator and IPropertyTypeMigrator, but this approach seemed to work as well :)

Do you have an example of how it is handled for color picker / multi value editors (dropdown, radiobutton list and checkbox list)?
The custom Button Picker use much same approach for prevalues - an array of objects, but I noticed the validate of configuration assume the property is named items when returning ValidationResult 😅
umbraco/Umbraco-CMS#20283